1. Managing GitHub with Terraform

Managing GitHub with Terraform

If a service can be managed with API most probably you will find it in an impressive list of Terraform providers. Yes, GitHub is there, too. TwinDB hosts software in GitHub, it felt wrong I don’t manage it with Terraform yet, so I decided to give it a go.

Prerequisites

Directory layout

I keep all GitHub related stuff in a separate directory. GitHub also has its own state file. The separate state file ensures some level of protection against mistakes that can impact other parts of infrastructure.

Let’s see what’s in the directory.

State file configuration. The state file is stored in a separate file in an S3 bucket.

Input variables for GitHub provider.

main.tf contains the provider configuration, users.tf – members of TwinDB organization, teams.tf – TwinDB teams and membership, repos.tf – configures repositories. twindb-repo is a Terraform module that defines TwinDB repository, branch protections, webhooks etc.

Now about each in more details.

Provider configuration

Before you can do anything with GitHub you need to configure the provider.

I use my personal token , but it’s better to create some role account and use its token.

Where to store the token and how to use it? Well, I store it in an ignored file terraform.tfvars and use it in a Makefile.

The terraform.tfvars can be also generated from environment variables in Travis or whatever you use for CI/CD.

Environment variables is another convenient way to pass variable to Terraform. If you define a TF_VAR_github_token environment variable, Terraform will use it as the github_token variable.

Organization members

I think you cannot create users in GitHub with Terraform, but it is possible to define them as members of an organization.

It’s also possible to add your own SSH/GPG keys, but I don’t see much value in that. It is impossible to create/delete other users or manage their keys.

Organization Teams

Quite handy to keep track of teams and members in a teams.tf.

Repositories

The most exciting part of course is repositories. A repository configuration consists of the repository configuration itself, webhooks, collaborators, branch protection, etc. It is better to put that all in a module and reuse it for all repos (obviously only for those that share configuration). That way it will be easier to keep the repo configuration uniform because you will avoid copy-pasting.

For example, this is how the module is used for the TwinDB backup repo.

And this is the module definition.

Importing existing repositories

It’s a good chance you already have repos in GitHub and you’d like to import them into Terraform. The Terraform docs suggests a command:

This unfortunately is not going to work with repos defined in modules. Terraform requires this piece of code to be present and putting resource "github_repository" "backup" confuses Terraform.

The workaround for that is following. 1) Rename existing repo backup to something temporary backup_x. 2) Add the module "backup" and run apply. It will create a blank repo backup. 3) Remove that backup repo and rename backup_x back to backup. Now the repo is in Terraform.

Another nasty problem with the GitHub provider is that it cannot create a repo with default branch anything but master. At TwinDB we use a Gitflow model where the default branch is develop. When you create a repo in GitHub it contains only one branch – master. Setting it to anything else will fail. I wish the provider would just gave a warning that the develop branch doesn’t exist and didn’t fail. For now I will create repos manually and then import them in Terraform.

Conclusion

Overall I like the GitHub provider, it definitely helps to keep GitHub in a good shape. There are shortcomings, too.

  • GitHub provider doesn’t support some GitHub features. For example, it’s not possible yet to configure GitHub pages.
  • There is no smooth way to use anything but master as a default branch.
Previous Post Next Post