In my last post, Learning Terraform, I committed to learning Terraform. I’ve started reading Terraform: Up and Running: Writing Infrastructure as Code. It has given me a great primer thus far. I’m converting my CloudFormation templates to Terraform. At this point, this is my preferred tool for building infrastructure as code.

The Basics

You’ll want to install the Terraform command-line interface. I ran into issues with the Homebrew version of Terraform, so I installed the binary. Terraform provides a single binary with zero dependencies, which warms my heart.

I love shell aliases. So, I set up the following aliases for my terraform activities.

alias tf='terraform'
alias tfa='terraform apply'
alias tfd='export TF_LOG=DEBUG'
alias tfdl='unset TF_LOG'
alias tfdr='terraform destroy'
alias tfp='terraform plan'
alias tfs='terraform show'
alias tft='export TF_LOG=TRACE'

These are my most common commands while developing new templates.

If you’re on Windows 10, install a Linux shell, or if you’re stuck in the past, use Cygwin.

Add the aliases to your .bash_profile on either platform, then run this command.

source ~/.bash_profile

I recommend using a Terraform IDE extension for completion and syntax highlighting.

Remember the golden rule of Terraform before diving too deep.

The master branch of the live repository should be a 1:1 representation of production.

If you’re chanting incantations to get your Terraform-managed resources deployed, step back and rethink your approach. Following this rule will also allow Terraform to serve as documentation.

Testing

Like any other code starting with tests is a great idea. Proving your code does what it’s supposed to do is a cornerstone of great software.

Testing Terraform requires the following.

The tests you write should verify that your resources made it to your target environment. Perform tests using output values fed to simple scripts that check if resources exist after deployment.

Tools like Terratest can give you a leg up if you want to get more sophisticated.

Providers

Learning Terraform requires an understanding of providers, and they are a vital feature that harnesses the true power of Terraform. Terraform supports numerous providers, of which I’ve used the New Relic and AWS providers with great success.

Terraform Success

Importing a New Relic dashboard went surprisingly smoothly. It was painful to pick apart the extra attributes after import, but it saved time. I’ve added a new Terraform feature proposal to GitHub. Upvote that issue if you’re interested in saving even more time.

Data Sources

Data sources are a core concept that allows you to query resources using various APIs. Data sources are read resources outside of Terraform’s control.

These read-only resources exist outside your Terraform module and may have resided entirely in another or outside of Terraform.

Terraform data sources are necessary for resources that Terraform does not manage.

The syntax is super simple. This code example from the Terraform website is the perfect use case. If we would like to query an AMI to associate with a launch configuration, we do it.

# Find the latest available AMI that is tagged with Component = web
data "aws_ami" "web" {
  filter {
    name   = "state"
    values = ["available"]
  }
  filter {
    name   = "tag:Component"
    values = ["web"]
  }
  most_recent = true
}

Stack Overflow has a post that answers the Terraform data source use cases in more detail.

State

Terraform’s magic is its ability to manage resource states. State management can be painful. Kudos to HashiCorp for taking on the state management challenge. If you’re working on a team, this feature is necessary to ensure the stability of your resources. You can get by without considering this feature if you’re not working on a team.

If we’re a team managing state, we’ll need to ensure we don’t cross the streams. Luckily, Terraform can store its state in a remote data source. Keeping the state in a remote location allows you to edit the same resources on a team by keeping the state in a central location.

Multiple backends can store Terraform’s state.

The most common backend is S3, and it’s simple and works well.

terraform {
  backend "s3" {
    bucket="<Bucket Name>"
    key="<Bucket Key Where State Will Be Stored>"
    region="us-east-1"
  }
}

The only thing you need to do is add that to your Terraform template. If you add this later, you’ll need to reinitialize the terraform state.

Remember

  • Some Terraform state is eventually consistent. If a resource fails to deploy, you’ll need to rerun it after fixing the problem. Use the depends_on meta-argument to avoid some of this back-and-forth work.
  • Valid plans can and will fail. Terraform can’t handle every edge case in the universe. Failures are usually caused by not importing existing resources.
  • Commit to only using Terraform to manage your resources. If you use user interfaces instead, weird errors will occur.

Modules

A Terraform module is a directory after running this command.

terraform init

Terraform input variables and output values control your module’s behaviors. Access child module outputs with this syntax.

module.MODULENAME.OUTPUTVARIABLENAME

The calling module should handle the provider definition.

Here’s an example of calling a module.

provider "aws" {
    region = "us-east-1"
}
module "webserver_cluster" {
    source = "../modules/services/webserver-cluster"
}

This code assumes your modules folder lives outside of the Terraform template.

Remember

  • When creating a reusable module, prefer using a different resource. Separating resources will allow callers of your module to extend your module with custom rules.
  • Version your modules to avoid breaking dependent code
  • Make your modules configurable for added flexibility.

Import

Behold the Terraform import command.

terraform import aws_s3_bucket.jeffbaileywebsite jeffbaileywebsite

Run this command when you have existing resources you would like to manage with Terraform. This command will be your best friend if you’re migrating from CloudFormation templates.

Importing is awkward, but adding the resource with a local name will run the import command.

resource "aws_s3_bucket" "bucket" {
}

Once complete, you can run this command to get a representation of the resource you are importing.

terraform plan

If you set up the aliases above, you can type tfs to run the same command.

Copy the output of the resource you are importing and rerun tfs. The output will give you complaints about read-only fields like in a Terraform template. Remove the invalid fields and rerun tfs to see if your local state is valid. If it is, you can run this command.

terraform apply

Check that you haven’t deleted your infrastructure, then commit your template. Now you’re off to the races with Terraforming further changes to your resources in the future.

Challenges

Any tool comes with challenges, and here are some of the problems you will encounter.

ProblemSolutions
Avoiding an override of your remote state with local stateDeploy changes with a build pipeline that only allows one deployment at a timeUse remote state and diligently run the terraform plan command locally to capture the latest changes.
Ensuring resources are in the correct stateCreate unit and integration tests to validate that your resources deployed as expectedUse an isolated testing environment to validate all your changes and run your tests.

Small challenges for great rewards!

Conclusion

While learning Terraform might save your life, it’s not all roses and sunshine, and there will be problems using it like any other tool.

While creating an AWS Cost and Usage Report, an internal server error occurred. The aws_cur_report_definition failed to deploy unless targeting the us-east-1 region. Everything worked when I changed the template to us-west-2 instead of us-east-1; CloudFormation might have been more helpful.

Adoption of Terraform within your team will need a culture change. The team will need to understand and appreciate the benefits of Terraform. Editing a dashboard in a slick user interface is convenient. While editing a dashboard is convenient, it doesn’t share the intent with other team members. Pull requests will prompt your team to question a dashboard change. Your team will also have an opportunity to learn about new features added to a dashboard.

The bottom line

Changing a dashboard can cause your employer to lose millions in lost revenue. If your widget says everything is fine, but it’s not, was the convenience of the UI worth the cost? Ensure your team sees the value before asking them to delve into Terraform.

Overall, Terraform is excellent and is getting better. I’m committed to using it for my IaC efforts in the future.

Continue Learning Terraform

If you work for a company with stringent compliance workflows, watch this video from Ellie Mae. These guys automated everything to capture every change everywhere.

Related Content