Jeff Bailey
Written by Jeff Bailey
Learning resources, opinions, and facts about technology.

Learn Terraform

Reading Time: 8.4 minutes
Listen to this post

In my last post, Learning Terraform, I committed to learning Terraform. I’ve started reading Terraform: Up and Running: Writing Infrastructure as Code. It has given me a great primer thus far. I’m converting my Cloudformation templates to Terraform. I’m all in at this point and am convinced that this is my preferred tool for building infrastructure as code.

The Basics

You’ll want to install the Terraform command-line interface. I ran into some issues with the Homebrew version of Terraform, so I installed the binary. Terraform provides a single binary with zero dependencies, which warms my heart.

I love shell aliases. So, I set up the following aliases for my terraform activities.

alias tf='terraform'
alias tfa='terraform apply'
alias tfd='export TF_LOG=DEBUG'
alias tfdl='unset TF_LOG'
alias tfdr='terraform destroy'
alias tfp='terraform plan'
alias tfs='terraform show'
alias tft='export TF_LOG=TRACE'

I’ve found these to be my most common commands while developing new templates.

If you’re on Windows 10, you can install a Linux shell, or if you’re stuck in the past, you can use Cygwin.

On either platform, add the aliases to your .bash_profile, then run this command.

source ~/.bash_profile

I recommend using a Terraform IDE extension for completion and syntax highlighting.

Don’t forget the golden rule of Terraform before diving too deep.

The master branch of the live repository should be a 1:1 representation of production.

If you’re chanting incantations to get your Terraform managed resources deployed, step back and rethink your approach. Following this rule will also allow Terraform to serve as documentation.

Testing

Like any other code starting with tests is a great idea. Proving your code is doing what it’s supposed to do is a cornerstone of great software.

Testing Terraform requires the following.

The tests you write should verify that your resources made it to the environment you are targeting. Perform tests by using output values fed to simple scripts that check if resources exist after deployment.

Tools like Terratest can give you a leg up if you want to get more sophisticated.

Providers

Learning Terraform requires an understanding of providers, and they are a vital feature that harnesses the true power of Terraform. Terraform supports numerous providers, of which I’ve used the New Relic and AWS providers with great success.

Importing a New Relic dashboard went surprisingly smoothly. It was a bit painful to pick apart the extra attributes after import, but it saved time. I’ve added a new Terraform feature proposal to GitHub. Upvote that issue if you’re interested in saving even more time.

Data Sources

Data sources are a core concept that allows you to query resources using various APIs. Data sources are read resources outside of Terraform’s control.

These read-only resources exist outside of your Terraform module, and they may have resided in another terraform module or outside of Terraform entirely.

Terraform data sources are necessary for resources that Terraform does not manage.

The syntax is super simple. This code example from the Terraform website is the perfect use case. If we would like to query an AMI to associate with a launch configuration, we do it.

# Find the latest available AMI that is tagged with Component = web
data "aws_ami" "web" {
  filter {
    name   = "state"
    values = ["available"]
  }

  filter {
    name   = "tag:Component"
    values = ["web"]
  }

  most_recent = true
}

Stack Overflow has a post that answers the Terraform data source use cases in more detail.

State

Terraform’s magic is its ability to manage resource states. State management can be painful. Kudos to Hashicorp for taking on the state management challenge. If you’re working on a team, this feature is necessary to ensure the stability of your resources. If you’re not working on a team, you can get by without considering this feature.

If we’re a team managing state, we’ll need to ensure we don’t cross the streams. Luckily Terraform provides the ability to store your state in a remote data source. Keeping the state in a remote location allows you to edit the same resources on a team by keeping the state in a central location.

Multiple backends can store Terraform’s state.

The most common backend is S3, and it’s simple and works well.

terraform {
  backend "s3" {
    bucket="<Bucket Name>"
    key="<Bucket Key Where State Will Be Stored>"
    region="us-east-1"
  }
}

The only thing you need to do is add that to your Terraform template. If you add this later, you’ll need to reinitialize the terraform state.

Remember

  • Some Terraform state is eventually consistent. If a resource fails to deploy, you’ll need to run it again after fixing the problem. Use the depends_on meta-argument to avoid some of this back and forth work.
  • Valid plans can and will fail. Terraform can’t handle every edge case in the universe. Failures are usually caused by not importing existing resources.
  • Commit to only using Terraform to manage your resources. If you use user interfaces instead weird errors will occur while using it.

Modules

A Terraform module is a directory after running this command.

terraform init

We’re all using Terraform modules by way of using the Terraform command-line interface. Terraform input variables and output values control your module’s behaviors. Access child module outputs with this syntax.

module.MODULENAME.OUTPUTVARIABLENAME

The calling module should handle the provider definition.

Here’s an example of calling a module.

provider "aws" { 
    region = "us-east-1"
} 

module "webserver_cluster" { 
    source = "../modules/services/webserver-cluster" 
}

This code assumes your modules folder lives outside of the Terraform template.

Remember

  • When creating a reusable module, always prefer using a separate resource. Separating resources will allow callers of your module to extend your module with custom rules.
  • Version your modules to avoid breaking dependent code
  • Make your modules configurable for added flexibility

Import

Behold the Terraform import command.

terraform import aws_s3_bucket.jeffbaileywebsite jeffbaileywebsite

Run this command when you have existing resources you would like to manage with Terraform. If you’re migrating from CloudFormation templates, this command will be your best friend.

Importing is awkward, but adding the resource with a local name will run the import command.

resource "aws_s3_bucket" "bucket" {
}

Once complete, you can run this command to get a representation of the resource you are importing.

terraform plan

If you set up the aliases above, you can type tfs to run the same command.

Copy the output of the resource you are importing and rerun tfs. The output will give you complaints about read-only fields like in a Terraform template. Remove the invalid fields and rerun tfs to see if your local state is valid. If it is, you can run this command.

terraform apply

Check that you haven’t deleted your infrastructure, then commit your template. Now you’re off to the races with Terraforming further changes to your resources in the future.

Challenges

Any tool comes with its challenges, and here are some of the problems you will encounter.

ProblemSolutions
Avoiding an override of your remote state with local stateDeploy changes with a build pipeline that only allows one deployment at a time

Use remote state and diligently run the terraform plan command locally to capture the latest changes.
Ensuring resources are in the state they are supposed to be inCreate unit and integration tests to validate that your resources deployed as expected

Use an isolated testing environment to validate all your changes and run your tests.
Small challenges for great rewards

Conclusion

While learning Terraform might save your life, it’s not all roses and sunshine, and there will be problems using it like any other tool.

While creating an AWS Cost and Usage Report, an internal server error occurred. The aws_cur_report_definition failed to deploy unless targeting the us-east-1 region. When I changed the template to target us-east-2 instead of us-west-2, everything worked. CloudFormation might have been more helpful.

Adoption of Terraform within your team will need a culture change. The team will need to understand and appreciate the benefits of Terraform. Editing a dashboard in a slick user interface is convenient. While it’s convenient to edit a dashboard, it doesn’t share the intent with other team members. Pull requests will prompt your team to question a dashboard change. Your team will also have an opportunity to learn about new features added to a dashboard.

The bottom line

Changing a dashboard can cause your employer to lose millions in lost revenue. If your widget says everything is fine, but it’s not, was the convenience of the UI worth the cost? Make sure your team sees the value before asking them to delve into Terraform.

Overall, Terraform is great and is getting better. I’m committed to using it for my IaC efforts in the future.

Continue Learning Terraform

If you work for a company that has stringent compliance workflows watch this video from Ellie Mae. These guys automated pretty much everything to capture every change everywhere.

  • Terrascan – A static code analyzer for Infrastructure as Code.
  • Terraform Validate – A python package that allows users to define Policy as Code for Terraform configurations.