Infrastructure Provisioning Update: Terraform and CloudFormation

Over the past few months, the DevOps team here at Flux7 has noticed a growing trend among our projects. An increasing number of client assessments result in the use of Terraform by HashiCorp in support of DevOps automation and more specifically, infrastructure as code (IaC). We thought we’d devote today’s blog to why we are becoming heavier Terraform users and its benefits. And, we’ll also share the situations in which we recommend its use to clients, as well as situations where we might recommend the use of both AWS CloudFormation and Terraform.

What is Terraform?

First, let’s give those of you new to Terraform a quick introduction. Terraform is an open source, cloud agnostic tool that allows you to build, change and version infrastructure via configuration files. These config files describe the components you need to run anything from the simplest application to entire infrastructures. Said another way, Terraform allows you to define infrastructure as code.

Terraform’s leading features include:

• The ability to generate an execution plan describing what it will do to reach the desired state, and then execute it to build the described infrastructure. Terraform separates the planning phase from the execution phase by using the concept of an execution plan. And, notably, Terraform presents a detailed and readable summary of the changes that will be applied.  
• Terraform’s modular design, which makes it easier to create a catalog of approved Terraform modules that are easily consumed by other teams within a large enterprise.
• Terraform’s simple, unified syntax allows almost any resource to be managed without learning new tooling. By capturing all the resources required, the dependencies between them can be resolved automatically so that operators do not need to remember and reason them out.
• Last, Terraform enables operators to write reusable scripts, which gives visibility to infrastructure updates.

How Does Terraform Differ from AWS CloudFormation?

If you are saying to yourself, this sounds a lot like AWS CloudFormation, you’d be right. The two tools offer similar base functionality. However, Terraform has three features that make its use a better fit in some situations (we’ll cover these situations in just a minute).

1. Terraform’s modularity allows you to easily write reusable code. This approach can, for example, keep you from having to copy and paste code from one environment to the next when deploying the same app in several locations. Once written, you simply place your template code in a module, which you can then deploy with the use of a URL.
   
   For example, we are working with a client that uses Atlassian BitBucket for its code repository, including its Terraform modules. Several services, managed by disparate teams, use these modules all by using the BitBucket URL in their code. Additionally, Terraform modules are configurable which means that you don’t have to recreate the wheel every time you want it to perform differently, you only need to alter the parameters.
   
   The big benefit of this modular approach is that it allows the infrastructure team to create modules with best practices codified in the template. When developers then use the module, they are building from a best practice foundation. Moreover, if this customer’s infrastructure team needs to update the module for any reason, the change is reflected as a new version, with a new URL. If the service teams would like to use the new release, all they need to do is update the URL, saving time and effort (and potential human error) in updating. This managed library of Terraform modules helps ensure compliance with infrastructure and security best practices, while saving developer resources.
   
   AWS CloudFormation allows for similar modularity but there are subtle differences. For example, in CloudFormation, a module can imported but only if the module resides in an Amazon Simple Storage Service (S3) bucket. While this has benefits, it does require an extra step of ensuring that the modules in S3 are always up to date with the version control system. This is why we feel that Terraform provides an easier use of modules.
2. Terraform is platform agnostic, working with all the major cloud providers, e.g. AWS, Google Cloud, Microsoft Azure, etc. It also works with PaaS and SaaS providers like BitBucket and GitHub. While the number of supported platforms is quite lengthy, you can access the full list here
3. HashiCorp Terraform provides the ability to write a loop in the code which makes it easy to write code in Terraform as you can create nearly identical resources with ease, e.g., several distributed systems applications such as MondoDB require that you create at least three EC2 instances, and in three different availability zones. Rather than writing a template for each of the instances, we can use the loop to create them with less code.

While these features are helpful in some situations, a drawback to Terraform is that unlike AWS CloudFormation, it isn’t natively integrated with new AWS features. This is a natural result of AWS ensuring its own tools are the first to benefit from any new features and functionality it develops. So, for example, while the new AWS Service Catalog service was available for AWS CloudFormation at launch, Terraform customers are still waiting on this functionality.

When to use Terraform?

In looking at when these features best serve companies, we see that Terraform is being used by companies that:

1. Share infrastructure-as-code between teams and services. In this case, teams have equal access to Terraform modules developed by infrastructure and security teams to ensure compliance with internal (and perhaps even external) standards or requirements. When modules are updated, teams can easily update their own services without a great deal of time or effort, helping ensure ongoing compliance.
2. Implement a hybrid approach as Terraform can just as easily create resources in AWS as it does in Azure, the Google Cloud, and VMware.

AWS CloudFormation is usually a better fit for organizations:

1. Using AWS Service Catalog as it works exclusively with CloudFormation. Together they easily enable self-serve IT in AWS environments as users can easily define AWS Service Catalog products using AWS CloudFormation templates.
2. Who want new AWS features and functionality first. As a core AWS service, AWS often brings new features to AWS CloudFormation first among its toolset. For example, the first tool Service Catalog integrated with was CloudFormation. Moreover, new AWS functionality will often be found in AWS CloudFormation before it is available in Terraform.

Terraform Tool Ecosystem

As you might expect, the ecosystem of tools that can effectively be used with Terraform is similar to those you’d use with CloudFormation. For example, just like with CloudFormation, we often find our AWS consulting team using Jenkins with Terraform for Continuous Integration. (You could also use tools like Bamboo or TeamCity with Terraform.)

We have used this combination most recently for the customer we mentioned earlier. They sought to build a CI/CD service that would allow it to easily build and modify its AWS infrastructure. Specifically, we built pipelines for Code, Infrastructure, and Configuration with Jenkins and Terraform. For this customer, the pipeline is configured in Jenkins for deploying Terraform templates to the specified environments.

Terraform & CloudFormation

Although Terraform and AWS CloudFormation are  provisioning tools with similar functionality, we do often see situations where using both tools can make sense. The reason is that Terraform is able to run CloudFormation templates and can act as a very powerful wrapper around it.

As you would expect, AWS CloudFormation has more AWS support, e.g., all AWS QuickStarts are published with a CloudFormation Template. Thus, it makes sense to combine the power of the two tools at times. To give you an example, we have a customer using Terraform at large. For them, we used CloudFormation provided by AWS for the Transit VPC wrapped inside of Terraform. The intent was that this will

(a) reduce the effort of translating from CloudFormation to Terraform, and

(b) enable the customer to consume any future updates from AWS without having to re-translate.

Similarly, for another customer, we used Terraform at large. However, the customer specifically wanted to use AWS Service Catalog for its Sandbox accounts. In this case, we took a portion of the Terraform code and translated it into CloudFormation so that AWS Service Catalog, which only works with CloudFormation, can be used.

We are increasingly being asked about the relative benefits of Terraform and AWS CloudFormation, of which there are many. Regardless of the solution you use, implementing Infrastructure as Code to increase standardization, reduce risk, and promote developer productivity will help you reap the benefits of DevOps automation that much faster. For additional reading about CI/CD, Infrastructure as Code, and DevOps automation, please subscribe to our blog.