Welcoming The Release Of AWS CloudFormation Drift Detection

Infrastructure as Code has revolutionized how we manage infrastructure and AWS CloudFormation has played a foundational role in this process. As AWS Premier Consulting Partners, at Flux7 we have many customers using AWS CloudFormation to deploy their infrastructure. Yet, the benefits of AWS CloudFormation aren’t just about deploying, as it is instrumental in maintaining and providing future upgrades to their infrastructure. In this regard, AWS CloudFormation has had many major improvements, one of the biggest of which were changesets. And today Amazon takes these benefits another stride forward with the release of drift detection for AWS CloudFormation.

One of the biggest concerns our customers have regarding AWS CloudFormation is an upgrade to a landing zone stack; they want to ensure the upgrade does not put the stack in a bad or unstable state that brings down production applications. Features, like allowing users to take self-service corrective actions on stacks, and reviewing the changes using changesets, has been extremely beneficial in instilling confidence in users of AWS CloudFormation in the stability of their stacks. 

However, one issue that had remained a concern is: What happens if someone makes manual changes to the setup which render the stack to a state where it can neither be updated or deleted without causing any downtime? To address this concern, AWS has introduced drift detection for AWS CloudFormation.

A resource within a stack is considered to have drifted if its actual configuration differs from its expected configuration. A stack is considered to have drifted if one or more of its resources vary from its expected configuration state. AWS CloudFormation generates an expected state of the stack by compiling the following information:

• Resource properties specified in the stack template and parameters
• Default values set by AWS CloudFormation (but not explicitly mentioned in the templates
• Default values set by the underlying services responsible for creating resources.

This information is generated upon stack creation and is updated whenever a stack is successfully updated. AWS CloudFormation uses this information against the actual resource properties to determine if the resources within a stack have drifted.

The new drift detection feature can be accessed through the AWS CloudFormation console as well as through CLI/SDKs. To understand how it works, we created a simple stack consisting of an Amazon EC2 instance and a security group associated with the instance.

One can see that the ‘Drift Status’ reports that the actual configuration of all the resources of the stack match with that of the stack template. To demonstrate how drift detection works, we made a change to the security group ‘AppServerSg’ to open up a new port. To detect if the stack has drifted, AWS CloudFormation provides a new action on the stacks. Select the stack for which the drift needs to be detected and click on Actions -> Detect drift for current stack.

The Detect Drift dialog box appears and asks for confirmation. Upon confirmation, the Detection status appears as DETECTION_IN_PROGRESS. It might take a few minutes depending on the number of resources in the stack. Once detection is complete, the result is updated in the dialog box as well as the Stack Overview page in the AWS CloudFormation console. In this example, the stack has indeed drifted and thus the result is displayed as such. 

One can also view which resources in the stack have drifted.

To go further and understand what exactly has changed in the underlying resource, click on the stack and then click on view details within the Drift Status heading.

This will take you to the Drift Details page for the stack. Here we can expand any resource to understand the actual resource configuration and expected configuration.

As you can see here, the expected configuration, based on the stack template, is that security group ingress rules has only port 22 open. Meanwhile, the actual configuration shows that in addition to port 22, port 80h as also been added to ingress rules. 

All the actions described above can be automated using CLI/SDK which makes it really easy to have automated drift detection in place both for cloud governance and to reduce any accidental downtime due to updates on a drifted stack.

Drift detection for AWS CloudFormation will go a long way towards helping our customers gain confidence in making AWS CloudFormation updates to critical stacks, especially those stacks that other stacks depend on. This confidence will in turn help improve our customer’s experience year-over-year as they upgrade their landing zones to meet the ever-growing agility needs of the business.