How to Create a Serverless Testing Framework with GitHub and Jenkins

We all know that a proper testing framework accelerates a team’s productivity and efficiency. And, testing serverless is no different; a proper testing framework to ensure accuracy and reliability is vital as if the serverless architecture fails, the business can suffer any number of consequences that impact reputation, customer satisfaction, revenue and more.  In addition, remediating issues can be resource-intensive. As a result, today we’ll share the steps we took to integrate Jenkins with a serverless testing framework to improve visibility and ease administration of Ansible Playbooks at Flux7. Following these steps, you, too, can grow visibility and decrease administrative overhead while lowering remediation costs.

This journey began for us at Flux7 when we started using Ansible Playbooks to deploy infrastructure and knew we needed a testing framework to test every role on Ansible. Promoting a test-driven culture, Flux7 uses Molecule to test Ansible Roles.  To further improve, automate and streamline its test processes, we created an automated serverless framework with integration of Ansible role repositories which contain Molecule test-cases.

How To Create a Serverless Testing Framework

A serverless architecture is a combination of Backend as a Service (BaaS) and Function as a Service (FaaS). Hence, it is more likely to be called Platform as a Service (PaaS) as it includes an integration of separate, distributed services that must be tested both independently, and together. As a result, creating a testing framework that allows for easy repeatability in testing is important. To do so, we took a templatized testing approach with:  

A Lambda function that is used for customer deployments and custom configurations. 

A CI/CD pipeline that auto-deploys serverless architecture; it allows changes or additions to be made to architecture and propagates a company-wide framework for writing serverless code. 

The testing framework allows DevOps engineers to test, check lint, verify code and converge. It not only reduces maintenance costs and testing efforts, but also provides a high return on investment (ROI) for DevOps teams looking to optimize their Agile processes.

Before we jump further into the details of the framework, let’s take a quick look at its architecture.

Serverless Testing Framework with Github and Jenkins Architecture

AWS SAM and Lambda

In AWS, serverless applications integrate Lambda functions, event sources, and other resources that work together to perform tasks. We think that Lambda is one of the best Serverless AWS services because it runs as the main backend service for code. And, it allows  developers to write and run their code (with any dependencies) for virtually any type of application or backend service — all with zero administration. For more information, refer to AWS Lambda.

We use AWS Service Application Model (SAM) to create and manage our serverless infrastructure. To deploy SAM, we create a Jenkins job. In turn, SAM uses an AWS CloudFormation template to develop Infrastructure which includes Lambda and other AWS resources. 

Jenkins Pipeline for SAM Deployment

Serverless Test Framework 

We use an API gateway to create a small integration server for Jenkins that manages incoming requests based on a Git event. This API gateway acts as an HTTP server where routes and endpoints are defined by configuration, and each route is associated with a resource to manage that route. In a serverless architecture, such handlers are often FaaS functions. For more information, refer to Amazon API Gateway.

A Jenkins-Job was created for all the repos, eliminating the need for the Jenkinsfile. When setting up a new repository, the needed webhooks must be created. If a repository exists already, only webhooks can be added to it. An API gateway will be used as a webhook for the GitHub repository. After integration, all Ansible repositories are connected to the Jenkins-Job. For a set of Ansible role repos, this Jenkins build will be invoked.

Jenkins Pipeline to Test Ansible Roles

We use two different methods to invoke the Jenkins build:

1. Nightly Jenkins Build

   We are most excited about the Jenkins Nightly-Build portion of the serverless testing framework as it helps check logs quickly without manual intervention. Unlike plug-ins that may help with this task, our solution will not cease functioning if you change the version of Jenkins you’re working with; using a Lambda function allows the process to remain automated regardless.
   
   The nightly Jenkins build matches Ansible repos from the GitHub repos based on a specified pattern (Eg: repos with molecule-testing tag). It then runs appropriate test jobs to make sure that outside factors do not break code. It ensures that validations are followed using the latest LTS, and provides feedback of changes made, related to Jenkins infrastructure and configurations. As you can see in the graphic below, the solution communicates failures to specific Slack channels for high visibility and quick action.
   
   Automated nightly builds help check logs on a daily basis without the need for any manual intervention, providing high visibility of failures with zero administration. Nightly builds help ensure quality of builds and most importantly help us maintain a solid infrastructure. 
   
   Slack Notification Logs

2. Pull Request and Push Event Builds

   The second method we use to invoke the Jenkins build is the pull request or push event build:
   
   Pull Request – To run a test scenario, our developers raise a pull request or push an event to invoke the Jenkins testing build. When a pull request is initiated, a Jenkins build is also initiated. 
   
   Push Event – To push an event, developers follow a standard GitHub code push which then properly commits code for a specific Ansible scenario. The commit message format should be: <Your Message #Commit>
   
   Example commit messages you may use include:
   
    

   No.	Commit Purpose	Commit Message
   1	Test	Your Message #test
   2	Lint	Your Message #lint
   3	Converge	Your Message #converge
   4	Destroy	Your Message #destroy
   5	Verify	Your Message #verify

Improve Test Efficiency and More

In addition to the innate benefits of a serverless architecture, the serverless testing framework has several benefits of its own. First and foremost, it improves test efficiency and code quality as automation checks code each time a Jenkins build is invoked, thereby removing the opportunity for human error. It also eliminates the need for additional jobs as the same Jenkins job can be used for all repositories which is helpful because it saves engineers’ time and efforts. The serverless test framework also optimizes test scenarios while facilitating maximum test coverage. 

Importantly, it reduces maintenance costs in testing efforts. While we’ve seen anecdotal evidence of this, industry accounts show that automated testing frameworks provide a high return on investment to DevOps teams. Specifically, the solution has  afforded us the opportunity to run more tests without increasing costs, all in the same amount of time. 

We use Ansible playbooks to improve our service delivery model.  Promoting a test-driven culture, We use Molecule to test Ansible Roles and have now built upon that foundation with an automated serverless testing framework. For enterprises using — or interested in using — Ansible roles in their organization, this framework will provide greater test speed, efficiency and accuracy, while reducing test maintenance costs and overall risk as it results in a consistently concrete infrastructure.