From End to Beginning: Automation Testing in the CI/CD Pipeline
Get a rundown of the stages involved in testing as well as best practices and tools to integrate it within your CI/CD pipeline.
Join the DZone community and get the full member experience.Join For Free
In a previous article, we discussed the various use cases wherein agile teams would automate their test cases. One of the cases was when teams want to integrate tests with every build and have a continuous integration as a part of the build process.
In this article, we will address integrating tests in a continuous integration/continuous delivery platform.
Let’s start with the basics first.
What Are Continuous Integration and Continous Delivery?
In simple terms, continuous integration allows development teams to integrate their code into a shared repository. This helps in maintaining code quality and identifying potential issues with the local version of the code at an early stage.
Continuous delivery is often called "Continuous Deployment" as well. Everything that is being continuously merged by the development team is continuously deployed to the live environment. Continuous delivery is often coined as "Continuous Deployment" as well.
Since most developers work in parallel, continuously integrating their code into one repository would mean that the master branch is continuously updated with new features. In order to ensure that there is no compromise in the code quality with so many changes happening rapidly, testing must happen at the same pace.
It should be no surprise that manual testing in this environment would not be the best approach in order to achieve this. Automated testing is the key to successful testing in the CI/CD pipeline.
Stages of Continuous Delivery
Develop: The developer builds the code according to the project requirements or the feature requests.
Writing tests: Once the code is written, tests need to be written for the same. At this point, these tests as usually unit tests written by the developers.
Local Testing: This is then locally tested in order to check whether all the tests passing and to ensure the code does not break. Often, a percentage is set as the pass rate which the tests running need to meet.
Rebase and Resolve conflict: In an actual development scenario, there will be multiple people merging their code. The developer needs to make sure that his branch is updated at all times. Updating the branch with the latest merged code is called "rebasing." Once it’s rebased, there will likely be some conflicts which need to be resolved. After that, the tests are run again against the rebased code.
Commit: Once the tests have passed, the code is then ready to be committed with all the changes.
Build: The source code developed is then combined to build a deployment artifact which can be run on an instance, like the server if the environment is on-premises. This code is now ready to be deployed to different testing environments.
UAT: The code is then deployed to a test server where testers start to test the feature. These tests can be automated as well as manual.
Merge: If the commit that’s under testing is approved by the testers, this is then merged into the master branch.
Production deployment: Once the code is merged, this merged code is then deployed to production.
The above process needs to be done with every build coded by the developers.
Where Does Automation Testing Fall in This CI/CD Pipeline?
Automated testing ideally happens once the build stage has been completed and the code can be deployed. Unit tests, UI tests, and integration tests can all be run at this stage. These tests help in ensuring that the code meets a standard of quality.
This phase can last from a few minutes to a couple of hours depending on how the automation that needs to be run is architected.
Tests can be run in parallel in order to execute them more quickly. If a code fails during the test phase, the build can be rejected without further investing in any manual testing time.
Tools Used for CI/CD
- Jenkins: Jenkins is an open source tool which is used for continuous integration. It’s free to use and jobs can be configured both by the interface as well as scripts.
- Travis CI: This tool is free of cost for open source projects, hosted by GitHub.
- Gitlab: Gitlab is a version control tool which has its own cloud-based CI methodology. It is supported on multiple platforms which have both free and paid versions.
- Bamboo: Bamboo is a CI tool by Jira. If your organization uses Jira, then it would be beneficial to check this tool out. It supports automated merging on approval of tickets as well.
Best Practices for CI/CD Pipeline to Make the Best out Of Test Automation
- Incremental changes: It is always advisable to follow a feature-by-feature approach. If the feature is really big, it is good to break it down into smaller and more quicker to test features. This is important in terms of automation because if there is an issue, it is easier to figure out the root cause. If your commit is too big, isolating the cause of an issue would be a tough task.
- Identify what can be automated: It is very common for teams to dive fast and say "Let’s automate everything," but this is a common mistake. We must know the purpose of automation and identify the test cases which should be automated.
- Parallel Tests: Tests should be run in parallel in order to make testing more efficient and timely. It can greatly reduce the time taken to run tests and thus give the results much quicker. But it’s not sufficient to just execute these tests in parallel; it is also important to scale the server size where the tests are running in order to actually make them faster.
Automating tests is an important part in the successful deployment of projects while maintaining a standard of quality. Ensuring tests are run at every stage gives good transparency on the quality of the code. Bugs can be discovered at an early stage and any delays which can be caused by it can be planned in a timely manner. Having a CI/CD pipeline with tests integrated helps in speeding up the testing and deployment process.
Published at DZone with permission of Arjun Sharma. See the original article here.
Opinions expressed by DZone contributors are their own.