Continuous Testing Using a Shift-Left Testing Approach
Continuous Testing Using a Shift-Left Testing Approach
To the left, to the left, everything single test needs to shift to the left.
Join the DZone community and get the full member experience.Join For Free
Is the concept of adopting a continuous everything model a daunting task for your fast moving business? Read this whitepaper to break down and understand one of the key pillars of this model in Continuous Governance: The Guardrails for Continuous Everything.
The software industry is changing at a very rapid pace and the time to market has become more critical than ever before. The waterfall model was one of the most prominent software models that were used by organizations, but it had a major drawback – testing was not done at every stage of software development, due to which highly critical bugs were left unnoticed until the software reached the production stage. Such a scenario could also lead to business losses, like buggy software being shipped to the end-customer, a problem that is present in every traditional develop-first-then-test models. This gave rise to more evolving models like Shift-Left testing and Continuous Delivery testing where testing is considered an integral part of the development function.
What Is Shift-Left Testing?
In a traditional development model, testing is performed only at the end of the development cycle in the Requirement gathering (Planning), Analysis, Design, Development, Testing, and Deployment phases. There is a cost associated with testing, identifying, and categorizing issues, and pushing the development changes into the product. The costs might rise if the test/verification team comes across an issue of very high severity (release blocker issue) at the end of the lifecycle. Such issues could even stall the entire release.
With Shift-Left testing, testing is no longer performed in isolation by the test/verification team, but it is performed in sync with the product development. With such an approach, the testing teams are more empowered since they work in close collaboration with other stakeholders of the project, like the product planning team, project planning team, development team, release team, etc. As testing is tightly coupled with the development process, bugs can be identified and resolved at a much faster pace. Though the development and test team work in sync from the beginning of the project, a transition to DevOps is equally important using the Shift-Right approach. The combined approach (of Shift-Left testing and Shift-Right testing) is called Continuous Testing.
As mentioned earlier, Shift-Left testing is more about performing the development and testing jobs in more synchronization; on the other hand, Continuous Testing synchronizes testing with Development and Ops in order to achieve the necessary development and business goals. Continuous Testing brings the best practices of Shift-Left testing by integrating DevOps as well.
The Quality Assurance (QA) team plays a vital role in the process of Continuous Delivery, a process by which new changes are released to the end-user in a more sustainable manner. Their job in Continuous Delivery is to accelerate the deployment by ensuring that bugs are solved by the development team and changes are pushed to the relevant branch such as production environment or staging environment. Continuous Testing ensures that bugs unearthed and fixed at the right time so that a good-quality product is shipped to the end customer.
In simpler terms, Shift-Left testing stresses on the importance of testing at early stages of the SDLC (software development lifecycle). Continuous Testing adds the factor of DevOps to the overall equation. This valuable addition makes it possible to get issues fixed and ship changes to the end-customer at a faster pace.
Differences Between Continuous Testing and Shift-Left Testing
Now that we are aware of Shift-Left testing and Continuous Testing, let’s have a look at some of the key differences between the two
- Testing beyond UI: In Shift-Left testing, there is an increased amount of focus on UI and UX testing before the release. However, a web application or website is much more than impressive UI. Along with effective UI, performance also plays an important role in shaping up the product. As a developer, you have to ensure that transition between the pages is smooth/glitch free, the right mix of web and local components, that a hybrid approach is used, CPU consumption is minimal, thermal requirements are met, and more. These factors which go beyond the look and feel of a website/web application are taken into consideration during Continuous Testing. Hence, Continuous Testing is more optimal since it takes into account UI, UX, and performance metrics which are the key elements for any kind of product.
- More real-time scenarios in testing: It is a known fact that bugs can occur at any stage of product development. This makes product testing an integral part of the development life-cycle. Shift-Left testing focuses on testing at early stages, i.e., the pre-production stage of development, whereas Continuous Testing focuses on pre-production to post-production part of the development cycle. There are some bugs that occur only when your system is being tested under extreme conditions and the possibility of unearthing such kind of issues at an early stage is unlikely since the components/modules are still under design/development.
This is where Continuous Testing can play a pivotal role in performing thorough testing of the product. Consider a mobile application; the app might perform well when it is executing under the ideal conditions, but the performance would drastically vary if the testing is performed on mobile devices that have a low configuration. Along with the performance, the look and feel of a web app might vary from one device to another. There is a huge amount of fragmentation as far as mobile and tablet devices are concerned and it becomes more complicated with the different versions of OSs that exist for the devices. In order to ensure that your web app works perfectly fine on the designated devices and respective browsers, thorough cross-browser testing has to be performed, something that cannot be done on a web app which is still in the pre-production phase. This is where Continuous Testing plays a pivotal role since it takes real-time scenarios into consideration.
- Increased learnings, more pro-activeness: Shift-Left testing empowers the testing/verification team since it provides an opportunity to the testers to work in close collaboration with the product managers, project managers, developers, other test engineers, etc. This helps in inculcating a development mindset within the test team. Such an approach helps them in visualizing issues during the product planning and development phase. Due to this, testers can report issues at early stages of the product development cycle. They feel more motivated since there is a learning opportunity involved in each interaction. In some cases, the thought process of the test team can become linear in approach.
Continuous Testing is performed on a more polished version of a product since it is performed at every stage of product development and testing. Hence, the severity and type of issues that a tester comes across during the process of Continuous Testing can be very different from those unearthed during Shift-Left testing. This makes the test team more informed about critical issues and they can incorporate those learnings at the beginning of each testing cycle. The overall motivation and proactiveness of the test team is more since there is more learning and they can utilize those learnings for the betterment of the product.
- Continuous Integration and Continuous Deployment: Today’s modern development practices make use of CI (Continuous Integration) and Continuous Deployment (CD). CI is a common practice that is used to simplify the overall software release process. CD, on the other hand, ensures that important features and bug-fixes are shipped to the customer on a regular and timely basis. By implementing CI, important changes are merged with the production branch on a regular basis and the same changes are shipped to the customer using CD. Before the changes are shipped to the end customer, they have to be tested and this is where Continuous Testing can play a vital role in the verification of those changes. As the product development reaches a more mature phase, the number of CI and CD cycles would also increase. In order to make sure that a quality product is shipped to the customer, continuous test cycles of Continuous Testing should also be performed (at every stage of CI and CD).
Shift-Left Testing – A Subset Of A Successful Continuous Testing
So we have looked into differences between the Shift-Left testing and continuous testing. But the question that still remains is whether we should consider them as two entirely different testing methodologies? The answer to that would be no! They offer differences, but as we looked at earlier, Continuous Testing involves everything from shift-left to shift-right testing. So, it would be wrong to consider them as mutually exclusive. Rather, Shift-Left testing can be related as a subset of Continuous Testing. When incorporated well, Shift-Left testing with continuous testing could unleash much faster and robust product in the market.
More and more organization functions are transitioning to practices like Continuous Testing since the testing process is more severe, extensive and thorough. However, in order to achieve a better ROI, it is crucial to integrate Shift-Left testing with Continuous Testing. Shift-Left testing should serve as the foundation for your continuous test cycles. Also, before your organization functions transition to DevOps, make sure that all the dependent functions are ready for the change since each building block is critical in the process.
Published at DZone with permission of Himanshu Seth . See the original article here.
Opinions expressed by DZone contributors are their own.