Major Impediments to Continuous Testing
Major Impediments to Continuous Testing
Check out some major impediments to continuous testing.
Join the DZone community and get the full member experience.Join For Free
Along with continuous delivery and continuous integration, continuous testing enters the game. The notion started spreading a good five years ago, yet today, continuous testing is not ubiquitous. Read our article to find out more about a not-so-easy transformation of the testing universe.
The Start of the Continuous Testing Era
Why all the fuss about continuous testing? How come testing procedures, usually left almost forgotten up to the end of the SDLC in the past, are now required at every project stage and have finally acquired the same level of significance as development in terms of project delivery? Historically, test teams were given far less time and resources than dev teams, as testing was considered a sort of supporting service, and responsibility for the project outcome was mostly laid on the genius of a developer.
See the time/significance interdependency of the SDLC phases in the picture below.
However, things have changed to a great degree due to the transformation of industry appetites. Today’s releases are much more frequent than they used to be, say, five years ago, and the price of a mistake is much higher now. Consequently, an app that is released monthly, weekly, daily, or more frequently demands a different approach to code writing and testing as compared to the traditional waterfall-based cycles and will be created under the laws of Agile and DevOps. It's only natural that extreme appetites of the IT-industry spurred a new-level, extreme development, and testing.
Another contributing factor to the rise of a new testing era was actually the rise and fall of test automation. Test automation, when applied appropriately, gave us a speedy delivery, cost reduction, and eliminated the human factor from our testing procedures. Nevertheless, since test automation isn’t an option for many projects, and many companies still have to do the job manually, the global IT-community had to admit that something went wrong with such a brilliant thing as test automation and it didn’t work as intended. It failed indeed, and 70% of global testing, which is still done manually, confirms that.
Continuous Testing Steps in…
At this sad moment, continuous testing steps in and joins continuous delivery, which, frankly speaking, came as no surprise as “continuous delivery needs continuous testing”, according to Forrester.
Continuous testing was the bullet to hit the two above-mentioned objectives — speedy and high-quality delivery according to the best agile practices and rehabilitation of test automation. Let’s study the notion in more detail.
Continuous testing was designed to enable uncovering of defects and bugs at the earliest stage possible through “atomic” tests, as Adam Auerbach, Vice President of Quality Engineering at EPAM Systems, calls them in his article. These small, independent units, which have no dependencies with other tests, are launched each time a change is introduced to the system. In such a way, CT ensures quick and reliable feedback on the current state of the system, i.e. tells if the change has negatively affected the system and gives go or no-go results to the developer.
Key items of continuous testing are people, processes, and technologies. As for people, continuous testing implies empowering developers with testing capabilities. CT philosophy urges the developers and agile teams to perceive testing as part of their responsibility. Processes within CT vision are characterized by ultimate automation. Technologies should be cutting-edge and open-source that support automation and are easy-to-use for developers.
However, as it goes, these three notions are also the key impediments to the introduction of continuous testing at a global scale. Why? See below.
Key Items = Key Impediments to Continuous Testing
What should a company leader do to make its people not simply sit at their desks from 9 to 5 but continuously add value to the processes they are involved in? This is not an easy nut to crack; however, this issue, if resolved, will help a company apply CT philosophy in its daily activities. People are very reluctant to wear many hats, though CT demands exactly this. “The people involved, developers and testers, must become part of this new culture. There should be a DevOps organization set up to hire or retrain people into more well-rounded engineers who can wear many hats,” the Definite Guide to Continuous Testing says. Developers capable of worthy testing and testers enabled to write a healthy code, that’s what this new culture requires. Here’s the tricky bit: how many companies are truly adequately staffed for this transformation?
Moreover, attitude towards automation, which lies in the core of CT, can also be a challenging aspect. As automation has proven to be quite an ambiguous benefit from time to time (due to its high cost and numerous application limitations), management along with the staff may consider manual, not automated tests to be a safer option for the project. This opinion is reasoned inImpediments for software test automation: A systematic literature review. In this paper, the authors have studied, among others, behavioristic impediments to test automation application. See the abstract below accompanied by the picture for better comprehension.
“Consider the case where automation is not viewed as useful. This will likely reduce the willingness to invest in the automation by allocating people and time to the work. Less time and people available to do the work will likely lead to low-quality automation. Low-quality automation is not as useful as high-quality automation, which reduces the perceived value and the feedback loop is closed.”
That being said, CT, which actually is about multitalented and super involved people, can stall right at its start. To avoid this, consider continuous talent training programs and ensure that well-tuned corporate processes and cutting-edge technologies accomplish the appetite for perfection and value of your talents.
As a risk-based alternative to the traditional all-encompassing testing practices, the processes in CT should be tuned so as to quickly unveil fresh bugs after adding new code to the system through targeted automated tests. Fail fast is the mantra of the followers of the CT-philosophy. Why worry about such a trifle thing as a small bug if 100% testing is impossible anyway? The thing is, such bugs can potentially cause a specific risk to materialize, which a company identifies as significant for the app in question. That’s why the processes should be corrected accordingly, to enable targeted automation testing of small code portions.
Got it, what’s wrong here? As it happens, automation has a limited application scope. Under the left-shift testing rules that apply to CT, “Developers test as they go, with test automation including functional testing, performance testing, monitoring, and security testing” (DevOps). Among the above-mentioned test types, only two — functional and performance testing — can be automated with a sufficient degree of reliability.
As Don Prather, Technical Program Manager at Axway Software, states in his article, tests launched during continuous testing should meet flexibility, coverage, and effectiveness requirements. In other words, tests must be flexible so that they can be run at any time; test coverage should be maximum within a given period to test; the effectiveness of the tests must ensure immediate detection of hard-to-pinpoint defects. Thus, functional and performance tests are perfect candidates for CT-implementation due to their nature, as they are almost never done manually and meet the above requirements.
However, for security tests automation, this area is still in its infancy. Justin Arbuckle, vice president of worldwide transformation and chief enterprise architect at Chef, highlights that many enterprises hold onto manual security testing because of their concern with verifiable and auditable trails. “Companies want to have someone reviewing every line of code to make sure it’s up to regulation standards and there aren’t any holes leaving a company open to breaches.” Hence a huge lump of testing work is still done manually, not to mention UI/UX testing or exploratory testing.
For the developers to be able to test out their work, developer-friendly tools are required. In terms of availability, this is not a problem anymore because today, you can find various automation tools for many application purposes. However, the cost of such technologies can be devastating to the efficiency of the project, and CT, as well as CI and CD, are all about achieving the highest efficiency possible.
Do Nguyen (LogiGear) supports the idea and includes start-up costs to the major barriers of continuous testing. “The initial effort required to pick, set up, and configure the tools and frameworks to get to a running automation process can be daunting. The heavy investment for research time and substantial financial investment coupled with the added possibility of interrupting your day-to-day operations have been a big deterrent for many organizations.”
Is there a chance to do without top-notch testing tools in achieving CT-objectives? The Definite Guide to Continuous Testing says No. The fact is, many companies stating that they stick to the CT-approach in their projects continue working with “legacy” tools, which sufficed for waterfall. “They [legacy tools] are an impediment to testing at the speed of agile. They don’t support automation in the right way for a continuous integration environment and they are tools that developers refuse to use. There needs to be a new set of tools that won’t get in the way of speed and quality; tools that enable quality to be built into the process and not “just” test the application as in the past.”
As we can see, the revolution in the testing industry cannot run as smoothly as planned. However, much was, is, and will be done to enable a full-scale transformation of the sector. CT promises a lot: speed, quality, and new revenues, and in spite of many initial blockers, it will find its way to the acknowledgment by the global IT-community, like Agile, DevOps, CI and CD did.
Published at DZone with permission of Veronika Chizh . See the original article here.
Opinions expressed by DZone contributors are their own.