Overcoming Testing Challenges In Agile
Overcoming Testing Challenges In Agile
With more more frequent transformations in technology than ever before, Quality Engineers and the testing they conduct cannot be left out of an Agile transformation.
Join the DZone community and get the full member experience.Join For Free
You've been hearing a lot about agile software development, get started with the eBook: Agile Product Development from 321 Gang.
Agile software development introduces ways of working which is very different from traditional software development. The differences can be seen in the way people collaborate with each other, develop and test software by adopting new and improved technical practices, leverage technology to create smart solutions and focus on delivering new, delightful customer experiences. The crux of Agile software development lies in delivering working software frequently with preference to shorter iterations. Unlike traditional ways, developers and testers work together continuously and in parallel. While this methodology works wonders for organizations as it benefits their customers, embracing Agile is not an easy task for most of the Agile teams and even more difficult for the testing community.
Working as Quality Engineer in an Agile environment is challenging. The expectations from this role have changed and evolved over last decade or so. Quality Engineers in Agile teams have faced challenges such as frequently changing requirements, incomplete and unclear requirements, lack of focused testing, insufficient unit testing by developers, less examples and scenarios by Product Owner, etc. Apart from these common challenges, below are other challenges that doesn't' get highlighted but have larger impact on the overall software delivery.
In Agile, testing is not done in phases or after the entire development is over. Instead, it is a continuous activity, right from the start of the project until the end of it, and goes on in iterations. For teams that are new to Agile, often these iterations become mini-waterfalls, making the testers wait until the user stories are developed. Testers are left with the last 2-3 days of the iteration to test the entire iteration backlog. This results in less time for testing, sometimes compromising on the scope and quality of testing. This happens because the team adopts the process but not the Agile mindset.
To overcome this situation, teams and other functional groups should be encouraged to adopt Shift Left mindset. With the focus on continuous testing, it is important to bring in Quality Engineers early in the project lifecycle and not after development is complete. Throughout the project duration and during iterations, testers should work in collaboration with developers, business analysts, and the Product Owner. They should get involved during the inception phase and understand requirements, ask queries, start thinking about overall testing strategies (both manual and automated), think about acceptance test scenarios, integration scenarios, and write test cases. During the iterations, start designing test cases early, prepare test data and review test environment. Get test scenarios reviewed from the Product Owner and developers. This will ensure that everyone has the same understanding of the user requirements. And when user stories are ready, start testing incrementally, communicate the results to developers and the Product Owner and seek their feedback. Continue with this approach and testers won’t be left with a long backlog to finish in a hurry.
Upskilling to Automation and Other Technical Areas:
Agile without automation has limited success. Cost reduction and speed being high on the management executive's agenda, there is no other alternative to automation. This leaves no room for having a mix of manual and automation test engineers. Keeping pace with fast changing technology is another challenge. Upskilling remains one of the hardest challenges for test engineers. Often it is observed that upskilling is more of a mental block than capability issue, and it takes time and patience to bring in this change. These changes are real and test engineers have no choice but to learn new skills. They have to go through learning new languages, different automation tools, understand automation frameworks, and technical practices such as TDD, BDD, and ATDD, to name a few.
To overcome these challenges, quality engineers should be put through a time bound and systematic training plan. Management support is most crucial during this transition. Leverage in-house competency centers and training academies to facilitate training. Define skill-based maturity levels so that the transformation is smooth and practical, and have mentors mentoring them during this transformation. After the initial grooming and mentoring, they may start to work on automation on their own while their work is being peer-reviewed with constant feedback. Working in pairs also works great initially. Management may also consider making a repository of knowledge and FAQs or conducting knowledge sharing sessions periodically. Make continuous learning a habit, and with time they will be transformed into a complete test engineer.
Poor Automation Strategy
Automation is not limited only to testing but extends all the way from continuous code integration to continuous deployment. Poor automation strategy can defeat the very purpose of going Agile. Automation can take months when envisioned poorly. Building, testing, deploying and monitoring are the important functions for automation. Often it is observed that each of these functions define their own automation strategy which could be completely disjointed from each other. Other issues include, but are not limited to:
Starting point of the automation;
Availability of resources with required knowledge and skill sets;
Having the right set of automation tools and the availability of license;
Availability of infrastructure and environment;
An alignment between testing and business priorities;
The scope of automation and the approach; and,
Automation is a big change that involves huge effort and the complexity is also very high. Organizations usually look for internal experts and start with trial-and-error knowing that some of the decisions can go wrong, or they hire experts who have deep understanding and expertise in the area. The decision is mostly influenced by a Make or Buy analysis. Once the decision is taken, create a task committee who will be authorized to drive the change. Take small steps, select a pilot project, train people, appoint champions, and monitor results constantly to provide periodic updates. Lessons from pilots have always helped to fine tune the strategy and bring in more areas under automation.
Risk of Poor Test Coverage
Agile thrives on changing requirements, starting with minimum known requirements, and knowing that requirements will evolve. This means continuous development, continuous integration, and continuous testing. Frequently changing code base has a risk of compromising on test coverage missing on testing certain areas.
Metrics around test coverage and maintaining traceability across different artifacts are effective ways to overcome this challenge. Another solution is automation. The automation strategy should address areas that are musts for testing and will deliver maximum benefits. Definition of Done should include testing metrics. Every test case and every user story should have backward and forward traceability. Tools should be leveraged to identify missing test coverage supported by metrics. Seamless tools integration, easy accessibility to artifacts, and regular source code analysis are some of the measures that can be taken to ensure all requirements are adequately covered from a coverage perspective.
Misalignment of Objectives With Other Stakeholders
In organizations where testing is an independent verification and validation function and is a shared service for IT, the misalignment of objectives on the part of various IT teams can create a lot of burden for the testing function. The issue gets even more complicated when certain sections of IT works in an Agile way while others don’t. Testers have to do testing for different IT teams in parallel. Misalignment and varying priorities mean constant changes in plan and execution, often resulting in long release cycles and sometimes fatigue. More often than not, it is the testing function that becomes an easy scapegoat for schedule overrun.
The best way to overcome this challenge is to do release planning at the portfolio level, involving all stakeholders rather than operating in silos. During the release planning meeting, dependencies, risks, and a release plan for each team should be clearly called out and communicated to all. A steering committee comprised of stakeholders from different groups holds the Scrum of Scrums where risks, issues, dependencies, and current and upcoming backlog are discussed and appropriate action is taken to make sure that the program remains on track. The testing activities are planned and re-planned depending on the outcome of these meetings. Since Scrum of Scrums involves a forward looking agenda, QE gets ample insight and scope to plan their activities.
Quality Engineers are now shouldering increased responsibilities; they are dealing with complex environments all the time, collaborating very frequently with different teams. This leaves them with stretched bandwidth to complete their tasks. Strong support and cooperation from others is what they need for them to be successful. Proper time management and effective communication are key factors for successful deliveries and overcoming these challenges. If managed properly, there is no reason why quality engineers won't succeed in an Agile environment. Happy Testing!
Opinions expressed by DZone contributors are their own.