What Continuous Delivery Means for Testers, QA, and Software Quality
Continuous delivery touts faster time to market and better quality software, but how should testers and QA adapt?
Join the DZone community and get the full member experience.Join For Free
If you’ve been a software tester for any length of time, you’ve likely noticed the shift toward continuous delivery, whereby businesses and project and operations teams aim to safely and quickly release new builds to production, ostensibly at the push of a button. The realization of continuous delivery means faster feedback, improved time to market, increased quality, and a better customer experience, though not necessarily in that order.
What’s All the Fuss About Continuous Delivery?
Continuous Delivery gives developers rapid feedback on their code, which leads to improved productivity. In theory, code can be written, tested, reviewed, merged, and integration and acceptance tested before it even gets into a tester’s hands. Rapid, reliable and high-quality releases mean happier customers, which often translates into increased business revenue.
Development and operations teams are justifiably excited by the idea of moving faster and, with support from the business, they tend to be the driving forces behind a transition to continuous delivery. But with all the focus on development and operations, it’s easy for testers to feel left out in the cold.
Challenges of Working in a Continuous Delivery Environment
It’s not unusual for testers to struggle with the transition to continuous delivery. In this brave new world, there’s a strong focus on:
- Automating as much testing as possible
- Delivering features in small chunks, many without a user interface to test against
- Features developed in just hours or days, with little time for planning and preparation
- Accelerated time to market over perfect software
These changes in perspective require testers to not only optimize their existing skill set but to learn some new tricks.
When software is delivered continuously, testing needs to be continuous as well. It must be integrated with every stage of the development process, from identifying the story as a business requirement to determining if customers like and are using a feature once it is delivered to the production environment.
Strong technical skills are required when working in a continuous delivery environment, but soft skills, such as the ability to question and apply critical thinking to requirements, products, practices, and processes, are invaluable as well. When working on a continuous delivery project, it’s helpful to think about
- Whether the team is building the right thing in the first place.
- Whether the business requirements can be captured as executable specifications.
- Whether you can add value by working with the team to identify and express acceptance criteria, scenarios and examples.
- Whether processes are working or can be improved.
- Where the waste is and how it can be removed.
- How to approach non-functional testing, performance, security, accessibility, etc.
- What devices, browsers, operating systems, and environments the product needs to support and how you can test those.
- What data and configuration are required across all of the environments in your build pipeline.
Much of a tester’s job relies on effective communication. Testers should be comfortable discussing bugs and issues directly with individuals or with a team. Clearly explaining your goals, how you plan to achieve them, and the problems you might face will go a long way toward generating support in the form of more testable products.
A Layered Test Strategy
In the Continuous Delivery world, it’s helpful to think about layering testing to focus on different levels of architecture and functionality that includes:
- Exploratory testing
- Automated user acceptance testing
- Automated integration testing
- Unit testing
Now think about how you’re going to accomplish this model. Do you need to learn how to program to better understand what’s going on, or to automate day-to-day tasks? Or should you focus on what the test infrastructure looks like and the various abstractions it needs to contain? Coverage is an important consideration as well.
As the testing expert, it’s important to think about the tests your team needs, in order to deliver the quality your customers expect.
When working in a Continuous Delivery environment, tests should be as lean as possible. Exploratory Testing is ideal, but you need some way of managing the process so other team members can pick it up.
One option is to use a Session-Based approach that defines the scope of the testing as a kind of mission statement or “charter.” That charter might consist of a set of ideas or heuristics, or it might take the form of a tour with an itinerary or set of landmarks to visit, including helpful resources for the journey. Another option is a Persona-Based Test that focuses on the needs and objectives of a specific user.
Ultimately, how you define your exploratory tests is up to you as long as you keep in mind the following principles:
- Keep the tests lean
- Timebox your tests
- Provide enough information in the mission statement or charter for someone else to pick them up if necessary
- Document the outcome of your testing
Test Coaching and Testing Tools
Even if you are technically inclined, there’s no need to dive right into the code. Instead, encourage the developers to take a test-driven approach to developing their code. Help them identify edge cases and scenarios. Steer them toward implementing a Behavior or Acceptance Test Driven Development (BDD/ATDD) approach by working closely or directly with them.
When you’re not coaching, research and implement tools to make exploratory testing faster, easier and more effective. Consider agile testing tools to help you manage testing across the entire team or organization or other tools to help you interrogate and exercise your system.
For example, automated scanners can identify flaws and vulnerabilities in your solution, but whatever tools you use, endeavor to become an expert in supporting your team with testing solutions developers haven’t thought of yet.
Focus on the Benefits
It’s been my experience that testers struggle when trying to adopt the Continuous Delivery worldview, particularly when they come from a more traditional waterfall-style background.
Rather than focusing on the skills Continuous Delivery requires, it’s helpful to focus on the benefits Continuous Delivery delivers:
- Accelerated time to market
- Ability to build the right product
- Improved productivity and efficiency
- Reliable builds
- Improved quality
- Improved customer satisfaction
I use these benefits as a set of heuristics to help guide my thinking and decision-making.
As a team, if we’re failing in one of those areas, what can I do to help? A combination of the techniques I’ve discussed above goes a long way toward identifying a solution.
Opinions expressed by DZone contributors are their own.