Mobile app development has introduced a range of new challenges for software testers, and it seems that many teams are struggling to overcome them. A recent Evans Data survey of 500 developers found that 75% of mobile apps ship with between 1 and 10 bugs, and 20% of apps are averaging between 11 and 50 bugs. Is there something seriously wrong with our testing strategies? Are business pressures pushing software out the door before it’s really ready?
Budgets and resources may be tight, but a lack of testing always leads to a lack of quality and polish. There’s no such thing as defect-free software, but it’s not enough to point at a rapid patching plan. And yet, that’s exactly what many developers are doing. A full 80% push updates containing bug fixes at least monthly, and about a third of developers push updates out weekly.
Has this new continuous release model led to a relaxation in quality standards?
Finding the Right Mix
Regardless of how many manual testers you have, or how much time is available, you can always craft a plan to maximize coverage based on what you have. To boost the quality of your software and find more defects as quickly as possible, you should adopt a mixed approach that integrates automated testing into the build pipeline, and frees up testers to go off-script and engage in exploratory test sessions.
The right balance will depend on your project. You have to think carefully about what tests to automate. Key repeatable processes, vital user functionality and stabilized features that are unlikely to evolve much are strong candidates for automation. By starting out with manual tests, you can nail the steps and scripts down, and then automate them to prevent the workload from becoming unmanageable as the project grows.
If you want to get the most value out of your tester’s skills, you won’t confine them to traditional scripted testing. Exploratory testing beats scripted testing when it comes to finding serious defects as quickly as possible. It also leads to greater satisfaction and fulfillment for testers, but you have to implement it properly.
Maximize Testing Efforts
It’s important to visualize how the software works and strategize about where time could best be spent. What’s most important to the end user? Put yourself in their shoes. This is context-driven testing, but you still need good charters defining areas to focus on, and efficient data collection and management for all of your manual test sessions.
You can run exploratory testing in parallel with the automated test suite. It doesn’t require a specific block of time, you design and implement tests that make the best use of all available time. It’s challenging, exciting, and it really pushes testers to take responsibility, think about how to continuously improve their approach, and make the absolute best use of their time.
Tackling Mobile App Testing Challenges
Any mobile app testing approach is going to have to take fragmentation, usability, security, connectivity, and a range of other factors into account. It’s a complex challenge, and with the growth of the Internet of Things, that challenge is growing every day. Mobile software testers have to prioritize in terms of target devices, they can’t cover every platform version thoroughly. This is all about prioritizing correctly to get maximum value from the limited resources on hand.
A holistic approach that encourages and empowers testers to look at the big picture and test based on user expectations is the best way to uncover those serious defects faster, and make the most of limited testing time. Instead of walking the same well-worn paths, they are free to deviate if they spot something worth investigating.
If you’re in any doubt about the efficacy of an exploratory approach, then you should set up some metrics and put it to the test for yourself. You may find the same serious defects with a scripted approach eventually, but it’s going to take you longer. The benefits of finding serious defects quickly so you can get to work on them early should not be under estimated.
The vast majority of apps are going to ship with defects, and updating software after release is the norm, but with the right approach you can still find and eliminate the most serious defects before you ship.