Test Coverage Planning Is an Empirical Game

As 2015 comes to a close, are you testing for the whole digital experience, on the right devices, operating systems and browsers?

This is the question that app developers, testers and product managers ask themselves as app quality becomes a major focus for not only development and testing teams, but for the entire organization. The stakes are raised for delivering a high-quality end user experience across mobile and web apps. In short, the quality of digital experiences, across mobile and web, matter more than ever. So how can you know what the right mobile devices, operating systems and web browsers are for testing your app?

• Start with what you know. As organizations seek to understand their end-users’ app usage, they obtain unique web traffic analytics and other insights coming directly from their customers specific devices. Such information is extremely useful for making decisions around the next app capability, enhanced quality for specific devices in specific locations and more. With that in mind, I recommend you use this level of analytics for your mobile, web, responsive web strategies. It only makes sense, organizations should test for the conditions, devices, and platforms that its customers are using.
• Pair with industry data. There's no better guide for your testing than real enterprise data. I’ve just published the third edition of the Digital Test Coverage Index to help you determine the right test matrix of mobile devices, operating systems and web browsers that represents your customers, markets and geographies – you can get it here. We aggregated data from our customers on over 4,000 devices and combined it with mobile market sales data and our own analysis, to create a list of the most relevant devices, OS versions and web browsers. It guides you in the right direction based on target geographies and market data so you can optimize your test coverage strategy.
  
• Keep an eye out for the future. Test coverage planning is a continuous project. For instance, did you know the average shelf life for a device is less than 9 months and shrinking, fast? What was the top device a year ago is now totally irrelevant, and your test coverage plan must reflect these constant changes. In the Digital Test Coverage Index, we've included a calendar showing what mobile devices and OS versions are either new to the market or releasing in Q1 2016, and a 2015 Mobile Retrospective that looks back at the year's major releases. This should help you not only react when new devices, OS versions or browser versions hit, but help you proactively prepare for the market updates.

Optimizing a test coverage strategy that matches end user scenarios, keeps up with the constantly evolving mobile and digital markets, and is prepared for what is yet to come is definitely a challenging part of mobile testing. Developing a test coverage strategy that addresses customer usage data, market data and the future of the market, however, will set your app apart from the rest and lead to the highest-quality end user experience. For help in aggregating data, download the Digital Test Coverage Index here.