What Developers Need to Know: 3 Steps for Effective Mobile Testing

DZone 's Guide to

What Developers Need to Know: 3 Steps for Effective Mobile Testing

A master at mobile app development and testing offers some advice for planning and executing an effective mobile test strategy.

· Web Dev Zone ·
Free Resource

This article is featured in the new DZone Guide to Dynamic Web and Mobile Development. Get your free copy for more insightful articles, industry statistics, and more!

These days, developers are handling an increasing share of the software testing workload. This is a good thing, as it means testing is taking place earlier and quality is being owned by more of the organization. And with people accessing the internet on mobile devices far more than traditional desktops, it's essential that developers are equipped to create and execute an effective mobile testing strategy. Gartner has estimated that this year, more than 50 percent of users will go to a tablet or smartphone first for all online activities, so software development today must have mobile in mind from the start. However, developers aren't always prepared to take a strategic approach to testing: they're accustomed to writing code, not testing it. This article offers a crash course in creating an effective mobile test strategy for developers who need to get up to speed and make decisions quickly.

Here, we'll explore three specific, practical areas needed for developers to get started on a mobile test strategy. First, we'll cover how to choose an automated testing framework. Second, we will review the pros and cons of testing using emulators and simulators versus real devices. And, finally, we offer suggestions on how to decide which tests to automate— and which to continue testing manually.

Tip #1: How to Choose an Automated Testing Framework

There are several frameworks to choose from when it comes to automated testing. Choosing one can be intimidating, but it's an important first step.

Selenium is the industry standard and is a widely-used framework. It's a great choice if you need to automate tests across a variety of browser and operating system combinations. But for mobile testing, in particular, you'll likely want to use Appium, Selenium's mobile cousin. You should also consider native frameworks, such as XCUITest for iOSand Espresso for Android.

It's important to remember that you don't just have to choose one. You should have multiple frameworks to choose from because some lend themselves better to specific use cases.

Speaking of use cases, before you choose a mobile test automation framework, you must understand what you need to test. Here is a checklist that can act as a starting point for determining your mobile testing requirements:

  • Which mobile platforms do you need to test on? Are you looking at iOS, Android, or both? Also, which versions are the most important?

  • Is your app native, hybrid, or mobile web? If it's mobile web, which browsers (and versions) are the most important for testing?

  • Which elements of your app are you most concerned with testing? GUI? Memory and resource use? Load and stress? Basic functionality? All of these?

  • Are there any specialized/unique features that you need to test? Do you have a clear strategy for testing them?

  • Do you need code-level testing? Black-box, white-box, or gray-box?

  • Approximately how many different tests (within an order of magnitude) do you anticipate running? How many of those can be run in parallel?

  • Which test scripting languages do your test engineers have the most experience with? Do they have a preference (e.g. are they happy with Python and JavaScript but don't want to touch Ruby or C# — or vice versa)?

These questions offer a great starting point for developers needing to get over the hurdle of determining which frameworks to test on the path to mobile test automation. However, don't feel you have to choose just one. It's a good idea to adopt a test infrastructure that allows you to change it up between different frameworks as needed. Cloud-based testing supports this flexibility. If you set up your test environment on a single framework on your own infrastructure, you will need to make a lot of changes in order to switch frameworks if you find yourself needing to use another. But when you use a public testing cloud that supports multiple frameworks, you're all set to switch back and forth as needed.

Tip #2: Test on Both Emulators/Simulators and Real Devices

The use of emulators and simulators to test mobile applications has historically been met with resistance. This is based on the perception that if you're not testing on a real device, you're not really testing at all. I beg to differ! Although real devices do give more accurate test results, using them alone is by no means ideal for continuous testing and continuous delivery. Testing with real devices can be costly, too. Due to budget issues, some organizations don't bother at all with real devices as they're too expensive, and, instead, opt for emulators and simulators.

The reality is that effective and efficient mobile app development requires both emulators/simulators and real devices. An emulator, as the term suggests, emulates the device software and hardware on a desktop PC, or as part of a cloud testing platform. The Android (SDK) emulator is one example. A simulator, on the other hand, delivers a replica of a phone's user interface and does not represent its hardware. The iOS simulator for Apple devices is one such example. Emulators enable parallel testing in a way that can't be achieved having to manually prepare each emulator for the tests. Further, automation is easier with emulators as the tests can be executed without manual intervention and can be controlled remotely.

Image title

A combination of emulators, simulators, and real devices provides the best coverage for all use cases. 

The bottom line? Using both emulators/simulators and real devices offers the best coverage and performance at a reasonable cost.

Tip #3: Determine Which Tests to Automate

Believe it or not, your goal shouldn't be to automate every test. Automated and manual testing are both required for an effective mobile testing strategy. Automated mobile testing can be applied to cumbersome tasks and it's very useful for regression testing at scale, as well as for load tests. To effectively identify which tests are the best candidates for automation and prioritize them accordingly, start with the following questions:

  • Which tests will be performed most often? These are often the best ones to automate.

  • Which tests involve purely technical elements (such as whether an application starts as expected in a given browser), as opposed to tests that are subjective in nature (such as how users interact with a UI)? Purely technical tests are easy to automate; those that have subjective elements may need to be performed manually in many cases.

  • Which tests take the longest to perform manually? Automating the most time-consuming tests will deliver the greatest benefits in efficiency

Answering these questions will help get you started in the right direction. At SauceCon 2018, Angie Jones presented a great framework on how to choose which tests to automate. Her session offers a thorough, structured plan for determining which tests merit automation.


With developers taking on more of the software testing burden, it's important for them to know how to get up to speed quickly with testing. And when developers can add software testing to their list of skills — particularly mobile testing — it makes them more well-rounded and valuable to their organizations. With the tips from this article, developers will be able to plan and execute an effective mobile test strategy.

This article is featured in the new DZone Guide to Dynamic Web and Mobile Development. Get your free copy for more insightful articles, industry statistics, and more!

application testing, mobile app testing, mobile application development, web dev

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}