Mobile Testing Basics: Manual Vs Automated Testing

Introduction

Mobile devices hold a very special place today in everyone’s life. Our life is dependent on them for buying groceries, ordering food, learning, paying bills, banking requirements, yes almost everything. Mobile applications are ruling the world currently with easy accessibility, good UI, excellent responsiveness, solving our day to day requirements with the tap of fingers. 

Efficient testing, which ensures quality, and a great user experience are vital for the success of a mobile application. In this article, we will see the basics of mobile testing related to manual and automation testing.

What is Mobile Testing?

Let’s start from the very basic and understand what mobile testing is.

As the name implies, any testing that is done on a mobile device is called mobile testing. It can have two variations:

• Mobile device testing: This means testing the device itself, and also involves some interaction with the hardware of a mobile device. For e.g., screen resolution, Wi-Fi, Bluetooth, SD card, internal hardware, camera, radio, etc.
• Mobile application testing: This means testing the mobile application, and involves the software working on the mobile device. E.g., screen layout, buttons, speed of transactions, performance, security of the network, etc.

Why Do We Need Mobile Testing?

Today we have a huge variation in mobile devices in the market. After mobile application development, we cannot be sure that the application will work on all of them. So then how do we ensure that our mobile application works on the devices we target? We can do so by effective testing. We require it because of the below variations in the mobile devices:

Types of Operating Systems

• Android
• iOS
• Windows
• Blackberry

Types of Devices

• Mobile phones
• Tablets
• E-book readers
• Smartwatches

Types of Applications

Below are the three types of applications that we run on mobile devices:

• Native apps: These are specifically developed to work on a platform i.e. iOS or Android. They have access to mobile device’s GPS (location), contact list, camera, SMS, etc. These are the apps that are present in the app store (Google play store/Apple app store) for download. E.g., WhatsApp
• Mobile web apps: These are actually websites that work on a mobile browser and are different from the native apps, as in, they don’t need to be installed. E.g., financial times
• Hybrid apps: These applications are a mix of Native and Mobile web applications. They broadcast the content from websites in the form of mobile applications. E.g., Instagram. More details on the type of mobile applications here.

Types of Mobile Testing

We can perform mobile testing in two ways:

1. Manual Mobile Testing
2. Automated Mobile Testing

Manual Mobile Testing

Manual mobile testing is performed by QA engineers manually. It is performed without using any automation script or automation tool. The QA team manually performs all actions on the real mobile devices and captures the test results. It is evident that manual execution of test cases will require more time and effort but we cannot subside it completely. There are scenarios that require human attention and an eye for detail.

Examples of such scenarios are Usability Testing, Exploratory Testing, etc.

Advantages

• Manual testing is perfect for user experience-related scenarios. For testing usability, UI intuitiveness, ease of use, etc., we require a real user who can perform these tasks and provide feedback based on the human experience.
• No training required for learning automation tools or scripting because all tasks will be performed manually by the user.
• Complex scenarios that cannot be automated are always tested using manual testing. For example, gaming applications.
• For small projects, automation can become expensive and it is better to use manual testing in such cases.
• Manual testing is a better choice when it comes to finding out the reason behind memory crashes and application hanging.
• Help perform real-life scenarios and identify probable issues.

Disadvantages

• More prone to error since it is performed by humans.
• May require more effort and time for test case execution.
• Requires more human resources for test execution.

When To Use Manual Mobile Testing

• Usability testing: Tests the user-friendliness, intuitiveness, ease of use of the mobile application. 
• Exploratory testing: The user manually explores the mobile application without any prior test cases. It is based on the creativity and thinking of the user hence can be only manual.
• Device crashes: There are many reasons for devices crashes and to identify the root cause is a tricky task. Sometimes, due to memory leaks, the device may lose the memory available for mobile apps. Therefore, memory leaks are one of the reasons for a mobile app crash. To identify the possible loopholes in the code which are causing memory leaks, testers and developers do memory profiling and debugging using a combination of both BlackBox and Whitebox testing techniques.
• New functionality: While releasing a new functionality/utility through mobile application in the market, it is tough to know how the customers would receive it. If they would like it or not. Therefore, initially, it is recommended to test this functionality manually only. Once functionality is final and stable and is well-accepted by the users, automating the test cases is the next logical step. 
• Ad-hoc testing: It is the least formal method of testing. The user performs testing without any documentation to find bugs. More about Ad-hoc testing here.
• Localization testing: Requires testing that the application fulfills all requirements, logos, text, messages as per a specific region/country.

Automated Mobile Testing

Automated mobile testing uses automation tools and scripts to execute test cases on mobile devices. Human interaction is not required in the case of automated mobile testing. We can always automate the repeating test cases, like regression test cases in the test suite. Also, the testing which is impossible to perform manually like performance test cases needs to be automated. We may think that we can automate the whole testing process and then sit back and relax. This cannot be the case because we can never automate testing for the user experience; we require a real user for that purpose. 

Advantages

• Automation mobile testing is more reliable than manual testing since it is not prone to manual errors. 
• Automation tests can run 24 hours a day, hence it will provide more coverage than manual testing in the same time frame.
• Parallel testing can be performed using automation tools. Several test cases can be run simultaneously; this will increase the coverage tremendously. One such tool is Testsigma, which allows parallel testing on thousands of environments without any setup hassles.
• It is a better choice for large organizations and big projects. Once automation scripts are ready then the Return On Investment (ROI) is higher.
• There are some scenarios which are difficult to perform manually like performance testing (load, stress, volume, etc.). In this case, we take the help of automated testing tools.
• Helps in the early detection of bugs and thus reduces the fixing cost later in the development stages.
• Test cases are reusable; the same repetitive test cases can be reused several times without any hassle.

Disadvantages

• Cannot execute user experience-related test cases efficiently. We can better rely on manual testing for that purpose.
• It is not advisable to use automation testing for a small application with small features because it may take more time to write scripts for the test cases rather than manually executing them.
• Maintenance of test scripts according to the changing requirements and environments is usually time-consuming. There is one tool that may save your precious maintenance time because it does not require coding skills and environment setup. 

When to use Automated Mobile Testing

• Parallel testing: Many test cases can be run in parallel on multiple real devices using automation tools.
• Large projects: For large projects, it is advisable to use automation tools, because manual testing may take more effort, human resources, and time.
• Repetitive test cases: Repetitive cases like sanity and regression test cases should be automated.
• Performance test cases: Load, stress, volume, and endurance test cases can not be executed manually because they usually require large-scale test execution.
• Complex scenarios for manual execution: Complex calculations and other complicated scenarios which are difficult for manual execution are executed via automation tools.

Conclusion

Now we understand the pros and cons of both manual and automated mobile testing. Hence, it is clear that both have their own importance and usage. Some scenarios are better run manually while a few are best to be automated. Together with the functional test cases, we require test cases that evaluate battery consumption, network connectivity, performance, memory usage, memory leaks, etc. Few of these can be checked manually and for a few checks, we may use automated tools. As such, a hybrid approach to testing is the most successful method.