Top 5 Open-source Automation Tools for iOS and Android

Test Automation nowadays plays a very important role in product testing. There are multiple positives to go implement test automation. This includes minimizing test execution time, ensuring larger coverage in less time in critical release phases, and reliable and repetitive runs during product development phase to ensure no regression issues are introduced. And it minimizes the risk of human error or negligence while doing repetitive regression testing cycles.

There are a lot of tools available in market, both open source and paid. Though there are pros and cons of using both paid and open source tools, the latter category of tools are widely used across the automation testing community. Choosing the right automation tool according to your project requirements is very tricky.

Below are the 5 top open-source iOS and Android automation tools.

1) Calabash (For Android and iOS)

What it is – Calabash is an open source acceptance testing framework for Android and iOS automation. Calabash provides a separate library for Android and iOS automation. It’s a cross-platform framework and supports cucumber which enables you to express the behaviour of your app in natural English language implementing BDD (Behavior Driven Development).

Calabash provides a bridge that allows Cucumber tests to run and validate on iOS and Android. Cucumber tests are written as a list of statements which forms number of test scenarios. All the statements in Cucumber are defined using Ruby.

In Calabash, a Cucumber statement must be defined only once but can be repeated for different scenarios of a Cucumber script.

The actual tests are written in Gherkin, backed with Ruby code, and run in the context of the Cucumber Framework.

Pros

1. Large Community Support.
2. Simple and expressive English-like test statements
3. Supports all actions on screen such as swipe, pinch, rotate, tap.
4. Large and enthusiastic community.
5. Cross-platform development support (same code works for Android as well as iOS devices)

Cons

1. After failure of test steps, all subsequent steps will be skipped. This can cause a more severe product issue to be missed out.
2. Takes time to run tests as it always installs app first by default. However this setting can be overridden by a hook that we can configure in code.
3. Requires Calabash framework to be instrumented inside ipa file for iOS.
4. We must have code of app in case of iOS.
5. Not friendly to languages other than Ruby.

2) Appium (Android and iOS)

What it is – Appium is an open-source test automation framework by Sauce Labs for use with native, hybrid, and mobile web apps. Appium library functions inside the framework make calls to the Appium server running in the background which operates the connected device. It uses JSONWireProtocol internally to interact with iOS and Android apps using Selenium’s WebDriver.

Unlike Calabash, which mainly supports development in Ruby, Appium can be used in framework with language of your choice from Java, Python, Ruby, and all other that Selenium WebDriver supports.

An Appium server gets hosted on a Node server. One can start an Appium server by firing a set of Node commands. If using the Appium Standalone Application as your server (downloaded from Appium website), the Inspector tool provides the ability to locate/identify/operate on all locators of your application.

Pros 

1. Supports multiple languages.
2. Does not require Source code access.
3. Cross platform script development.
4. Large community support.
5. Supports script recording on Mac
6. Can extract identifiers using Appium Server application’s ‘Inspector’ tool.
7. In built support for ‘Selendroid’ by Appium Server desktop application.
8. It also uses the vendor provided framework: UIAutomation for iOS, UIAutomator and Selendroid for android 4.2+ and 2.3+ respectively.
9. Supports physical device as well as Emulators.
10. Supports native, hybrid and mobile web application automation.

Cons

1. Frequent unstable releases for Appium Server desktop application.
2. Script recording not available for Application server desktop application for Windows OS.

3) Robotium (Android)

What it is – Robotium is an open-source test framework for developing functional, system, and acceptance test scenarios. It is very similar to Selenium but is only for Android. It’s listed under Apache License 2.0.

Because of simplicity and ability to create powerful and robust automation scenarios it’s gaining popularity in test automation community.

It uses run time binding to GUI components. It installs a test case suite as an application on an Android device or emulator and provides a real environment for execution of tests.

Pros 

1. Easy to write test scripts in minimal time.
2. Automation of pre-installed apps is possible.
3. Automatically follows current activity.
4. Faster and robust test execution as compared to Appium due to run time binding with GUI components.
5. Can work without access to code or knowing implementation of app.
6. Support for Activities, Dialogs, Toasts, Menus, Context Menus and other Android SDK controls.

Cons

1. Can’t handle flash or web components.
2. Supports development in Java.
3. Can be slow on older devices.
4. Does not support automation of iOS devices which can be deal breaker where test strategy involves automation of both android and iOS device automation.
5. No inbuilt support for record and playback. Paid tools available for recording such as TestDroid and Robotium Recorder.

4) Frank (iOS)

What it is – Frank is an iOS app automation framework that provides the ability to write tests scenarios in structured English sentences using Cucumber.

Frank requires it to be compiled inside application under test, meaning changes are mandatory to your source code. It’s a tool that uses a combination of Cucumber and JSON commands that are sent to a server running inside your native application, and leverages UISpec to run the commands.

Pros 

1. Test scenarios are written in understandable English sentences with help of cucumber framework.
2. Symbiote – Live inspection tool is included.
3. Proves efficient when team is having experience on web automation framework with selenium, cucumber.
4. Active community support.
5. Continuously expanding library.

Cons

1. Limited support for gestures.
2. Bit difficult to run tests on device.
3. Modification to configuration file is required to run on real devices.
4. Recording feature not available.

5) UIAutomator (Android)

What it is – UIAutomator is test framework by Google that provides advanced UI testing of native Android apps and games. It’s a Java library containing API to create functional UI tests and also an execution engine to run the tests. This library comes with the Android SDK. Many tutorials are available for beginners to get started. The advantage is it gives special privileges to JUnit test cases while running to access different processes. Even though it’s good and simpler for native apps automation, it has very limited or almost no support web view automation. It only supports devices with API level 16 and above, however that should not be a big factor or deal breaker as most apps these days support API level 19 and above.

Pros

1. Simple and easy to learn tutorials are available.
2. Library supported and maintained by google community.
3. Third party paid integration with cloud based test management available.

Cons 

1. Supports only android 4.1 and above.
2. Script recording is not supported.
3. Support is focused in for Java.
4. You can’t get current activity or Instrumentation.
5. ‘WebView’ is not currently supported and hence hybrid apps.
6. Library support is in Java only so it’s very hard if someone wants to blend it with cucumber which is in ruby.           However Java has its own BDD frameworks which can be used, not much in practice though.