Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Top 5 iOS Testing Frameworks

DZone's Guide to

Top 5 iOS Testing Frameworks

Read on for a comparison of a few of the leading iOS mobile app testing framework options by their features and test coverage.

· Mobile Zone
Free Resource

Download this comprehensive Mobile Testing Reference Guide to help prioritize which mobile devices and OSs to test against, brought to you in partnership with Sauce Labs.

As the market matures in the mobile app space, iOS application developers have a multitude of iOS testing frameworks to leverage. Let's go over some of the most popular frameworks (in alphabetical order), their strengths, features, test coverage options, and how they are licensed.

Appium

From appium.io:

"Appium is an open-source tool for automating native, mobile web, and hybrid applications on iOS and Android platforms. Native apps are those written using the iOS, Android, or Windows SDKs. Mobile web apps are web apps accessed using a mobile browser (Appium supports Safari on iOS and Chrome, or the built-in 'Browser' app on Android). Hybrid apps have a wrapper around a 'webview' -a native control that enables interaction with web content."

Sauce Labs is very active in the development of Appium. Jonathan Lipps, Director of Open Source at Sauce, is the Appium project lead, and a number of Sauce Labs developers frequently contribute to the project. Your Appium tests can run across emulators, simulators and real devices on the Sauce Labs platform.

Strengths
  • You can choose your own underpinning framework, like XCTest, for unit testing.
  • Cross-platform support allows for reuse of test scenarios across mobile and web channels.
  • You can write tests with your favorite tools using any WebDriver-compatible language such as Java, Objective-C, and JavaScript.
  • Full framework is open source.
Sample Code for WebDriver:
driver.findElement(By.id("com.example.app:id/radio0")).click();
driver.findElement(By.id("com.example.app:id/radio1")).click();
driver.findElement(By.id("com.example.app:id/radio2")).click();
driver.findElement(By.id("com.example.app:id/editText1")).click();
driver.findElement(By.id("com.example.app:id/editText1")).sendKeys("Simple Test");
driver.findElement(By.name("Answer")).click();

Apple Xcode Test

From developer.apple.com:

"Use the XCTest framework to write unit tests for your Xcode projects that integrate seamlessly with Xcode's testing workflow. Tests assert that certain conditions are satisfied during code execution, and record test failures (with optional messages) if those conditions are not satisfied. Tests can also measure the performance of blocks of code to check for performance regressions, and can interact with an application's UI to validate user interaction flows."

Strengths"
  • Tooling is incredibly easy to embed in the application.
  • Provides unit, UI, and performance testing.
  • Uses Objective-C and Swift so developers can use their existing skill set.
  • Can run tests automatically as a "bot" in the CI/CD process on Xcode Server.
Sample Code:
- (void) testAdditionPerformance { [self measureBlock: ^{
		// set the initial state
		[calcViewController press: [calcView viewWithTag: 6]]; // 6
		// iterate for 100000 cycles of adding 2
		for (int i = 0; i < 100000; i++) { [calcViewController press: [calcView viewWithTag: 13]]; // +
			[calcViewController press: [calcView viewWithTag: 2]]; // 2
			[calcViewController press: [calcView viewWithTag: 12]]; // =
		}
	}];
}

Calabash

From calaba.sh:

"Calabash enables you to write and execute automated acceptance tests of mobile apps. It's cross-platform, supporting Android and iOS-native apps. It's also open-sourced and free, developed and maintained by Xamarin."

(Editor's Note: Xamarin announced that they have discontinued active development of Calabash as of April, 2017).

Strengths:
  • Uses Cucumber to describe the test scenarios using BDD. This is a great feature for QA teams because it makes information clear and easy to digest.
  • Cross-platform support allows for reuse of test scenarios across mobile and web channels.
Sample Cucumber Feature:
Feature: Refund item

Scenario: Customer returns a faulty microwave
Given Customer has bought a microwave for $100
And she has a receipt
When she returns the microwave
Then Customer should be refunded $100

EarlGrey

From the Google Open Source Blog:

"Brewing for quite some time, we are excited to announce EarlGrey, a functional UI iOS testing framework. Several Google apps like YouTube, Google Calendar, Google Photos, Google Translate, Google Play Music and many more have successfully adopted the framework for their functional testing needs."

Strengths:
  • Easy to include in an iOS project, either directly or using CocoaPods.
  • A flexible framework, with powerful synchronization features across internal components.
  • Full framework is open source.
Sample Code:
// Objective-C
- (void) testInvokeCustomSelectorOnElement { [[EarlGrey selectElementWithMatcher: grey_accessibilityID(@"id_of_element")]
	performAction: [GREYActionBlock actionWithName: @"Invoke clearStateForTest selector"
	performBlock: ^(id element, NSError * __strong * errorOrNil) { [element doSomething];
		return YES; // Return YES for success, NO for failure.
	}]];
}

OCMock

Homepage: ocmock.org

OCMock provides a framework to create stub objects in your iOS app. It comes in two options- a static library that is used for iOS development, and a framework used for OS X development.

Strengths:
  • One of the easiest ways to add mock objects to existing unit tests.
  • Uses Objective-C so developers can use their existing skill set.
  • Full framework is open source.
Sample Code- Adding to an XCTest:
- (void) testExample {
	XCTAssertEqual(0, [self.calculator calculateTax]);
	id mockSubscriptionManager = OCMClassMock([SubscriptionManager class]);
}

Conclusion: Which iOS Testing Framework Should I Use?

Based on your needs and experience with other languages, the iOS testing framework to select may be obvious. Or you may need to try all the frameworks listed to see which one works best for you.

Either way, at this point, the market is mature enough that there is a framework out there that fits the needs of almost any developer, and serves a variety of tasks-whether you simply need to automate testing service calls in iOS, or want to run UI tests in parallel across multiple mobile and web platforms.

Analysts agree that a mix of emulators/simulators and real devices are necessary to optimize your mobile app testing - learn more in this white paper, brought to you in partnership with Sauce Labs.

Topics:
mobile ,mobile testing ,ios ,appium ,xcode ,calabash

Published at DZone with permission of Vince Power. See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}