Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

New XCUITest Features With Xcode 9: Hands-on Exploration

DZone's Guide to

New XCUITest Features With Xcode 9: Hands-on Exploration

The XCTest framework now offers new features like parallel and multi-app UI testing for Apple platforms, including iOS.

· Mobile Zone
Free Resource

Download this comprehensive Mobile Testing Reference Guide to help prioritize which mobile devices and OSs to test against, brought to you in partnership with Sauce Labs.

At WWDC 2017, there was a great session on What's New in Testing which was mainly about new features of XCTest and XCUITest frameworks.  The team working on developer tools at Apple has made huge improvements in the area of  UI Testing with XCUITest and Continuous Integration with Xcode Server. In this post, we will explore all the new features with practical examples in Xcode 9 and command line. There is Github repo  Xcode9-XCTest created as part of this exploration.  You can reference this post to that GitHub repository to try out the things by your own.

There were a lot of new things announced related to testing for Apple platforms, especially iOS and macOS. Some of them are

  • XCUISiriService
  • Localization Testing
  • Async Testing
  • UI Test Performance Improvement
  • Activities, Attachments, and Screenshots
  • Multi-App Testing
  • Headless Testing and Parallel Testing in xcodebuild
  • Inbuilt Xcode Server

There are many enhancements in testing processes including Siri integration, waiting in XCTest, Core Simulators in xcodebuild, and much more. Let's dive into these, one by one.

XCUISiriService

Using XCUISiriService, we can now interact with the Siri interface. We can control Siri by passing voice recognition text from our XCUITests and Siri will act accordingly. Let's imagine we want to open an Apple News app using XCUISiriService- we can do that from our test.

func testXCUISiriService() {
        XCUIDevice().siriService.activate(voiceRecognitionText: "Open News")
 }

I wrote a detailed blog about controlling Siri from XCTest using XCUISiriService a few months ago when it was announced with Xcode 8.3, and also created a demo project on Github called XCUISiriServiceDemo. API syntax has changed a bit but these examples are still valid.

A Practical Example

Open the Xcode9-XCTest project in Xcode 9 and run the testXCUISiriService() test from the Xcode9_XCTestUITests.swift file. You can see that Siri will open the Apple News app.

Localization

With Xcode 9, we can use the Xcode scheme to run tests in different languages and regions. When we edit the scheme, we see the test options to run tests using specific languages and regions. We can easily test our app with different languages and regions by changing the scheme settings.

Async Testing

There are many situations where we need to wait until certain things to happen to carry on our test execution, like opening a document, waiting for a response from the server, et cetera, but it's most common in UI testing scenarios. Until now, XCTest handled async testing by creating expectations and the test will wait until the expectations are fulfilled. The XCTest timeout will result in test failure. The expectations are tightly coupled with XCTestCase.

Now, XCTest has a new class, XCTWaiter, which allows us to define expectations explicitly and is decoupled from XCTestCase. It has an initializer as a public API and different waiting conditions covered. XCTWaiter waits for a group of expectations to be fulfilled. We can now define the expectations like this:

wait(for: [documentExpectation], timeout: 10)
 // OR
XCTWaiter(delegate.self).wait(for: [documentExpectation], timeout: 10)
// OR
let result = XCTWaiter.wait(for: [documentExpectation], timeout: 10)
if result ==.timeout {
   ...
}

The list of the newly added classes and sub-classes  are as follows:

There is one more handy API that we can use to wait for the existence of XCUIElement.

XCUIElement.waitForExistance()

A Practical Example

Open the Xcode9-XCTest project in Xcode 9 and run the testXCWaiterfeatures() test from Xcode9_XCTestUITests.swift file. I have already shared a detailed article a few months ago on Asynchronous iOS Testing in Swift with XCWaiter; there is also a demo project on GitHub here.

UI Testing Performance Improvements

Removal of Snapshot

As we are aware, XCUIApplication is a proxy app that interacts with the main app used to launch, terminate, and query the main application. Apple was using the snapshot tool for communication between the XCUI app and main app, which was causing performance issues in terms of time and memory. Now, snapshot has been removed and replaced with the remote query and query analysis technique to improve the performance of the XCUItests. We should have faster execution of the tests.

First Match API

Apple also introduced First Match API to speed up the response to queries. The First Match API will exit early as soon as it finds the first XCUIElement, rather than querying everything. We can use firstMatch on any XCUIElement

let button = app.navigationBars.buttons["Done"].firstMatch

Now the query will have a faster magnitude and no memory spike.

Match First Vs Match All
app.buttons.firstMatch //Not Good idea

app.buttons["Done"].firstMatch // Better

app.navigationsBars.buttons["Done"].firstMatch //best

Activities, Attachments, and Screenshots

There are big improvements in the area of organizing tests for readable reports, taking screenshots of XCUIElements, and attaching rich data to test reports. There were three different concepts introduced this year:

  • Activity: Grouping of actions by giving meaningful names.
  • Screenshot: Taking screenshots of fullscreen or a specific XCUIElement.
  • Attachments: Attaching rich data to XCTest reports.

Activities

UI tests are usually long-running with a lot of actions happening, like tapping buttons and swiping. Until now, XCTest reports showed all the actions in the test reports, which were not particularly readable. Activities is the way to organize those actions into groups by giving them meaningful names, so XCTest results will use that activity name in the result to make it more readable. You can read more about activities on Apple's official documentation.

We can sprinkle activities on any set of actions, like launching the app in a clean state.

XCTContext.runActivity(named: "Given I have launched app in clean state") { _ in
   XCUIApplication().launch()
   XCUIApplication().launchArguments = ["-StartFromCleanState", "YES"]
}

Now that we have defined an activity to launch the app in a clean state, when we run the test, we will see "Given I have launched app in clean state" in the report, which is more readable. We can still access underlying actions by expanding the activity.

ScreenShots

Apple also announced the new API to take a screenshot of the full screen as well as specific XCUIElements. There is a new protocol, XCUIScreenshotProviding, to provide a screenshot of the current UI state, and there are two new classes, XCUIScreen and XCUIScreenshot, to capture screenshots of the app or UIElement state. We can take a screenshot using following code snippet:

let screen = XCUIScreen.main
let fullscreenshot = screen.screenshot()

Attachments

An Attachment can be used to attach rich data to test reports. It may be data from a test, improved triage additional logs, or post processing workflow. The data may be in the form of raw binary data, string, property list, code object, files, or images. We can attach the screenshots to activities using an attachment. We can add an attachment to an activity like this:

XCTContext.runActivity(named: "When I add attachment to activity") { (activity) in
            let screen = XCUIScreen.main
            let fullscreenshot = screen.screenshot()
            let fullScreenshotAttachment = XCTAttachment(screenshot: fullscreenshot)
            fullScreenshotAttachment.lifetime = .keepAlways
            activity.add(fullScreenshotAttachment)
}

A Practical Example

Open the Xcode9-XCTest project in Xcode 9 and run the testActivitiesScreenShotsAttachments() test from the Xcode9_XCTestUITests.swift file. You can see the test reports in Xcode are taking an activity name rather than actions. In another test, we have attached screenshots to the activity.

Multi-App Testing

Previously, we could test only one application, the target application for the XCUITest target. There was no way to interact with other apps like Settings or News from XCUITest. XCUIApplication() is used to launch, terminate, and query an application under test. It can only access the target application under test and doesn't test other applications. There was always a need to access other applications, like Settings or app extensions.

With Xcode 9, we can test multiple application using XCUIApplication. It has three main changes; now XCUIApplication()

  •  has an initializer, which takes bundleIdentifier so that we can pass the bundleID of our app,
  •  has a new activate() method to activate an app from the background,
  •  has States to monitor changes in the application. States has cases like runningBackground, runningForeground, etc.

We can test two applications in one XCTest using below code:

func testTwoApps() {
  let app1 = XCUIApplication(bundleIdentifier: "com.app1.xyz ")
  let app2 = XCUIApplication(bundleIdentifier: "com.app2.xyz")
  // Launch App1
    app1.launch()
  // Launch and test App 2
   app2.launch()
   app2.launchArguments = ["-StartFromCleanState", "YES"]
// test App1 again by activating it from back ground
   app1.activate()
}

The activate() method automatically waits for another app to become active, but there might be situations where we want to manually wait for the app to be activated; this can be done by using predicates and XCUIApplication states.

let app1ActiveExpectation = XCTNSPredicateExpectation(predicate: NSPredicate(format: "state == \(XCUIApplication.State.runingForeground.rawValue"), object: app1)

wait(for: [app1ActiveExpectation], timeout: 10)

As we can see, we can interact with any app within the simulator or device as long as we know the bundle identifier. This is a huge improvement in the UI testing of iOS apps.

A Practical Example

Open the Xcode9-XCTest project in Xcode 9 and run the testMultipleApps()   test from the Xcode9_XCTestUITests.swift file. We are interacting with the main application and settings app.

Note that currently, the test is failing while launching settings, but the simulator is launching the settings app regardless.

xcodebuild: Headless Testing

There are some huge improvements in the xcodebuild command line tool, which is used for analyzing, building, testing, and archiving an iOS application. xcodebuild will use the core simulator to run tests so we don't see simulators running when running our tests from the command line. This means we won't see simulators on CI servers. It will be totally headless.

A Practical Example

Clone or download  Xcode9-XCTest project and run xcodebuild command. One thing to note that , there is no simulator launching while tests are running.

$ git clone https://github.com/Shashikant86/Xcode9-XCTest
$ cd Xcode9-XCTest
$ xcodebuild -scheme "Xcode9-XCTest" -destination 'platform=iOS Simulator,name=iPhone 7 Plus,OS=11.0' build test CODE_SIGN_IDENTITY="" CODE_SIGNING_REQUIRED=NO

xcodebuild: Parallel Testing

xcodebuild support parallel device testing. It means we can run test in multiple simulator and devices as long as devices are provisioned also we can run test on devices which are wirelessly connected to server. We just need to pass multiple destinations to xcodebuild.

A Practical Example

This would be the same as the above clone, or you could download the Xcode9-XCTest project and run the xcodebuild command by passing additional destinations options.

$ xcodebuild -scheme "Xcode9-XCTest" -destination 'platform=iOS Simulator,name=iPhone 7 Plus,OS=11.0'-destination 'platform=iOS Simulator,name=iPhone 7 Plus,OS=10.3' build test CODE_SIGN_IDENTITY="" CODE_SIGNING_REQUIRED=NO

Inbuilt Xcode Server

Xcode Server is a continuous integration system provided by Apple. Until now, Xcode Server needed a macOS Server application to run Xcode bots. Now, Xcode Server no longer needs a macOS Server application. It's inbuilt in Xcode and can be accessed from Xcode Preference. Xcode Server has improved provisioning, uses a core simulator (headless) for running tests, parallel testing on multiple devices, and localization control.

A Practical Example

I have shared a detailed article about how to setup Xcode Server using Xcode 9, called Xcode 9 + Xcode Server = Comprehensive iOS Continuous Integration. Feel free to navigate to that article to learn more about Xcode Server setup from scratch.

Watch below how to create an Xcode bot for our demo project, Xcode9-XCTest.

New features in XCTest framework like parallel testing, automatic provisioning, and multi-app testing will definitely be useful for all the iOS teams across the world. Apple is investing time in improving testing and continuous integration practices. I hope it will continue in upcoming years.

Analysts agree that a mix of emulators/simulators and real devices are necessary to optimize your mobile app testing - learn more in this white paper, brought to you in partnership with Sauce Labs.

Topics:
xcode ,wwdc 2017 ,xctest ,swift ,mobile testing ,test automation ,ui testing

Published at DZone with permission of Shashikant Jagtap. See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}