Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Controlling Siri and Asynchronous Testing With XCTest

DZone's Guide to

Controlling Siri and Asynchronous Testing With XCTest

The latest Xcode beta has some promising new tools for testing your iOS apps. From better error messaging to new results, your tests should be much smoother.

· Mobile Zone
Free Resource

Launching an app doesn’t need to be daunting. Whether you’re just getting started or need a refresher on mobile app testing best practices, this guide is your resource! Brought to you in partnership with Perfecto

Apple has released the new Xcode 8.3 beta 2 with the new Swift 3.1 snapshot, which is available to download from your Apple developer account. There are a lot of new additions and changes in Xcode 8.3 beta 2. You can read the release notes if you have an Apple developer account. There is a handy class added to the XCTest framework to enable interaction with Siri with XCUI Test, which is XCUISiriService. In this post, we'll look at how to enable interaction with Siri and asynchronous testing with XCTest

Xcode 8.3 Beta 2

Newly added classes XCUISiriService and XCTWaiter are available in Xcode 8.3 beta 2. You can get it from the Downloads section of the developer account. Xcode 8.3 needs macOS version 10.12 and above. You can download the compressed ZIP file, which is around 4.52 GB. If you already have a previous version of the Xcode, then remove it — or you can keep it, but you have to switch between Xcode DEVLOPER_DIR. Once downloaded, you can extract the file to install Xcode 8.3 beta and wait for the installation of Xcode and the command line tools. Once Xcode 8.3 beta 2 is fully installed with all the command line tools, we can drag it into the /Applications path. Now, we have to switch to the new Xcode version by running the following command:

$ sudo xcode-select --switch /Applications/Xcode-beta.app/


This will set new DEVELOPER_DIR, and we are ready to use Xcode 8.3. Make sure you are using the correct toolchain using the xcrun --find swift command, which will show current toolchain you are using.

$ xcrun --find swift

/Library/Developer/Toolchains/swift-3.1-DEVELOPMENT-SNAPSHOT-2017-01-22-a.xctoolchain/usr/bin/swift


Now, make sure you export the toolchain and are using the correct version of Swift, which is Apple Swift version 3.1-dev at the moment. You can easily do that by running the following commands.

$ export TOOLCHAINS=swift
$ swift --version 

Apple Swift version 3.1-dev (LLVM 40fb70e1b6, Clang 658ce8b57d, Swift d6c7fe1067)
Target: x86_64-apple-macosx10.9


This will ensure that you are using Swift 3.1. Now we are good to try new the features of the XCTest framework.

XCUISiriService

The XCTest framework now has a new class, XCUISiriService, which has an 'activate' method, which takes strings (voice recognition text) and passes them to Siri. This string is then processed by Siri and returns the result. The siriService is a part of the XCDevices class and can be used like so:

XCUIDevice.shared().siriService.activate(voiceRecognitionText: "Open News")


Let's cover how to use the XCUISiriService to interact with Siri by creating a sample application with a few tests.

Create an Xcode Project

Let's create a new Xcode project as an iOS app with unit and UI testing templates and name it XCUISiri. Now we should have a default iOS app with template UI test code.

Enable Siri From Device/Simulator

Now we have to enable Siri from a simulator or a real device. Let's enable it for a simulator by Selecting the 'Hardware' menu and selecting Siri. In the simulator, we can enable Siri from the settings menu as shown below.

Run Tests

Now we are all set to try XCUISiriService. Let's add one more test to use XCUiSiriService.

func testXCUISiriOpenExistingApp() {

        XCUIDevice.shared().siriService.activate(voiceRecognitionText: "Open News")

}


We can easily guess that the code above is telling Siri to open Apple's 'News' application. Let's run the test by pressing CMD+U.

You can see that the tests have opened Siri and passed the 'Open News' string to it. Siri then acts on it and opens the Apple News app. In this way, we can literally pass any string to Siri and let Siri act on it.

Potential Benefits of XCUISiriService

There might be potential benefits we can get out of this XCUISiriService class in XCUI tests. Here are some possibilities

  • We can use Siri to open an app (potentially).
  • We have the ability to automate everything that Siri can do.

XCTWaiter 

Current Waiting in XCUI Test

The XCTest framework allows developers to write unit and UI tests for iOS and macOS apps. Apple introduced Xcode UI testing in the WWDC 2015, which allows us to write UI tests within the Xcode. As part of the Xcode 8.3 release, Apple has added a couple of new classes to the XCTest framework to support asynchronous testing. It means there are no handlers involved while waiting for the XCUIElement to appear. Previously, we had the waitForExpectations(timeout: handler:) method combined with XCTestExpectation to test asynchronous code, which looks like this:

let predicate = NSPredicate(format: "exists == 1")
let query = XCUIApplication().buttons["Button"]
expectationForPredicate(predicate, evaluatedWithObject: query, handler: nil)
waitForExpectationsWithTimeout(3, handler: nil)


This piece of code will wait for three seconds to find the button, and it will fail after three seconds if it doesn't find said element. We have passed nil to the handler, which invokes error once the timeout is reached. This causes the test to fail, as well as an error to be thrown. It's very generic and not often useful while debugging. Luckily, we now have better control over errors and handlers with XCWaiter.

XCTestWaiter

The XCTest framework now has XCTWaiter class to wait for the element. The list of the newly added classes and sub-classes are as follows:

XCWaiter class returns results of the expectations in the form of boolean. It returns an enum of four possible situations — .completed, .timedOut, .incorrectOrder, or .invertedFulFilment. We can simply add an extension to XCUIElement or add a simple function that returns the result of the waiter function, which returns one of the results from XCWaiter.

func waiterResultWithExpextation(_ element: XCUIElement) -> XCTWaiterResult {
        let myPredicate = NSPredicate(format: "exists == true")
        let myExpectation = expectation(for: myPredicate, evaluatedWith: element,
                                      handler: nil)

        let result = XCTWaiter().wait(for: [myExpectation], timeout: 5)
        return result
}


There is no callback block or completion handler. The helper method simply returns a boolean indicating if the element appeared. Let's briefly go through each of the boolean results returned by XCTWaiterResult to understand what it is doing.

Completed

This returns true when defined expectations are fulfilled or completed. It returns false when expectations are not met within specified timeout. Tests will likely to fail when expectations are not met.

TimedOut

This will return true when expectations are not met within the timeout period. We can define how to fail our test case by providing useful error messages for debugging.

Incorrect Order

This tells us the expectations that succeeded and which are still waiting to succeed. We know what is taking time to diagnose the problem.

InvertedFulfilment

Not entirely sure how it works, but as per docs, if an expectation is set to have inverted behavior, then fulfilling it will have a similar effect of failing to fulfill a conventional expectation.

Readable Expectations

There are some new expectations added to the XCTest framework, which are XCTNSPredicateExpectationXCTKVOExpectation, and XCTDarwinNotificationExpectation. I think the idea behind that is to make expectations more readable and customizable. Previously, we have to write expectations with handlers. Now we can write expectation like this:

let myExpectation = XCTKVOExpectation(keyPath: "exists", object: element,
                                              expectedValue: true)


We can then pass this expectation to the XCWaiter to get a boolean result. You can find some examples in the demo repo here.

The Benefits of XCWaiter

Better Waiting Strategy

XCWaiter gives us boolean results from each expectation, which allows us to define our XCTest with better control. This might help to deal with the flakiness of tests. XCWaiter results simply return whether an element appeared or not.

Ability to Define Multiple Expectation

We can define more than one expectation and wait for XCWaiter to return results for each of them. As mentioned above, expectations can be written in a more readable way.

Handling Timeouts and Errors

Previously, XCtest used to fail if expectations were not fulfilled within the specified timeout, and the error was very generic. Now we have the .timedOut  result from XCWaiter so that we can control test failures with proper error messages.

Reduce Flakiness

XCWaiter gives us more control over how we want to define test failures. This will reduce the flakiness of our tests. The XCWaiter results can also give us the ability to fail or pass tests with proper error messages.

Highlight Performance Issues

XCWaiter gives us the ability to define multiple expectations, and each expectation can attempt to be fulfilled within a timeout period. The .incorrectOrder result  tells us how many expectations are fulfilled and how many still waiting to be fulfilled. This gives us an indication why those expectations are slow and why and where we might have some performance issues. We can diagnose the slowness to make the test and application faster

GitHub Repo XCWaiter Demo

I have created a GitHub repo, Xcode83_Demo, with a sample iOS application to give the XCWaiter a try and experiment with XCWaiter features. Feel free to clone it and give it a go.

Github Source Code for XCUISiriService

  • You can get the repo from GitHub: XCUISiriServiceDemo. Try it yourself, make sure you have Xcode 8.3 beta and latest Swift toolchain.
$ git clone https://github.com/Shashikant86/XCUISiriServiceDemo/
$ cd XCUISiriServiceDemo
$ open XCUISiri.xcodeproj


Now, if you run tests using CMD+U then you should see tests running.

Conclusion

The new class XCUISiriService in XCTest exposed the ability to automate AI-related tasks with XCUI Tests. Also, XCTWaiter makes asynchronous testing possible with XCTest. Hopefully, we will get some more insight once the stable version of Xcode 8.3 is released.

Keep up with the latest DevTest Jargon with the latest Mobile DevTest Dictionary. Brought to you in partnership with Perfecto.

Topics:
xcode ,swift 3.0 ,ios development ,mobile ,tutorial

Opinions expressed by DZone contributors are their own.

THE DZONE NEWSLETTER

Dev Resources & Solutions Straight to Your Inbox

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

X

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}