XCUITest - The Emerging iOS UI Test Automation Framework

Over the last year, there has been a growing trend of iOS development teams adopting XCUITest and additional frameworks built on top of the XCTest interface.

Development teams have started to adopt XCUITest to get fast and reliable feedback. There are a few clear drivers to this growing adoption:

1. Intuitive – Using XCUITest is quite intuitive for developers as it runs from within the XCode IDE.
2. Fast – Test execution against iOS devices is faster than any other UI test automation tool due to the framework architecture.
3. Reliable – Due to the architecture of the framework, test execution using XCUITest generates more reliable results and eliminates flakiness.
4. Mature – The APIs and the framework became significantly more mature during the last year.
5. Test maintenance – Since the app has been instrumented, the framework works directly at the object level which reduces maintenance efforts that usually happen due to changes in the applications.

An iOS Instrumented Testing Frameworks and Terminology Review

• XCTest – Apple’s official framework for writing unit tests for classes and components at any level. These tests, like the app itself, can be written in Swift/Objective C.
• XCUITest – A UI testing framework that is built on top of XCTest. It includes additional classes (such as UIAccessibility). These tests can be written in Swift or Objective C. The tests are packaged in a test ipa (iOS packaged application) runner that executes the tests on the AUT(application under test) ipa.
• KIF (Keep It Functional) – A iOS native app that wraps XCTest as well using undocumented iOS APIs. It requires the developer to add the KIF framework to the project. It has a simple and intuitive syntax. [tester enterText:@”[email protected]” intoViewWithAccessibilityLabel:@”Login User Name”];
  [tester enterText:@”thisismypassword” intoViewWithAccessibilityLabel:@”Login Password”];
• Earlgrey – Similar to KIF although developed by Google. EarlGrey has an advanced a synchronization mechanism which means you don’t need explicit waits/sleeps. (For example, if tapping a button triggers a network request, EarlGrey will wait for the network request to finish before proceeding with the test). EarlGrey uses matchers extensively (read selection API here), these give you the flexibility to interact with elements and write assertion logic in a variety of ways with simple APIs.
• Cucumberish – Test automation framework for Behavior Driven Development (BDD) that integrates into XCode and uses the iOS interfaces XCTest/XCUITest.

The Challenge

The challenge we find is that although the above test frameworks can solve significant challenges where other test automation frameworks cannot, in many cases, teams adopt these frameworks before considering the proper setup and infrastructure. The promise of XCUITest depends on the fact that the execution of the tests will be on a robust, reliable, and scalable lab infrastructure.

Even though the adoption of those automation frameworks grow, many teams still execute their tests on simulators/local devices from the developers' workstation. Those teams understand that they get significantly more value from executing XCUITests and therefore continue to consider leveraging them even more by executing them as part of the CI processes in order to provide continuous feedback on real devices and end-user conditions.

XCUITest Advanced Capabilities

Perfecto recently released advanced support for the above frameworks in order to enable development teams to leverage the advantages mentioned above, while leveraging Perfecto’s cloud-based capabilities. In addition, Perfecto extended the XCUITest framework by adding the ability to control and set up the device the same way end users do, by which enabling teams to validate their apps will function as expected in the real world.