Pros and Cons of Using XCTest for iOS Testing

The thing with frameworks (like XCTest framework) that integrate tightly with development environment is that those frameworks typically provide easy-to-use and seamless workflow with other components and features of the development environment. Also, there has been lots of changes in the iOS app test automation landscape lately due to Apple’s decision of deprecating UI Automation from Xcode.

Now, if you haven’t used XCTest before and you started out with it, there are few things worth of mentioning before diving in. Let’s start with a brief introduction.

A Brief Introduction to XCTest Framework

Like Android Studio, Xcode provides feature-rich software testing capabilities for developers that can significantly help enhancing the stability of the software. Furthermore, Xcode provides XCTest and XCUITest that are extremely helpful for achieving and building better quality software. Thoroughly tested apps, and regardless of whether they're done at unit-level or UI-level, improve the user experience and accelerate adoption of those sort of apps.

Frankly, XCTest is not a new framework but it has evolved quite well with Xcode releases. In fact, the XCTest framework was introduced with Xcode 5 a couple of years ago. What XCTest basically does is allow its users to do unit testing for Xcode projects (iOS apps among them) as it is currently considered as one of the top options for iOS app testing. Writing any tests with XCTest is a trivial task to iOS developers because XCTest is fully compatible with both Objective-C and Swift. In addition, all test classes developers create in Xcode project are basically subclasses of XCTestCase.

XCTest tests can be executed in a simulator or real physical device. If you use real devices locally, you just need to make sure all provisioning side of things (provisioning profile) is valid and set properly to test the target. The test methods used in XCTest are instance methods so no parameters are passed nor they return a value. For this, the name also begins with "test" and all added tests are visible in Test Navigator of your Xcode project.

Pros of the XCTest Framework

Many of today’s test automation frameworks are based on frameworks and layers that development tools provide with them. In case of iOS this is XCTest and with Android most of the frameworks are based on JUnit and Android instrumentation.

The advantages of XCTest over the other frameworks for iOS app testing are as follows.

Easy to Learn and No Additional Components Required

Xcode provides everything to get started with test automation with XCTest. Xcode’s XCTest is an integral part of the tool so testing is fairly easy to start and convenient to work with.

Native iOS Language Support

Basically, writing tests with the same programming language that your application is built with is not a requirement, but it gives some confidence for developers to create tests for their apps. The other way around, some developers may also think this differently and prefer other additional languages, frameworks, and tools to be used in test creation. Nevertheless, there is no learning curve or language barriers to get started with XCTest test creation.

Xcode Test Recorder

This is mostly a feature for UI tests, but as XCTest is closely related with XCUITest, the UI recording is possible with Xcode environment. The UI testing capabilities with Xcode include UI recording, generating code out of those recordings, run tests identically as those were intended while UI test was recorded. Record-and-playback testing tools have plenty of benefits and can get testers do the job even without understanding underlying software. In addition, recording tests again is possible or simply editing the generated piece of code with whatever changes have been done for the app itself. In short, record and playback tools provide a fast way to create tests, a decent level of accuracy, and is not too sensitive to user input errors.

Integrating With Continuous Integration

Integrating XCTest with continuous integration is also easy. Xcode allows XCTest tests to be executed using command-line scripts/shell and seamlessly integrated with Xcode’s continuous integration bots. However, there is a way to integrated XCTest scripts and the development environment with more widely used CI systems, such as Jenkins, but there are lots of things you need to know and sometimes – despite XCTest and Xcode have provided this capability for some time – tests end up failing just for no reason.

Faster Than On-Top-of-It Frameworks

Many other frameworks are relying on XCTest. Despite the fact that those frameworks provide more features and higher abstraction levels for testing and better capabilities, they still rely on the foundation of XCTest. This naturally makes XCTest faster as you don’t need to rely on abstraction APIs, albeit in many cases lightweight.

Cons of the XCTest Framework

XCTest isn’t perfect, but it provides some excellent basic functionality and capabilities to exercise tests on your iOS apps. As XCTest has improved by each version of Xcode, there are few cons you should consider.

No Cross-Platform Support

Apply typically builds everything to be available only on their own tools, devices, and environments. As this is understandable in the case of integral testing API and functionalities, many of testers prefer cross-platform test automation frameworks and have that one test script to work on both platforms, Android, and iOS. One great option, which is also an XCTest-based framework on iOS, is Appium. On Android, Appium utilizes Selendroid and uiautomator as its foundation and converts those commands written in Selenium Webdriver to native commands.

Limited Programming Language Support

As this is clearly one of iOS app development’s advantages, it is also one of its cons. Basically, app developers and tests created for those apps are limited to build with Objective-C or Swift. Again, abstraction frameworks on top of XCTest will expand the programming language selection and provide testers a freedom of choice with their tools and programming languages. 

Flaky

XCTest isn’t that new, but it still has some issues with basic usability and in the robustness of creating tests for iOS apps. Doing some the basic unit tests and running on those in the simulator may work just fine, but when you try to run the same tests on devices, you’ll see that there is some room for stability improvements with XCTest and Xcode in general.

Conclusion

XCTest is a valid choice for basic unit-level testing of iOS apps and functionality. However, you need to add a lot of those sort of basic tests to thoroughly cover the logic, and that makes functional UI testing more suitable for iOS apps.

What are your experiences with XCTest and do you feel it will be the number one choice for app developers? Weigh in with a comment below!