WWDC '18: What's New in Code Coverage, XCTest, and XCUITest

DZone 's Guide to

WWDC '18: What's New in Code Coverage, XCTest, and XCUITest

Learn what's new in testing in Xcode 10, like code coverage reports and parallel testing, and some tips for better testing.

· DevOps Zone ·
Free Resource

At WWDC18, there was a session on What's New in Testing describing new features in code coverage, XCTest, and XCUITests. It was presented by the Honza and Ethan. This session consisted of

  • xccov for code coverage
  • Test Selection from Scheme
  • Randomizing Tests
  • Parallelizing Tests
  • Tips for Parallelizing Tests

Fortunately, I have already covered most of the things presented at this talk in my previous blog posts.

  • As xccov was launched with Xcode 9.3, I have written a detailed blog post covering all the features of xccov on my personal blog as well on Medium.
  • Parallel Testing features of Xcode 10 have been announced in Xcode 10, and I already covered most of the parallel testing in an Xcode 10 in action blog post, which is also published on Medium.

You can always watch the live demo in the session anytime. However, we will cover the features mentioned at the session briefly.

Apple has released new command line tool xccov with Xcode 9.3 for inspecting the contents of Xcode code coverage reports. We can explicitly enable code coverage for the scheme by editing the scheme and ticking the box "Code Coverage" from the "Test" action.

Once the tests run with code coverage data, Xcode will generate code coverage reports into the default derived data directory located at ~/Library/Developer/Xcode/DerivedData and you will see the code coverage reports generated at Logs/Test directory. We will see code coverage with the .xccovreport and .xccovarchive extensions. Inside the Logs/Test directory, there is a coverage report with the extension .xccovreport, and the coverage archive, with the extension .xccovarchive respectively. As per the main page of this utility, "The coverage report contains line coverage percentages for each target, source file, and function/method that has coverage information. The coverage archive contains the raw execution counts for each file in the report." We can achieve the following things with xccov at the moment:

  • View the Code Coverage Reports from the terminal.
  • Spit out the JSON from the code coverage reports.
  • List all the files for which code coverage has been generated.
  • Look at the code coverage report for one specific file.

Watch how to view code coverage in action:

This reduces the pain of using third-party tools to display the Xcode code coverage reports in a nice format. We can also skip the code coverage for the targets we don't care about.

With Xcode 10, we can select the specific tests from schemes, randomize the tests, and execute the tests in parallel. We can enable parallel testing by updating the scheme, and in the Test action, we can select "Options" against the test bundle to choose the parallelization option. We can select the location, as well.

The tests are extended to parallelizing test suites within a single simulator by creating clones of the simulators. Xcode creates different runner processes under the hood and each process gets specific tests assigned.

It's possible to run the XCTest from the command line using the xcodebuild tool. With Xcode 10, we have a few more options to enable parallel testing:

  • -maximum-concurrent-test-device-destinations NUMBER : the maximum number of device destinations to test on concurrently
  • -maximum-concurrent-test-simulator-destinations NUMBER : the maximum number of simulator destinations to test on concurrently
  • -parallel-testing-enabled YES|NO : overrides the per-target setting in the scheme
  • -parallel-testing-worker-count NUMBER : the exact number of test runners that will be spawned during parallel testing
  • -maximum-parallel-testing-workers NUMBER : the maximum number of test runners that will be spawned during parallel testing

These options allow us to execute the XCTests in parallel as we have done it from Xcode 10.

At the end of the session, there are three awesome tips shared:

  • Keep the test class small; if the test class gets big with too many tests, then split it into two or more classes. This will avoid the situation of one class getting dominated in parallel testing.
  • Keep the performance tests in a separate bundle and never run them in parallel.
  • Find the right tests for the parallelization.

Some awesome improvements happened this year in the parallel testing of both XCTest and XCUITests. One more session is going to happen on Testing Tips and Tricks where we will get more insight on how to perform testing better. I will publish another post about that. Stay tuned.

devops ,mobile ,xctest ,xcuitest ,wwdc

Published at DZone with permission of Shashikant Jagtap , DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}