Continuous Integration for iOS and macOS

This is an article from DZone's 2023 Development at Scale Trend Report.

For more:

Read the Report

The no-code approach to continuous integration (CI) on mobile projects works reasonably well when teams start with one or two developers, a small project, and a cloud service. Over time, as a team grows and a project becomes more complex, it is natural to transition to self-hosted runners for faster feedback, more reliable tests, and code in production. This is the low-code approach to automation, as it has evolved. 

Figure 1: CI - Mac Studio with iPhone used as a self-hosted runner

Many Xcode projects can use very similar workflows tailored to current project needs. Since the Apple ecosystem constantly evolves, this approach allows developers to stay current in our tools and methods. For example, a recent switch by Swift Package Manager required a couple updates to scripts that update dependencies. The goal of this article is to share common low-code workflows for Xcode projects and strategies for automation.

Low-Code Automation Strategies

Automation workflows depend on the project strategies that development teams adopt. Things that should be considered include whether to write tests, the type of tests, if CI should be included, and so forth. These considerations are team decisions. Let's review some examples of strategies that work for multiple projects. 

Continuous Integration: 10-Minute Limit

Limiting CI execution time to 10 minutes provides quick feedback to developers. When tests take longer, it's a sign to refactor tests or move some to the suite of nightly tests. iOS Simulator is a great way to run tests fast as it can execute multiple tests in parallel. 

Unit Tests

It is a good goal for unit test suites to be lightning fast, have no network access, and take advantage of random order of execution. Code often has to be extracted into a framework or Swift Package so that launching unit tests does not launch the mobile app, nor does it require linking the app. All dependencies are abstracted away. The biggest value is to enable code refactoring with confidence and see how code works, not just how it reads. 

Application Tests

Application tests build and link whole mobile apps. It's okay to access a network and load dependencies because application tests verify integration between components, unlike unit tests, which only test a method or a function. There are few strategies for testing integration with the back end of the app: either the separate QA server, mock server, or server deployed on localhost. 

Application tests could be used with any of those methods; the goal is to execute code that “talks” to the back end without performing UI interactions. 

UI Tests

Simulate a user interacting with a mobile app. This kind of test catches most bugs, but it is the most fragile and could be flaky. To provide fast feedback to developers, we only run the main happy path of the app and not every feature. Avoid testing every failure path — just test one or two to verify how errors are presented to users. 

Device Tests

Some projects might need to run tests on devices, not only on iOS Simulator. This is especially true when, let’s say, an iOS app is using features not available on Simulator, such as push notifications, cameras, and cellular networks. Since device tests are more expensive to run, it is prudent to limit device tests to only those tests that Simulator could not run. 

Here is where self-hosted runners shine as they can have iOS connected via USB cable and launch tests on that device, as shown in Figure 1 above. 

Continuous Delivery: Manual Deploy Button

Unlike web-based projects, mobile projects do not deploy every commit or change. The usual cadence for mobile releases to TestFlight is about one to two times a week, or even once every two weeks. In each release, developers offer testers a small list of features to test and give feedback on. 

And once a month — or may be once every two months — the team promotes that TestFlight build from an internal to an external group of testers. External groups can include some users who volunteered to give early feedback. 

Nightly Tests

At night, development teams can execute long-running tests since runners are not in use by continuous integration and other jobs. 

Nightly UI Tests

The goal of nightly UI tests is to test every feature of the app and exercise error conditions and less frequently used scenarios. This requires the team's discipline to attend to broken tests or failures from the previous night's run. If there is an opportunity to refactor nightly tests to run faster, those can be moved to the regular UI test suite. 

Sometimes a team cannot immediately figure out a fix for broken UI tests, and in that case, they can be marked as a bug, reported as a GitHub issue, and marked as an "expected-to-fail" test. 

Reliability Tests

Run the same UI tests (and application tests) multiple times to expose intermittent failures of the test, infrastructure, and code. Legacy code that was not test driven often behaves unreliably; reliability tests allow you to see stats about behavior. 

Other Automation

Other routine tasks, which can be automated by GitHub Actions workflows, include periodic dependency updates or opening a secure tunnel to debug failures. 

Dependencies Update

Updating workflows creates proposed changes to the code and submits pull requests (PRs), which go through the usual CI process. A couple of the most common updates that teams have to do periodically are: 

• Third-party – Swift Packages, Ruby Bundles, Node dependencies, and GitHub Actions are all updated once a week, and on Mondays, teams can review a green PR given it passed all tests and is ready for merge. See this GitHub project for an example.
• Back-end API – Mobile development teams can have a workflow that kicks off an update for the API to pick up the latest changes from GraphQL made by the back-end team. For projects that use GraphQL client code generation, the job outcome is a PR that uses updated back-end code. This workflow can be triggered manually to update client code when a server team deployed changes to QA. 

Screen Share to CI Machine

Sometimes developers need to access a CI machine to debug and investigate failures. Screen share workflow opens a one-time tunnel into a CI machine, giving the developer screen share and full access to the UI. This is one of the advantages of self-hosted runners: Developers can inspect what happened, unlike cloud runners that will destroy the state of the failure when the job finishes. Take a look at the screen-share.yml workflow in this example project. 

Workflows and Pipelines

Let's review some examples of GitHub Action workflows and pipelines. 

Continuous Integration

CI can start from a simple workflow of building projects and executing tests. 

Figure 2: CI badge in README with links

Here is a working example of a workflow that builds a sample app and runs tests for an Xcode project: 

name: Simulator Tests

on:
  pull_request:
    branches:
      - main
  push:
    branches:
      - "**"
      - "!experiment/**"

defaults:
  run:
    shell: zsh –login –errexit –pipefail {0}

jobs:
  build-and-test:
    runs-on: self-hosted
 
    steps:
    - uses: actions/checkout@v3
    - name: Build
      run: > 
        xcodebuild -scheme MyApp -sdk iphonesimulator -destination name=iphone-${{ runner.name }}
        build-for-testing | xcbeautify
    - name: Run tests
      run: >
        xcodebuild -scheme MyApp -sdk iphonesimulator -destination name=iphone-${{ runner.name }}
        test | xcbeautify

Note: You can view the workflow file on GitHub here.

The runs-on: self-hosted command is used to make the workflow run on a self-hosted runner instead of the GitHubprovided macOS runner. Both steps for the build and run tests just have the xcodebuild command, the same way a developer runs on a local machine. This makes it easier to reproduce failures locally as developers can just copy and paste commands from the CI.

Another thing to note is the name of destination simulator, iphone-${{ runner.name }}, where the naming convention of name simulators is used without spaces to make it easier to avoid double quotes and to assign each runner its own simulator. 

For more examples, check out this GitHub gist for a build-and-test workflow from a real-life project (the app name has been changed to MyApp). There are three shell scripts included to make build-and-test runs more reliable and to report test coverage. 

Continuous Delivery

When delivering to iOS or macOS App Stores, applications must pass reviews and have a "what’s new" blurb, and it makes sense to bundle some meat of functionality into a release to be worthy of a download for users. Unlike web app projects that can do continuous deployment to production for every minute change, the App Store has record of past versions and release notes. Continuous delivery, unlike continuous deployment, involves the manual step of making the release and providing the release notes. A go-to tool for Xcode continuous delivery is TestFlight. The process of submitting TestFlight releases can be automated either with the help of fastlane or a shell script. 

TestFlight With Fastlane

If a project is already using fastlane to build and upload apps to App Store Connect, we can create the automation workflow similar to what's found here on GitHub. Things to note about this workflow: 

• Only triggered manually by a developer and requires a string to say what to test
• Sets an app build number to match a GitHub Actions run number, and it uses version offset if needed – This allows users to easily find what run made which version of the app. We use a forever-incrementing build number across app version numbers; that is to say, 1.10(255) and 2.15(256) are versions produced by runs 255 and 256, respectively.
• Tags a commit with the version number
• Unlocks the keychain for code signing
• Runs the fastlane release command
• Uploads build artifacts and saves the Xcode archive on the CI machine in Xcode Organizer 
• Uploads debug symbols (dSYM) to Crashlytics

TestFlight Without Fastlane

For projects without fastlane, developers can use Apple's altool and a shell script, as shown in the GitHub gist here. Note that the workflow launches the export-xcode-archive.zsh script, which invokes xcrun altool.

Automated Dependencies Update

The workflow found here will update Ruby Gems and Swift Packages and make PRs to run through a normal CI workflow and code review. Things to note about this workflow: 

• Sets the concurrency group with automatic cancellations of previous runs to allow a single PR at a time
• Clones swift packages into a fresh temporary directory to trick Xcode into updating
• Updates local swift package dependencies
• Reports the pull request number and URL in runner logs

There is also an action to update GitHub Actions: GitHub Actions Version Updater.

Conclusion

Many no-code approaches to continuous integration are quickly outgrown by development teams and become harder to maintain and debug as the complexity of the project grows. Using a low-code approach, developers can take advantage of more than 15,000 available actions from the GitHub Marketplace to construct powerful workflows. Declarative YAML workflows can be supplemented with shell scripts to run commands on a runner. 

Using a self-hosted runner gives development teams better control over the runner environment and speeds up execution by having the system set up and ready for the next job. The automated workflows described in this article can be applied with small changes to iOS/iPadOS or macOS projects.

Additional Resources:

• self-hosted-runner-example GitHub project
  • update-dependencies.yml
  • screen-share.yml 
  • ci.yml
• build-and-test.yml GitHub gist
• test-flight.yml GitHub gist
• export-xcode-archive.zsh GitHub gist
• Getting Started With GitHub Actions Refcard 
• Become a Simulator Expert [video] 
• Expected failures documentation  

This is an article from DZone's 2023 Development at Scale Trend Report.

For more:

Read the Report