Fearless iOS Continuous Delivery With TestFlight and Phased Release

The major obstacles to continuous delivery are the lack confidence in the release candidate and fear of critical bugs in production. While releasing an iOS app, there is always the fear and risk of bugs in production, no matter how much time is spent on testing in the pre-production environment. It's because we are unsure that our beta testing environment mimics the production environment. There are loads of third party iOS beta testing services available in the market, but Apple has its own beta testing service platform called TestFlight to give us confidence in our testing.

In previous posts, we have seen how to setup end to end continuous delivery pipeline for iOS app with Fastlane. Using this setup, we can deliver any commit or merge to a GitHub branch to TestFlight, causing multiple builds uploaded to iTunes Connect. It would be a good idea to maintain those builds in TestFlight and distribute them to relevant people to test.

Why TestFlight?

There are various beta testing platforms available to test ad-hoc builds, like Fabric or HockyApp. There are eleven of them mentioned in this great blog post, TestFlight Alternatives, which compares all of them. TestFlight has a major enhancement, as mentioned in What's New in iTunes Connect at WWDC 2017 session 302. There are multiple reasons why we shouldn't use other beta testing platforms; some of them are as follows:

• TestFlight is Apple's official beta testing platform, so there is no possibility of breaking it if the API is changed.
• It's easy to distribute the builds to anyone by adding testers to internal or external testers.
• No need to maintain devices and provisioning profiles for ad-hoc distribution.
• No need for separate configurations to distribute.ipa file like other beta testing platforms.

This is a small list, but there are various other reasons that you should think of getting rid of these third party beta testing services. The top reason would be the recent enhancement by Apple in TestFlight, which we will explore in this post.

Multiple Builds and Groups

As per the latest upgrade in TestFlight, we can create multiple groups in and distribute specific builds to a specific group. In an Agile world, we can create a group for QA Engineers, Product Owners, and Stakeholders. We can continuously deliver the builds to TestFlight and assign the build to a specific group to get early feedback. As suggested in the WWDC talk, we can create Alpha, Beta, and Release candidate groups. As builds mature, we can promote them to a final Release Candidate group. We can also create a group for experimentation to check specific features separately.

Switch Between the Builds

There might be a tester added to multiple groups, and they can now see the previous builds to which they have access and select another build for testing. This feature will be very helpful for QA Engineers to test multiple features easily and without hassle. The previous builds can hold the test information so that testers can identify the information to test. Now TestFlight builds will also show the session count and the crash count. Apple is also planning to extend the testers limit to 10,000 testers at some point this year.

Phased Release

The practice of Continuous Delivery enables us to deploy new builds to TestFlight, but production is a unique environment. No matter how much testing we have done in pre-production, we have to watch the behavior of the app in production once released. If we are about to release a new feature which involves risk, or we are less confident about it but we want to test how it performs in production without taking too much risk, then phased releases can solve the problem.

When the feature is enabled for a phased release, it will be distributed to certain users for seven days. The limited users will receive the updates and try the build in the production to verify the new feature. The percentage of users selected are shown in the figure below:

There is a difference between the phased release and feature flagging; you can read the main difference on the blog here, but it is similar.

Enable Phased Release

There is a new radio button in iTunes Connect called "Use phased release" which can be selected to enable the phased release for the build. We can select it for any build which we are less confident about and we don't want to distribute to 100% of our users.

Roll Out or Rollback

There are two possibilities after we release the build using phased release: either our feature is working fine in production, or our feature has bugs. We have the ability to deal with both of these situations.

• When the phased release is OK, we can deliver it to 100% of users by pressing the "Release to All Users" button. This will stop the phased release and everybody will get the updated version.
• When the phased release has minor issues, we can pause the release for a maximum of 30 days and we can submit another build with the new fix.
• When the phased release has serious issues, we can remove the current version from sale.

Advocate Continuous Delivery

The newly announced features like multi-build, multi-group, and phased release are the catalyst for continuous delivery. These new features give developers the confidence to manage risky releases by taking a calculated risk. There will be big changes likely to happen in iOS projects all over the world.

• iOS developers won't be scared about making multiple TestFlight builds, provided they are automated.
• Everybody will understand the value of an automated release/deployment process.
• With the help of groups in TestFlight, we can get early feedback from QA, Product Owners, and Stakeholders. Once everyone is happy, releasing is a click away.
• Product owners won't be scared to make frequent releases using phased release, as there won't be major risk even if the application is broken in production.

Delivering the product (iOS apps) to end users quickly and whenever needed is the key to success. It can be easily achieved by setting up the Continuous Delivery pipeline and using the newly added features like Multi Build TestFlight and phased release.