Continuous Integration is an integral part of the iOS development process, which gives early feedback when something breaks during application development. In an agile application development, producing working app is not only writing good code but also setting up infrastructure to deliver it continuously. The process of Continuous Integration and automating the build can help to achieve Continuous Delivery. There are various Continuous Integration tools available in the market that can be used on the iOS team. The CI server solution can be self-hosted or cloud-hosted. You can get a whole list of CI servers here with recommendations.
In iOS development, we have few options for a self-hosted CI server, including Xcode Server, Jenkins, and TeamCity. If you are looking for cloud solutions, then there are some promising services, such as TravisCI, CircleCI, BuddyBuild, Bitrise, and GreenhouseCI.
There are multiple tools, so the challenge is which one to choose and to evaluate why one is better than the other. Cloud CI servers are easy to set up and are managed by other companies. They simply just work, if configured correctly. The other things is that when using cloud CI solutions, we don’t need someone to manage servers. However, you have to be careful of various things like privacy, cost, ease of use, availability, and platform support while using cloud-based solutions.
Choosing a self-hosted CI over a cloud CI has its pros and cons, so you should consider what is best for your team. Xcode Server seems to be an easy option for the self-hosted CI, as it is managed by Apple and deeply integrated with Xcode. In this post, we'll see the pros and cons of using Xcode Server for iOS Continuous Integration.
Brief Introduction to Xcode Server
Xcode Server is a Continuous Integration platform developed by Apple for analyzing, building, testing, and archiving iOS and MacOS apps. Painful parts in iOS development are certificates and provisioning profiles. It became more hectic when we wanted to set up certificates and profiles on the CI server.
The latest version of Xcode Server solved lots of issues with certificates and profiles. Xcode Server is very tightly coupled with Xcode, so it became painless to set up and use Xcode Server to perform Continuous Integration. Apple has very comprehensive documentation on Xcode Server setup for CI. However, I wrote a short tutorial to set up the latest MacOS Server to run Xcode bot. You can refer those guides to get started with Xcode Server. Xcode Server has following features:
- Free or cheap solution.
- Self-hosted and deeply integrated with Xcode.
- Device testing.
- Over-the-air (OTA) installation.
- Managed and maintained by Apple.
Xcode Server has all these features, but some people struggled to get Xcode Server working properly. Let’s see the pros and cons of using Xcode Server.
Xcode Server Pros
Xcode Server is only designed to work with Apple operating systems. It stands out from other self-hosted CI solutions like Jenkins and TeamCity because it’s managed by Apple. The other options like Jenkins and TeamCity are managed by third-party companies and mostly written in Java or some other programming language. Xcode Server has improved a lot since it was released the first time with Xcode 5. The latest versions of MacOS Server and Xcode Server have all the features needed to perform efficient Continuous Integration.
Easy to Set Up
As mentioned in Apple’s guide to Continuous Integration, Xcode Server can be set up with a few clicks if you have a Mac or Mac Mini and the MacOS Server. In my previous post on setting up CI with Xcode Server, I wrote down some basic steps to get up and running with Xcode Server and Xcode bot creation. Any iOS or MacOS engineer can set that up within a few hours.
It’s almost free if you have paid Apple developer account. You can get the MacOS Server by redeeming your code. It will cost just under $20 if you are not an Apple developer. This cost if comparatively low, as well, as it’s just a one-time investment.
Deep Integration With Xcode
The Xcode bots are directly integrated into the real Xcode, not any Xcode plug-in, and tied to the source code and schemes. Xcode Server uses the macOS Server. It’s real Xcode running on a server, so it’s a deeply integrated CI Server with Xcode. Developers can see all the test reports and code coverage reports straight into Xcode running on a development Mac. The developers can create, edit, and manage Xcode bots from the local machine using Xcode. There is no need to go to a web interface to manage CI jobs. However in Jenkins or TeamCity, we need to visit web interfaces to manage CI jobs.
Painless Certificates and Provisioning Profiles Setup
One of the painful things in iOS development is setting up certificates and provisioning profiles. Apple introduced a new way of code signing at WWDC 2016 in the talk What’s new in Xcode app signing. The process became easy and painless with Xcode 8. While using Xcode Server, we need to copy all the Certificates and Profiles to the /Library/Developer/XcodeServer/Certificates and /Library/Developer/XcodeServer/ProvisioningProfiles directories and we are done.
Built-In Code Coverage
Apple introduced this nice feature of generating code coverage in WWDC 2015 in the talk Continuous Integration and code coverage in Xcode. The process of generating code coverage is easy; the only thing we need to do is tick the box while creating the Xcode bot. The developer doesn’t need to use any other third-party tools like Fastlane or Xcov to configure code coverage.
Now, code coverage will be generated for the for each integration. The cool thing is that it shows code coverage file-by-file so that we know where to add unit tests.
Built-In Real Device Testing
Xcode Server supports testing in the simulators as well as in the real devices. It has a built-in test service as shown in the above image, reducing the burden of configuring Fastlane Scan or scripting xcodebuild to run our unit and UI tests with XCTest. Once devices like iPhones and iPads are attached to the server, then they start appearing in the Devices tab so that we can select then for our app testing.
Distributed Test Execution Without Building
Previously, there was a need to build an app before testing. Then, Xcode 8 introduced a nice feature called Build for Testing in the talk Advanced testing and Continuous Integration, which generates an
xctestrun file in the Derived Data that can be used to run tests without building using the latest
xcodebuild so that we can share this file with multiple Xcode Servers to enable parallel and distributed testing.
Build Hosting and OTA Installation
Xcode Server can
Archive and host a build on the Xcode Server machine. The hosted build can be distributed to QA and product people to have a look. The build can be downloaded using Server address in a browser like Safari from the devices. There will an option to install the build on the devices. This reduces burden of using a build hosting service like Crashlytics or HockyApp, as test users can download the build straight from Xcode Server.
Rest API and Environmental Variables
There is an option to use Xcode Server APIs and Environmental variables so that we can script Xcode Server according to our project needs. We can use Xcode Server environmental variables in the pre-integration or post-integration scripts in the ‘Trigger” sections. We have the ability to define our own environmental variables.
Xcode Server gives us ability to run bash scripts, i.e., triggers, before integration and after integration. Triggers can be extremely useful for setting up dependencies, i.e., CocoaPods, Carthage, or Swift Packages, before running build as part of the pre-integration triggers. Xcode Server also has post-integration triggers that can be used to run other scripts to do something or send email notifications.
Xcode Server eliminated the need to use a lot of Fastlane tools as most go the things are pre-configured. Fastlane is a nice tool build on top of Apple’s command line tools for the easy configuration of complex scripts. It automated iOS Development tasks using multiple tools to enable continuous delivery. Xcode Server eliminated need for Scan, Xcov, Gym, and Sigh and match.
Physical Access to MacOS Server
Xcode Server being an self-hosted CI solution, we have physical access to machine and MacOS Server installed on the machine. We can tweak Xcode Server as per our need if required. We can have fun controlling the CI server with the customization option.
Xcode Server Cons
As we have seen, Xcode Server has lots of useful things, but there are some disadvantages to using Xcode Server. Most of the previous old pains of code signing and certificates has been removed in the latest Xcode. There are still some issues whiles using Xcode Server.
No Pull Request Support
Assuming most of us are using GitHub as a code hosting service, we need to create pull requests before we merge code in the main branch. Xcode Server supports integration on different GitHub branches, but it still doesn’t support testing of pull requests, which is the most painful part of app development. In order to enable pull requests testing, we have to write our own GitHub webhook and generate Xcode Bots using Xcode Sever API, or we have to use open-source tools like Buildasaur, which require additional setup.
Build Never Finished
Some developers complained that Xcode bot keeps running forever if something goes wrong, and debugging is extremely difficult. This problem occurs when there is an issue with code signing.
Extra Management of Servers and Bots
Xcode Server being a self-hosted CI solution, it might not run on its own eve we configured it correctly. Someone need to take care of machines and Servers to make sure everything this running smoothly. It gives rise to hiring TechOps resources to manage those servers or use developers time to manage servers, which is costly.
OTA Installation Fails on Public Domains
Some Xcode Server users complain about configuring Xcode Server with public domain. OTA installation doesn’t work all the time on the public domain. It throws a "couldn’t connect" error. The issue was that once you clicked on the Install button, nothing happened, or that the Install button never appeared. It needs additional encryption, SSL Certification, and port setup to get that properly working. This post explains the pain of using Xcode Server.
No Cross-Platform Support
Xcode Server is designed to work only with Apple operating system so there isn’t any possibility to use those server machines for any other platform like Android.
Not Suitable for the Distributed Teams
Xcode Server can be used effectively with small iOS teams but if you have a big organization and want to share Xcode Server all over the distributed teams, then it's probably hectic to manage. It requires additional setup for domains and networking.
Xcode Server is a valid Continuous Integration solution for iOS apps as it’s maintained by Apple and has built-in static analysis, testing, building, archiving, and reporting mechanisms to reduce the burden on developers. However, we need to consider the cost of managing CI server machines.
What are your experiences with Xcode Server? Do you feel it will be the number one choice for app developers? Weigh in with a comment below!