Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Cache Carthage to Speed Up iOS Continuous Integration

DZone's Guide to

Cache Carthage to Speed Up iOS Continuous Integration

Learn how to set up Carthage on TravisCI and use it to speed up continuous integration for your iOS apps, complete with code. And no need for CocoaPods!

· Mobile Zone
Free Resource

Download this comprehensive Mobile Testing Reference Guide to help prioritize which mobile devices and OSs to test against, brought to you in partnership with Sauce Labs.

There is hardly any iOS project that doesn't have any dependency. Almost every project has to manage some sort of dependencies, either internal or third party. Every major programming language has its own dependency management system. Swift has various options to manage the dependencies. Apple has been working on its own Swift Package Manager for ages. Seriously, ages- iOS developers are keen to get the official dependency manager that has been supported by Apple but it seems like the things progressing with tortoise speed. The Swift Package Manager still not supports iOS project. CocoaPods is the most commonly and traditionally used dependency management for iOS projects as developers didn't have any other option. CocoaPods is a magic, black magic, superpower, incantation that f***s up your Xcode project to get things working. It builds dependencies, creates a workspace, executes some scripts, and many other things. CocoaPods works well, but very few developers have understood that what has been happened under the hood when they run pod install command. Everyone is happy if everything works, but when pod install fails then everybody gets panic. Nobody knows how to fix that, it takes hours, days, or weeks to sort out that mess created by CocoaPods. Love it or hate it, either way, you have to use it.

The Birth of Carthage

Somone understood the pain that iOS Developers are having with CocoaPods and produced a baby called Carthage to be an alternative for the CocoaPods. Carthage only works for the dynamic frameworks and gives developers an ability to take full control of what they are doing with Xcode project. It simply builds the frameworks and developers have to manually link those frameworks to respective build phases of the target. Carthage works great but it has few drawbacks

  • Carthage only works with dynamic frameworks.
  • Carthage has small community of developer contributing to the project.
  • There are few manual steps involved while using Carthage.
  • The major drawback was that it is so slow, especially on CI servers. Carthage checks out and builds each framework for every build. This is definitely not a great approach for CI, as builds will be incredibly slow and lengthy. There was no clean solution to cache the Carthage/ directory if nothing changed. In this post, we will see how to cache the Carthage directory on CI servers and speed up the builds.

Carthage on CI

The most common use of Carthage is to run a Carthage update or script as part of the CI script which modifies all the dependencies to use the latest version changes Cartfile.lock file, which is not correct, causing the checkout and building of all the dependencies. This will make the build the CI builds utterly slow. So, the ideal way to use Carthage on CI would be:

  • Check that Cartfile.lock exists or not. If this file doesn't exist then we have to bootstrap all the dependencies using Carthage bootstrap.
  • If Cartfile.lock exists but not changed, we don't need to perform a bootstrap or update operation and we can use contents inside Carthage directory from the previous build.
  • If Cartfile.lock exists and changes some dependencies, then we have to only bootstrap those outdated dependencies rather than all.

Now that we know the mechanism of how to use Cartfile.lock status to perform a bootstrap or skip bootstrap, the next step would be to write a script to do that. There are lots of ways we can achieve this, depending on the environmental variables provided by your CI server. The script to use with Jenkins can look like this:

[ -f Cartfile.resolved ]; then
  CARTFILE_CHANGED=`git diff --stat $(GIT_PREVIOUS_SUCCESSFUL_COMMIT) $(GIT_COMMIT) | grep '\|' | awk '{print $$1}' | grep Cartfile.resolved`     
  if test "$(CARTFILE_CHANGED)" ; then          
    echo "## STEP: Updating Carthage" 
    carthage bootstrap --platform iOS --no-use-binaries
   else          
    echo "## Carthage is up to date"      
  fi 
else      
  echo "## STEP: Installing Carthage Dependencies" 
  carthage bootstrap --platform iOS --no-use-binaries 
fi

The Jenkins Git Plugin sets up that variables to compare the status of the Cartfile.lock. The full explanation can be found in this blog post here. There is an open-source tool to compare the status of Cartfile.lock called CartfileDiff which can be used to determine the changes in the Cartfile.lock if your CI server doesn't have specific environment variables. We will see how to use CartfileDiff later in the post.

Carthage Setup

Caching the Carthage/ directory is easy on the self-hosted CI servers however the cloud-based CI servers launch fresh virtual machine for every builds. This makes caching of Carthage directory difficult. Most of the cloud-based CI providers for iOS have recognized this problem and provided the mechanism to caching e.g Travis has Cache for most of the services. There might be other CI services like CircleCI might be doing the same things. Let's see how to use TravisCI cache service with the demo project.

Let's create a single view project in Xcode called Carthage-CI including the UI Test target. Now, we will get XCFit framework which is a small library that I wrote for BDD in the iOS project using Carthage. Create Cartfile and add

github "Shashikant86/XCFit"

Now run carthage update --platform iOS to download and build all the dependency. Add the built framework to the UI Test target manually drag and drop and run the Carthage script as part of build phase as per the Carthage documentation. Now that, we have framework inside the app, it's time to write the script to use Carthage intelligently on CI. Make a couple of executable files in the project

$ mkdir script 
$ touch script/bootstrap 
$ touch script/intelligent-bootstrap 
$ chmod +x script/bootstrap
$ chmod +x script/intelligent-bootstrap 

Now that we have created two files in script/bootstrap the script we will run Carthage bootstrap with dependencies passed to that script. The script/intelligent-bootstrap script we will do the magic of determining the Cartfile.lock status. In the script/bootstrap file, insert the following code:

#!/bin/bash

carthage bootstrap $@ --platform iphoneos --no-use-binaries || exit $?

You may need to install CartfileDiff tool at this stage by following the CartfileDiff README. This will bootstrap the number of dependencies passed to this script. Now, in the script/intelligent-bootstrap file, insert this code:

#!/bin/sh

SCRIPT_DIR=$(dirname "$0")
BOOTSTRAP="$SCRIPT_DIR/bootstrap"
CACHED_CARTFILE="Carthage/Cartfile.resolved"

if [ -e "$CACHED_CARTFILE" ]; then
  OUTDATED_DEPENDENCIES=$(cartfilediff "$CACHED_CARTFILE" Cartfile.resolved)

  if [ ! -z "$OUTDATED_DEPENDENCIES" ]
  then
    echo "Bootstrapping outdated dependencies: $OUTDATED_DEPENDENCIES"
    "$BOOTSTRAP" "$OUTDATED_DEPENDENCIES"
  else
    echo "Cartfile.resolved matches cached, skipping bootstrap"
  fi
else
  echo "Cached Cartfile.resolved not found, bootstrapping all dependencies"
  "$BOOTSTRAP"
fi

cp Cartfile.resolved Carthage

This script will use CartfileDiff tool to determine if Cartfile.lock has been changed or not and execute the bootstrap accordingly. If there are no changes in the Cartfile.lock then, the bootstrap step will be skipped.

Caching Carthage on TravisCI

Now that we have all the setup to determine the status of the Cartfile.lock file, the next thing is to configure the TravisCI to run those steps with cached Carthage directory. Create a.travis.yml file at the root of the project and insert the following code:

{
  "language": "objective-c",
  "osx_image": "xcode9",
  "cache": {
    "directories": [
      "Carthage"
    ]
  },
  "before_install": [
    "curl -L -O https://github.com/Carthage/Carthage/releases/download/0.25.0/Carthage.pkg",
    "sudo installer -pkg Carthage.pkg -target /",
    "curl -L -O https://github.com/YPlan/CartfileDiff/releases/download/0.1/CartfileDiff.pkg",
    "sudo installer -pkg CartfileDiff.pkg -target /",
    "gem install fastlane --no-ri --no-rdoc --no-document"
  ],
  "install": true,
  "script": [
    "script/intelligent-bootstrap",
    "fastlane scan -s Carthage-CI"
  ],
  "os": "osx"
}

Note that we are using cache from TravisCI of Carthage/ directory. We also have to download and install the CartfileDiff tool and Fastlane to run tests for the scheme Carthage-CI. Now we need to enable the project on Travis to trigger the build.

Source Code and TravisCI Job

I have created the repository on Github to demonstrate what we said earlier. The repository is Carthage-CI and you can see the TravisCI job here. Look at the job history below, the latest job saved almost 3 minutes because of caching. As XCFit is a very small library, if you have bigger libraries, this approach would be a time saver.

Conclusion

Using the Caching features of the CI servers and smart strategy to perform the Carthage bootstrap, we can reduce the build time drastically on the CI server. How are you using Carthage on CI, what are your experiences? Feel free to comment below.

Analysts agree that a mix of emulators/simulators and real devices are necessary to optimize your mobile app testing - learn more in this white paper, brought to you in partnership with Sauce Labs.

Topics:
continuous delivery ,devops ,dependency management ,swift ,deployment ,ios ,mobile ,mobile app development ,continuous integration

Published at DZone with permission of Shashikant Jagtap, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}