Speeding up Android Builds with an Incremental Java Compiler

This post was modified from its original version, we’ve added more intergrations with the annotation processors and included new stats from the upcoming release.

Since we launched JRebel for Android last year, we’ve learned a lot about the Android build system, how it behaves in real-world projects, and where actual build time bottlenecks occur. Most of this invaluable feedback came from our JRebel for Android users and provided us ideas on how to make it even faster!

Today, I’m pleased to announce that the JRebel for Android now includes an incremental compiler that makes the performance of code and resource updating even snappier.

When you use the default toolchain, specifically Android Studio and Gradle, there are two major tasks in the build process of an Android app that are not entirely incremental. First is the compilation of Java source files and the other is resource packaging. In this post, I wanted to share some details about the compilation aspect and how we made it incremental, what challenges still exist, and where it can take you performance wise.

Background

Although incremental Java compilation is not new, it’s not fully incremental in all use cases. Incremental Java compilation was enabled in Eclipse. However after moving to Android Studio and Gradle, the Android Gradle plugin enables the incubating feature of Gradle’s incremental compiler by default from version 2.1 and later.

On the down-side, it does not support any annotation processors, for example, if you’re using Dagger, Dagger 2, or Butter Knife in your project, it’s fairly useless. And since any annotations need to be processed every time a change the to code is made (not even to the annotations themselves), the compile time for the project piles up and takes quite a chunk of the total build time. Just run your build with –profile and see how long javac takes to run. If you feel like doing something nice today, you can also ping me on Twitter about the results you got; I’d love to see your mileage: @stensuitsev.

Incremental Compiler in JRebel for Android

Recently we’ve shipped an incremental Java compiler for JRebel for Android. One of the principal objectives was to ensure that it supported annotation processors. The challenge with annotation processors is that each of them requires a custom integration. So far we’ve built a handful of these, to support the most prominent and widely used libraries. By default when using JRebel for Android, you will have the incremental compiler enabled for your project. And it will try its best to speed up your compile times unless it encounters an annotation processor that we don’t yet support.

Here’s the list of the libraries that have integrations for at the time of writing:

• Butter Knife
• Parceler
• Icepick
• Dagger
• Dagger2
• AutoValue
• Greenrobot
• EventBus
• Dart
• Henson
• Deeplink
• Epoxy

So if you don’t use the annotation processor at all, or if you use a library mentioned in the list above or a combination of these two, then you’ll make use of all the benefits of JRebel for Android’s incremental compilation.

The list of the libraries we integrate with is not final, and we’d be super happy if you could let us know which libraries with annotation processors you are using! Leave a comment below or find me on Twitter if you prefer.

How Does Incremental Compilation Speed up Your Build Times?

Now let’s look at the numbers and measure the impact of incremental compilation with JRebel for Android. I’ll be using an internal project codenamed BFA (big freaking application) that we have specifically built for testing large code and resource based use cases.

A quick summary of BFA:

• 4706 application classes
• 14497 Android resources
• 84 libraries (Dagger 2, Butter Knife)
• 7971 library classes

For a simple comparison, here are the same metrics for the GoogleIO 2015 application:

• 618 application classes
• 922 Android resources
• 28 libraries
• 6157 library classes

We’ll take a look at the total time it takes to see any code or resource changes without and with the incremental compiler enabled and how much time was spent on the compilation in total. For each measurement, five compilation runs were done, and the median used to eliminate outliers. If you’re interested, I have a fairly old MacBook from 2013, with these specs: 2.4GHz i5, 16GB of memory and 250GB SSD.

In the first scenario, we’ll modify the onCreate() method of the main Activity of the BFA application and use “Apply changes”. Here is the output from Gradle profile with no incremental compiler. “Task Execution” is the time it took to execute all the build tasks needed for JRebel for Android to apply changes. Let’s take a look into “Task Execution” to see where we’re spending our time.

You can see that the majority of time is spent on the compilation task “compileFreeLegacyFatDebugJavaWithJavac” and it’s 12.8 seconds.

We also ran this measurement with the Instant Run enabled, and got the following report:

You can see that the compilation times are in the same ballpark as for the JRebel for Android without the incremental compiler. It also makes sense because the both just use the same Java compilation build task.

Now let’s look at how JRebel for Android with the incremental compiler handles this challenge. Below you can see the corresponding profile report.

So, the compile time went from 12.8 seconds to 1.6 seconds (a 8X improvement!), because the incremental compiler in JRebel for Android avoided most of the no-op work and recompiled only the necessary classes.

Are There Any Downsides to the Incremental Compilation?

When doing an initial compilation the compiler needs to read in and parse all of the project’s classes to perform dependency analysis. This data will be used to determine which classes need to be recompiled for any given change.

So to lay all my cards on the table, I want to show you the task breakdown for the first clean install as well. Here’s the profile report for the compile task of a clean install.

The time spent on the compilation was 15.9 seconds, which is roughly the same as when we just applied a change. It is not a surprising result since every change triggers approximately the same amount of work. Now let’s look at the breakdown with the incremental compiler turned on:

Initial compilation takes 31 seconds, which is longer since the incremental compiler has to build the data structures and capture the essential information about the classes you compile. Notice that in the grand scheme of things the slowdown is pretty much negligible. The clean build takes so much time (2:20 in the example above) that a couple of additional seconds won’t make any difference.

It becomes a very different story when you actually work on your application. Then most of the Gradle’s work is already done, and the benefits of the incremental compilation can’t be ignored. The initial compilation overhead is won right back the next time you compile the project. And after that, you’re going up the curve in overall time savings.

Summary

I wanted this post to accomplish two things. First I wanted to announce the amazing achievement of the JRebel for Android team in shipping the incremental Java compiler for Android. Secondly, I wanted to highlight some of the challenges we had to overcome when we designed and developed this capability. These challenges were focused around annotation processors which require additional custom integrations to support incremental compilation.

All the effort developing the JRebel for Android incremental compiler has resulted in a significant reduction in build time, and you can see in the above examples how incremental compilation saves you time every time you build your project. And JRebel for Android becomes even more efficient at updating code. With this, you don’t have to sit and wait for your build to finish and quickly get back to coding.

Now I know you want to try out all of this goodness on your project. In fact, nothing would make me happier… well, maybe if you buy it outright, I’d be happier, but you should be smart and try it out first. It’s really easy to do, hit the button below and follow the instructions to enable it for your project.