Mono-Repo Build With Gradle

Sometimes, we are faced with a project that is both open source and part proprietary. Here’s how I address the challenges of synchronization amongst builds.

When faced with a partly open-source and partly closed-source project, it is common to use a Git sub-tree to synchronize them. The open-source project is added in a folder of the closed-source project, and all development happens in that root project (alias the mono-repo). On top of that, we want our build tool to handle the sub-tree project as it is a part of the mono-repo.

This is the tricky part, as the sub-tree must be buildable and must be buildable from the mono-repo. In cases of multi-module projects, we also want to express dependencies between modules of those projects and have our IDE detect those dependencies.

One solution is provided by Gradle: it is called includeBuild. This solution works great but has a few drawbacks for this use case.

Problem With includeBuild

When using the includeBuild method, if you run a gradle test at the root of the project, it will run tests only on projects "natively" in the root project. This can be all right when you only need to include libraries with their own life cycle, but in our use case, we want to have a single task to build and test everything.

With  includeBuild, we would have:

$ gradle run
> Task :other-module:run
com project: Hello World

Even if we have a task named run in one module of the sub-tree, only the task from module "natively" in the root is ran.

With our approach, all tasks with the given name will be launched.

$ gradle run
> Task :other-module:run
com project: Hello World
> Task :my-project:module-b:run
oss project: Hello World

You can check out the code from this example project on GitHub.

How it Works

The main idea is to include the settings.gradle of the sub-tree in the mono-repo project. However, in order to keep the name constant when declaring dependencies, we must play a little bit with the paths of the projects.

Setting Up the Sub-Tree

The sub-tree does not declare its projects in settings.gradle but in the other file; here it is oss-settings.gradle.

All declared projects are children of a new root project — here, :my-project and this project are also included.

The settings.gradle imports this file and changes the paths:

apply from: ‘oss-settings.gradle’
def fixPath
fixPath = { project ->
  String relativeProjectPath = project.projectDir.path.replace(settingsDir.path, “”)
  project.projectDir = new File(relativeProjectPath.replace(“/my-project/”, ‘’))
  project.children.each fixPath
}
rootProject.children.each fixPath

Dependencies between modules can be expressed as usual but it must include this new root project.

dependencies {  
  compile project(‘:my-project:module-a’)
}

Setting up the Mono-Repo

Once the sub-tree itself is set up, a few tweaks must be made to the mono-repo project.

Settings.gradle must apply the oss-settings.gradle, set the path correctly, include the root project of the sub-tree, and then add its own projects as usual.

apply from: ‘sub-tree/oss-settings.gradle’
def fixPath
fixPath = { project ->
  String relativeProjectPath = project.projectDir.path.replace(settingsDir.path, “”)
  project.projectDir = new File(relativeProjectPath.replace(“/my-project/”, ‘sub-tree/’))
  project.children.each fixPath
}
rootProject.children.each fixPath
include ‘:my-project’
project(‘:my-project’).projectDir = “$rootDir/sub-tree” as File
include ‘:other-module’

def fixPathfixPath = { project ->  String relativeProjectPath = project.projectDir.path.replace(settingsDir.path, “”)  project.projectDir = new File(relativeProjectPath.replace(“/my-project/”, ‘sub-tree/’))  project.children.each fixPath}rootProject.children.each fixPath

include ‘:my-project’project(‘:my-project’).projectDir = “$rootDir/sub-tree” as File

include ‘:other-module’

Results

In this example project, running the run task in the sub-tree directory gives:

$ gradle run
> Task :my-project:run
oss project: Hello World

And, running the same task in the mono-repo gives us:

$ gradle run
> Task :other-module:run
com project: Hello World
> Task :my-project:module-b:run
oss project: Hello World

Plugins Sharing

We often use custom plugins directly in the buildSrc to share build logic. This behavior can be kept using the following method.

In the sub-tree project, add your buildSrc  directory and add an extra build file to declare the plugin:

apply plugin: ‘groovy’
apply plugin: ‘java-gradle-plugin’
gradlePlugin {
  plugins {
    ossPlugin {
      id = “oss-plugin”
      implementationClass = “OSSPlugin”
    }
  }
}

In the mono-repo, the buildSrc project must include this project as a runtime dependency

build.gradle:

dependencies {
  runtime subprojects
}

settings.gradle:

include ‘:oss-buildSrc’
project(‘:oss-buildSrc’).projectDir = “$rootDir/../sub-tree/buildSrc” as File

With this technique, plugins from the plugins of the sub-tree project can be used in the mono-repo projects, and the mono-repo can declare its own plugins.

When running the tasks from the sub-tree:

> Task :my-project:module-b:customOSSTask
OSSPlugin is applied

When running the tasks from mono-repo:

> Task :other-module:customComTask
ComPlugin is applied
> Task :other-module:customOSSTask
OSSPlugin is applied
> Task :my-project:module-b:customOSSTask
OSSPlugin is applied

Conclusion

This is the solution I found to handle this kind of use case. If you have encountered similar situations, I’d be very interested to know how you handled these kinds of issues. Feel free to let me know what you think in the comments below.