Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Trimming the (Build) Tree with Bazel

DZone's Guide to

Trimming the (Build) Tree with Bazel

Bazel caches your tests, which can help minimize the work your build needs to do. Here's a quick demonstration of the technique.

· Big Data Zone ·
Free Resource

Hortonworks Sandbox for HDP and HDF is your chance to get started on learning, developing, testing and trying out new features. Each download comes preconfigured with interactive tutorials, sample data and developments from the Apache community.

Jonathan Lange wrote a great blog post about how Bazel caches tests. Basically: if you run a test, change your code, then run a test again, the test will only be rerun if you changed something that could actually change the outcome of the test. Bazel takes this concept pretty far to minimize the work your build needs to do, in some ways that aren’t immediately obvious.

Let’s take an example. Say you’re using Bazel to “build” rigatoni arrabiata, which could be represented as having the following dependencies:

recipe

Each food is a library which depends on the libraries below it. Suppose you change a dependency, like the garlic:

change-garlic

Bazel will stat the files of the “garlic” library and notice this change, and then make a note that the things that depend on “garlic” may have also changed:

dirty

The fancy term for this is “invalidating the upward transitive closure” of the build graph, aka “everything that depends on a thing might be dirty.” Note that Bazel already knows that this change doesn’t affect several of the libraries (rigatoni, tomato-puree, and red-pepper), so they definitely don’t have to be rebuilt.

Bazel will then evaluate the “sauce” node and figures out if its output has changed. This is where the secret sauce (ha!) happens: if the output of the “sauce” node hasn’t changed, Bazel knows that it doesn’t have to recompile rigatoni-arrabiata (the top node), because none of its direct dependencies changed!

The sauce node is no longer "maybe dirty" and so its reverse dependencies (rigatoni-arrabiata) can also be marked as clean.

The sauce node is no longer “maybe dirty” and so its reverse dependencies (rigatoni-arrabiata) can also be marked as clean.

In general, of course, changing the code for a library will change its compiled form, so the “maybe dirty” node will end up being marked as “yes, dirty” and re-evaluated (and so on up the tree). However, Bazel’s build graph lets you compile the bare minimum for a well-structured library, and in some cases avoid compilations altogether.

Hortonworks Community Connection (HCC) is an online collaboration destination for developers, DevOps, customers and partners to get answers to questions, collaborate on technical articles and share code examples from GitHub.  Join the discussion.

Topics:
big data ,bazel ,cache

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}