The New Go 1.5 Vendor Handling

When Go was first released, a gap between what developers needed and what the go tool provided became apparent. That gap was the management of vendor packages, which quickly lead to the rise of applications such as Godep and GPM to fill that need. With the release of Go 1.5 comes changes to the go toolchain that starts to natively address the gap. In this post I'll talk about how it works and how you can leverage it.

The Need

Packages in Go need to live within the GOPATH. The go toolchain makes certain assumptions and problems arise when packages are outside the GOPATH or when you have multiple directories in the GOPATH.

Traditionally, you can only have one version of a package in your GOPATH and using go get retrieves the latest version on the master branch. It isn't setup to retrieve a specific version. What happens when two developers get different versions of a dependent package to the one they are working on? Or, what happens when you're working on two packages that require different versions of the same dependency? You can easily end up with unreproducible environments.

To fix this tools such as Godep and GPM stepped in with hacks around the problem. For example, Godeps allows you to take a snapshot of your dependent packages to store in a subdirectory and then others could restore that version.

If only the Go toolchain helped solve this problem.

The Vendor Directory

With the release of Go 1.5 there is a new opt-in feature (that's planned to be on by default in the future). It provides a way to specify vendor packages in a way the Go tool can work with.

First, you need to set the environment variable GO15VENDOREXPERIMENT to 1. This is how you opt-in at the moment. Once you do that your project can have a special vendor directory containing your dependencies. The layout looks like:

- GOPATH/github.com/example/foo
  |
  - main.go
  - vendor/
     |
     - github.com/
       |
       - foo/bar
       - Masterminds/vcs

Your project lives in the GOPATH normally. But, your projects dependencies can live in the vendor/ directory. When go looks up a package referenced in the source it will look in the vendor/ directory, then look in the GOPATH, then look in the GOROOT (where standard library packages such as fmt live).

This allows you to have versions of dependent packages specific to a package in on place with no hacks.

This even works recursively. For example, if one of your dependent packages has a vendor directory with dependencies in it those will resolve as well. And, if two dependent packages have different versions of the same dependency the right one will be used.

This recursive nature can introduce sideways problems so I recommend paying careful attention when using it.

Management Tooling

The go tool in 1.5 does little more than what I just described. So, how can you manage the packages in the vendor/ directory?

This is where I'll get a little opinionated. When the release of Go 1.5 we pivoted Glide to be a support tool. The goal is to help others be successful with this new way to work with dependencies.

Glide provides a way to document, fetch, and manage the dependencies within the vendor/ directory while working cleanly with the go toolchain. It tries to compliment the functionality in Go.

You can use Glide to repeatedly install the same dependency versions between different environments and developers. You can think of this as something similar to npm, composer, bundler, and the package managers used by other languages.

If you want to vendor the dependencies from vendors, that is store the source files for the dependent packages in your source control system, Glide will work as well.

Glide has lots of other management features and we plan to continue development down this road to try and make things easy.

Note: there are other tools for managing dependencies in the vendor/ directory. When choosing one be sure you understand the purpose of the tool and where it's going.

The Bright Future

Other languages and platforms have come to some great patterns and the Go toolchain with the surrounding ecosystem is heading in a good direction. I look forward to the bright future ahead of us.