A few weeks ago at the Jenkins User Conference, I went to see Andrew Phillips, VP of Products at XebiaLabs, present a packed session titled “Building an Enterprise Continuous Delivery Machine Around Jenkins”. His talk focused on the components of a Continuous Delivery environment, and whether it made sense, for each component, to build a new solution, buy a tool, or use Jenkins.
Some general tips before we go down the list: Andrew recommended that users stay close to the “core” of what Jenkins is meant to do. For anything that is not directly addressed by Jenkins, consider what new plugins you need to install, and how many “run a script” tasks need to run. If what you need requires tons of either of these, then reconsider Jenkins.
For a better visual, you can refer to the “Jenkins onion” slide. The further away from the core of Jenkins functionality (continuous integration, test execution, artifact repository, and test result management), the less suitable Jenkins is for that particular task.
The point of the session, according to Andrew, was “To provide an inventory of what components make up a continuous delivery stack, and secondly to look at which components are well-filled by Jenkins, and which are perhaps not-so-well filled.” When asked why bring up when Jenkins shouldn’t be used, he replied that part of being a mature community is recognizing when a tool won’t work well, and “if you’re going to Jenkins User Conference, your ultimate goal is to learn how to use Jenkins more effectively.” This is the list of components he covered, and what was the best solution for each:
- Continuous Integration: If Andrew had said anything other than “use Jenkins” he might have started a riot.
- Test Execution: Jenkins has plugins available for many different testing tools at both the code and system level, as well as integrations with hosted test services. Jenkins is great for this task as well.
- Artifact Repository: Using Jenkins as an artifact repository can be tricky, as plugins must be usable from the build process, and not all the features are easily available out-of-the-box. Either buy a repository tool like Artifactory, or use Jenkins if you have a very simple use case and don’t use Maven.
- Test Result Management: Jenkins’ plugins for testing tools each have their own different visualization style. On top of that, these plugins have low or no ability to generate custom reports without hacking a plugin. The answer: buy a tool or build your own. Despite the effort, build is a good way to go because there aren’t that many tools out there.
- Environment Provisioning: Assuming you’re not using a PaaS or Docker, Jenkins is not the way to go. Jenkins’ only concept of remote systems is slaves, which is not a good model for target requirements. There are plenty of tools you can buy that integrate with Jenkins.
- Application Release Automation: Deployments may be part of your environment processing tool, otherwise you’ll have to include a lot of “run this script” tasks. There are tons of options you can buy out there, and Jenkins should only be used if deployments are handled by a technology like Cloud Foundry.
- Release Coordination: Doing this in Jenkins requires a ton of plugins or Jenkins Workflow. If the process is fully automated, then Jenkins is perfect. If your organization has a lot of non-technical product managers who need to see what’s going on, you should buy a tool instead.
Some of these tools were obviously not what Jenkins was built for, but when you’re using an open source tool, it’s important to use it in a way that doesn’t disrupt your system down the line.