Over a million developers have joined DZone.

Ensuring Corporate Standards in Pipelines With Custom Marker Files

DZone's Guide to

Ensuring Corporate Standards in Pipelines With Custom Marker Files

Pipeline as Code has revolutionized how Continuous Delivery pipelines are defined in Jenkins. Custom Marker Files address some of the concerns that come along with it.

· DevOps Zone ·
Free Resource

Is the concept of adopting a continuous everything model a daunting task for your fast moving business? Read this whitepaper to break down and understand one of the key pillars of this model in Continuous Governance: The Guardrails for Continuous Everything.

Pipeline as Code revolutionized how Continuous Delivery pipelines are defined in Jenkins by checking in the Pipeline as a Jenkinsfile in your repository instead of storing the definition locally in Jenkins. This becomes especially useful when leveraging the direct integrations Jenkins has with Github and Bitbucket. In this case, Jenkins will scan the entire organization for repositories containing Jenkinsfiles and then create the associated pipelines in Jenkins automatically.

The Jenkinsfile approach is great for many use cases, but many times, larger organizations would like to use Pipeline as Code while still setting certain standards to be used throughout the organization. Pipeline as Code allows free reign to developers to create their own Jenkinsfiles without any regard to corporate standards or practices. This may be fine with a small implementation, but as the as the number of projects and repos increases, there is a higher chance of teams deviating from the best practices developed by the shared services team. Additionally, there are many instances in which you may just want to ensure that certain commands run before or after the general pipeline, such as requiring a cleanup to occur after any build runs in the organization. The Custom Marker File feature in the CloudBees Jenkins Platform was created to address some of these concerns.

As blogged about earlier, Custom Marker Files allow you to associate repositories in your SCM that have a given identifier with a generic pipeline. For example, instead of having the same Jenkinsfile defined in every Java project, Custom Marker Files will instead allow you to define one pipeline that should be used by all repositories that have a pom.xml file (with pom.xml being the identifier that lets you know that this is a Java project). This functions as an easy way to get new teams onboarded because once they create a new Java Project in GitHub or Bitbucket, Jenkins will start building the project without the need to create a new Jenkinsfile.

In many cases, fully templatizing the Jenkinsfile for the entire organization as described above may be too restrictive, which is why Custom Marker Files also allow you to set standards while still giving teams the flexibility to create their own Jenkinsfiles. Let’s go through an example of how this would work in a GitHub organization (it works with Bitbucket teams, as well). Make sure to have updated versions of the CloudBees Pipeline: Templates plugin.

  • Click New Item > GitHub Organization and give it a name to create a new GitHub organization.
  • In the Configuration page, enter in the GitHub Organization and Scan Credentials.
  • In the Project Recognizers section, delete the option Pipeline Jenkinsfile and click Add > Custom Script.
  • Enter in the information as shown below:
  • Click Save to start scanning your organization

Notice that in this case, the Marker File is set to a Jenkinsfile. This means that any repository or branch in the organization with a Jenkinsfile will be detected. The Pipeline section then defines the pipeline that will be used for all projects containing the Jenkinsfile Marker File. Let’s analyze this Pipeline script.

  • Lines 1-3 prepend a preflight check stage that allows custom actions such as setting up the build environment before the main Jenkinsfile pipeline.
  • Line 5 uses the readTrusted function to read the Jenkinsfile file into a variable. readTrusted allows you to read files from the project that is being checked out without the requirement of being on a node.
  • Line 6 then actually runs the Jenkinsfile pipeline by using the evaluate function. evaluate is similar to load but does not require to be run on a node.
  • Lines 8-10 appends a postflight cleanup stage in which cleanup commands can be run after the pipeline has completed.

Looking at the Stage View above, you can see that the main pipeline ran with the preflight checks stage prepended and the postflight cleanup stage appended.

This method of reading in the Jenkinsfile for Custom Marker Files can be useful for requiring additional actions to be run across the organization. The example in this blog is very simple, but this can also be a good starting place in the future to analyze the Jenkinsfile that developers create and to ensure that it meets certain standards. An example of this would be to read the Jenkinsfile in a variable (look at Line 5) and parse that variable to make sure that certain functions are used or that it adheres to a certain schema. With all of these different options, Custom Marker Files in the CloudBees Jenkins Platform help you strike a balance between standardizing and providing adequate freedom for your developers.

Are you looking for greater insight into your software development value stream? Check out this whitepaper: DevOps Performance: The Importance of Measuring Throughput and Stability to see how CloudBees DevOptics can give you the visibility to improve your continuous delivery process.

jenkinsfile ,devops ,pipelines ,custom marker files

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}