DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Last call! Secure your stack and shape the future! Help dev teams across the globe navigate their software supply chain security challenges.

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workloads.

Releasing software shouldn't be stressful or risky. Learn how to leverage progressive delivery techniques to ensure safer deployments.

Avoid machine learning mistakes and boost model performance! Discover key ML patterns, anti-patterns, data strategies, and more.

Related

  • A Concise Guide to DevSecOps and Their Importance in CI/CD Pipeline
  • Implementing CI/CD Pipelines With Jenkins and Docker
  • Zero to Hero on Kubernetes With Devtron
  • 7 Reasons Why Companies Should Apply DevOps and CI/CD Practices to Their Data Pipelines

Trending

  • DZone's Article Submission Guidelines
  • A Complete Guide to Modern AI Developer Tools
  • Is Agile Right for Every Project? When To Use It and When To Avoid It
  • Automating Data Pipelines: Generating PySpark and SQL Jobs With LLMs in Cloudera
  1. DZone
  2. Testing, Deployment, and Maintenance
  3. Deployment
  4. Five Best Practices for GoLang CI/CD

Five Best Practices for GoLang CI/CD

Create CI/CD workflows with Artifactory and GoLang.

By 
Ankush Chadha user avatar
Ankush Chadha
·
Aug. 06, 19 · Opinion
Likes (3)
Comment
Save
Tweet
Share
15.3K Views

Join the DZone community and get the full member experience.

Join For Free

±For developers programming in long-established languages like Java, JavaScript, or Python, the best way to build continuous integration and delivery (CI/CD) workflows with Artifactory is pretty familiar. A mature set of dependency management systems for those languages and container solutions, such as Docker, provide a clear roadmap.

But if you’re programming your applications in GoLang, how hard is it to practice CI/CD with the same kind of efficiency?

As it turns out, it’s gotten easier, especially with some of the latest innovations in Go. 

Best Practices

At JFrog, we’re big fans of GoLang, as we use it as the language for several of our flagship solutions. We practice what we promote, too, using Artifactory at the heart of our CI/CD. Here are some of the practices we can recommend:

1. Use Go Modules

Unlike many established programming languages, initial releases of GoLang didn’t provide a common mechanism for managing versioned dependencies. Instead, the Go team encouraged others to develop add-on tools for Go package versioning.

That changed with the release of Go 1.11 in August 2018, with support for Go modules. Now, the native dependency management solution for GoLang, Go modules are collections of related Go packages that are versioned together as a single unit. This enables developers to share code without repeatedly downloading it. 

A Go module is defined by a go.mod file in the project’s root directory, which specifies the module name along with its module dependencies. The module dependency is represented by the module name and version number.

If you haven’t adopted Go modules yet, you’ll need to follow the steps below:

  1. Use go mod init to generate a go.mod file if previous package managers were used.
  2. Use go mod tidy if other package managers were not used. This command will generate a populated go.mod file.
  3. If the version is v2 and above, you will need to change the module name to add the corresponding suffix, update to the import path, add module aware static analysis tools, and finally update code generator files, such as .proto files to reflect the new import path.

For example, here is a go.mod file for a publicly available Go module for a structured logger:

module github.com/sirupsen/logrus
require (
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/konsorten/go-windows-terminal-sequences v1.0.1
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/stretchr/objx v0.1.1 // indirect
github.com/stretchr/testify v1.2.2
golang.org/x/sys v0.0.0-20190422165155-953cdadca894
)


Note that the version numbers must conform to semver convention (for example, v1.2.1 instead of 20190812, or 1.2.1) as required by the go command. You should avoid using pseudo versions like the one shown above (v0.0.0-yyyymmddhhmmss-abcdefabcdef) — although commit hash pseudo-version were introduced to bring Go Modules support to untagged projects, they should be only used as a fallback mechanism. If the dependencies you need have release tags, use those tags in your required statements.

In your own code, you can import this Go module along with other dependencies:

import (
"fmt"
"io/ioutil"
"net/http"
"os"
// Public Go Module for logging
log "github.com/sirupsen/logrus"
)


Then you can reference the module functions in your GoLang code:

 // Send text to the log
log.Printf("Hello Log!")


2. Use GOPROXY to Ensure Immutability and Availability

Once you maintain your GoLang dependencies as versioned modules, you can keep them as immutable entities by setting up a GOPROXY. In this way, you can always guarantee what a specific version of a module contains, so your builds are always repeatable.

Will you be using Go modules that are publicly available open-source packages? Ones you create and share with the OSS community? Modules you keep private to just your team? Here are ways to do each, or all at once, and ensure that those unchanging versioned modules are always available for your builds.

3. Use Artifactory Repository Layouts

When you build your app (using Go build), where will the binary artifacts generated be stored? Your build process can push those intermediate results to repositories in a binaries repository manager like Artifactory.

For these intermediate Go artifacts, you’ll use Artifactory’s generic repositories. Structuring those repositories can help you control the flow of your binaries through development, tests, and production with separate repositories for each of those stages. To help you do this, use the Custom Repository Layout feature of Artifactory with those generic repositories.

Create a custom layout that is similar to the following:

[org]/[name<.+>]/[module]-[arch<.+>]-[baseRev].[ext]

When you configure the custom layout, you should test the artifact path resolution to confirm how Artifactory will build module information from the path using the layout definitions.

Test path resolution

Test path resolution


4. Build Once and Promote

With your repositories properly set up for your Go builds, you can start to move them through your pipeline stages efficiently. 

Many software development procedures require a fresh complete or partial build at each staging transition of development, testing, and production. But as developers continue to change shared code, each new build introduces new uncertainties; you can’t be certain what’s in it. Even with safeguards to help assure deterministic builds that may still require repeating the same quality checks in each stage.  

Instead, build your Go-based microservices once, then promote them to the next stage once promotion criteria such as tests or scans are met. If you plan to containerize your Go microservice, the same principle applies: build each Docker image once and promote it through a series of staging repositories. In this way, you can guarantee that what was tested is exactly what is being released to production.

Build promotion


5. Avoid Monolithic Pipelines

Instead of a single, monolithic pipeline for your app, It’s better to have several, with each one building, testing, scanning, and promoting a different layer. This helps make your CI/CD process more flexible with different teams responsible for each layer, fostering a “fail fast” system that helps catch errors early.

For example, deploying a containerized application can typically be composed of five pipelines:

  1. Build the Go application using the JFrog CLI. This pipeline pulls the source code, builds the application with Artifactory repositories for dependencies and output binaries, and tests and promotes from a dev repo to a staging repository in Artifactory.
  2. Build a base layer of the containerized app, e.g. a Docker framework. Static or dynamic tags can be used based on company’s risk and upgrade policies. For example, If a dynamic tag such as alpine:3.10 is used as a base layer of Docker framework, then all patch updates will be included each time the Docker framework is built. This pipeline will include build, test and promote stages.
  3. Pull the promoted artifacts produced by the prior pipelines and build a containerized app.This will also have build, test, scan and promote stages. 
  4. Build a Helm chart that points to a statically tagged & promoted version of a containerized app that was produced by prior pipeline.
  5. Deploy the containerized Go app to Kubernetes using the Helm chart.

Artifactory acts as your “source of truth” for your Go builds, providing a GOPROXY for both public and private modules and storing compiled binaries. Using the JFrog CLI to build your Go app helps the build process interact with Artifactory, and capture the build info that makes your builds fully traceable. Here is a sample snippet:

// Configure Artifactory
jfrog rt c
// Configure the project's repositories
jfrog rt go-config
// Build the project with go and resolve the project dependencies
from Artifactory.
jfrog rt go build --build-name=my-build --build-number=1
// Publish the package we build to Artifactory.
jfrog rt gp go v1.0.0 --build-name=my-build --build-number=1
// Collect environment variables and add them to the build info.
jfrog rt bce my-build 1
// Publish the build info to Artifactory.
jfrog rt bp my-build 1

 

To explore this more, take a look at our example demonstration files for GoLang CI/CD  that we’ll be using at GopherCon.

Continuous Integration/Deployment Golang Repository (version control) Software development app Dependency Docker (software) Pipeline (software)

Published at DZone with permission of Ankush Chadha, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • A Concise Guide to DevSecOps and Their Importance in CI/CD Pipeline
  • Implementing CI/CD Pipelines With Jenkins and Docker
  • Zero to Hero on Kubernetes With Devtron
  • 7 Reasons Why Companies Should Apply DevOps and CI/CD Practices to Their Data Pipelines

Partner Resources

×

Comments

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends: