Why Is a Pull Versus a Push Pipeline Important?

With GitOps, automated delivery pipelines roll out changes to your infrastructure when changes are made by pull request to Git. But GitOps goes further than that and it also makes use of tools that compares the production state of your application with what’s under source control and alerts you if your running cluster doesn’t match your desired state.

Recently, there’s been much discussion around what the differences are between push and pull automated CICD pipelines, what we mean by that when we originally coined the concept and why it’s an important component of GitOps. In this post we shed some light on some common questions about GitOps and in particular we address how and why a Pull pipeline rather than a Push pipeline is necessary:

• Can I use a CI server to orchestrate convergence in the cluster?

• Should I abandon my CI tool?

• Why is a PULL vs a PUSH pipeline important?

• What is an Observed State?

Can I use a CI server to orchestrate convergence in the cluster?

Your CI server is not an orchestration tool. You need something that continually attempts to make progress (until there are no more diffs). CI fails when it encounters a difference. In combination with a human operator, a CI server can be made to force convergence, but this creates other issues. For example your CI scripts might not be able to enforce an idempotent and/or atomic group of changes.

Also, if something breaks (and it will), then you could end up being in a partial and unknown state that requires a full reboot to fix. It is easier to use dedicated orchestrators like Kubernetes and Terraform to provide convergence rather than relying on your CI tool to do that.

Should I abandon my CI tool?

No! Please keep on using your CI server to orchestrate the development of merging to mainline, building and testing.

But you shouldn’t use CI servers to do continuous delivery. A GitOps pipeline is an improvement on existing CICD pipelines where the CI engine (like Jenkins) is responsible for orchestrating updates but then may not complete correctly and simply fail. CI is used to merge and integrate updates with the trunk, and with GitOps we rely on Kubernetes or the cluster to internally manage deployments based on those trunk updates. We also call this the “pull” model for CD.

Why is a PULL vs a PUSH pipeline important?

Most CICD tools available today use a push-based model. A push-based pipeline means that code starts with the CI system and then continues its path through a series of encoded scripts to push changes to the Kubernetes cluster. The reason you don’t want to use your CI system as the basis for your deployments is because of the potential to expose credentials outside of your cluster. While it is possible to secure your CI scripts, you are still working outside the trust domain of your cluster which is not recommended.

In a pull pipeline, a Kubernetes Operator deploys new images from inside of the cluster (in our case, Weave Cloud provides one for you). The operator notices when a new image has been pushed to the registry. Convergence of the cluster state is then triggered and the new image is pulled from the registry, the manifest is automatically updated and the new image is deployed to the cluster. With a pipeline that pulls an image from the repository, your cluster credentials are not exposed outside of your production environment.

What is an Observed State?

We cannot say what actual state is. We can only observe it. This is why diffs are so important. Observability is an integral part of GitOps, where we alert on a divergence and then converge the "actual" system if it differs from "the source of truth."

How do you know whether or not a cluster update succeeded?

When you want to know whether a transactional update in Git has led to a correct update in the cluster:

• The cluster needs to be observed and compared with what’s in Git.

• If deployments are forthcoming - or a convergence-- you should be notified for success.

• On a divergence there should be an alert that triggers a convergence.

See GitOps Part 3 - Observability for more information.

Final Thoughts

Push vs. Pull is just one important piece to the GitOps pipeline. We’re not advocating that you don’t use a CI tool, or that you don’t incorporate automated tests or that you deploy straight to production from development. Pull refers to the Kubernetes Operator installed to the cluster that watches the image repository for new updates. Depending on the policy that you’ve set for your deployments, these updates get applied and depending on where the operator is installed, and on how you’ve organized your development pipelines, it could be to development, staging or QA.

For more answers to common questions about GitOps, please refer to the Official GitOps FAQ.