What is Continuous Integration? 11 Key Practices and Principles

Continuous Integration, like DevOps, comes from Agile and relies on repeatable steps for quick development.

If you have multiple developers working on the same project, then you have one central problem:

Integration.

All those lines of code that one developer wrote will have to be checked and combined with the lines of code a different developer wrote, and then they’ll have to be tested together to make sure it all works properly.

The question is, how often should you perform integrations?

In the early days of software development, integrations might’ve been performed after weeks or months of coding. But with that much time in between writing the code and combining it, you’ll most likely have more than a few bugs and errors that will take a lot of time to fix. Which is why Continuous Integration (CI) was created.

Why wait to integrate different strings of code when you can do it often and all the time? That’s the question CI answers. Let’s take a look at what Continuous Integration is and the practices and principles that form its foundation.

What is Continuous Integration?

According to Aaron Cois, CI is a technique designed to continually merge “source code updates from all developers on a team into a shared mainline. This continual merging prevents a developer’s local copy of a software project from drifting too far afield as new code is added by others, avoiding catastrophic merge conflicts.”

It’s common practice today for integrations to happen at least once per day, if not more often. Continuous Integration was coined by Grady Booch and then picked up and expanded by Kent Beck and Extreme Programming (XP).

CI has become common practice in DevOps and Agile culture, and for very good reason: CI helps you detect and locate errors before they grow into major problems. But CI isn’t the only tool for keeping your code bug-free throughout the development cycle.

11 Continuous Integration Practices and Principles

Make sure to check off all 11 Continuous Integration practices and principles for total implementation.

The following practices come from Continuous Integration and Extreme Programming advocate, Martin Fowler. They describe different steps and strategies for making CI practical and straightforward.

1. Maintain a Single Source Repository

Every software project involves many different files that need to be integrated together to construct a final product. To manage all that data, you need a Source Management Tool (SCM), like Github. Then all of your code can go into one central repository.

This includes:

• Test scripts.
• Properties files.
• Database schema.
• Install scripts.
• Third-party libraries.

You can even put in IDE configurations to make it easier for people to share IDE setups.

The key features of SCM’s are allowing you to create multiple branches and manage various streams of development using a mainline — a single branch of the project under development. Continuous Integration is the process of combining separate pieces of code into the mainline and testing it to ensure everything works together.

2. Automate the Build

Automating the build means turning the sources you have into a running system — automatically. With the tools available today, you shouldn’t ever have to do it manually. Besides, telling people to type commands and click around different dialog boxes will inevitably produce more mistakes that you’re trying to avoid in the first place.

Whatever tool you use, it should analyze what needs to be changed as part of the process. Jenkins, for example, is one of the most popular Continuous Integration and build automation tools available to DevOps teams. 

3. Make Your Build Self-Testing

If you want to catch bugs quickly and often, then you should include automated tests in the build process.

While not perfect, self-testing code saves you a ton of time and has become very popular among Extreme Programming and Test Driven Development (TDD) enthusiasts.

You want to be able to run a series of automated tests on your base of code that shows it is free of any major defects. In a way, you can’t practice Continuous Integration without automated testing. And while XP and TDD are popular methods for producing self-testing code, you don’t have to use them to create automated tests.

As long as tests can be executed from a single command and check a large part of the code - causing the build to fail if the test fails - you’re good to go.

One of the most popular automated testing tools is Unified Functional Testing (UFT). It comes loaded with API and web services testing, continuous testing, mobile testing, and a whole lot more.The only caveat to automated testing is, it doesn’t catch everything.

That’s why to make Continuous Integration as successful as possible, you should rely heavily on automated testing, but make room for manual testing, too.

4. Everyone Commits To the Mainline Every Day

The practice of committing to the mainline is what most people think of when discussing Continuous Integration. And they should. It’s the key to the whole process.

Daily integration allows developers to alert other developers about changes they’ve made, and it’s this frequent communication that allows changes and developments to occur rapidly without too many errors.

Typically, this practice works in the following way:

• A developer updates their working copy to match the mainline.
• They resolve conflicts with the mainline (if applicable).
• They build on their local machine.
• They test their build.
• If they pass, they commit it to the mainline.

Doing this often, sometimes multiple times a day, reveals any conflicts between developers, and if a conflict is found, it can be remedied swiftly, that means errors can be found and fixed within a few hours of them springing up.

It’s when errors remain in the mainline for weeks or months that a software project stretches out for many more months or years than it should.

Bottom line: the more frequently developers commit to the mainline, the less conflicting errors you’ll find and the faster you can fix any that you have.

5. Every Commit Should Build the Mainline on an Integration Machine

By using an integration machine, developers can ensure that their builds were successful and that it’s ready to commit to the mainline.

There are 2 ways to accomplish this:

1. Manual build.
2. Continuous Integration server.

A manual build is similar to a local build that you would perform before a commit to the repository. You head over to the integration machine, check the head of the mainline, and begin the integration build. If the build succeeds, it’s ready for a commit. A continuous integration server (like Jenkins) will monitor the repository for you.

When a commit against the repository is complete, the server automatically initiates a build and notifies the “committer” of the results.

6. Fix Broken Builds Immediately

If the mainline build fails, it needs to be fixed immediately. That’s the point of CI — to always be developing on a stable base. The reality is, the mainline build will break from time to time. Not a big deal.

However, it shouldn’t happen all the time. If it is, that means other developers aren’t building locally before performing a commit. At all times, fixing the build should be the top priority for every team member. One way to fix a build fast is to revert back to the latest commit from the mainline. Then debug the problem on a development workstation.

7. Keep the Build Fast

Rapid feedback is the hallmark of Continuous Integration. No build should take a long time if you’re following CI practices and principles. For example, Extreme Programming uses a 10-minute build.

The faster the build, the faster you can detect bugs, and the faster you can integrate your build into the mainline.

8. Test in a Clone of the Production Environment

Every test you perform should be in an environment that virtually matches the original production environment.

If there are differences in the environment, you’ll get different and unreliable results. That is, every difference presents a potential risk of getting a result that you won’t get in actual production.

That’s why it’s critical the test environment mirrors the production environment, including but not limited to:

• The database software.
• The operating system.
• The libraries.
• The IP addresses and ports.
• The hardware.
• Etc.

With that said, there are obvious limits.

You can’t possibly run a test for every variation of an end user’s desktop software, making sure your software perfectly interacts with every single other piece of software out there. Nonetheless, the goal is to replicate the environment as perfectly as possible.

9. Make It Easy for Anyone to Get the Latest Executable

One of the major reasons why Continuous Integration was created was because it used to take so long to build the right software.

With old school methods like Waterfall, every specification had to be considered and documented in advance. And if the software wasn’t built to spec, it was difficult to go back and make sweeping changes.

Continuous Integration leverages a natural human trait: the ability to see that’s something isn’t right and needs changing. We’re much better at that than dreaming up all possible functions and designs ahead of time. That’s why CI incorporates constant commits to the mainline.

Every developer should be able to run the latest executable for demonstrations, exploratory testing, or anything else.

10. Everyone Can See What's Happening

A big reason why CI works so well is because it relies on constant communication. The way to ensure consistent communication between developers and product owners is allowing everyone to see the state of the system and the changes made to it. In other words, letting anyone see the mainline build.

Most CI software will allow you to see build progress and the state of the mainline build. But some developers like to get creative and hook up a continuous display to the build system. This can take the form of:

• Lights that glow green when the build works and red when it fails.
• Red and green lava lamps, where bubbles in the red lamp indicate the build has been broken for too long.
• Dancing rabbits.
• A ringing bell.
• And so on.

You should also ensure you can see who is currently building and what changes they’ve made, along with a history of changes so everyone on the team knows where the project is and where it’s going.

11. Automate Deployment

Multiple environments are required in Continuous Integration for running commit tests and secondary tests. Plus, you’re moving executables between these environments often, sometimes multiple times a day. This should be automated.

What you’ll need are scripts that deploy the application into the environment of your choice seamlessly. Automatic deployment is cost-effective, reduces errors, and speeds up the process tremendously. In addition to automated deployment, you also need automated rollback. If bad things happen (and they do), you want to be able to quickly revert back to the last known stable state. The added benefit of being able to rollback a bad build is it encourages developers to deploy more frequently.