Which CI is Best For My Team?
Which CI is Best For My Team?
This article lists some considerations developers should have when looking for a continuous integration tool, as well as some tool pros and cons.
Join the DZone community and get the full member experience.Join For Free
At the beginning of developing and building a new application or product, before writing any business logic, it's necessary to make some difficult decisions. These decisions can include which cloud hosting platform to use, which continuous integration (CI) tool to use, and various development configuration choices. These decisions can dictate the speed and success of your engineering efforts. Therefore, choosing the right CI tool can make or break the velocity and quality of a team's code production.
I'm a former software engineering instructor at Hack Reactor @ Galvanize, and currently a software engineer at Forge. I have experience teaching and using most of the options detailed below. Like most engineering problems, a discussion of the trade-offs and constraints as well as an understanding of your team’s specific requirements can help you to pick the best CI tool for you.
In this article, I will discuss how to choose the best CI for your team.
You may alsod enjoy: Choosing the Right Continuous Integration Tool
Trade-offs and Constraints
Before I discuss the details of specific CI tools, here are some dimensions to help you to make an informed decision for your team. Some of these considerations may be very important to you and your team; others may not be important at all.
Does the tool/service offer parallelization? How does cost scale with speed? I expect most developers would agree that the speed of the test and iteration cycle is one of the most important considerations when evaluating a CI tool.
An important feature is test/prod parity. Does the particular tool offer identical staging and production environments to ensure parity?
Ease of setup and configuration is a commonly requested feature of CI tooling. All the amazing features are great, but they are useless if they are locked behind a wall of incomplete or confusing documentation and complex configuration.
As per the point above, usability is another important aspect of the product. Will new developers on your team be able to learn and interact with your CI system? How steep is the learning curve?
Lower is better assuming feature parity. Some tools are prohibitively expensive, especially for teams working on open-source projects. Other tools are free or available at a reduced cost for these teams.
How easy is it to contact a support representative? Some teams might find this invaluable while others may not. What is the quality of their documentation? Do they provide examples of common applications and operations? Are those examples available in multiple languages?
How widely used is the tool? Does it have a vibrant community of fellow developers trying to solve similar problems with the tool?
Does it play nice with Docker, AWS or GCP services? In a modern dockerized development world, container support is vital for most teams. Does the service integrate with GitHub nicely?
Are fragile and complex hacks required to make your multi-faceted app behave and test as expected? How easy is it to attach dependent services?
CI Tool Options
Here are some CI tool options that are currently leading the market, as well as some pros and cons of each.
- Capabilities to construct complicated sequential build processes to handle complex use cases.
- Large community.
- Online examples and troubleshooting available.
- Github integration.
- Easy setup.
By using Circle CI, users can add an image that is in a predetermined state for testing via containerization.
- Poor UI/UX — Circle CI has a number of rough edges. For example, Circle CI does not automatically refresh the workflow page when a task in the pipeline completes or fails. There are also a number of other small workflow issues that could be improved.
- Downtime/denial of service — I've personally experienced some very inconvenient service outages. While looking at the up-time records for the last year, I'm sure others have as well.
- Aggressive GitHub permissions — This is cited by Circle CI users as one of the major cons of using the product.
- Free for open-source projects.
- CI environment provides multiple run-times, data stores, and so on.
- Hosting projects on Travis CI means you can effortlessly test your library or applications against multiple run-times and data stores without installing them locally.
- You get what you pay for — This is a great tool for hobby projects and small open-source teams but may fail to support the complex needs of a larger engineering team or project.
- You have to pay to add Travis CI to a private Github repository.
- Configuration for complex testing suites and workflows may be challenging.
- CodePipeline offers amazing integrations with existing AWS products with plugins similar to AWS Lambda development.
- No servers to provision or set up.
- Offers a fully-managed continuous delivery service that connects to your existing tools and systems.
- Parallel execution.
- Simple to set up and relatively easy to use.
- AWS IAM access controls and permissions in a large and dynamic team.
- Enormous array of features and customized workflows to fit virtually any development team.
- Option to pay by usage.
- No upfront fees.
- Cost — AWS services can be expensive.
- Platform lock-in.
- Lack of integration with source control providers other than GitHub or AWS CodeCommit.
- Separation of concerns and easy debugging — GitHub actions are just consecutive Docker runs. This makes it easy to reason about and debug.
- The individual actions in a workflow are isolated by default, making it much easier to reason about and debug.
- Vibrant developer community available as a resource.
- GitHub Actions are a more generalized tool — they do many things other than run CI.
- Environmental configurations for testing staging/DB and prod/DB parity is challenging
- Due to security reasons there is no support for pull requests from forks. If you're using a workflow that involves forks, that makes GitHub actions largely unusable as CI/CD tool.
- Limited quality and breadth of published GitHub actions.
- Less actionable documentation with limited real-world examples. This is a subjective claim but supported by others in the developer community. Check for yourself by comparing GitHub Actions documentation with the more detailed documentation provided by Heroku or AWS.
- Amazing documentation and examples
- Natural integration with Heroku hosted apps and a Heroku Pipeline
- Consolidated Heroku billing
- Great UI and developer experience
- Lower level of complexity for attaching dependent services, assuming the dependency is in Heroku's Add-On catalog (e.g. Postgres) compared to the roll-your-own mentality of Circle CI
- No waiting in the queue
- Best-in-class pipeline integration
- It is not possible to use Heroku CI to test container builds.
- Cannot use Heroku CI without using Heroku Pipelines or dynos for hosting
- Heroku CI costs $10/month
Which is the Best CI Tool For You?
I've discussed some trade-offs and constraints as well as some key details specific to common continuous integration services that engineering teams should consider before selecting a CI tool.
My current favorite is Heroku CI because I’m already using the Heroku universe (hosting, Heroku Pipelines) at my firm. I like Heroku CI because it runs tests on Performance Dynos in a disposable Heroku app on each merge to master. So, I don't have to wait for other developers' apps to build or tests to run. It also seamlessly integrates with Github and Heroku Pipelines. Generally speaking, Heroku CI is a real pleasure to work with.
If you aren't using Heroku, then my second favorite is Circle CI. Despite some UI/UX and quality of life issues, it is versatile, quick to set up, and cloud-platform agnostic. You can configure complex workflows with environmental set-up, integration testing, tear-down, and parallelized tasks with plenty of detailed tutorials and a large community of users.
As an engineering team lead, it's important for you to make an informed decision based on your team’s constraints and requirements, knowledge of the market, and trade-offs to pick the CI tool that is right for you.
Selecting Your Local Continuous Integration Tool
What CI/CD Tool Should I Use?
Published at DZone with permission of Nicholas Vincent-Hill . See the original article here.
Opinions expressed by DZone contributors are their own.