Continuous integration - some tips
A continuous integration server is a must for every non-basic project. Here are some of my tips for working with CI servers.
IRC, email, campfire - use what fits best for you. Ideally, the
notification should come with some info about the code changes that
triggered the build. We use IRC - we've got an IRC channel for every project we work on, so the notifications go to the correct channel.
It's a huge topic. It's so difficult to keep the build time short for
every project. I have to admit we've got projects where build time takes
20 minutes. My goal is to get down to below 10 minutes for existing
projects and try to keep the new projects below 3 minutes. Slow builds
have a very bad impact on so many things. The feedback is late. People
tend not to run builds on local machines.
It's great that you have a CI server, but the goal is also that all
tests can be run on every developer machine in a decent time.
That's the most surprising effect of using tests and CI. Once we got to
the point where many of our projects have the slow build problem we
investigated our technology choices. In many cases the slowness came
from using Selenium. Selenium is the only "right" tool when you have an
app that is a mix of server-side logic and client-side code (JS). It seems, surprisingly, that many of our projects could be turned into one-page CoffeeScript apps.
Obviously it's not an easy goal (and not always possible). It's
changing the way we architecture our apps. If the split is achieved you
can test the CS code in isolation from the server-side and then test the
server-side API, also in separation, but without Selenium.
Selenium is cool, but it has the drawbacks of being slow and unreliable.
If you're using Selenium, then you already know what I mean. I hate this fact but sometimes we've got build failures which shouldn't appear, they're most of the time timing issues. It's a hard situation to have. The important thing to remember is that we (the team) are all in it. Every team member should investigate the problem and try to fix them - sometimes adding a sleep (a hack, I know) solves the problem.
Who's responsible for failures?
I like the idea that the last person who triggered a failing build is
responsible for fixing it. It's hard to get to this point, especially if
your build is unreliable. This leads to:
Never leave a broken build
This is the biggest sin when working with CI servers. If I want to start coding and I see that a build is broken and no one seem to care then it's very depressing to me. In practice, it means that you should commit your changes some time before you finish your work, so that the CI can run it and get you the result back. If it's broken, try to fix that as much as you can. It's less of a sin when the failure comes from unreliability, but leaving a broken build caused by your changes is unforgettable :)
A green, fast build run frequently on the CI server is a sign of a
healthy project. If there's anything broken with the build process, look
at it. It's often a smell showing you that something can be wrong in
your process. Don't ignore that.
Any tips from your experience?