Building a Culture of Testing

// this post by tim rosenblatt originally appeared on the ship.io blog.

i love this blog post from lou kosak on the testing culture at airbnb. it’s not so much a description of what they do now, but more a story of how they built testing into their culture. it’s a story full of hard work and grit.

talking about culture is very popular these days. in my opinion, the most important part of a company culture is the ability to find and stick with the important parts, while changing others. sticking with an existing habit is easy — it’s the change that is usually hard. we know from neuroscience that our brains like to establish routine habits and then put them on autopilot. groups of people can also resist change in interesting ways. it’s not that our brains are bad, or that groups of people are bad. this is simply how these things work.this is why i love this post so much. it is a powerful mindset to believe that improvement is possible but difficult, and to push forward anyways.

back in 2013, airbnb basically had no testing and a giant monolithic app that wasn’t going to make it easy to add tests for:

“the suite was slow and error-prone, our ci server was barely limping along, and most people had no idea how to run tests locally. new hires were told that testing was important, but when they saw that nobody was paying any attention to tests they quickly forgot about them.”

i definitely remember when i started writing automated tests (a very, very long time ago). the thing about testing is that it is a compound habit — it’s not a single thing that you do. it’s a combination of 10 or 20 things you do that build up to what we know as “testing”. bad habits in one area make it impossible to develop good habits in other areas, but good habits build upon each other. before i had this realization, i felt stuck, and i felt like i was constantly fighting the testing tools and the process. lou’s comment about new hires dropping the testing habit reminds me of this complexity.

on a personal note, this is something that i like about ship.io. we are building a ci system, and the team has an incredible background, having produced other ci, build, and deployment systems. when we talk about the importance of all of these habits, the team understands them intuitively and it helps us continually grow. plus, helping other developers run their tests is a great feeling. it makes life wonderful!

another aspect of this is when lou talks about how they started using pull requests — a change to the team’s core habits. they used to merge everything into master, but mature projects cannot be run this way. his explanation of how they changed this, by patiently being good role models, is a textbook example for anyone trying to improve habits:

“eventually, a few people decided to do something about it and started submitting pull requests for their changes. this was never introduced as a mandatory policy; we never disabled pushing to master or shamed people for doing so. but as those few, then a team, then several teams started doing this…it became clear that this process of peer review lead to less bad code hitting production, and therefore fewer outages.”

these are just a few of the highlights from a really thorough write up. give it a read !