How to Build a QA Strategy Like Spotify

The QA strategy (or lack thereof) that works well when your team is first starting out probably wasn't necessarily sophisticated. But teams that go through periods of growth often discover the hard way that QA that is "good enough" for a tiny start doesn't hold steady in the long run.

Spotify is a great example of how to scale development and QA practices successfully. Since their launch in 2008, Spotify has grown from 150 employees to over 2000. How do they keep product quality high for over 157 million users? Here are a few QA lessons shared by Spotify that growing development teams can model their own testing process after.

Prioritize Long-Term Reliability and Quality

One of the biggest hurdles faced by growing teams like Spotify is maintaining quality and coverage over time. As organizations gain momentum, opting for quick fixes and low-hanging fruit can easily become the norm. But to scale effectively, teams must prioritize development and testing choices that will provide the best long-term gains. At Spotify, developers work with the product team to prioritize testability and stability in the product over time.

"If you have a solid system to test, you need to add a lot of testability; I talk to the product owners saying that we need to add this to give you what you want. I know you didn't order this, and this will take us ten days extra, but this is what we have to do to have a product which we can have high reliability in the future." - Kristian Karl, Test and Development Manager

Structure Your QA Team to Align With Your Product Goals

Fast-moving organizations know that siloed QA teams don't scale. Spotify's development organization is composed of squads and tribes arranged around focus areas. This structure helps individuals collaborate across different functional areas more effectively, and ensures that the goals of the QA, product and development teams stay aligned.

In addition to this more tribe-squad team structure, Spotify stresses that the role of QA isn't to block releases or act as a gatekeeper, but to collaborate with developers and product teams to continually improve the product.

Leaving that role behind can be harder to do than it might sound like. It takes some effort as a QA to not have full control of what goes out to the end-user, but to try to have that would be detrimental to true engagement from the others in the dev-team, and also slow all of you down immensely. - Olof Svedström, QA Chapter Lead

Use Automation to Accelerate QA, Not Replace It

Testing automation is a key component of Spotify's ability to scale development. However, it isn't used as a panacea for every quality woe. Instead, Spotify's QA team uses testing automation as a tool to help their QA engineers be more effective, and focus more of their energy on their overall product quality goals.

"I would rather say that you can never replace humans with machines for testing. We can lose 20% of the testers that don't apply. The number of tests that you can do on this system is infinite. If I can get help with automation, I can focus on further, deeper and broader tests. That's what I want to achieve and get from automation." - Kristian Karl, Test and Development Manager

Learn More About How Enterprise Teams Build a Scalable QA Strategy

We recently took a look under the hood at a few big-name enterprise organizations with famously lean QA processes. In our guide, Agile QA at Scale, we dig into how companies like Spotify, Facebook and Atlassian have built QA strategies that scale, and the lessons that growing teams can learn from how they approach quality assurance.

Download the guide now for more insight into what it takes to build a software testing strategy that will scale with your organization.