How the Guardian Uses Agile to Handle 977M Impressions a Month
A case study into how the Guardian's development team has adopted agile practices in order to handle enormous amounts of traffic.
Join the DZone community and get the full member experience.Join For Free
Thomas Kaliakos is a Senior Software Developer at the Guardian news sites. He wrote a great blog post which described how Agile is implemented at the Guardian, and how they face the challenge of handling 977 million page impressions in a month.
Stay Away From Deadlines
Although sometimes sticking to certain time windows is essential (especially when there is a third party involvement), according to Kaliakos, deadlines are not a usual phenomenon at the Guardian’s Platforms team. Additionally, estimations can be a painful concept for any developer, especially when they are sometimes (mis)treated as deadlines by product managers. Developing big software systems is in its essence a very complex problem. For each task we have to consider creating a system that can scale to large amounts of traffic and is able to handle large traffic spikes, whilst still providing a quality service. Investigations and exploratory tasks are also very common, whilst the work schedule itself is very dynamic, with support requests and urgent feature requests occurring quite often. All these points underline why estimations can be very difficult. Initially they were used, but were abandoned in the process as they weren’t very accurate and it was commonly believed that they didn’t bring much value to the process.
Kaliakos says that “It is worth mentioning the psychological factor too; it is much less stressful not to have to estimate the time for a task if you spend more time on it than your original estimate. Finally it’s worth noting that the fact that the Guardian, through its online version, offers a service instead of a product makes this feasible (as there is no date for the next version of the product to be shipped to its clients) and this could be considered one of the advantages of software as a service in general.”
Planning Your Planning
Having no estimates according to Kaliakos means that there are no burn down charts, no team velocity, no sprint planning with their typical meanings. During the sprint planning, milestones are set in the sense of what would be the desirable outcome at the end of the sprint. These milestones can, of course, change and adapt if in the process the goal post has moved. Setting an ultimate target and always seeing the bigger picture is what helps us understand if we are moving towards the right direction. “For that purpose,” says Kaliakos. “There are quarterly and mid-quarterly review meetings where we try to do that. In the quarterly sessions the team’s objectives are defined and clear targets are set. The focus of the mid-quarter sessions is to review the status and the progress against those objectives, as well as raise any issues or blockers that need attention. Metrics and KPIs are also valuable tools in our arsenal. For example, approximately one year ago, the Content API went through a big restructure. It was migrated from an in-house datacenter to AWS and the majority of the code was reimplemented. As a result of these changes the system’s stability decreased. Instead of taking the usual approach and making a list of features along with deadlines for their delivery dates, a general goal was set to increase the system’s stability and started working towards it, whilst metrics and KPIs (like uptime) were used to check we were moving in the right direction. After a short time the system became robust and resilient, with 99,99% uptime.”
Only When It Makes Sense
It is a very common pitfall for Scrum to be applied in a ritualistic form. Whenever this happens the true meaning of being agile is lost. At the Guardian, Kaliakos says they try to use all the considered agile best practices, but only in a way that makes sense in each specific case. Even the sprint’s duration is not fixed; it may change to one week for example, if it fits better the circumstances. In conclusion, says Kaliakos “we try to apply all the well-known agile methodologies and practices but every time through a critical perspective, adapting and adjusting them. After all, this could be a great mentality to have not only in a software development process, but in different contexts too, like life…”
Published at DZone with permission of Yaniv Yehuda, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.