Most Development teams now claim they are doing Agile, and are usually doing some variant of Scrum. In this blog post, I propose three reasons why teams can struggle with Scrum or any form Agile development. And, conveniently, they all begin with R.
In the pre-agile days, requirements were usually very detailed and well thought out. For example, when I worked at Ericsson, before we moved to the Agile methodology RUP, the projects were based on a classical Waterfall model that was a very high standard (usually around CMM level 3). The requirements came from System experts who had lots of experience in the field, were very technically skilled and had incredible domain knowledge. Their full time job was to tease things out knowing that they had effectively only one chance to get it right.
In the Agile world, with shorter iteration cycles and more releases, there are chances to make something right when you get it wrong. Functionality can be changed around much more easily. This is good! It is easier for customer collaboration, and thus enables more opportunities to tweak, fine tune, and get all stakeholders working together to sculpt the best solution.
However, because the penalty for getting requirements wrong is not as great as it was in the Waterfall model, it can mean that the level of detail in requirements can start becoming so insufficient that time starts getting wasted. The key is to get the balance right. Enough detail so that there is a clear agreement across the dev team, customer, and any other stakeholders about what is coming next, but not so much detail that people are just getting bogged down in paralysis analysis and forgetting that they are supposed to be shipping regularly.
I suggest one process for helping get your requirements quickly and efficiently in this blog post.
In Scrum, you are supposed to release at the end of every sprint. This means instead of doing 1 - 4 releases a year you will be doing closer to 20 if not more. If your release is painful, your team will struggle with any form of Agile. For example, say a full regression test, QA, bug fixing, build, deploy, etc, takes 10 days (including fixing any bugs found during smoke testing). It means that 20 * 10 = 200 man days are going to releases. Whereas in the old world, with 4 releases it would just be 4 * 10 = 40 days. In case it's not obvious, that's a little regressive.
Now, the simple math tells us that a team with long release cycle will struggle to release regularly, and will thus struggle with any Agile methodology.
To mitigate this risk, it is vital that the team has superb CI with very high code coverage and works towards a very strict CI culture. This includes:
- Ensuring developers are running full regression tests on feature branches before merging into dev branches to minimize broken builds on dev branches.
- Fix any broken build as a priority.
- No committing to a broken build.
- Tests are written at high quality — they need to be as maintainable as your source code.
- A coding culture where the code is written in a style so it is easily testable.
- CI needs to run fast. There's no point in having 10,000 tests if they take 18 hours to run. Run tests in parallel if you have to. If your project is so massive that it really does take 18 hours to run automated tests, you need to consider some decomposition, for example, moving to a microservices architecture where components are in smaller, more manageable parts that can be individually released and deployed in a lot less than 18 hours.
By mastering automated testing, the release cycle will be greatly shortened. The dev team should be aiming towards a continuous delivery model where in theory any check could be released if your CI system says it's green. Now, this all sounds simple, but it is not. In practice, you need skilled developers to write good code and good tests. But the better you are at it, the easier you will be able to release, and the easier you will become truly agile.
Note: One of the reasons why the microservices architectural style has become popular is because it offers an opportunity to make releases easier. That's true. However, most projects are not big enough or complex enough to need this approach. Don't just jump on a bandwagon — justify every major decision.
The most popular agile methodology Scrum only defines 3 Roles:
- Product Owner
- Scrum Master
- Dev Team
That's it. But wait a second! No Tech Lead, no Architect, no QA, no Dev manager? Surely you need some of these on a project.
Of course you do. But this is often forgotten. Scrum is a mechanism to help manage a project — it is not a mechanism to drive quality engineering, quality architecture, and minimize technical debt. Using Scrum is only one aspect of your process. While your Scrum Master might govern the process, they don't have to govern architecture or engineering. If all the techies are just doing their own stories trying to complete them before the next show and tell and no one is looking at the big picture, the project will quickly turn into an unmanageable ball of spaghetti.
This is a classic mistake at the beginning of a project. At the beginning, there is no debt. There are also no features, no bugs, and, of course, no code! But the key point is that there is no technical debt. Everyone starts firing away at stories and there is an illusion of rapid progress, but if no one is looking at the big picture, the architecture, the technical debt, and the patterns after a few happy sprints will very quickly explode. Meaning that all that super high productivity in the first few weeks of the progress will quickly disappear.
To mitigate this, I think someone technical has to back away from the coding and focus on the architecture, technical leadership, enforcing code quality, good design, and ensure that non-functional targets of the system are defined and met. It is not always a glamorous job, but if it isn't done, complexity will creep in and soon make everything from simple bug fixing, giving estimates, and delivering new features will be much harder than they should be.
In the Scrum world, it is a fallacy to think every technical person must be doing a story. It is the equivalent of saying that everyone building a house has to be laying a brick, and then wondering why the house is never finished because the bricks never seem to line up.
Someone has to stand back and ensure things are lining up correctly, that tech debt is kept at acceptable levels, and any technical risks are quickly mitigated.
Until the next time, take care of yourselves.