In less than 15 years, Agile development has grown from a little-known alternative to traditional or "Waterfall" project management, into the de facto framework for managing the process of delivering software products.
Despite its rise in popularity, discussions about whether Agile, Waterfall, or even a blended approach was the best way to manage a new project remained common until fairly recently.
According to surveys conducted by Forrester, among others, only about 10 percent of respondents are using pure traditional development methodologies. This confirms that Waterfall is drying up as Agile continues to expand.
Is the "Waterfall vs Agile" debate still worth having, or is Agile clearly the best approach under every circumstance? We thought we would take a look.
In use for over four decades, Waterfall methodology is a sequential development process and generally relies on the following stages:
Once each stage is successfully completed, developers move on to the next stage. Since the process is sequential, a developer can't go back to the previous stage without having to start the entire project from the beginning. Because there is no room for error in the Waterfall approach, developers must start at the beginning and carry through until they finish all of the stages.
Despite its lack of efficiency, Waterfall's focus on sequential planning and documentation has a few potential advantages:
- Meticulous records are created and updated at each stage of the development process, making it easier for subsequent stages to understand the whole application because there is no guesswork about what was done prior. At each stage, the project is "baked" complete before the next stage starts.
- Clients and stakeholders like to know upfront what they're getting at the end of an engagement. Because detailed documentation of requirements, specifications, and project planning form the underpinnings of Waterfall, they start the project with a concrete idea of the cost, scope, and timeline for the entire project (though many would argue that they're getting a false sense of security, because everything goes out the window as soon as the product's goals or anticipated outcomes are altered).
While Waterfall methodology has its benefits, its inherent lack of flexibility poses a number of substantial risks.
- Because the entire project is built on the initial requirements, mistakes in requirements gathering or documentation can throw the process into disarray and cause painful rework if not found until later stages like testing and implementation.
- Bugs aren't discovered until very late in the process because quality assurance testing comes at the end of the project. Since everything is sequential, you might reach what appears to be the end only to start the process again.
- It doesn't accommodate the client or stakeholder's evolving needs, making new product development a challenge when needs are not fully known and change in requirements are inevitable.
Despite the challenges, there are instances in which Waterfall is appropriate, such as in cases when there is a clear picture of the final product, when clients can't change the scope once it starts, and when certainty is a higher priority than speed or flexibility. Examples, where Waterfall may still be appropriate, are mission critical redundant systems such as space and military systems.
As its name suggests, Agile is all about flexibility. What makes this so important is that software engineering isn't a particularly predictable or repeatable process, like building physical objects such as houses or machinery, which is what traditional project management was designed for.
Agile's hallmarks are a high degree of collaboration between client/stakeholder and the project team, iterative design and dividing work into small units that are tested throughout the process. Because of these structures, bugs are discovered and corrected earlier, feedback is gathered and incorporated more frequently, and - perhaps most importantly - the product can evolve organically over the entire product development process. In addition:
- Changes can be made throughout the project without significantly disrupting the process.
- The client or stakeholder can change their mind based on changes in the marketplace or other situational factors.
- Developers can respond to changes or improvements in available technology.
- Project priorities are evaluated throughout, thus reducing the risk of ending up with an irrelevant product at the end.
- Working in iterative cycles conditions the team for continuing to improve the product after its initial release.
- The high degree of structure inherent to Waterfall project management provides a narrow set of "guard rails" for less experienced project managers to operate within, whereas Agile processes can quickly devolve into "no process" when managed by someone who is new to the methodology.
- Clients or stakeholders expecting a finished product that adheres precisely to a predetermined and highly detailed set of specifications may be disappointed.
- Agile isn't well suited for projects that require a lot of detailed documentation up front.
Four Scenarios for how Using an Agile Approach Allowed for Even Better Products for our Clients
Scenario One: Creating the Right Hardware Prototype
For a recent hardware project, our team and the client shifted the goal/deliverables at least three times during a five-week timeline and having an Agile workflow allowed for this. At each iteration of the prototype, the hardware team discussed the outcome with the client and pivoted to build something slightly different based on their findings. This resulted in building the best hardware prototype that met both user/client needs and technology requirements.
Scenario Two: Shifting Focus to a Better Audience
During a client engagement, we identified a better growth opportunity if their business strategy, and the platform we were building, pivoted from a B2C to B2B focus.
By developing our software in an Agile process this allowed us to regularly review our priorities and work, and adapt where appropriate. Initially, our focus was on solving their immediate consumer needs - online acquisition. During the project, the needs changed focus to administrative/point of sale requirements for location owners that would use this platform. This generated more customers faster, with higher traffic, and got our client to more locations quicker.
The complexity of the system being built made it critical that the Dialexa design practice also had an Agile approach to be hands on throughout the development of the project - rather than a one and done approach at the start of the engagement.
Scenario Three: Using an Agile Approach to Manage Support Needs
For one of our clients, where we have developed a new internal platform to drive operational efficiencies for a core activity, we are operating our Support efforts using an Agile model. This means that through an active communication and prioritization framework with the client, the team can be focused on building, training, features, or addressing bugs - whatever is more important at the time to ensure the platform is operating and evolving as needed.
Scenario Four: Freedom to Change the Program With Ease
The team started their work with a plan for an Agile team where the focus was on the research and design at first and then there would be a shift to development after 25% of the time has passed. What ended up happening is that the team was required to work on the design for almost 75% of the planned time to address all the client’s user research and changing priorities/requirements, and the remaining 25% of the timeline was development. This change in the program ensured both the client and our team were confident that what we were building in the remaining time accurately addresses the client and user needs.
There are a few important reasons that we don't hear the "Waterfall vs Agile" debate as much as we used to:
- For the reasons mentioned above, Agile simply works better for the vast majority of software development projects.
- Because it works better, the number of experienced Agile practitioners is constantly increasing.
- The pace and volume of innovation continue to increase, thus reinforcing the need for a methodology that is able to tolerate change while the process is running.
Does this mean that the question of "Waterfall vs Agile" isn't worth asking? Not necessarily. While it's increasingly a forgone conclusion, there will always be edge cases that require a more structured process, and project managers would do well not to ignore the possibility that Waterfall, or even an approach that combines Agile and traditional project management methodologies, is the right solution.
We love Agile and use it almost exclusively with our customer and internal projects.