Agile Planning & Estimation
Planning and estimation are two of the main factors that can influence the outcome of any project. So with the wide adaption of agile methodologies in the IT world, questioning the effectiveness of the ordinary style of planning and estimation is starting to rise. Together we will try to explore the world of agile planning and estimation, to offer a better understanding of this issue and hopefully to be able to decide if it is the right thing to do for your organization or not.
1. Agile Planning & Estimation
The agile project management by default requires a specific kind of planning. Planning that is flexible enough to handle the nature of the agile environment. Due to its fluid nature, it might look like it's non-existent to the untrained eye. This is probably what led to the old myth of "There’s no planning in agile".
Agile planning like other types of planning needs estimating and measuring tools and methods to stay alive. However in spite of the fact that a lot of the general estimating and measuring rules and tools which are used in ordinary (none-agile) projects can be applied to agile projects (with some modifications to its style), its implications can vary and its implementations are different.
So how is the agile planning done? And how does it estimate assigned projects? There is no one answer for this, because although there are a lot of the shared ideas and concepts between the different agile methodologies, each one has its own distinct style of applying them. For this article I'll concentrate on the general ideas shared by most of the agile methodologies, like Scrum and Lean methodologies (If you are interested to learn more about these two methodologies check my article "Agile Methodologies - The Missing Path "³).
2. Explore Agile Planning & Estimation
2.1 Planning2.1.1 Why Agile Planning
As Mike Cohn defines it, "Agile planning balances the effort and investment in planning with the knowledge that we will revise the plan through the course of the project. An agile plan is one that we are not only willing, but also eager to change " This concept exists mainly to avoid the weakness of the ordinary (non-agile) planning. These weaknesses include¹:
- Concentrating on activities rather than delivered features
- Ignoring the prioritization
- Ignoring the existence of uncertainty
- Using estimations as commitments
Such weaknesses are what made the ordinary (non-agile) planning not able to keep up with the agile projects and their dynamic environments. For that the planning has to evolve to become agile as well.2.1.2 Agile Planning Levels
Agile planning needs to be done on 5 different and integrated levels of focus. They are Product Vision Planning, Product Roadmap Planning, Release Planning, Iteration Planning, and Daily Commitment Planning⁵. So let me explain each of them briefly.Product Vision Planning
The highest level of planning that concentrates on the future of the product and its ultimate goal with the possible changes to them and their likelihood. This creates the macro image of the product's world and its location in the universe of the stakeholders, allowing us to establish the general line of priorities and their estimations.
Plan shaping done by:
- 99% Product Owner & his team
- 1% Project Manager & his team
After knowing what the product world shall look like, the next step would be planning on partitioning this world into separate buildings and lining them on the road which we will follow to build the product's world. This allows us to estimate the releases with respect to the general functionalities and time needed.
Plan shaping done by:
- 70% Product Owner & his team
- 30% Project Manager & his team
After knowing all the buildings of the product's world, we start planning how each of these buildings will look like. Each of these buildings is a deliverable part of a value that is usable by the stakeholders. This allows us to estimate the iterations of each release and how they are related.
Plan shaping done by:
- 50% Product Owner & his team
- 50% Project Manager & his team
After deciding on how the assigned building in the product's world will look like, we start planning the required floors of each building and how each floor will look like from inside (for example, how many rooms and bathrooms). This allows us to be more specific about the assignment of the iterations with respect to the individual teams' abilities and experience.
Plan shaping done by:
- 30% Product Owner & his team
- 70% Project Manager & his team
So after knowing which part of the floor your team is building, we start planning on the steps that we will follow in finishing the assigned rooms (for example constructing the walls, windows, doors, etc). This will help us get more exact in our expectations and projections.
Plan shaping done by:
- 1% Product Owner & his team
- 99% Project Manager & his team
Although there is a tendency from a lot of the inexperienced agile methodology followers to ignore the Product Vision Planning and the Daily Commitment Planning, I believe it is a big mistake. Missing on the Product Vision Planning can end up easily with a 'great' product that is not useful or at least not at the accepted level of usefulness for the stakeholders. The arguably failure (since there is no official analysis of the project announced by any of its main members, things are still wildly arguable) of the Chrysler Comprehensive Compensation System (commonly referred to as "C3 Project") which was headed by the father of the Extreme Programming (XP) Kent Beck², in my humble opinion and from all the online discussions I went through, is due to a failure in the Product Vision Planning. The failure to anticipate the product's world and its complications in general is what stopped C3 from going into the second stage.
On other hand missing on the Daily Commitment Planning would result in a different kind of problems. Such problems include the continuously accumulated loss of accuracy of the project status updates which in turn result in mislead decisions and projections. If not noticed and corrected early enough, exceeding the budget and passing the deadline would be expected and in turn can result in the project failure. A good example of such a case would be the first 2 phases of a social media project called (Quraner) in which I was the Business Analyst and was involved heavily with the Technical Project Manger. The status updates and projections were positive all the time which was great news for everyone. However due to its daily accumulated loss of accuracy, we end up with projections that has nothing to do with reality. This mistake was fixed in later phases with our Daily Commitment Planning.2.1.3 Agile Planning Life Cycle
Due to the nature of agile methodology, the planning life cycle might seem strange to some people. Due to its short, repetitive and flexible nature, agile planning life cycle might look like as if it's done in a reverse order of the traditional (non-agile) projects. Some even may claim that agile planning is done as an after work procedure to fill out the missing documents. If such a thing is actually done, then it would be due to a misunderstanding of the main concepts of agile projects. Agile planning in general has a great emphasis on the following main concepts which differentiate it from the other planning types.Transparency
Agile methodologies by their nature are about minimizing the gap between the business world and the technical world. Agile planning is no different. Transparency is essential to make all stakeholders understand the result of their decisions and involvement.Understand Current Situation
Your project is the world you are creating. However as any other world, it has to exist in a universe; a universe which you don't have much control over. So do not assume what your universe will look like, go and see it yourself to understand it.There is always a place for improvement
Your project plan is a living creature. As any other living creature it will not stop growing, changing, and improving till it ceases to exist. So your planning is not done when the project's execution starts, you have to continually improve it.
Putting these concepts in mind will make you understand the various flavors of agile planning life cycle nature, in spite of the differences between them. So for example while Lean methodology asks for the planning life cycle to go as Think-Do-Check-Act, the Lean Startup methodology is referring to a modified version of it, Build-Measure-Learn (Figure 01). Both encourage flexibility; however I personally prefer the later one because of its emphasis on learning. Improvement is done through learning, so keep learning. If you are not sure about something, why waste time and effort on making up assumptions with no guarantees on their validity. Put your assumptions to test, and learn from the result. As a matter of fact Lean Startup looks at this testing as a"…more than just a theoretical inquiry; it is also a first product."⁴
Figure 01: Lean Startup Planning LC ⁴
2.2 Estimation:2.2.1 Estimation Projection:
Estimations in general are mostly driven from one or more of the following techniques¹:
- Expert Opinions: Through asking people with previous experience on the assigned tasks, estimation can be reached.
- Analogies: Through comparing known and unknown tasks to each other, we can find a relative scale for estimation.
- Disaggregation: Through breaking complicated tasks into more familiar and smaller ones, we can have more accurate estimation.
Agile estimation encourages the usage of all of these techniques combined together. An interesting style of doing so is called the Planning Poker which was first described by James Grenning in 2002. The interesting thing about this style, besides being fun, is its reasonably accurate estimation (If interested in learning more on using this style, check chapter 6 of Mike Cohn book "Agile Estimating and Planning").2.2.2 Estimation Validation:
Estimations are meant to be changed, so do get attached to them. Lean Startup encourages this through its "Innovation Accounting" which is a method for validating your estimations. This method is a loop of 3 steps⁴:
- Establish real data through creating the minimum viable product. This will allow you to see how realistic your projections are.
- Enhance your viable product to become closer to your projections.
- "Pivot or preserve" your projections. Depending on the results, you have to decide on keeping the projections as valid or changing them accordingly.
3. Using Agile Planning & Estimation
So going back to our question, is agile planning and estimation suitable for you? It depends on the project's agility level. Most projects usually are neither 100% agile nor 100% waterfall (non-agile), but they sit somewhere in the middle. This goes for the agility level of planning and estimation too.
An example of projects that tends to be closer to the waterfall style is usually the ones that deal with the government sector, the planning procedure tends to follow the ordinary (non-agile) type. Regardless of how flexible they try to make their planning procedure be, the outside forces of bureaucracy, the 100s of signatures, and the huge out dated contracts which they have to deal with, will not allow such concept to run smoothly.
On the other hand the private sector is another story. It seems that agile planning is actually appreciated in the projects that are not related to the governments. Due to the fast based environment the private sector is in and the ever changing competition, ordinary planning will not survive. What stakeholders want right now is not what the want in 6 months, due to the continually changing needs and requirements of the market. That's why agile planning becomes the natural thing to do.
Although agile planning seems to be the natural thing to do, people approach it in different ways, depending on the nature of their projects and their background experience. A good strategy to follow on deciding on the style of your planning would be to start with agile planning regardless of the situation. Then as things become clearer, let the planning stays agile if possible or switch to the ordinary style if the nature of the project requires you to.
In order for agile planning to work correctly, the client needs to have an understanding of the nature of agile projects. Educating the client on the nature of agile world can be possible if you are dealing with the decision maker directly.
Agile planning and estimation is different from the ordinary style. Its creation is a natural reaction of the evolving agile projects and organizations where ordinary planning and estimations fail to survive due to its rigid nature. Agile planning and estimation is misunderstood by a lot of people due to its short, repetitive and flexible nature. However if it is implemented correctly, agile planning and estimation can be the best solution for a lot of projects and organizations.
- Cohn , Mike . Agile Estimating and Planning . Prentice Hall, 2005. Print.
- Fowler, Martin. "C3." Martin Fowler. N.p., N.d., Web. 30 Dec. 2012.
- Kishawy, Mohamed. "Agile Methodologies - The Missing Path." DZone. MN, USA, 14 May 2012. Web. 30 Dec. 2012.
- Ries, Eric. The Lean Startup. New York: Crown Business, 2011. Print.
- Smits, Hubert. "Scaling Agile Processes: Five Levels of Planning." Pragmatic Marketing. N.p., 7 May 2007. Web. 30 Dec. 2012.
The postings on this site are my own and don't necessarily represent my company's positions, strategies or opinions.Google