Agile Methodologies - The Missing Path
In the past few years an increasing number of styles of software development models, referred to as agile methodologies have been gaining a lot of interest in the software development world. Promising an antidote to bureaucracy that old models were forcing, agile methodologies became very successful in gaining a huge number of followers and supporters all over the software landscape. Eventually the question changed from "Should we become agile?" to "Which agile methodology should we follow?" This article will try to answer this question by showing (in general) three of the main agile methodologies that have a long record of success. This article will also try to decide on the best way of implantation for these methodologies particularly in startup environments.
So let's start by going over the main concepts of the 3 chosen methodologies; Lean, Scrum, and Extreme Programming (XP).
1. Lean Methodology:
The first thing to understand about Lean methodology is that the commitment the organization has to give to its core values and the caring about how things are being achieved more than what the result was, very critical in Lean methodology. Lean adaptation depends on the following main concepts²:
The Spirit of Challenge: The spirit of challenge paired with creativity and courage to guide people to dream bigger, and not be satisfied with the current situation will help create the environment and the people who value the highest quality possible.
Kaizen Mind: Creating the mentality that there is always a place for improvement will result in flexible and objective mindsets. This will always make the adaptation of new technology or tool easier and smoother.
Genchi Genbutsu: This is about the idea of “go and see to deeply understand”. This concept is one of the main tools which will smooth the communication between the different organization's levels. This will assist the top levels to understand the language of the levels below them through getting the knowledge and the experience of the real work being done by these levels.
Team work: The importance of team work needs to be fully understood by the entire organization. However the Executive Management level will have a critical role in implementing this concept by showing how the success or failure is a success or a failure for the entire team. This is done not by ignoring the individual’s evaluation, but by giving the team evaluation a higher weight.
Respect: The organization's levels should not be looked at as the levels of respect the people should get. The people in the lower level of the organization are as important as the people in your level and the level above you. If any level disappears, the rest will be destined to fail. Understanding this and how important everyone in the organization is, will help in building this environment of respect for every individual and team.
These core concepts provide a great environment for building leaders in the organization. Jeffrey Liker and Gary Convis explain this in great details in their book The Toyota Way to Lean Leadership.
2. Scrum Methodology:
Scrum methodology is about enabling teams to be as productive and effective as possible. Through creating Self Organizing Teams (I agree with Alan Shalloway ⁴ on calling them Self Organizing Teams and not Self Managing Teams because an upper management will always be needed) and applying the best strategies that support the work of this type of teams. The following are the main strategies that are need to adapt Scrum Methodology:
Daily Scrums: These kinds of short and concentrated meetings at the beginning of each day are a very helpful tool to refocus the team on the objectives of the project they are on and to keep the entire team in harmony with all the new updates and changes.
Sprints: Dividing the project into small parts that can be accomplished in single sprints (2 to 6 weeks each) help in focusing the team's effort towards achieving the specified tasks.
Prioritization of tasks: According to the client requirements and the importance of the required tasks (based on the evaluation of potential risks and critical path for each requirement), each task gets categorized into a line of priority levels (for example: Critical, High, Medium, and Low). This minimizes the negative impact that the possible failure of achieving the goals of the sprint in full might have.
Continuous Reviews: Continuous reviews and evaluations of progress for each sprint and its integration in the entire project allow the team manager (usually called Scrum Master and he is not the team leader) to foresee the problems and bottlenecks more accurately and to take the corrective action needed.
Retrospectives: Going through the lessons learned at the end of each sprint and each project allow the team to expand their knowledge and learn from previous faults. This will also help the team to be open to constructive criticism and ideas for improvement.
3. Extreme Programming (XP) Methodology:
With the emphasis on team productivity, the XP methodology offers a very helpful set of strategies. The XP core concepts can be adapted through the following set of strategies ⁵:
Enhanced Communication: XP achieve enhanced communication on the following different channels:
- The team and the customer (via on-site customer)
- The individual and the team (via co-location of the team)
- The individual and the other individuals of the same team (via peer code reviews)
Flexibility & Adaptivity: XP achieve flexibility and adaptivity through keeping the design as simple as possible and coding what is needed only in a way that anticipates the inevitable change of requirements.
Accelerated Feedback: Through small releases the customer can give a quicker feedback and prevent any deeper diversion from the right path. Also applying Test Driven Development (TDD) will enhance the quality of the code and lower the time spent on communication between developers and testers due to lower defect rates.
Shared Responsibility: Through collective code ownership, the entire team will feel the meaning of shared responsibility to enhance the quality of the code. So whenever there is a need for refactoring, it is done without delays.
After understanding the main concepts of each of the previously mentioned agile methodologies, which one should you implement? Which one is better than the other? Answering these questions might seem very hard especially with the religious wars each of these methodologies' followers is fighting. Blinded with their love to their methodology, each team claims it as the ultimate solution for all problems in all situations. However, anyone who is not involved in such wars can see how silly this claim is. The more you get involved in software projects of different sizes and types, the clearer it becomes to see how the success of each of these methodologies (if implemented right) will be depending heavily on the situation it has to face. So while XP offers no help for managing multiple teams, Scrum excels, and while Scrum fails to deal with big numbers of teams of different levels, Lean shines.
So what should you do? It's easy; do not stick to one methodology. It's not a religion (although some takes it as one), be selective; take what can help you from each one and leave what might not suit your situation. If done right and integrated carefully to avoid conflicts, you will end up with your perfect agile methodology that is built for you specifically.
If we take a classical organization (not agile) and analyze its culture, most probably we will find that their structure is divided into 3 main levels: Executive Management, Project Management, and Development. Although each level needs to work with the one above and below it, each one has its different needs and problems. It will become clear that there is no one methodology that is capable of solving every problem of these 3 different worlds. Because of this, merging (selectively) the 3 agile methodologies (Lean, Scrum, XP) to address the needs of each of these levels and integrating them in a way to prevent conflicts between them become a must.
In general, I agree with Geoffrey Bourne's concept that implementing Lean methodology for the Executive Management level, Scrum methodology for the Project Management level, and Extreme Programming (XP) for the Development level (if done correctly) would solve most of the problems a classical organization might be facing. Each of these methodologies should be applied selectively to guarantee smooth integration between them as Figure002 shows below.
To test how accurate this interesting theory is, I initiated two investigating meetings. One was with a startup called (Fortress NetGaurd) specialized in security and automation software and had at one point a 20 multinational team members, and another with a software development and training company (they refused to allow me to include their name in my research) specialized in taxes software, deal mainly with the government sector and have about 150 team members.
The meeting was on the phone with the executive manger (California) of the first company and with a team leader (Colorado) in the second one. Both meeting started with big disappointment when both said they implemented neither Scrum nor XP methodologies and asked me what I mean by Lean! However out of desperation I continued the meetings and asked general questions on how they manage their teams and projects. It was really surprising to know that they are using a lot of the agile tools from different methodologies without even knowing it. They didn't use the right or known names for things, but they indeed used it. Both companies' CEOs started the companies and were involved in coding themselves at one point (Lean: Genchi Genbutsu). Both companies had the concept of daily stand up meetings for team members (Scrum: Daily Scrums). Both companies had on site representative of their client with related projects (XP: On site customer).
Although two investigating meetings are not sufficient enough to generalize a rule, I was glad that they didn't conflict with it. Regardless of the religious wars each of the three (Lean, Scrum, XP) methodologies' followers are fighting, some companies and individuals already decided on the selective integration of agile methodologies as the right way to follow.
A recent success story of this was published in a book called "The Lean Startup" for Eric Ries³. However the writer (who is the entrepreneur of this 'new' methodology too) decided to give this mix and match of agile methodologies -which helped in the great success of his startup- a name; "Lean Startup" methodology. In spite of the naming which is -clearly- for marketing reasons, I find his ideas very interesting. Although he doesn't mention it clearly, he actually follows a selective integration of Lean, Scrum, and XP just like what my research is suggesting.
To summarize this point, I believe that in spite of the shortage of researches and recorded success stories supporting the concept of mixing and matching the agile strategies and the selective integration of these methodologies, companies (especially startups and small companies with low switching costs) and individuals are doing it intentionally or unintentionally.
Agile methodologies are great weapons against most of the weakness of the old software development models. Through its concentration on closing up the big chasm between business and technical worlds and eliminating red tapes, it helps making Software a more efficient and productive world. This research studied in general, three of the main agile methodologies; Lean, Scrum, and XP, trying to find the best path to agile. However after the analysis of these methodologies, the research suggests a selective integration of the three methodologies as a better way of approaching the best agile solution.
1- Bourne, Geoffrey. "The Marriage of Lean, Scrum and Extreme Programming (XP): How to Align Agile Across an Organization." Executive Brief. N.p., August 2010. Web. 30 Dec. 2012. .
2- Liker, Jeffrey, and Gary Convis. The Toyota Way to Lean Leadership: Achieving and Sustaining Excellence through Leadership Development. New York: McGraw-Hill, 2012. Print.
3- Ries, Eric. The Lean Startup. New York: Crown Business, 2011. Print.
4- Shalloway, Alan. "Different Agile Approaches: 1st (XP, Scrum) and 2nd (Lean/Kanban) Generation Methods."Webinar Recordings. The Agilista PM, 3 Mar. 2010. Web. 30 Apr. 2012.
5- Shore, James, and Shane Warden. The Art of Agile Development. Sebastopol: OReilly, 2007. Print.
The postings on this site are my own and don't necessarily represent my company's positions, strategies or opinions.