For those who missed the earlier parts here are the links.
Enterprises when it comes to software development have been embracing agile for many numbers of years. However, many concepts from Lean Startup and Lean Thinking could be used in conjunction to improve the accuracy and predictability of software development and delivery. Many of these concepts are not contradictory but could be complementary. By combining the power of lean concepts and agile methodology, organizations could go a long way in eliminating waste and delivering software that adds customer value.
Lean advocates defining value in terms of Customer. When it comes to software development defining the value as to what the Customer wants is very important. Companies should keep the customers in mind and define value and not get overboard with the value defined by engineering or product experts alone. This lean approach of defining value from a Customer perspective and identifying activities that add value could help a long way to avoid waste.
The concept of MVP as defined by Lean Startup could be adopted by enterprises when it comes to software delivery. Many times, enterprises do a lip service to agile by following Scrum or Kanban processes to deliver software week after week. Many of the team structures in place need a high level of collaboration and coordination to actually deliver software that is immediately consumable. We would be looking into the team structures in detail later as well. Organizations should focus on building the MVP for the following reasons.
- Build MVP to infer validated learnings about who your customers are and what they want. The enterprise should use the same rigor when they are facing uncertainties.
- Even within the software built for enterprises where the Customers are known, having an MVP would help to deliver useful product increments rather than building software that lies idle.
In one of my projects we were working hard on creating an Architecture around a Legacy System, which must be decommissioned and we were facing variety of technical challenges. We had an announcement that our priorities have changed. We had a meeting with our Senior Leader where he made it clear that we must work on a complicated feature which he has come up in his mind. The Leader had complex ideas around how the functionality should work and believed that we are going to have crazy growth. This meant shift in focus and working on a rigid schedule.
The Lean Startup rightly talks about using Validated Learning to verify Leap of Faith Assumption. However, our teams made no attempts to verify the assumption that thousands of Customers would sign up for these discounts.
- There was no effort made to see if Customers would really like the asked feature or whether the feature would increase our sales. It was more a knee jerk reaction to information that our Competitor is offering such a feature.
- Similarly, there was not an effort made to understand whether the asked feature to be modeled is something that is supportable by our Systems. Teams tried to customize the system producing a to support the feature Had an effort been made to understand the technical capability at the very start, the whole approach towards could have been very different.
- The team went about implementing this complicated feature for the next two months. We all were working on a tight schedule. Once the support for the feature was offered, we just had 10+ Customers signing up in its entire life cycle. We had one of the engineers making a comment that had we hired a person to manually provide the functionality we would have supported him for the next 200 years; with the money, we have used it for building the software. We all were exhausted and as a team also felt that we had failed despite all our good efforts.
Had our teams rigorously attempted to validate the Hypothesis that we would have thousands of people lining for these feature by attempting to build a MVP even before building the actual systems and had used the Build/Measure/Learn to make decisions around pivot or preserve, so much of waste could have been avoided. Agile techniques like Scrum provide product owners the single authority to decide what will be developed and in which order. However, there are not necessary steps within the process to validate the vision of the stakeholders. So even enterprises when taking challenging enterprise projects should adopt the Build/Measure/Learn approach, validating the vision of business stakeholders or product managers as quickly as possible.
Is Software Development analogous to a factory where the same goods are repeatedly made or is it like a laboratory where something new is being invented? Unlike manufacturing where there are many inefficiencies, when it comes to mass production, for a software most of the challenge is around building the first copy of the software right and scalability is addressable and not as expensive as building the new software. Many organizations approach building software with a naïve belief that it would be rather used and end up building things/planning for scale/quality that are never being used. Organizations should imbibe a laboratory-like scientific mindset when building the software initially, by focusing on small audience, clearing ambiguities using MVP/Build-Measure-Learn approaches. Once it has validated learnings it should then attempt to switch its mode towards scalable, high quality products. This needs a fundamental change in the mindset of how organizations approach building software and how they approach quality and scalability concerns.
Build Versus Buy
On occasions an organization could incur waste not just by building software that customers don’t want but also by ending up building software that is readily procurable. In this era of packaged software available for subscription in Cloud model, organizations should have a strong decision-making system as to whether they are building the right kind of software that is a differentiator for the enterprise versus buying the software that meets the business needs. Any effort that multiplies the cost and delays the time with respect to adoption, when it comes to building a software versus directly consuming the software should also be treated as waste.
General form of Conway’s Law, states that: “organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations.” (Melwin E Conway, Conway’s Law)
Most of the software companies tend to operate in the fashion described. Teams of expertise (User Interface, Back-End APIs, Database, DevOps, Business Analysis) are formed and well-coordinated communication patterns/resources are established for teams to collaborate and deliver.
(Diagram Reference: Vlad Ungureanu Conway’s Law)
Perceived benefits in this approach are
- Team of experts focusing and specializing.
- Ability to handle multiple projects using shared expertise.
However, this approach results in significant challenges and people issues. I am detailing challenges we faced in a similar setup for one of my projects.
The teams were working on a Bottom-up approach. This meant a layer like a Database design was complete and the communication was used to agree and propagate changes on the top layer like Back End API and from there changes to the UI.
Each of these teams ended up having different reporting hierarchy and the teams were managing multiple competing priorities.
Every team had a different opinion on which project is the top priority for them. This means some teams would finish their work which never gets consumed for a significant time which is a waste.
Change is any requirement meant changes must be propagated across many different teams.
When things went wrong, people were more focused on passing the blame in the hierarchy rather than taking a collective approach to problem-solving.
At times disconnected groups led to disconnected design and teams had to spend a longer time correcting their designs.
I was intrigued by the idea of cross functional autonomous teams as proposed by Lean Startup and Lean Thinking and advocated by Scrum.
- Teams were cross functional and had many different skills to help with the success of the project
- Every member in the group was fully dedicated to achieving the overall goal.
- The teams took on a collective mission
- The teams operated with a high level of independence.
In hindsight, rather than having multiple separate teams working on crucial projects organizations should build a self-contained team, working on a collective mission. This would have greatly reduced many of the above-mentioned problems.
In manufacturing Toyota applied lean thinking and used small general-purpose machines as against large specialized machines to produce a variety of parts in small batches. Though the manufacturing has effectively utilized small batches most of product development happens only like large batches. Lean Startup advocates doing just enough work as quickly as possible to know that what we are building is what Customer wants. This is very different from the small batches advocated by Scrum and Kanban which just focuses on the software delivery alone.
In manufacturing using Pull techniques factories produce only when there is a demand. For lean startups the focus is to build things only when the Customer wants them and doing things as minimally as possible. It recommends approaches like AB testing to serve two variants to quickly figure out features Customer’s preferences and build upon it. While the pull approach advocated by Kanban just focuses around allocating resources when they become available by controlling the amount of work, there is never a mandated end customer focus to identify whether what we want to build is what the Customer wants. The Pull model should be extended to verify that what we are building is what the customer wants to prevent waste.
We explored the evolution of many different agile and lean techniques. While Agile is efficient in prescribing how to develop a working software, it is still unable to give answers as to what is being developed is useful. The shortcomings of agile could be very well addressed by adopting some of the recommendations from the Lean Methodology and Lean Startup that have been summarized.
- Beck, K. B., Mike; Bennekum, Arie van; Cockburn, Alistair; Cunningham, Ward; Fowler, Martin; Grenning, James; Highsmith, Jim; Hunt, Andrew; Jeffries, Ron; Kern, Jon; Marick, Brian; Martin, Robert C.; Mellor, Steve; Schwaber, Ken; Sutherland, Jeff; Thomas, Dave. (retrieved 2020). Manifesto for Agile Software Development. http://agilemanifesto.org
- J P Womack &D T Jones (1996) Lean Thinking Banish Waste and Create Wealth in your Corporation
- Mary Poppendieck (2002), Principles of Lean Thinking
- Robert Cecil Martin (2002) Agile software development: principles, patterns, and practices
- Kenneth S. Rubin (2012) Essential Scrum
- Ken Schwaber and Jeff Sutherland (2012) The Scrum Guide TM
- David Anderson (2010) Kanban: Successful Evolutionary Change for Your Technology Business
- E Reis Lean Startup: Lean Startup
- E Reis & Team (retrieved 2020) www.theleanstartup.com
- Vlad Ungurean Conway’s Law in Software Development 2019, https://firstname.lastname@example.org/conways-law-in-software-dev-3aa6324ead52