However, in most organizations, software development efficiencies are subjected to infinite analysis while customer demand effectiveness isn’t mentioned at all. We neither measure nor calculate feature value in the beginning nor do we measure whether or not value was actually captured as the customer or product manager had predicted. In many large companies project ROI might be calculated as part of the portfolio management process, but rarely does anyone follow up to see if that ROI was in fact attained. So development teams have feedback mechanisms while customer/product teams have few except possibly at a macro level (product sales for example). Product managers are left with gut feel for most of their feature-level decisions. No wonder 50%+ of developed features are rarely or never used. Product management teams aren’t the culprit here however, lack of adequate feedback mechanisms are.
If we look at a typical Agile project team, there is the development sub-team and the product/customer sub-team. Roles, responsibilities and feedback mechanisms have been defined, so for example, the product team identifies stories, writes stories, prioritizes them, and develops acceptance criteria (both automated acceptance tests and feature showcase evaluations). So there are micro-level (feature and story) feedback mechanisms to steer the development effort. But what about feedback from product management to the business? These are often much more tenuous, such as overall sales that tell how a product is doing at a macro level, but say nothing about individual features. Internal IT products have even less feedback in most cases. Lack of feedback leads to feature bloat since it’s always easy to succumb to customer requests and internal demands to improve the product.
At the Agile Brazil 2011 conference I was listening to Josh Keriesvky’s talk on his Lean Startup experience, and gravitated to thinking about this problem. Here’s a starter list of ideas about potential solutions:
- Develop Customer Demand Effectiveness measures for every product management organization and team.
- Calculate relative or monetary value for every feature.
- Use relative benefit dials such as increasing customer happiness, reducing customer risk, improving our internal collaboration culture, to evaluate feature value.
- Build feature usage information into software to provide feedback to product management. Product manages could thereby gain insight into what was actually used and what wasn’t.
- Develop feature evaluation experiments into your feature identification and prioritization process. For example do A/B testing on features.
- False feature analysis. Give users access to a feature, that when selected pops up a message something like “this feature is under development. When completed are you ‘very likely’, ‘somewhat likely,’ or ‘not likely,’ to use it. Josh talked about an expensive feature that his company decided not to implement because of the results from this type of test.
- Use short, focused surveys to take a measure of customer happiness. For example, “What was your experience using capability Y (capability being several features)? Awesome, OK, Not so Hot.
While there has been a lot alluded to in the Agile and Lean communities about value-based development, the actual practices to support this objective are not yet sufficient. There aren’t sufficient feedback mechanisms at the feature level to help mitigate the constant push towards feature bloat. Maybe taking a look at some of the ideas from Lean Startup and other sources can help.
These are some of my preliminary ideas. What are your ideas about this issue?