Lean tools: Feedback

Feedback is one of the foundations of the Agile movement and of any iterative methodology for developing software. Lean tools for amplifying learning feature Feedback as one of the core concepts too.

Development and production

A fundamental distinction that must be cleared before diving into the tools, is that software development is not comparable to physical production. Summing up a long discussion, here's what we sometimes errouneously think:

Design -> Architect  -> Product development
Coding -> Bricklayer -> Factory work

However, the real mapping is this:

Design and coding -> Architect  -> Product development
Build             -> Bricklayer -> Factory work

I heard this proposition for the first time at the Italian Agile Day 2010 in Paolo Perrotta's keynote.

The manual work in software development is almost completely automated nowadays; what's left is knowledge work that cannot be organized as a factory for producing cars, even with the practices of Lean manufacturing. Product development however is present in Lean too, and it's where the following practices come from.

Feedback and feedforward

It's impossible to get a design right the first time, before coding a single line: the start of a project is the instant in which we know less about the problem, and so the amount of design that we product should take in account that.

Feedback as a Lean tool is more general: the theory of control tells us that feedback-based system are more complex than feedforward ones but also much more robust to anomalies.

The most physical example of feedback is temperature regulation. It would be ignominiously difficult to model a house or a room and the heat that it transfers from and to the surrounding environment, and then producing the right amount of heat from electricity to maintain the required temperature. It's only a matter of time before small errors in measurement and in the model parameters make the temperature drift away from the target: even if you just put in 0.1W more than required, the system isn't stable.

And finally, since there is much variability during the day, the calculations will be far off as no control system can predict if it's raining or sunny without feedback. Heating your house in a sunny day because a model said so doesn't seem very smart.

Instead, any thermostat has an internal temperature sensor, and activates heating when the temperature goes below a certain threshold (with some hysteresis). Inexpensive and much simpler with respect to calculating when to enable heating.

Feedback in software development

We typically think of feedback as delivering working software at the end of a day or an iteration, or as our test suite running. However, feedback can be inserted nearly everywhere in the development cycle.

In fact, the original 1970 paper, endlessly cited in discussions of feedback, contains a double cycle for waterfall:

Thus if 30 years before the Agile manifesto software development was already thinking of feedback at every level, we can do the same:

• pretotyping and the Lean Startup movement professes experimentation to check that the requirements are actually useful, and produce value that can be measured with split testing. This should be performed before starting development at all.
• Domain-Driven Design involves talking with domain experts very often to check the model of the world we are extracting from them is not missing or "travisando" pieces of the puzzle. Analysis is not done by segregating an analyst in a room crunching documents.
• Big Design Up Front is universally despised in favor of little or no design up front, with the code emerging from successive iterations. Whether a starting point for the design is useful is not important in this discussion: the tricky part is being open to change our decisions down the road, especially if we took them before coding.
• Test-first approaches like TDD forces us to write end-to-end and unit tests before the code they exercise, in order to give us feedback about an API (class names, method signatures and ease of use) immediately.
• The walking skeleton practice prescribes to build and deploy an empty system (which just says Hello world) to validate the technological viability: giving us feedback on the difficult of updating a schema or invalidating caches before we start to satisfy functional requirements.

Conclusions

It's not only a question of introducing feedback in a process: we are already doing that almost everywhere, with bug tracker, test suites and similar tools. But we should also strive for improving the quality of the feedback, making it easier to generate, and tune the frequency with which we receive it, which should be not too low but also not too high to make the system unstable (like telling your colleagues to change the feature to be developed every day).