A Note On Quality
The Agile Zone is brought to you in partnership with JetBrains. Learn how Agile Boards in YouTrack are designed to help teams plan, visualize and manage their work in an efficient manner, with support for both Scrum and Kanban processes.
“We never use a screwdriver in the last week. We hammer the screws in. We slam solder on the connections, cannibalize parts from other televisions if we run out of the right ones, use glue or hammers to fix switches that were never meant for that model. All the time management is pressing us to work faster, to make the target so we all get our bonuses.” Worker in a Soviet television factory quoted in Milgrom & Roberts: Economics, Organization and Management.
Every hacker, at one time or another has committed the software equivalent of hammering in screws when a deadline approaches. It’s understood that quality suffers, but quality can mean many things. Here we talk about three kinds of quality: design quality, conformance quality and total quality.
Design Quality is a statement of intent, a measure of how the product as designed would appeal to the market’s true needs and desires if it were made perfectly. It has nothing to do with the hidden details of the internal architecture and everything to do with user. The original iPhone had great design quality.
Conformance Quality is the degree to which the actual product reflects that design.
A product with few bugs, but many issues closed as “Works as designed” may have good conformance quality and poor design quality. Usability issues are design quality problems. Designing a product no one wants, that costs too much to operate or that isn’t competitive are others. In our simple model of software development, what we’ve labeled “quality” is conformance quality. We’ll add design quality when we add customers to the model.
Typical bugs are conformance failures—the product doesn’t perform as specified. But if you host your software, so are performance problems, deployment problems, hardware issues, scaling problems. All detract from the user experience and create a gap between the intended value and what you delivered.
Total Quality is the combination of design quality and conformance quality. The combination is not additive: poor design quality destroys the product’s value, even if the conformance quality is very high. A product has good total quality when the implementation conforms closely to a design that meets the market’s expectations.
We distinguish between design quality and conformance quality for a reason: Most software organizations invest far more on finding bugs than they do on the quality of their requirements, usability, or the competitiveness of the overall design. For startups, at least, this is starting to change as more adopt the Lean Startup approach that makes customer development (essentially market research) an equal partner with product development.
 The distinction between these three definitions of quality is stolen with pride from Kaoru Ishikawa.