Software Quality Metrics that One Shouldn't Use
Everyone knows about the improtance of quality metrics, but if you're using these metrics you might not be getting the full picture.
Join the DZone community and get the full member experience.Join For Free
Typically in an Agile software development team setup, after every sprint, the teams would have to produce a sprint report, used to review the sprint and also to report the status of the sprint to the line manager/coach. This report normally would have details about the number of lines of code produced by the team that sprint, the number of unit tests, integration tests, acceptance tests, exploratory tests, code coverage and the number of code reviews that sprint. This report will be scrutinized and critiqued if the metrics show a downward trend.
And this activity is done mostly on the first day of the sprint, which, ideally, should have been spent in sprint planning, but is invariably taken up to analyze various repositories.
This has been the case and is still the story of Scrum Masters across the globe. These metrics were and are used by the team’s manager and their manager and so on to ascertain the productivity of the team or to secretly rank the teams.
Scrum Masters often question the reason for capturing such metrics.
They may guess right and come to the conclusion that they are asked to measure these metrics for their teams because it is easy to understand, the number of lines of code increasing supposedly meaning that the teams are not idling away. High coverage means that we have great tests, or the developers know how to write unit tests.
It is a matter of time before the reality hits such teams who diligently measure these metrics that the metrics they gather have no relation to the predictability or quality of the product they write nor can decisions be made based on the metrics gathered.
The number of lines: The number of lines produced is a very easy-to-understand metric, but it is not consistent as programming languages differ. And of course, as the product develops, lots of refactoring has to be done to add new features, which increases the lines of code exponentially. In addition, a lot of commented code and a lot of "TODOs:" add to the code volume, making this metric not very healthy and not indicative of anything regarding the product or the quality of software.
Coverage: This exposed the number of lines of code not tested by unit tests. Not every single line can be unit tested; sometimes we just don’t want to test, so it is of no value.
The number of unit tests and number of code reviews: The increasing trend doesn't mean anything to the developers. Tests were not capturing all use cases, code which have a high number of tests still didn't cover the scenario the customer had, and limited code reviewers with rights to code check-in were the bottleneck to get any code delivered.
These facts clearly indicate that one has to gather metrics that generate value, show indications of the quality of the software, and are actionable!
Opinions expressed by DZone contributors are their own.