144 developers, managers, testers, and executives participated in the study - each from a company with more than 1,000 employees and 50 developers. Most organizations were located in North America. The study found that 58% of the developers polled said most of their software bugs were caused by insufficient testing time, poor testing procedures, or limited testing infrastructure. 31% cited design flaws as the key contributor.
Building anything other than a unit test can be time consuming, and some developers don't even bother with the unit tests. Finger pointing is prevalent during bugfix time. Developers point to their managers saying that they're more interested in speed and quantity rather than quality. Some are also frustrated when company leaders won't invest in the computing resources and tools that can make testing faster, easier, or at the very least make it possible (some developers find that all of the resources and equipment are already being used when they go to test). Of course, some developers have bad programming habits too. If they don't have to fix their own code when bugs are found, some people will just get the job done and leave someone else with the clean up.
Defective software can have seriously harmful effects. Bugs found late in the production cycle usually affect release dates, according to the study. 44% of the respondents said that the last significant software bug cost their companies an average of $250,000 and about 20 hours to clean up. Here are some more highlights from the study:
- Only 12% of organizations had automated test systems. On the other side of the coin, only 10% did all their testing manually.
- 46% said they don't have enough time to test properly (this is in line with another survey done by JBoss)
- 36% said that their organizations don't test enough before product releases.
- 53% said that testing is limited by compute resources (slow builds or unavailable resources).
The final statistics showed that developers who believed their organizations set aside enough time for testing had less problems from bugs and spent half as much time fixing those bugs.
Do you think poor testing is the biggest problem in today's software development shops?
Is there any other things that frustrate you more than testing? For example: Writing documentation, having no equipment available to run code, code with no comments, or a change in requirements for the phase that you are working on?