Earlier this year, RebelLabs released its annual Developer Productivity report. It was based off a performance survey which collected responses from over 1500 developers, testers, architects, and many more interestingly named job titles. The great news is RebelLabs gave $0.50 for every completed survey to a great charity called Dogs for Good (formerly Dogs for the Disabled), which provide assistance dogs to disabled children.
As with all problems which have many solutions, it can often be tricky to determine which is the best solution for you. What works for other people may not work for you. When it comes to profilers, there are many on the market, and it’s often tough to determine what the differences are and which is best for your application.
This blog post looks at some of the report insights and answers the following questions:
- Which tools find the highest number of bugs?
- Should I be using more than one tool?
Let’s look at the results!
Almost Half (46.6%) of Survey Respondents Use Multiple Profilers
The wily among you will have noticed that the first image in this blog contains percentages which do not add up to 100%. The reason for this isn’t basic math incompetence. No, not this time. This time it’s because the question allowed for multiple choice answers, in case people tended to use more than one profiling tool on their applications. It turns out that it was very common for teams to use multiple tools as each tool tends to be better in certain phases or the application lifecycle, or perhaps preferable when chasing down a particular type of performance bug.
Wanna Find Bugs? Roll Your Own!
Most performance bugs were found by custom in house tools. This might be because those who write the custom tools work with very particular applications and have the specific application knowledge to expose certain bugs that other tools may not. We might also be able to infer that if a team were able to write their own in house performance tools, they are likely to be closer to the nirvana that is a performance expert. As such they would be able to find more performance bugs than others anyway.
Tools which did very well include JProfiler, XRebel, NetBeans profiler, JMC and JProbe. From a personal point of view, it was really exciting to see XRebel, a relatively new profiler on the block finding some of the most bugs out of any tool. This may well be because it is designed to be used in development, a place where it’s easier to find and eliminate bugs.
Can You Really Blame Your Tools?
Well, from the graph below, you might be able to blame not having enough tools, as there’s a definite trend showing that you can find more performance issues if you use multiple tools. This helps to show there is a right tool for the right job. The job however may vary, for instance, you might be trying to find and fix a specific types of IO issues or code bottleneck. You might have a prefered tool for each of those tasks. You might want to use JMC in production, but XRebel in development as each tool is better suited to different environments.