Over a million developers have joined DZone.

Bug Bounties Considered Harmful

Are bug bounty programs getting Boba Fett results at Greedo prices? Take a look at the analysis and insight this article offers on bug bounties.

See Gartner’s latest research on the application performance monitoring landscape and how APM suites are becoming more and more critical to the business, brought to you in partnership with AppDynamics.

Bug bounties are a large part of today’s cyber security research landscape. They’ve become more lucrative for security researchers, have shown tangible benefits to sponsoring companies, and have helped secure the information of billions.

They’re also a ripoff.

Economics of Bug Bounties. Sponsoring companies love bug bounties - and why shouldn’t they? They encourage hundreds of engineers to analyze their production systems. And they actually find security vulnerabilities that are responsibly disclosed! Which is great - companies love responsible disclosure, and they should. It allows them to get fixes out into production prior to the vulnerabilities becoming well known. And they pay for the service, after all, right? I mean, they pay for the vulnerabilities people find, so, you know, win-win, right?

Well, not so much. Companies sponsoring bug bounty programs are getting a screaming deal, and they know it. The security researchers? Not so much.

Application security testing is difficult - for real, it’s tough to do. Take a look at some of the leading researchers over at GitHub’s bug bounty program for example, and look at the kind of vulnerabilities they’re finding. The leaders are doing really strong, detailed work, and the front runner especially. So, how does this work for his personal economic situation?

Well, GitHub doesn’t list the amounts paid, but they do score vulnerabilities via a point value. I can’t find the maximum points they’ll grant for a single vulnerability, so looking at the leader, I’ll assume 5000 points is the maximum. I’ll also assume a global reward ceiling of $10K, and a floor of $200 (these are the ranges GiHub gives us), with a linear correlation between granted points and payouts. So, GitHub’s leader has, today, 30,750 points across 14 vulnerabilities. Looking at individual scores, scaling those scores, relating to possible payouts, and taking into account that these have been found over a two-year period, the leader in GitHub’s bug bounty program made a little over $30K per year.

This analyst is doing top shelf work - and he made just over $30K per year doing it. Glassdoor gives the average salary for a security researcher as just over $106K.

So how is this working out economically for the lead researcher with GitHub’s bug bounty program? Look at the numbers - he gave himself roughly a 70% salary cut, so, not well.

And this is the leader in the program, by a significant amount. The next closest researcher has a bit over 13K points.

Commoditization of Security Research. Not only is participating in bug bounty programs less lucrative than actually having, you know, a job, but penetration testing and security research is beginning to be commoditized. Bugcrowd is a company that crowdsources application penetration testing, and pretty successfully it seems. The concept is simple - they help companies set up bug bounty programs, either time-boxed or ongoing, and they suggest that companies use crowdsourced bounty programs instead of dedicated penetration testers. And it certainly makes economic sense - as we saw above, experienced penetration testers are expensive. Much more so than equivalent bug bounty participants.

We’ve seen how well these kinds of things have worked for general software development. Anybody logged into elance lately?

Moreover, these kind of programs are not an adequate substitute for security-minded system design and development, nor are they for end-state penetration testing. System security needs to be built into products, not bolted on at the end prior to a bug bounty program - or, for that matter, a penetration test. System and application security is much more important, and much more difficult to implement, than that. Crowdsourced programs, while better than nothing, are not an adequate replacement for experienced security analysis. Thousands of average security analysts won’t find that one complex, difficult-to-find problem that a more experienced analyst can track down. And that one problem may very well be the one that compromises your system.

I guarantee GitHub has top-notch cyber security engineers on staff. Bug bounties are just the icing on their corporate cake.

What to do? So bug bounty programs aren’t great for experienced penetration testers. You’re not going to make a living from it. Nor are they a substitute for security minded development and development processes and highly trained cyber security staff. Companies that rely on bug bounty programs to secure their systems are going to be disappointed, as are engineers that try to make a living from them.

So how do you fix this? Well, first, bug bounty programs should pay more. There’s a variety of markets for vulnerabilities after all - the FBI just forked out over a million dollars for an iPhone vulnerability. Companies like Zerodium have recently entered the market, providing reseller services to security researchers for zero-day vulnerabilities. If companies are serious about bug bounty programs, they’ll need to match the prices those vulnerabilities can fetch on the open market - they’ll need to match the amounts that buyers like Zerodium (and their clients) are willing to pay.

And that’s lots more than $10K. Put an extra zero or two on the end and you’ll be in the ballpark.

What if you’re an engineer? Well, look, bug bounty programs are a good way to get experience. But honestly, you’re not going to pull much money out of them. You can, however, study the actual exploits, and get some insight into how they were tracked down. This is really valuable information, and gives you a potential leg up on your peers, especially if you’re able to use what you’ve learned.

And I’m not going to tell you not to try to earn some extra money either, but be aware, you’re going to need to keep your day job.

The Performance Zone is brought to you in partnership with AppDynamics.  See Gartner’s latest research on the application performance monitoring landscape and how APM suites are becoming more and more critical to the business.

security,bug bounty programs

The best of DZone straight to your inbox.

Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}