When it Comes to Application Security, 'Doing Your Homework' Matters

They say software is eating the world, very true, but it has become even more clear that OSS components are eating the software world. This amazing revolution is driving unimagined gains in innovation and efficiency in our ability to deliver software. Think Uber, here is a new leader in the transportation industry without owning a single vehicle. Every major Enterprise and even most medium and small companies are software producers – and free and open software components are driving this dramatic shift in our world.

For several years, we at Sonatype have been messaging the idea that effectively managing your software supply chain is critical, the next great borrow from Edwards Deming and the manufacturing industry. Sonatype was ahead of the curve for sure, but the idea is now becoming mainstream as highlighted by recent Forrester and Gartner reports: DevOps and Supply Chain Management, and Sonatype DevSecOps.

How Do We Effectively Manage Our Software Supply Chain

So what drives effective supply chain management? At the core is accurate, timely, and comprehensive component intelligence. Without complete and accurate data, the fanciest reports and the slickest integrations have little value.

From a technical perspective, the best measure of accuracy and completeness is a fewer number of false positives and false negatives; a false positive is a known security vulnerability assigned to a component which does not have the vulnerability, a false negative is missing a component that has a known vulnerability. So let’s dive into both false positives and false negatives and illustrate how Sonatype’s Lifecycle Solution is so unique.

False Positives

It’s been my experience, that so many of the application scanning tools that scan and report CVE’s on components include a LOT of false positives – as much as 70%. A high false positive rate generates a tremendous amount of useless work, negates any attempt at automation, and may even undermine your entire compliance process.

Why do you think false positives are so prevalent? So many other solutions scrape data from public sites such as NSVDB, VULN-DB and other sources and do naive (but automated) things like “name-matching” or “fuzzy logic” to try to determine which component the vulnerability exists. Couple that with mistakes in the original vulnerability description from the public sites and incorrect fixes, and the result is many false positives.

The only accurate way to determine which component has a known vulnerability is to have an application security expert review the CVE and then examine the source code that contains the vulnerability, confirm the CVE is correct, and then precisely identify the component with the vulnerability. Time-consuming research is key here.

False Negatives

While false positives are clearly bad and create gross inefficiencies, false negatives have a far greater impact on the risk of your deployed applications. It only takes one false negative to enable a breach – and with the average breach costing 3.1 million dollars, this is no small thing.

But what are some of the contributing factors to false negatives? Here are a few data points I have seen in my work with many companies using various application scanning products.

Trusting That a Reported Fix to a Vulnerability Was Correctly Implemented

Let me give you one example, CVE-2014-0114 on Apache’s commons-beanutils. This supposedly affects all versions through 1.9.1; Sonatype’s research team reviewed the supposed fix that was put into 1.9.2 and determined that 1.9.2 and subsequent versions are still vulnerable when using the default configuration. So the fix is not sufficient by itself; and more importantly, using any version of this component without the required customization leaves your critical application vulnerable to an attack.

Assuming the Vulnerable Code Is Only in One Component

When Sonatype’s data team researches and confirms a vulnerable component, they check their entire eco-system for other components using this same vulnerable code marking them as vulnerable also. The most prominent example of this is the commons-collections vulnerability of 2015; Sonatype found thousands of other components with this same vulnerable code. What’s even more alarming, for this vulnerability, just the presence of any of these components in your class path makes your application susceptible to an attack.

Incorrectly Identifying Which Component Has the Vulnerability

Use of name matching or fuzzy logic always results in many false negatives (and false positives).

We Do Our Homework

Because Sonatype uses a “boots-on-the-ground” approach with a committed team of security researchers, going to the source code to identify the correct component and confirm any supposed fixes; and searching all components in the eco-system for the same vulnerability, we have the lowest false positive and false negative rate in the market.

Or quite simply, we do our homework.