BSIMM9: A Decade of Software Security Science

The Building Security In Maturity Model (BSIMM) project turned ten this year, with ten years of careful observation of the best software security practices in real companies. BSIMM9, the ninth iteration of the report, describes the software security initiatives of 120 firms in detail. By this point, the BSIMM is widely recognized as a de facto standard for assessing (and then improving) software security initiatives (SSIs) through objective measurement. The report itself assembles and organizes the software security activities uncovered in individual firms and draws conclusions about best practices, how actual SSIs mature and evolve, and the state of software security within and across verticals. Unlike a prescriptive methodology, the BSIMM represents the software security activities performed by real-world organizations — not what they should do but what they actually do.

When an organization asks for a BSIMM assessment, we send a team of specialized consultants to conduct in-depth, in-person interviews with key security personnel from the software security group (SSG), relevant development teams, and the legal, compliance, training, threat intelligence, and incident response groups. Using the data we gather, we score the organization’s existing efforts in 116 specific software security activities organized into twelve practices. We can directly compare a particular firm’s measurement to the rest of the BSIMM population and draw important conclusions about software security maturity in the firm.

As the culmination of a decade of objective, observation-based work in the field, BSIMM9 stands out from past reports in three specific ways. In this short article, I will describe three newly uncovered themes that illustrate the direction of software security evolution.

Architectural Convergence and the Cloud

Over the past ten years, we have tracked several industry verticals found within the BSIMM population. For example, we have reported on Financial Services, Independent Software Vendors (ISV), and Technology firms since the days of BSIMM1. In BSIMM9, we see evidence of new convergence on a similar cloud architecture among ISVs, Internet of Things (IoT) companies, and Cloud firms — three of the BSIMM’s distinct verticals. Until recently, these firms had distinct tech stacks, software methodologies, and approaches to software security. To be sure there was some overlap, what we see now is convergence to a common architecture. You might say that cloud architecture is the tail wagging the software security dog.

What Does This Mean When it Comes to BSIMM Measurement?

With BSIMM9, we have added three new activities to the BSIMM model, which we will carefully track going forward. All three new activities are found in the software environment practice. Taken together, the three new activities serve to emphasize that software security in the cloud is becoming mainstream. Put simply, common cloud architectures require similar software security approaches.

The three new measurement activities were added to the already existing 113 in BSIMM8, for a total of 116 activities in a BSIMM9 measurement. Here are the activities as described in the BSIMM report itself. (Note, you can and should download the report and read about all 116 activities! See https://bsimm.com/.)

SE3.5: Use orchestration for containers and virtualized environments

The organization uses automation to scale container and virtual machine deployments in a disciplined way. Orchestration processes take advantage of built-in and add-on security controls to ensure each deployed container and virtual machine meets predetermined security requirements. Setting security behaviors in aggregate allows for rapid change when the need arises. Of course, orchestration platforms are themselves software that, in turn, requires security patching and configuration. If you use Kubernetes, make sure you patch Kubernetes.

We have been tracking containerization, DevOps, and associated software security mechanisms closely since BSIMM7. Now that virtual machines and container environments come in the hundreds instead of the tens, the orchestration is required to keep everything on track. Containers can be a boon for software security as long as they are kept up to date and properly functioning in a collective manner. This is something that leading Cloud firms have done for some time, but now other kinds of firms are catching up.

SE3.6: Enhance application inventory with operations bill of materials

A list of applications and their locations in production environments is essential information for any well-run enterprise (see [CMVM2.3 Develop an operations inventory of applications]). In addition, a manifest detailing the components, dependencies, configurations, external services, and so on for all production software allows organizations to secure all the things. That is, to react with agility as attackers and attacks evolve, compliance requirements change, and the number of items to patch grows quite large. Knowing all the components in running software—whether they’re in private data centers, in clouds, or sold as box products—allows for timely response when unfortunate events occur.

I have argued that the killer app for software security is an up-to-date inventory of all software (see this IEEE article). However, an app level view alone is no longer enough. Cloud architectures tend to be even more widely distributed than other kinds of Internet architectures, meaning there are pieces and parts of functionality spread all over the net. Keeping track of which parts of your applications are where, what their versions are, who produces them, and their basic software security state is essential. This goes double for open source components and libraries.

SE3.7: Ensure cloud security basics

Of course, you already do [SE1.2 Ensure host and network security basics are in place], right? Someone must ensure that basic requirements are met in cloud deployments as well. In the increasingly software-defined world, you must explicitly implement security features and controls (some of which may be built in) at least as good as those built with cables and physical hardware. Nothing is as automatic as it seems.

DevOps puts more of the “ops” in the hands of “dev” than ever. In the old days, machines “in the cloud” were controlled and managed by IT. In the new world, somebody has to do the controlling, installing, marshaling, patching, provisioning, and monitoring, and that somebody is the DevOps team.

Together, the three new activities described in BSIMM9 demonstrate the architectural convergence we are observing in the field. Ignore these activities at your own peril.

Adding the Retail Vertical

A new vertical has emerged in the BSIMM data pool — retail. This is not the first time we have added a new vertical (we added healthcare in BSIMM6), but it is the first time we have added a vertical and been struck by its initial maturity when it comes to software security.

Because of models like the BSIMM, which carefully describe real software activities carried out every day in real firms, firms just getting started in software security (say, after a massive data breach) can get up to speed more quickly than ever. In retail, as new models focusing on e-commerce become critical to sustaining a healthy business, software security comes right along.

Retail represents 10 firms of BSIMM9’s 120; its average SSG is only 3.2 years old and has fewer than eight full-time people. But as a vertical, retail already outperforms the average BSIMM population in the practices of Configuration Management and Vulnerability Management, Software Environment, and Architecture Analysis, while keeping pace in eight of the remaining nine practices — only compliance and policy lags as a practice. Retail’s stellar debut also shines a scary spotlight on the healthcare and insurance verticals, which continue to lag in software security.

In some sense, the BSIMM data allow us to “predict the future” when it comes to software security. We know what mature firms and verticals look like, and we know how they got there. So, we can copy the good things and avoid the known pitfalls while creating a new SSI. This is great news for both industry verticals and geographies who may currently be behind when it comes to software security. Very quick progress is not only possible, but it is also demonstrably happening.

Software Security Continues to Thrive and Grow

The most obvious result of BSIMM9 is that software security is thriving as a field. We know this because we have been tracking the field closely for a decade.

Not only do we have more data in BSIMM9 than we had in BSIMM8, but the scope of organizations we’ve assessed over the past year has also likewise grown. The BSIMM now reports on data from 120 firms. The number of developers whose work in BSIMM firms is described by BSIMM9 grew by 45 percent (from 290k to 415k). Even more impressive, the number of full-time software security practitioners in BSIMM9 firms (represented in both a firm’s SSG and its Satellite) grew by 65 percent (from 4769 to 7891). Want a stellar tech career? Pick software security.

In our view, BSIMM9 firms (and their Boards) are more aware of the resources needed to produce real software security results, not just software security theatrics. This is a good thing.

BSIMM9 incorporates and organizes the largest set of data collected involving software security anywhere on the planet Earth. By measuring your firm with the BSIMM measuring stick, you can directly compare and contrast your software security approach to some of the best firms in the world. You can find out where you stand, and then you can plan to move forward. Measurement is key.

You might have read an earlier version of the BSIMM. You might even have had a BSIMM assessment. Perhaps, you are a member in good standing of the thriving BSIMM Community (our big conference happens this month). Or, maybe you are new to the software security measurement party (in which case, welcome!). I encourage you to download BSIMM9 and consider how your organization can improve its software security initiative concretely — or start one if you don’t have one already.

Software security is never going to get easier, but by sharing what we know, learning from one another’s accomplishments (and missteps), and always building on existing best practices, we can make it better.