Welcome back to my blog series, the CISO’s View. In my last article, CISO’s View: metrics part 1, we started looking at metrics and why they are the foundation of a successful security program. Today, we’ll look at how we derive metrics that communicate value in a way that’s tied to the company strategy. Hopefully, by the time we are done, you too will agree that metrics are the foundation of a successful security program that is viewed as a strategic business enabler.
Integrating Security into the Company Strategy
Okay, so where do we start? The shift in culture begins with actively participating in helping to develop the company strategy showing how risk management practices both help to achieve the strategic goals and provide an up to date measure on the execution against that strategy. Risk is an interesting word that can have different nuanced meanings. The normal use of risk is the probability and impact that something we care about will suffer loss. This is typically measured using several forms of predictive analytics. For this article, I am using a more general definition of risk as the measure of uncertainty of outcome. This is useful in the context of business strategy as it allows for a formal methodology of setting and measuring execution against business strategy – both the positive and negative effect of uncertainty on that outcome. If your methodology only focuses on loss then you miss the ability to execute on those happy emergent opportunities from positive uncertainty identification. For further reading, check out the OpenFAIR framework to learn more about risk modeling.
External Inputs to Company Strategy
Strategy starts with the external inputs and pressures on the company in its drive to compete in the marketplace and provide value to its owners or shareholders. This shouldn’t be anything new, so please allow me to state the obvious. These inputs can take the form of market dominance, profitability, revenue needs, etcetera, and become the basis driving corporate strategy. Our job is to be engaged and knowledgeable about these inputs and engage in shaping strategy.
Strategy development is finding the way to meet these external inputs. The focus of this article is how to apply a formal process to these strategies to measure execution and measure the uncertainty in meeting those strategic goals. We need to determine the scenarios that lead to achieving each strategic goal, break down each scenario into the discrete events that lead to a successful outcome and identify events that could impede or prevent the successful outcome.
Each scenario’s events are then modeled using the predictive risk analytics that determines the points of uncertainty surrounding each event remembering to capture both positive and negative outcomes of uncertainty.
Decomposition into Company KPI & KRI
These models get decomposed down to the specific measures in the predictive risk equations and translated into the first level of business metrics expressed as Key Performance Indicators (KPI) and Key Risk Indicators (KRI). These KPIs and KRIs become the basis of tracking execution against strategy and insight into the factors that can impact success.
Key Performance Indicator
This is an actual measure of something. There are two main types of performance measures, the positive measure of progress to completion of the specific events that lead to the completion of the goal, and the negative measure of the level of effort expended in working on the specific events.
Key Risk Indicator
This is the measure of the uncertainty surrounding desired event completion. Again, this is both a predictive analytic score and the degree of measured deviation or uncertainty inherent in the KRI score.
KPI & KRI to Technical Control Metrics Integration
Okay, now that we’ve determined what to measure in a manner that is tied to the company strategy, we need to figure out how to measure it. This is identifying the business processes and applications that are involved and determining what data is available to measure against. This should feel familiar, as the data in question is what you’ve probably used to date. The difference is instead of expressing technical metrics such as vulnerability remediation rates, you use the same data to show it applies to the identified KPIs and KRIs.
Technical Metrics Automation
This is the most overlooked step in a successful metrics driven security program. You’ve done the work figuring out how to communicate your security metrics in a manner that communicates the value your organization provides and found data that can be used as the source of those metrics. Now what? You’re probably looking at all the work to generate the metrics and either decided it’s not worth the effort or decided to generate the metrics on some infrequent basis such as quarterly or annually.
Automation is Critical
Invest in an enterprise initiative to build a board level reporting dashboard that automates the collection, generation, and reporting of these metrics. Looking back at what we’ve discussed, you’ve probably realized that nothing here is security specific – these same challenges are faced by all IT and business areas. This reporting platform can drive massive efficiency gains – bottom line value – across the enterprise to justify its initial implementation and will continue to show its value in rightsizing spending on what is considered cost center keep-the-lights-on funding.
The goal is to require every new project initiative to consider the data it generates, how that data can be leveraged for metrics, and the method of automated collection into the reporting platform. Every project seeks funding by promising to deliver some value to the business; the reporting platform is a formalized way of actually showing the value delivered.
Okay, that was a bit of a deep dive today on the how behind metrics. Part 3 ties everything together showing how we can communicate to the business and leverage them in a successful security program.