Who’s Buying DevOps Assessments?

What is a DevOps Maturity Assessment?

Assessments measure the keys to success that will help you improve. Assessments should deploy polling across all of your teams to gain valuable insight about DevOps maturity and those competencies that drive the most value for your organization. In addition to looking at the levels of maturity, they should look at the impact that each of these has in delivering value to your customers.

Industry experts have identified five key competency areas. These major competency areas are broken down into sub-competencies and measured.

The best approach is rooted in industry standard maturity models such as the Capability Maturity Model (CMMI). It can be extended for measuring software development maturity to DevOps by applying levels of maturity 1 - 5 for each of the major DevOps competency areas and each related sub-competency.

Key competencies areas include:

Automation

• Unit tests

• Test Automation

• Continuous Build

• Continuous Deployment

Continuous Improvement

• Key Performance Indicators (KPI)

• Service Level Agreements (SLA)

• Metric Tracking

• Remediation

Independence

• Isolated Configuration

• Code Dependencies

• Isolated Infrastructure

Operationalization

• Monitoring and Alerting

• Operations Knowledge Transfer

• Documentation

What It Does

The five key business metrics can be used to measure the results, but a similar set of measurements must be developed to measure DevOps itself. To do this it is important to define the capabilities that are critical to DevOps. These can provide a list of top-level competencies against which teams will be measured. To provide further detail in measuring these competencies they can be broken down into sub-competences.

By measuring DevOps in your organization, you set a baseline from which your teams can grow. In addition, this sets a standard that your company can work together as a whole. In addition, by taking an outcome-based approach and measuring at regular intervals you can ensure that, in addition to maturing, your company is delivering results.

How It Works

A Maturity Assessment measures DevOps implementation across the entire organization. This is for enterprises that need to create a competency model that reflects what DevOps is supposed to do for the organization. This can be accomplished by tracking, testing, correlating existing metrics and logs, and making this data visible through a DevOps Maturity Assessment.

With a good assessment and maturity model, you can help successfully drive a consistent DevOps model across mid-size and very large organizations. An understanding that there are many competencies which comprise DevOps allows you to track maturity in a consistent way across a large number of teams that are often not consistent in their practices.

By providing a mechanism and process to roll this data up into competencies and team scores you can simplify these numbers to make them easier to understand and track consistently across the organization.

Using an outcome-based approach and clearly measurable competencies you can successfully drive DevOps in large and diverse environments. By beginning with the end in mind you can ensure that improvements in DevOps competencies are driving towards the business results you want to achieve.

Buyer’s Profile

If you are looking to make investments in DevOps you’ll want the most bang for your buck. With an assessment, you can make the right investments at the right time, avoid frustration, and know where you stand.

Executives who are buying assessments don’t just measure DevOps maturity, they also want to measure the business metrics that are important. By understanding the business results you are trying to achieve, you can answer the “why” of DevOps. We can help you ensure that not only is your organization maturing but that that maturity is delivering tangible results.

We analyze five key business metrics which measure a wide range of business drivers. You should be seeing regular improvement in these metrics if your DevOps transformation is succeeding:

1. Business Agility & Speed to Market - as measured by Cycle Time

2. Infrastructure Investment - as measured by Capital Expenditure (CapEx)

3. Stability and Scale - as measured by availability

4. Customer Support - as measured by Customer Satisfaction

5. Product Quality - as measured by Rollback Percentage

Very often, someone who is new to the role wants to do an assessment. Typically they’ve been in it six months to a year. They are often charged with digital transformation and want to know what's going on.

Many buyers want to see assessments performed in a very hands-on way. They don’t want it to be a hands off consultant style exercise that comes across in theoretical terms. They want to know what teams had these results and here's what you should should do with its team. Buyers want hands-on, “face to face” and constructive and genuine.

Most buyers want the assessment was done in seven or eight weeks and able to start within a couple of weeks. Deadlines are tight and adhered to.

Enterprise’s Profile

In 2018, if your enterprise had more than 50 IT execs that need to be surveyed, you may have wanted to use a DORA online survey as part of your assessment. However, after the Google acquisition of DORA on December 20, 2018, DORA has stopped accepting new customers, but things may change by the end of 2019. Until then, don’t worry! You can still use a similar online survey that is used for less than 50 participants, but it won’t include industry data as comparisons. In both situations, also have a consultant do face-to-face interviews with IT execs that last one hour apiece. Don’t do groups — do them one on one. Also do a telemetry analysis analyzing ServiceNow tickets and any other data coming in off the infrastructure.

Enterprises that have the most to gain from assessments are at least mid-sized and most often very large. Smaller companies that are doing devops often have grown the capability from the ground up as they deployed their applications initially. Larger enterprises also have the most probability of having to find siloes that they need to break down.

Enterprises that are in the middle of building their DevOps teams often see a lot of value from assessments. They can best deploy resources where they can get the most bang for the buck.

A Use Case

We had one client who was just starting on their formal DevOps journey and saw that they had silos of data and really didn’t think they were doing much DevOps.

After the assessment, it turned that they were pretty much industry average. They weren't too bad! They expected to hear they were terrible so they had some low hanging fruit to improve on. The CIO is new and the VP of Infrastructure is new. Still, we came out with three things that were simple-to-implement things that'll show quick wins in six months.