Mobile DevOps Metrics that Matter

Creating a secure DevOps culture helps companies accelerate mobile app release cycles and securely deliver the new features and capabilities that users crave. Automating the continuous integration/continuous delivery (CD/CD) pipeline speeds time to market to meet the demands of the business.

But app store ratings and reviews aren't the only important measures of performance. As more NowSecure customers embark on the journey to DevOps, they increasingly focus on a few key performance indicators.

Mature DevOps shops use metric-driven scorecards or dashboards to track their progress and drive continuous improvement. These metrics generally support two main categories of improvements to the mobile app software development lifecycle (SDLC): speed and quality.

As with all things DevOps, the goal is to automate everything and let systems do the work, which in turn improves performance. What follows is a guide to some of the metrics NowSecure and our customers recommend adopting as part of a secure DevOps discipline.

Speed Demon

DevOps aims to shrink release cycles to get mobile app updates into the hands of users sooner. Whether the goal is to release apps weekly, daily, or even multiple times per day, speed coupled with quality is the name of the game. Here some key metrics related to velocity.

Cycle Time Value (CTV)

CTV refers to the time it takes to go from commit/build to production, spanning all the steps of build, test, stage, and push to production.

Why It Matters

Slowing down the cycle with last-minute security testing and/or lengthy security assessments creates friction for developers and the business. With DevOps teams driving toward a CTV of three hours or fewer, security often causes the biggest delay. Manual testing alone takes hours, days, or even weeks. The more you can automate mobile appsec testing earlier in the cycle, the better. Break the build if necessary, but instead focus on automatically security testing your new code and build every day to reduce CTV and eliminate manual errors.

Mean Time to Resolve (MTTR)

 In DevOps parlance, MTTR measures how long it takes to resolve an issue, whether it's an incident, functional bug or security bug.

Why It Matters

Time is money, and wasting time on false positives or complex issues that are difficult to find and fix frustrate dev teams and slow the machine. Tap testing tools that pinpoint where security bugs exist and provide remediation instructions so they can be fixed quickly.

Dwell Time Value (DTV)

In the DevOps world, DTV is the length of time a breach spends in the wild before it's detected and remediated. The longer it takes, the more damage the hacker can inflict.

Why It Matters

Dwell time directly correlates with defect escape rate (DER) as described below. Each escaped security bug expands the attack vector which can lead to lengthy breaches. Taking shortcuts in security testing to rush release increases risk. And worse, pen testing only one release per year means each release that contains new code could be at the risk of an undetected breach. Regularly continuous security testing reduces the chance of vulnerabilities and speeds resolution to shrink DTV.

Quality Control

Speed doesn't mean anything unless DevOps organizations balance it with quality, and everyone involved in software delivery has a role in its success. What follows are a few metrics focused on quality.

Error Rate Value (ERV)

The error rate refers to the number of bugs or vulnerabilities per lines of code.

Why It Matters

With the goal of delivering high-quality software, DevOps seeks to minimize errors to maintain optimal productivity. Automated tooling at all levels reduces human error and process gaps. Providing developers with security training and automatically testing new lines of compiled code each day can minimize the error rate and help drive continuous improvement.

Defect Escape Rate (DER)

The DER tracks how many defects and bugs make it to production.

Why It Matters

It's easier to fix a bug before it reaches production, and cheaper, too. IBM research shows it can cost 100 times as much to fix a bug in production than catching it during development. Defect ratios tend to be high when people rely on manual pen testing performed once per year versus automated security testing. Zeroing in on DER helps you gauge the effectiveness of your security testing program.

App Risk Score

Our company's unique NowSecure Risk Score is a proprietary algorithm computed from CVSS-scored findings in our test engine that enables organizations to baseline the relative risk of every single security test and track it over time. Our solution conducts static, dynamic and behavioral testing on real mobile devices.

Why It Matters

Many of the other metrics cited here use a single number to track current state, history, and trends over time to drive visibility and continuous improvement. But until now there hasn't been an easy way to gauge security and risk. Now DevOps and security teams can achieve that visibility with the single NowSecure Risk Score. Armed with this metric, organizations can drive improvement to better protect customer data and corporate reputation.

For more insight into DevOps and how to take a phased approach to the transformation, consult our "Phased Approach to Securing DevOps for Mobile Apps" ebook for guidance. And by all means, hit up @NowSecureMobile on Twitter to share what DevOps metrics are top of mind for you or humblebrag about the ones you're most proud of your company achieving.