Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Benchmarking Release Management

DZone's Guide to

Benchmarking Release Management

Release management is essential is the digital age and helps deliver quality products to end users faster.

· DevOps Zone ·
Free Resource

Learn more about how CareerBuilder was able to resolve customer issues 5x faster by using Scalyr, the fastest log management tool on the market. 

It goes without saying that the backbone of all organizations today is digital, and independent of the end-products you provide, there is an inherent need to drive your features and solutions from conception (the initial idea) through to production (the end customer) as quickly, safely, and efficiently as possible.

That is, every organisation requires effective IT Release Management.

Image title

So, what is IT Release Management? Let us start with a definition (or two):

From Wikipedia:

  • "Release managementis the process of managing, planning, scheduling and controlling a software build through different stages and environments; including testing and deploying software releases."

  • "Enterprise release managementis a multi-disciplinary IT governance framework for managing software delivery and software change across multiple departments in a large organization."

Note: The former is what a DevOps engineer might gravitate to, i.e. someone with the task of building and deploying code to their individual system or services, and the latter something that might be handled by an IT Portfolio Manager (or Enterprise Release Manager), that is someone interested in coordinating a family of system releases (or project releases) using methods like SAFe “Agile Release Trains” to maintain collective velocity, phase alignment and a unified implementation.

A Visual Definition of Release Management

Alternatively, I might describe it using the following visual, a set of vertical practices that is responsible for supporting delivery through the system lifecycle (i.e. Analysis, Design, Development, Test and Implementation), a set of horizontal practices responsible for driving transformation across the enterprise (or divisions) and/or a “hybrid” mix of both.

Image title

Measuring Release Management

Independent of whether your perspective is vertical (system focussed) and/or horizontal (enterprise focussed), there is little argument in the importance of getting both activities completed correctly and having the various teams working together as effectively as possible.

However, agreeing on benchmarking may not be trivial as the two tasks, although obviously related, are quite different.  With the above in mind, I decided to put together two sets of “release” metrics that I thought would be useful in providing insight with respect to current behaviour and capability: the ten best metrics for Enterprise Release Management and the ten best metrics for System (Deployment) Release.

Note: Special thanks to some of our enov8 customers for sharing some of these.

Enterprise Release Management Metrics

Typically, the responsibility of an IT Enterprise Release Manager the main goals are to ensure that a family of project releases/changes, which are often related and contain dependencies, can move through the software development lifecycle, from conception to production, in a controlled and aligned fashion.

Key data and trends to be captured would be

  1. Size of Enterprise Release by Projects (Project Releases Registered)

  2. Size of Enterprise Release by System Deployments

  3. Size of Enterprise Release by Features (or Stories)

  4. Project Releases Status (Success/Failure)

  5. Project Releases Delivered on Time

  6. Project Release Delay

  7. Number of Project De-Scoped*

  8. Number of Systems De-Scoped*

  9. Number of Features De-Scoped*

  10. Root Cause of Project Release Delay

*Note: In ERM, de-scoping is most typically caused by a project failing to meet its release gate obligations and being removed from the enterprise release as quickly as possible and thus avoid an impending “train crash."

System (Deployment) Release Metrics

Typically, the responsibility of a DevOps “System Release” Engineer, the main goals are reliability and velocity i.e. the engineer want to get the job done as safely and quickly as possible.

Key system* deployment data and trends would be

  1. System Deployment Frequency

  2. System Deployment Feature (Story) Size

  3. System Deployment Status (Success/Failure)

  4. System Deployment Time (ideally versus SLA)

  5. System Deployment Outage Time (outage time for deployment itself)

  6. System Post-Deployment Outage Time (outage caused by unsuccessful deployment)

  7. System Post-Deployment Incidents (Tickets) due to a deployment

  8. Mean Time to issue detection

  9. Root Cause of System Deployment Issue (Delay, Incident or Outage)

  10. Mean Time to remediation

Note: In the case of DevOps you would expect these metrics to be of use across the lifecycle. Therefore, the word System could quite logically be replaced with System Instance, where you would have instances in Integration, UAT, Staging & Production.

Food for Thought

Hopefully you found some of the above release metrics useful & maybe thought provoking.

Ultimately metrics need to be aligned to your own operating model.

However, in closing I’d say this:

Good release metrics should always do the following:

  • Clarify release strategy, i.e. direct good practice
  • Understand existing release capability i.e. strengths & weaknesses and
  • Drive release enhancement i.e. continuous improvement

What else would you add to the above “Release Management” Metrics?

Further Reading

Find out more about how Scalyr built a proprietary database that does not use text indexing for their log management tool.

Topics:
release management ,devops ,metrics ,benchmarking ,software testing

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}