Benchmarking Release Management
Benchmarking Release Management
Release management is essential is the digital age and helps deliver quality products to end users faster.
Join the DZone community and get the full member experience.Join For Free
Learn more about how CareerBuilder was able to resolve customer issues 5x faster by using Scalyr, the fastest log management tool on the market.
It goes without saying that the backbone of all organizations today is digital, and independent of the end-products you provide, there is an inherent need to drive your features and solutions from conception (the initial idea) through to production (the end customer) as quickly, safely, and efficiently as possible.
That is, every organisation requires effective IT Release Management.
So, what is IT Release Management? Let us start with a definition (or two):
"Release managementis the process of managing, planning, scheduling and controlling a software build through different stages and environments; including testing and deploying software releases."
"Enterprise release managementis a multi-disciplinary IT governance framework for managing software delivery and software change across multiple departments in a large organization."
Note: The former is what a DevOps engineer might gravitate to, i.e. someone with the task of building and deploying code to their individual system or services, and the latter something that might be handled by an IT Portfolio Manager (or Enterprise Release Manager), that is someone interested in coordinating a family of system releases (or project releases) using methods like SAFe “Agile Release Trains” to maintain collective velocity, phase alignment and a unified implementation.
A Visual Definition of Release Management
Alternatively, I might describe it using the following visual, a set of vertical practices that is responsible for supporting delivery through the system lifecycle (i.e. Analysis, Design, Development, Test and Implementation), a set of horizontal practices responsible for driving transformation across the enterprise (or divisions) and/or a “hybrid” mix of both.
Measuring Release Management
Independent of whether your perspective is vertical (system focussed) and/or horizontal (enterprise focussed), there is little argument in the importance of getting both activities completed correctly and having the various teams working together as effectively as possible.
However, agreeing on benchmarking may not be trivial as the two tasks, although obviously related, are quite different. With the above in mind, I decided to put together two sets of “release” metrics that I thought would be useful in providing insight with respect to current behaviour and capability: the ten best metrics for Enterprise Release Management and the ten best metrics for System (Deployment) Release.
Note: Special thanks to some of our enov8 customers for sharing some of these.
Enterprise Release Management Metrics
Typically, the responsibility of an IT Enterprise Release Manager the main goals are to ensure that a family of project releases/changes, which are often related and contain dependencies, can move through the software development lifecycle, from conception to production, in a controlled and aligned fashion.
Key data and trends to be captured would be
Size of Enterprise Release by Projects (Project Releases Registered)
Size of Enterprise Release by System Deployments
Size of Enterprise Release by Features (or Stories)
Project Releases Status (Success/Failure)
Project Releases Delivered on Time
Project Release Delay
Number of Project De-Scoped*
Number of Systems De-Scoped*
Number of Features De-Scoped*
Root Cause of Project Release Delay
*Note: In ERM, de-scoping is most typically caused by a project failing to meet its release gate obligations and being removed from the enterprise release as quickly as possible and thus avoid an impending “train crash."
System (Deployment) Release Metrics
Typically, the responsibility of a DevOps “System Release” Engineer, the main goals are reliability and velocity i.e. the engineer want to get the job done as safely and quickly as possible.
Key system* deployment data and trends would be
System Deployment Frequency
System Deployment Feature (Story) Size
System Deployment Status (Success/Failure)
System Deployment Time (ideally versus SLA)
System Deployment Outage Time (outage time for deployment itself)
System Post-Deployment Outage Time (outage caused by unsuccessful deployment)
System Post-Deployment Incidents (Tickets) due to a deployment
Mean Time to issue detection
Root Cause of System Deployment Issue (Delay, Incident or Outage)
Mean Time to remediation
Note: In the case of DevOps you would expect these metrics to be of use across the lifecycle. Therefore, the word System could quite logically be replaced with System Instance, where you would have instances in Integration, UAT, Staging & Production.
Food for Thought
Hopefully you found some of the above release metrics useful & maybe thought provoking.
Ultimately metrics need to be aligned to your own operating model.
However, in closing I’d say this:
Good release metrics should always do the following:
- Clarify release strategy, i.e. direct good practice
- Understand existing release capability i.e. strengths & weaknesses and
- Drive release enhancement i.e. continuous improvement
What else would you add to the above “Release Management” Metrics?
Opinions expressed by DZone contributors are their own.