Are Your DevOps Metrics Aligned With the Business Objectives?
In this article, take a look at whether your DevOps metrics are aligned with your business objectives.
Join the DZone community and get the full member experience.Join For Free
An enterprise-wide Agile-DevOps transformation typically starts with a Vision followed by many items such as articulation of expected business outcome, a Transformation Roadmap, a set of Key Performance Indicators, an implementation plan, and so on. From the enterprise level these items get detailed out to the other levels from a Business Unit, Division, through the portfolio, program, and finally to the Agile-DevOps teams.
A key aspect of this translation is the alignment of the objectives at all levels with the stated business outcome. Business stakeholders and leadership are interested in the outcome which could include risk and cost reduction, higher market share, revenue increase, operational efficiency, and so on.
At the foundation level, this alignment manifests as a mapping between the team level metrics and the Program level KPIs. This article identifies the challenges in the alignment at this level and proposes a few tips to secure alignment.
Let us start with identifying the impact if the team level metrics and the program level KPIs are not aligned. There could be several repercussions because of this lack of alignment.
Enterprise DevOps transformations come with a certain cost that is arrived at using certain predictions, assumptions, and a high-level plan. One of the assumptions in computing the cost is that the organizational layers will align their respective objectives and action plans with one another eventually falling in line with the enterprise vision and objectives. However, if there is a break in the alignment during the transformation journey the Line of Business (LoB) or divisional leaders and their staff have to revisit their plans and rework on items that they would have already completed. This would often call for undoing certain things and starting all over again. Depending on the criticality and volume this rework could result in severe cost overrun eating away significant portions of the transformation budget.
Rework is time-consuming and it not only costs money but often results in delays as well. With a lack of alignment of program and team objectives and the resulting rework, the teams will have to change their DevOps strategy. For example, they may have to change their continuous integration/delivery/deployment approach to minimize the delay. This often leads to application misconfiguration and quality issues. As a result, the enterprise would have to face the reality of delayed entry into the market with newer solutions and features. This would potentially result in loss of revenue and market positioning.
The cumulative effect of cost overrun and delayed solution delivery is that the enterprise fails to meet the objectives of the DevOps transformation. This not only affects the morale of the leadership it also diminishes the confidence of the stakeholders. As a consequence, the corrective measures or future transformation initiatives would receive less budget and feeble commitment from them.
While the enterprise faces challenges of achieving the business objectives the staff at the ranks would lose motivation because of the rework due to lack of alignment of objectives. Often this results in employee turnover further affecting the DevOps journey. It would be an uphill task for the managers to pep up their teams to perform and retain key talent to work on the DevOps initiative.
What could be the reasons for the possible misalignment and how could we address these?
Here are a few.
- Vague Definition and incorrect mapping – Are the business outcomes defined without ambiguity? If in the first place there is a lack of clarity on what the enterprise wants to achieve through the DevOps transformation, the middle management and the ranks in the hierarchy will have a hard time articulating the end objectives. They may go with their own interpretations and probably in different directions.
To address this the enterprise should first consider their long-term business roadmap and their near-term priorities to define the objectives of the DevOps transformation. The leadership should articulate these in simple terms easy for everyone to understand. Based on these objectives they should determine the metrics that would be used to measure progress. They could use approaches such as Objectives-Key Results or Goal-Question-Metrics in this exercise. While the metrics could be specific to the context of the transformation, we have seen enterprises focusing typically on the following key metrics
- Deployment frequency
- Lead time to deliver
- Change fail rate
- Time to restore service (MTTR)
The leadership should then allow the programs to articulate these to the teams and define the processes and tools to track and report them.
- Lack of commitment from the program management and the Team – Has the Leadership done a good job at taking along the middle management in the DevOps transformation journey? Do the program management executives and the implementation teams see the value of the transformation and understand their specific roles in the journey?
Securing commitment depends to a large extent on the organization’s culture and the readiness of the teams to embrace the transformation. The leadership should include change management as a key component of the DevOps transformation. They should address the ‘What’s in it for me’ question and demonstrate their commitment to the transformation. They may also address the human resource aspects such as new roles, opportunities, and the impact of the transformation on the career path. They may consider a pilot to test their hypothesis, the metrics, the processes and tools and use the pilot results to secure a commitment from the ranks.
- Lack of tools – Does the enterprise equip the managers and their teams with the necessary tools for value mapping, planning, automation, metrics collection, analysis, and reporting? Does it have the right training and communication infrastructure to meet the needs of the transformation initiative?
Successful adoption of DevOps has shown that the teams have been enabled with adequate infrastructure and the appropriate tools to map their work with the objectives of the higher-level – program and portfolio. They have also been equipped with the tools to implement DevOps practices where the teams would start with automated code review, unit and integration testing, and progressively automate the other areas through build, continuous integration, continuous deployment. Eventually, they achieve automation of the entire delivery pipeline, automated infrastructure provisioning, and automated application monitoring.
- Inadequate Communication – Is there a clear communication channel to provide timely and adequate information to the staff involved in the transformation?
The objectives of the overall transformation and the program/LoB level objectives should be communicated precisely and unambiguously. This communication should not be just a one-time during the commencement of the journey. There should be a continuous communication flow re-stating the objectives and summarizing what is happening across the organization with respect to the transformation. The leadership could townhalls, newsletters, and broadcast mails and collaboration platforms such as Jive/SharePoint/MS Teams to communicate with the layers of the organization.
The leadership should use communication as a tool to drive changes in the Dev and Ops teams in order to
- Define their shared vision
- Collaborate better
- Share responsibilities among Dev, QA, and Ops in production releases
- Report progress and metrics throughout the lifecycle
- Address the dichotomy between Dev and Ops team related to ‘speed and delivery’ with ‘Stability and Reliability’
- Weak Governance – What is the governance mechanism for the transformation program – is it too rigid or is it just right with the right balance of control and freedom? How does the leadership get information on the progress of the transformation? Do they get the right data at the right time to make key decisions? Do they get information to validate that the outputs of the different groups are aligned with the business outcome?
Modern enterprises have demonstrated the effectiveness of lightweight governance to periodically review the Agile-DevOps transformation and measure business benefits. A typical governance framework is illustrated below.
This governance framework could be used to periodically review the transformation and get answers to the questions such as the following.
- Are we are focusing on the right outcome?
- Do the KPIs align with and support the measurement of the business outcome?
- Does the program deliver value at the defined milestones?
- Do the Agile-DevOps teams have the capacity and capability to deliver the business outcomes?
- What are the training/re-skilling needs, what are the tools/infrastructure needs?
- What is the feedback from the stakeholders on the output delivered by the transformation so far? Is there timely adequate communication with the stakeholders?
- What are the areas of improvement and how these improvement actions should be prioritized?
An enterprise embracing the Agile-DevOps transformation should make sure that the KPIs at the program and team levels are always aligned to the business objectives. To ensure this alignment the leadership should address the five aspects discussed above and support the middle management and teams to work towards delivering the expected business outcome.
Opinions expressed by DZone contributors are their own.