DevOps — Metrics, Goals, and Waste
DevOps — Metrics, Goals, and Waste
Barry Chandler, DevOpsDays London Organizer, reviews how DevOps is measured, as well as the Japanese lean engineering practices it evolved from.
Join the DZone community and get the full member experience.Join For Free
DevOps involves integrating development, testing, deployment and release cycles into a collaborative process. Learn more about the 4 steps to an effective DevSecOps infrastructure.
This article is about waste and how I discovered that the greatest cause of waste doesn’t come from what we do but from not having a common understanding as to why we are doing it.
It then goes onto introduce a seldom discussed Lean methodology that can address the root causes of this waste, the origins of which come from the same Japanese manufacturing industries that gave rise to the adoption of Lean thinking in today’s Enterprises.
Last, I explore the methodologies relationship with DevOps and why it’s important to acknowledge it.
A DevOps Journey – Sailing the Ship From “Measure Everything” to “The Goal of the Enterprise”
If it moves, measure it?
I became involved in DevOps in 2014 when our team’s agile transformation was in full swing. I had transferred a lot of responsibility to the team members which gave me more time to look at improving the way in which we work across my technology division in general. At the end of that year, Barclays initiated a group wide initiative to focus on Design, Lean, Agile and DevOps. Barclays recognized that the IT landscape had moved significantly and that they needed to change their processes accordingly.
Consequently, there was a desire to understand whether we (the various technical divisions) are successful at delivering change as a result of following the group wide initiatives, specifically aligning to the Investment Bank’s DevOps Transformation team’s priorities. The DevOps Transformation Team is responsible for setting a vision, the standards, and an approach that aims to make it easier for teams to adopt best practice whilst trying to ensure our infrastructure estate is manageable and consistent across the enterprise.
This led to a number of conversations on metrics, which isn’t a surprise given it’s one of the pillars of DevOps. Not the first time the phrase “If it moves, measure it” was referenced…a phrase I’ve never been comfortable with.
Clearly there are hundreds, perhaps thousands of moving parts, inputs, outputs, internal, and external factors that can influence a delivery process, let alone an enterprise operating in any sector. In addition to this, measuring human interactions is often open to interpretation and therefore far from an exact science.
Miller’s law recognizes that most people can hold 5 objects in memory, plus or minus 2. So that being the case: Why measure everything? Where do you draw the line? How do you know what’s of significance?
The extent to which we measure isn’t of principal importance. Having the ability to measure a vast array of parameters is desirable, but looking at all of them together is a different matter altogether and is more of a distraction than a provider of meaningful insights. Choosing which metrics to use is akin to picking your battles, focus on what’s important to you at that given moment in time and ignore the rest.
The best metrics-related one pager I’ve come across is the KPI Institute’s Key Performance Indicators Infographic1 which outlines metrics are related to SMART Objectives and have associated KPI’s that support them.
Fig. 1 Excerpts from: The KPI Institute – Key Performance Indicators Infographic 1
It’s the outcomes, realized by DevOps or otherwise, which are of primary importance. Second to that are the outcomes of the DevOps adoption for those organizations undergoing a transformation, something many organizations currently have in common.
The Goal of DevOps
So we are now talking in terms of outcomes which has to be a good thing. This leads me to think about the goal of DevOps and what it is. Asking our good friend Google, more often than not this can be summarized as “delivering better quality software cheaper and faster.” However, other people have a different perspective and look at it from an enterprise viewpoint. They describe the goal of DevOps as enabling organizations to realize their primary objectives, that being to maximize profit (in the private sector) or to provide the best services possible (in the public sector).
In my opinion, both are equally valid, albeit not expressed in a particularly smart manner. With that said, this doesn’t appear to be the complete picture. In my head I have Patrick Debois’s “Optimize the whole” diagram which expands the principles of DevOps (Culture, Automation, Measuring and Sharing) to all divisions or teams of the Enterprise.
Fig 2. Optimize the whole – Take from jedi.be blog – Codifying DevOps Area Practices2
This has always resonated with me, Technology being at the heart of the enterprise enabling the various teams to meet their objectives, or not as the case may be.
Following further research I realized that the goals of an enterprise map align to its hierarchy, starting with the overall goal, objectives, KPI’s, and measures. These are then broken down through the organizational structure and expressed to a lower level of detail, with each path ideally taking us from the over goal to those expressed across divisions such as Dev and Ops. Clearly this is a blue sky scenario and I doubt all enterprises are thinking about this. However, for DevOps it helps to position what we are trying to achieve and put it in the context of our organization.
At the time I was quite content knowing we had a better appreciation for what we are measuring and why — understanding goals should cascade through the enterprise. I still felt this was a fairly high level viewpoint of an operational model and there was clearly more to think about and observe.
A Night Out With Enlightenment
Earlier this year I was at a leaver's drinks and got the chance to catch up with a former colleague of mine called Andy. He’s the type of person who refuses to accept the status quo if he sees something which is not good enough. As a result, Andy would research specific subject matter to the extent I think he is the most well-read person I know. For those of you who have read The Phoenix Project3, which I hope most of you have, he has some similarities to Erik, albeit more hands on. At the end of the week we would always go to our local pub and along with a few other colleagues we would debate the state of our projects, team, Enterprise, and industry. Some people didn’t understand why we would take these conversations into our social lives but for us this wasn’t (and still isn’t) a job.
Before long we were discussing my DevOps role and my observations on a hierarchy of goals and subsequent measurements. Almost instantly Andy said what I was crudely describing what was the essence of Hoshin Kanri. He went onto explain Hoshin Kanri is a management approach originating from the legendary Japanese manufacturing industry, which also gave birth to many Lean concepts used to facilitate the transformation of today’s organizations so that they are better positioned to meet their goals. He continued to expand on its key disciplines, the conversation felt like completing a jigsaw puzzle, finally revealing the bigger picture.
The next day I began my own research and reading in an attempt to put it into context of DevOps and how it can help my organization.
A Whistlestop Tour of Hoshin Kanri and Why It’s So Important
There are several references to the meaning or western translation of Hoshin Kanri, all of which are variations on “A methodology for strategic direction setting”4. My favourite Hoshin Kanri metaphor is “a ship in a storm travelling in the right direction”5.
I discussed earlier that goals, objectives, KPI’s, and measures can be represented at different levels throughout the enterprise, starting at the top and cascading downwards, becoming ever more granular in the process.
This is the approach Hoshin Kanri takes but in addition to this, it stresses the need to make these drivers and measures visible throughout the enterprise. As well as exposing the organizational drivers from the top of the organizational hierarchy to the bottom (i.e. vertical alignment), this visibility provides an opportunity to co-ordinate and collaborate with teams you interact with (i.e. horizontal alignment).
The Hoshin Kanri’s process flow can be described in 3 phases repeated through the Enterprise hierarchy, each having multiple steps:
- Creation – The initial vision, goals, objectives, KPIs, and measures along with the tactics used to realize them.
- Communication and delivery – Conveying the strategic plans down and plan execution.
- Review – Monitor the plan execution, adjusting, and continually improving it.
Its benefits include:
- Enterprise Awareness – Providing clarity on the goals, objectives, and how they are measured so that everyone is aware whether the enterprise you are part of is heading in the right direction or not.
- Positive contribution – Having become enterprise aware, people are better positioned to establish whether their decisions are in the best interests of the organization (whether that be for initiating a specific project, hiring resources, introducing a new technology, prioritising a task etc.).
- Reduced Conflict – Increasing the likelihood of teams and individuals collaborating for the greater good by making it easier to establish their goals as well as the common goals of the teams you interact with are aligned.
- Combats siloed mentality – Collective understanding with regards to what the Enterprise is trying to achieve and how it intends to go about it is less likely to lead to the rise of silos following their own path.
- Strategic foundations – Changing strategic direction becomes easier as a result of the moving parts working in harmony (horizontal alignment as well as vertical).
Fig 3. The Target State: Achieving the goal of the enterprise by having vertical and horizontal alignment throughout.
Fig 4. A representation of an enterprise without vertical alignment along with silos operating independently of each other, not able to achieve the overall goal. There are plenty of variations on the theme.
Having looked at the benefits of Hoshin Kanri closely and spoken to many people in different enterprises, in different positions from Managing Directors downwards, it’s clear that virtually all of them face issues to varying degrees that could be addressed by implementing Hoshin Kanri.
Smaller organizations and start ups may face less of a challenge as they can rely on informal means of communication to keep moving in the right direction…the captain can shout across the ship’s deck and still be heard.
But this is not so on larger ships…greater discipline and focus is required to ensure the path is being followed as intended. That’s not to say smaller ships are more nimble and quicker. That maybe the case but it’s also possible to achieve agility and speed with these bigger vessels as well as benefiting from the size they bring.
One thing is for certain, you could be working within an enterprise that is ultra lean and has optimal delivery agility, but if the goals and objectives of the organization are not managed vertically and horizontally, the biggest source of waste is very likely to still be out there.
Is it as easy as that?
Some may think I’ve made the successful implementation of Hoskin Kanri sound trivial and they might be right.
Like any enterprise transformation there is a danger or trying to change everything at once and subsequently not achieving anything. It’s important to recognise how quickly you can change and set a direction accordingly.
Complacency can also be a significant risk factor that could impact a successful Hoshin Kanri adoption. This is not something you can do for a few months and then expect it to maintain its momentum, it needs to be a way of life, forever present and visible.
In addition to this, people’s behavior may change if and when different (better) objectives are being measured, causing some people to feel exposed and become defensive. I’ve seen similar behaviour when I introduced monitors to display delivery metrics dashboards (e.g. Jira burndowns, TeamCity build statuses, SonarQube code quality analysis) in the workplace. Initially a couple of people hid their data, making improvements before they were prepared for it to be seen by a wider audience. Whilst it wasn’t my intent to cause such a reaction, ultimately the effect was positive, and these individuals are now happier than they were before because their results are now transparent as well as meeting expectations.
DevOps is part of the solution but it cannot realize the goal of the enterprise in isolation
So We Now “Get” the Basics of Hoshin Kanri, What’s It Got to Do With DevOps?
To start, it plays a massive role in defining the culture of the enterprise with key values of visibility, transparency and continuous improvement at its core, the same as DevOps. In many ways, Hoshin Kanri provides a blueprint for a positive enterprise culture.
More often than not it is senior managers who engage in strategy and policy making activities, if these behaviors are present at the top of the organizations hierarchy, there’s a greater chance it will filter down alongside its goals, objectives, and measures.
The most significant similarity is the horizontal alignment paradigm. This is DevOps – teams collaborating together, optimising the flow of work in order to realise common goals in the best interests of the enterprise. I see DevOps as an extension of the horizontal alignment base class, containing attributes which are unique to Technology. This is also why we see DevOps in combination with Security (DevOpsSec), it’s another extension of horizontal alignment. Equally this can (and needs to apply) to other teams both within technology, outside, or a combination thereof.
Recognizing that the basis of DevOps can be modeled and mapped to a framework which has scope covering the entire enterprise helps us understand that focusing on DevOps in isolation might not reap any significant benefits in enterprise terms. For that to happen we need to start at the top and “Optimize the whole”.
Arriving At the First Port — Vive L’evolution!
I hope you have enjoyed our journey together and, like me, you feel that the principles and techniques of Hoshin Kanri need to be in place in order to realize the full benefits of Leaning processes, Agile delivery, and applying DevOps principles to the enterprise.
In all my years working in technology I believe we are witnessing the biggest changes in our industry, akin to a revolution in many ways. There are lots of examples: CX/UX approaches to provide the best experience from software, the drive to automate and commoditize, and the explosion of the open source movement which is challenging the long established software giants, etc.
However, when I picture the act of revolution in my head I see scenes from the musical Les Miserables. When I then think of revolution within an enterprise I don’t think this type of battle with the board members works and will land you in a lot of trouble!
In the context of the enterprise, everyone throughout the hierarchy needs to be pulling in the same direction, with visibility, transparency, and sustainability at the core of our new ways of working. I think the only way we can achieve the full benefits of Hoshin Kanri, Lean thinking, and DevOps is for enterprise evolution as opposed to revolution.
At Barclays, we are at the start of our journey. Agile thinking and delivery is bedding in, Lean approaches and DevOps are gathering momentum. We’re aware that there’s room for improvement, there always is of course, and I’m confident we’ll embrace these new ways of working to meet the goals of the overall group.
Next port of call: DevOpsDays London April 2016.
I would like to thank:
- Christoforos Nikitas, Nick Salt and Simon Paynter for reviewing this article.
- Christoforos Nikitas for being able to draw Fig 3. and 4. faster and better than me.
- Des and Pete for making me read The Goal all those years ago – Little did I know…
Published at DZone with permission of Barry Chandler . See the original article here.
Opinions expressed by DZone contributors are their own.