Cloud Metrics: A Tool to Measure Success of Cloud Adoption

Introduction

As organizations are becoming bigger and more complex, it is becoming difficult to meet the business expectations and alignment of business and IT. Measuring the performance and progress of organizations, cloud adoption gives the cloud team an opportunity to identify areas where they need to focus more in order to ensure that the cloud center of excellence (CCoE) is functioning properly and delivering the business value.

Today, businesses are adopting the cloud as part of the transformation journey to improve business agility and customer relationships while delivering operational efficiencies. However, most organizations have no mechanism to measure the effectiveness of cloud adoption. Also, there exists very little guidance on the measurement of cloud adoption effectiveness within the organization. Most organizations focus on the definition, completeness of the cloud adoption, and maturity of the cloud development processes rather than the effectiveness of cloud adoption measurement.   

To ensure that cloud adoption is progressing well, and is also adding value to the organization, there is a need to define metrics for measuring its effectiveness, which is mostly done by the CCoE of the organization. However, cloud metrics are still an emerging field and there are no defined standards. Cost, time to market, ROI, and technology are the prominent areas that determine which workloads should be moved to the cloud. 

A CCoE is a cross-functional team that leads cloud Go/No-Go, cloud migration, cloud adoption, and cloud operations within an organization. The CCoE is responsible for developing and managing the cloud strategy, governance, and best practices that the rest of the organization can leverage to transform the business using the cloud. 

This article describes how cloud metrics help in measuring the progress of cloud adoption implementation and its maturity. It covers the benefits of having cloud metrics and also captures the approach used for defining the cloud metrics. It describes the cloud adoption metrics for measuring the progress of cloud adoption in the short term and for measuring the value delivered by the cloud in the long term.

Why Measure

Traditionally, CCoE has been measured on the immediate benefits that it provides to the organization. Although CCoE has a significant impact on how the IT organization aligns with business, if CCoE metrics are only limited to measuring business-IT alignment, they would fail to measure the long-term impact of cloud adoption and more importantly its impact on business as an enabler and catalyst of change. 

Based on the author’s interactions with various customers, the following are the practical challenges that we encountered in measuring the effectiveness of cloud adoption: 

• Effectiveness of cloud adoption is rarely assessed
• No established industry-standard metrics for measuring cloud effectiveness
• Cloud adoption context significantly differs between organizations
• Different stakeholders have different perceptions of cloud adoption
• Most cloud metrics aim to measure cloud value in IT terms — not in business terms
• Measurement data concerning cloud adoption often does not exist within the organization
• Objectives of cloud adoption change during the implementation path
• Most employees belonging to an organization do not know about the cloud initiative 

Cloud metrics not only help in gauging the long-term impact of cloud adoption by an organization but also help in measuring the true value of the cloud, which will enable the organization to plan more effectively and efficiently. 

Measuring cloud metrics are necessary to:

• Switch away from a CapEx intensive model to a subscription model
• Adopt a utility model and on-demand consumption vs. a one-time investment
• Free up resources from mundane technological activities to roles that contribute more value to the business
• Reduce TCO  
• Improve ability to seize new business opportunities
• Improve business agility

The cloud adoption success measurement should:

• Increase in revenue margin
• Improve ROI 
• Improve quality of user experience
• Improve elastic scaling cost 
• Improve level of automation
• Improve transaction latency and throughput
• Increase open source adoption 

For the cloud adoption program to be successful, it needs to be periodically monitored and measured with respect to a set of defined cloud metrics.  The status of these metrics can be captured, presented, and communicated effectively using cloud adoption scorecards or dashboards to the cloud governing body. 

Approach for Cloud Metrics Identification

Cloud adoption needs to be measured in a manner that is reflective of its impact on the business. Also, it is important to measure and monitor the progress of CCoE while cloud adoption is being implemented and until the time it reaches a level of maturity that delivers value to the business. 

Cloud adoption metrics estimate the progress of CCoE during the early stages of cloud adoption. It also helps in recognizing the efficiency and effectiveness of CCoE to ensure that the true value of the cloud is delivered to the business.  

For the CCoE to be successful, it needs to be periodically monitored and measured with respect to a set of defined cloud adoption metrics. The status of these metrics can be captured, presented, and communicated effectively to an organization's steering committee at regular intervals. Since cloud adoption goals and processes are very broad, YoY (year on year) measurement comparison for all cloud metrics would be a good indicator of cloud program penetration across the organization. The identification of cloud metrics will be done by adopting the CCoE value scorecard method (CCVSCM) which is shown below:

As shown in the figure above, the business strategy of the organization is the preliminary input to the cloud adoption metrics model, which consists of cloud adoption metrics inputs and planning inputs. These inputs would be utilized in all planning activities across the organization. Thus by measuring the planning effectiveness and efficiency, it would clearly represent the value of cloud adoption to business, as these planning activities have an impact on all five aspects of the organization (i.e. finance, customers, technology, processes, and governance). 

The most commonly used cloud adoption metrics for an organization are defined in the table below. These are purely based on the authors' experience. Again, these vary from domain to domain and customer to customer. 

Conclusion

Establishing a cloud center of excellence helps organizations keep complexity and misalignment at bay. But it is not inconceivable that CCOE may itself fall prey to this complexity trap if it is not well defined, managed, and maintained. Even if it is, it may be of no value to the organization if the well-defined artifacts are not used by the organization or if the cloud adoption processes are not adhered to.

To ensure that cloud adoption is progressing well, and is also adding value to the organization, there is a need to define metrics for measuring the progress and performance of cloud adoption. 

The cloud adoption metrics are defined in terms of effectiveness, agility, and alignment to business goals and strategies. Once cloud adoption is properly implemented and business entities are engaged, it will improve the overall planning activities of the organization and not just IT.