Cloud Metrics: A Tool to Measure Success of Cloud Adoption

Learn how cloud adoption metrics measure the progress of cloud adoption in the short term and measure the value delivered by the cloud in the long term.

Dr Gopala Krishna Behara

CORE ·

raju Myadam

Nov. 01, 21 · Opinion

Likes (1)

Comment

Save

11.3K Views

Introduction

As organizations are becoming bigger and more complex, it is becoming difficult to meet the business expectations and alignment of business and IT. Measuring the performance and progress of organizations, cloud adoption gives the cloud team an opportunity to identify areas where they need to focus more in order to ensure that the cloud center of excellence (CCoE) is functioning properly and delivering the business value.

Today, businesses are adopting the cloud as part of the transformation journey to improve business agility and customer relationships while delivering operational efficiencies. However, most organizations have no mechanism to measure the effectiveness of cloud adoption. Also, there exists very little guidance on the measurement of cloud adoption effectiveness within the organization. Most organizations focus on the definition, completeness of the cloud adoption, and maturity of the cloud development processes rather than the effectiveness of cloud adoption measurement.

To ensure that cloud adoption is progressing well, and is also adding value to the organization, there is a need to define metrics for measuring its effectiveness, which is mostly done by the CCoE of the organization. However, cloud metrics are still an emerging field and there are no defined standards. Cost, time to market, ROI, and technology are the prominent areas that determine which workloads should be moved to the cloud.

A CCoE is a cross-functional team that leads cloud Go/No-Go, cloud migration, cloud adoption, and cloud operations within an organization. The CCoE is responsible for developing and managing the cloud strategy, governance, and best practices that the rest of the organization can leverage to transform the business using the cloud.

This article describes how cloud metrics help in measuring the progress of cloud adoption implementation and its maturity. It covers the benefits of having cloud metrics and also captures the approach used for defining the cloud metrics. It describes the cloud adoption metrics for measuring the progress of cloud adoption in the short term and for measuring the value delivered by the cloud in the long term.

Why Measure

Traditionally, CCoE has been measured on the immediate benefits that it provides to the organization. Although CCoE has a significant impact on how the IT organization aligns with business, if CCoE metrics are only limited to measuring business-IT alignment, they would fail to measure the long-term impact of cloud adoption and more importantly its impact on business as an enabler and catalyst of change.

Based on the author’s interactions with various customers, the following are the practical challenges that we encountered in measuring the effectiveness of cloud adoption:

Effectiveness of cloud adoption is rarely assessed
No established industry-standard metrics for measuring cloud effectiveness
Cloud adoption context significantly differs between organizations
Different stakeholders have different perceptions of cloud adoption
Most cloud metrics aim to measure cloud value in IT terms — not in business terms
Measurement data concerning cloud adoption often does not exist within the organization
Objectives of cloud adoption change during the implementation path
Most employees belonging to an organization do not know about the cloud initiative

Cloud metrics not only help in gauging the long-term impact of cloud adoption by an organization but also help in measuring the true value of the cloud, which will enable the organization to plan more effectively and efficiently.

Measuring cloud metrics are necessary to:

Switch away from a CapEx intensive model to a subscription model
Adopt a utility model and on-demand consumption vs. a one-time investment
Free up resources from mundane technological activities to roles that contribute more value to the business
Reduce TCO
Improve ability to seize new business opportunities
Improve business agility

The cloud adoption success measurement should:

Increase in revenue margin
Improve ROI
Improve quality of user experience
Improve elastic scaling cost
Improve level of automation
Improve transaction latency and throughput
Increase open source adoption

For the cloud adoption program to be successful, it needs to be periodically monitored and measured with respect to a set of defined cloud metrics. The status of these metrics can be captured, presented, and communicated effectively using cloud adoption scorecards or dashboards to the cloud governing body.

Approach for Cloud Metrics Identification

Cloud adoption needs to be measured in a manner that is reflective of its impact on the business. Also, it is important to measure and monitor the progress of CCoE while cloud adoption is being implemented and until the time it reaches a level of maturity that delivers value to the business.

Cloud adoption metrics estimate the progress of CCoE during the early stages of cloud adoption. It also helps in recognizing the efficiency and effectiveness of CCoE to ensure that the true value of the cloud is delivered to the business.

For the CCoE to be successful, it needs to be periodically monitored and measured with respect to a set of defined cloud adoption metrics. The status of these metrics can be captured, presented, and communicated effectively to an organization's steering committee at regular intervals. Since cloud adoption goals and processes are very broad, YoY (year on year) measurement comparison for all cloud metrics would be a good indicator of cloud program penetration across the organization. The identification of cloud metrics will be done by adopting the CCoE value scorecard method (CCVSCM) which is shown below:

Figure: CCVSCM approach to identify cloud adoption metrics

As shown in the figure above, the business strategy of the organization is the preliminary input to the cloud adoption metrics model, which consists of cloud adoption metrics inputs and planning inputs. These inputs would be utilized in all planning activities across the organization. Thus by measuring the planning effectiveness and efficiency, it would clearly represent the value of cloud adoption to business, as these planning activities have an impact on all five aspects of the organization (i.e. finance, customers, technology, processes, and governance).

The most commonly used cloud adoption metrics for an organization are defined in the table below. These are purely based on the authors' experience. Again, these vary from domain to domain and customer to customer.

Customer Metrics
Objective	Metrics	CCOE Involvement	Benefits
Business value due to cloud adoption	Business process efficiency: Improvements made to the business process (quality, costs) through architecture initiatives. % of delivered capabilities versus proposed capabilities None of the business processes improved/re-organized	Enable simplified and standardized solutions that are modular	Improved reliability, quality, and efficiency
Application latency	Migrate more applications to cloud from on-premise. Improve response time	Design of cloud-native applications	Faster response compared to on-premise solution
Time to market	Improvement in decision making cycle (product/business) due to the alignment of business processes and goals with the application and technology components	Improve the business-IT alignment	IT is agile and meets the business needs quickly. e.g.: Quick introduction of new products/offerings into the market
Operational integrity	Impact of architecture initiatives on the operational integrity aspects: Application availability statistics Trending of high priority incidents	Facilitate the architecting & design of reliable solutions that are easy to operate	Reduces downtime of IT applications
Modernizing IT	Improve the traffic to website, better buying experience, retain the existing customers and better customer satisfaction	Modernizing IT, revamping legacy applications	Providing benefits to customers through a better user experience with low cost

Financial Cloud Metrics
Objective	Metrics	Cloud COE Involvement	Benefits
CapEx Reduction	CapEx reduction through the prevention of non-standard, one-off solutions Business Process Optimization % of processes reduction % of reduction in average time taken for process execution % in process efficiencies	Demonstrate the benefits of cross-functional initiatives	Avoid duplicate projects/investments
Optimize the resource cost	Cost of all resources tagged or untagged Cost per resource % of total bill charged back	Identify and consolidate all the resources that are contributing towards the cost	Ensure better resource utilization that is directly proportional to optimizing the cost
Enabling organizations to use latest technologies	Benefits from using the latest technologies (e.g.: reduce CapEx through the usage of SaaS, PaaS, IaaS solutions, and new technologies) Infrastructure consolidation Cloud enablement % of server consolidations % of infrastructure running on demand License optimization % reduction in licenses Adoption of open source	Innovation driving efficient investments through the usage of new technologies	Ensure better ROIs and minimum technical debts
Asset Reuse	Maximize the IT asset reuse (applications and servers): Number of shared services, sharing of server infrastructure, and application reuse None of the projects leveraging enterprise solution Cost reduction (due to reuse) Reusability % increase in reusable services across organization Sharing of server infrastructure Application reuse	Improve the IT asset reuse through better governance	Application and Server infrastructure reuse has a significant positive impact on the organization
OpEx Reduction	Portfolio simplification: OpEx reduction through the simplification of application and server portfolio Transparency of operating costs Application portfolio rationalization % of application decommission % of application consolidation % of legacy transformations Shared services % of FTEs shared across resources No. of shared services across organization Application health analysis % of reduction in ticket volumes Transformation initiatives Number of transformation initiatives executed within defined SLAs (on-time delivery, within budget, etc.)	Support rationalization to simplify the portfolios. Bring cost transparency to run IT	Portfolios are easier to manage and are less expensive to support

Technology Metrics
Objective	Metrics	CCOE Involvement	Benefits
Open-source technologies usage	Is the organization leveraging open source technologies?	Tools identification and usage	Reduce the workload of development and operations
Automation	What level of automation is currently in place at the organization? Workload automation Bursting Resource health	Provide immediate responses to events, such as configuration changes or sudden increases in workload	Reduce the manual workload of managing infrastructure
Service availability	Number of service interruptions Comparison of service downtime between cloud and On-premise environment	Helps in setting up more robust cloud environment	Arrive in better service availability

Process Metrics
Objective	Metrics	CCOE Involvement	Benefits
Regulatory compliance	No. of Instances that audit teams have consulted cloud repositories for legal, regulatory compliance documentation Percent of policies in compliant state (%) Cost-optimized over time ($) Cost-optimized per policy over time ($)	Support legal/regulatory compliance through cloud repositories/activities	Provide an insight into the IT controls that are in place
Resource optimization	Size of instances, scale on-demand Run them during off-peak hours	Deciding on instance and scaling. Removing old volume snapshots	Leverage automation to perform resource optimization
Risk Mitigation	% of compliance with security policies % variance in schedule Number of incidents related to unsubscribe Severity of exceptions		Used to drive preventive measures that can be taken to avoid potential risks generated from cloud computing adoption
Improved delivery model	Bimodal mode of service delivery and engagement with stakeholders Catalog-based self-service consumption with high automation applied in entire lifecycle Chargeback and billing with federated engagement and empowerment of consumers	Design and manage the cloud service catalog Define chargeback model	Streamline approval process and reduce deployment time from months to days to hours. Bring financial transparency with bill of IT

Governance Metrics
Objective	Metrics	Cloud COE Involvement	Benefits
IT staffing	What the current spending is on operations and development	Create a clear breakdown, to assess the impact of the cloud on organization workload	Reduce the operational overhead of running IT. Frees up the workforce to focus more on building and maintaining applications.
Training	Number of training courses provided on cloud migration and cloud-native adoption across the organization	Improve the cloud awareness across the organization	Improve the cloud adoption
Collaboration	What type of tools, knowledge, and technologies are supporting the cloud initiative across the organization	Promote the tools and technologies across the organization	Enhance the collaboration and eliminate the duplicate usage of tools and technologies
Senior management support	Assess the impact of cloud adoption across the organization How effectively KPI’s are met	Be part of the executive team of the organization and provide updates about CCOE to senior management	Provide updates to senior management about every stage of the cloud journey

Conclusion

Establishing a cloud center of excellence helps organizations keep complexity and misalignment at bay. But it is not inconceivable that CCOE may itself fall prey to this complexity trap if it is not well defined, managed, and maintained. Even if it is, it may be of no value to the organization if the well-defined artifacts are not used by the organization or if the cloud adoption processes are not adhered to.

To ensure that cloud adoption is progressing well, and is also adding value to the organization, there is a need to define metrics for measuring the progress and performance of cloud adoption.

The cloud adoption metrics are defined in terms of effectiveness, agility, and alignment to business goals and strategies. Once cloud adoption is properly implemented and business entities are engaged, it will improve the overall planning activities of the organization and not just IT.

Cloud computing Adoption Metric (unit) Measure (physics) application Open source IT Business process

Published at DZone with permission of Dr Gopala Krishna Behara. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

Trending