The $200 Billion Industry That's Causing Massive Global Disruption

Ten years ago, Amazon Web Services (AWS) invented a multi-billion dollar industry responsible for all Amazon profits today. This, in turn, helped Netflix to achieve a $156 billion valuation in just fifteen years, with a stratospheric growth rate of 77 times.

By any standard that’s an incredible achievement as according to McKinsey, one of the greatest obstacles to sustained growth is the ability to scale IT resources. However, with the availability of almost unlimited cloud-based computing from Amazon, Microsoft and Google, that is no longer an issue.

This article describes how this $200 billion industry is causing the biggest technology upheaval since 1981 when the IBM PC was launched. It will explain the business benefits of cloud computing, and the most common ways to get started.

What is Cloud Computing?

To put it in context, when you use the web you are connected to:

• Your device: A mobile, desktop or laptop computer.
• The local network: The immediate Wi-Fi network.
• The Cloud: The thousands of commercial services you use every day which are hosted on 3rd party cloud servers. These include Google Search, Gmail, Dropbox and Microsoft Office.

Cloud computing, however, refers to more than just simple data storage and utility services. It has been extended into the provision of general-purpose computing, and was invented almost by accident by Amazon in 2006.

How Did It All Start?

Twenty years ago, Amazon executives were frustrated that projects estimated at weeks to complete, took months just to provision the hardware. This included the network, compute resources and storage, as each team started from scratch.

In an attempt to resolve this, Amazon set up an internal division to provide standardized compute resources, delivering the world’s first Infrastructure-as-a-Service (IaaS). By 2006, it had developed into a viable business, and Amazon Web Services (AWS) was launched with two products, the Simple Storage Service (S3) and associated compute platform, Elastic Compute Cloud (EC2).

Initially appealing to small start-ups, just seven years later, AWS was a major player in the market, winning a $600 million contract for the CIA against competition from market leader IBM. By 2017, it delivered over $17 billion in annual revenues, nearly 10% of the entire Amazon group turnover.

"[AWS has] been radically disruptive to everything. HP is breaking up, IBM is divesting and retooling businesses, and Oracle is moving its revenue stream to the cloud as fast as it can". – John Furrier,  Forbes Magazine

AWS is Leading the Pack

The chart below illustrates how AWS currently dominates cloud computing, with a market share of 34%, which is greater than it’s next four competitors, in an industry which is growing at an annual 40% growth rate.

What’s the Benefit?

Two common misconceptions about cloud computing are that it’s not secure, and it’s all about reducing cost. If you believe either of these, you may be missing the point.

“Nearly all the high-profile breaches in the past year have targeted on-premises IT infrastructure. Maintaining in-house infrastructure is clearly not a guarantee of information security” – Tripp Smith – Clarity Insights

The benefits of cloud computing include:

• Agility: It’s possible spin up an entirely new hardware architecture from whiteboard to keyboard within a day. There’s no need to waste months justifying capital expenditure, and ordering and deploying hardware. As cloud computing is available on a pay-as-you-use basis, it provides a significant capability to experiment, and with it, the ability to adapt quickly to events.
• Innovation: One outcome of the potential for agility is the ability to innovate. To try out new ideas, test prototypes and quickly shut them down without significant impact if they don’t work out. This lets you fail fast, one of the key tenets of successful innovation.
• Scalability: As much an advantage to a small IT department as a fast-growing start-up, you can deploy a tiny server with a few gigabytes of storage, and quickly scale it to a massive supercomputer. As an example, the Snowflake Elastic Data Warehouse can scale from a single database server to a massive 128-node behemoth within seconds, without any disruption to service.
• Cost Control: With compute resources potentially charged by the second, and the ability to shut down servers when not needed, there’s every opportunity to control costs. As an example, the largest general-purpose AWS machine with 64 CPUs and 256Gb memory cost just $3.71 per hour, plus storage at $116 per month for a terabyte of SSD.
• Adaptive Technology: As cloud services are hosted and managed by a third party, service upgrades and new features are seamlessly delivered and automatically deployed. There’s no need to schedule downtime for system upgrades or software patch releases as these are transparently deployed.
• Operate locally across the globe: As the largest providers (AWS, Microsoft, and Google) provide data centers around the world, it’s possible to deploy a globally distributed solution. This means everyone from small start-ups to multinational enterprises, can start locally, and then expand globally.

“Invention requires two things: The ability to try a lot of experiments, and not having to live with the collateral damage” – Andrew Jassy, CEO,  Amazon Web Services

Although it’s understandable to think of cloud computing as simply the on-premise systems hosted by somebody else, it also brings the potential advantage of extending Agile DevOps methods to hardware provision.

Using an Infrastructure-as-Code process, it’s possible to quickly, deploy, test and adjust system configuration. This means, for example, it’s possible to quickly readjust the hardware to cope with an anticipated increase in demand, for example at month end or in preparation for a Black Friday sale.

Common Use Cases

Data Warehouse-as-a-Service (DWaaS)

Data Warehouse and cloud-based Big Data analytics services are available from a number of vendors. Often this is used for near real-time data ingestion, or to capture, store, and analyze massive data volumes.

While you could simply “lift and shift” to the cloud, the real benefits are available from the DWaaS providers. The clear leaders in this field are Snowflake Computing, AWS Redshift and Google BigQuery. Essentially, there’s no hardware or software to install, just load your data and run queries.

Software-as-a-Service (SaaS)

Often considered the top of the cloud food chain, Software-as-a-Service is an entire software solution delivered as a subscription-based service. In addition to the benefits of buy don’t build, it has the advantage of avoiding the on-going cost of maintenance, as applying upgrades and new features are automatically deployed.

The best examples of SaaS include Salesforce, the customer relationship management system, and Microsoft Office 365, in addition to niche products like Slack for enterprise cross-platform collaboration, and the Intuit suite for financial accounting.

Backup-as-a-Service (BaaS)

Often one of the easiest ways to start, effectively this uses the Agile and elastic nature of cloud computing to provide backup and recovery services. AWS provide a range of solutions including supporting applications on-premises, entirely in the cloud or a hybrid of both.

Disaster Recovery-as-a-Service (DRaaS)

Especially valuable to SMEs, this provides the possibility of fail-over to a cloud-based disaster recovery solution without the expense of building and maintaining another data centre.

Desktops-as-a-Service (DaaS)

Virtual desktops are quickly replacing desktop devices as a fast-emerging way to support a more Agile way of working. In addition to supporting a hot desking method of working, it means IT departments can efficiently standardise content and security patches across the entire estate.

Cloud-based virtual desktops are the next logical step, and provide an additional level of disaster recovery, as the desktop is available from any device, anywhere there’s a web connection. Prices are charged on an hourly or monthly rate, and start as little as $29 per month for a Linux based machine.

Infrastructure-as-a-Service (IaaS)

If correctly implemented using an Infrastructure-as-Code process, it’s possible to quickly configure, deploy, test and remove a highly resilient and secure hardware setup within hours. In addition to avoiding the lengthy budgeting, planning and inevitable delays in installing new hardware, this converts a capital investment into an adaptable operational expense, and helps smooth cash flow.

Conclusion

Having worked in Business Intelligence and Data Warehousing my entire career, the benefits of cloud-based Data Warehousing are clear. The ability to quickly scale up the hardware, and securely store almost infinite data volumes, make the cloud the perfect location for Big Data analytics.

However, I’d argue the benefits of cloud deployment extend across the entire IT and business landscape.

The ability to quickly deploy, test, and reconfigure both hardware and software is a game-changer. It allows hard-pressed IT departments to truly innovate. To try new technologies, to prototype new solutions, and quickly build and tear-down full-size pilot or test systems. It’s enabled Amazon, a company that started less than 25 years ago, to become larger than all its bricks-and-mortar competitors combined, and to compete with Apple to become the most valuable company in the world.

A final word of caution. In the same way Amazon, Netflix, Airbnb, and Uber disrupted their respective markets by being Agile and innovative, the advantages of extreme agility from cloud deployment are causing disruption in almost every industry, and the cloud service providers are at the vanguard of that change. If you don’t adapt to the new reality, it may be your market that’s disrupted.

“[The] best time to deploy a public cloud strategy was two years ago, the second-best time is now.”  Tripp Smith, CTO, Clarity Insights

Disclaimer: The opinions expressed in my articles are my own, and will not necessarily reflect those of my employer (past or present) or any client I have worked with.

Article first published on www.Analytics.Today as "The $200 billion industry that's causing massive global disruption."