Big Cloud Still Runs Most Containers on VMs; What Does that Mean for the Rest of Us?

If bare metal provides the best raw performance, why do hyperscalers still insist on running their own infrastructure on virtual machines? The answer reveals what the companies running the world’s most complex infrastructure really think about cloud architecture.

Research by the analyst firm ReveCom shows how the major cloud providers overwhelmingly deploy their containerized workloads on virtual machines rather than on bare metal servers. In addition to relying primarily on VMs to support their in-house operations (with the exception of Google), they also rely on VMs instead of bare metal to support the services they offer customers. Those findings are based on reviews of documentation and interviews with engineers and executives at AWS, Google, Microsoft, and DigitalOcean.

ReveCom reports that the major cloud providers opting for VMs instead of bare metal for their containerized infrastructure serves as a case study that has significant implications for enterprise IT teams. If the companies that pioneered modern cloud infrastructure rely on virtualization, the technology should be more than sufficient for organizations running private clouds and hybrid environments.

The Performance Gap Has Closed

The concept of running containers on bare metal gained prominence roughly a decade ago. Part of the rationale was performance optimization: containers running directly on servers could eliminate the virtualization overhead, resulting in better throughput and lower latency. However, this approach introduces added complexity, since bare metal usually requires more manual management than virtualized environments do and is not conducive to scaling.

Since then, more recent results indicate that containers over VMs are catching up in terms of performance, with netperf benchmark tests showing that instances of VMs retain 99% of the performance compared to bare metal, according to ReveCom data. Meanwhile, there has long been a consensus that VMs offer more direct operational control, isolation, security, and other benefits compared to bare metal.

Modern hypervisors are highly efficient. AWS developed Nitro and ESXi specifically to mitigate the performance penalties of virtualization. By transferring host management functions to dedicated custom hardware, Nitro delivers a lightweight hypervisor with performance nearly identical to bare-metal performance. VMware demonstrated near-bare-metal performance for virtualized AI/ML with VMware Cloud Foundation. 

This proves that virtualization and high performance are no longer mutually exclusive.

And ReveCom's findings support this conclusion. In most cases, container performance on virtual machines now matches or closely resembles that of bare metal systems. Its marginal raw performance advantage over bare metal rarely justifies the added operational complexity.

AWS Defaults to Virtualization

Amazon Web Services hosts most of its containerized workloads on virtualized hosts. Bare metal deployments are targeted and performed only when direct hardware access is essential.

This includes testing custom silicon, supporting specialized hardware features, and meeting certain regulatory or licensing requirements that mandate physical isolation. But these are the exceptions, not the rule.

By default, virtualization is enabled for both the AWS control plane and customer workloads. Multi-tenancy, isolation, and fleet management requirements favor the use of VMs. Economics also favors virtualization. Consider the overhead if every Kubernetes cluster instance were running on dedicated bare-metal. Data center sprawl would be unsustainable and too expensive.

Microsoft Teams Runs on Virtual Machines

And Microsoft is mirroring AWS's approach. Kubernetes deployments supporting Microsoft Teams and Microsoft 365 primarily rely on virtual machines, according to ReveCom research.

VMs offer complete elasticity with straightforward lifecycle management, Sean McKenna, director of product, Azure Cloud Native, told ReveCom. Dynamic scaling to meet changing needs requires minimal operational overhead, McKenna said.

Day two operations, such as cluster upgrades, also benefit from the VM model. The process involves provisioning new virtual machines on the target Kubernetes version and disabling the old ones. Simplicity and scalability make VMs the default choice for most workloads, according to ReveCom research.

Microsoft's McKenna acknowledged that some scenarios favor bare metal. High-performance computing and specialized AI workloads sometimes require direct hardware access. Some services use a hybrid model, combining both approaches. Yet virtualization remains the basis for the vast majority of Microsoft's containerized infrastructure.

Google's Dual Strategy

Google's internal infrastructure highlights the company's dual strategy, where VMs are the primary approach, with one notable exception.

Borg, Google's core container orchestration system, runs on bare metal. That system predates modern virtualization and manages a substantial portion of Google's internal compute environment. Thus, the architecture is effectively grandfathered in, and Google has no plans to virtualize it. Everything else operates within virtual machines (VMs). Most Google clusters rely on virtualization, with containers hosted on VMs for workloads outside the core Borg infrastructure. Key services such as Gmail and YouTube also utilize this VM-based setup.

For Google, virtualization outperforms bare-metal in almost all cases. VMs enable a hybrid approach, in which sensitive environments or legacy systems receive additional isolation.

DigitalOcean Practices What It Preaches

For its internal platform, DigitalOcean uses VMs. Most internal workloads run as containers on those VMs. Internal applications, such as the Gradient GenAI Platform, run on the App platform within the Virtual Machine Layer.

DigitalOcean vice president of engineering Archana Kamath was clear when speaking to ReveCom about the preference. For all internal applications and use cases, the company uses VMs. Bare metal isn't an option. VMs are easier to use and maintain, she said.

This VM-based architecture also matches what customers deploy. VMs support familiar cloud constructs with predictable operational behavior. While customers seeking maximum flexibility can use bare metal environments, DigitalOcean states that such environments require deep expertise in kernel management, networking, and security. 

Recovery and Operations Become Difficult With Bare Metal

VMs support fast node recovery and multitenancy. The operational model for Kubernetes/containerized workloads is much simpler. DigitalOcean and many of its customers prefer Kubernetes on VMs and containers.

What Does That Mean for Enterprise Teams?

The hyperscalers have made a clear strategic choice. They rely on virtualized infrastructure for their essential services because virtual machines typically provide greater overall benefits compared to the marginal performance gains of bare-metal systems.

Security and isolation are the primary concerns. VMs define boundaries between workloads that containers do not. Operational scalability favors virtualization. Provisioning, scaling, and lifecycle management become easier with VMs serving as the foundation.

For DevOps and platform engineering teams evaluating their own infrastructure strategies, the data points in one direction. VMs remain the default method for running containers on-premises and in the cloud.

Bare metal still works. Performance-intensive workloads may justify the additional operational complexity. For most use cases, however, the disadvantages outweigh the benefits. If virtualization works for the giants working on the biggest scale, it will work for everyone else.