Great conversation with Rob Malnati, V.P. Marketing, and Pete Mastin, Product Evangelist, at Cedexis who just returned from the Content Delivery Summit.
Content delivery is changing as the number of content delivery networks (CDNs) and cloud hosting services continue to grow. Web pages have grown exponentially. It is not surprising to hear that a single web page has more than 300 round trip calls to the server. Likewise – the growth in software downloads has soared.
All of this bloat has happed at the same time users have become more impatient than ever about the speed at which they are serviced on the web. Everyone has seen the stats – if a page is slow to load – people leave. Site bloat has made it increasingly difficult for content providers and broadcasters to achieve great performance for end users.
It is increasingly popular to use multiple Origins for a web site to improve user experience (UX). A single data center is no longer the norm. CDNs strive to have vendor diversity in order to optimize the end UX, but this is often insufficient. Modern architectures are broadly distributed. These applications are loosely coupled and require a new breed of tools. Many media and gaming companies like PBS and ROBLOX are opting to use multiple CDNs to gain greater improvements in UX.
In the past, when you had your own data center you could have five nine redundancy. Today everyone builds for failure.
When a user base has significant geographic dispersion, it can cause problems. If you are running on a cloud that happens to be in Virginia, your user in Maryland may be happy but users in California may not be. But geographic location is not the only part of the story. Peering, congestion and routing also hinder performance.
Furthermore, Internet connections and peer services are not guaranteed. Thousands of streams can overwhelm connections. User-generated content (UGC) like YouTube is a long-tail catalog. If something is popular it is deployed to many servers. Long tail is typically not in the caches of CDNs and you have to go back to the origin to retrieve it. New content does not fit well with existing architecture. We need to look at new architecture (e.g. CDN with multiple origins – hybrid CDN) delivering multiple types of content from different origins.
Cedexis likes to say they “make the unmanaged internet managed” by making tools that enable their clients to monitor and optimize performance throughout deployment and delivery. This is critical for developers, as they need to know the delivery performance of all the applications they are developing.
Given the growth of cloud solutions and the fact load balancers don’t transfer from one cloud to another, evolution is necessary.
Cedexis has built and maintains a next generation Global Traffic Management (GTM) system that defines load balancing for the 21st century. Back in the old days (10 years ago), people used to just load balance local servers in a data center. This was a single origin with no failover. The algorithms to do this load balancing was simplistic – typically “round robin” where the servers were given traffic one after the other.
Then as time went on, the need for a primary and secondary locations (remember disaster recovery?) came about – and load balancers had to be improved to meet these new needs.
Then – as the desire for an “Active-Active” configuration came about – again – GTM had to be upgraded once again. New methods for distributing traffic had to be invented. The most popular was geographically based traffic management – sometimes called “Geo” for short. This type of traffic management attempted to determine the location of the user requesting the service – and then route that user to the servers closest to him or her. A typical case was to have servers in a cloud or data center in California and others in Virginia. Everyone east of the Mississippi River was routed to Virginia and everybody west of the Mississippi was routed to California.
There are many problems with this model. While it’s certainly better than round robin there are many cases where peering, congestion and micro-outages on cloud services make this a terrible choice. If a service is having an outage – using Geo you could be routing traffic directly to a dead service. Congestion between middle mile carriers can also cause latency issues even when the servers are physically closer to the users.
Because of these many issues – technologists have been driven to “performance-based load balancing” for their services. What this means is that measurements are taken of the cloud or CDN services and then traffic is routed to the best performing cloud or CDN.
The measurements usually consist of latency, throughput or availability of the cloud or CDN services in question. There are two ways these measurements can be made – synthetic measurements or Real User Measurements.
Synthetic monitoring can be deceptive since it’s based on measurements from a well connected data center and does not take into account data from the last mile of delivery. For this reason, Synthetic measurements should be avoided.
Real user monitoring (RUM) to all public and private infrastructure users at the ISP level is needed to optimize the UX. What is needed is a global traffic manager (GTM) that will give you visibility into multiple clouds and across an infinite number of networks that make up the internet.
A GTM, like Cedexis, is able to take billions of measurements to provide a clear picture of how data is flowing over the last mile. This data is crowd-soured from more than 950 contributing websites. More data means more insight into the activity on the network and the ability to predict problems and route around them before a roadblock happens.
Everything comes down to improving the UX. Everything you do in DevOps needs to focus on the end user and using the cloud to deliver the best UX. Look for a GTM to help optimize UX for your end users.