DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
What's in store for DevOps in 2023? Hear from the experts in our "DZone 2023 Preview: DevOps Edition" on Fri, Jan 27!
Save your seat
  1. DZone
  2. Software Design and Architecture
  3. Microservices
  4. Creating a Self-Routing Varnish Cluster

Creating a Self-Routing Varnish Cluster

We take a look at how to create a self-routing Varnish cluster without the need for a dedicated routing tier, meaning you don't need a a load balancer or app router.

Reza Naghibi user avatar by
Reza Naghibi
·
Feb. 16, 17 · Tutorial
Like (1)
Save
Tweet
Share
7.52K Views

Join the DZone community and get the full member experience.

Join For Free

In this article, I will explain how to create a highly available, self-routing sharded Varnish Cache cluster. This is similar to a standard sharded cluster with one exception, there is no dedicated routing tier. Each node in the cluster can route the request to the proper destination node, by itself.

From an architecture standpoint, having a self-routing cluster is an interesting proposition because it means no load balancer or application router is needed. All we need is DNS. Routing logic, which before had a dedicated piece of hardware or software, is now distributed across all participating nodes.

Self-Routing Architecture

In this example, we have three nodes, each with one-third of our content. This is a standard sharded architecture using the request URL as the shard key. The DNS name is:

  • content.example.com

This DNS name does a simple A, AAAA, or even CNAME round robin across our three nodes, with the following hostnames:

  • node1.example.com
  • node2.example.com
  • node3.example.com

The architecture will work as depicted in Figure 1 below:

Self Routing Diagram

Figure 1: Self-routing cluster using DNS round robin

To set up a self-routing cluster, we first need to define the cluster and content origin:

vcl 4.0;

backend node1 {
  .host = "node1.example.com";
  .port = "80";
}

backend node2 {
  .host = "node2.example.com";
  .port = "80";
}

backend node3 {
  .host = "node3.example.com";
  .port = "80";
}

backend content {
  .host = "content-origin.example.com";
  .port = "80";
}


Next, define a hash director for our cluster:

sub vcl_init
{
  new cluster = directors.hash();
  cluster.add_backend(node1, 1);
  cluster.add_backend(node2, 1);
  cluster.add_backend(node3, 1);
}


We leverage this director to do our content sharding by hashing on the URL and route the request to its destination:

sub vcl_recv
{
  set req.backend_hint = cluster.backend(req.url);
  set req.http.X-shard = req.backend_hint;

  if (req.http.X-shard == server.identity) {
    set req.backend_hint = content;
  } else {
    return(pass);
  }
}


Above, we check if the hash director points to this node or to another node (lines 34 to 38). If it points to this node, we set the backend to our content server and Varnish will now go through its standard caching logic. If the hash director points to another node, we simply pass the request to that node. This logic will repeat on that node and the request will hash to itself, normal caching logic will happen, and the response will pass back through the originating node.

That's it, we now have a self-routing sharded cluster! The full VCL can be found here.

There are some considerations for the code above. First, we are using server.identity to identify the server against our backend list (line 34). Server.identity defaults to the hostname of the node. In some cases, the hostname may not match the VCL backend name, so you can change the value of server.identity by using the -i parameter when starting varnishd to match your VCL backends.

Also, we are using the hash director to do our shard routing. The hash director has known rehashing limitations when it comes to resizing your cluster or when an entire node is marked as down, which is another kind of resizing. Starting in Varnish Cache 5.0, a new and improved director has been added, the shard director, courtesy of our friends at UPLEX. The shard director addresses these limitations and basically gives you textbook consistent hashing.

Finally, we kept the architecture simple for example purposes. Probes and health checks have been omitted and backends are all single units and not highly available nested directors. More on this later.

Redirect Routing Configuration

There is another method of self-routing we can employ that accomplishes the same thing, but without the need to pass (or proxy) the request to the destination shard. This method uses standard HTTP redirects and this instructs the client to make a new connection directly to the appropriate destination shard.

Using the above VCL, we just need to alter vcl_recv to this:

sub vcl_recv
{
  set req.backend_hint = cluster.backend(req.url);
  set req.http.X-shard = req.backend_hint;

  if (req.http.X-shard == server.identity &&
      req.http.X-shard + ".example.com" == req.http.Host) {
    set req.backend_hint = content;
  } else {
    set req.http.X-redir = "https://" + req.http.X-shard +
      ".example.com" + req.url;
    return(synth(302, "Found"));
  }
}

sub vcl_synth
{
  if (resp.status == 302) {
    set resp.http.Location = req.http.X-redir;
    return (deliver);
  }
}


Instead of doing a pass to the destination shard, we do an HTTP 302 redirect (lines 38 to 40). This means the client will create a new dedicated request directly to our intended destination. Using redirection means we do not have the overhead of passing the request through the originating node. This means less cluster overhead, but it also means your client will be doing an extra round trip. This is usually a positive trade off if you plan on transferring large amounts of content.

The full VCL can be found here.

Varnish Plus

Varnish Plus can add tremendous value to this architecture. First, we can use VHA and add nodes to our shards which would act as additional traffic capacity without any extra overhead. This means we can now choose from a pool of servers for any single request. This not only maximizes performance, but it serves to gives us greater fault tolerance when we have unexpected downtime, including IP failover. Varnish Cache Plus also ships with DNS backend support, simplifying configuration, and VMODs for managing millions of custom shard routes.

We can also leverage Massive Storage Engine and scale the storage on each node to tens of terabytes with little to no performance impact. It will also keep its cache intact across restarts and reboots. We can follow best security practices and enable end to end TLS on all traffic. Finally, since clients are connecting directly to this server, we can leverage TCP Acceleration and optimize connections for ultra-low latencies.

Image title

Figure 2: Self-routing cluster with VHA, TLS, MSE, and TCP Acceleration

If you are looking for enhanced security, greater architectural control, or to scale up your content delivery while controlling costs, then Varnish Extend is the complete enterprise-grade DIY private or hybrid CDN solution.

In Conclusion

This post highlights just a few ways we can scale content caching while keeping our architecture simple and optimizing for availability and performance. The key here is VCL. By having the ability to express our architecture using simple logic and rules, architectures which were once considered out of reach can now become standard and these now standard architectures can be easily modified to match your exact requirements!

cluster Architecture Requests Shard (database architecture)

Published at DZone with permission of Reza Naghibi, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • A Complete Guide to AngularJS Testing
  • What Should You Know About Graph Database’s Scalability?
  • Why Every Fintech Company Needs DevOps
  • Java Development Trends 2023

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: