Zero Trust, Build High Scale TLS Termination Layer

Let me tell you about the TLS termination system I built. We needed to support custom domains at scale, which meant HAProxy handling thousands of certificates and terminating TLS for high-traffic services.

The old playbook was simple: decrypt at the load balancer, send HTTP to your app servers, call it a day. But that plaintext traffic between your load balancer and backends? That’s a security team's nightmare in 2025. Zero Trust means exactly that — trust nothing, encrypt everything, even your “internal” traffic.

Introduction

In this presentation, I’m going to discuss a solution for implementing secure communication over HTTPS in modern architectures. The solution involves specific technologies such as load balancing, TLS termination, and certificate management. It’s important to note that there are multiple ways to implement HTTPS and secure traffic handling; this is simply one approach I chose to explore.

The Problem

When thinking about implementing secure, modern TLS termination systems at scale, many questions naturally arise — or at least they did for me. Some of these include:

• How will DNS be managed?
• Where will certificates be stored?
• How will certificates be provisioned?
• Will the solution scale, and if so, to what extent?

Over the course of this presentation, I’ll answer these questions and explain the solution based on my experience.

Architectural Overview

The system looks like this:

When a user hits my-domain.com, they should see a green lock — no certificate warnings, no “this site is insecure” nonsense. Making that work at scale with no downtime means coordinating DNS, certificate provisioning, secure storage, and the load balancer layer.

Here’s how each piece fits together.

DNS System

We used CNAME records for all custom domains pointing to our HAProxy load balancer. When you're managing hundreds of domains, CNAMEs beat A records — you can change backend infrastructure without updating every customer's DNS.

For certificate validation, DigiCert uses HTTP-based DCV. They fetch a validation token from:

http://my-domain.com/.well-known/pki-validation/fileauth.txt

This is just plain HTTP (port 80), so no certificates are needed. We configured HAProxy to serve this path from a shared storage location where we dropped the validation tokens. Once DigiCert verified the token, they issued the certificate, and we loaded it into HAProxy for HTTPS termination.

Cert Storage System

Certificates live in our internal KMS. Each HAProxy node runs a sync agent that maintains a local state file, tracking which certs are currently synced. The agent periodically checks the central server, compares against what should be there, and pulls any diffs.

Here’s the key part: we don’t reload HAProxy. The agent uses HAProxy’s runtime API to hot-inject new certificates directly into memory. Zero downtime, zero connection drops.

New cert issued? The agent fetches it from KMS, writes it to /etc/haproxy/certs/, and pushes it into HAProxy via a socket command. HAProxy starts using the new cert immediately.

Certificate Provisioning Flow

Here’s how we automate certificate issuance with DigiCert:

1. Request cert: Send a Certificate Signing Request (CSR) to DigiCert's API for the customer domain.
2. Get DCV token: DigiCert returns an HTTP validation token.
3. Serve the token: We expose it at http://my-domain.com/.well-known/pki-validation/fileauth.txt via HAProxy.
4. DigiCert validates: They fetch the token from that URL to prove we control the domain.
5. Cert issued: DigiCert returns the certificate, we push it to KMS.
6. Agent syncs: Within 5 minutes, our sync agent picks up the new cert and hot-injects it into HAProxy via the runtime API.

The entire flow is automated. From when a customer adds a domain to serving HTTPS with a valid cert takes about 10 minutes. The bottlenecks are usually DigiCert validation time and DNS propagation.

Load Balancer

HAProxy is the load balancer where TLS termination happens. If you're not familiar with the concept: a load balancer sits in front of your application servers and distributes traffic across them. This prevents any single server from getting hammered. But load balancers do more than just balance — they also handle TLS termination.

Here’s why that matters: all internet traffic is encrypted with TLS (the “S” in HTTPS). Your application servers can’t process encrypted data directly — someone has to decrypt it first. That’s the load balancer’s job. It decrypts incoming HTTPS requests, then forwards them to your backends.

Now, what happens after decryption? You have two options:

1. Send plaintext HTTP to backends – This was common in old on-prem datacenters with trusted internal networks. Don’t do this anymore. If an attacker gets inside your network (and they will eventually), plaintext traffic is a gift.
2. Re-encrypt before sending to backends – The load balancer decrypts the client’s TLS connection, then re-encrypts it using internal certificates before forwarding to app servers. This is the Zero Trust approach — encrypt everything, trust nothing.

We went with option 2. No plaintext traffic, even inside our VPC.

Why HAProxy?

There are plenty of load balancer options: Nginx, Envoy, F5 BIG-IP, AWS ALB. We chose HAProxy because:

• It can mount thousands of certificates from a local directory, so all we had to do was get certificates to that local directory.
• It has a dynamic runtime API, so hot loading certificates was possible and prevented any downtime.
• It was optimal in terms of cost and speed.

Our Setup

We run 3 HAProxy instances behind AWS NLB (3 Availability Zones). Each node handles:

• TLS termination for thousands of customer domains
• SNI-based certificate selection
• Re-encryption to backends
• DigiCert DCV token serving

Config:

frontend https_front
    bind *:443 ssl crt /etc/haproxy/certs/ alpn h2,http/1.1
    mode http
    default_backend web_servers

# Backend: Your application servers
backend web_servers
    mode http
    balance roundrobin
    server app1 192.168.1.10:80 check
    server app2 192.168.1.11:80 check

Let me break down what’s happening here:

The bind *:443 ssl crt /etc/haproxy/certs/ alpn h2,http/1.1line:

• Listens on port 443 (HTTPS)
• Enables SSL/TLS termination
• Loads all certificates from /etc/haproxy/certs/ (HAProxy reads every .pem file in that directory)
• Supports both HTTP/2 and HTTP/1.1 via ALPN

The frontend is where client connections land. The backend is where your actual application servers live — in this case, two servers using round-robin distribution.

Here’s the critical part: HAProxy loads all certificates into memory at startup, but we don’t need to restart to add new ones. The sync agent I mentioned earlier uses HAProxy’s runtime API to inject new certificates directly into memory while HAProxy is running. A customer adds a new domain, DigiCert issues the cert, our agent picks it up from KMS and hot-swaps it into HAProxy — all without dropping a single connection.

This is one of HAProxy’s killer features for managing certificates at scale.

Server Name Indication (SNI)

I should explain how one HAProxy instance serves thousands of different domains from a single IP address — a key piece that makes this system work.

Back in the day, only one certificate was allowed per IP address. The TLS handshake happened before the client could tell the server which domain it wanted, so the server had no way to know which certificate to present. If you had 1,000 customer domains, you needed 1,000 IP addresses. Wasteful and expensive.

SNI (Server Name Indication) changed everything. It’s a TLS extension where the client sends the requested hostname in plaintext during the initial handshake. HAProxy reads that hostname and selects the matching certificate from its in-memory store.

Here’s how it works:

• HAProxy loads all certificates from /etc/haproxy/certs/ into memory at startup.
• New certificates get hot-injected via the runtime API (no disk reads on the request path).
• When a client connects with SNI for mydomain.com, HAProxy looks up the cert in memory.
• HAProxy presents the correct certificate, TLS handshake completes.

One IP, thousands of domains, all with valid certificates. No disk I/O per request — everything’s in memory.

Edge cases: Ancient clients (pre-2010) don’t support SNI, but we haven’t seen any in production. The hostname is sent in plaintext with standard SNI; Encrypted Client Hello (ECH) in TLS 1.3 encrypts this, though client and server adoption is still rolling out.

Conclusion

Building TLS termination at scale comes down to a few key pieces: automated certificate provisioning, secure centralized storage, stateful reconciliation across nodes, and zero-downtime cert injection.

Our stack — DigiCert for issuance, internal KMS for storage, sync agents with local state tracking, and HAProxy’s runtime API — handles 3,000 domains without manual intervention. A customer adds their domain, and 10 minutes later, they’re serving HTTPS with a valid certificate. Everything in between is automated.

The biggest win? HAProxy’s runtime API. Hot-swapping certificates in memory means we rotate hundreds of certs daily with zero downtime. No reloads, no dropped connections, no user impact.