Scaling On-Demand TLS: Optimizing Caddy Server for 50,000+ Customer Domains on a Single VPS

Introduction: The Multi-Tenant Domain Challenge

In modern Software-as-a-Service (SaaS) and multi-tenant platforms, allowing customers to point their custom domains to your application is a standard requirement. However, managing SSL/TLS certificates for tens of thousands of external domains presents a massive operational hurdle. Traditional approaches involving automated cron jobs, Let's Encrypt rate-limit navigation, and complex web server reloads quickly break down under the weight of 50,000+ domains.

Enter Caddy Server and its groundbreaking On-Demand TLS capability. Instead of pre-generating certificates for every possible domain, Caddy can obtain a certificate on-the-fly during the initial TLS handshake. While this feature is incredibly powerful, deploying it at a scale of over 50,000 domains on a standard Virtual Private Server (VPS) requires precise configuration, rigorous security implementations, and underlying system optimization to prevent resource exhaustion and service degradation.

Understanding Caddy's On-Demand TLS Mechanism

To optimize the system, we must first understand how On-Demand TLS operates. When an unencrypted or unrecognized HTTPS request reaches Caddy, the server executes the following lifecycle:

The TLS Handshake Begins: The client requests a secure connection specifying the Server Name Indication (SNI).
The Authorization Check: Caddy intercepts the handshake and sends an internal HTTP request to a predefined management endpoint to verify if the domain is permitted to receive a certificate.
Certificate Acquisition: If authorized, Caddy interacts with an ACME provider (such as Let's Encrypt or ZeroSSL) to solve the challenge, obtain the certificate, and complete the handshake.
Caching: The certificate is stored in memory and on disk for future requests.

Without strict guardrails, this mechanism leaves your VPS highly vulnerable to distributed denial-of-service (DDoS) attacks, where malicious actors flood your server with arbitrary domain requests, exhausting your ACME rate limits and server memory.

Step 1: Implementing a Robust 'Ask' Endpoint

The single most critical security component of On-Demand TLS is the ask directive. You must never run On-Demand TLS in production without configuring an authorization endpoint. Caddy sends a GET request to your backend with the domain as a query parameter (e.g., [https://your-api.internal/check-domain?domain=customer.com](https://your-api.internal/check-domain?domain=customer.com)). If your backend returns an HTTP 200 OK status, Caddy proceeds; any other status causes Caddy to abort the handshake instantly.

Optimizing the Ask Backend

High-Performance In-Memory Lookups: Do not query a heavy relational database (like PostgreSQL or MySQL) directly during the TLS handshake. Instead, use an in-memory data store like Redis or a distributed cache. The lookup must resolve within milliseconds.
Strict Sanitization: Ensure your backend validates that the domain format is structurally valid and explicitly matches an active tenant in your database before issuing a 200 response.

Step 2: Advanced Caddyfile Configuration for Scale

To support 50,000+ domains on a single VPS, we must fine-tune Caddy's global options and storage backend. Below is an optimized production-ready configuration structure:

{
  email [email protected]
  on_demand_tls {
    ask [http://127.0.0.1:8080/validate-domain](http://127.0.0.1:8080/validate-domain)
    interval 1m
    burst 10
  }
  storage file_system /var/lib/caddy
}

In this block, the burst and interval parameters act as an essential rate-limiter for certificate generation. A burst of 10 with an interval of 1 minute ensures that even if thousands of valid domains suddenly point to your server simultaneously, Caddy will pace the ACME requests safely to avoid hitting Let's Encrypt API limits.

Switching Storage Providers (Optional but Recommended)

By default, Caddy saves certificates to the local file system. When managing over 50,000 certificates, the local directory structure will house hundreds of thousands of small files (certificates, private keys, and metadata). To optimize performance and ensure seamless future scaling, consider using a database or object storage provider via Caddy plugins, such as caddy-dns modules combined with a Redis storage backend.

Step 3: Linux OS and Network Stack Tuning

A standard VPS Linux installation is not configured out-of-the-box to handle tens of thousands of concurrent open TLS connections. To prevent Caddy from bottlenecking at the operating system level, you must adjust the kernel parameters via /etc/sysctl.conf.

Increasing File Descriptor Limits

Every active TLS connection and certificate file on disk consumes a file descriptor. Update your system limits by adding the following lines:

fs.file-max = 2097152 (Increases global file descriptor limits)
Modify Caddy's systemd service file (/etc/systemd/system/caddy.service) to include LimitNOFILE=65536 to ensure the process itself can handle heavy traffic.

Optimizing the TCP IP Stack

To smoothly handle high connection volumes, apply these kernel adjustments to recycle closed connections faster and expand buffers:

net.core.somaxconn = 4096
net.ipv4.tcp_fin_timeout = 15
net.ipv4.tcp_max_tw_buckets = 1440000

Step 4: Memory and CPU Resource Planning

How much VPS power do you actually need for 50,000 domains? Because Caddy loads certificates into memory dynamically on-demand, memory consumption scales relative to active concurrent connections, not total registered domains.

As a rule of thumb, a baseline certificate cache for 50,000 domains on disk requires minimal storage space (approx. 2-3 GB). However, active RAM utilization will vary. For a mid-tier VPS running 4 vCPUs and 8GB RAM, Caddy can easily maintain thousands of active concurrent connections without breaking a sweat, provided the ask endpoint remains responsive and network latency is kept to a minimum.

Conclusion

Scaling On-Demand TLS to 50,000+ custom domains on a VPS is not only possible but highly efficient when leveraging Caddy Server's elegant design. By safeguarding your infrastructure with a high-performance Redis-backed ask endpoint, configuring strict ACME rate limits within your Caddyfile, and optimizing the underlying Linux kernel, you create a self-sustaining, secure, and cost-effective routing layer capable of handling enterprise-grade traffic volumes without operational overhead.