Scaling On-Demand TLS with Caddy Server: Optimizing for 50,000+ Customer Domains on a Single VPS

Introduction: The Multi-Tenant Custom Domain Challenge

In the modern Software-as-a-Service (SaaS) and e-commerce platform landscape, allowing users to point their custom domains (e.g., shop.customer.com) to your platform is a standard requirement. However, managing SSL/TLS certificates at scale for tens of thousands of independent domains presents severe infrastructure hurdles. Traditional web servers require complex automation scripts, cron jobs, and frequent configuration reloads to provision certificates via Let's Encrypt or ZeroSSL. This approach introduces significant overhead, replication lag, and potential downtime.

Caddy Server revolutionizes this paradigm with its built-in On-Demand TLS capability. Instead of pre-generating certificates for thousands of domains, Caddy provisions them on-the-fly during the TLS handshake of the very first request. While this sounds like magic, running it smoothly for over 50,000 domains on a single Virtual Private Server (VPS) requires careful architectural planning, fine-tuning, and robust security guards. This guide details how to optimize Caddy for large-scale enterprise deployments without draining your VPS resources.

The Core Architecture: How On-Demand TLS Works

Before diving into optimization, it is crucial to understand the lifecycle of an On-Demand TLS request. When an un-cached domain hits Caddy, the server initiates a synchronous sequence:

The Handshake: A client establishes a TLS connection with a specific Server Name Indication (SNI).
The Permission Check: Caddy hits an internal or external HTTP endpoint to verify if the domain is authorized to use your service.
ACME Issuance: If authorized, Caddy requests a certificate from an ACME provider (Let's Encrypt or ZeroSSL).
Storage and Serving: Caddy saves the certificate to the filesystem or a database and completes the TLS handshake.

Without strict limits, this process leaves your infrastructure vulnerable to Distributed Denial of Service (DDoS) attacks, certificate rate-limiting exhaustion, and high disk I/O bottlenecks. Below are the definitive optimizations required to stabilize this setup for 50,000+ domains.

1. Implementing a High-Performance Ask Endpoint

The single most critical security and performance barrier in On-Demand TLS is the ask directive. If you leave this unconfigured, Caddy will attempt to provision a certificate for any domain pointed at your server's IP address, quickly hitting ACME rate limits and locking you out of certificate issuance.

Security Rule: Never enable On-Demand TLS without an optimized, secure, and fast 'ask' endpoint. It is the gatekeeper of your infrastructure.

To support 50,000+ domains, your ask endpoint must respond within milliseconds. Avoid querying a heavy relational database (like PostgreSQL or MySQL) directly during the TLS handshake. Instead, implement a fast caching layer using Redis or an in-memory Key-Value store. The endpoint should return an HTTP status 200 OK if the domain is allowed, and a 4xx status if it is not.

2. Production-Ready Caddyfile Configuration

The following optimized block shows how to structure your global options and site block within the Caddyfile to maximize resource efficiency and manage network traffic effectively.

{
    # Global options
    on_demand_tls {
        ask [http://127.0.0.1:8080/validate-domain](http://127.0.0.1:8080/validate-domain)
        interval 2m
        burst 10
    }
    
    # Storage optimization
    storage file_system /var/lib/caddy
} 

:443 {
    tls {
        on_demand
    }
    
    # Proxy to backend application
    reverse_proxy 127.0.0.1:3000 {
        header_up Host {host}
        header_up X-Real-IP {remote_host}
    }
}

In this setup, the interval and burst parameters act as rate limiters for certificate generation. A burst of 10 certificates with a cooldown interval prevents sudden traffic spikes from overwhelming the server or triggering ACME provider blocks.

3. System-Level and Kernel Adjustments

Running a high-traffic proxy serving 50,000 domains means your VPS will experience thousands of concurrent TCP connections. The default Linux kernel configurations are usually not optimized for this volume. You must increase the file descriptor limits and optimize the network stack via /etc/sysctl.conf.

Increase File Limits: Ensure that Caddy can open enough files and sockets by configuring ulimit -n 65535 in your systemd service file.
TCP Window Tuning: Enable TCP window scaling and reuse to process connections faster without exhausting the local port range.
Optimize Buffer Sizes: Read and write buffers should be adjusted to balance memory usage with network throughput.

Adjust your kernel parameters by adding these lines to your system configurations:

fs.file-max = 2097152
net.core.somaxconn = 4096
net.ipv4.tcp_fin_timeout = 15
net.ipv4.tcp_tw_reuse = 1

4. Storage Layer: Moving Beyond the Local Filesystem

By default, Caddy stores certificates as flat files on the local disk. When managing over 50,000 domains, this can result in more than 150,000 files (including certificates, metadata, and private keys) organized in complex nested directories. This causes high inode consumption and significant disk I/O bottlenecks on standard VPS SSDs.

To mitigate this risk, consider migrating your Caddy storage to a distributed or high-performance plugin database layer such as caddy-dns or a Redis storage plugin ([github.com/gamalan/caddy-tlsredis](https://github.com/gamalan/caddy-tlsredis)). Storing certificates in Redis keeps the handshake processes localized to memory speeds and allows you to scale horizontally by adding more VPS instances behind a Load Balancer in the future.

5. Monitoring, Logging, and Rate Limiting

Maintaining long-term uptime requires proactive visibility. Caddy's native internal metrics can be exported directly into Prometheus and Grafana. Pay close attention to these vital health indicators:

caddy_tls_certificates_total: Track the growth curve toward your 50,000-domain milestone.
http_requests_total: Watch for unexpected spikes in specific unknown host requests.
ACME Errors: Filter logs for 429 status codes, indicating that your application is hitting Let's Encrypt rate limits due to misconfigured authorization checks.

Conclusion: A Scalable, Self-Healing Infrastructure

Optimizing Caddy Server for On-Demand TLS transforms a monumental task—managing SSL certificates for 50,000 custom domains—into a reliable, automated, and low-maintenance operation. By safeguarding your environment with a high-speed Redis-backed ask validation endpoint, fine-tuning Linux kernel network parameters, and scaling the storage layer out of the local filesystem, a cost-effective single VPS can easily handle enterprise-level loads. Implement these steps sequentially, monitor your system metrics meticulously, and watch your platform scale smoothly without administrative friction.