DPTCloud
Back to articles
Technology Insight

Scaling n8n with Queue Mode and Docker Compose: Handling Millions of Webhooks Daily

June 4, 2026

Introduction: The Bottleneck of Single-Instance Workflow Automation

In modern enterprise architectures, data integration and automated workflows are the backbone of efficient operations. As organizations scale, the volume of incoming events—such as webhooks from CRM systems, payment gateways, and e-commerce platforms—can grow exponentially. n8n has emerged as a premier source-available workflow automation tool, offering immense flexibility. However, running n8n in its default default mode (single-instance) presents significant limitations when tasked with handling high-concurrency traffic or millions of webhooks per day.

When a single n8n instance receives an overwhelming surge of webhook requests, it must handle incoming HTTP connections, parse payloads, execute complex workflow logic, and maintain state simultaneously. This tightly coupled architecture can lead to memory exhaustion, CPU throttling, and dropped webhook events, resulting in data loss. To build a resilient, enterprise-grade automation pipeline, implementing n8n Queue Mode via Docker Compose is the definitive solution.

Understanding n8n Queue Mode Architecture

n8n Queue Mode separates the responsibilities of API management, workflow execution, and background processing into specialized, decoupled components. Instead of a single server managing everything, the workload is distributed across multiple nodes using a message broker and a centralized database.

A robust Queue Mode deployment consists of the following architectural pillars:

  • Main Webhook Node: A dedicated n8n instance optimized exclusively to listen for incoming webhook triggers, quickly accept payloads, push them to the queue, and return a 200 OK response to the sender. This minimizes connection hold times.
  • Redis (Message Broker): Acts as the high-performance memory queue (using BullMQ under the hood) that buffers incoming execution jobs, ensuring no webhooks are lost during traffic spikes.
  • Worker Nodes: Independent n8n instances that poll jobs from Redis, execute the actual workflow nodes, process data, and connect to third-party APIs. Workers can be scaled horizontally depending on the queue size.
  • PostgreSQL (Primary Database): The centralized relational database that stores workflow definitions, execution history, user credentials, and session data.
  • Main UI Node: A dedicated instance for users to build workflows, manage credentials, and view execution logs without impacting the performance of running workers or webhook receivers.

Prerequisites and Infrastructure Planning

Before deploying the stack using Docker Compose, ensure your infrastructure meets the necessary prerequisites. For handling millions of webhooks daily, a multi-core VPS or cloud instance (AWS EC2, DigitalOcean, or Hetzner) is highly recommended.

Minimum Recommended Resources

  • CPU: 4 vCPUs (Compute-optimized instances preferred)
  • RAM: 8 GB to 16 GB RAM (To comfortably support Redis and multiple worker processes)
  • Storage: NVMe SSD with automated log rotation, as execution history can grow rapidly
  • Software: Docker Engine v20.10+ and Docker Compose v2.0+ installed
Important Note: Because n8n handles sensitive API keys and credentials, ensuring your deployment is secured behind a Reverse Proxy (such as Nginx, Traefik, or Caddy) with automated Let's Encrypt SSL certificates is non-negotiable for production environments.

The Complete Docker Compose Blueprint

Below is the comprehensive, production-ready docker-compose.yml file configured for n8n Queue Mode. This setup spins up PostgreSQL, Redis, an n8n Main node (UI and Webhook receiver), and two dedicated n8n Worker nodes.

version: '3.8'

services:
  postgres:
    image: postgres:15-alpine
    container_name: n8n-postgres
    environment:
      - POSTGRES_USER=${DB_USER:-n8n_user}
      - POSTGRES_PASSWORD=${DB_PASSWORD:-secure_db_pass}
      - POSTGRES_DB=${DB_NAME:-n8n_db}
    volumes:
      - postgres_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U $$POSTGRES_USER -d $$POSTGRES_DB"]
      interval: 5s
      timeout: 5s
      retries: 5
    restart: always

  redis:
    image: redis:7-alpine
    container_name: n8n-redis
    command: redis-server --appendonly yes
    volumes:
      - redis_data:/data
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      timeout: 5s
      retries: 5
    restart: always

  n8n-main:
    image: docker.n8n.io/n8nio/n8n:latest
    container_name: n8n-main
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    environment:
      - N8N_HOST=${N8N_HOST}
      - N8N_PORT=5678
      - N8N_PROTOCOL=https
      - NODE_ENV=production
      - DB_TYPE=postgresdb
      - DB_POSTGRESDB_HOST=postgres
      - DB_POSTGRESDB_PORT=5432
      - DB_POSTGRESDB_DATABASE=${DB_NAME:-n8n_db}
      - DB_POSTGRESDB_USER=${DB_USER:-n8n_user}
      - DB_POSTGRESDB_PASSWORD=${DB_PASSWORD:-secure_db_pass}
      - EXECUTIONS_MODE=queue
      - QUEUE_BULL_REDIS_HOST=redis
      - QUEUE_BULL_REDIS_PORT=6379
      - N8N_ENCRYPTION_KEY=${N8N_ENCRYPTION_KEY}
    ports:
      - "127.0.0.1:5678:5678"
    restart: always

  n8n-worker-1:
    image: docker.n8n.io/n8nio/n8n:latest
    container_name: n8n-worker-1
    command: worker
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    environment:
      - NODE_ENV=production
      - DB_TYPE=postgresdb
      - DB_POSTGRESDB_HOST=postgres
      - DB_POSTGRESDB_PORT=5432
      - DB_POSTGRESDB_DATABASE=${DB_NAME:-n8n_db}
      - DB_POSTGRESDB_USER=${DB_USER:-n8n_user}
      - DB_POSTGRESDB_PASSWORD=${DB_PASSWORD:-secure_db_pass}
      - EXECUTIONS_MODE=queue
      - QUEUE_BULL_REDIS_HOST=redis
      - QUEUE_BULL_REDIS_PORT=6379
      - N8N_ENCRYPTION_KEY=${N8N_ENCRYPTION_KEY}
    restart: always

  n8n-worker-2:
    image: docker.n8n.io/n8nio/n8n:latest
    container_name: n8n-worker-2
    command: worker
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    environment:
      - NODE_ENV=production
      - DB_TYPE=postgresdb
      - DB_POSTGRESDB_HOST=postgres
      - DB_POSTGRESDB_PORT=5432
      - DB_POSTGRESDB_DATABASE=${DB_NAME:-n8n_db}
      - DB_POSTGRESDB_USER=${DB_USER:-n8n_user}
      - DB_POSTGRESDB_PASSWORD=${DB_PASSWORD:-secure_db_pass}
      - EXECUTIONS_MODE=queue
      - QUEUE_BULL_REDIS_HOST=redis
      - QUEUE_BULL_REDIS_PORT=6379
      - N8N_ENCRYPTION_KEY=${N8N_ENCRYPTION_KEY}
    restart: always

volumes:
  postgres_data:
  redis_data:

Step-by-Step Deployment Guide

Step 1: Environment Variables Configuration

Create an .env file in the same directory as your docker-compose.yml file. Populate it with secure credentials and your system variables:

N8N_HOST=n8n.yourdomain.com
N8N_ENCRYPTION_KEY=generate_a_long_random_string_here
DB_USER=n8n_prod_admin
DB_PASSWORD=super_secure_password_123
DB_NAME=n8n_production

Step 2: Launching the Stack

Run the following command to start all services in detached mode:docker compose up -d

Verify that all containers are healthy by inspecting the running processes:docker compose ps

Optimizing for High-Volume Webhook Processing

Deploying Queue Mode is only the first step. To ensure your stack can continuously ingest and process millions of webhooks daily without experiencing performance degradation, apply these critical production optimizations:

1. Fine-Tuning Execution Data Retention

By default, n8n saves every single node execution status to the database. Writing millions of records to PostgreSQL daily will rapidly cause disk bloat and degrade query performance. Add these variables to your n8n-main and worker environments to limit database writes:

  • EXECUTIONS_DATA_PRUNE=true: Automatically removes older execution data.
  • EXECUTIONS_DATA_MAX_AGE=48: Keeps data for only 48 hours.
  • EXECUTIONS_DATA_PRUNE_TIMEOUT=3600: Runs the pruning process every hour.
  • EXECUTIONS_DATA_SAVE_ON_ERROR=all: Only save complete data when a workflow fails, reducing successful execution overhead.

2. Horizontal Scaling of Workers

If you observe that your Redis queue size is increasing and workflows are delayed, you can scale your worker instances effortlessly without modifying the configuration file. Execute the following Docker Compose command to dynamically scale your workers:docker compose up -d --scale n8n-worker-1=4

3. Splitting Webhook Traffic and UI Traffic

For extreme workloads, it is recommended to spin up a dedicated instance configured with N8N_ENFORCE_SETTINGS_FILE_AS_SOURCE=true acting strictly as a webhook processor, placing it behind a load balancer that routes all /webhook/* paths to it, while routing standard traffic to the UI node.

Conclusion: Enterprise Reliability Made Simple

By shifting from a single-instance setup to n8n Queue Mode via Docker Compose, you transform n8n from a simple automation utility into a resilient, high-throughput enterprise integration engine. The decoupled synergy of Redis and PostgreSQL ensures that even during unexpected traffic anomalies, your systems will buffer, queue, and process every single event reliably. Implement these configurations today to future-proof your organizational workflows and build automation that scales boundlessly.