Optimizing CI/CD Costs with Woodpecker CI Multi-Worker Deployment on Low-Cost VPS
A guide to optimizing CI/CD costs using Woodpecker CI with a distributed multi-worker architecture on low-cost VPS, ideal for small businesses and startups.
Image credit: 937983 · Pixabay Content License
In today’s fast-paced software development landscape, deploying an efficient and cost-effective CI/CD system is essential—especially for small businesses and startups. Woodpecker CI is a lightweight, open-source CI/CD platform designed for self-hosted environments. In this guide, we’ll walk through deploying Woodpecker CI in a distributed multi-worker architecture across low-cost VPS instances to optimize expenses while maintaining performance and scalability.
Introduction to Woodpecker CI and Benefits of Multi-Worker Architecture
Woodpecker CI is a powerful, self-hosted CI/CD solution built for simplicity and efficiency. It supports multiple programming languages and integrates seamlessly with Docker-based workflows. One of its key strengths is the ability to run in a multi-worker configuration, allowing you to distribute build, test, and deployment tasks across multiple machines.
This distributed setup reduces the load on any single server, accelerates pipeline execution, and improves reliability. By leveraging low-cost VPS instances for worker nodes, you can achieve high performance at minimal cost.
Prerequisites and Environment Setup
To deploy Woodpecker CI with a multi-worker setup on low-cost VPS, you will need:
- At least two VPS instances (one master and one or more workers).
- Stable network connectivity between all VPS instances.
- Linux-based operating system (Ubuntu 20.04 or later).
- Docker and Docker Compose installed on all VPS nodes.
Deploying Woodpecker CI Master and Worker Nodes
We’ll deploy Woodpecker CI using a master-worker model. The master manages pipelines, while workers execute jobs.
1. Install on the Master Server
On the master VPS, create a docker-compose.yml file:
version: '3.8'
services:
woodpecker-server:
image: woodpecker/woodpecker-server:latest
restart: unless-stopped
ports:
- 8000:8000
volumes:
- /var/lib/woodpecker:/var/lib/woodpecker
environment:
- WOODPECKER_DEBUG=true
- WOODPECKER_HOST=https://your-master-domain.com
- WOODPECKER_SECRET=your-secret-key
- WOODPECKER_AGENT_SECRET=your-agent-secret
- WOODPECKER_DATABASE_DRIVER=sqlite3
- WOODPECKER_DATABASE_SOURCE=/var/lib/woodpecker/woodpecker.db
networks:
- woodpecker
woodpecker-agent:
image: woodpecker/woodpecker-agent:latest
restart: unless-stopped
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /var/lib/woodpecker:/var/lib/woodpecker
environment:
- WOODPECKER_SERVER=woodpecker-server:9000
- WOODPECKER_AGENT_SECRET=your-agent-secret
- WOODPECKER_DEBUG=true
networks:
- woodpecker
networks:
woodpecker:
driver: bridge
Run the following command:
docker-compose up -d
2. Install on the Worker Server
On the worker VPS, create a docker-compose.yml file with only the agent service:
version: '3.8'
services:
woodpecker-agent:
image: woodpecker/woodpecker-agent:latest
restart: unless-stopped
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /var/lib/woodpecker:/var/lib/woodpecker
environment:
- WOODPECKER_SERVER=your-master-domain.com:9000
- WOODPECKER_AGENT_SECRET=your-agent-secret
- WOODPECKER_DEBUG=true
networks:
- woodpecker
networks:
woodpecker:
driver: bridge
Run the command:
docker-compose up -d
Configuring Master-Worker Connectivity
To ensure the worker connects to the master, verify the following:
- WOODPECKER_SERVER on the worker must point to the master’s IP or domain.
- WOODPECKER_AGENT_SECRET must match on both master and worker.
- Port 9000 must be open on the master for incoming connections.
Verification and Testing
To confirm the worker is connected:
- Access the master’s web interface at
https://your-master-domain.com. - Log in with your credentials.
- Go to the Agents section and verify the worker appears with a Online status.
Troubleshooting Common Issues
- Worker fails to connect: Verify
WOODPECKER_SERVERandWOODPECKER_AGENT_SECRETvalues. - Worker not visible in UI: Check the agent logs:
docker logs woodpecker-agent. - Builds not running: Ensure the Docker daemon is active and the worker has access to the Docker socket.
Cost Optimization
Using low-cost VPS instances (e.g., 1GB RAM, 1 vCPU) for worker nodes significantly reduces operational costs. Start with one master and one worker, then scale horizontally as needed. Woodpecker CI performs efficiently even on limited hardware, making it ideal for budget-constrained environments.
References
High-RAM Cloud VPS Plans Optimized for AI, Databases & Cache
Enterprise RAID 10 NVMe SSDs, CMC & FPT District 9 uplinks with Anycast Anti-DDoS and 99.99% SLA.
VPS-4E
10% VAT included
- CPU: 4 vCPU
- RAM: 8 GB
- SSD: 50 GB NVMe
