Self-Hosting Windmill.dev on a VPS: The Ultimate Open-Source Alternative to AWS Lambda for Python and TypeScript Background Jobs

Introduction: The Hidden Costs of Serverless Cloud Functions

For years, AWS Lambda has been the go-to standard for executing background jobs, microservices, and scheduled tasks. The promise of infinite scalability and a pay-as-you-go pricing model lured engineering teams worldwide. However, as production workloads scale, many organizations face a harsh reality: serverless infrastructure can become unpredictably expensive, complex to debug locally, and constrained by rigid limitations like the 15-minute execution timeout.

Enter Windmill.dev—a powerful, developer-first, open-source developer platform that serves as a direct, self-hostable alternative to AWS Lambda, Temporal, and Retool. By hosting Windmill on your own Virtual Private Server (VPS), you regain absolute control over your computational environment, eliminate vendor lock-in, and unlock an intuitive UI to manage, monitor, and orchestrate background tasks using standard Python and TypeScript. This comprehensive guide walks you through the architectural benefits of Windmill and provides a step-by-step roadmap to hosting it on your infrastructure.

Why Choose Windmill over AWS Lambda?

While AWS Lambda excels at handling highly spiky traffic, it introduces significant friction for routine background jobs, long-running ETL processes, and internal automation scripts. Here is why modern engineering teams are shifting to self-hosted Windmill:

No Execution Time Limits: Unlike Lambda's strict 15-minute cutoff, Windmill tasks running on your VPS can execute for hours or days, making them ideal for heavy data processing and long-running workflows.
Substantial Cost Predictability: Instead of navigating complex API gateway and compute-duration billing, you pay a fixed, predictable monthly fee for your VPS hardware.
Native Python and TypeScript Support: Windmill treats code as first-class citizens. You write plain scripts, and Windmill automatically parses the arguments to generate a clean, web-based UI form for execution and debugging.
Built-in Workflow Orchestration: You can chain individual scripts into complex, stateful DAGs (Directed Acyclic Graphs) directly within the dashboard, replacing expensive orchestration layers like AWS Step Functions.
Integrated Secrets Management: Environment variables, API tokens, and database credentials are securely managed and injected into scripts at runtime without requiring AWS Secrets Manager setup.

Architecture Overview: Windmill on a VPS

Windmill is built with high performance and low footprint in mind, written primarily in Rust and Go. When deployed via Docker Compose on a single VPS, the architecture comprises a few lightweight, highly efficient components:

Windmill Server (Backend): Handles API requests, serves the frontend UI, manages the schedule runner, and coordinates job distribution.
PostgreSQL Database: Act as the central state store, holding script versions, execution logs, user permissions, and workflow states.
Windmill Workers: Distributed execution agents that pull jobs from the queue and run your Python or TypeScript code. Because they are decoupled, you can easily scale workers horizontally across multiple VPS instances if your workload grows.

Note: A basic 2 vCPU and 4GB RAM VPS from providers like DigitalOcean, Hetzner, or Linode is more than sufficient to process hundreds of thousands of background jobs per day efficiently.

Step-by-Step Deployment Guide via Docker Compose

1. System Requirements and Prerequisites

Before launching Windmill, ensure your VPS meets the following baseline requirements:

A clean installation of Ubuntu 22.04 LTS or newer.
Docker and Docker Compose v2 installed.
A domain or subdomain pointed to your VPS IP address (e.g., windmill.yourcompany.com) for secure HTTPS access.

2. Configuring the Docker Compose Environment

Connect to your VPS via SSH and create a dedicated directory for your Windmill stack. We will set up a standard docker-compose.yml file utilizing the official, pre-configured production images provided by the Windmill team.

Create your configuration directory and navigate into it:

mkdir -p /opt/windmill && cd /opt/windmill

Next, generate an environment file named .env to store essential variables, including the master database password and your application's base URL:

DATABASE_URL=postgres://postgres:YourSecurePasswordHere@db:5432/windmill?sslmode=disable
BASE_URL=[https://windmill.yourcompany.com](https://windmill.yourcompany.com)
WM_TOKEN=YourGeneratedRandomLongToken

Deploy the containers using the optimized Compose blueprint. This stack configures the core server, an embedded PostgreSQL database, and dedicated workers for handling asynchronous heavy-lifting tasks. Execute the deployment with:

docker compose up -d

Once the services initialize, verify that all containers are healthy by checking the runtime logs. You can now access the control panel by navigating to your configured domain name. The default administrator credentials allow immediate setup upon your first login.

Developing Background Jobs with Python and TypeScript

One of Windmill’s greatest strengths is how it transforms standard scripts into production-ready background jobs without requiring proprietary SDK boilerplate. It relies heavily on type hints to generate user interfaces automatically.

Example: Creating a Python Background Job

In Windmill, a Python script is structured as a simple main function. The platform automatically inspects the function parameters and generates an execution form. Here is an example of an asynchronous reporting script:import time def main(report_name: str, target_email: str, retry_count: int = 3): """ Generates and dispatches a comprehensive business analytics report. """ print(f"Initializing background job: {report_name}") # Simulate data aggregation time.sleep(5) print(f"Successfully generated report. Dispatching to {target_email}...") return {"status": "success", "delivered_to": target_email}

When saved inside the Windmill workspace, this script can be executed manually via a web form, triggered via an HTTP webhook, or scheduled using cron syntax (e.g., every Monday at 8:00 AM) to handle heavy recurring backend logic seamlessly.

Enterprise-Grade Monitoring and Security Practices

Transitioning from a managed cloud provider like AWS to a self-hosted environment means you take ownership of operational safety. Implement these essential best practices to maintain a reliable production stack:

1. Automatic Backups

Since PostgreSQL holds the entire configuration state of your tasks and workflows, schedule nightly logical dumps of your database using pg_dump and stream them securely to an offsite S3-compatible storage bucket.

2. Resource Isolation (Sandboxing)

By default, Windmill allows you to configure workers to run jobs inside restricted execution environments. Ensure untrusted scripts or heavy computations are bound by Docker memory and CPU limits to prevent a single faulty infinite loop from freezing your entire VPS backend infrastructure.

3. Centralized Logging

Windmill captures standard output (stdout) and standard error (stderr) streams for every script execution, making troubleshooting trivial compared to digging through complex AWS CloudWatch log groups. Integrate these logs with uptime monitoring tools to stay alerted to unexpected job failures immediately.

Conclusion: Embracing Serverless Independence

Self-hosting Windmill.dev on a VPS represents a modern architectural paradigm shift. It combines the seamless developer experience and ease of use typically found in high-end SaaS tools with the economic predictability, performance advantages, and sovereignty of self-hosted open-source software. By migrating your background jobs from AWS Lambda to a dedicated Windmill setup, you unlock absolute execution freedom, reduce administrative overhead, and drastically trim down your monthly cloud expenditures.