Deploy Multi-Task AI Coding Agent on 4GB RAM VPS with Qwen Agent and LiteLLM
A detailed guide to deploying a multi-task AI coding agent on a 4GB RAM VPS using Qwen Agent and LiteLLM, offering a self-hosted alternative to Cursor Bolt New. This solution...
A detailed guide to deploying a multi-task AI coding agent on a 4GB RAM VPS using Qwen Agent and LiteLLM, offering a self-hosted alternative to Cursor Bolt New. This solution enables developers to automate coding tasks, integrate multiple AI models, and manage inference via a unified API gateway—all without relying on cloud services. The setup runs efficiently on modest hardware, leveraging Docker and lightweight models for optimal performance.
Deploying a Multi-Task AI Coding Agent on a 4GB RAM VPS
As demand for AI-assisted programming grows, self-hosting an AI coding agent offers developers greater control, privacy, and cost efficiency. With a 4GB RAM VPS, you can run a capable multi-task AI coding agent using Qwen Agent and LiteLLM—two open-source tools that support model aggregation, automation, and local execution. This guide walks through deploying a fully self-hosted system that replaces cloud-dependent tools like Cursor Bolt New, enabling local AI coding assistance with minimal infrastructure overhead.
Overview of the Solution: Qwen Agent + LiteLLM
Qwen Agent, developed by Alibaba Cloud, is an AI agent capable of writing, debugging, analyzing, and documenting code. When paired with LiteLLM, a unified API gateway for multiple large language models, you can create a powerful, self-hosted coding assistant. LiteLLM abstracts access to various models—Qwen, GPT, Claude, Mistral, and others—through a single endpoint, simplifying integration and reducing complexity.
On a 4GB RAM VPS, you can run both Qwen Agent and LiteLLM simultaneously without a GPU by using optimized or quantized models. LiteLLM supports model routing, rate limiting, and fallback strategies, making it ideal for production-like environments where reliability and flexibility are key.
Prerequisites and Installation
Prerequisites
- 4GB RAM or higher VPS (Ubuntu 20.04/22.04 recommended)
- Linux operating system
- Docker and Docker Compose
- Python 3.9 or higher
- Git
Installation
Run the following commands on your VPS to install required tools:
sudo apt update
sudo apt install -y docker.io docker-compose git python3 python3-pip
sudo usermod -aG docker $USER
newgrp dockerRestart your system or re-login to apply Docker group changes.
Deploying Qwen Agent and LiteLLM
Create a project directory and clone the repositories:
mkdir ai-coding-agent
cd ai-coding-agent
git clone https://github.com/qwen-agent/qwen-agent.git
git clone https://github.com/BerriAI/litellm.gitNow, create a docker-compose.yml file to orchestrate both services:
version: '3.8'
services:
qwen-agent:
image: qwen-agent:latest
container_name: qwen-agent
ports:
- "8080:8080"
volumes:
- ./data:/data
environment:
- QWEN_API_KEY=your_qwen_key
- MODEL_NAME=qwen-7b
restart: unless-stopped
litellm:
image: berriai/litellm:latest
container_name: litellm
ports:
- "8000:8000"
environment:
- LITELLMMODELS=qwen-7b,gpt-3.5-turbo,claude-3-haiku
- LITELLMSERVER=true
restart: unless-stoppedStart the services:
docker-compose up -dVerification and Testing
After deployment, verify the services are running:
docker psCheck the health endpoints:
http://<your-vps-ip>:8080/health– Qwen Agent health checkhttp://<your-vps-ip>:8000/health– LiteLLM health check
Test the API from a client:
curl -X POST http://<your-vps-ip>:8000/completion \
-H "Content-Type: application/json" \
-d '{\"model\": \"qwen-7b\", \"prompt\": \"Write a Python function to calculate factorial\"}'Replacing Cursor Bolt New
Cursor Bolt New is a cloud-based AI coding tool that may raise concerns about data privacy, ongoing costs, and network dependency. By using a self-hosted setup with Qwen Agent and LiteLLM, you gain:
- Full control over source code and data
- Lower long-term operational costs
- Integration with existing CI/CD pipelines and IDEs
- Customizable AI models based on workload needs
This approach delivers similar functionality to Cursor Bolt New while offering greater flexibility, security, and scalability.
Troubleshooting
- API connection failed: Check if Docker Compose is running, ports are open, and firewall rules allow traffic.
- Memory exhaustion: Use smaller models (e.g., Qwen-1.8B), enable model quantization, or upgrade RAM.
- API returns errors: Verify environment variables, inspect Docker logs, and validate model configuration.
References
High-RAM Cloud VPS Plans Optimized for AI, Databases & Cache
Enterprise RAID 10 NVMe SSDs, CMC & FPT District 9 uplinks with Anycast Anti-DDoS and 99.99% SLA.
VPS-4E
10% VAT included
- CPU: 4 vCPU
- RAM: 8 GB
- SSD: 50 GB NVMe
