Building a Multi-Tasking Self-Hosted AI Coding Agent on a 4GB RAM VPS with Qwen-Agent and LiteLLM: The Ultimate Open-Source Alternative to Cursor and Bolt.new

Introduction: The Shift Toward Self-Hosted AI Development

The developer ecosystem has experienced a massive paradigm shift with the rise of AI-powered development tools. Platforms like Cursor and Bolt.new have fundamentally transformed how we write, debug, and deploy code. However, relying entirely on proprietary cloud-based services introduces distinct challenges for enterprise environments and independent developers alike: recurring subscription costs, strict API rate limits, and significant data privacy concerns regarding proprietary codebases.

The solution lies in self-hosting. While running large language models (LLMs) traditionally demands enterprise-grade GPUs, recent optimizations in open-source models and agentic frameworks have changed the game. This guide provides a comprehensive, production-ready blueprint to build and deploy a multi-tasking AI Coding Agent on a constrained 4GB RAM Virtual Private Server (VPS). By combining Alibaba's highly efficient Qwen-Agent framework with LiteLLM as a unified proxy layer, you can establish a robust, privacy-centric alternative to commercial tools at a fraction of the cost.

Architectural Overview: Efficiency by Design

Running an advanced AI agent on a 4GB RAM footprint requires strict architectural discipline. Attempting to host a large model locally on such limited hardware would immediately trigger Out-Of-Memory (OOM) faults. Therefore, our architecture decouples the compute layer (the LLM) from the orchestration layer (the agent framework).

Qwen-Agent: A powerful open-source framework by Alibaba, specifically optimized for tool-use, code generation, and long-context comprehension. It manages the agent's memory, planning, and tool execution loop.
LiteLLM: Acts as a universal API gateway. It translates standard OpenAI-format requests into calls for highly optimized, cost-effective external APIs (such as DeepSeek, OpenRouter, or Qwen Cloud), allowing the VPS to focus entirely on agent logic rather than heavy model inference.
Docker Workspace: A sandboxed environment running on the VPS where the agent can safely read, write, and execute code without compromising the host system.

Step 1: Preparing the VPS Environment

Before deploying our software stack, we must optimize the underlying Linux operating system to maximize available memory and ensure stability under load. We recommend starting with a clean installation of Ubuntu 22.04 LTS or Ubuntu 24.04 LTS.

1.1 Configuring Swap Space

On a 4GB RAM system, configuring a swap file is critical to handle occasional memory spikes during code compilation or deep context parsing.

Pro-Tip: While SSD-backed swap is significantly slower than physical RAM, it acts as a vital safety net to prevent the Linux kernel from killing your agent processes during intense workloads.

Execute the following commands to allocate a 4GB swap file:

sudo fallocate -l 4G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

1.2 Installing System Dependencies

Update your package manager and install Docker Compose, which will orchestrate our containerized ecosystem:

sudo apt-get update && sudo apt-get upgrade -y
sudo apt-get install -y docker.io docker-compose git curl python3-pip python3-venv

Step 2: Configuring LiteLLM as the Universal API Proxy

LiteLLM is the backbone of our resource-saving strategy. It standardizes input/output formats, allowing Qwen-Agent to communicate seamlessly with any backend model provider while abstracting complex authentication and endpoint configurations.

2.1 Creating the LiteLLM Configuration

Create a dedicated directory and define your config.yaml file. For optimal coding performance on a budget, we recommend routing to models like DeepSeek-V3, DeepSeek-R1, or Qwen-2.5-Coder-72B-Instruct via cost-effective providers.

model_list:
  - model_name: coding-model
    litellm_params:
      model: openrouter/deepseek/deepseek-chat
      api_key: "os.environ/OPENROUTER_API_KEY"
      max_tokens: 8192
      temperature: 0.2

2.2 Deploying LiteLLM via Docker Compose

Define a docker-compose.yml file to run the LiteLLM proxy service continuously in the background:

version: '3.8'

services:
  litellm:
    image: ghcr.io/berriai/litellm:main-latest
    ports:
      - "4000:4000"
    environment:
      - OPENROUTER_API_KEY=your_actual_api_key_here
    volumes:
      - ./config.yaml:/app/config.yaml
    command: ["--config", "/app/config.yaml", "--port", "4000", "--host", "0.0.0.0"]
    restart: always

Launch the service by running docker-compose up -d. Your local proxy is now accessible at http://localhost:4000.

Step 3: Implementing the Multi-Tasking Qwen-Agent

With our API proxy established, we can now configure the orchestration layer. Qwen-Agent uses an extensible tool architecture, enabling the AI to interact with the file system, execute terminal commands, and perform web searches to gather documentation.

3.1 Setting Up the Python Environment

Isolate your application dependencies within a secure virtual environment:

python3 -m venv venv
source venv/bin/activate
pip install qwen-agent custom-tools-dependencies

3.2 Writing the Core Agent Script

Create a file named agent_core.py. This script initializes the Qwen-Agent, connects it to our local LiteLLM proxy, and defines a secure workspace where the agent can autonomously write and modify code.

import os
from qwen_agent.agents import Assistant
from qwen_agent.tools import register_tool

# Define the LLM configuration pointing to our local LiteLLM proxy
llm_cfg = {
    'model': 'coding-model',
    'model_server': 'http://localhost:4000/v1',
    'api_key': 'None',  # Handled by our LiteLLM environment config
}

# Define system instructions to optimize the agent for multi-tasking software engineering
system_instruction = """You are an expert, autonomous AI Coding Agent.
Your task is to plan, write, debug, and document production-grade software.
Utilize the provided system tools to read/write files and execute code within the workspace sandbox.
Always verify your code by writing and running unit tests before finalizing your output."""

# Initialize the multi-tasking agent with standard file and code execution tools
bot = Assistant(
    llm=llm_cfg,
    system_message=system_instruction,
    function_list=['code_interpreter', 'file_management']
)

print("AI Coding Agent successfully initialized and awaiting directives.")

Step 4: Building a Simple Web Interface

To truly replace tools like Cursor or Bolt.new, we require an intuitive user interface. We can build a rapid, lightweight frontend using Streamlit, keeping memory overhead minimal while delivering a rich, interactive user experience.

Install Streamlit via pip: pip install streamlit, then construct your UI script (app.py) to capture user prompts, display the live agent execution thoughts, and show modified source files in real-time.

import streamlit as st
# UI components and interaction loops mapping back to agent_core.py go here
st.title("🤖 Self-Hosted AI Coding Agent")
st.caption("Powered by Qwen-Agent & LiteLLM on a 4GB RAM VPS")

user_input = st.text_area("Describe the feature or bug fix you want to implement:")
if st.button("Execute Task"):
    st.info("Agent is analyzing workspace and planning execution...")
    # Trigger agent reasoning loop...

Production Optimization and Security Considerations

Operating an autonomous agent that handles file writes and command executions requires strict guardrails. To maintain a secure, high-performance environment, adhere to the following best practices:

Workspace Sandboxing: Never run the agent directly on the host OS. Ensure that the code_interpreter and file operations are strictly bound to an isolated Docker container with limited CPU and memory allocations.
Strict Context Management: As codebases grow, the context window fills rapidly, increasing token consumption. Implement periodic context pruning or leverage Qwen-Agent's built-in RAG (Retrieval-Augmented Generation) capabilities to pass only relevant files to the prompt.
Automated Backups: Configure your agent's workspace directory as a Git repository. Instruct the agent to create automatic commits before making sweeping changes, allowing you to roll back unintended modifications instantly.

Conclusion: Democratizing AI-Driven Development

By leveraging smart orchestration rather than raw computing power, we have successfully created a highly capable, multi-tasking AI Coding Agent on an affordable 4GB RAM VPS. This setup eliminates dependency on costly SaaS platforms, protects intellectual property by routing data through your chosen secure APIs, and gives you total control over your development workflow.

As open-source models continue to advance in efficiency and reasoning capabilities, self-hosted developer agents will shift from a luxury alternative to a standard engineering standard. Deploy your instance today and experience the future of autonomous development.