Guide to Build an Offline AI Code Assistant on a 4GB RAM VPS with CodeGPT, CodeLlama, and Ollama
A detailed guide to building an offline AI Code Assistant on a 4GB RAM VPS using Ollama, CodeGPT, and CodeLlama, integrated with the VS Code Extension. Covers installation,...
Overview of the Offline AI Code Assistant System
In today's fast-evolving software development landscape, having an AI assistant that supports code generation directly within your development environment is highly beneficial. This guide walks you through building an offline AI Code Assistant on a 4GB RAM VPS using open-source language models like CodeGPT and CodeLlama via Ollama, integrated with the VS Code Extension. This setup enables developers to write code quickly, accurately, and securely without relying on third-party APIs.
By hosting the system locally, you maintain full control over data privacy and security, making it ideal for internal development teams, enterprises, or organizations seeking a self-hosted AI solution for coding assistance.
Requirements and Environment Preparation
To deploy this system, you will need:
- A VPS with at least 4GB RAM and 2 vCPU (minimum).
- Operating System: Ubuntu 22.04 LTS or equivalent.
- Stable internet connection for initial model downloads.
- Software: Ollama, VS Code, and the VS Code Extension for Ollama.
Installation and Configuration Steps
Follow these steps on your VPS:
1. Install Ollama
Run the following command to install Ollama:
curl -fsSL https://ollama.com/install.sh | sh
2. Download CodeLlama and CodeGPT Models
Download the CodeLlama 7B model (or 13B if sufficient RAM is available):
ollama pull codellama:7b
If you prefer CodeGPT (compatible with GPT-3.5), download the corresponding model:
ollama pull codegpt:7b
3. Start Ollama and Verify Model Installation
Check that the model is successfully downloaded by running:
ollama list
Ensure the model appears in the list. Then, test the model by running:
ollama run codellama:7b "Write a Python function to calculate factorial"
4. Install VS Code Extension
On your local machine, open VS Code and install the Ollama Extension from the Extensions Marketplace.
After installation, open Settings and configure:
- Set OLLAMA_HOST to your VPS IP address (e.g., http://192.168.1.100:11434)
- Set the default model (e.g., codellama:7b)
Testing and Performance Optimization
Once configured, test the system by:
- Opening a source code file (Python, JavaScript, etc.) in VS Code.
- Using the Code Assistant toolbar or Ctrl+Shift+P to access Ollama.
- Requesting the model to generate or modify code.
To optimize performance:
- Limit input token count to reduce memory usage.
- Use smaller models if RAM is constrained.
- Run Ollama as a service to ensure automatic restarts after failures.
Troubleshooting and Support
If you encounter issues:
- Connection to Ollama failed: Verify port 11434 is open and Ollama is running.
- RAM exceeds limits: Reduce model size or upgrade VPS RAM.
- Model not visible in VS Code: Confirm the model is downloaded and Ollama is active.
Refer to the official Ollama documentation and Ollama GitHub for detailed support.
References
For further information on deploying self-hosted LLMs, consider these resources:
High-RAM Cloud VPS Plans Optimized for AI, Databases & Cache
Enterprise RAID 10 NVMe SSDs, CMC & FPT District 9 uplinks with Anycast Anti-DDoS and 99.99% SLA.
VPS-4E
10% VAT included
- CPU: 4 vCPU
- RAM: 8 GB
- SSD: 50 GB NVMe
