Deploy and Run Local LLM AI on VPS with Ollama and Open WebUI
Detailed guide to deploying and running Local LLM AI on VPS using Ollama and Open WebUI, including setup, performance optimization, and security. Ideal for businesses needing full...
Introduction
Deploying and running Local Large Language Model (LLM) AI on a Virtual Private Server (VPS) is an ideal solution for businesses that require full data control, enhanced security, and reduced operational costs compared to cloud-based AI platforms. With Ollama and Open WebUI, you can build a self-hosted AI system that supports models like Llama 3, Mistral, and Phi-2, giving you complete control over software and hardware.
This guide provides a detailed walkthrough of deploying and operating a Local LLM AI system on a VPS using Ollama and Open WebUI. It covers installation, configuration, performance optimization, and security hardening, making it suitable for both beginners and system engineers scaling to larger deployments.
Requirements and Preparation
To successfully deploy a Local LLM AI system on a VPS, ensure the following prerequisites are met:
- Cloud VPS with high RAM (minimum 16GB, preferably 32GB or more for larger models)
- OS: Ubuntu 22.04 LTS or Debian 11+
- SSH access to the VPS
- Stable internet connection
- Docker and Docker Compose installed (required for Open WebUI)
Deploy Ollama and Open WebUI
Ollama is a lightweight tool for downloading, running, and managing local LLMs. Open WebUI provides a user-friendly web interface to interact with Ollama-hosted models.
Install Ollama
Run the following command on your VPS to install Ollama:
curl -fsSL https://ollama.com/install.sh | sh
Download LLM Models
After installing Ollama, download your desired models. For example, to download Llama 3:
ollama pull llama3
Or download a smaller model like Phi-2:
ollama pull phi
Install Open WebUI
Open WebUI can be deployed using Docker. Create a docker-compose.yml file:
version: '3.8'
services:
open-webui:
image: ghcr.io/open-webui/open-webui:main
container_name: open-webui
ports:
- "8080:8080"
volumes:
- ./data:/app/data
environment:
- OLLAMA_BASE_URL=http://host.docker.internal:11434
restart: always
Then run:
docker-compose up -d
Optimize Performance and Security
To ensure optimal performance and high security, follow these steps:
- Limit Ollama memory usage by setting
OLLAMA_NUM_GPUandOLLAMA_NUM_THREADS. - Set
OLLAMA_HOST=0.0.0.0:11434to allow external connections. - Use Nginx as a reverse proxy with HTTPS to secure access.
- Configure UFW firewall to open only ports 80, 443, and 11434.
- Set up automatic updates for Ollama and Open WebUI.
Verify and Test
To verify your system is running correctly:
- Open a browser and navigate to
http://:8080 - Log in to Open WebUI (default: no account, create a new one)
- Select the downloaded LLM model (e.g., Llama 3, Phi)
- Enter a test query and verify the response.
Troubleshooting
Common issues and solutions:
- Cannot connect to Ollama: Check if Ollama is running with
systemctl status ollamaand ensure port 11434 is open. - Open WebUI not showing models: Verify the
OLLAMA_BASE_URLenvironment variable and confirm Ollama listens on port 11434. - Memory shortage: Reduce the number of concurrent models or upgrade VPS RAM.
References
High-RAM Cloud VPS Plans Optimized for AI, Databases & Cache
Enterprise RAID 10 NVMe SSDs, CMC & FPT District 9 uplinks with Anycast Anti-DDoS and 99.99% SLA.
VPS-4E
10% VAT included
- CPU: 4 vCPU
- RAM: 8 GB
- SSD: 50 GB NVMe
