Back to articles
AI & Cloud

Running Local LLM on VPS GPU: A ChatGPT Alternative with Fixed Costs and Absolute Data Control

A detailed guide on running local LLMs on GPU-powered VPS to replace ChatGPT with fixed costs, absolute data control, and high performance. Includes setup, configuration,...

DPTCloud Engineering4 min read
Running Local LLM on VPS GPU: A ChatGPT Alternative with Fixed Costs and Absolute Data Control

Image credit: sasint · Pixabay Content License

Introduction

In an era where AI services like ChatGPT are widely adopted, many businesses and individuals seek self-hosted LLM (Large Language Model) solutions to maintain data control, reduce operational costs, and ensure security. This guide provides a detailed walkthrough of running a local LLM on a GPU-powered VPS to replace ChatGPT with fixed costs, absolute data control, and high performance.

By hosting your own LLM on a VPS with GPU support, you retain full control over input, output, and processing data. This approach is ideal for organizations with high security requirements, sensitive data, or long-term cost optimization goals.

Requirements and Preparation

To run a local LLM on a GPU VPS, you need:

  • GPU VPS with CUDA-compatible GPU (e.g., NVIDIA T4, A10G, L4, A100)
  • RAM of 32GB or more (64GB or higher recommended)
  • SSD storage with at least 500GB capacity
  • Linux operating system (Ubuntu 22.04 or CentOS 8)
  • Docker and NVIDIA Docker installed

Refer to: High-RAM Cloud VPS for Self-Hosted LLMs & API Gateways | Cloud VPS RAM lớn cho Self-Host LLM & API Gateway

Installation and Configuration

We'll use llama.cpp or text-generation-webui to run a local LLM on your GPU VPS.

1. Install NVIDIA Driver and CUDA

On Ubuntu 22.04:

# Update system
sudo apt update && sudo apt upgrade -y

# Install NVIDIA driver
sudo apt install nvidia-driver-535 -y

# Install CUDA Toolkit (GPU-specific)
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-ubuntu2204.pin
sudo mv cuda-ubuntu2204.pin /etc/apt/preferences.d/cuda-repository-pin-600
sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/7fa2af80.pub
sudo add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/ /"
sudo apt-get update
sudo apt-get -y install cuda

2. Install Docker and NVIDIA Container Toolkit

# Install Docker
sudo apt install docker.io -y
sudo systemctl enable docker
sudo systemctl start docker

# Install NVIDIA Container Toolkit
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt update
sudo apt install nvidia-docker2 -y
sudo systemctl restart docker

3. Run LLM with text-generation-webui

# Create model storage directory
mkdir -p /opt/llm && cd /opt/llm

# Download model (e.g., Llama 3 8B)
wget https://huggingface.co/meta-llama/Llama-3-8B/resolve/main/consolidated.00.pth

# Run container
docker run -it --gpus all -p 7860:7860 -v /opt/llm:/models --name llm-webui ghcr.io/oobabooga/text-generation-webui:latest

Optimization and Cost Control

To maintain fixed costs and optimize performance, consider:

  • Auto-scaling: Use Smart VPS Auto Scaler based on AI predictions to adjust resources dynamically.
  • Performance monitoring: Use nvtop or nvidia-smi to track GPU, RAM, and CPU usage.
  • Model optimization: Choose smaller models (7B, 13B) or apply quantization (4-bit, 8-bit) for lower resource needs.

Verification and Testing

After setup, verify LLM functionality by accessing http://:7860 and entering a query.

Run performance checks:

# Check GPU status
nvidia-smi

# Monitor GPU usage
watch -n 1 nvidia-smi

# Monitor RAM and CPU
htop

ChatGPT Alternative Comparison

Comparison Point Running Local LLM on GPU VPS ChatGPT (Public)
Cost Fixed cost (VPS + GPU) Usage-based (often higher)
Data Control Absolute (data stays on-premise) No control (data sent to cloud)
Performance Customizable and optimized Limited by platform
Customization High (model, prompt, API) Very low

Troubleshooting

  • GPU not detected: Verify NVIDIA driver, install nvidia-docker, and restart Docker.
  • Model fails to load: Check disk space, network connectivity, and model format.
  • Low performance: Optimize model (quantization), increase RAM, or reduce batch size.

References

DPTCloud Tier III Infrastructure

High-RAM Cloud VPS Plans Optimized for AI, Databases & Cache

Enterprise RAID 10 NVMe SSDs, CMC & FPT District 9 uplinks with Anycast Anti-DDoS and 99.99% SLA.

View 22 VPS Plans
ECO · vn-02Recommended

VPS-4E

297.000 ₫ / mo

10% VAT included

  • CPU: 4 vCPU
  • RAM: 8 GB
  • SSD: 50 GB NVMe
ECO · vn-02

VPS-5E

614.900 ₫ / mo

10% VAT included

  • CPU: 8 vCPU
  • RAM: 16 GB
  • SSD: 100 GB NVMe
ADVANCED · vn-03Recommended

VPS-4T

548.900 ₫ / mo

10% VAT included

  • CPU: 4 vCPU
  • RAM: 8 GB
  • SSD: 50 GB NVMe

Related Articles

View all
ZaloZalo SupportDiscord0384 753 478