Deploy AI Agent on VPS: Detailed A-Z Guide with Docker and GPU
A detailed guide to deploying an AI Agent on VPS using Docker and GPU, covering installation, configuration, performance optimization, and operational checks. Step-by-step...
Introduction
AI Agents are powerful automation applications capable of performing tasks such as data processing, user interaction, or system control. For efficient deployment in self-hosted environments, a Virtual Private Server (VPS) is a popular choice due to its flexibility, high performance, and GPU integration capabilities. This guide provides a step-by-step walkthrough of deploying an AI Agent on a VPS using Docker and GPU, from initial setup to operational verification.
System Requirements and Preparation
To deploy an AI Agent on a VPS, you need:
- A VPS with at least 8GB RAM and 2 CPU cores (recommended: 16GB RAM and 4 CPU cores).
- A GPU supporting CUDA (e.g., NVIDIA Tesla, RTX 30xx/40xx) or an integrated GPU supporting OpenCL.
- Linux operating system (Ubuntu 20.04 LTS or later).
- SSH access to the VPS.
- Docker and Docker Compose installed.
- NVIDIA GPU driver (if using NVIDIA GPU).
Visit High-RAM Cloud VPS for Self-Hosted LLMs & API Gateways to select a suitable VPS.
Install and Configure Docker with GPU
First, install Docker and Docker Compose on the VPS:
# Update the system
sudo apt update && sudo apt upgrade -y
# Install Docker
sudo apt install docker.io docker-compose -y
# Enable and start Docker
sudo systemctl enable docker
sudo systemctl start docker
For NVIDIA GPUs, install the driver and Docker runtime:
# Add NVIDIA repository
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin
sudo mv cuda-ubuntu2004.pin /etc/apt/preferences.d/cuda-repository-pin-600
sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/7fa2af80.pub
sudo add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/ /"
sudo apt update
# Install GPU driver
sudo apt install nvidia-driver-535 nvidia-cuda-toolkit -y
# Install NVIDIA Docker runtime
sudo apt install nvidia-docker2 -y
sudo systemctl restart docker
Test GPU access in Docker:
docker run --gpus all nvidia/cuda:11.0-base nvidia-smi
Deploy AI Agent with Docker
Create a docker-compose.yml file to define the AI Agent service:
version: '3.8'
services:
ai-agent:
image: your-ai-agent-image:latest
container_name: ai-agent
runtime: nvidia
environment:
- CUDA_VISIBLE_DEVICES=0
volumes:
- ./data:/app/data
ports:
- "8000:8000"
deploy:
resources:
limits:
cpus: '2'
memory: 8G
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu]
Start the AI Agent:
docker-compose up -d
Verify and Optimize Performance
To verify the AI Agent is running:
# Check container status
docker ps
# View logs
docker logs ai-agent
# Monitor GPU usage
nvidia-smi
To optimize performance, consider:
- Limiting GPU resources in Docker Compose.
- Using KServe, Ray, and GPU Binpacking for load balancing.
- Applying Fuse OverlayFS for storage driver optimization.
Troubleshooting
If the AI Agent fails to run:
- Check Docker logs:
docker logs ai-agent. - Ensure the GPU driver is correctly installed and Docker has GPU access.
- Test GPU access with
docker run --gpus all nvidia/cuda:11.0-base nvidia-smi. - Run
nvidia-smion the VPS to verify GPU detection.
Refer to Deploy SurrealDB on Cloud VPS for additional VPS configuration insights.
Conclusion
Deploying an AI Agent on a VPS using Docker and GPU is an effective way to self-host AI applications locally. With this detailed A-Z guide, you can set up, configure, and optimize your AI Agent in a self-hosted environment. Ensure thorough testing and performance tuning to achieve optimal results.
For more AI solutions, explore Dedicated Compute Servers for AI & Inference Workloads.
High-RAM Cloud VPS Plans Optimized for AI, Databases & Cache
Enterprise RAID 10 NVMe SSDs, CMC & FPT District 9 uplinks with Anycast Anti-DDoS and 99.99% SLA.
VPS-4E
10% VAT included
- CPU: 4 vCPU
- RAM: 8 GB
- SSD: 50 GB NVMe
