Back to articles
AI & Cloud

Building a Private AI Knowledge Base with AnythingLLM and Docker on Low-Spec VPS

A detailed guide on building a Private AI Knowledge Base using AnythingLLM and Docker on low-spec VPS, ideal for small businesses and individuals seeking to self-host a secure,...

DPTCloud Engineering4 min read
Building a Private AI Knowledge Base with AnythingLLM and Docker on Low-Spec VPS

Why Build a Private AI Knowledge Base on a Low-Spec VPS?

In an era where AI is increasingly prevalent, having a Private AI Knowledge Base enables businesses and individuals to maintain full control over their data, ensure internal security, and avoid dependency on public AI platforms like OpenAI or Google. Many people hesitate due to high costs associated with powerful servers or cloud AI solutions. However, with AnythingLLM and Docker, you can deploy an internal AI knowledge system on a low-spec VPS (e.g., 2GB RAM, 1 CPU, 20GB SSD) — ideal for small businesses, individuals, or development teams.

AnythingLLM is an open-source platform that allows you to upload internal data (documents, notes, APIs, etc.) and create an AI system that understands and answers questions based on that data. Combined with Docker, deployment becomes simple, manageable, and reusable across multiple servers.

Requirements and Environment Setup

To successfully deploy, you need:

Deploying AnythingLLM with Docker

Follow these steps to install AnythingLLM on your VPS:

1. Install Docker and Docker Compose

On your Ubuntu VPS, run the following commands:

# Update the system
sudo apt update && sudo apt upgrade -y

# Install Docker
sudo apt install docker.io docker-compose -y

# Enable and start Docker
sudo systemctl enable docker
sudo systemctl start docker

2. Create the docker-compose.yml File

Create a new directory and create the `docker-compose.yml` file:

mkdir anythingllm && cd anythingllm
nano docker-compose.yml

Paste the following content into the file:

version: '3.8'

services:
  anythingllm:
    image: anythingllm/anythingllm:latest
    container_name: anythingllm
    ports:
      - "3001:3001"
    volumes:
      - ./anythingllm_data:/app/data
    environment:
      - NODE_ENV=production
      - DATABASE_URL=sqlite:///app/data/database.sqlite
      - PORT=3001
    restart: unless-stopped

3. Start the Service

docker-compose up -d

After running the command, AnythingLLM will start on port 3001. Access http://IP_VPS:3001 to log in for the first time (create an account and password).

Adding Data and Configuration

After logging into the AnythingLLM interface, you can:

  • Upload documents (PDF, Word, TXT, etc.)
  • Integrate APIs or URLs to extract data
  • Select an AI model (default uses Ollama or supports OpenAI API if available)
  • Set access controls, authentication, and security settings

Verification and Testing

To verify the system is working:

# Check Docker status
docker-compose ps

# View logs
docker-compose logs -f anythingllm

Access the web interface and test asking a question based on the uploaded documents. If the system responds accurately, deployment is successful.

Troubleshooting Common Issues

  • Cannot access port 3001? Check the firewall (UFW) or VPS security settings: sudo ufw allow 3001
  • Cannot upload data? Verify permissions on the `anythingllm_data` directory and file formats.
  • Slow performance? Use lightweight AI models (e.g., Llama 2-7B or Mistral) or increase RAM if possible.

References and Resources

For further learning:

Building a Private AI Knowledge Base on a low-spec VPS not only reduces costs but also ensures data security and internal control — an ideal solution for modern businesses.

DPTCloud Tier III Infrastructure

High-RAM Cloud VPS Plans Optimized for AI, Databases & Cache

Enterprise RAID 10 NVMe SSDs, CMC & FPT District 9 uplinks with Anycast Anti-DDoS and 99.99% SLA.

View 22 VPS Plans
ECO · vn-02Recommended

VPS-4E

297.000 ₫ / mo

10% VAT included

  • CPU: 4 vCPU
  • RAM: 8 GB
  • SSD: 50 GB NVMe
ECO · vn-02

VPS-5E

614.900 ₫ / mo

10% VAT included

  • CPU: 8 vCPU
  • RAM: 16 GB
  • SSD: 100 GB NVMe
ADVANCED · vn-03Recommended

VPS-4T

548.900 ₫ / mo

10% VAT included

  • CPU: 4 vCPU
  • RAM: 8 GB
  • SSD: 50 GB NVMe

Related Articles

View all
ZaloZalo SupportDiscord0384 753 478