Back to articles
Security & Anti-DDoS

Deploy OpenWebUI Connecting to Self-Built VLLM Cluster for Secure Shared AI Platform Across Enterprises

Detailed guide on deploying OpenWebUI connected to a self-built VLLM cluster to create a secure shared AI platform for enterprises, with comprehensive security measures from network...

DPTCloud Engineering4 min read
Deploy OpenWebUI Connecting to Self-Built VLLM Cluster for Secure Shared AI Platform Across Enterprises

Introduction

Deploying a shared AI platform across an enterprise requires robust security, performance, and scalability. In this context, building a self-hosted VLLM (Very Large Language Model) cluster integrated with OpenWebUI offers an effective solution. This guide provides a detailed walkthrough of deploying such a system on high-security VPS infrastructure at DPTCloud, incorporating comprehensive security measures from infrastructure to application layers.

Requirements and Preparation

To successfully deploy the system, you need:

  • Cloud VPS with L3/L4/L7 Anycast Anti-DDoS at DPTCloud
  • Network Security & 99.98% SLA Infrastructure at DPTCloud
  • Software: Docker, Docker Compose, OpenWebUI, VLLM, Nginx, Let's Encrypt (SSL)
  • Hardware: Minimum 16GB RAM, 4 CPU cores, 100GB SSD or larger

Deploying the VLLM Cluster

We will deploy VLLM across multiple nodes to form a cluster. Each node will run a VLLM instance, using Redis as the state storage and inter-node communication system.

VLLM Configuration

On each node, create a docker-compose.yml file:

version: '3.8'
services:
  vllm:
    image: vllm/vllm:latest
    command: [
      "--model", "meta-llama/Llama-2-7b-chat-hf",
      "--host", "0.0.0.0",
      "--port", "8000",
      "--tensor-parallel-size", "1",
      "--trust-remote-code"
    ]
    ports:
      - "8000:8000"
    volumes:
      - ./models:/models
    restart: unless-stopped
    deploy:
      resources:
        limits:
          memory: 16G
        reservations:
          memory: 8G

Connecting Nodes

Use Redis to synchronize state. On each node, start Redis:

docker run -d --name redis -p 6379:6379 redis:alpine

Then configure VLLM to connect to Redis using the REDIS_URL environment variable.

Deploying OpenWebUI

OpenWebUI provides a user-friendly interface to interact with the VLLM cluster.

OpenWebUI Configuration

Create a docker-compose.yml file for OpenWebUI:

version: '3.8'
services:
  openwebui:
    image: openwebui/openwebui:latest
    ports:
      - "8080:8080"
    environment:
      - VLLM_API_BASE_URL=http://vllm-cluster:8000
      - REDIS_URL=redis://redis:6379
    volumes:
      - ./data:/app/data
    restart: unless-stopped

System Security

To ensure enterprise-level security:

  • Integrate WireGuard VPN for secure system access.
  • Use Cloudflare Tunnels to protect remote access without requiring a VPN.
  • Apply UFW firewall and IP-based access control.
  • Scan Docker images for vulnerabilities before deployment using Trivy.

Testing and Optimization

Run the following commands to test:

docker-compose up -d
curl http://localhost:8000/v1/models

If the response returns a list of models, the system is operational.

Alternative Solutions and Advanced Security

For enterprises requiring maximum security:

Secure Remote Access Solutions

To access OpenWebUI remotely without a VPN:

  • Set up Cloudflare Tunnel with a free Let's Encrypt SSL certificate.
  • Enforce IP-based and time-based access policies.
  • Ensure OpenWebUI is protected with OAuth or API key authentication.

Troubleshooting

  • Cannot connect to VLLM: Verify port 8000 is open, Redis is running, and VLLM is active.
  • OpenWebUI does not display models: Check the VLLM_API_BASE_URL environment variable and network connectivity.
  • Low performance: Increase RAM, use GPU if available, or reduce the number of concurrent models.

Conclusion

Deploying OpenWebUI connected to a self-built VLLM cluster delivers a secure shared AI platform for enterprises. With security measures spanning from infrastructure to application layers, the system ensures safety and efficiency for internal AI operations.

DPTCloud Tier III Infrastructure

High-RAM Cloud VPS Plans Optimized for AI, Databases & Cache

Enterprise RAID 10 NVMe SSDs, CMC & FPT District 9 uplinks with Anycast Anti-DDoS and 99.99% SLA.

View 22 VPS Plans
ECO · vn-02Recommended

VPS-4E

297.000 ₫ / mo

10% VAT included

  • CPU: 4 vCPU
  • RAM: 8 GB
  • SSD: 50 GB NVMe
ECO · vn-02

VPS-5E

614.900 ₫ / mo

10% VAT included

  • CPU: 8 vCPU
  • RAM: 16 GB
  • SSD: 100 GB NVMe
ADVANCED · vn-03Recommended

VPS-4T

548.900 ₫ / mo

10% VAT included

  • CPU: 4 vCPU
  • RAM: 8 GB
  • SSD: 50 GB NVMe

Related Articles

View all
ZaloZalo SupportDiscord0384 753 478