Building a Centralized AI Prompt Guardrail System Using Open WebUI and Pipelines on a VPS

Introduction to Enterprise AI Governance

As generative AI and Large Language Models (LLMs) integrate deeper into corporate workflows, they bring unprecedented efficiency. However, this rapid adoption introduces significant risks, including data leaks (IP and PII), toxic content generation, and compliance violations. Organizations can no longer rely on individual users to practice safe prompting. There is a critical need for a centralized intermediary that inspects, filters, and sanitizes both incoming prompts and outgoing responses.

This technical guide details how to build a centralized AI Prompt Guardrail system using Open WebUI and its powerful Pipelines architecture, deployed on a self-hosted Virtual Private Server (VPS). By implementing this architecture, your organization establishes a secure gateway, ensuring that all AI interactions strictly adhere to corporate governance and safety standards before reaching the underlying LLMs.

---

Understanding the Architecture: Open WebUI, Pipelines, and Guardrails

To build a robust control plane, we leverage two core open-source technologies combined with a defensive engineering concept known as Guardrails.

• Open WebUI: A highly customizable, feature-rich user interface designed for LLMs. It serves as the centralized user hub, replacing direct API access and providing granular user management, authentication, and logging.
• Pipelines: An extensible plugin framework native to Open WebUI. It allows developers to intercept the request-response lifecycle. By writing custom Python scripts, you can parse, modify, or block prompts and responses dynamically.
• Guardrails: The logical rulesets embedded within the Pipelines. These rules act as a dual-layer firewall: Input Guardrails filter sensitive or malicious prompts, while Output Guardrails ensure the model's response is accurate, non-toxic, and free of proprietary data.

The Core Objective: No prompt reaches the LLM, and no response reaches the user, without passing through our centralized verification layer.

---

Prerequisites and VPS Environment Setup

Before beginning the deployment, ensure your Virtual Private Server (VPS) meets the minimum requirements to handle traffic routing and concurrent pipeline processing comfortably.

Recommended VPS Specifications

• CPU: 4 Cores (Dedicated recommended)
• RAM: 8 GB minimum (16 GB preferred if running light embedding models locally)
• Storage: 50 GB NVMe SSD
• OS: Ubuntu 22.04 LTS or Ubuntu 24.04 LTS

Initial Server Provisioning

Connect to your VPS via SSH and update the system packages to their latest versions:

sudo apt update && sudo apt upgrade -y
sudo apt install curl git build-essential -y

Next, install Docker and Docker Compose, as running Open WebUI and Pipelines via containers ensures isolated environments and simplifies dependency management:

curl -fsSL [https://get.docker.com](https://get.docker.com) -o get-docker.sh
sudo sh get-docker.sh
sudo usermod -aG docker $USER

Log out and log back in to apply the docker group permissions.

---

Step-by-Step Deployment: Open WebUI and Pipelines via Docker Compose

We will configure a single docker-compose.yml file to orchestrate both Open WebUI and the Pipelines container. This approach ensures they share a secure internal network for ultra-low latency communication.

1. Create the Directory Structure

mkdir -p ~/ai-gateway/pipelines
cd ~/ai-gateway

2. Configure the Docker Compose File

Create a docker-compose.yml file with the following infrastructure configuration:

version: '3.8'

services:
  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    restart: always
    ports:
      - "3000:8080"
    volumes:
      - open-webui-data:/app/backend/data
    environment:
      - OPENAI_API_BASE_URLS=http://pipelines:9099
      - OPENAI_API_KEYS=filtered-gateway-key
      - WEBUI_SECRET_KEY=super-secret-encryption-key
    depends_on:
      - pipelines

  pipelines:
    image: ghcr.io/open-webui/pipelines:main
    container_name: pipelines
    restart: always
    ports:
      - "9099:9099"
    volumes:
      - ./pipelines:/app/pipelines
    environment:
      - PIPELINES_URL=http://pipelines:9099
      - OPENAI_API_BASE_URL=[https://api.openai.com/v1](https://api.openai.com/v1)
      - OPENAI_API_KEY=your-actual-upstream-llm-key

volumes:
  open-webui-data:

In this configuration, Open WebUI does not talk directly to OpenAI or your internal LLM server. Instead, it treats the Pipelines container (port 9099) as its upstream OpenAI provider. Pipelines, in turn, forwards approved requests to the actual upstream API after inspection.

3. Launching the Services

Run the following command to start the environment in detached mode:

docker compose up -d

Verify that both containers are running optimally by checking their health status:

docker compose ps

---

Implementing the Guardrail Logic in Pipelines

Now that the infrastructure is active, we must design the guardrail logic. Open WebUI Pipelines expects a standard Python file structure implementing specific hook functions like inlet (for input filtering) and outlet (for output filtering).

Writing a Custom Guardrail Pipeline

Create a file named guardrail_pipeline.py inside your ./pipelines directory on the VPS. This blueprint demonstrates how to block corporate PII leakages (such as credit cards or internal project code names) and filter malicious intent:

import re
from typing import List, Union, Generator

class Pipeline:
    def __init__(self):
        self.name = "Enterprise Security Guardrail"
        # Define forbidden keywords or regex patterns
        self.forbidden_patterns = [
            r"(?:\d{4}-){3}\d{4}", # Standard Credit Card pattern
            r"(?i)\b(Project-X|Confidential-Alpha)\b" # Internal codenames
        ]

    async def inlet(self, body: dict, user: dict) -> dict:
        """Executes before the prompt is dispatched to the LLM"""
        print(f"Inspecting incoming prompt from user: {user.get('email')}")
        
        messages = body.get("messages", [])
        if not messages:
            return body

        # Extract the latest prompt text
        latest_prompt = messages[-1].get("content", "")

        # Scan for sensitive information
        for pattern in self.forbidden_patterns:
            if re.search(pattern, latest_prompt):
                raise Exception("Security Violation: Your prompt contains restricted corporate data or PII.")

        # Modify prompt slightly to enforce corporate context if needed
        return body

    async def outlet(self, body: dict, user: dict) -> dict:
        """Executes before the response is shown to the user"""
        print("Inspecting model generation response...")
        # Implement toxic content or hallucinations filtering here
        return body

Once saved, Open WebUI detects the pipeline automatically. Any user attempting to pass a credit card number or discuss "Project-X" will immediately receive a clean, localized security exception instead of leaking data to public LLM APIs.

---

Production Optimization and Security Hardening

Running an enterprise gateway requires strict operational hardening. Ensure the following production configurations are met prior to onboarding users:

1. Enforce SSL/TLS via a Reverse Proxy: Do not expose port 3000 directly to the internet. Install Nginx or Caddy on your VPS to handle Let's Encrypt SSL certificates, routing all external traffic over secure HTTPS (port 443).
2. Configure IP Whitelisting: If this gateway is restricted to internal corporate employees, utilize firewall configurations like ufw to restrict incoming connections solely to your corporate VPN IP ranges.
3. Audit Logging: Retain the standard outputs of the pipeline container. Route these logs to an external security information and event management (SIEM) tool to monitor compliance trends and catch bad actors attempting jailbreak prompts repeatedly.

---

Conclusion

By coupling Open WebUI with Pipelines on a centralized VPS, you successfully transition from an uncontrolled AI landscape to an enterprise-grade, monitored AI ecosystem. This setup ensures that your business can embrace the immense productivity gains of Large Language Models while strictly minimizing compliance, security, and data leakage vectors. Begin by deploying the basic keyword and regex filters outlined above, and systematically scale your guardrails using advanced embedding models or semantic checks as your organization's AI needs grow.