Zero-Trust Security for Internal AI Infrastructure: Securing OpenWebUI with Tailscale ACLs and OAuth2 Proxy
The Imperative for Zero-Trust in Internal Enterprise AI
As enterprise adoption of Large Language Models (LLMs) skyrockets, organizations are rapidly deploying self-hosted, open-source user interfaces like OpenWebUI to interface with internal models (such as Llama 3 or Mistral via Ollama). While this approach keeps proprietary corporate data inside the local infrastructure, it introduces a massive security challenge: How do you ensure that only authorized employees can access these powerful AI tools, while completely hiding the infrastructure from the public internet?
Traditional perimeter-based security (the "castle-and-moat" model) is no longer sufficient. In modern corporate environments, compromised credentials or a lateral movement exploit can give malicious actors unfettered access to internal web applications. This is where a Zero-Trust Architecture (ZTA) becomes mandatory. Under a Zero-Trust framework, the system operates on a strict principle: never trust, always verify. Every request to the internal AI system must be authenticated, authorized, and encrypted, regardless of where the request originates.
This technical blueprint demonstrates how to architect a production-grade, Zero-Trust ingress for OpenWebUI using two industry-standard open-source tools: Tailscale ACLs and OAuth2 Proxy.
---The Architecture: Layered Defense-in-Depth
To successfully protect OpenWebUI, we rely on a layered defense-in-depth strategy that decouples network transport from application authentication. The architecture consists of three core components:
- Network Isolation Layer (Tailscale): OpenWebUI and its underlying LLM orchestration APIs are completely stripped of public IP addresses. They run inside an encrypted mesh VPN (Tailnet).
- Identity Verification Layer (OAuth2 Proxy): A reverse proxy that sits directly in front of OpenWebUI, intercepting all HTTP requests and validating them against your corporate Identity Provider (IdP) like Google Workspace, Microsoft Entra ID, or Okta.
- Application Layer (OpenWebUI): The frontend user interface, which now only receives traffic that has already been verified by both the network and the identity proxy.
Security Axiom: If an unauthorized user attempts to scan your public infrastructure, they should find absolutely zero open ports pointing toward your internal AI systems. The application simply does not exist to the public web.---
Step 1: Network-Level Segmentation with Tailscale ACLs
Tailscale provides an encrypted overlay network built on top of the WireGuard® protocol. By deploying Tailscale across your infrastructure, internal servers can communicate securely regardless of physical location. However, simply putting OpenWebUI on a Tailnet isn't enough; we must enforce strict micro-segmentation using Tailscale Access Control Lists (ACLs).
By default, Tailscale allows all nodes on a network to talk to each other. To implement Zero-Trust, we change this default behavior to a "default deny" stance and explicitly define who can access the AI frontend. Below is an example of a production Tailscale ACL configuration defined in JSON:
{
"groups": {
"group:data-science": ["[email protected]", "[email protected]"],
"group:ai-admins": ["[email protected]"]
},
"hosts": {
"ai-frontend": "100.101.102.103"
},
"acls": [
{
"action": "accept",
"src": ["group:data-science", "group:ai-admins"],
"dst": ["ai-frontend:80", "ai-frontend:443"]
}
]
}With this configuration, even if an employee is part of your global Tailnet, they cannot even initiate a TCP handshake with the ai-frontend server unless they are explicitly added to the group:data-science or group:ai-admins groups. Network visibility is strictly restricted on a need-to-know basis.
Step 2: Enforcing Identity with OAuth2 Proxy
While Tailscale ACLs guarantee that only verified corporate devices can reach the server network, we still need to verify the *identity* of the specific user session and handle centralized single sign-on (SSO) logging. This is where OAuth2 Proxy acts as a robust gatekeeper.
OAuth2 Proxy intercepts incoming traffic to OpenWebUI. If a user does not have a valid session cookie, the proxy redirects them to your organization’s OpenID Connect (OIDC) provider (e.g., Okta or Entra ID). Once the user successfully authenticates at the IdP level, OAuth2 Proxy validates the returned JWT token, establishes an encrypted session cookie, and forwards the authorized request to OpenWebUI.
Example: Production Docker Compose Configuration
To implement this seamlessly, you can deploy OAuth2 Proxy and OpenWebUI together using Docker Compose. In this setup, OpenWebUI does not expose any ports to the host machine—only OAuth2 Proxy listens on the Tailscale network interface.
version: '3.8'
services:
oauth2-proxy:
image: quay.io/oauth2-proxy/oauth2-proxy:v7.6.0
container_name: oauth2-proxy
ports:
- "100.101.102.103:443:4180" # Binds only to the Tailscale IP
environment:
- OAUTH2_PROXY_PROVIDER=oidc
- OAUTH2_PROXY_CLIENT_ID=your-idp-client-id
- OAUTH2_PROXY_CLIENT_SECRET=your-idp-client-secret
- OAUTH2_PROXY_OIDC_ISSUER_URL=[https://accounts.google.com](https://accounts.google.com)
- OAUTH2_PROXY_COOKIE_SECRET=highly-secure-random-32-char-string
- OAUTH2_PROXY_UPSTREAM=http://openwebui:8080
- OAUTH2_PROXY_EMAIL_DOMAINS=company.com
- OAUTH2_PROXY_HTTP_ADDRESS=0.0.0.0:4180
restart: always
openwebui:
image: ghcr.io/open-webui/open-webui:main
container_name: open-webui
expose:
- "8080"
environment:
- WEBUI_AUTH=true
- OLLAMA_BASE_URL=http://ollama-backend:11434
volumes:
- open-webui-data:/app/backend/data
restart: always
volumes:
open-webui-data:In this architecture, notice that the openwebui container has no ports directive mapping to the host network. It is entirely isolated within the internal Docker bridge network. The *only* entry point is through port 4180 on OAuth2 Proxy, which is securely bound exclusively to the machine's Tailscale private IP address.
Step 3: Configuring OpenWebUI for Header-Based Authentication
To optimize the user experience and avoid double-authentication (where a user logs into your enterprise IdP and then has to type another password into OpenWebUI), we pass identity context upstream.
OAuth2 Proxy can be configured to inject specific HTTP headers—such as X-Forwarded-Email, X-Forwarded-User, and X-Forwarded-Groups—into the request after authentication succeeds. OpenWebUI can read these incoming headers to automatically provision accounts and log users into their personalized workspaces. This achieves a frictionless, enterprise-grade Single Sign-On (SSO) workflow while keeping access control strictly centralized at the identity provider level.
Key Enterprise Security Benefits
By implementing this Zero-Trust stack, your internal AI infrastructure gains substantial security advantages:
- Elimination of the Public Attack Surface: Because your servers do not expose ports to the public internet, they are immune to automated botnets, zero-day HTTP exploits, and external distributed denial-of-service (DDoS) attacks.
- Granular Audit Logging: Every attempt to access your AI models generates a centralized audit log at the IdP level, the OAuth2 Proxy layer, and the Tailscale control plane. This satisfies strict compliance requirements (e.g., SOC2, ISO 27001).
- Immediate Offboarding: When an employee leaves the company and is deactivated in your central identity provider, their access to the internal AI models is revoked instantly across both the network layer (Tailscale) and the application layer (OAuth2 Proxy).
Conclusion: Balancing Velocity and Security
Deploying powerful internal AI tools like OpenWebUI shouldn't require compromising your enterprise security posture. By shifting away from open public endpoints and embracing a Zero-Trust architecture leveraging Tailscale ACLs and OAuth2 Proxy, security teams can confidently deliver advanced AI capabilities to internal users. This setup ensures that proprietary corporate data, models, and conversations remain secure, private, and fully authenticated at all times.