DPTCloud
Back to articles
Technology Insight

Building a Shared AI Coding Assistant for Agencies Using Tabby and Ollama on Cloud Servers

June 4, 2026

Introduction: The Agency Dilemma in the Age of AI Coding

In the fast-paced world of software development and digital agencies, efficiency is the ultimate competitive advantage. The rise of AI-powered coding assistants like GitHub Copilot and ChatGPT has undeniably revolutionized developer workflows, boosting velocity and reducing routine boilerplate work. However, for a growing agency, relying entirely on public, third-party commercial AI tools introduces a triad of challenges: escalating subscription costs, potential intellectual property leaks, and a lack of customization for proprietary frameworks or coding standards.

When you have dozens of developers working across multiple client projects, paying a monthly per-seat license fee quickly compounds into a significant operational expense. More critically, enterprise clients frequently mandate strict non-disclosure agreements (NDAs) that prohibit uploading their codebases to public cloud AI models. Agencies are left caught between a rock and a hard place: sacrifice AI-driven velocity or risk costly data compliance violations.

Fortunately, a powerful, enterprise-grade alternative has matured. By combining Tabby (an open-source, self-hosted AI coding assistant) with Ollama (a lightweight, highly efficient LLM runner) on a centralized cloud server, your agency can deploy a shared, secure, and blazing-fast AI coding assistant. This comprehensive guide will walk you through the technical blueprint, architectural benefits, and step-by-step implementation of this self-hosted solution.

---

Why Tabby and Ollama? The Ultimate Self-Hosted Synergy

Building an internal AI assistant requires two primary software components: an LLM engine to serve the models and a specialized application layer to handle code completion logic, context fetching, and IDE integration. Here is why the Tabby-Ollama combination stands out for agencies:

  • Tabby: Specifically designed as an open-source, self-hosted alternative to GitHub Copilot. It offers native extensions for popular IDEs (VS Code, JetBrains), supports repository indexing for context-aware suggestions, and provides multi-tenant capabilities suitable for a shared agency environment.
  • Ollama: Renowned for its simplicity and optimized resource management. Ollama allows you to seamlessly download, manage, and run state-of-the-art open-source LLMs like Llama 3, StarCoder2, or DeepSeek-Coder with minimal configuration.
"By pairing Tabby's slick IDE integration with Ollama's efficient backend model management, agencies achieve a fully private, zero-license-fee AI coding infrastructure that scales with their team."
---

Architectural Overview & Cloud Server Requirements

To serve an entire agency smoothly, a centralized topology is highly recommended over local developer installations. This approach ensures that individual developer machines do not require expensive GPU hardware, and it allows the agency to centralize security policies and custom codebase indexing.

Hardware Recommendations

The choice of your cloud server (AWS, Google Cloud, DigitalOcean, or specialized GPU providers like Vast.ai or RunPod) depends directly on the number of concurrent developers and the size of the model you intend to run. For a smooth user experience, low latency (Time-to-First-Token) is critical for inline code completions.

Team Size Recommended Model Minimum GPU Hardware VRAM Requirement
5 - 15 Developers DeepSeek-Coder-1.3B / StarCoder2-3B 1x NVIDIA T4 or RTX 4000 8 GB - 16 GB VRAM
15 - 50 Developers DeepSeek-Coder-7B / Llama-3-8B 1x NVIDIA A10G or RTX 4090 24 GB VRAM
50+ Developers DeepSeek-Coder-14B / Multi-Model Setup 1x NVIDIA A100 or 2x RTX 3090 24 GB - 40 GB+ VRAM
---

Step-by-Step Deployment Guide

Follow these structured steps to configure and launch your shared agency AI assistant on a Linux-based cloud server equipped with Docker and NVIDIA drivers.

Step 1: Setting Up Ollama with Docker

First, we deploy Ollama via Docker, ensuring it has access to the host machine's NVIDIA GPU for hardware acceleration. Create a unified docker-compose.yml file to orchestrate both services.

version: '3.8'

services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama-service
    volumes:
      - ./ollama_data:/root/.ollama
    ports:
      - "11434:11434"
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
    restart: unless-stopped

Run docker compose up -d ollama to start the container. Once active, download your chosen code-generation model into Ollama by executing:

docker exec -it ollama-service ollama run deepseek-coder:6.7b

Step 2: Configuring and Launching Tabby

With Ollama serving the underlying LLM, we append Tabby to our infrastructure configuration. Tabby will connect to Ollama via its local HTTP API network. Update your docker-compose.yml to include the Tabby service:

  tabby:
    image: tabbyml/tabby:latest
    container_name: tabby-service
    command: serve --model experimental::ollama/deepseek-coder:6.7b --device cuda
    ports:
      - "8080:8080"
    volumes:
      - ./tabby_data:/data
    depends_on:
      - ollama
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
    restart: unless-stopped

Execute docker compose up -d to bring up the entire stack. Tabby will now be accessible via port 8080, acting as the centralized bridge for your development team.

---

Securing and Scaling the Infrastructure

Exposing a raw HTTP port (8080) directly to the internet is a severe security risk, especially when dealing with proprietary source code. To make this production-ready for an agency, implement these critical layers:

  1. Reverse Proxy and TLS Encryption: Use Nginx, Caddy, or Traefik to handle incoming traffic over port 443, securing connection paths via Let's Encrypt SSL/TLS certificates. This guarantees all code snippets sent from IDEs to the server are fully encrypted in transit.
  2. Network Access Control: Restrict traffic to the cloud server using firewall rules (Security Groups). Allow connections to port 443 exclusively from your agency's office static IP address or your corporate VPN gateway.
  3. Tabby Authentication: Enable Tabby's built-in user management and token-based authentication feature via its admin UI. Every developer must generate a unique API token to connect their IDE extension, allowing administrators to audit access and revoke tokens when a developer offboards.
---

Client Integration: Setting up VS Code and JetBrains

Once the server is secured and running under an enterprise domain (e.g., [https://ai-coder.youragency.com](https://ai-coder.youragency.com)), onboarding your engineering team takes less than five minutes per developer:

For Visual Studio Code:

  • Search for and install the official Tabby extension from the VS Code Marketplace.
  • Open the extension settings and locate the Server Endpoint configuration field.
  • Input your secured URL: [https://ai-coder.youragency.com](https://ai-coder.youragency.com).
  • Provide the personal access token generated from the Tabby Admin console.

The extension will instantly establish a connection. As developers write code, inline ghost text suggestions will appear automatically, powered entirely by your centralized cloud server.

---

The Strategic Advantages for Digital Agencies

Transitioning from a public SaaS model to a centralized, self-hosted AI assistant yields profound strategic dividends:

1. Absolute Data Sovereignty & Client Compliance: Because no data ever leaves your controlled cloud environment, you can confidently assure high-security clients (such as financial institutions, fintech startups, or healthcare enterprises) that their source code is completely isolated and safe from third-party AI training loops.

2. Context-Aware Custom Indexing: Tabby allows you to index your agency's historical repositories and internal boilerplates. This means the AI won't just suggest generic code; it will suggest code styled exactly like your agency’s best practices, leveraging internal libraries and architecture patterns seamlessly.

3. Massive ROI and Cost Predictability: Instead of paying $10–$20 per user every month, your costs are capped at a flat, predictable cloud server monthly fee. As your agency scales from 20 to 100 engineers, your infrastructure costs scale marginally rather than linearly, drastically improving gross margins.

---

Conclusion

Embracing AI coding assistants is no longer optional for agencies wanting to stay competitive, but how you implement them matters. Deploying Tabby and Ollama on a secure cloud server gives your agency the best of both worlds: the hyper-productivity of modern AI development paired with the rigorous security, privacy, and cost-efficiency demanded by corporate enterprise clients. Take ownership of your development tools today, protect your IP, and build a smarter, faster agency infrastructure.