Back to articles
AI & Cloud

Optimizing Docker Images with Distroless: Solution to Reduce 90% Image Size and Enhance System Security

Detailed guide on optimizing Docker images using Distroless to reduce up to 90% image size, enhance system security, and suit lightweight LLM self-hosting environments on NVMe VPS....

DPTCloud Engineering4 min read
Optimizing Docker Images with Distroless: Solution to Reduce 90% Image Size and Enhance System Security

Introduction to Distroless and Benefits for Self-Hosted LLM Environments

Distroless is a set of Docker images built from scratch, containing only the necessary libraries to run applications, completely removing system tools like shell, package managers, and unnecessary libraries. This makes Distroless an ideal solution for self-hosted environments such as AI Gateways (Ollama, LiteLLM) on NVMe VPS, where disk space and RAM resources are strictly limited.

Compared to standard Docker images like alpine, ubuntu, or debian, Distroless images can reduce size by up to 90%. For example, a simple Python application may occupy 500MB on alpine but only 50MB on Distroless. Additionally, by eliminating system tools, Distroless significantly reduces the attack surface, enhancing system security and lowering the risk of exploitation through system vulnerabilities.

Setting Up and Configuring Docker Images with Distroless

To use Distroless, you need to select an image that matches your application's language and runtime environment. Distroless offers multiple images for popular languages such as python, nodejs, golang, and java. Below is an example with a Python application:

# Dockerfile using Distroless Python
FROM gcr.io/distroless/python3-debian12

# Copy application source code into the container
COPY . /app

# Set the working directory
WORKDIR /app

# Install required libraries
RUN pip install --no-cache-dir -r requirements.txt

# Run the application
CMD ["python", "app.py"]

To check the image size after building, you can use the following command:

docker images --format "{{.Repository}}:{{.Tag}} \t {{.Size}}" | grep "your-image-name"

Compared to the python:3.11-alpine image (approximately 100MB), a Distroless image can be over 50% smaller.

Testing and Troubleshooting OOM Errors on NVMe VPS

On NVMe VPS with limited RAM, optimizing Docker image size is the first step to avoid Out of Memory (OOM) errors. However, it is essential to thoroughly check system resources. Below are the steps for testing and troubleshooting:

  • Check RAM and swap usage: Use the command free -h to view available RAM and swap status.
  • Adjust Docker RAM limits: In the Docker daemon configuration file /etc/docker/daemon.json, add:
{
  "default-shm-size": "64m",
  "memory": "1g",
  "memory-swap": "2g"
}
  • Set timeouts for Ollama/LiteLLM: In the application configuration, reduce the timeout to prevent prolonged processing from consuming excessive resources.
  • Handle OOM errors: When an OOM error occurs, check logs using journalctl -u docker.service or docker logs <container-id>. If the error is related to OOM, reduce the image size or increase RAM/swap.

To improve stability, you can use LiteLLM AI Gateway with Redis caching and rate limiting to manage load more effectively.

References and Official Documentation

DPTCloud Tier III Infrastructure

High-RAM Cloud VPS Plans Optimized for AI, Databases & Cache

Enterprise RAID 10 NVMe SSDs, CMC & FPT District 9 uplinks with Anycast Anti-DDoS and 99.99% SLA.

View 22 VPS Plans
ECO · vn-02Recommended

VPS-4E

297.000 ₫ / mo

10% VAT included

  • CPU: 4 vCPU
  • RAM: 8 GB
  • SSD: 50 GB NVMe
ECO · vn-02

VPS-5E

614.900 ₫ / mo

10% VAT included

  • CPU: 8 vCPU
  • RAM: 16 GB
  • SSD: 100 GB NVMe
ADVANCED · vn-03Recommended

VPS-4T

548.900 ₫ / mo

10% VAT included

  • CPU: 4 vCPU
  • RAM: 8 GB
  • SSD: 50 GB NVMe

Related Articles

View all
ZaloZalo SupportDiscord0384 753 478