Optimizing Docker Images with Distroless: Solution to Reduce 90% Image Size and Enhance System Security
Detailed guide on optimizing Docker images using Distroless to reduce up to 90% image size, enhance system security, and suit lightweight LLM self-hosting environments on NVMe VPS....
Introduction to Distroless and Benefits for Self-Hosted LLM Environments
Distroless is a set of Docker images built from scratch, containing only the necessary libraries to run applications, completely removing system tools like shell, package managers, and unnecessary libraries. This makes Distroless an ideal solution for self-hosted environments such as AI Gateways (Ollama, LiteLLM) on NVMe VPS, where disk space and RAM resources are strictly limited.
Compared to standard Docker images like alpine, ubuntu, or debian, Distroless images can reduce size by up to 90%. For example, a simple Python application may occupy 500MB on alpine but only 50MB on Distroless. Additionally, by eliminating system tools, Distroless significantly reduces the attack surface, enhancing system security and lowering the risk of exploitation through system vulnerabilities.
Setting Up and Configuring Docker Images with Distroless
To use Distroless, you need to select an image that matches your application's language and runtime environment. Distroless offers multiple images for popular languages such as python, nodejs, golang, and java. Below is an example with a Python application:
# Dockerfile using Distroless Python
FROM gcr.io/distroless/python3-debian12
# Copy application source code into the container
COPY . /app
# Set the working directory
WORKDIR /app
# Install required libraries
RUN pip install --no-cache-dir -r requirements.txt
# Run the application
CMD ["python", "app.py"]
To check the image size after building, you can use the following command:
docker images --format "{{.Repository}}:{{.Tag}} \t {{.Size}}" | grep "your-image-name"
Compared to the python:3.11-alpine image (approximately 100MB), a Distroless image can be over 50% smaller.
Testing and Troubleshooting OOM Errors on NVMe VPS
On NVMe VPS with limited RAM, optimizing Docker image size is the first step to avoid Out of Memory (OOM) errors. However, it is essential to thoroughly check system resources. Below are the steps for testing and troubleshooting:
- Check RAM and swap usage: Use the command
free -hto view available RAM and swap status. - Adjust Docker RAM limits: In the Docker daemon configuration file
/etc/docker/daemon.json, add:
{
"default-shm-size": "64m",
"memory": "1g",
"memory-swap": "2g"
}
- Set timeouts for Ollama/LiteLLM: In the application configuration, reduce the timeout to prevent prolonged processing from consuming excessive resources.
- Handle OOM errors: When an OOM error occurs, check logs using
journalctl -u docker.serviceordocker logs <container-id>. If the error is related to OOM, reduce the image size or increase RAM/swap.
To improve stability, you can use LiteLLM AI Gateway with Redis caching and rate limiting to manage load more effectively.
References and Official Documentation
- Distroless GitHub: Official documentation for Distroless images.
- Dockerfile Reference: Detailed guide on Dockerfile syntax.
- Docker Daemon Configuration: Docker daemon configuration guide.
- High-RAM Cloud VPS for Self-Hosted LLMs & API Gateways: Suitable VPS solutions for self-hosted LLMs.
High-RAM Cloud VPS Plans Optimized for AI, Databases & Cache
Enterprise RAID 10 NVMe SSDs, CMC & FPT District 9 uplinks with Anycast Anti-DDoS and 99.99% SLA.
VPS-4E
10% VAT included
- CPU: 4 vCPU
- RAM: 8 GB
- SSD: 50 GB NVMe
