Optimizing Keycloak Auth Server Memory on Low-Spec Linux VPS with Quarkus Tuning

Introduction: The Challenge of Hosting Keycloak on Low-Spec VPS

Identity and Access Management (IAM) is a critical component of modern software architecture. Keycloak, an industry-standard open-source IAM solution, offers robust security features including Single Sign-On (SSO), social login, and identity brokering. However, deploying Keycloak on a resource-constrained Linux Virtual Private Server (VPS)—such as an entry-level instance with 1GB to 2GB of RAM—frequently presents a significant engineering challenge.

Legacy versions of Keycloak relied on the WildFly application server, which carried a massive memory footprint and was notoriously difficult to slim down. Fortunately, modern Keycloak distributions are built on top of Quarkus, a Kubernetes-native Java framework designed for low memory consumption and rapid boot times. Despite this architectural leap, default Keycloak configurations remain optimized for enterprise-grade hardware. Running Keycloak out-of-the-box on a low-spec VPS will inevitably lead to high memory utilization, sluggish performance, or worse, termination by the Linux Out-Of-Memory (OOM) Killer. This guide provides a production-tested approach to tuning Quarkus and the Java Virtual Machine (JVM) to run Keycloak efficiently on low-memory environments.

1. Understanding the Memory Landscape of Keycloak on Quarkus

Before adjusting configuration files, it is vital to understand where Keycloak allocates its memory. A running Keycloak instance divides its memory consumption into three primary areas:

• JVM Heap Memory: Where active Java objects reside. Keycloak uses this for session management, token processing, and caching.
• JVM Off-Heap (Metaspace & Direct Memory): Memory utilized by the JVM itself for class metadata, thread stacks, and JIT compilation.
• Infinispan Cache: An embedded distributed cache used by Keycloak to keep user sessions, authentication sessions, and brute-force protection data in memory.

On low-spec hardware, the goal of tuning is to restrict both Heap and Off-Heap memory while optimizing the Infinispan cache to avoid excessive swapping to disk.

2. JVM Tuning: Adjusting Heap and Garbage Collection

The most direct way to control Keycloak's memory footprint is by managing the JVM parameters via the JAVA_OPTS_APPEND environment variable. By default, the JVM dynamically determines heap size based on available system memory, which can be overly aggressive on low-spec servers.

Setting Explicit Heap Limits

For a VPS with 1GB to 2GB of total RAM, allocating a fixed heap size prevents the JVM from competing with operating system processes. We recommend setting the initial heap (-Xms) and maximum heap (-Xmx) to identical values. This avoids the CPU overhead of dynamically resizing the heap during peak loads.

Recommended Configuration for a 1GB RAM VPS:
-Xms256m -Xmx256m

Recommended Configuration for a 2GB RAM VPS:
-Xms512m -Xmx512m

Choosing the Right Garbage Collector

While the Garbage-First (G1) collector is the modern standard for large applications, it carries noticeable memory overhead for internal data structures. For low-memory environments, the Serial Garbage Collector (Serial GC) or the Parallel Garbage Collector is often superior. The Serial GC operates on a single thread, reducing memory fragmentation and overhead at the cost of slight pause times, which are negligible for low-to-medium traffic authentication endpoints.

To explicitly enable the Serial GC, append the following flag:

-XX:+UseSerialGC

3. Quarkus Optimization Strategies

Quarkus provides internal build-time and runtime optimizations that significantly impact how Keycloak utilizes system resources. By adjusting the keycloak.conf file or setting environment variables, we can prune unused features.

Disabling Unused Features

Keycloak bundles several features that may not be necessary for your specific deployment. Disabling components like metrics, health checks, or specific database drivers reduces the total number of loaded classes, thereby shrinking the JVM Metaspace footprint.

# keycloak.conf
features-disabled=admin-fine-grained-authz,authorization
metrics-enabled=false
health-enabled=false

Optimizing Database Connection Pooling

Keycloak utilizes Agroal for database connection pooling within the Quarkus framework. Each open connection consumes network buffers and memory threads. In a low-spec environment, large pools provide diminishing returns and exhaust resources rapidly.

Reduce the maximum pool size to mirror your low-core CPU availability. A lean database configuration prevents memory spikes under concurrent login requests:

# keycloak.conf
db-pool-initial-size=2
db-pool-max-size=5
db-pool-min-size=2

4. Configuring Infinispan Caches for Low Memory

Infinispan handles user sessions, login failures, and realm data caching. By default, Infinispan keeps an unlimited or highly permissive number of entries in memory. On a VPS with low RAM, we must enforce strict eviction policies.

To tune Infinispan, copy the default configuration file (cache-ispn.xml) from the Keycloak distribution and reference it in your keycloak.conf:

cache=config/cache-ispn.xml

Inside your custom cache-ispn.xml, define a maximum number of entries (max-count) for user sessions and client sessions to force eviction of idle data:

Enforcing a low max-count ensures that inactive sessions do not quietly accumulate and exhaust system memory over time.

5. Linux OS-Level Tweaks: Swap and Overcommit

Tuning Keycloak internally is only half the battle; the underlying Linux OS must be configured to handle low-memory stress gracefully without triggering abrupt application termination.

Configuring a Swap File

Running a Java application on a VPS without swap space is highly dangerous. If Keycloak breaches physical memory limits, the kernel will kill it instantly. A 1GB or 2GB swap file acts as a safety valve, allowing the operating system to offload idle pages from RAM to disk.

1. Create a 2GB swap file: sudo fallocate -l 2G /swapfile
2. Set appropriate permissions: sudo chmod 600 /swapfile
3. Format as swap: sudo mkswap /swapfile
4. Activate swap: sudo swapon /swapfile

To ensure system responsiveness, reduce the system swappiness value to 10. This tells the kernel to avoid using swap space unless absolutely necessary, preserving fast physical RAM for active Keycloak operations.

sudo sysctl vm.swappiness=10

Conclusion: A Lean, Secure Identity Provider

Optimizing Keycloak on Quarkus for a low-spec Linux VPS requires a multi-layered approach. By constraining the JVM heap, deploying a lightweight garbage collector, pruning unused Quarkus features, throttling connection pools, and setting up Linux swap safeguards, you can run a highly stable authentication server on a budget. Monitoring performance metrics post-deployment remains crucial, but with these configurations, your Keycloak instance will remain resilient, highly responsive, and resource-conscious.