Building a Lightweight, S3-Compatible Object Storage Clusters in Vietnam Using GarageHQ

Introduction: The Quest for Cost-Effective, Localized Object Storage

In the modern cloud architecture landscape, AWS S3 has become the de facto standard for object storage. However, for many businesses operating in Vietnam, relying solely on global public cloud providers introduces distinct challenges: high international bandwidth costs, fluctuating latency, and compliance concerns regarding local data residency. While enterprise-grade self-hosted alternatives like Ceph and MinIO exist, they are notoriously resource-intensive, demanding substantial CPU and RAM overhead that prices out budget-friendly, low-spec Virtual Private Servers (VPS).

Enter GarageHQ (Garage), an open-source, lightweight distributed object storage service designed specifically to run on cluster topologies composed of low-end, heterogeneous hardware. By utilizing a unique architecture based on a conflict-free replicated data type (CRDT) and a consistent hashing ring, Garage allows engineering teams to construct a robust, S3-compatible storage cluster across cheap, geo-distributed nodes. This post provides a technical blueprint for implementing a multi-node GarageHQ cluster using three low-spec VPS instances hosted by local Vietnamese providers.

Why GarageHQ for Low-Spec Hardware?

Traditional distributed storage engines like Ceph require heavy metadata servers and significant memory footprints to map data blocks. MinIO, while simpler, is fundamentally designed for larger multi-disk arrays and can struggle with clustering over wide-area networks or constrained hardware. GarageHQ breaks this mold through several architectural advantages:

Minimal Memory Footprint: Runs comfortably on instances with as little as 512MB to 1GB of RAM, leaving host resources free for other lightweight application workloads.
No Dedicated Metadata Servers: Metadata is distributed across the cluster natively, eliminating single points of failure and reducing complex coordination overhead.
Network Tolerance: Designed from the ground up to cope with the realities of commodity internet connections, making it ideal for connecting different VPS providers across Hanoi, Da Nang, and Ho Chi Minh City.

Architecture Design & Prerequisites

To achieve optimal redundancy and split-brain prevention, a minimum of three nodes is required to form a consensus quorum. For this deployment blueprint, our topology consists of three low-spec VPS instances from localized Vietnamese providers (such as Viettel IDC, VNPT, CMC Telecom, or regional commodity providers like Vietnix and AZDIGI):

Node 1 (Hanoi): 1 vCPU, 1GB RAM, 20GB SSD NVMe
Node 2 (Ho Chi Minh City): 1 vCPU, 1GB RAM, 20GB SSD NVMe
Node 3 (Da Nang / Alternative Zone): 1 vCPU, 1GB RAM, 20GB SSD NVMe

Note: For optimal performance and data resilience, ensure that the inter-node latency between these regions remains below 50ms, which is standard for internal Vietnamese routing networks.

Network and Firewall Configuration

Garage relies on two primary ports that must be carefully exposed and firewalled:

RPC Port (Default: 3901): Used for internal communication, data replication, and cluster consensus. This must be restricted explicitly to the public or private IP addresses of the three participating nodes.
S3 API Port (Default: 3900): The endpoint exposed to your applications for standard S3 API operations (PutObject, GetObject, etc.). This can be exposed publicly or reverse-proxied via Nginx/Caddy.

Step-by-Step Deployment Guide

Step 1: Installing the Garage Binary

Because Garage is written in Rust, it compiles down to a single, highly efficient static binary. Download the latest stable release on all three nodes:

wget [https://garagehq.deuxfleurs.fr/dist/v0.9.0/x86_64-unknown-linux-musl/garage](https://garagehq.deuxfleurs.fr/dist/v0.9.0/x86_64-unknown-linux-musl/garage)
chmod +x garage
sudo mv garage /usr/local/bin/

Step 2: Configuring the Cluster Nodes

Create a configuration file located at /etc/garage.toml on each server. Below is a production-ready template tailored for Node 1. Repeat this process for Node 2 and Node 3, ensuring you update the rpc_public_addr and local path parameters accordingly:

metadata_dir = "/var/lib/garage/meta"
data_dir = "/var/lib/garage/data"

bind_public_rcp = "0.0.0.0:3901"
bind_s3 = "0.0.0.0:3900"

rpc_secret = "GENERATED_HEX_SECRET_KEY_SHARE_AMONG_ALL_NODES"

[rpc_public_addr]
node1 = "103.xxx.xxx.1:3901"

Generate a secure, random hex string for the rpc_secret using openssl rand -hex 32. This secret key must be identical across all members of the cluster to authenticate inter-node traffic securely.

Step 3: Initializing and Joining the Cluster

Start the Garage daemon on all nodes. For production systems, wrap the binary execution in a standard systemd service file to manage restarts and boot logging. Once the daemons are running, connect to Node 1 to check the status and explicitly instruct it to connect to the peer nodes:

garage status
garage node connect node2_ip:3901
garage node connect node3_ip:3901

Verify that all nodes are visible in the connection topology by running garage status again. At this stage, the nodes are aware of one another but no data has been assigned to them.

Step 4: Configuring the Layout and Replication Factor

Garage uses a flexible system of "layouts" to define how data is distributed across physical locations. To commit your nodes to the storage ring and configure a replication factor of 3 (meaning every object is fully copied to all three distinct physical VPS instances), run the following configuration commands:

garage layout assign  --zone vn-north --capacity 1
garage layout assign  --zone vn-south --capacity 1
garage layout assign  --zone vn-central --capacity 1

garage layout apply --version 1

By defining explicit zones (vn-north, vn-south, vn-central), Garage intelligently ensures that data copies are segregated geographically, protecting your system even if an entire regional data center faces an extended power or network outage.

Exposing the S3 API and Integrating with Applications

With the cluster unified, you can now provision standard S3 credentials and buckets. Run the following commands to create an API key pair and an initial bucket infrastructure:

# Create an API Key
garage key create my-app-key

# Create a Bucket
garage bucket create my-company-backups

# Link Key permissions to Bucket
garage bucket allow my-company-backups --read --write --key my-app-key

The console will output an Access Key ID and a Secret Access Key. You can seamlessly plug these credentials into standard dev tooling such as the AWS CLI, MinIO Client (mc), rclone, or direct application SDKs (boto3, aws-sdk-go). Simply override the target endpoint URL to point to your localized cluster: http://your-vps-ip:3900.

Performance Optimization and Maintenance for Low-Spec Nodes

Running distributed storage on constrained hardware requires careful operational tuning to prevent out-of-memory (OOM) errors and disk I/O bottlenecks:

Linux Virtual Memory Tuning: Low-spec VPS instances can suffer from aggressive swapping. Adjust your sysctl configurations by setting vm.swappiness = 10 to keep metadata caches strictly in RAM while avoiding unnecessary disk thrashing.
Reverse Proxy Layering: It is highly recommended to place a lightweight reverse proxy like Caddy or Nginx in front of your S3 API port. This allows you to handle SSL/TLS termination efficiently and implement strict rate-limiting policies before queries hit the Garage engine.
Monitoring and Telemetry: Garage natively exports Prometheus metrics via an internal HTTP endpoint. Pair this with a lightweight monitoring agent to keep track of CPU usage, storage capacity growth, and replication queue lag.

Conclusion

Building an S3-compatible storage cluster doesn't require a massive enterprise budget or complex infrastructure teams. By choosing GarageHQ and leveraging affordable, locally hosted Vietnamese VPS instances, companies can achieve ultra-low latency data access, fulfill local data regulations, and maintain independent control over their object storage layer. With minimal upkeep and absolute resource efficiency, this architecture proves that high-availability engineering can scale down just as elegantly as it scales up.