Building High-Availability Shared Storage: Deploying GlusterFS on 3 Budget VPS Instances

Introduction: The Challenge of High-Availability Storage on a Budget

In modern cloud architecture, achieving High Availability (HA) is often synonymous with spiraling infrastructure costs. For businesses running web clusters, content management systems (CMS) like WordPress, or containerized applications via Docker Swarm and Kubernetes, a shared file system is critical. If your shared storage goes down, your entire application delivery pipeline halts.

Traditionally, enterprise-grade HA storage required expensive Storage Area Networks (SAN) or premium cloud-native managed services. However, by leveraging GlusterFS—an open-source, software-defined distributed file system—and three budget-friendly Virtual Private Servers (VPS), you can build a resilient, replicated storage cluster that withstands node failures without breaking the bank. This guide will walk you through the architecture, prerequisites, and step-by-step deployment of a 3-node GlusterFS cluster.

Why GlusterFS and Why a 3-Node Minimum?

GlusterFS aggregates various storage servers (known as bricks) into a single, unified network file system. In a replicated configuration, data written to one node is synchronously mirrored across other nodes in the cluster.

But why do we specifically need three nodes instead of two? The answer lies in data integrity and split-brain prevention:

• The Split-Brain Dilemma: In a 2-node cluster, if the network connection between the two servers breaks, both servers might assume the other is dead. If they both continue writing data independently, the file system becomes catastrophically corrupted.
• The Quorum Solution: By introducing a third node, GlusterFS implements a quorum mechanism. A volume remains writable only if a strict majority (more than 50%) of the nodes can communicate with each other. If one node fails in a 3-node cluster, the remaining two nodes maintain quorum (2 out of 3), allowing your applications to continue running seamlessly without data corruption risk.

Pre-requisites and Environment Preparation

Before executing commands, you need to provision your infrastructure. For this setup, we will use three budget VPS instances running Ubuntu 22.04 LTS or Ubuntu 24.04 LTS. To ensure optimal performance and minimal latency, select a VPS provider that offers private networking and locate all three instances within the same datacenter region.

Network Architecture Mapping

We will define our cluster nodes using the following internal IP mapping. Ensure your provider\'s firewall permits unrestricted communication between these internal IPs over GlusterFS ports.

Important Note on Storage: For production workloads, it is highly recommended to attach a secondary unformatted block storage volume to each VPS for the GlusterFS bricks, rather than using the root filesystem partition. This prevents storage leaks from crashing your main operating system.

Step-by-Step Deployment Guide

Step 1: Configuring Hostnames and DNS Resolution

GlusterFS relies heavily on consistent hostname resolution. Log into each VPS via SSH and set the respective hostnames. For example, on Node 1:

sudo hostnamectl set-hostname node1.storage.local

Next, open the /etc/hosts file on all three nodes and append the following lines to ensure they can resolve each other without a public DNS server:

10.0.0.11 node1.storage.local node1
10.0.0.12 node2.storage.local node2
10.0.0.13 node3.storage.local node3

Step 2: Preparing the Storage Bricks

Assuming you have attached a secondary disk (e.g., /dev/vdb) to each server, we need to format it with the XFS file system, which is highly optimized for GlusterFS workloads. Run these commands on all three nodes:

sudo mkfs.xfs -f /dev/vdb
sudo mkdir -p /mnt/gluster_brick
echo "/dev/vdb /mnt/gluster_brick xfs defaults 0 0" | sudo tee -a /etc/fstab
sudo mount -a

Now create the specific directory that will serve as the inner brick:

sudo mkdir -p /mnt/gluster_brick/brick

Step 3: Installing GlusterFS Server

To ensure you receive the latest stable updates and performance patches, add the official GlusterFS upstream PPA repository. Execute the following sequence on all nodes:

sudo apt install software-properties-common -y
sudo add-apt-repository ppa:gluster/glusterfs-11 -y
sudo apt update
sudo apt install glusterfs-server -y
sudo systemctl enable glusterd
sudo systemctl start glusterd

Verify that the service is running actively by executing sudo systemctl status glusterd.

Step 4: Creating the Trusted Storage Pool

Now we must link the independent servers into a single trusted storage cluster. This step only needs to be executed from Node 1 (node1.storage.local). Run the following peer probe commands:

sudo gluster peer probe node2.storage.local
sudo gluster peer probe node3.storage.local

To confirm that the cluster integration was successful, check the peer status from Node 1:

sudo gluster peer status

The output should state that Peer 2 and Peer 3 are Connected.

Step 5: Initializing the Replicated Volume

With the storage pool trusted, we can now define our high-availability volume. We will create a 3-way replicated volume named ha_shared_volume. Run this command exclusively on Node 1:

sudo gluster volume create ha_shared_volume replica 3 \
node1.storage.local:/mnt/gluster_brick/brick \
node2.storage.local:/mnt/gluster_brick/brick \
node3.storage.local:/mnt/gluster_brick/brick

Once initialized, activate the volume so it becomes accessible to clients:

sudo gluster volume start ha_shared_volume

Verify the health, configuration parameters, and status of your new distributed volume with:

sudo gluster volume info

Connecting Clients with Auto-Failover Capability

Your HA shared storage system is now live, but to utilize it, your client applications (such as Nginx web servers or application servers) need to mount it. GlusterFS provides a native client driver that automatically handles failover. If the specific node a client mounts through goes offline, the client seamlessly shifts its network I/O traffic to one of the surviving nodes in the cluster.

On your client server, install the client package:

sudo apt update && sudo apt install glusterfs-client -y

Create a mount point and bind the volume using the native GlusterFS mount type:

sudo mkdir -p /var/www/shared_data
sudo mount -t glusterfs node1.storage.local:/ha_shared_volume /var/www/shared_data

To guarantee the volume mounts automatically upon server reboots, append this entry to your client\'s /etc/fstab file:

node1.storage.local:/ha_shared_volume /var/www/shared_data glusterfs defaults,_netdev 0 0

SEO Tip: Using the _netdev mount option is a critical best practice. It instructs the operating system to delay mounting the filesystem until network interfaces are completely up, preventing boot hangs.

Testing High Availability and Failover Capabilities

The true value of this infrastructure lies in its resilience. Let\'s validate that our cluster behaves correctly during a simulated infrastructure crash.

1. On your client machine, create a dummy file inside the mounted directory: touch /var/www/shared_data/test_file.txt.
2. Log into Node 2 and Node 3 to verify that test_file.txt has instantly replicated to their respective local brick directories (/mnt/gluster_brick/brick/).
3. Simulate a catastrophic hardware failure on Node 1 by completely shutting down its server instance or stopping its network interface.
4. Return to your client server and run an I/O test, such as writing a new file or appending data. You will observe that the file system remains fully readable and writable. The client driver automatically redirected operations to Node 2 or Node 3 because a majority quorum was successfully maintained.

Conclusion and Operational Best Practices

Deploying GlusterFS on three budget-friendly VPS instances proves that high availability does not require an enterprise budget. By eliminating single points of failure at the storage layer, you provide a bulletproof foundation for your web applications and services.

As you move this architecture into production, remember to monitor disk utilization closely across all nodes, implement automated backups of your underlying bricks, and ensure that your private network latency remains consistently low. With software-defined storage, structural engineering quality matters far more than premium hardware pricing.