Building a Self-Hosted Web3 Decentralized IPFS Pinning Service Using IPFS Cluster for dApps

Introduction: The Decentralization Dilemma in Web3 Storage

InterPlanetary File System (IPFS) has become the de facto standard for decentralized storage in the Web3 ecosystem. From hosting decentralized application (dApp) frontends to storing NFT metadata and smart contract assets, IPFS provides the content-addressed foundation that modern blockchain applications require. However, a common misconception among developers is that uploading a file to IPFS guarantees its permanent availability. In reality, IPFS nodes aggressively garbage-collect unpinned data to free up disk space.

To keep data available, it must be "pinned." While commercial pinning services exist, relying on a single third-party provider introduces a centralized point of failure, potential API bottlenecks, and recurring subscription costs. For enterprise-grade dApps requiring complete control over data sovereignty, compliance, and availability, building a self-hosted, multi-node IPFS Pinning Service using IPFS Cluster on Virtual Private Servers (VPS) is the optimal architectural choice.

This technical guide provides a step-by-step blueprint for configuring a production-ready, self-hosted IPFS Pinning Service capable of automating data replication across multiple nodes, ensuring maximum uptime and data integrity for your Web3 projects.

Understanding the Architecture: IPFS Kubo vs. IPFS Cluster

Before diving into the configuration, it is critical to understand the distinction between a standalone IPFS node and an IPFS Cluster setup:

• IPFS Kubo (formerly go-ipfs): This is the core daemon responsible for connecting to the peer-to-peer (P2P) network, fetching content-addressed blocks, and pinning data locally.
• IPFS Cluster: This is a separate distributed application that sits on top of multiple IPFS nodes. It orchestrates allocation, manages a shared global pinset, and ensures that data is automatically replicated across the cluster based on predefined replication factors.

Key Benefit: If one VPS node goes offline due to a hardware failure or network split, IPFS Cluster automatically re-allocates and repins the affected content to other active nodes in the network, maintaining the target replication factor without human intervention.

Prerequisites and Environment Setup

To build a resilient cluster, we recommend a minimum architecture of two or more VPS instances distributed across different geographic regions (e.g., AWS, DigitalOcean, or Linode). Each VPS should meet the following baseline specifications:

• OS: Ubuntu 22.04 LTS or newer
• CPU/RAM: 2 vCPUs and 4GB RAM minimum (IPFS can be memory-intensive during heavy DHT provider operations)
• Storage: High-speed NVMe SSD (size depends on your dApp's data footprint)
• Network: Public IPv4 address with open ports for P2P communication (Ports 4001 for IPFS, 9094/9096 for IPFS Cluster)

Step 1: Installing IPFS Kubo and IPFS Cluster

First, SSH into your primary VPS node (Node 1) and update the package repository. We will download the pre-compiled binaries provided by Protocol Labs via the official distributions platform.

1.1 Install IPFS Kubo

Execute the following commands to download and install the core IPFS daemon:

wget [https://dist.ipfs.tech/kubo/v0.26.0/kubo_v0.26.0_linux-amd64.tar.gz](https://dist.ipfs.tech/kubo/v0.26.0/kubo_v0.26.0_linux-amd64.tar.gz)
tar -xvzf kubo_v0.26.0_linux-amd64.tar.gz
cd kubo
sudo ./install.sh
ipfs --version

Initialize the IPFS repository. For a server deployment, use the server profile to optimize routing and minimize background bandwidth consumption on public DHT networks:

ipfs init --profile server

1.2 Install IPFS Cluster Service and CLI

Next, install the cluster daemon (ipfs-cluster-service) and the command-line interface tool (ipfs-cluster-ctl):

wget [https://dist.ipfs.tech/ipfs-cluster-service/v1.0.8/ipfs-cluster-service_v1.0.8_linux-amd64.tar.gz](https://dist.ipfs.tech/ipfs-cluster-service/v1.0.8/ipfs-cluster-service_v1.0.8_linux-amd64.tar.gz)
tar -xvzf ipfs-cluster-service_v1.0.8_linux-amd64.tar.gz
cd ipfs-cluster-service
sudo cp ipfs-cluster-service /usr/local/bin/

wget [https://dist.ipfs.tech/ipfs-cluster-ctl/v1.0.8/ipfs-cluster-ctl_v1.0.8_linux-amd64.tar.gz](https://dist.ipfs.tech/ipfs-cluster-ctl/v1.0.8/ipfs-cluster-ctl_v1.0.8_linux-amd64.tar.gz)
tar -xvzf ipfs-cluster-ctl_v1.0.8_linux-amd64.tar.gz
cd ipfs-cluster-ctl
sudo cp ipfs-cluster-ctl /usr/local/bin/

Step 2: Configuring the IPFS Cluster Swarm

An IPFS Cluster relies on a shared secret key (a 32-byte hex-encoded string) to authenticate and secure communication between authorized nodes. This prevents unauthorized peers from joining your pinning network.

2.1 Generate the Cluster Secret

On Node 1, generate a unique cluster secret and export it to your environment:

export CLUSTER_SECRET=$(od -vN 32 -An -tx1 /dev/urandom | tr -d ' 
')
echo "Your Cluster Secret: $CLUSTER_SECRET"

Note: Save this secret key in a secure location. You will need to apply the exact same key to all subsequent nodes joining the cluster.

2.2 Initialize the Cluster Configuration

Initialize the configuration file on Node 1 using the generated secret:

ipfs-cluster-service init

Open the generated configuration file located at ~/.ipfs-cluster/service.json and replace the null or default secret value with your newly generated 32-byte string. Ensure that the replication_factor_min and replication_factor_max values are adjusted according to your infrastructure scale (e.g., set to 2 for a two-node redundant setup).

Step 3: Launching Services and Connecting Node 2

To ensure system stability, both IPFS Kubo and IPFS Cluster should run as background system services via systemd.

3.1 Configure systemd Services

Create a systemd unit file for IPFS Kubo at /etc/systemd/system/ipfs.service:

[Unit]
Description=IPFS Kubo Daemon
After=network.target

[Service]
User=ubuntu
ExecStart=/usr/local/bin/ipfs daemon --migrate=true
Restart=on-failure

[Install]
WantedBy=multi-user.target

Create a similar systemd unit file for the cluster service at /etc/systemd/system/ipfs-cluster.service, making sure to pass the CLUSTER_SECRET environment variable within the configuration block.

sudo systemctl daemon-reload
sudo systemctl enable --now ipfs
sudo systemctl enable --now ipfs-cluster

3.2 Joining Secondary Nodes to the Cluster

On Node 2, install Kubo and IPFS Cluster using the identical binaries. Initialize IPFS Cluster on Node 2, then manually edit its service.json to insert the identical CLUSTER_SECRET generated on Node 1.

To bind Node 2 to Node 1, bootstrap the cluster service using Node 1's multiaddress:

ipfs-cluster-service daemon --bootstrap /ip4/[NODE_1_IP]/tcp/9096/p2p/[NODE_1_CLUSTER_PEER_ID]

Verify that both nodes see each other by executing: ipfs-cluster-ctl peers ls. The terminal should display both peer IDs marked as online.

Step 4: Integrating the Pinning Service with dApps

With the cluster operational, your dApp can now interact with it programmatically. IPFS Cluster exposes a fully compliant IPFS Pinning Service API (IPFS PSA) endpoint at port 9094. This endpoint adheres to the standardized OpenAPI specification adopted by mainstream Web3 frameworks like Ethers.js, Web3.js, and Helia.

When your dApp front-end or backend server initiates an IPFS add or pin command via the cluster endpoint, the file is accepted by the primary node and concurrently distributed to the secondary node according to your replication rules. This architecture completely eliminates centralized data hosting dependencies, offering an immutable, redundant infrastructure tailored for scalable Web3 deployments.

Conclusion

Transitioning from commercial cloud storage providers to a self-hosted IPFS Cluster represents a significant leap forward in realizing true technical decentralization. By following this guide, your organization secures data sovereignty, mitigates the risks of unexpected service shutdowns, and provides your dApp users with verified, decentralized high-availability storage. As your network scales, scaling up capacity is as simple as launching an additional VPS and bootstrapping it into your existing cluster matrix.