Building a Self-Hosted Web3 Decentralized IPFS Pinning Service with IPFS Cluster for dApps

Introduction: The Storage Challenge in Decentralized Applications

InterPlanetary File System (IPFS) has become the backbone of decentralized storage for Web3 applications, powering everything from NFT metadata to frontend hosting for decentralized applications (dApps). However, a common misconception among newcomers is that uploading a file to IPFS guarantees its permanent availability. In reality, IPFS nodes routinely garbage-collect unpinned data to free up resources. If your dApp relies on temporary nodes or public gateways, critical assets risk vanishing from the network.

To ensure high availability, developers typically turn to centralized pinning services. While convenient, relying on third-party providers introduces points of failure, potential censorship, and unpredictable API costs that conflict with the core ethos of web3. The alternative? Building your own self-hosted, enterprise-grade IPFS Pinning Service using IPFS Cluster on a Virtual Private Server (VPS). This guide provides a production-ready architectural walkthrough to help you deploy, orchestrate, and scale a private pinning network for your dApps.

Understanding the Architecture: IPFS vs. IPFS Cluster

Before diving into terminal commands, it is essential to understand how the components interact. A standalone IPFS daemon handles data storage, block routing, and peer-to-peer communication. However, managing data replication across multiple distinct IPFS nodes manually is highly inefficient.

IPFS Cluster solves this by acting as an orchestration layer on top of your IPFS daemons. It coordinates the allocation, pinning, and replication of Content Identifiers (CIDs) across a swarm of nodes. When your dApp requests a pin via the IPFS Cluster API, the cluster distributes the pin request across its members based on consensus algorithms, ensuring redundancy and automated state synchronization.

Key Benefit: By separating the storage layer (IPFS) from the consensus/orchestration layer (IPFS Cluster), you can scale your pinning service horizontally by adding more VPS instances as your dApp's data footprint expands.

Step 1: System Requirements and VPS Provisioning

For a production-grade Web3 pinning service, your VPS instances require adequate disk I/O, memory, and bandwidth to handle concurrent peer connections. The following baseline specifications are recommended per node:

• OS: Ubuntu 22.04 LTS or Ubuntu 24.04 LTS (64-bit)
• CPU: 2 vCPUs minimum (Compute-optimized preferred)
• RAM: 4GB minimum (8GB recommended, as IPFS bitswap processes can be memory-intensive)
• Storage: NVMe SSD (Size depends on your dApp data needs, minimum 100GB recommended)
• Network: 1 Gbps unmetered or high-allowance bandwidth with a dedicated static Public IPv4 address

Firewall Configuration

Before installing software, configure your firewall (e.g., UFW) to allow essential ports while blocking unauthorized access to internal management interfaces:

1. Port 22/TCP: SSH access (Restrict to trusted IPs if possible).
2. Port 4001/TCP & UDP: IPFS Swarm port (Must be open to the public internet for global peer connections).
3. Port 9094/TCP: IPFS Cluster HTTP API proxy (Restrict access or protect with basic auth/TLS).
4. Port 9096/TCP & UDP: IPFS Cluster Swarm (Must be open to other cluster peers for consensus).

Step 2: Installing Kubo (Go-IPFS) and IPFS Cluster

We will utilize the official pre-built binaries provided by Protocol Labs. Log into your VPS via SSH and execute the following steps to download and install the binaries.

Installing Kubo (IPFS Daemon)

wget [https://dist.ipfs.tech/kubo/v0.26.0/kubo_v0.26.0_linux-amd64.tar.gz](https://dist.ipfs.tech/kubo/v0.26.0/kubo_v0.26.0_linux-amd64.tar.gz)
tar -xvzf kubo_v0.26.0_linux-amd64.tar.gz
cd kubo
sudo ./install.sh
ipfs version

Installing IPFS Cluster Service and CLI

wget [https://dist.ipfs.tech/ipfs-cluster-service/v1.1.0/ipfs-cluster-service_v1.1.0_linux-amd64.tar.gz](https://dist.ipfs.tech/ipfs-cluster-service/v1.1.0/ipfs-cluster-service_v1.1.0_linux-amd64.tar.gz)
tar -xvzf ipfs-cluster-service_v1.1.0_linux-amd64.tar.gz
cd ipfs-cluster-service
sudo cp ipfs-cluster-service /usr/local/bin/

wget [https://dist.ipfs.tech/ipfs-cluster-ctl/v1.1.0/ipfs-cluster-ctl_v1.1.0_linux-amd64.tar.gz](https://dist.ipfs.tech/ipfs-cluster-ctl/v1.1.0/ipfs-cluster-ctl_v1.1.0_linux-amd64.tar.gz)
tar -xvzf ipfs-cluster-ctl_v1.1.0_linux-amd64.tar.gz
cd ipfs-cluster-ctl
sudo cp ipfs-cluster-ctl /usr/local/bin/

Step 3: Initializing and Configuring the Services

To run these applications reliably in a production environment, we must initialize their configurations and configure them to run as background system services via systemd.

Initializing IPFS

Initialize the IPFS repository. For a server deployment, use the server profile to optimize network traffic and prevent the node from scanning local network subnets:

ipfs init --profile server

Configuring the IPFS Cluster

IPFS Cluster requires a unique secret key shared among all members of the cluster to authenticate communication. Generate this identity secret on your first node:

CLUSTER_SECRET=$(od -vN 32 -An -tx1 /dev/urandom | tr -d ' 
')
echo "export CLUSTER_SECRET=$CLUSTER_SECRET" >> ~/.bashrc
source ~/.bashrc
ipfs-cluster-service init

Important: Save this 32-byte hex string. Any subsequent peer nodes you add to this cluster must use this exact same CLUSTER_SECRET value in their configuration.

Step 4: Creating Systemd Services for High Availability

Create systemd service files to ensure that both Kubo and IPFS Cluster start automatically upon system boot and recover from unexpected crashes.

Kubo Systemd Configuration

Create /etc/systemd/system/ipfs.service:

[Unit]
Description=IPFS Daemon
After=network.target

[Service]
User=root
Environment=IPFS_PATH=/root/.ipfs
ExecStart=/usr/local/bin/ipfs daemon --migrate=true
Restart=on-failure
KillSignal=SIGINT

[Install]
WantedBy=multi-user.target

IPFS Cluster Systemd Configuration

Create /etc/systemd/system/ipfs-cluster.service, making sure to explicitly pass your cluster secret:

[Unit]
Description=IPFS Cluster Daemon
Requires=ipfs.service
After=ipfs.service

[Service]
User=root
Environment=CLUSTER_SECRET=YOUR_GENERATED_SECRET_HERE
ExecStart=/usr/local/bin/ipfs-cluster-service daemon
Restart=on-failure

[Install]
WantedBy=multi-user.target

Enable and start both services:

sudo systemctl daemon-reload
sudo systemctl enable ipfs --now
sudo systemctl enable ipfs-cluster --now

Step 5: Connecting Multi-Node Swarms for Redundancy

If you are deploying multiple VPS nodes to form a highly resilient pinning network, copy the CLUSTER_SECRET to the secondary nodes during their initialization. Start the secondary nodes by bootstrapping them directly to the multiaddress of your primary node:

ipfs-cluster-service daemon --bootstrap /ip4/PRIMARY_NODE_IP/tcp/9096/p2p/PRIMARY_CLUSTER_PEER_ID

Verify that your nodes are successfully communicating across the decentralized network by running the monitoring command:

ipfs-cluster-ctl peers ls

Step 6: Integrating with Your Web3 dApp Workflow

Your self-hosted decentralized pinning service is now fully operational. To integrate it seamlessly into your Web3 application deployment workflow (such as hardhat scripts, frontend build pipelines, or backend minting workers), you can utilize standard IPFS Pinning Service API endpoints.

To interact with the service programmatically via curl or pinning SDKs, direct requests to your cluster HTTP API endpoint:

ipfs-cluster-ctl pin add QmYourDAppCIDHere

This triggers asynchronous distributed synchronization across your entire private cluster infrastructure, making the data instantly available globally via public IPFS gateways such as ipfs.io or cloudflare-ipfs.com.

Conclusion

By shifting from commercial pinning vendors to a self-hosted IPFS Cluster on your own VPS infrastructure, you unlock ultimate control over your dApp’s data lifecycle. This architecture eliminates third-party platform risks, scales cost-effectively, and reinforces the true decentralized nature of your Web3 platform. As your application grows, you can seamlessly integrate load balancers and expand your storage tier, ensuring your decentralized assets remain persistent, rapid, and truly unstoppable.