Back to articles
DevOps & Systems

New Generation Distributed Storage Architecture Optimizing Costs with JuiceFS and Ceph OSD on 3 Cheap VPS

A modern distributed storage solution on 3 low-cost VPS using JuiceFS and Ceph OSD enables small and medium enterprises to build scalable, reliable, and cost-effective storage...

DPTCloud Engineering5 min read
New Generation Distributed Storage Architecture Optimizing Costs with JuiceFS and Ceph OSD on 3 Cheap VPS

Image credit: TheDigitalArtist · Pixabay Content License

A modern distributed storage solution on 3 low-cost VPS using JuiceFS and Ceph OSD enables small and medium enterprises to build scalable, reliable, and cost-effective storage systems. This architecture leverages open-source technologies for distributed file management and decentralized data storage, ideal for Docker, databases, and internal storage. This guide details setup steps, configuration, testing, and optimization for Docker, databases, and internal storage.

Overview of the New Generation Distributed Storage Architecture

In the context of increasing data storage demands, businesses need a distributed storage solution that is scalable, highly reliable, and cost-effective. The new generation distributed storage architecture combining JuiceFS and Ceph OSD on 3 low-cost VPS is an efficient choice. JuiceFS is a distributed file system based on Redis and an object storage system, enabling POSIX file access. Ceph OSD (Object Storage Device) is the data storage component in a Ceph cluster, handling distributed storage, backup, and data recovery. Together, these technologies enable the construction of a powerful distributed storage system that can run on affordable NVMe-based VPS infrastructure, such as that provided at NVMe Cloud VPS for Docker, Nginx & Databases.

The main advantages of this architecture include:

  • Cost Optimization: Using 3 low-cost VPS instead of investing in dedicated storage systems or high-end server clusters.
  • Scalability: Easily expand the cluster by adding new VPS nodes.
  • High Reliability: Ceph provides automatic data replication and recovery in case of failure.
  • Easy Integration: Seamlessly integrates with Docker, Nginx, and database applications.

Requirements and Environment Preparation

To deploy this architecture, you need:

  • 3 Linux-based VPS servers (Ubuntu 20.04/22.04 or CentOS 8/9).
  • At least 20GB of NVMe SSD storage on each VPS.
  • Stable network connectivity between the VPS nodes.
  • SSH access to each VPS.

Before starting, ensure the following tools are installed on each VPS:

sudo apt update
sudo apt upgrade -y
sudo apt install -y git curl wget

Set the hostname for each VPS for easier management:

sudo hostnamectl set-hostname vps1
sudo hostnamectl set-hostname vps2
sudo hostnamectl set-hostname vps3

Setting Up Ceph OSD and JuiceFS

On each VPS, install Ceph OSD and JuiceFS. Below are the configuration steps for each component.

Configuring Ceph OSD

On each VPS, install Ceph OSD and initialize the storage directory:

sudo apt install -y ceph-osd
sudo mkdir -p /var/lib/ceph/osd/ceph-0
sudo ceph-osd --mkfs --osd-id 0 --mkkey --osd-data /var/lib/ceph/osd/ceph-0

Start the Ceph OSD service:

sudo systemctl start ceph-osd@0
sudo systemctl enable ceph-osd@0

Configuring JuiceFS

On one VPS (choose VPS1 as the management server), install JuiceFS and create the distributed file system:

curl -sSL https://get.juicefs.com | sh
juicefs format --storage=redis --metaaddr=redis://localhost:6379 --data-shards=1 --name=juicefs-storage redis://localhost:6379
juicefs mount /mnt/juicefs redis://localhost:6379

The value redis://localhost:6379 is the Redis address used to store metadata. You can replace it with a separate Redis server if needed.

Testing and Performance Optimization

After configuration, test the system operation:

  • On VPS1, check the JuiceFS status:
juicefs status redis://localhost:6379
  • On VPS2 and VPS3, check the Ceph OSD status:
sudo ceph osd tree
sudo ceph -s

To optimize performance, you can adjust parameters such as the number of data shards, block size, and Redis configuration to match your usage needs.

Troubleshooting and References

Common issues include:

  • Unable to mount JuiceFS: Check Redis connectivity and ensure JuiceFS is correctly installed.
  • Ceph OSD not running: Check logs at /var/log/ceph/ceph-osd.log.
  • High latency: Ensure stable network connectivity between VPS nodes and use NVMe SSD storage.

Refer to the official documentation for more details:

To learn more about self-hosting applications on VPS, refer to the guide on Building a Self-Hosted Mini Vercel on ARM VPS with Coolify and Nixpacks.

DPTCloud Tier III Infrastructure

Recommended NVMe Cloud VPS Plans to Deploy This Setup

Enterprise RAID 10 NVMe SSDs, CMC & FPT District 9 uplinks with Anycast Anti-DDoS and 99.99% SLA.

View 22 VPS Plans
ECO · vn-02

VPS-2E

94.600 ₫ / mo

10% VAT included

  • CPU: 2 vCPU
  • RAM: 2 GB
  • SSD: 30 GB NVMe
ECO · vn-02Recommended

VPS-3E

170.500 ₫ / mo

10% VAT included

  • CPU: 2 vCPU
  • RAM: 4 GB
  • SSD: 50 GB NVMe
BASIC · vn-01Recommended

VPS-05

457.600 ₫ / mo

10% VAT included

  • CPU: 4 vCPU
  • RAM: 8 GB
  • SSD: 50 GB NVMe

Related Articles

View all
ZaloZalo SupportDiscord0384 753 478