Cost-Effective Persistent Storage in Docker Swarm: A Guide to Implementing JuiceFS with Backblaze B2

Introduction to High-Availability Storage in Container Orchestration

In modern cloud-native architectures, container orchestration platforms like Docker Swarm offer a lightweight, highly efficient mechanism for managing microservices. However, one of the most persistent operational hurdles when running stateful workloads across a multi-node cluster is handling Persistent Volumes (PVs). Unlike Kubernetes, which features a robust, deeply integrated ecosystem for CSI drivers, Docker Swarm requires a deliberate architectural strategy to ensure that data remains consistent, available, and performant, regardless of which node executes a specific container container replica.

Traditional shared file systems, such as Network File System (NFS), introduce single points of failure (SPOF) and scale poorly across geographic zones or public cloud boundaries. On the other hand, distributed storage blocks like Ceph or GlusterFS demand intensive compute overhead, complex maintenance, and high infrastructure costs. This blog post explores a highly optimized alternative: combining JuiceFS, a high-performance open-source POSIX file system, with Backblaze B2, one of the industry's most cost-effective cloud object storage solutions, to establish a permanent, resilient, and budget-friendly shared volume system for Docker Swarm clusters.

The Architectural Dilemma: Stateful Containers in Docker Swarm

By design, Docker Swarm excels at scheduling stateless containers across a pool of worker nodes. When a container requires data persistence (e.g., databases, content management systems, user upload directories), the storage must be mounted dynamically to whatever node the Swarm scheduler selects. If Node A fails and the container fails over to Node B, the underlying data must be immediately accessible on Node B without data corruption or latency spikes.

The Challenge: Standard local Docker volumes are tied to the physical host. If a service replicates or migrates to another server, local data does not follow it. A truly decentralized, network-accessible storage abstraction layer is mandatory.

To overcome this, engineers need a storage engine that fulfills three distinct pillars:

• POSIX Compliance: Applications should read and write data using standard file system calls without modifying code.
• Strong Consistency: Changes made by a container on one node must be instantly visible to containers on other nodes.
• Cost Efficiency: Storage costs should scale linearly with actual usage, omitting the financial burden of pre-provisioning large, idle block storage volumes.

Why Choose JuiceFS and Backblaze B2?

The combination of JuiceFS and Backblaze B2 provides an elegant synergy that addresses performance and budgetary constraints simultaneously.

1. Understanding the JuiceFS Architecture

JuiceFS revolutionizes cloud storage by decoupling data storage from metadata management. It splits every file into chunks and stores them in object storage (like Backblaze B2), while storing the corresponding file metadata (file trees, permissions, modification times) in a high-speed database engine such as Redis, PostgreSQL, or MySQL. This architecture allows JuiceFS to deliver low-latency POSIX operations because directory listings and file lookups happen via the metadata engine instantly, bypassing the high-latency HTTP requests typically required by raw object storage.

2. The Cost Advantage of Backblaze B2

While Amazon S3 and Google Cloud Storage are standard choices, their egress fees and per-gigabyte pricing models can quickly balloon a startup's or mid-sized enterprise's operational expenditures. Backblaze B2 offers S3-compatible API endpoints at a fraction of the cost—frequently saving up to 4x to 5x on raw storage fees with minimal or zero data egress charges when paired with specific CDN partners. By routing JuiceFS data chunks to Backblaze B2, teams can scale to terabytes of persistent data without breaking their infrastructure budget.

Step-by-Step Implementation Guide

To implement this architecture, we will execute a deployment strategy consisting of setting up the metadata engine, provisioning the Backblaze B2 bucket, formatting the JuiceFS file system, and integrating it directly into Docker Swarm as a shared plugin or system mount.

Prerequisites

Before proceeding, ensure you have gathered the following components:

1. A functional Docker Swarm cluster with at least one Manager and two Worker nodes.
2. A Backblaze B2 account with an application key ID and application key possessing read/write permissions.
3. A centralized metadata database. For production, a managed Redis or PostgreSQL instance is recommended. For testing, a highly available SQLite or self-hosted Redis instance works.

Step 1: Provisioning the Backblaze B2 Bucket

Log in to your Backblaze web console and create a new private bucket dedicated to your Swarm cluster storage. Ensure you note down your Bucket Name, S3 Endpoint URL (e.g., s3.us-west-004.backblazeb2.com), Key ID, and Application Key. These credentials act as the primary target for JuiceFS data blocks.

Step 2: Formatting the JuiceFS File System

JuiceFS requires formatting prior to mounting. Install the JuiceFS CLI on an administrative machine or run a temporary Docker container to execute the format command. In this example, we assume you are using a Redis instance as your metadata engine:

juicefs format \
    --storage s3 \
    --bucket [https://your-backblaze-bucket-name.s3.us-west-004.backblazeb2.com](https://your-backblaze-bucket-name.s3.us-west-004.backblazeb2.com) \
    --access-key YOUR_B2_KEY_ID \
    --secret-key YOUR_B2_APPLICATION_KEY \
    redis://:your-redis-password@redis-server-ip:6379/1 \
    swarm-persistent-storage

This command initialises the metadata schema inside Redis and links it to the designated Backblaze B2 object bucket under the file system name swarm-persistent-storage.

Step 3: Deploying the JuiceFS Volume Plugin on Docker Swarm Nodes

To enable Docker containers to consume this volume transparently, we must install the official JuiceFS Docker volume plugin on every node within the Docker Swarm cluster. Run the following command on each node:

docker plugin install juicefs/juicefs-volume-plugin:latest \
    --grant-all-permissions

Once installed, configure the plugin with your database and storage credentials so that it can autonomously mount the file system when a service spins up. Alternatively, you can mount JuiceFS at the host operating system level (via /etc/fstab) to /mnt/juicefs on all nodes, allowing Docker to use a standard bind mount. The host mount approach often simplifies debugging and plugin lifecycle management.

Step 4: Defining the Persistent Volume in a Docker Compose / Stack File

With the infrastructure prepared, you can now deploy services via Docker Stacks using standard YAML definitions. Below is an example of a replicated WordPress deployment consuming our cost-effective, persistent JuiceFS volume:

version: '3.8'

services:
  wordpress:
    image: wordpress:latest
    ports:
      - "8080:80"
    deploy:
      replicas: 3
      placement:
        max_replicas_per_node: 1
    volumes:
      - shared-wp-data:/var/www/html

volumes:
  shared-wp-data:
    driver: local
    driver_opts:
      type: none
      o: bind
      device: /mnt/juicefs/wordpress_data

In this architecture, even if the three WordPress replicas are distributed across separate physical servers, they write concurrently to /mnt/juicefs/wordpress_data. JuiceFS securely syncs modifications to Backblaze B2 while caching metadata changes instantly via Redis, guaranteeing that all application nodes share an identical, real-time view of the files.

Production Best Practices: Performance Optimization and Security

To run JuiceFS and Backblaze B2 reliably under heavy production traffic, apply these critical optimizations:

• Enable Local Caching: Configure JuiceFS with a dedicated local SSD cache directory (e.g., --cache-dir=/tmp/juicefs-cache --cache-size=102400). This drastically accelerates read operations by storing frequently accessed data chunks locally on the host's high-speed drive instead of downloading them repeatedly from Backblaze B2.
• Metadata Backup: Your data is only as healthy as your metadata. Configure automated, hourly snapshots of your Redis or PostgreSQL instance. If you lose your metadata database, rebuilding the file structure from raw data chunks in object storage is extraordinarily complex.
• Network Proximity: Ensure your Docker Swarm nodes are provisioned in data centers geographically close to your assigned Backblaze B2 region to minimize latency overhead during write operations.

Conclusion

Building a resilient, production-ready storage architecture for Docker Swarm does not require a massive enterprise budget or overly convoluted SAN solutions. By abstracting the file system layer with JuiceFS and backing it with the highly economical, durable architecture of Backblaze B2, organizations can achieve high availability, strict POSIX compliance, and infinite horizontal scaling. Implementing this workflow ensures your decentralized container workloads remain highly operational, stateful, and secure at a fraction of conventional cloud computing costs.