zapfs

ZapFS

A distributed object storage system with an S3-compatible API, written in Go.

Architecture

flowchart TB
    subgraph Clients["S3 Clients"]
        C1["AWS SDK"]
        C2["s3cmd"]
        C3["mc"]
    end

    subgraph MetadataLayer["Metadata Service (:8082 HTTP, :8083 gRPC)"]
        direction TB
        API["S3 API Gateway<br/>• AWS Signature (SigV4/V2/Presigned/POST)<br/>• Bucket & object operations<br/>• Multipart, versioning, ACLs"]

        subgraph InternalClients["Internal Clients"]
            MC["Manager Client<br/>• IAM sync<br/>• Bucket registration<br/>• Placement"]
            FC["File Client<br/>• Chunk read/write<br/>• Streaming uploads<br/>• Replication"]
        end

        API --> MC & FC
    end

    subgraph ControlPlane["Manager Cluster (:8050 gRPC, :8051 Raft)"]
        MG1["Manager 1<br/>(Leader)"]
        MG2["Manager 2"]
        MG3["Manager 3"]

        MG1 <-->|Raft| MG2 & MG3
    end

    subgraph DataPlane["File Servers (:8081 gRPC)"]
        F1["File Server 1"]
        F2["File Server 2"]
        F3["File Server N"]
    end

    subgraph Storage["Storage Layer"]
        FS["Local/S3 Storage<br/>• Content-hash dedup<br/>• Erasure coding (RS)<br/>• RefCount GC<br/>• Replication (2+ copies)"]
    end

    subgraph Database["MySQL / Vitess"]
        DB1[("buckets, objects<br/>multipart_uploads<br/>ACLs, policies")]
        DB2[("usage_events<br/>usage_daily")]
        DB3[("tasks<br/>(task queue)")]
    end

    Clients -->|HTTP| API
    MC -->|gRPC| ControlPlane
    FC -->|gRPC| DataPlane
    DataPlane -->|gRPC| ControlPlane
    F1 & F2 & F3 --> FS
    MetadataLayer -->|SQL| Database

Components

Quick Start

Docker Compose (Development)

cd docker
docker compose up -d

# Test with AWS CLI
aws --endpoint-url=http://localhost:8082 s3 mb s3://test-bucket
aws --endpoint-url=http://localhost:8082 s3 cp README.md s3://test-bucket/
aws --endpoint-url=http://localhost:8082 s3 ls s3://test-bucket/

Kubernetes (Helm)

cd k8s/zapfs
helm dependency update
helm install zapfs . -n zapfs --create-namespace

Configuration

IAM Credentials

Default development credentials (change in production):

Configure via cmd/config/iam.toml or the Manager Admin API:

# Create a new user
curl -X POST http://localhost:8060/v1/iam/users \
  -H "Content-Type: application/json" \
  -d '{"username": "myuser"}'

# Create access key
curl -X POST http://localhost:8060/v1/iam/users/myuser/access-keys

Environment Variables

Development

Building

# Build binary
go build -o zapfs .

# Run tests
go test ./...

# Generate mocks
make mocks

Running Locally

# Start manager (bootstrap node)
./zapfs manager --bootstrap --node_id=manager-1 --raft_dir=/tmp/raft

# Start metadata service
./zapfs metadata --manager_addr=localhost:8050 --db_driver=sqlite

# Start file server
./zapfs file --node_id=file-1

Performance

ZapFS is designed for high-throughput object storage. Benchmarks use MinIO WARP.

Quick Benchmark

# Start benchmark environment
docker compose -f docker/docker-compose.benchmark.yml up -d

# Run quick benchmark (1 min)
./scripts/benchmark.sh --profile quick

# Run standard benchmark (5 min) with output
./scripts/benchmark.sh --profile standard --output ./results

Benchmark Profiles

See docs/benchmarks/ for detailed methodology, environment setup, and recorded results.

Acknowledgments

ZapFS draws inspiration from several excellent open-source projects:

Some code patterns and S3 compatibility implementations were adapted from SeaweedFS under Apache 2.0 license.

License

ZapFS is open-core software with dual licensing:

Community Edition (Default)

The default build includes all core S3 functionality:

go build -o zapfs .

Enterprise Edition

Enterprise features require the enterprise build tag and a valid license key.

Get your license at zapfs.io — Start with a 14-day free trial.

go build -tags enterprise -o zapfs-enterprise .

To activate your license, place the license key file at /etc/zapfs/license.key or set the ZAPFS_LICENSE_KEY environment variable:

# Option 1: File-based
sudo mkdir -p /etc/zapfs
sudo cp ~/Downloads/zapfs-license.key /etc/zapfs/license.key

# Option 2: Environment variable
export ZAPFS_LICENSE_KEY=$(cat ~/Downloads/zapfs-license.key)

Enterprise Features

See LICENSE and LICENSE.enterprise for details.

Enterprise Support

For enterprise licensing, support, and custom deployments:

• Portal: zapfs.io — Manage licenses and subscriptions
• Documentation: zapfs.io/docs — Setup guides and API reference

Star History