README
¶
Blobcache
Blobcache reimagines application state as Volumes holding hash-linked data structures. Volumes can be persisted locally, encrypted, and accessed over the network. Hash-linked data structures are efficient to sync and transfer. Corruption is always detected, and the root is a cryptographic commitment to the whole data structure.
Blobcache is a universal backend for E2EE applications.
Quick Docs
Getting Started
For non-user machines (cloud, homelab) it is recommended to run the docker image. For user machines, it is recommended to run blobcache as a systemd service when you log in.
Installation
There is an install script for systemd based systems.
You can run that with just install-systemd.
The install script does 2 things:
- Installs the blobcache executable to
/usr/bin/blobcache - Copies blobcache.service into
$HOME/.config/systemd/user/
Then you can manage the service using systemctl --user as you would normally.
The service is not enabled by default (you can do that with systemctl --user enable blobcache), so this install method is also appropriate for getting the binary into /usr/bin without launching a background process.
Setting BLOBCACHE_API
On user machines, you will probably want to set the BLOBCACHE_API environment variable.
This would set the variable to the unix socket used by default in the systemd service described above.
export BLOBCACHE_API="unix:///run/user/$(id -u)/blobcache.sock"
The environment variable is used by the blobcache command to talk to the blobcache daemon.
You can test that the CLI can connect to the daemon with
$ blobcache endpoint
JcJvfY9tFsfkHSwoMT8IEoSq1ZfxVYBAwpBRvJ0uUJA:[::]:6025
Docker
There is a docker image on the GitHub Container Registry, it can also be built with just docker-build.
Right now it only builds for linux-amd64.
After that you can test run with
docker run -it --rm ghcr.io/blobcache/blobcache:v0.0.1
Images are tagged with the git hash e.g. git-a0b1c3d and the version e.g. v0.0.1
The /state directory is where blobcache stores all of its state.
This is where you should mount a volume to persist data on the host.
You should also expose the peer port, so other instances can connect.
docker run \
-v /host/path/to/state:/state \
-p 6025:6025/udp \
ghcr.io/blobcache/blobcache:v0.0.1
Running a Node in Memory
This is a good option if you just want to play around with the API, and don't want to persist any data, or connect to peers.
$ blobcache daemon-ephemeral \
--serve-api unix://./blobcache.sock \
--net 0.0.0.0:6025
Running the daemon
The following command runs a daemon with state in the specified directory.
$ blobcache daemon \
--state $HOME/.local/blobcache \
--serve-api unix:///run/blobcache/blobcache.sock \
--net 0.0.0.0:6025
Once the daemon is running, you should be able to connect to it and start building your application on top of content-addressed storage.
License
The Blobcache implementation is licensed under GPLv3.
All of the clients are licensed under MPL 2.0
What this means is: if you improve Blobcache you have to make the improvements available, but if you are just using a client you can do whatever you want with it.
Directories
¶
| Path | Synopsis |
|---|---|
|
client
|
|
|
go
Package bcclient provides a client for the Blobcache API.
|
Package bcclient provides a client for the Blobcache API. |
|
cmd
|
|
|
blobcache
command
|
|
|
git-remote-bc
command
|
|
|
src
|
|
|
bchttp
Package bchttp implements a Blobcache service over HTTP.
|
Package bchttp implements a Blobcache service over HTTP. |
|
bclocal
Package bclocal implements a local Blobcache service.
|
Package bclocal implements a local Blobcache service. |
|
bclocal/internal/blobman/shard
Package shard implements a single Shard in the database A Shard is a single directory on disk, called the Shard's "root".
|
Package shard implements a single Shard in the database A Shard is a single directory on disk, called the Shard's "root". |
|
bclocal/internal/dbtab
package dbtab contains constant IDs for each Table
|
package dbtab contains constant IDs for each Table |
|
bclocal/internal/pdb
package pdb has utilities for working with the Pebble database.
|
package pdb has utilities for working with the Pebble database. |
|
bcremote
Package bcremote implements a blobcache.Service over Blobcache Network Protocol (BCNP).
|
Package bcremote implements a blobcache.Service over Blobcache Network Protocol (BCNP). |
|
blobcache/blobcachetests
package blobcachetests provides a test suite for blobcache.Service.
|
package blobcachetests provides a test suite for blobcache.Service. |
|
internal/bcnet
bcnet implements the Blobcache Protocol (BCP).
|
bcnet implements the Blobcache Protocol (BCP). |
|
internal/bcp
Package bcp implements the Blobcache Protocol (BCP).
|
Package bcp implements the Blobcache Protocol (BCP). |
|
internal/sbe
package sbe implements simple binary encoding formats for serializing and deserializing data.
|
package sbe implements simple binary encoding formats for serializing and deserializing data. |
|
internal/schemareg
Package schemareg provides a registry for Schemas.
|
Package schemareg provides a registry for Schemas. |
|
internal/volumes/gitvol
Package gitvol implements a Volume in terms of Git
|
Package gitvol implements a Volume in terms of Git |
|
schema
Package schema implements Schemas for blobcache volumes.
|
Package schema implements Schemas for blobcache volumes. |
|
schema/bcfuse
Package bcfuse implements a filesystem interface for blobcache.
|
Package bcfuse implements a filesystem interface for blobcache. |
|
schema/bcgit/gitrh
package gitrh provides a Server implementation of the Git remote helper IPC protocol
|
package gitrh provides a Server implementation of the Git remote helper IPC protocol |
|
schema/hydra
Package hydra implements a Mergeable Schema
|
Package hydra implements a Mergeable Schema |
|
schema/jsonns
Package jsonns provides a simple namespace implementation.
|
Package jsonns provides a simple namespace implementation. |
|
schema/statetrace
package ledger implements an append-only ledger where state transitions can be verified
|
package ledger implements an append-only ledger where state transitions can be verified |