README
¶
Blobcache
Blobcache reimagines application state as Volumes holding hash-linked data structures. Volumes can be persisted locally, encrypted, and accessed over the network. Hash-linked data structures are efficient to sync and transfer. Corruption is always detected, and the root is a cryptographic commitment to the whole data structure.
Blobcache is a universal backend for E2EE applications.
Quick Docs
Getting Started
For non-user machines (cloud, homelab) it is recommended to run the docker image. For user machines, it is recommended to run blobcache as a systemd service when you log in.
Installation
There is an install script for systemd-based linux, and macOS.
curl blobcache.io/install.sh | sh
The install script does 2 things:
- Installs the blobcache executable to
/usr/bin/blobcache - Copies blobcache.service into
$HOME/.config/systemd/user/
Then you can manage the service using systemctl --user as you would normally.
The service is not enabled by default (you can do that with systemctl --user enable blobcache), so this install method is also appropriate for getting the binary into /usr/bin without launching a background process.
Setting BLOBCACHE_API
On user machines, you will probably want to set the BLOBCACHE_API environment variable.
This would set the variable to the unix socket used by default in the systemd service described above.
export BLOBCACHE_API="unix:///run/user/$(id -u)/blobcache.sock"
The environment variable is used by the blobcache command to talk to the blobcache daemon.
You can test that the CLI can connect to the daemon with
$ blobcache endpoint
JcJvfY9tFsfkHSwoMT8IEoSq1ZfxVYBAwpBRvJ0uUJA:[::]:6025
Docker
There is a docker image on the GitHub Container Registry, it can also be built with just build-images.
Right now it only builds for linux-amd64.
After that you can test run with
docker run -it --rm ghcr.io/blobcache/blobcache:latest
Images are tagged with the git hash e.g. git-a0b1c3d and the version e.g. v1.2.3
The /state directory is where blobcache stores all of its state.
This is where you should mount a volume to persist data on the host.
You should also expose the peer port, so other instances can connect.
docker run \
-v /host/path/to/state:/state \
-p 6025:6025/udp \
ghcr.io/blobcache/blobcache:latest
Running a Node in Memory
This is a good option if you just want to play around with the API, and don't want to persist any data, or connect to peers.
$ blobcache daemon-ephemeral \
--serve-ipc ./blobcache.sock \
--net 0.0.0.0:6025
Running the daemon
The following command runs a daemon with state in the specified directory.
$ blobcache daemon \
--state $HOME/.local/blobcache \
--serve-ipc /run/blobcache/blobcache.sock \
--net 0.0.0.0:6025
Once the daemon is running, you should be able to connect to it and start building your application on top of content-addressed storage.
License
The Blobcache implementation is licensed under GPLv3. All of the clients are licensed under MPL 2.0 As a genreal rule: everything north of the UNIX or HTTP API is MPL and everything south is GPL.
What this means is: if you improve Blobcache you have to make the improvements available, but if you are just using a client you can do whatever you want with it.
If you need additional clarity with regard to licensing, please reach out. We can add headers to the invidual files if necessary.
Directories
¶
| Path | Synopsis |
|---|---|
|
client
|
|
|
go
Package bcclient provides a client for the Blobcache API.
|
Package bcclient provides a client for the Blobcache API. |
|
cmd
|
|
|
blobcache
command
|
|
|
git-remote-bc
command
|
|
|
examples
|
|
|
goexample1
command
|
|
|
radixkv
command
This is a program which will connect to a Blobcache according to BLOBCACHE_API.
|
This is a program which will connect to a Blobcache according to BLOBCACHE_API. |
|
src
|
|
|
bchttp
Package bchttp implements a Blobcache service over HTTP.
|
Package bchttp implements a Blobcache service over HTTP. |
|
bcipc
Package bcipc implements the Blobcache Protocol over UNIX sockets
|
Package bcipc implements the Blobcache Protocol over UNIX sockets |
|
bclocal
Package bclocal implements a local Blobcache service.
|
Package bclocal implements a local Blobcache service. |
|
bclocal/internal/blobman/shard
Package shard implements a single Shard in the database A Shard is a single directory on disk, called the Shard's "root".
|
Package shard implements a single Shard in the database A Shard is a single directory on disk, called the Shard's "root". |
|
bclocal/internal/dbtab
package dbtab contains constant IDs for each Table
|
package dbtab contains constant IDs for each Table |
|
bclocal/internal/pdb
package pdb has utilities for working with the Pebble database.
|
package pdb has utilities for working with the Pebble database. |
|
bcremote
Package bcremote implements a blobcache.Service over Blobcache Network Protocol (BCNP).
|
Package bcremote implements a blobcache.Service over Blobcache Network Protocol (BCNP). |
|
blobcache/blobcachetests
package blobcachetests provides a test suite for blobcache.Service.
|
package blobcachetests provides a test suite for blobcache.Service. |
|
internal/backend/gitvol
Package gitvol implements a Volume in terms of Git
|
Package gitvol implements a Volume in terms of Git |
|
internal/bcnet
bcnet implements the Blobcache Protocol (BCP).
|
bcnet implements the Blobcache Protocol (BCP). |
|
internal/bcp
Package bcp implements the Blobcache Protocol (BCP).
|
Package bcp implements the Blobcache Protocol (BCP). |
|
internal/bcsys
TODO: eventually remove this file, all errors should be in src/blobcache
|
TODO: eventually remove this file, all errors should be in src/blobcache |
|
internal/schemareg
Package schemareg provides a registry for Schemas.
|
Package schemareg provides a registry for Schemas. |
|
schema
Package schema implements Schemas for blobcache volumes.
|
Package schema implements Schemas for blobcache volumes. |
|
schema/bcfuse
Package bcfuse implements a filesystem interface for blobcache.
|
Package bcfuse implements a filesystem interface for blobcache. |
|
schema/bcgit/gitrh
package gitrh provides a Server implementation of the Git remote helper IPC protocol
|
package gitrh provides a Server implementation of the Git remote helper IPC protocol |
|
schema/hydra
Package hydra implements a Mergeable Schema
|
Package hydra implements a Mergeable Schema |
|
schema/jsonns
Package jsonns provides a simple namespace implementation.
|
Package jsonns provides a simple namespace implementation. |
|
schema/radixkv
Package radixkv implements a radix tree key-value store in a Blobcache volume.
|
Package radixkv implements a radix tree key-value store in a Blobcache volume. |
|
schema/statetrace
package ledger implements an append-only ledger where state transitions can be verified
|
package ledger implements an append-only ledger where state transitions can be verified |