README
¶
mlfs — POSIX filesystem
A FUSE filesystem with PostgreSQL metadata, content-addressed file slices, durable write-back cache, copy-on-write, ownership leases, invalidation, POSIX ACLs, and local or remote storage paths.
Go module: github.com/scitrera/memorylayer-storage/mlfs.
Initial version: 0.7.1, licensed under AGPL-3.0-only.
The mlfs daemon mounts the filesystem. mlfs-admin provides administrative operations and mlfs-bench provides workload/metrics tooling. Run each command with -help. Linux and FUSE are required for mount tests.
From the repository root:
cd mlfs
go test ./...
go build ./...
The root go.work resolves sibling modules for development. Published module
versions use mlfs/vX.Y.Z tags. See development and releases
and operations.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package bridge is the EXPORTED mlfs↔VFS metadata-only bridge (handoff Item C): it moves files between mlfs and blobgw by SYNTHESISING metadata over the shared casstore chunks — no file bytes are ever read or written in either direction.
|
Package bridge is the EXPORTED mlfs↔VFS metadata-only bridge (handoff Item C): it moves files between mlfs and blobgw by SYNTHESISING metadata over the shared casstore chunks — no file bytes are ever read or written in either direction. |
|
cmd
|
|
|
mlfs
command
Command mlfs is the L2 POSIX filesystem daemon: it wires the PostgreSQL metadata engine, a casstore-backed chunk store with a durable write-back disk cache, the data path, and the FUSE bridge into a real mount.
|
Command mlfs is the L2 POSIX filesystem daemon: it wires the PostgreSQL metadata engine, a casstore-backed chunk store with a durable write-back disk cache, the data path, and the FUSE bridge into a real mount. |
|
mlfs-admin
command
Command mlfs-admin is the L2 operations CLI.
|
Command mlfs-admin is the L2 operations CLI. |
|
mlfs-bench
command
Command mlfs-bench is a standalone probe for an mlfs mount, meant to be deployed onto any pod that mounts mlfs storage.
|
Command mlfs-bench is a standalone probe for an mlfs mount, meant to be deployed onto any pod that mounts mlfs storage. |
|
internal
|
|
|
acl
Package acl implements just enough POSIX.1e ACL handling for default-ACL INHERITANCE on file creation.
|
Package acl implements just enough POSIX.1e ACL handling for default-ACL INHERITANCE on file creation. |
|
bench
Package bench is a POSIX filesystem benchmark for an mlfs mount.
|
Package bench is a POSIX filesystem benchmark for an mlfs mount. |
|
cache
Package cache is mlfs's local disk cache and durable write-back staging in front of the casstore-backed chunk store.
|
Package cache is mlfs's local disk cache and durable write-back staging in front of the casstore-backed chunk store. |
|
chunkstore
Package chunkstore stores file slices through casstore.
|
Package chunkstore stores file slices through casstore. |
|
coord
Package coord is mlfs's multi-node coordination plane (L2.8).
|
Package coord is mlfs's multi-node coordination plane (L2.8). |
|
e2e
Package e2e holds the hermetic cross-module integration test that proves the converged storage stack works end to end: the REAL blobgw control-plane server (controlplane.Server over embedded NATS, with a tenantbind credential provider and an in-memory dedup index) wired to the REAL mlfs node data path (chunkstore.NewRemote → remoteindex.NatsRequester / s3http / casstore), with an httptest server standing in for S3.
|
Package e2e holds the hermetic cross-module integration test that proves the converged storage stack works end to end: the REAL blobgw control-plane server (controlplane.Server over embedded NATS, with a tenantbind credential provider and an in-memory dedup index) wired to the REAL mlfs node data path (chunkstore.NewRemote → remoteindex.NatsRequester / s3http / casstore), with an httptest server standing in for S3. |
|
fileio
Package fileio is mlfs's file data path: it turns POSIX read/write at byte offsets into slice writes (chunk store) + slice records (meta engine), and resolves overlapping slices on read so newer writes win.
|
Package fileio is mlfs's file data path: it turns POSIX read/write at byte offsets into slice writes (chunk store) + slice records (meta engine), and resolves overlapping slices on read so newer writes win. |
|
fsstat
Package fsstat holds small, dependency-free helpers for reporting filesystem usage: human-readable byte sizes, on-disk directory accounting, and statfs capacity.
|
Package fsstat holds small, dependency-free helpers for reporting filesystem usage: human-readable byte sizes, on-disk directory accounting, and statfs capacity. |
|
fuse
Package fusebridge maps FUSE operations onto the metadata engine and file I/O layer.
|
Package fusebridge maps FUSE operations onto the metadata engine and file I/O layer. |
|
gc
Package gc is mlfs's refcount-free, mark-and-sweep garbage collector.
|
Package gc is mlfs's refcount-free, mark-and-sweep garbage collector. |
|
meta
Package meta implements the PostgreSQL filesystem metadata engine.
|
Package meta implements the PostgreSQL filesystem metadata engine. |
|
metrics
Package metrics is the mlfs daemon's in-process instrument registry.
|
Package metrics is the mlfs daemon's in-process instrument registry. |
|
mountlock
Package mountlock provides a single-node "is a daemon live?" guard built on a PID lock file in the data directory.
|
Package mountlock provides a single-node "is a daemon live?" guard built on a PID lock file in the data directory. |
|
remoteblob
Package remoteblob implements the casstore blobstore.Storage interface for an mlfs node's direct-S3 data plane (ADR-001 §2.2, §5).
|
Package remoteblob implements the casstore blobstore.Storage interface for an mlfs node's direct-S3 data plane (ADR-001 §2.2, §5). |
|
remoteindex
Package remoteindex implements snapshot.DedupStore over the blobgw control plane using NATS request-reply and the ctlproto wire contract (ADR-001 §2.2, §2.5).
|
Package remoteindex implements snapshot.DedupStore over the blobgw control plane using NATS request-reply and the ctlproto wire contract (ADR-001 §2.2, §2.5). |
|
remotepresign
Package remotepresign implements a node-side client that requests presigned S3 URLs from blobgw's control plane over NATS request-reply using the ctlproto wire contract (ADR-001 §2.2, §2.5, §2.9).
|
Package remotepresign implements a node-side client that requests presigned S3 URLs from blobgw's control plane over NATS request-reply using the ctlproto wire contract (ADR-001 §2.2, §2.5, §2.9). |
|
s3http
Package s3http is the node-side HTTP transfer primitive for the direct-S3 data plane described in ADR-001 §2.2 and §5.
|
Package s3http is the node-side HTTP transfer primitive for the direct-S3 data plane described in ADR-001 §2.2 and §5. |
|
testpg
Package testpg provides Postgres-backed test isolation for mlfs.
|
Package testpg provides Postgres-backed test isolation for mlfs. |
|
wal
Package wal is mlfs's local data write-ahead log.
|
Package wal is mlfs's local data write-ahead log. |
Click to show internal directories.
Click to hide internal directories.