Help us shape the future of Go! Take the 2026 Go Developer Survey

meta

package
v0.7.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 8, 2026 License: AGPL-3.0 Imports: 12 Imported by: 0

Documentation

Overview

Package meta implements the PostgreSQL filesystem metadata engine. Directory entries, file slices, locks, ownership leases and change records are updated transactionally. Each metadata write appends a change record in the same transaction, preserving the ordering observed by other nodes.

Index

Constants

View Source 
const (
	FlockShared    = 'R'
	FlockExclusive = 'W'
	FlockUnlock    = 'U'
)

flock ltype values stored in flock.ltype (and passed by the bridge).

View Source 
const (
	RenameNoReplace = 0x1 // RENAME_NOREPLACE: fail with EEXIST if dst exists
	RenameExchange  = 0x2 // RENAME_EXCHANGE: atomically swap src and dst
	RenameWhiteout  = 0x4 // RENAME_WHITEOUT: overlayfs; not supported (ENOSYS)
)

Rename flags (subset of renameat2). Values match the kernel's RENAME_* (uapi/linux/fs.h) so the FUSE bridge can forward in.Flags verbatim.

View Source 
const (
	MayExec  uint8 = 1
	MayWrite uint8 = 2
	MayRead  uint8 = 4
)

Access permission bits (match the low POSIX mode bits).

View Source 
const (
	ChangeMknod   = opMknod
	ChangeMkdir   = opMkdir
	ChangeUnlink  = opUnlink
	ChangeRmdir   = opRmdir
	ChangeRename  = opRename
	ChangeSetattr = opSetattr
	ChangeWrite   = opWrite
	ChangeSymlink = opSymlink
	ChangeLink    = opLink
)

Exported mirror of the changelog op codes: the fs_changelog wire contract, consumed by the change feed (L2.8.5) to decide what kernel cache to invalidate. Kept as aliases so internal call sites stay unchanged.

View Source 
const (
	TypeFile      uint8 = 1
	TypeDirectory uint8 = 2
	TypeSymlink   uint8 = 3
	TypeFIFO      uint8 = 4
	TypeBlockDev  uint8 = 5
	TypeCharDev   uint8 = 6
	TypeSocket    uint8 = 7
)

File types stored in filesystem metadata.

View Source 
const (
	SetMode  uint16 = 1 << 0
	SetUID   uint16 = 1 << 1
	SetGID   uint16 = 1 << 2
	SetSize  uint16 = 1 << 3
	SetAtime uint16 = 1 << 4
	SetMtime uint16 = 1 << 5
)

SetAttr mask bits select which Attr fields SetAttr applies.

View Source 
const (
	XattrCreate  uint32 = 0x1 // pure create: fail with EEXIST if the name exists
	XattrReplace uint32 = 0x2 // pure replace: fail with ENODATA if it does not
)

SetXAttr flags (match linux <sys/xattr.h>).

View Source 
const ChunkSize = 1 << 26 // 64 MiB

ChunkSize is the logical chunk granularity: a file is a sequence of chunks, each addressed by index, each holding overlaid slices.

View Source 
const Schema = `` /* 3747-byte string literal not displayed */

Schema is the mlfs metadata DDL, idempotent.

  • node / edge / symlink: the directory tree.
  • slice_ref: the per-chunk slice list, with one row per write. Copy-on-write copies slice rows; garbage collection is mark-and-sweep over this table. No chunk_refcount table (by design).
  • flock / plock: POSIX locks (table created in L2.1; lock ops land later).
  • xattr: extended attributes ((inode,name)→value); also stores POSIX ACL blobs (system.posix_acl_*). The kernel enforces these (mount EnableAcl / CAP_POSIX_ACL, L2.8.7), reading/writing them through this generic store.
  • ownership: per-scope write-affinity lease with a `generation` fence.
  • fs_changelog: durable, ordered (BIGSERIAL lsn) change log; every write appends here in the same transaction.
  • mlfs_counter: monotonic counters (inode allocation).

Variables

This section is empty.

Functions

This section is empty.

Types

type Attr 

type Attr struct {
	Typ       uint8
	Mode      uint16
	Uid       uint32
	Gid       uint32
	Atime     int64 // seconds
	Mtime     int64
	Ctime     int64
	Atimensec uint32
	Mtimensec uint32
	Ctimensec uint32
	Nlink     uint32
	Length    uint64
	Rdev      uint32
	Parent    Ino
	// StoreClass is the file's compression class (0 = default/policy-decides,
	// 1 = uncompressed for model/tensor data). Set at create by file type, read on
	// every write to stamp the slice so the codec follows the file. Mirrors
	// chunkstore.StoreClass numerically; meta stays decoupled from chunkstore (the
	// bridge maps between them).
	StoreClass uint8
	// ContentHash is the whole-file sha256 as lowercase hex, no algorithm prefix
	// (empty until the file is first flushed). It matches blobgw's
	// ObjectInfo.ContentHash encoding byte-for-byte (gateway hexSum), so the
	// mlfs↔VFS bridge can register a blob_ref whose content_hash equals this value
	// without re-hashing. Written by SetContentHash on flush/close; read here.
	ContentHash string
	Full        bool
}

Attr is a node's POSIX attributes.

type ChangeEntry 

type ChangeEntry struct {
	LSN     int64
	Ino     Ino
	Op      uint8
	Payload []byte
	TS      int64
}

ChangeEntry is one row of the fs_changelog, returned by ScanChangelog.

type Context 

type Context struct {
	Uid uint32
	Gid uint32
	Pid uint32
}

Context carries the caller identity for ownership + (future) permission checks. A zero Context (root) is fine for internal callers.

func Background 

func Background() *Context

Background returns a root-owned context.

type Engine 

type Engine struct {
	// contains filtered or unexported fields
}

Engine is the mlfs metadata system-of-record over a PostgreSQL database (via database/sql; the caller supplies the *sql.DB and driver). It is safe for concurrent use.

func Open 

func Open(db *sql.DB, region uint8, opts ...Option) *Engine

Open wraps a database/sql handle. region is this node's 8-bit region id; a random 8-bit session id is mixed in so concurrently-running nodes draw distinct inode prefixes. Call Migrate before use. Optional Options (e.g. WithLogger) tune the engine; the logger defaults to slog.Default().

func (*Engine) Access 

func (e *Engine) Access(ctx context.Context, inode Ino, mask uint8, c *Context) syscall.Errno

Access reports whether caller c may access inode with the given r/w/x mask.

func (*Engine) AcquireOrResume 

func (e *Engine) AcquireOrResume(ctx context.Context, scopeKey, node string, ttlMs int64) (lease Lease, held bool, err error)

AcquireOrResume takes (or renews) the lease on scopeKey for node, for ttlMs. It succeeds if the scope is unowned, expired, or already owned by node; taking it from an expired owner bumps the generation. It returns the lease and whether node now holds it. A live lease held by another node is returned with held=false (no change).

func (*Engine) ChangelogHorizon 

func (e *Engine) ChangelogHorizon(ctx context.Context) (int64, error)

ChangelogHorizon returns the highest lsn currently in fs_changelog (0 when empty). Capture this as a checkpoint before doing work you might want to replay, then TrimChangelog up to it once the work is durable.

func (*Engine) ClearAllLocks 

func (e *Engine) ClearAllLocks(ctx context.Context) error

ClearAllLocks deletes every flock and plock row. The FUSE daemon calls this once at mount startup. Locks are advisory state tied to open file descriptions; the kernel tears all of them down when a mount goes away, so a lock row can never legitimately outlive the mount that took it. In the single-node deployment there are also no other live mounts, so any rows present at startup are stale leftovers from a previous (possibly crashed) instance. Without this, a crash while holding a lock leaves a row under a dead session id that every future locker sees as a live foreign holder — wedging that file's locking permanently (EAGAIN forever). Multi-node will instead reap by session liveness rather than clearing wholesale (L2.8).

func (*Engine) ClearLocksForSID 

func (e *Engine) ClearLocksForSID(ctx context.Context, sid uint64) error

ClearLocksForSID deletes only the flock/plock rows owned by one session id — the multi-node-safe form of startup reaping (L2.8). At mount startup a node calls this with its OWN sid to clear its stale rows from a prior crashed incarnation without touching peers' live locks (which ClearAllLocks would wrongly nuke). The liveness reaper calls it with a dead peer's sid once the peer is confirmed gone (absent from the node-liveness set).

func (*Engine) ClearPlocks 

func (e *Engine) ClearPlocks(ctx context.Context, inode Ino, owner uint64)

ClearPlocks drops all byte-range (POSIX) locks held by owner on inode. The FUSE bridge calls this on flush (every close(2)), matching the POSIX rule that closing any fd releases the process's record locks on that file. flock locks are released separately via Flock(..., 'U') on release.

func (*Engine) Clone 

func (e *Engine) Clone(ctx context.Context, srcIno, dstIno Ino) syscall.Errno

Clone implements copy-on-write: it copies every slice_ref row of srcIno onto dstIno (referencing the SAME physical slice ids — zero data movement) and sets dstIno's length, all in one transaction. The shared chunks stay live because dst's new slice rows reference them; mark-and-sweep GC only collects a chunk once NO slice row (in any file) references it. dstIno should be a freshly created, empty regular file.

func (*Engine) CloneRange 

func (e *Engine) CloneRange(ctx context.Context, srcIno, dstIno Ino, srcIdx, dstIdx, nChunks uint32) syscall.Errno

CloneRange copies-on-write a chunk-aligned, whole-chunk run from srcIno into dstIno: it replaces dstIno's slice rows for chunk indices [dstIdx, dstIdx+nChunks) with copies of srcIno's rows for chunk indices [srcIdx, srcIdx+nChunks), referencing the SAME physical slice ids (zero data movement, shared chunks — same GC liveness rule as Clone). It does NOT touch dstIno's length; the caller (the data path) extends it to the copy end. Used by copy_file_range for the fast path where both offsets and the length sit on chunk boundaries. nChunks == 0 is a no-op.

func (*Engine) Close 

func (e *Engine) Close(ctx context.Context, inode Ino) syscall.Errno

Close drops one open handle. If it was the last handle on an inode that is already unlinked (sustained), the inode and its data rows are finally deleted — POSIX open-but-unlinked semantics.

func (*Engine) ContentHash 

func (e *Engine) ContentHash(ctx context.Context, inode Ino) (string, syscall.Errno)

ContentHash returns a file's persisted whole-file content hash (empty until the file is first flushed). The mlfs↔VFS bridge reads it to register a blob_ref without re-hashing. A missing inode returns ENOENT.

func (*Engine) Create 

func (e *Engine) Create(ctx context.Context, parent Ino, name string, mode uint16, c *Context) (Ino, *Attr, syscall.Errno)

Create creates a regular file and returns its inode (ready to open).

func (*Engine) Fenced 

func (e *Engine) Fenced(ctx context.Context, scopeKey string, generation int64) (bool, error)

Fenced reports whether scopeKey is still at the given generation — the check a write performs (via "AND generation=$held") to know it has not been fenced out. A missing scope reports false.

func (*Engine) Flock 

func (e *Engine) Flock(ctx context.Context, inode Ino, owner uint64, typ uint8) syscall.Errno

Flock acquires ('R'/'W') or releases ('U') a whole-file BSD lock for owner. Non-blocking: EAGAIN if a conflicting holder exists.

func (*Engine) GetAttr 

func (e *Engine) GetAttr(ctx context.Context, inode Ino) (*Attr, syscall.Errno)

GetAttr returns a node's attributes.

func (*Engine) GetAttrOrNil 

func (e *Engine) GetAttrOrNil(ctx context.Context, inode Ino) *Attr

GetAttrOrNil returns the attr or nil (helper for return paths).

func (*Engine) GetXAttr 

func (e *Engine) GetXAttr(ctx context.Context, inode Ino, name string) ([]byte, syscall.Errno)

GetXAttr returns the value of extended attribute name on inode, or ENODATA if it is not set. (We do not distinguish a missing inode from a missing name on the read path; the kernel's preceding lookup already validated the inode.)

func (*Engine) Getlk 

func (e *Engine) Getlk(ctx context.Context, inode Ino, owner uint64, typ uint8, start, end uint64) (uint8, uint64, uint64, uint32, syscall.Errno)

Getlk tests whether the requested byte-range lock could be placed. It returns the first conflicting lock (its type/range/pid) or F_UNLCK if none conflict. 

func (e *Engine) Link(ctx context.Context, inode, newParent Ino, newName string, c *Context) (*Attr, syscall.Errno)

Link creates a hard link newName under newParent pointing at an existing non-directory inode, bumping its link count. Directory hardlinks are refused (EPERM), matching POSIX. Returns the linked inode's updated attributes.

func (*Engine) ListLockSIDs 

func (e *Engine) ListLockSIDs(ctx context.Context) ([]uint64, error)

ListLockSIDs returns the distinct session ids that currently hold any flock or plock row. The liveness reaper diffs this against the live-node set to find dead sessions whose locks must be cleared.

func (*Engine) ListXAttr 

func (e *Engine) ListXAttr(ctx context.Context, inode Ino) ([]string, syscall.Errno)

ListXAttr returns the names of all extended attributes on inode (sorted).

func (*Engine) LiveSliceIDs 

func (e *Engine) LiveSliceIDs(ctx context.Context) (map[uint64]struct{}, error)

LiveSliceIDs returns the set of slice ids currently referenced by some file (the distinct slice_id values in slice_ref). This is the live set for refcount-free, mark-and-sweep GC: any stored slice NOT in this set is an orphan (its last referencing inode was deleted) and may be reclaimed once it is older than the GC safety window. Copy-on-write shares a slice_id across inodes, so a slice stays live until its last reference is gone — exactly what DISTINCT captures.

func (*Engine) LiveSliceIDsIn 

func (e *Engine) LiveSliceIDsIn(ctx context.Context, ids []uint64) (map[uint64]struct{}, error)

LiveSliceIDsIn returns the subset of ids that are currently referenced by some file (have a slice_ref row). It is the batched form of LiveSliceIDs: a cursored GC checks one window of stored slice ids at a time instead of loading the entire live set into memory (TECH_DEBT #28), so peak RAM is bounded by the batch, not the total slice count. An empty input returns an empty set.

func (*Engine) LockSID 

func (e *Engine) LockSID() uint64

LockSID exposes this engine's lock-session id so the daemon can reap its own stale rows at startup (ClearLocksForSID) and publish it to the node-liveness set for peer reaping.

func (*Engine) Lookup 

func (e *Engine) Lookup(ctx context.Context, parent Ino, name string) (Ino, *Attr, syscall.Errno)

Lookup resolves a name in a directory to its inode + attributes.

func (*Engine) Migrate 

func (e *Engine) Migrate(ctx context.Context) error

Migrate applies the schema and seeds the root inode + inode counter. Safe to call repeatedly.

func (*Engine) Mkdir 

func (e *Engine) Mkdir(ctx context.Context, parent Ino, name string, mode uint16, c *Context) (Ino, *Attr, syscall.Errno)

Mkdir creates a subdirectory.

func (*Engine) Mknod 

func (e *Engine) Mknod(ctx context.Context, parent Ino, name string, typ uint8, mode uint16, rdev uint32, c *Context) (Ino, *Attr, syscall.Errno)

Mknod creates a non-directory node (file, fifo, device, socket).

func (*Engine) NewSlice 

func (e *Engine) NewSlice(ctx context.Context) (uint64, error)

NewSlice allocates a globally-unique slice id (batched from the shared counter). Slice ids name a slice's data in the chunk store.

func (*Engine) NodeID 

func (e *Engine) NodeID() string

NodeID returns this mount's coordination identity (empty in single-node mode).

func (*Engine) Open 

func (e *Engine) Open(ctx context.Context, inode Ino) (*Attr, syscall.Errno)

Open verifies the inode exists, registers an open handle, and returns attrs.

func (*Engine) OpenRef 

func (e *Engine) OpenRef(inode Ino)

OpenRef registers one open handle on inode (no I/O). The FUSE bridge calls this on Open/Create so the inode is retained if it is unlinked while open.

func (*Engine) Read 

func (e *Engine) Read(ctx context.Context, inode Ino, indx uint32) ([]Slice, syscall.Errno)

Read returns the slices written to chunk indx, in write order (overlay resolution into the visible slice list is a follow-up port of slice.go). 

func (e *Engine) ReadLink(ctx context.Context, inode Ino) ([]byte, syscall.Errno)

ReadLink returns a symlink's target.

func (*Engine) ReadSlices 

func (e *Engine) ReadSlices(ctx context.Context, inode Ino, indx uint32) ([]SliceRef, syscall.Errno)

ReadSlices returns the slice_ref rows for a chunk in WRITE order (ascending id). The data path resolves overlap; this returns the raw history.

func (*Engine) Readdir 

func (e *Engine) Readdir(ctx context.Context, inode Ino, plus bool) ([]*Entry, syscall.Errno)

Readdir returns a directory's entries (including "." and ".."); attributes are filled when plus is true.

func (*Engine) ReapOrphanedInodes 

func (e *Engine) ReapOrphanedInodes(ctx context.Context) (int, error)

ReapOrphanedInodes deletes crash-orphaned ("zombie") inodes: node rows whose link count has reached zero AND that no edge points at, i.e. a file that was unlinked while open ("sustained") whose holder crashed before the final Close could delete it. Such a node has no name and no live handle, but its slice_ref rows still pin its chunks (LiveSliceIDs counts them as live), so slice GC can never reclaim them until the node itself is removed. This calls deleteInodeRows for each, which clears node/symlink/slice_ref/xattr in the same transaction; freed slices then become reclaimable on the next GC pass.

RootInode is never reaped (it is parentless by design). Safe offline only: like ClearAllLocks it assumes no live mount is mid-sustain (single-node), so it must run from mlfs-admin against a quiesced filesystem, never a live one.

func (*Engine) Refresh 

func (e *Engine) Refresh(ctx context.Context, scopeKey, node string, generation, ttlMs int64) (ok bool, err error)

Refresh extends node's lease on scopeKey only if it still owns it at the given generation. Returns false if the scope was stolen (fenced out).

func (*Engine) ReleaseOwnership 

func (e *Engine) ReleaseOwnership(ctx context.Context, scopeKey, node string, generation int64) (bool, error)

ReleaseOwnership voluntarily gives up scopeKey by expiring its lease, but only if this node still holds it at generation. A successor's AcquireOrResume then sees the expired lease and steals it (bumping the generation), which fences any late write from this node. Used by the ownership-handoff path (L2.8.4): the prior owner releases so the requester can take over immediately rather than waiting out the TTL. Returns true if the release applied.

func (*Engine) RemoveXAttr 

func (e *Engine) RemoveXAttr(ctx context.Context, inode Ino, name string) syscall.Errno

RemoveXAttr deletes extended attribute name from inode, or ENODATA if unset.

func (*Engine) Rename 

func (e *Engine) Rename(ctx context.Context, parentSrc Ino, nameSrc string, parentDst Ino, nameDst string, flags uint32, c *Context) (Ino, *Attr, syscall.Errno)

Rename moves srcName under parentSrc to dstName under parentDst, atomically in one SQL transaction (so a rename across ownership scopes is a single commit — the plan's "rename across owners works as one SQL transaction"). An existing destination is replaced: a file is unlinked, an empty directory is removed; a non-empty directory yields ENOTEMPTY. RenameNoReplace fails with EEXIST if the destination exists. RenameExchange atomically swaps the two named entries (both must exist). RenameWhiteout (overlayfs) is not supported and returns ENOSYS.

func (*Engine) Rmdir 

func (e *Engine) Rmdir(ctx context.Context, parent Ino, name string, c *Context) syscall.Errno

Rmdir removes an empty subdirectory.

func (*Engine) ScanChangelog 

func (e *Engine) ScanChangelog(ctx context.Context, after int64, fn func(ChangeEntry) error) error

ScanChangelog yields changelog entries with lsn > after, in lsn order. This is the durable, replayable backstop for the change feed (L2.8).

func (*Engine) ScopeOf 

func (e *Engine) ScopeOf(ctx context.Context, inode Ino) (string, error)

ScopeOf returns the ownership scope key for inode. Exported for the coordinator (lease handoff, admin tooling) and tests; the write path uses the unexported scopeOf directly.

func (*Engine) SetAttr 

func (e *Engine) SetAttr(ctx context.Context, inode Ino, set uint16, attr *Attr, c *Context) (*Attr, syscall.Errno)

SetAttr updates the selected attributes (set mask) and bumps ctime. The caller c authorizes ownership-changing fields: only the owner or root may chmod/chgrp, and only root may chown to a different uid (EPERM otherwise).

func (*Engine) SetClock 

func (e *Engine) SetClock(now func() time.Time)

SetClock overrides the clock (tests).

func (*Engine) SetContentHash 

func (e *Engine) SetContentHash(ctx context.Context, inode Ino, hash string) syscall.Errno

SetContentHash records a file's whole-file content hash (see Attr.ContentHash): the lowercase-hex sha256 of the file's bytes, matching blobgw's ContentHash encoding. The bridge calls it on flush/close once the running or recomputed digest is finalized. It is a plain attribute UPDATE (no changelog / fence): the value is derived from the file's already-committed data, so a stale-generation writer would have failed its own data writes' fence first; this only stamps the resulting digest. A no-op (0 rows) on a missing inode returns ENOENT.

func (*Engine) SetCoordinator 

func (e *Engine) SetCoordinator(nodeID string, owner OwnerView)

SetCoordinator enables multi-node ownership fencing: nodeID identifies this mount and owner drives lease acquisition/liveness. Passing a nil owner (or not calling this at all) leaves the engine in single-node mode. Call once, before the engine starts serving.

func (*Engine) SetMetricsRecorder 

func (e *Engine) SetMetricsRecorder(r MetaRecorder)

SetMetricsRecorder installs the meta-DB RED + fence recorder (nil clears it). Call once before serving. A nil engine or nil recorder leaves recording off; the engine never assumes a recorder is present.

func (*Engine) SetStoreClass 

func (e *Engine) SetStoreClass(ctx context.Context, inode Ino, class uint8) syscall.Errno

SetStoreClass records a file's compression class (see Attr.StoreClass). The bridge calls it once, right after creating a file, when its type classifies as non-default; the class is then inherited by every slice the file writes. It is a plain attribute UPDATE (no changelog / fence): the class is set by the creating client on a brand-new inode before any data is written, so there is no concurrent writer to fence against. A no-op (0 rows) on a missing inode returns ENOENT.

func (*Engine) SetXAttr 

func (e *Engine) SetXAttr(ctx context.Context, inode Ino, name string, value []byte, flags uint32) syscall.Errno

SetXAttr sets extended attribute name=value on inode, honoring the create/replace flags. ENOENT if the inode is gone, EEXIST/ENODATA on a flag violation.

func (*Engine) Setlk 

func (e *Engine) Setlk(ctx context.Context, inode Ino, owner uint64, typ uint8, start, end uint64, pid uint32) syscall.Errno

Setlk acquires (F_RDLCK/F_WRLCK) or releases (F_UNLCK) a byte-range lock for owner. Non-blocking: EAGAIN if another owner holds a conflicting range. 

func (e *Engine) Symlink(ctx context.Context, parent Ino, name, target string, c *Context) (Ino, *Attr, syscall.Errno)

Symlink creates a symbolic link with the given target.

func (*Engine) TrimChangelog 

func (e *Engine) TrimChangelog(ctx context.Context, upTo int64) (int64, error)

TrimChangelog deletes every entry with lsn <= upTo and returns the number removed. upTo <= 0 is a no-op. Pass the minimum LSN that all change-feed consumers have durably acknowledged.

func (*Engine) TrimChangelogKeepLast 

func (e *Engine) TrimChangelogKeepLast(ctx context.Context, keep int64) (int64, error)

TrimChangelogKeepLast retains only the newest keep entries (by lsn), deleting the rest. keep <= 0 deletes nothing. Use this for a simple bounded-size log when there is no external consumer dictating the safe horizon.

func (*Engine) TrimChangelogOlderThan 

func (e *Engine) TrimChangelogOlderThan(ctx context.Context, age time.Duration) (int64, error)

TrimChangelogOlderThan deletes entries whose timestamp is older than age, measured against the engine clock. age <= 0 deletes nothing.

func (*Engine) Truncate 

func (e *Engine) Truncate(ctx context.Context, inode Ino, size int64) syscall.Errno

Truncate sets the file length to size. On shrink it destroys data beyond size — slices entirely past it are deleted, a slice straddling it is trimmed — so a later grow reads zeros (POSIX semantics), not the old bytes. Grow is metadata-only (the tail is sparse). 

func (e *Engine) Unlink(ctx context.Context, parent Ino, name string, c *Context) syscall.Errno

Unlink removes a file entry; the node (and its symlink row) is deleted when its link count reaches zero AND it is not open; an open file is kept (sustained) and deleted on the last Close (POSIX open-but-unlinked).

func (*Engine) Write 

func (e *Engine) Write(ctx context.Context, inode Ino, indx uint32, off uint32, slice Slice) syscall.Errno

Write records a slice over chunk indx at byte offset off and extends the file length if needed.

type Entry 

type Entry struct {
	Inode Ino
	Name  string
	Attr  *Attr
}

Entry is a directory entry.

type Ino 

type Ino uint64

Ino is a 64-bit inode number. mlfs partitions the space as a 16-bit region+session prefix and a 48-bit per-node counter (see inode.go). 

const RootInode Ino = 1

RootInode is the fixed inode of the filesystem root.

type Lease 

type Lease struct {
	ScopeKey      string
	OwnerNode     string
	ExpiresUnixMs int64
	Generation    int64
}

Lease is the state of one scope's ownership row. Generation is the fencing token: it strictly increases each time ownership changes hands, so a stale owner's writes (guarded by `AND generation = $held`) are rejected after the scope has been stolen. SQL is the system of record; the lease is a write-affinity hint, and generation is the correctness primitive (L2.8 threads it through every metadata UPDATE).

type MetaRecorder 

type MetaRecorder interface {
	RecordMetaOp(op string, start time.Time, err *error)
	FenceRejected()
}

MetaRecorder is the engine's optional metrics hook. *metrics.Registry implements it. RecordMetaOp takes the op label, the op's start time, and a pointer to the error (nil = success), so a call site can `defer e.rec.RecordMetaOp("read_slices", time.Now(), &err)`. FenceRejected counts a write rejected because the ownership generation moved (multi-node).

type Option 

type Option func(*Engine)

Option configures an Engine at construction time (see Open).

func WithLogger 

func WithLogger(l *slog.Logger) Option

WithLogger sets the slog.Logger used for operational diagnostics (e.g. the underlying cause behind an EIO returned to FUSE). A nil logger is ignored so the engine never logs through a nil handler.

type OwnerView 

type OwnerView interface {
	// Enabled reports whether multi-node fencing is active. False short-circuits
	// the whole scope/fence path for the single-node fast path.
	Enabled() bool
	// Ensure makes this node the owner of scope (acquiring or renewing the
	// lease) and returns the generation it holds. held=false means another live
	// node owns it: the caller returns ESTALE (the kernel/caller retries, by
	// which point a handoff may have completed).
	Ensure(ctx context.Context, scope string) (gen int64, held bool, err error)
}

OwnerView is the engine's hook into multi-node ownership coordination (L2.8). Before any metadata mutation, the write path calls Ensure(scope) to acquire-or-confirm this node's lease on the scope; the returned generation is the fencing token re-checked inside the write transaction (see checkFence).

Single-node mode wires noopOwner, whose Enabled() reports false — the engine then skips scope resolution and the fence outright, preserving the exact single-node behavior and cost. The authoritative fence always lives in SQL (the ownership row); an OwnerView only decides liveness and accelerates discovery (e.g. the NATS-backed coordinator in internal/coord).

type SQLOwner 

type SQLOwner struct {
	// contains filtered or unexported fields
}

SQLOwner is a pure-PostgreSQL OwnerView: it drives the ownership-lease primitives (AcquireOrResume / Refresh) already on the Engine. It makes the engine multi-node-SAFE without any external coordinator — the fence is fully authoritative in SQL. It is also the reference an accelerator (the NATS-backed coordinator in internal/coord) is measured against: that swap replaces the liveness/refresh traffic with KV+Watch but keeps this exact fence contract.

Lifecycle: construct with NewSQLOwner, register via Engine.SetCoordinator, then run Run(ctx) in a goroutine to keep held leases alive. Ensure is called on the write path (cheap in-memory fast path when the lease is already held); the authoritative re-check happens in the write tx via checkFence.

func NewSQLOwner 

func NewSQLOwner(e *Engine, nodeID string, ttl, refresh time.Duration) *SQLOwner

NewSQLOwner builds a SQL-backed owner for nodeID. ttl is the lease lifetime (a foreign owner is considered dead after it); refresh is how often Run renews held leases (should be <= ttl/2). Zero values fall back to 30s / 10s.

func (*SQLOwner) Enabled 

func (o *SQLOwner) Enabled() bool

Enabled reports that multi-node fencing is active.

func (*SQLOwner) Ensure 

func (o *SQLOwner) Ensure(ctx context.Context, scope string) (int64, bool, error)

Ensure acquires-or-confirms this node's lease on scope. The fast path returns a still-valid cached generation with no SQL round-trip; otherwise it calls AcquireOrResume. held=false means a live foreign owner holds the scope.

A stale cache (we believe we hold it but were stolen) is harmless: Ensure returns the cached generation, and checkFence in the write tx rejects it against the authoritative SQL row, yielding ESTALE. Run also drops the lease on the next refresh.

func (*SQLOwner) Run 

func (o *SQLOwner) Run(ctx context.Context)

Run renews every held lease on a ticker until ctx is cancelled. A lease whose Refresh fails (the scope was stolen) is dropped, so the next write re-acquires or reports ESTALE. Idle eviction is intentionally omitted from this first cut (see docs/L2.8_IMPLEMENTATION.md §4); a held scope stays held until lost.

type Slice 

type Slice struct {
	Id   uint64
	Size uint32
	Off  uint32
	Len  uint32
}

Slice is one write of data over a chunk: a (sliceId, size) source, read from [Off, Off+Len). Slices overlay in write order within a chunk.

type SliceRef 

type SliceRef struct {
	Pos uint32
	Slice
}

SliceRef is a raw slice_ref row: a slice at chunk-relative byte position Pos, sourced from chunk-store slice ID at [Off, Off+Len). Pos is what the data path needs to overlay slices (newer wins); meta.Slice (Id/Size/Off/Len) omits it.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.