Documentation
¶
Overview ¶
Package cache is mlfs's local disk cache and durable write-back staging in front of the casstore-backed chunk store.
A Write lands in a staging file that is fsync'd (file AND directory) BEFORE the call returns, so an acknowledged write survives a kill -9. A background uploader drains staging into the backing chunk store; on restart, recover() re-uploads any staging entries left behind by a crash. Clean (already-uploaded) cache entries are LRU-evictable; staging entries are never evicted (they are the only durable copy until uploaded). Every cache/staging file embeds a sha256 header that is verified on read, so corruption is detected, not served.
DiskCache implements chunkstore.Store, so it is a drop-in decorator.
Index ¶
- Variables
- type Config
- type DiskCache
- func (c *DiskCache) CachedLocally(id uint64) bool
- func (c *DiskCache) Close() error
- func (c *DiskCache) Exists(ctx context.Context, id uint64) (bool, error)
- func (c *DiskCache) Flush(ctx context.Context) error
- func (c *DiskCache) Read(ctx context.Context, id uint64) ([]byte, error)
- func (c *DiskCache) ReadAt(ctx context.Context, id uint64, off int64, p []byte) (int, error)
- func (c *DiskCache) ReadBatch(ctx context.Context, ids []uint64, concurrency int) (map[uint64][]byte, chunkstore.BatchReadStats, error)
- func (c *DiskCache) Remove(ctx context.Context, id uint64) error
- func (c *DiskCache) Write(ctx context.Context, id uint64, data []byte, class chunkstore.StoreClass) error
Constants ¶
This section is empty.
Variables ¶
var ErrStagingCorrupt = errors.New("cache: corrupt staging entry quarantined")
ErrStagingCorrupt reports that one or more acknowledged staging writes failed their checksum and were quarantined (not silently dropped). Returned by Flush/Close when no Config.OnCorrupt handler is set.
Functions ¶
This section is empty.
Types ¶
type Config ¶
type Config struct {
Dir string // clean read-cache root (the reclaimable, evictable cache/ tree)
// StagingDir, when set, places the DURABLE write-back data — staging/ (the
// only copy of an acked-but-not-yet-uploaded write) and corrupt/ (quarantined
// acked writes) — on a separate filesystem from Dir. Empty = under Dir (the
// original layout). Splitting them lets Dir live on fast/ephemeral storage
// (instance-store NVMe) without weakening the "an acked write survives node
// termination" guarantee: point StagingDir (and the WAL) at a persistent disk.
StagingDir string
MaxBytes int64 // clean-cache size cap (0 = unlimited)
MaxItems int // clean-cache item cap (0 = unlimited)
// MinFreeFraction evicts the clean cache to keep at least this fraction of the
// cache *filesystem* free (0 = disabled; e.g. 0.15 = treat the cache as full
// once free space ≤ 15% of the volume). It lets the LRU self-size to each
// node's disk without a hard-coded byte cap, and composes with MaxBytes (the
// cache is trimmed to satisfy whichever bound bites first). Only the clean
// cache is reclaimable — staging (undurable writes) is never evicted — so the
// floor is best-effort when staging + other on-disk usage already exceed it.
MinFreeFraction float64
AutoUpload bool // run the background uploader (off → drain only on recover/Flush)
UploadInterval time.Duration // background drain cadence (default 1s)
// UploadConcurrency bounds how many uploads a single drain runs in parallel
// (≤1 = serial). It applies to BOTH drain paths: the batched path (backings
// implementing chunkstore.BatchWriter) flushes up to this many ~pack-sized
// batches at once; the per-slice fallback uploads up to this many slices at
// once. On the high-latency -remote path serial uploads are round-trip-bound,
// so a bounded pool multiplies throughput. This is INTRA-drain concurrency
// only — the upload loop is still a single nudge-coalesced goroutine.
UploadConcurrency int
// OnCorrupt is invoked when a staging entry fails its checksum on drain.
// The bytes are first quarantined (moved to corrupt/) so the acknowledged
// write is preserved for forensics/manual recovery, then this fires with
// the quarantine path. Use it to alert/log. If nil, Flush/Close surface
// ErrStagingCorrupt instead; recovery on open never blocks the mount.
OnCorrupt func(id uint64, quarantinePath string, cause error)
// OnError is invoked when a background drain (driven by the upload loop or a
// write nudge) fails. Those drains have no caller to return the error to, so
// without a sink the failure would be swallowed. If nil, errors are logged
// via slog.Default(). Synchronous drains (recover/Flush/Close) still return
// their errors to the caller and do NOT route through OnError.
OnError func(error)
// Metrics, if set, records cache hit/miss and write-back upload counters and
// exposes the live staging depth. nil disables instrumentation (all calls are
// nil-safe).
Metrics *metrics.Registry
}
Config tunes the cache.
type DiskCache ¶
type DiskCache struct {
// contains filtered or unexported fields
}
DiskCache wraps a backing chunkstore.Store with durable staging + an LRU read cache.
func New ¶
func New(backing chunkstore.Store, cfg Config) (*DiskCache, error)
New opens (or creates) the cache at cfg.Dir over backing, replays any crash-left staging entries, and starts the background uploader when cfg.AutoUpload is set.
func (*DiskCache) CachedLocally ¶
CachedLocally reports whether slice id is resident in the local clean cache or the durable staging dir, with NO backing-store round trip (unlike Exists, which falls through to the backing store / S3 on a local miss). The readahead prefetcher uses it to skip slices that are already resident.
func (*DiskCache) ReadBatch ¶
func (c *DiskCache) ReadBatch(ctx context.Context, ids []uint64, concurrency int) (map[uint64][]byte, chunkstore.BatchReadStats, error)
ReadBatch serves many slices at once: local hits are read from the cache, and misses are fetched in ONE coalesced call to the backing (when it is a chunkstore.BatchReader — casstore's cross-key pack coalescing) then populated into the clean cache exactly like a Read miss, so the LRU/checksum invariants are untouched. Falls back to per-id Read when the backing can't batch. Best-effort: an unreadable id is simply absent from the result.
func (*DiskCache) Write ¶
func (c *DiskCache) Write(ctx context.Context, id uint64, data []byte, class chunkstore.StoreClass) error
Write stages data durably (fsync'd) and returns; the uploader moves it to the backing store. The acknowledged write survives a crash via the staging file. class is persisted WITH the staged bytes so a recovery drain (which has no inode context) packs the slice under the right codec.