Documentation
¶
Overview ¶
Package wal is mlfs's local data write-ahead log.
Each in-flight write appends a durable (fsync'd) record describing the slice it produced — (seq, op, ino, indx, off, slice_id, geometry) — BEFORE the metadata commit to PostgreSQL. If the daemon crashes after the data is staged (internal/cache) but before the slice_ref row commits, startup Replay re-runs the metadata write so no acknowledged write is lost. Replay is idempotent via the slice_id natural key (meta.Write uses ON CONFLICT DO NOTHING).
Records are framed [u32 len][u32 crc32(payload)][payload]; a torn tail (short read or bad CRC, only possible at the end of the last segment after a crash) is truncated on Open. Segments roll at a size threshold; Checkpoint drops segments whose records have all been committed.
Index ¶
Constants ¶
const DefaultSegmentBytes = 64 << 20 // 64 MiB
DefaultSegmentBytes is the size at which a segment rolls.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Record ¶
type Record struct {
Seq uint64
Op OpType
Ino uint64
Indx uint32
Off uint32
SliceID uint64
Size uint32
SliceOff uint32
SliceLen uint32
}
Record is one logged operation. For OpWrite the slice fields describe the data; for OpTruncate, Off carries the new length's low 32 bits and Indx the high bits (callers encode as needed).
type WAL ¶
type WAL struct {
// contains filtered or unexported fields
}
WAL is an append-only, segmented, fsync'd log. Safe for concurrent use.
func Open ¶
Open opens (or creates) the WAL in dir, truncating a torn tail left by a crash and positioning to append after the highest valid record.
func (*WAL) Append ¶
Append assigns a sequence number, writes the record framed + fsync'd, and returns the assigned seq. The record is durable when Append returns.
func (*WAL) Checkpoint ¶
Checkpoint drops fully-committed segments: any non-current segment whose highest seq is <= upToSeq is deleted (its records have all landed in SQL).
Source Files