Documentation
¶
Overview ¶
Package wal implements a versioned, length-prefixed, CRC32C-checksummed Write-Ahead Log for the gograph durability stack.
The on-disk format is documented in FORMAT.md alongside this package. Each frame is self-describing; readers stop cleanly at the first torn or corrupted frame and report the byte offset where the cut occurred, leaving the file otherwise untouched.
Example ¶
Example shows the core write-ahead-log loop: open a writer, append a few opaque payload frames, Sync them durably, then reopen the file with a Reader and replay every frame back in append order.
package main
import (
"fmt"
"os"
"path/filepath"
"github.com/FlavioCFOliveira/GoGraph/store/wal"
)
func main() {
dir, err := os.MkdirTemp("", "wal-example")
if err != nil {
panic(err)
}
defer func() { _ = os.RemoveAll(dir) }()
path := filepath.Join(dir, "wal")
// Append three records. A WAL payload is opaque bytes as far as the
// log is concerned; the durability stack above it (store/txn) gives
// them meaning. Group-commit: several Appends, then a single Sync.
w, err := wal.Open(path)
if err != nil {
panic(err)
}
for _, rec := range [][]byte{[]byte("alpha"), []byte("bravo"), []byte("charlie")} {
if err := w.Append(rec); err != nil {
panic(err)
}
}
if err := w.Sync(); err != nil {
panic(err)
}
if err := w.Close(); err != nil {
panic(err)
}
// Replay: a fresh Reader iterates the frames in the order they were
// appended, stopping cleanly at the first torn frame (none here).
r, err := wal.OpenReader(path)
if err != nil {
panic(err)
}
defer func() { _ = r.Close() }()
count := 0
err = r.Replay(func(f wal.Frame) error {
count++
fmt.Printf("frame %d: %s\n", count, f.Payload)
return nil
})
if err != nil {
panic(err)
}
fmt.Printf("replayed %d frames\n", count)
}
Output: frame 1: alpha frame 2: bravo frame 3: charlie replayed 3 frames
Index ¶
- Constants
- Variables
- func Encode(w io.Writer, f Frame) (int, error)
- type Frame
- type Reader
- type Stats
- type WALFile
- type Writer
- func (w *Writer) Append(payload []byte) error
- func (w *Writer) AppendCtx(ctx context.Context, payload []byte) error
- func (w *Writer) Close() error
- func (w *Writer) DurableOffset() int64
- func (w *Writer) Stats() Stats
- func (w *Writer) Sync() error
- func (w *Writer) SyncCtx(ctx context.Context) error
- func (w *Writer) SyncGroup() error
- func (w *Writer) Truncate() (int64, error)
- func (w *Writer) TruncatePrefix(upTo int64) (int64, error)
Examples ¶
Constants ¶
const CurrentVersion uint16 = 1
CurrentVersion is the WAL format version this package writes. Readers must accept all versions <= CurrentVersion; older versions are intentionally permitted so a fresh build can replay archives produced by previous releases.
const HeaderSize = 4 + 2 + 4 + 4
HeaderSize is the fixed number of bytes occupying the frame header (magic + version + length + crc32c).
Variables ¶
var ( // ErrBadMagic indicates the next four bytes did not match Magic. ErrBadMagic = errors.New("wal: bad frame magic") // ErrUnsupportedVersion indicates the frame version is newer // than this build knows how to parse. ErrUnsupportedVersion = errors.New("wal: unsupported frame version") // ErrCRCMismatch indicates the frame's CRC32C did not match the // re-computed value. ErrCRCMismatch = errors.New("wal: crc32c mismatch") // ErrTornFrame indicates the underlying reader returned EOF // before the frame was fully read. ErrTornFrame = errors.New("wal: torn frame at end of input") // ErrTornFrameMasksData indicates a frame's declared payload length // over-declared past the end of input AND the bytes it would have // consumed contain at least one further valid (CRC-checking) frame. // This is genuine mid-stream corruption masquerading as a benign torn // tail: a corrupt length field swallowed durable frames that follow it. // Unlike [ErrTornFrame] (a benign final partial write), this is a hard // error — it MUST fail-stop so the durable frames the bad length hid are // never silently dropped. It is a DISTINCT sentinel (it deliberately does // not wrap [ErrTornFrame]) so recovery's corruption classifier treats it // as corruption rather than a benign tail. ErrTornFrameMasksData = errors.New("wal: torn frame hides later valid frames (corrupt length)") // ErrFrameTooLarge indicates the frame's declared payload length // exceeds maxFrameSize. A length field this large is treated as // corruption: the frame is rejected before any allocation, so a // crafted or corrupted length cannot force a large one-shot make. ErrFrameTooLarge = errors.New("wal: frame payload length exceeds maximum") )
Errors returned by the reader.
var ErrPrefixTruncateUnsupported = errors.New("wal: TruncatePrefix requires a path-backed writer (use Open, not OpenWith)")
ErrPrefixTruncateUnsupported is returned by Writer.TruncatePrefix on a Writer created via OpenWith (a synthetic, path-less file handle). The crash-safe prefix truncation works by writing the surviving suffix to a sibling temp file and atomically renaming it over the WAL path, so it requires a real filesystem path — which only Open records.
var ErrWALLocked = errors.New("wal: WAL directory is locked by another process")
ErrWALLocked is returned by Open when another process already holds the exclusive OS-level lock on the WAL directory. It signals that the WAL is in active use and the caller must not open a second writer against it — doing so would silently interleave frames and corrupt the log.
var ErrWriterClosed = errors.New("wal: writer is closed")
ErrWriterClosed is returned by methods on a Writer that has already been closed.
var Magic = [4]byte{'G', 'G', 'W', 'A'}
Magic is the 4-byte identifier prefix of every WAL frame: ASCII "GGWA".
Functions ¶
Types ¶
type Frame ¶
Frame is the in-memory representation of one WAL frame.
func Decode ¶
Decode reads the next frame from r. It returns ErrTornFrame when the reader ends mid-frame (clean tail truncation), ErrBadMagic on a missing magic, ErrUnsupportedVersion on a newer-than-supported version, and ErrCRCMismatch on integrity failure. Any other error is propagated from the underlying reader.
type Reader ¶
type Reader struct {
// contains filtered or unexported fields
}
Reader iterates the frames of a WAL file. It is read-only and stops cleanly at the first torn or corrupted frame, reporting the byte offset where the cut occurred via Reader.TailOffset.
Reader is not safe for concurrent use; create one Reader per goroutine that wishes to iterate.
func NewReader ¶
NewReader builds a Reader over an io.Reader. closer may be nil if the caller owns the resource.
func OpenReader ¶
OpenReader opens path for read-only frame iteration.
func (*Reader) Close ¶
Close releases any underlying resource passed to NewReader or OpenReader.
func (*Reader) Frames ¶
Frames returns an iterator over every frame in the WAL. The iterator stops at the first error; call Reader.TailError / Reader.TailOffset after iteration to inspect why.
func (*Reader) Replay ¶
Replay applies apply to every frame in the WAL in order. If apply returns an error, replay stops with that error returned to the caller. After Replay returns, TailOffset/TailError describe where and why iteration stopped (frame-level errors).
func (*Reader) TailError ¶
TailError returns the error that ended iteration (typically ErrTornFrame, ErrCRCMismatch, or ErrBadMagic), or nil when iteration ended at clean EOF.
func (*Reader) TailOffset ¶
TailOffset returns the byte offset (from the start of the input) where iteration stopped. After a successful iteration to EOF this equals the file size; after a torn frame this equals the start of the torn frame.
type Stats ¶
type Stats struct {
Frames uint64 // total frames appended
Bytes uint64 // total bytes appended (header + payload)
Syncs uint64 // total Sync calls
// SyncFailed counts Sync calls that failed at the flush/fsync
// I/O layer. Calls rejected because the writer was already
// poisoned by an earlier failure are not counted (mirroring how
// context-cancelled calls are not counted).
SyncFailed uint64
}
Stats is a snapshot of a Writer's lifetime counters. Counters are monotonic; subtract two snapshots to compute deltas. Values are read with sync/atomic.LoadUint64, so they may race slightly behind in-flight operations but never observe a torn value.
type WALFile ¶ added in v0.6.0
type WALFile interface {
io.Writer
// Reader is required by [Writer.TruncatePrefix], which reads the
// surviving suffix of the file (the frames committed after the
// captured watermark) before atomically replacing the file with a
// suffix-only copy. *os.File and *testfs.FaultFile both satisfy it.
io.Reader
io.Seeker
// Sync flushes OS write buffers to durable storage.
Sync() error
// Truncate resizes the file to size bytes.
Truncate(size int64) error
// Close releases underlying OS resources.
Close() error
}
WALFile is the minimal open-handle interface that Writer requires of its underlying file. *os.File and *testfs.FaultFile both satisfy it, which lets fault-injection tests substitute a synthetic file without touching production paths.
It is exported so an external filesystem backend (the deterministic- simulation harness, internal/sim) can name it as the return type of its [walFS].OpenFile method and thereby satisfy the unexported walFS interface, exactly as github.com/FlavioCFOliveira/GoGraph/store/snapshot.File is exported for the snapshot seam. Production callers open WAL files via Open (which wraps *os.File) and never reference this type directly; tests and the simulator reach for OpenWith / OpenFS.
Concurrency: a WALFile is used by a single Writer whose own mutex serialises every access; any implementation's further guarantees are its own.
type Writer ¶
type Writer struct {
// contains filtered or unexported fields
}
Writer is a single-writer append-only log file. Callers append frames with Writer.Append and durably commit them with Writer.Sync; group-commit is achieved by appending several frames before a single Sync.
Writer is safe for concurrent calls to Writer.Append / Sync / Stats; all mutations serialise on an internal mutex.
A Writer fail-stops on commit failure: the first flush or fsync error in Writer.Sync permanently poisons the writer. The un-synced suffix of the file — which may hold the flushed frames (including the commit marker) of the very transaction whose Sync just failed — is physically discarded, and every subsequent Append/Sync returns the original error. Without the poison, a later transaction's successful fsync would make the failed transaction's frames durable even though its commit was never acknowledged, and recovery would replay it: a phantom commit violating Atomicity and Durability. A poisoned Writer accepts only Writer.Close; the owner must discard it and re-open the WAL, which re-validates the tail.
func Open ¶
Open opens or creates the WAL file at path for append-only writing. The file is created with mode 0o600 (owner read/write only) if it does not already exist; existing complete frames are preserved and new frames are appended after them. The restrictive mode keeps the full graph mutation stream from being world-readable.
When the existing file ends in a benign torn frame (ErrTornFrame — the crash-mid-write-after-last-fsync case), Open truncates the file to the last durable frame boundary and fsyncs it before returning, so new frames are never appended after torn junk that every reader would stop at; see discardTornTail. Files whose scan stops at genuine corruption (for example ErrCRCMismatch or ErrBadMagic) are left byte-for-byte intact.
func OpenFS ¶ added in v0.6.0
OpenFS builds a path-backed Writer over a caller-supplied filesystem backend. Unlike OpenWith (which produces a path-less Writer that rejects Writer.TruncatePrefix with ErrPrefixTruncateUnsupported), OpenFS records the path and routes the truncation's temp-write/rename/remove/parent-dir-fsync through fsys, so a Checkpointer can reclaim the WAL prefix over the injected filesystem. The caller transfers ownership of the opened handle: Writer.Close will close it.
OpenFS is the seam the deterministic-simulation harness (internal/sim) uses to run the full snapshot + WAL + checkpoint stack against its in-memory [SimDisk]; production code uses Open.
Unlike Open, OpenFS does NOT acquire the OS-level WAL directory lock (flock(2)/O_EXCL has no analogue on an injected backend, and OpenFS callers are single-writer by contract) and does NOT scan for or discard a benign torn tail. The caller MUST therefore pre-truncate any benign torn tail to the last durable frame boundary (recovery reports it as github.com/FlavioCFOliveira/GoGraph/store/recovery.ReplayResult.WALTailOffset) BEFORE calling OpenFS, exactly as the production Open does internally via discardTornTail; appending after un-discarded torn junk would strand every new frame behind bytes that every reader stops at — a Durability violation.
func OpenWith ¶
OpenWith builds a Writer over an already-open file handle. The caller transfers ownership: Writer.Close will call f.Close().
This constructor exists primarily for tests that inject a *testfs.FaultFile; production code should use Open.
func (*Writer) Append ¶
Append writes one frame with the given opaque payload to the underlying file. The frame is buffered in process memory; call Writer.Sync to durably commit.
On a writer poisoned by an earlier Sync failure, Append rejects the frame and returns the original sync error; see the Writer type documentation.
func (*Writer) AppendCtx ¶
AppendCtx is the context-aware variant of Writer.Append. ctx.Err() is checked before acquiring the internal mutex and again before writing; on cancellation returns the wrapped ctx.Err.
func (*Writer) Close ¶
Close flushes any buffered frames, calls Sync once, and releases the underlying file.
On a writer poisoned by an earlier Sync failure, Close skips the flush (which would buffer new data) but performs a second-chance truncation followed by a best-effort fsync. The truncation to durableSize discards any suffix that poison() may have failed to discard on a device in transient distress. The subsequent fsync makes the reduced inode metadata durable; it is safe to issue because the Truncate has already shrunk the file — the fsync can only confirm the reduced EOF, never resurrect data above it.
func (*Writer) DurableOffset ¶ added in v0.3.1
DurableOffset returns the file size, in bytes, covered by the last successful fsync — the byte length of every frame durably committed so far. The value always lands on a frame boundary: a transaction commits only after its [OpCommit] marker has been appended and fsynced, so the durable prefix is always a whole number of complete frames.
It is the watermark a non-blocking checkpoint captures (see store/checkpoint): taken under the store's quiesce boundary ([txn.Store.RunUnderCommitLock], which drains in-flight group commits) the returned offset equals appendedSize — every committed frame is durable and none is mid-flight — so it is exactly the prefix a self-sufficient snapshot folds and Writer.TruncatePrefix may discard.
Concurrency: safe for concurrent use; it reads durableSize under the internal mutex.
func (*Writer) Sync ¶
Sync flushes the buffered frames to the OS and then issues the per-commit data sync (fdatasync(2) on Linux, os.File.Sync / fsync elsewhere; see dataSync) so the appended frames and the grown file size reach durable storage before returning. It must be invoked at every transaction commit boundary.
The first flush or fsync failure permanently poisons the writer: the un-synced suffix of the file is discarded and every subsequent Append/Sync returns the original error; see the Writer type documentation.
func (*Writer) SyncCtx ¶
SyncCtx is the context-aware variant of Writer.Sync. ctx.Err() is checked before acquiring the internal mutex; on cancellation returns the wrapped ctx.Err without flushing.
func (*Writer) SyncGroup ¶ added in v0.3.1
SyncGroup durably commits the caller's already-appended frames, coalescing the fsync with those of every other committer whose frames are buffered at the same time — PostgreSQL-XLogFlush-style group commit. It returns nil only after a single data sync (fdatasync on Linux, fsync elsewhere; see dataSync) has made durable every byte up to and including the caller's last appended frame (its OpCommit marker); a caller therefore acknowledges its commit only once the marker is on stable storage, exactly as Writer.Sync does, but without paying a private fsync per commit.
Contract ¶
SyncGroup must be called AFTER the caller has appended all of its frames via Writer.Append / Writer.AppendCtx and is the durability barrier for those frames. It captures the current appendedSize as the caller's watermark, then:
- If the writer is already poisoned, it returns the sticky error immediately (the un-synced suffix, including this caller's frames, was discarded by an earlier failed sync).
- If a previous sync has already advanced durableSize past the caller's watermark (a concurrent leader covered it), it returns nil without any I/O — the follower fast path.
- Otherwise, if no leader is flushing, the caller becomes the LEADER: it flushes the buffer and fsyncs once, covering its own and every other buffered committer's frames, publishes the new durableSize, and wakes the followers. If a leader is already flushing, the caller waits on the group condition until durableSize covers its watermark or the writer poisons.
Durability, atomicity, and failure semantics ¶
- DURABILITY: success is returned only after the fsync covering the caller's marker completes. Because all appends serialise (the buffer is FIFO and O_APPEND lands every write at EOF), a marker whose end offset is <= the flushed appendedSize is made durable by that flush's fsync; there is no prefix-only fsync on a local file system.
- ATOMICITY: the on-disk frame stream is unchanged from the per-commit path — each transaction's ops are contiguous and followed by its OpCommit marker — so a crash mid-leader-fsync recovers each fully-marked transaction and discards the unmarked tail exactly as before.
- FAIL-ALL: if the leader's flush or fsync fails, [Writer.poison] discards the entire un-synced suffix (every group member's frames and markers) and broadcasts; every waiter then observes the sticky error and fails its own commit. No member may believe it committed when the shared fsync failed.
Cancellation ¶
SyncGroup is intentionally NOT context-aware. Once a committer's frames are in the shared buffer they cannot be un-appended (later committers' frames sit after them, and the transaction sequence is consumed), so abandoning the wait while the group still fsyncs the frames would make the transaction durable — recovery replays a fully-marked transaction — while returning an error to the caller, risking a double apply on retry. The caller's deadline is honoured earlier, at the cancellable single-writer acquire ([Store.BeginCtx]); after the append point the commit is in-flight-durable and the wait for its covering fsync runs to completion (the marker either becomes durable or the writer poisons and fails it). This matches PostgreSQL: a backend cannot un-write WAL it has already inserted.
Concurrency: safe for concurrent calls; it serialises on the same internal mutex as Append/Sync and guarantees a single leader per fsync round.
func (*Writer) Truncate ¶
Truncate empties the WAL: flushes any buffered frames, truncates the underlying file to zero bytes, and fsyncs the result so the empty state is durable on disk before returning. Subsequent Writer.Append calls write to offset 0 of the freshly-empty file.
Truncate is intended to be called by the checkpointer after a snapshot covering all WAL frames has been durably persisted; on success every frame previously durable in the WAL is logically folded into the snapshot.
Lifetime counters in Writer.Stats are NOT reset; the returned int64 reports the number of bytes that were in the file at the moment of truncation (after the in-memory buffer was flushed), which is the canonical measure of WAL bytes freed by this call.
On error the WAL may be in a partially-truncated state; callers should not continue using the Writer.
func (*Writer) TruncatePrefix ¶ added in v0.3.1
TruncatePrefix crash-safely discards the WAL bytes in [0, upTo) and preserves every byte in [upTo, end) — the frames committed after the watermark a checkpoint captured. It is the WAL-prefix-reclamation primitive a non-blocking checkpoint uses: the snapshot folds the prefix [0, upTo), the suffix [upTo, end) holds transactions committed concurrently while the snapshot was written lock-free, and recovery replays that surviving suffix on top of the self-sufficient snapshot.
Why a copy-and-rename, not an in-place rewrite ¶
The WAL is opened O_APPEND, so writes cannot be repositioned; more importantly, rewriting the file in place (truncate to zero, then write the suffix back) has a fatal crash window: a crash after the truncate but before the suffix is rewritten leaves the suffix — committed transactions NOT present in the snapshot — permanently lost, a Durability violation. Instead TruncatePrefix writes the surviving suffix to a sibling temp file, fsyncs it, then atomically renames it over the WAL path and fsyncs the parent directory. rename(2) is atomic: a crash before the rename leaves the original full WAL intact (recovery = snapshot + full replay); a crash after it leaves the suffix-only WAL (recovery = snapshot + suffix replay); both reconstruct the exact committed state. This mirrors the file-granularity WAL reclamation of RocksDB (delete whole log files below the flush point) and PostgreSQL (recycle whole segments below the redo LSN), adapted to GoGraph's single un-segmented WAL file.
Contract and ordering ¶
The caller MUST hold the store's quiesce boundary ([txn.Store.RunUnderCommitLock]) so no concurrent Writer.Append races the rename or the durableSize/appendedSize/buffer reset, and MUST have made the covering snapshot fully durable (data fsync + snapshot publish + parent-dir fsync) BEFORE calling this. upTo must be a value previously returned by Writer.DurableOffset (a frame boundary) and must satisfy 0 <= upTo <= durableSize; an out-of-range upTo is rejected without touching the file. upTo == 0 is a no-op (nothing to reclaim).
On success the returned int64 is the number of bytes reclaimed (upTo) and the Writer continues against the suffix-only file. On a path-less Writer (OpenWith) it returns ErrPrefixTruncateUnsupported.
Error handling splits on the atomic rename:
- A failure BEFORE the rename (suffix copy, or the rename itself) leaves the ORIGINAL full WAL intact and the Writer usable; the error is returned and the caller may retry the checkpoint — the prefix is still present, so nothing is lost.
- A failure AFTER the rename (parent-dir fsync, or the reopen of the new inode) cannot be undone: the on-disk state has already advanced to the durable suffix-only WAL and the old inode is unlinked. The Writer is therefore POISONED (fail-stop): the error is returned and every subsequent Append/Sync returns it, so the owner must discard the Writer and re-open the WAL, which re-validates the already-correct on-disk suffix. The committed data is safe on disk; only the in-memory handle is abandoned.
Source Files