Documentation
¶
Overview ¶
Package state provides the durability core — an append-only Log and content-addressed Blobs — that the engine (Phase 2) sits on for commit/resume and that obs (Phase 6) reads to project OTel spans.
The Log is etcd-style framed (length + CRC32C + payload + 8-byte pad — see event.go); the fold scans from offset 0 and silently truncates on the first short read or CRC mismatch, which is the torn-tail recovery the design requires. The Blobs store is content-addressed (sha256 v1; the ref format carries the algorithm name so BLAKE3 can land additively).
Slice 1.5 ships the primitives; the engine in Phase 2 orchestrates them into the content-address-then-pointer-swap commit boundary (CLAUDE.md invariant: artifacts in Blobs first, then a `node.completed` event appended + fsync'd).
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ( // ErrBadRef wraps every refusal to parse a ref (wrong prefix, wrong length, non-hex, // wrong algorithm). Callers use errors.Is to distinguish from missing / corruption. ErrBadRef = errors.New("state: malformed blob ref") // ErrCorruption fires when Get reads a file whose recomputed sha256 doesn't match the // ref's address. Detects silent disk corruption. ErrCorruption = errors.New("state: blob content does not match its address") )
Functions ¶
Types ¶
type Blobs ¶
type Blobs interface {
Put(content []byte) (ref string, err error)
Get(ref string) ([]byte, error)
}
Blobs is the content-addressed artifact store seam. One concrete impl (FSBlobs) + one fake (InMemoryBlobs). Hash is hidden behind the ref string so a future BLAKE3 impl can land additively (refs would be "awf-d1:blake3:<hex>" and parse via the same prefix split).
type Event ¶
type Event struct {
Seq uint64 `json:"seq"`
Epoch uint32 `json:"epoch"`
TS time.Time `json:"ts"`
Path string `json:"path"`
Type string `json:"type"`
PayloadRef string `json:"payload_ref,omitempty"`
Data json.RawMessage `json:"data,omitempty"`
}
Event is the on-disk record per phase-1 design §D. The JSON shape is the wire format; the frame (encodeFrame/decodeFrame) is the bytes that land on disk.
Field semantics:
- Seq, Epoch, TS: assigned by the Log on Append (caller leaves them zero-valued).
- Path: opaque attribution — the engine's addressing path (Phase 2's engine/path package produces it). Slice 1.5 never parses it.
- Type: opaque event-kind name. Phase 2's engine defines the vocabulary (run.started, node.completed, branch.taken, …); slice 1.5 carries arbitrary strings.
- PayloadRef, Data: optional. PayloadRef is a content-addressed pointer into Blobs (large payloads live there); Data is inline structured payload for small events. Both omitempty.
func FoldFile ¶
FoldFile reads + decodes every committed event from the log at path WITHOUT taking a writable file handle. Use this when you need to observe a log that another process or goroutine may be actively writing to — unlike OpenLog, FoldFile never truncates a torn tail (the file is opened read-only). Torn-tail records at the tail are silently skipped per scanFile's semantics.
Single-writer discipline is preserved: only OpenLog/OpenLogExclusive take a write handle; FoldFile is observer-only.
type FSBlobs ¶
type FSBlobs struct {
// contains filtered or unexported fields
}
FSBlobs is the filesystem-backed CAS — one file per content address, sharded by the first two hex chars of the address.
func OpenBlobs ¶
OpenBlobs prepares the blob store rooted at the given directory. Creates the root directory (and the `sha256/` subdirectory) if they don't already exist.
func OpenBlobsReadOnly ¶ added in v0.6.0
OpenBlobsReadOnly returns a filesystem CAS handle without probing or preparing the store. Observers can therefore start against an empty state directory without creating blobs/sha256 as a side effect. A later Get performs the first filesystem access and naturally returns a wrapped fs.ErrNotExist when the requested blob (or the store itself) is absent.
func (*FSBlobs) Get ¶
Get reads the blob at ref and re-verifies its sha256 against the ref's address. Returns ErrBadRef for malformed refs, ErrCorruption for content/address mismatch, or a wrapped fs.ErrNotExist for refs the caller didn't Put.
type FileLog ¶
type FileLog struct {
// contains filtered or unexported fields
}
FileLog is the production impl — one append-only file per run. Live agent event sinks may append concurrently with interpreter commits.
func OpenLog ¶
OpenLog opens (or creates) the log file at path. On an existing file, it scans to the first short read or CRC mismatch and truncates the file there (torn-tail recovery), then initializes the next Seq to (max-seen + 1) and the Epoch to (max-seen + 1). On a fresh file, Seq starts at 0 (first Append gets Seq=1) and Epoch is 0.
OpenLog does NOT create intermediate directories — the parent must exist. (The engine in Phase 2 owns the run directory layout, so plumbing MkdirAll here would couple the primitive to a layout it doesn't own.)
func OpenLogExclusive ¶
OpenLogExclusive opens (or creates) the log file at path WITH O_EXCL — i.e. the open atomically fails if the file already exists. This is the race-free first-run primitive: `awf run` calls it to mint a new run.id's log, and the existing OpenLog (which tolerates and torn-tail-recovers an existing file) stays as the resume primitive.
On collision, the returned error wraps fs.ErrExist — callers route with errors.Is(err, fs.ErrExist) for the "run id already exists, use awf resume" message.
Unlike OpenLog, there's no torn-tail-recovery branch here: a fresh file can't have a torn tail. The function is intentionally minimal.
func OpenLogExisting ¶ added in v0.6.0
OpenLogExisting opens an existing log for append and performs the same scan/torn-tail repair as OpenLog, but never creates the file. Resume uses this only after it owns the run lock, ensuring that observers and losing resume contenders cannot mutate a log merely by trying to open it.
func (*FileLog) Append ¶
Append assigns Seq/Epoch/TS and writes the framed JSON record. Does NOT fsync — caller invokes Sync at durability-critical events.
func (*FileLog) Close ¶
Close fsyncs and closes. If Sync fails we still attempt Close (releasing the FD is more important than the trailing close-error), but the Sync error is what we return.
type InMemoryBlobs ¶
type InMemoryBlobs struct {
// contains filtered or unexported fields
}
InMemoryBlobs is the Blobs fake: a map from hex-hash to content. Matches FSBlobs' error contract (ErrBadRef for malformed, wrapped fs.ErrNotExist for missing, ErrCorruption never fires in-memory because content can't be tampered with). Safe for concurrent Put/Get — a mutex serializes store access so parallel branch goroutines (e.g. T10 fan-out) can commit concurrently without racing.
func NewInMemoryBlobs ¶
func NewInMemoryBlobs() *InMemoryBlobs
NewInMemoryBlobs mints an empty in-memory blob store.
func (*InMemoryBlobs) ClearFault ¶
func (b *InMemoryBlobs) ClearFault()
ClearFault resets the FailPutAfterN hook. See InMemoryLog.ClearFault.
func (*InMemoryBlobs) FailPutAfterN ¶
func (b *InMemoryBlobs) FailPutAfterN(n int)
FailPutAfterN — see InMemoryLog.FailAppendAfterN.
type InMemoryLog ¶
type InMemoryLog struct {
// contains filtered or unexported fields
}
InMemoryLog is the Log fake: events live in a slice, Sync/Close are no-ops. Used by Phase 2's engine tests (the fake backend's conformance suite).
func NewInMemoryLog ¶
func NewInMemoryLog(clk clock.Clock) *InMemoryLog
NewInMemoryLog mints a fresh in-memory log. The optional clk must be non-nil; pass clock.System{} for "real" time or a clock.Fake for deterministic tests.
func (*InMemoryLog) Append ¶
func (l *InMemoryLog) Append(e Event) error
func (*InMemoryLog) ClearFault ¶
func (l *InMemoryLog) ClearFault()
ClearFault resets the FailAppendAfterN hook to "no fault programmed." Idempotent on already-cleared. Conformance harness calls this before the resume so a programmed crash doesn't replay on the resume's own Appends.
func (*InMemoryLog) Close ¶
func (*InMemoryLog) Close() error
func (*InMemoryLog) FailAppendAfterN ¶
func (l *InMemoryLog) FailAppendAfterN(n int)
FailAppendAfterN configures the fake so the first n Appends succeed and the (n+1)-th fails with an "induced fault" error. FailAppendAfterN(0) fails the very first call. One-shot: call #(n+1) and beyond succeed normally.
Mirrors container.Fake.FailExecAfterN's semantics; lets slice 2.4's commit test crash *between* Blobs.Put and Log.Append for a chosen attempt.
func (*InMemoryLog) Fold ¶
func (l *InMemoryLog) Fold() ([]Event, error)
func (*InMemoryLog) Reopen ¶
func (l *InMemoryLog) Reopen() error
Reopen simulates a FileLog.OpenLog of an existing file: bumps the internal epoch counter to (last-event.Epoch + 1) so subsequent Appends carry the new epoch. On an empty log (no events) Reopen is a no-op — matches FileLog's "fresh file → epoch=0" semantic.
Conformance harness's crash-then-resume choreography calls this between the simulated crash (programmed via Fail*AfterN) and the resume's first Append (run.resumed). Mirrors what production FileLog.OpenLog does for free on an existing file; the fake doesn't get that for free because it doesn't open files.
Returns error for signature symmetry with FileLog.OpenLog (which can fail on I/O); the in-mem impl can't actually fail, but callers shouldn't rely on that — Phase 4 Docker conformance may swap in a file-backed log where the signature matters.
func (*InMemoryLog) Sync ¶
func (*InMemoryLog) Sync() error
type Log ¶
Log is the durability seam. One concrete impl (FileLog) + one fake (InMemoryLog).
- Append assigns Seq (monotonic across reopens), Epoch (incremented on each open of an existing log), and TS (from the injected Clock); the caller fills Type, Path, PayloadRef, Data.
- Sync fsyncs the file; caller decides per-event criticality (the engine in Phase 2 calls Sync on node.completed, run.*, signal.received — the design's durability-critical events — and lets high-frequency io.chunk/agent.event ride the next fsync).
- Fold rescans the file and returns the full event sequence. No internal cache (the access pattern is dominated by Append; Fold is called on resume / for debugging).
- Close fsyncs and releases the file handle.
Source Files