Documentation
¶
Overview ¶
Package writer is the single-writer goroutine described in ARCHITECTURE § 7 and § 8. It appends JSONL and mirrors traces into SQLite with session inference inside the same goroutine.
Daily rotation per § 7.4: file date is decided by *write time*, not by trace.TsStart. A trace whose ts_start is yesterday but whose finalize happens after midnight lands in today's file. Consumers reconcile via ts_start if they care about request-arrival grouping.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type AppendResult ¶
type AppendResult struct {
JSONLPath string // e.g. "data/2026-05-27/a1b2c3d4.jsonl"
JSONLOffset int64 // pre-write byte offset where the line starts
Err error
}
AppendResult is returned via Record.Reply (if non-nil) after the line is written. M3 uses Offset to populate SQLite's jsonl_offset column.
type Record ¶
type Record struct {
Trace trace.Trace
KeyHash string // 16-hex full key_hash; writer uses KeyHashShort for filename
}
Record is one item on the writer channel. It carries the trace plus a key_hash for file naming. We compute key_hash here (not in the trace struct) because it's not part of the JSONL schema — it's only used by the writer to pick the file, and by M3's SQLite mirror.
type Stats ¶
type Stats struct {
DropWriterFull int64 // TrySend dropped because channel was full
DropJSONLFail int64 // JSONL append failed (disk full, EIO, etc.)
DropSQLiteFail int64 // SQLite upsert failed (busy timeout, schema err, …)
Appended int64 // successful JSONL appends
Indexed int64 // successful SQLite upserts (≤ Appended)
}
Stats are the cumulative drop / overflow counters this writer exposes.
type Writer ¶
type Writer struct {
// contains filtered or unexported fields
}
Writer owns the JSONL writing pipeline. Exactly one goroutine writes to disk; producers send Records on a bounded channel via TrySend.
func New ¶
func New(dataDir string, chanCap int, store *sqlite.Store, ctrs *counters.Counters, mediaExt *media.Extractor, mediaEnabled *atomic.Bool, coord *storage.Coordinator, clock func() time.Time) *Writer
New creates a Writer with the given chan capacity. Call Start to launch its goroutine. Use TrySend to enqueue records. Call the stop fn (from Start) on shutdown to drain the channel and wait for background gzip workers.
store may be nil; tests and pure-JSONL deployments leave it nil. ctrs may be nil; counters skipped if so. clock may be nil; defaults to time.Now.
mediaExt and mediaEnabled (Phase K) may both be nil; either nil means "no extraction." When both are non-nil, the writer reads mediaEnabled on each finalize and, if true, invokes mediaExt.Extract on the trace after the JSONL line is on disk. Extraction failures log WARN inside the extractor and never block the writer or affect MediaCount past the returned slice length (capture should never interfere).
coord (v0.1.1) is the storage coordinator. nil disables lease arbitration entirely — the writer opens/closes files without notifying retention. Production main wires a non-nil coord whose DataDir matches the writer's dataDir; the coord-side eviction loop can then safely interleave with appends.
func (*Writer) SetIdleTimeout ¶ added in v0.1.1
SetIdleTimeout configures the idle-close threshold. Must be called BEFORE Start. Zero (the default) disables idle-close entirely. Non-zero values are clamped at the sweep level: the writer ticks at idleTimeout / 2 (or once a minute, whichever is shorter) and closes any handle whose last append is at least idleTimeout ago.
func (*Writer) SnapshotStats ¶
SnapshotStats returns a copy of the current counters. Safe for concurrent callers (M4's /healthz will use this).
Source Files