Documentation
¶
Overview ¶
Package wal is the write-ahead log: CRC-framed records appended to numbered segment files, replayed in order to reconstruct the in-memory index after a crash. A record frames a length, a body (type + payload) and a CRC32C of the body; on replay a truncated or torn final record ends the log cleanly (crash recovery), while a complete record with a bad CRC is surfaced as corruption. Append-style and group-commit-friendly.
Index ¶
- Constants
- Variables
- func Replay(data []byte, h Handlers) error
- func ReplayDir(dir string, h Handlers) error
- type Handlers
- type SegmentWriter
- func (sw *SegmentWriter) Checkpoint() error
- func (sw *SegmentWriter) Close() error
- func (sw *SegmentWriter) Sync() error
- func (sw *SegmentWriter) Write(p []byte) (int, error)
- func (sw *SegmentWriter) WriteRecords(id signal.SeriesID, payload []byte) error
- func (sw *SegmentWriter) WriteSamples(id signal.SeriesID, ts []int64, values []float64) error
- func (sw *SegmentWriter) WriteSeries(id signal.SeriesID, s signal.Series) error
- func (sw *SegmentWriter) WriteSide(payload []byte) error
- type Writer
Constants ¶
const ( // DefaultMaxSegmentBytes is the size at which the writer rotates to a new segment. DefaultMaxSegmentBytes = 32 << 20 // 32 MiB )
Variables ¶
var ErrCorrupt = errors.New("wal: corrupt record")
ErrCorrupt is returned when a complete record fails its CRC check.
Functions ¶
func Replay ¶
Replay reads every complete record from data and dispatches it to h. It stops cleanly at end-of-log or a torn final record (returning nil), and returns an ErrCorrupt-wrapping error on a complete record whose CRC fails. Records already applied before the stopping point are kept.
Types ¶
type Handlers ¶
type Handlers struct {
OnSeries func(id signal.SeriesID, s signal.Series) error
OnSamples func(id signal.SeriesID, ts []int64, values []float64) error
OnRecords func(id signal.SeriesID, payload []byte) error
OnSide func(payload []byte) error
}
Handlers receives decoded records during Replay. Unset handlers skip their record type.
type SegmentWriter ¶
type SegmentWriter struct {
// contains filtered or unexported fields
}
SegmentWriter appends WAL records to numbered segment files in a directory, rotating to a fresh segment once the current one reaches the size limit. Replaying the directory in order (ReplayDir) reconstructs the logged state. Not safe for concurrent use.
func Create ¶
func Create(dir string, maxBytes int) (*SegmentWriter, error)
Create opens (creating the directory if needed) a segmented WAL writer. A non-positive maxBytes uses DefaultMaxSegmentBytes. If the directory already holds segments from a prior run, Create **resumes**: it opens a fresh segment numbered beyond the existing ones (never truncating them), so ReplayDir can still recover the prior segments before the next SegmentWriter.Checkpoint discards them.
func (*SegmentWriter) Checkpoint ¶ added in v0.2.0
func (sw *SegmentWriter) Checkpoint() error
Checkpoint discards every segment written so far and starts a fresh one. Call it after a full head flush, whose part durably supersedes those records. It rotates to a new segment, then best-effort deletes the obsolete ones. (Part commit and this deletion are not atomic across the two stores: a crash in between can leave a deleted-part's records to be replayed, a benign re-flush — deduped on merge for metrics, a rare duplicate for append-only records.)
func (*SegmentWriter) Close ¶
func (sw *SegmentWriter) Close() error
Close syncs and closes the current segment.
func (*SegmentWriter) Sync ¶
func (sw *SegmentWriter) Sync() error
Sync flushes the current segment to stable storage.
func (*SegmentWriter) Write ¶
func (sw *SegmentWriter) Write(p []byte) (int, error)
Write implements io.Writer, appending to the current segment and tracking its size so the writer knows when to rotate.
func (*SegmentWriter) WriteRecords ¶ added in v0.2.0
func (sw *SegmentWriter) WriteRecords(id signal.SeriesID, payload []byte) error
WriteRecords logs a stream's opaque engine-encoded record payload, rotating first if the current segment is full.
func (*SegmentWriter) WriteSamples ¶
WriteSamples logs a run of samples for one series, rotating first if the current segment is full.
func (*SegmentWriter) WriteSeries ¶
WriteSeries logs a series registration, rotating to a new segment first if the current one is full.
func (*SegmentWriter) WriteSide ¶ added in v0.2.0
func (sw *SegmentWriter) WriteSide(payload []byte) error
WriteSide logs an opaque engine-encoded side-store delta, rotating first if the current segment is full.
type Writer ¶
type Writer struct {
// contains filtered or unexported fields
}
Writer appends framed records to an io.Writer (typically a segment file). It reuses internal buffers, so it is not safe for concurrent use.
func (*Writer) WriteRecords ¶ added in v0.2.0
WriteRecords logs a run of records for one stream: its signal.SeriesID then an opaque, engine-encoded payload (the record engine owns the column encoding, keyed by its schema; the WAL stays signal-agnostic). Replaying it hands the payload back to the engine via Handlers.OnRecords.
func (*Writer) WriteSamples ¶
WriteSamples logs a run of samples for one series: its signal.SeriesID then the (timestamp, value) pairs. ts and values must have the same length.
func (*Writer) WriteSeries ¶
WriteSeries logs a series registration: its signal.SeriesID and full identity (Resource + Scope + data-point attributes). Replaying it reconstructs the series.
Source Files