persistence

package
v0.1.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 13, 2026 License: MIT Imports: 14 Imported by: 0

Documentation

Overview

Package persistence provides checkpoint flushing, batched writes, and crash repair, following upstream session persistence and JSONL storage. Persistence abstracts locate/load/append/list/snapshot operations. JSONLBackend appends events and supports checkpoints. H02 shards session IDs across independent locks and queues; background writers flush at size or time thresholds to reduce file operations. Crash repair closes unmatched turns as interrupted while preserving persisted chunks and tool events.

Index

Constants

View Source
const (
	// DefaultShardCount is a power of two for efficient indexing.
	// WithShardCount can increase it for workloads with many concurrent sessions.
	DefaultShardCount = 16
	// DefaultFlushInterval sets the asynchronous flush window in milliseconds.
	// Each tick flushes all buffered sessions in the shard.
	DefaultFlushInterval = 100 * time.Millisecond
)

Variables

This section is empty.

Functions

func ResetJSONLIOStats

func ResetJSONLIOStats()

ResetJSONLIOStats resets counters for tests and benchmarks, not normal production use.

Types

type JSONLBackend

type JSONLBackend struct {
	// contains filtered or unexported fields
}

JSONLBackend persists events in JSONL files. dir is the storage root; batchSize and flushInterval trigger flushing by count or time. Session hashes select a fixed shard using shardMask, requiring a power-of-two shard count. flushCh signals shard writers; closeOnce/closeCh coordinate shutdown, and wg lets Close wait until all background writers exit.

func NewJSONL

func NewJSONL(dir string, batchSize int, opts ...Option) (*JSONLBackend, error)

NewJSONL creates the backend and its directory if needed. A session flushes when its buffered count reaches batchSize or the time window expires. Nonpositive batchSize defaults to one. Options override DefaultShardCount and DefaultFlushInterval.

func (*JSONLBackend) Append

Append buffers an event in its session's shard and signals an immediate flush at batchSize. Disk writes run asynchronously in the shard writer.

func (*JSONLBackend) Close

func (j *JSONLBackend) Close() error

Close stops background writers, flushes remaining buffers, and waits for exit. Repeated calls are safe. Append after Close is unsupported.

func (*JSONLBackend) Flush

func (j *JSONLBackend) Flush(ctx context.Context, id brand.SessionID) error

Flush synchronously persists a session's buffered events. It takes ownership of the buffer and writes under writeMu, serialized with background flushing. Callers typically use it at turn/end checkpoints for durability.

func (*JSONLBackend) FlushAll

func (j *JSONLBackend) FlushAll(ctx context.Context) error

FlushAll synchronously flushes each shard in order, typically before exit or in tests.

func (*JSONLBackend) List

func (j *JSONLBackend) List(ctx context.Context) ([]brand.SessionID, error)

List returns all session directories.

func (*JSONLBackend) Load

Load reads the header and events and repairs interrupted sessions. It reads disk rather than pending buffers; call Flush or FlushAll first to observe recent appends. Historical reads need no preliminary flush.

func (*JSONLBackend) Locate

func (j *JSONLBackend) Locate(ctx context.Context, id brand.SessionID) (string, error)

Locate implements Persistence.

func (*JSONLBackend) SaveHeader

func (j *JSONLBackend) SaveHeader(ctx context.Context, h *session.SessionHeader) error

SaveHeader writes or replaces the session header.

func (*JSONLBackend) ShardCount

func (j *JSONLBackend) ShardCount() int

ShardCount returns the actual shard count for tests and monitoring.

func (*JSONLBackend) Snapshot

func (j *JSONLBackend) Snapshot(ctx context.Context, id brand.SessionID) (int, error)

Snapshot returns the complete event count after synchronously flushing the session.

type JSONLIOStats

type JSONLIOStats struct {
	PooledBufferHits     uint64 `json:"pooledBufferHits"`
	MarshaledBytes       uint64 `json:"marshaledBytes"`
	MarshaledEvents      uint64 `json:"marshaledEvents"`
	HeaderMarshaledBytes uint64 `json:"headerMarshaledBytes"`
}

JSONLIOStats is a thread-safe snapshot of H05 performance counters.

func ReadJSONLIOStats

func ReadJSONLIOStats() JSONLIOStats

ReadJSONLIOStats returns the current H05 counters.

type Option

type Option func(*jsonlOptions)

Option configures JSONLBackend through functional options.

func WithFlushInterval

func WithFlushInterval(d time.Duration) Option

WithFlushInterval sets the asynchronous flush window.

func WithShardCount

func WithShardCount(n int) Option

WithShardCount sets the shard count, rounding up to a power of two.

type Persistence

type Persistence interface {
	// Locate returns the persistent session path.
	Locate(ctx context.Context, id brand.SessionID) (string, error)
	// Load reads the session header and all events.
	Load(ctx context.Context, id brand.SessionID) (*session.SessionHeader, []session.SessionEvent, error)
	// Append buffers an event for the batch window.
	Append(ctx context.Context, id brand.SessionID, ev session.SessionEvent) error
	// Flush writes buffered events to disk as a checkpoint.
	Flush(ctx context.Context, id brand.SessionID) error
	// Snapshot returns the current event count for a session.
	Snapshot(ctx context.Context, id brand.SessionID) (int, error)
	// List returns all persisted session IDs.
	List(ctx context.Context) ([]brand.SessionID, error)
}

Persistence abstracts session persistence backends.

Directories

Path Synopsis
Package sqlite implements SQLite session persistence and FTS5 indexing (S03), following upstream storage/sqlite-session.
Package sqlite implements SQLite session persistence and FTS5 indexing (S03), following upstream storage/sqlite-session.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL