Documentation
¶
Overview ¶
Package shardedlog is a generic append-only JSONL store split across date-bucketed shards. Designed for "give me the latest N rows" queries without scanning every per-row directory — useful for any feature that wants a paginated history view (workflow runs, agent sessions, future log mirrors, …).
Layout on disk:
<Dir>/ 2026-05-15-01.jsonl (≤ ShardMax rows) 2026-05-15-02.jsonl 2026-05-16-01.jsonl
Newest rows sit at the END of the latest shard. Reading newest- first = list shards descending + scan each shard from bottom up.
Cost is bounded by ShardMax regardless of total history size: each page request reads one shard (~100 rows * ~100 bytes = ~10KB). Append cost is also bounded — we touch the latest shard only.
Index ¶
Constants ¶
const DefaultShardMax = 100
DefaultShardMax is the per-shard row cap. Each page request reads a single shard so this directly bounds scan cost.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Store ¶
type Store[T any] struct { // Dir is the on-disk directory holding shard files. Created on // first Append if it doesn't exist. Dir string // ShardMax bounds rows per shard file. Defaults to DefaultShardMax. ShardMax int // Now overrides time.Now for tests so the per-date bucket name // is deterministic. nil = time.Now().UTC(). Now func() time.Time // contains filtered or unexported fields }
Store is a sharded append-only JSONL log. Callers create one instance per logical stream (e.g. per workflow id, per session id). Concurrent appends to the same Store are serialised via the internal mutex.
Zero-value works except Dir is required.
func (*Store[T]) Append ¶
Append writes one entry to the active shard, rolling to the next shard when the current one hits ShardMax rows. Serial — safe for concurrent callers via the per-Store mutex.
Source Files