Documentation
¶
Overview ¶
Package readstate owns the per-space read/unread engine: the unread entries, the per-object seen-heads frontier and per-tag counters. See docs/read-tracking-proposal.md.
The engine persists two space-shared collections (one set per DB, like _meta):
- _read_unread — one row per unread change, deleted when read. A row's stateSeq is the apply seq at which the change became unread here; UnreadSince pages the space's rows on it.
- _read_state — one row per tracked object: frontier heads, pending remote heads, per-tag counters, and the object's last stateSeq — the dirty-object feed (ChangedSince) reads THIS table; there is no transitions log. A consumer re-pulls the object's unread snapshot and diffs against what it holds.
All mutating methods take the caller's tx context: the apply path calls Track inside the same WriteTx as the record mutations, so an unread entry is atomic with its change; marks and merges open their own tx at the service layer. Open creates the collections once, before the store is published; after that the engine holds no locks and no in-memory state — callers serialize per object (the apply path via the tree lock, merges/marks via the service's per-object mutex).
Read marking is forward-only: MarkRead covers the given changes and their causal ancestry. The ancestor walk runs over unread rows (each row carries its change's prevIds) and traverses gaps — read, self-authored, or untracked changes between unread ones — through the Resolver, pruning where a walked change's versionId drops below the object's minimum unread versionId (an ancestor's versionId is always smaller than its descendant's, so nothing unread can hide below that line). MarkReadUpTo never walks: range coverage by versionId is gap-immune.
Index ¶
- Constants
- Variables
- func MaxStateSeq(ctx context.Context, db anystore.DB, spaceId string) (uint64, error)
- func PurgeObject(ctx context.Context, db anystore.DB, objectId string) error
- func PurgeSpace(ctx context.Context, db anystore.DB, spaceId string) (purged int, err error)
- type Engine
- func (e *Engine) ChangedSince(ctx context.Context, since uint64, limit int) ([]ObjectState, error)
- func (e *Engine) ClearRecords(ctx context.Context, objectId string, recordIds []string, stateSeq uint64) error
- func (e *Engine) Counts(ctx context.Context, objectId string) (map[string]int, error)
- func (e *Engine) Frontier(ctx context.Context, objectId string) (heads, pending []string, err error)
- func (e *Engine) MarkRead(ctx context.Context, objectId string, changeIds []string) (MarkResult, error)
- func (e *Engine) MarkReadUpTo(ctx context.Context, objectId, upTo string) (MarkResult, error)
- func (e *Engine) MarkReadUpToChunk(ctx context.Context, objectId, upTo string, maxEntries int) (MarkResult, bool, error)
- func (e *Engine) MarkSeeded(ctx context.Context, objectId string) error
- func (e *Engine) MergeHeads(ctx context.Context, objectId string, heads []string) (MarkResult, error)
- func (e *Engine) NotifyState(objectId string, stateSeq uint64)
- func (e *Engine) Open(ctx context.Context) error
- func (e *Engine) SeedFrontier(ctx context.Context, objectId string, heads []string) (bool, error)
- func (e *Engine) Seeded(ctx context.Context, objectId string) (bool, error)
- func (e *Engine) SubscribeState(cb func(objectId string, stateSeq uint64)) (cancel func())
- func (e *Engine) TrackChange(ctx context.Context, t Track) error
- func (e *Engine) UnreadEntries(ctx context.Context, objectId string) ([]Entry, uint64, error)
- func (e *Engine) UnreadSince(ctx context.Context, since uint64, limit int) ([]Entry, error)
- func (e *Engine) WriteTx(ctx context.Context, fn func(txCtx context.Context) error) error
- type Entry
- type Loader
- type MarkResult
- type ObjectState
- type Resolver
- type SeqFunc
- type Track
Constants ¶
const ( UnreadCollectionName = "_read_unread" StateCollectionName = "_read_state" )
Variables ¶
var ErrNotOpen = errors.New("readstate: engine not opened")
ErrNotOpen is what the collection-backed methods return until Open ran; after a failed Open they return its error.
var ErrObjectDeleted = fmt.Errorf("readstate: %w", space.ErrObjectDeleted)
ErrObjectDeleted is what a Resolver returns for an object whose tree is deleted on this device. A mark or merge that meets it stops before persisting anything: the published frontiers of a purged object keep arriving (every device's reconcile replays them), and parking their heads as pending would re-create the state row the purge removed. Wraps space.ErrObjectDeleted for API callers.
var ErrSpaceNotResident = errors.New("readstate: space not resident")
ErrSpaceNotResident is the Resolver's refusal to answer from a space that is not loaded.
Functions ¶
func MaxStateSeq ¶ added in v0.6.0
MaxStateSeq returns the highest stateSeq persisted for spaceId, 0 when the space has no read-state rows (or the table does not exist yet). Read marks mint their stateSeq from the space's apply-sequence allocator and persist it only here, so the allocator's seed takes this into account next to the objects' apply watermarks.
func PurgeObject ¶ added in v0.6.0
PurgeObject removes objectId's read state — its unread rows and its state row — in the caller's ctx, so an object purge clears it in the same transaction as the object's row. Collections that were never created and rows that are already gone are not errors. Without this a purged object's row would outlive it: `sd` stays set, so a later materialization of the same id would skip first-sight seeding and keep the stale frontier. The object leaves the dirty feed with its row (ChangedSince reads state rows); the change-index deletion entry is the consumer's eviction signal.
func PurgeSpace ¶ added in v0.6.0
PurgeSpace removes every read-state row of spaceId: the state rows are read through the (space, stateSeq) index in chunks, and each chunk's objects lose their unread rows (object index) and state rows in one write transaction, so the sweep holds the writer for at most purgeSpaceChunk objects at a time and committed chunks survive a failure. The state rows enumerate the whole space — TrackChange persists an unread row and its object's state row together. Returns the number of objects purged and the first error; the rest is left for the next attempt. Space offload and the boot orphan sweep call it: a removed space's rows would otherwise stand in for the all-read seed if the space re-materializes.
Types ¶
type Engine ¶
type Engine struct {
// contains filtered or unexported fields
}
func New ¶
New builds the engine over the space's share of db. No I/O: Open creates the collections. A nil load leaves ErrSpaceNotResident to the caller.
func (*Engine) ChangedSince ¶
ChangedSince returns objects whose read state advanced past since, ascending by stateSeq, capped at limit (0 = no cap). One element per object (this reads the per-object state rows, not a log): the consumer re-pulls UnreadEntries for each dirty object and diffs against what it holds. Page by passing the last StateSeq back.
func (*Engine) ClearRecords ¶
func (e *Engine) ClearRecords(ctx context.Context, objectId string, recordIds []string, stateSeq uint64) error
ClearRecords drops deleted records from unread entries: an entry referencing only deleted records is removed (counters and feed updated); an entry also touching surviving records just sheds the deleted ids.
func (*Engine) Frontier ¶
func (e *Engine) Frontier(ctx context.Context, objectId string) (heads, pending []string, err error)
Frontier returns the object's current seen-heads frontier and pending remote heads.
func (*Engine) MarkRead ¶
func (e *Engine) MarkRead(ctx context.Context, objectId string, changeIds []string) (MarkResult, error)
MarkRead covers the given change ids and their causal ancestry. Ids not yet arrived on this device are persisted as pending heads. Runs in the caller's tx; the caller serializes per object.
func (*Engine) MarkReadUpTo ¶
MarkReadUpTo covers every unread change with versionId <= upTo ("" = everything). Range coverage needs no ancestry walk.
func (*Engine) MarkReadUpToChunk ¶
func (e *Engine) MarkReadUpToChunk(ctx context.Context, objectId, upTo string, maxEntries int) (MarkResult, bool, error)
MarkReadUpToChunk is MarkReadUpTo bounded to at most maxEntries covered entries (0 = unbounded); done=false means more remain below upTo. Covering a versionId prefix is a valid frontier advance (forward-only marking), so a caller can commit each chunk in its own tx and a crash mid-way just resumes — the huge-ReadAll path uses this to keep transactions bounded.
func (*Engine) MarkSeeded ¶
MarkSeeded records first-sight seeding as done WITHOUT touching entries or frontier — the consult-KV seed path uses it: the account's published frontiers, merged separately in the same tx, are the real seed. Idempotent.
func (*Engine) MergeHeads ¶
func (e *Engine) MergeHeads(ctx context.Context, objectId string, heads []string) (MarkResult, error)
MergeHeads applies another device's published frontier. Identical semantics to MarkRead: known ids cover their ancestry, unknown ids park as pending heads.
func (*Engine) NotifyState ¶
NotifyState fires the subscribed pings. Call AFTER the tx that produced stateSeq committed.
func (*Engine) Open ¶ added in v0.6.6
Open creates the collections and their indexes. The DDL takes any-store's write lock, so ctx must not carry a write tx and the caller must not hold one: the store build runs Open before the store is published, and every later method only reads and writes rows. A failed Open leaves the engine returning its error from every call until an Open succeeds; a no-op once it has.
func (*Engine) SeedFrontier ¶
SeedFrontier performs first-sight seeding: sets the frontier to the given heads (everything at or behind them is read), drops any unread entries that slipped in around the restore (their became- read transitions surface on the feed), preserves pending remote heads, and records the seed durably — all in the caller's tx. Idempotent: a second call on a seeded object is a no-op. Returns whether the seed ran.
func (*Engine) Seeded ¶
Seeded reports whether first-sight seeding already ran for the object. Durable: only SeedFrontier sets it, in the same tx as the seed itself, so the answer survives crashes on either side.
func (*Engine) SubscribeState ¶
SubscribeState registers a best-effort ping fired after a committed read-state change (new unread, mark, merge). cb runs synchronously on the notifying path — keep it small. Cancel is idempotent. A dropped ping is recovered by pulling ChangedSince from the consumer's cursor.
func (*Engine) TrackChange ¶
Track records one classified change from the apply path. Must run inside the apply's WriteTx (pass tx.Context()).
func (*Engine) UnreadEntries ¶
UnreadEntries returns the object's current unread set, ascending by versionId ("" upTo = all).
func (*Engine) UnreadSince ¶ added in v0.7.2
UnreadSince returns the space's unread entries whose stateSeq exceeds since, ascending by stateSeq, capped at limit (0 = no cap). Each entry has its own stateSeq, so paging on the last one skips nothing.
func (*Engine) WriteTx ¶
WriteTx runs fn inside a write transaction on the engine's DB — the tx opener for callers outside the apply path (marks, merges, reconcile). fn's error rolls the tx back and is returned.
A gap walk may need the space, and loading one can wait on an unreachable sync node. When fn fails with ErrSpaceNotResident the tx is rolled back, the Loader runs with the write lock released, and fn reruns once in a fresh tx.
type Entry ¶
type Entry struct {
ObjectId string
Dataset string
ChangeId string
VersionId string
AddSeq uint64
ApplySeq uint64
RecordIds []string
Tags []string
Key string
PrevIds []string
// StateSeq orders entries by when they became unread on this
// device.
StateSeq uint64
}
Entry is one unread change as persisted.
type MarkResult ¶
type MarkResult struct {
Removed []Entry
Frontier []string
// Pending are marked ids not yet arrived on this device; they are
// persisted and resolved when the change applies.
Pending []string
StateSeq uint64 // 0 when nothing changed
}
MarkResult reports what a mark/merge covered.
type ObjectState ¶
ObjectState is one dirty-object feed element: an object whose read state changed, at the stateSeq that change advanced it to.
type Resolver ¶
type Resolver func(ctx context.Context, objectId, changeId string) (prevIds []string, versionId string, ok bool, err error)
Resolver looks up a change the unread rows no longer (or never did) cover — the gap-traversal fallback backed by any-sync's change storage. ok=false means the change has not arrived on this device. It runs inside the caller's write tx, so it must not wait on the network: when the lookup needs the space loaded it fails with ErrSpaceNotResident, and WriteTx loads the space through the Loader with the lock released before rerunning the caller. The apply path runs on a resident space and has no retry: there the refusal fails the apply, which the next replay repeats.
type SeqFunc ¶
SeqFunc allocates the next per-space stateSeq. Wired to the space's ApplySeqAllocator so marks, merges, and applies share one monotonic cursor axis. Must be called with the tx held (allocation order = commit order).
type Track ¶
type Track struct {
ObjectId string
Dataset string
ChangeId string
VersionId string
AddSeq uint64
ApplySeq uint64 // stateSeq for the unread transition
RecordIds []string
Tags []string
Key string // supersede key; "" = none
PrevIds []string
// Tracked=false records no unread entry but still clears a
// superseded row (Key) and resolves a pending head.
Tracked bool
// SelfAuthored changes are born read and advance the frontier
// in place when their whole causal past is covered.
SelfAuthored bool
}
Track is the apply-path input: one tracked change, classified.