subscribe

package
v0.7.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 10, 2026 License: MIT Imports: 13 Imported by: 0

Documentation

Overview

Package subscribe owns the per-space live-query engine and the projected Event types the apply path hands to it. One Engine per loaded space; afterApply builds an Event via BuildEvent and calls Engine.OnApply, which routes each event to matching querySubs whose windows are maintained incrementally.

Two subscription scopes:

  • Shared-objects: fires on every event whose dataset is the per-space `objects` collection. Reached via Space.QueryObjects().
  • Per-(objectId, dataset): fires only on exact matches.

Per-sub window: entries map[string]*entry plus minRef/maxRef pointers for O(1) boundary access. When Limit > 0 the engine holds limit+1 entries, each with its sort tuple and last doc encoded, so the largest-tuple entry serves as a sentinel; single-event shifts (top-of-sort arrivals) absorb cleanly without an any-store query. When Limit == 0 there is no boundary and an entry is membership only.

A window at offset 0 with matches past its edge refills from storage (SubConfig.Refill) when a row leaves it, a row's rank becomes unknown, or a change's row cannot be read; the refill reads through Event.Source and looks up dropped rows for their reason. A refill deferred by an object's rebuild counts nothing, and Engine.Resync re-reads the window afterwards; a failed refill counts toward the drift budget (see settle), and the sub closes with space.ErrSubscriptionDrifted at DriftBudgetPercent of limit. Mailbox overflow closes with space.ErrSubscriptionOverflow. Both signal "resubscribe".

Events: the change's own event ships, under its VersionId, each touched row's transition from before the apply to after the records; the refill's event, with no VersionId and no ops, the transition from there to now.

Locking: a single engine.mu serializes register / close / OnApply. No per-sub locks. Holding engine.mu during the initial snapshot fences the apply path so the new sub never misses an event.

Index

Constants

View Source
const ObjectsDataset = properties.Dataset

ObjectsDataset is the CRDT dataset whose changes feed the shared-objects scope — the per-space `objects` collection, where every object's property values land regardless of object kind.

User-facing terminology calls these "property values"; the dataset is named "objects" because each row is one object's values record. Unrelated to typetype.DatasetPropertyDefs (type-definition metadata on type objects).

Variables

View Source
var ErrEngineClosed = errors.New("subscribe: engine closed")

ErrEngineClosed is returned by Subscribe after Close.

View Source
var ErrLimitWithoutSort = errors.New("subscribe: Limit > 0 requires Sort")

ErrLimitWithoutSort fires when a caller asks for Limit > 0 without supplying a Sort — the engine needs a total order to define "boundary" / "sentinel". Limit == 0 (unbounded) is fine without Sort.

View Source
var ErrRefillDeferred = errors.New("subscribe: refill deferred: storage is not final")

ErrRefillDeferred marks a read of rows an object is rebuilding: the engine counts nothing and re-reads the window at Resync.

Functions

This section is empty.

Types

type Engine

type Engine struct {
	// contains filtered or unexported fields
}

Engine is the per-space live-query engine. Cheap when no subs are registered: HasSubscribers is a single atomic load and the afterApply hook short-circuits on it.

func New

func New(spaceId string) *Engine

New constructs an empty Engine for spaceId.

func (*Engine) Close

func (e *Engine) Close() error

Close cancels every live sub. Idempotent.

func (*Engine) CloseObject added in v0.6.0

func (e *Engine) CloseObject(objectId string, reason error)

CloseObject closes every per-object subscription on objectId, across its datasets, with reason as the close error: the object was purged, so nothing will ever reach those subscriptions again. Runs after NotifyDeleted, so a Removed batch precedes the close. Shared and per-dataset subscriptions are untouched. The held set goes now; like an overflow close, the sub stays registered until the consumer's Close releases it. reason must be a plain sentinel (the close reason is an atomic value of one concrete type).

func (*Engine) HasSubscribers

func (e *Engine) HasSubscribers() bool

HasSubscribers reports whether any sub is live. Single atomic load — call before paying the cost of building the wire Event in afterApply.

func (*Engine) NotifyDeleted

func (e *Engine) NotifyDeleted(spaceId, dataset, objectId string)

NotifyDeleted emits a synthetic delete event for objectId in the given dataset scope, so live subscriptions drop the row and report RemoveDeleted — without an underlying CRDT apply.

The delete-callback path (spaceobjects.Store.DeleteTree, fired when any-sync catches a settings-tree deletion) uses this: it removes the materialized row directly, device-local, instead of writing a CRDT tombstone into the tree that is being reclaimed. There is no post-value lookup: a deleted record carries no post-apply doc, and applyRecord never reads one for a Deleted record.

The emitted SubscriptionEvent carries an empty VersionId (there is no underlying DAG change to draw an orderId from). A Removed{RemoveDeleted} therefore cannot be version-fenced — consumers must not drop a Removed via the VersionId replay fence; act on it directly.

func (*Engine) NotifyRecordsDeleted added in v0.5.0

func (e *Engine) NotifyRecordsDeleted(spaceId, dataset, objectId string, ids []string)

NotifyRecordsDeleted emits a synthetic delete for records ids of (objectId, dataset): the rows a purge removed from a shared dataset's collection. Like NotifyDeleted, the event carries no VersionId.

func (*Engine) OnApply

func (e *Engine) OnApply(ev Event, postValue PostValueFn)

OnApply is the apply-path hook. afterApply calls this with the same Event built once for the dispatcher, plus the PostValueFn that lets the engine evaluate filter / sort against the post-apply doc. A nil postValue serves an event of deleted records only; for any other record it reads as an error.

Cheap when there are no subs (HasSubscribers check returns false before this is even invoked by the caller). When subs exist, takes engine.mu briefly to fan to the matching ones.

func (*Engine) Resync added in v0.7.0

func (e *Engine) Resync(objectId string, source any)

Resync re-reads the windows an object's rebuild may have left behind: every window on objectId (one that cannot re-read closes), and every shared window that was deferred or holds the object's row, the latter by one lookup when the window is unbounded or past an offset.

func (*Engine) SpaceId

func (e *Engine) SpaceId() string

SpaceId returns the id of the space this engine serves.

func (*Engine) Subscribe

func (e *Engine) Subscribe(cfg SubConfig, snapshot SnapshotFn) (*Sub, error)

Subscribe registers a new querySub. The snapshot callback runs under engine.mu so the apply path is fenced between the snapshot read and the sub's registration — no events are missed and no events fire against an unregistered sub. The sub counts from before its snapshot: the apply path gates on HasSubscribers without the lock, and an apply that commits during the read must build its event and queue on the fence rather than skip the engine.

Returns space.ErrSubscribeUnsupported when the engine is closed. Returns ErrLimitWithoutSort when cfg.Limit > 0 and cfg.Sort is nil.

type Event

type Event struct {
	SpaceId   string
	ObjectId  string
	Dataset   string
	VersionId crdt.VersionId
	Records   []EventRecord
	// Source is the object the apply ran on, handed to a refill as is;
	// nil for a notification.
	Source any
}

Event is a single CRDT apply event handed to the engine.

Pipeline contract: receive change → apply to CRDT → commit the any-store tx → THEN build this event. By the time a consumer sees an Event, every projected $set/$unset is already durable in the controller's any-store; a Query against the same dataset run from the same process reflects the same state.

Carries the routing tuple plus enough about the change for the engine to classify each record (filter + sort + window membership) without re-querying:

  • VersionId — per-change DAG order. Forwarded on the wire as SubscriptionEvent.VersionId so fence-and-replay consumers can dedupe across snapshots.

  • Records — the post-apply effect of the change, projected to a flat list of $set / $unset ops per record. Ops payloads are deep-cloned off the controller's storage onto event-owned arenas, so an Event is safe to retain past the build call.

func BuildEvent

func BuildEvent(ch *crdt.Change, recordIds []string, derivedOps [][]crdt.Op, postValue PostValueFn) Event

BuildEvent projects (ch, recordIds, derivedOps, postValue) into the wire Event shape — runs the same per-record projection an engine caller would need, with deep-cloned op payloads. afterApplyFor builds this once per change before handing it to Engine.OnApply.

Safe to call with a nil change — returns a zero Event in that case; callers should check ch != nil themselves if they want to skip work.

type EventRecord

type EventRecord struct {
	Id string
	// Created is true when this change first materialised the record.
	// The engine does not read it: a consumer tells a creation from an
	// update by it.
	Created bool
	// Deleted is true when the change tombstoned this record. Ops is
	// empty in that case; the engine drops the entry from its held
	// set.
	Deleted bool
	// Unreadable: the post value could not be read when the event was
	// built; Ops is empty.
	Unreadable bool
	// Ops is the projected $set / $unset operations on the post-apply
	// record. Empty when Deleted is true. Surfaced verbatim to
	// SubscriptionEvent SubRecord.Ops for atomic-update consumers.
	Ops []space.EventOp
}

EventRecord is one record's worth of projected change inside an Event. Id is the record id within Dataset (for shared per-space datasets like "objects" this equals ObjectId).

type PostValueFn

type PostValueFn func(recordIndex int) (*anyenc.Value, error)

PostValueFn returns the post-apply value for one record in the change being built. Provided by the per-space layer (the spaceobjects Store holds the controller). Index is into ch.Records.

(nil, nil) is a record that no longer exists at this point in the timeline: tombstoned, dropped, a gated op that didn't land. An error is a row that could not be read, which is not the same as absent: the engine re-reads the window from storage or closes the subscription rather than treat the record as gone.

type RefillFn added in v0.7.0

type RefillFn func(source any) (RefillRead, error)

RefillFn opens one read of the sub's rows. source is Event.Source of the apply behind the refill (nil for a notification; the rebuilt object behind Resync). The engine closes the read. Storage not final: return an error wrapping ErrRefillDeferred.

type RefillRead added in v0.7.0

type RefillRead struct {
	// Rows yields the sub's query in order, as the snapshot did.
	Rows func(yield func(id string, doc *anyenc.Value)) error
	// Lookup reads one row by id, ignoring the query: nil when absent, a
	// tombstone or out of scope. A nil Lookup leaves a dropped row
	// displaced.
	Lookup func(id string) (*anyenc.Value, error)
	// Close ends the read.
	Close func()
}

RefillRead is one read a refill reconciles the held set from, in a single transaction so its rows and lookups agree.

type Scope

type Scope struct {
	Shared     bool
	AllObjects bool
	ObjectId   string
	Dataset    string
}

Scope locates which events a sub cares about.

Shared=true means "every event whose dataset == ObjectsDataset across every object in the space" — the property firehose. The ObjectId/Dataset fields are ignored in this case.

AllObjects=true means "every event of Dataset, whatever object it was applied on" — a shared dataset read across the space. ObjectId is ignored.

Otherwise the scope is "events whose ObjectId/Dataset match exactly".

type SnapshotFn

type SnapshotFn func(yield func(id string, doc *anyenc.Value)) error

SnapshotFn is the caller-supplied populator. It runs UNDER engine.mu (held by Subscribe), so apply events fired during the read block waiting on engine.mu and process after the new sub is registered. The callback yields every row its read returns, in sort order, as (id, post-doc); the engine extracts the sort tuple via cfg.Sort and inserts the entry, skipping a row with no doc or id. A windowed sub that then holds at most Limit rows holds every match: the read asked for Limit+1.

The callback must only read already-materialised rows. Anything that can apply a change — loading an object (its replay runs OnApply), writing to the DAG — needs engine.mu itself and deadlocks the engine. Resolve objects before calling Subscribe.

type Sub

type Sub struct {
	// contains filtered or unexported fields
}

Sub is the consumer-side handle returned by Engine.Subscribe. It implements space.QuerySubscription. The engine guarantees the mailbox carries only post-snapshot transitions (no replay of the snapshot itself).

func (*Sub) Close

func (s *Sub) Close() error

Close releases the subscription. Idempotent.

func (*Sub) Err

func (s *Sub) Err() error

Err returns the close reason: nil while live or after Close, space.ErrSubscriptionOverflow, space.ErrSubscriptionDrifted (drift budget spent, a row that could not be settled, a failed re-read), or space.ErrObjectDeleted (CloseObject).

func (*Sub) Events

func (s *Sub) Events() *mb.MB[space.SubscriptionEvent]

Events returns the underlying mb/v3 mailbox. Use Wait for batched delivery or WaitOne for single events.

type SubConfig

type SubConfig struct {
	Scope       Scope
	Filter      query.Filter // already ANDed with `_deletedAt missing`
	Sort        query.Sort   // required when Limit > 0
	Limit       int          // 0 = unbounded; no sentinel, no fast-reject
	Offset      int          // rows the snapshot skipped; a window past offset 0 never starts complete
	MailboxCap  int          // 0 → defaultMailboxCap; clamped to >= minMailboxCap
	DriftBudget int          // percent of limit; 0 → defaultDriftBudgetPercent; ignored when Limit == 0
	// Refill opens a re-read of the sub's rows under engine.mu (the
	// snapshot's contract). nil disables refills: departures count
	// toward the drift budget and an unreadable row closes the sub.
	Refill RefillFn
}

SubConfig is the input to Engine.Subscribe. Built by queryImpl from the chained Filter / Sort / Limit + caller's QueryOpts. The engine owns the parsed query.Filter / query.Sort after registration; the caller must not mutate them.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL