Documentation
¶
Overview ¶
Package delta is v5's delta publication engine (plan item P3.15a).
A projection published to the render thread must not be re-sent whenever anything about it changes. At 20,000 rows that is a multi-megabyte structured clone per keystroke, which is precisely the stall the off-thread model exists to remove. This engine turns a new state of the world into the ops needed to get there from the old one.
Two design decisions carry the guarantee, and both are about what an op says:
Ops name a key and an ANCHOR ("after this other key"), never an integer index. Index-based ops force the publisher to maintain a key→index map, and every insert then shifts every following index — O(N) bookkeeping per change, which is the cost being avoided. Anchors are O(1) to produce and the consumer already knows where its own keys are.
Change detection compares VERSIONS, not payloads. So the engine's resident index is one integer per key rather than a copy of the data, which is what M11 bounds — see memory_test.go.
The engine is not safe for concurrent use; it belongs to the worker that owns the projection.
Index ¶
- type Change
- type ChangeKind
- type Engine
- func (parseEngine *Engine) Apply(parseChanges []Change) ([]Op, error)
- func (parseEngine *Engine) ExportState() State
- func (parseEngine *Engine) Keys() []Key
- func (parseEngine *Engine) Len() int
- func (parseEngine *Engine) Publish(parseRows []Row) ([]Op, error)
- func (parseEngine *Engine) Reset()
- func (parseEngine *Engine) Version(parseKey Key) (uint64, bool)
- type Key
- type Op
- type OpKind
- type Row
- type State
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Change ¶
type Change struct {
Kind ChangeKind
Row Row
// AfterKey anchors an insert. Empty means the head. Ignored when the key is
// already published, since an upsert never relocates a row — use Publish if
// the order genuinely changed.
AfterKey Key
}
Change is one edit a producer already knows about.
type ChangeKind ¶
type ChangeKind uint8
ChangeKind names one producer-described change.
const ( // ChangeUpsert inserts a key or updates it if already present. ChangeUpsert ChangeKind = iota // ChangeRemove drops a key. ChangeRemove )
type Engine ¶
type Engine struct {
// contains filtered or unexported fields
}
Engine maintains a published projection and computes deltas against it.
func Restore ¶
Restore rebuilds an engine from an exported index.
This is what makes P3.14's guarantee possible. Without it, a reloaded worker starts with an empty engine and its first publish emits an insert for every row — a full re-send of a 20,000-row projection across the boundary, on a code edit. With it, the first publish after a reload emits only what actually changed, which for an edit that changed no data is nothing at all.
func (*Engine) Apply ¶
Apply publishes changes the producer already knows about, in O(change).
This is the path that makes P3.15a's guarantee about TIME as well as ops: nothing here touches a row that was not named. A producer that knows it inserted one row — a command handler, a SQLite hook — should use this rather than re-reading 20,000 rows so the engine can rediscover the same fact.
The cost is that the producer is now responsible for correctness: a missed change leaves the consumer permanently stale, with nothing to detect it. A producer that cannot guarantee completeness should use Publish, which pays O(N) to be sure.
func (*Engine) ExportState ¶
ExportState captures the engine's publication index.
func (*Engine) Publish ¶
Publish computes the ops that turn the published projection into rows, and adopts rows as the new published state.
Reading the whole new state is O(N) by necessity — the engine cannot know what changed without looking. What P3.15a requires, and what this delivers, is that the OPS are O(change): a 20,000-row projection with one edited row publishes exactly one op, not 20,000. A producer that can describe its own changes should use Apply instead and skip the scan entirely.
Duplicate keys are rejected rather than deduplicated. A projection with two rows claiming one key has no correct delta — the second silently winning would make the consumer's state depend on iteration order.
func (*Engine) Reset ¶
func (parseEngine *Engine) Reset()
Reset drops all published state, so the next Publish emits a full insert set.
For a projection whose subject changed entirely — a different query, a different table — where a diff against the previous subject would emit a remove-everything/insert-everything pair anyway, at twice the size.
type Op ¶
type Op struct {
Kind OpKind
Key Key
// AfterKey anchors an insert or a move. An empty AfterKey means the head of
// the projection. Unused by updates and removes.
AfterKey Key
// Payload is carried by inserts and updates only. Removes and moves do not
// resend data, which is most of the saving on a reorder.
Payload []byte
}
Op is one delta operation.
type Row ¶
type Row struct {
Key Key
// Version changes whenever Payload changes. It is the producer's
// responsibility and the engine's only means of detecting an update — a
// producer that reuses a version for changed data will publish a stale row,
// and one that bumps it needlessly will publish a redundant update.
Version uint64
Payload []byte
}
Row is one published row.
Payload is opaque: the engine never inspects it, which is what lets the same engine publish table rows, chart points, or search results without learning anything about them.
type State ¶
type State struct {
// Order is the published key order.
Order []Key `json:"order,omitempty"`
// Versions parallels Order.
//
// Parallel arrays rather than a map so the JSON is compact and the order is
// carried by the structure rather than needing a separate sort on restore.
Versions []uint64 `json:"versions,omitempty"`
}
State is an engine's publication index, extracted for transfer across a worker reload (plan item P3.14).
It is the index and nothing else: keys, their order, and their versions. No payloads, because the engine never held any — which is what makes carrying this across a reload cheap enough to be worth doing.
Source Files