Documentation
¶
Overview ¶
Package index coordinates the secondary indexes attached to a labelled property graph.
A Manager owns a set of named indexes (label bitmap, hash exact-match, B+ tree range) and fans out mutations to every index that subscribes to the affected property or label. The fan-out is best-effort sequential: failures in one subscriber do not abort the others (subscribers are independent and idempotent).
Index ¶
- Constants
- Variables
- type BuildLog
- type BuildResolver
- type Change
- type ChangeOp
- type Manager
- func (m *Manager) AbandonBuild(l *BuildLog)
- func (m *Manager) Apply(c Change)
- func (m *Manager) ApplyBatch(changes []Change)
- func (m *Manager) BeginBuild(resolve BuildResolver) *BuildLog
- func (m *Manager) Count() int
- func (m *Manager) CreateIndex(name string, sub Subscriber) error
- func (m *Manager) DropIndex(name string) error
- func (m *Manager) FinishBuild(l *BuildLog, fn func(reg RegisterFunc) error) error
- func (m *Manager) GetIndex(name string) (Subscriber, error)
- func (m *Manager) ListIndexes() []string
- type NodeSet
- func (s *NodeSet) Add(node uint64) (wasEmpty bool)
- func (s *NodeSet) AddRange(from, to uint64)
- func (s *NodeSet) AppendTo(dst []uint64) []uint64
- func (s *NodeSet) Bitmap() (bm *roaring64.Bitmap, shared bool)
- func (s *NodeSet) CanonicalBitmap() (bm *roaring64.Bitmap, shared bool)
- func (s *NodeSet) Cardinality() uint64
- func (s *NodeSet) Contains(node uint64) bool
- func (s *NodeSet) IsEmpty() bool
- func (s *NodeSet) Minimum() uint64
- func (s *NodeSet) OrInto(dst *roaring64.Bitmap)
- func (s *NodeSet) Remove(node uint64) (nowEmpty bool)
- func (s *NodeSet) RemoveRange(from, to uint64) (nowEmpty bool)
- func (s *NodeSet) ToArray() []uint64
- type RegisterFunc
- type ResolvedApplier
- type Serializer
- type Subscriber
Examples ¶
Constants ¶
const MaxBuildLogChanges = 1 << 17
MaxBuildLogChanges bounds the number of changes one BuildLog retains.
The bound is required rather than defensive: without it a sustained write workload concurrent with a long build would grow the log without limit. A recorded entry is a Change plus the resolution captured beside it, measured at 88 bytes against the Change's own 64, so the ceiling costs about 11 MiB of transient memory for a build that is saturated with concurrent writes, and is released when the build ends.
Reaching it is a saturation condition, and it is answered with ErrIndexBuildOverflow rather than by silently truncating: a truncated log would register an index missing exactly the writes this mechanism exists to catch. Only explicit transactions can commit into the window at all — the DDL's schema gate excludes every autocommit writer for the duration — so the ceiling corresponds to more than a hundred thousand index changes committed by concurrent explicit transactions during a single backfill.
Variables ¶
var ErrIndexBuildOverflow = errors.New("index: concurrent-change log overflowed during an index build")
ErrIndexBuildOverflow is returned by Manager.FinishBuild when more than MaxBuildLogChanges changes were fanned out while the index was being built, so the recorded catch-up log is no longer complete.
It is a saturation signal, not a corruption: nothing was registered, the half-built index is discarded, and the statement can simply be retried. It is reported rather than absorbed because absorbing it is precisely the defect Manager.BeginBuild exists to prevent — an index registered while some of the writes concurrent with its build were never applied to it.
var ErrIndexCorrupted = errors.New("index: serialized form corrupted")
ErrIndexCorrupted is returned by Serializer.Deserialize when the serialised form is structurally malformed or its CRC32C trailer does not match the payload. Callers (snapshot recovery in particular) treat this as "rebuild from the LPG" rather than as a fatal error.
var ErrIndexExists = errors.New("index: an index by that name already exists")
ErrIndexExists is returned by Manager.CreateIndex when the name is already in use.
var ErrIndexNotFound = errors.New("index: no index by that name")
ErrIndexNotFound is returned by Manager.DropIndex or Manager.GetIndex when the named index does not exist.
var ErrIndexValueTypeUnsupported = errors.New("index: value type not supported for serialization")
ErrIndexValueTypeUnsupported is returned by a generic index's Serialize / Deserialize methods when the value-type parameter is not in the supported on-disk encoding set.
The set is per implementation and is wider than one type. The B+ tree (graph/index/btree) encodes string, int64, int32, int, uint64, uint32, uint and float64; the hash index (graph/index/hash) additionally encodes []byte and bool. The engine relies on that breadth: its numeric companion index is keyed by float64, so a float64-keyed btree MUST be serialisable for a numeric index to survive a checkpoint. The authoritative list is the table under "Supported value-type encodings" in docs/persistence.md.
Callers whose value type is outside the set can convert to one of the supported types before registering the index for snapshot durability.
Functions ¶
This section is empty.
Types ¶
type BuildLog ¶ added in v0.14.1
type BuildLog struct {
// contains filtered or unexported fields
}
BuildLog records the changes fanned out by a Manager while an index is being built, so Manager.FinishBuild can replay them into that index before it becomes reachable. See the file comment for the argument that the replay is exact, and for why each change is RESOLVED as it is recorded rather than as it is replayed.
A BuildLog is created by Manager.BeginBuild and is valid only until Manager.FinishBuild or Manager.AbandonBuild retires it. It is safe for concurrent use: the manager records into it from every goroutine that fans a change out.
func (*BuildLog) Len ¶ added in v0.14.1
Len reports how many changes the log currently holds. It returns 0 for a log that has overflowed, whose contents were released. Intended for tests and observability.
func (*BuildLog) Overflowed ¶ added in v0.14.1
Overflowed reports whether more than MaxBuildLogChanges changes were fanned out during the build, which makes the log an incomplete account of the window.
type BuildResolver ¶ added in v0.14.1
BuildResolver answers, AT THE INSTANT A CHANGE IS FANNED OUT, the two questions a bound index would otherwise ask the graph when the change is replayed: the changed node's current raw value of the property under build, and whether that node is eligible for the index being built (live, and carrying the label under build).
current is returned in whatever representation the subscriber's own value projection accepts — for the engine's indexes an lpg.PropertyValue — and is nil when the node is absent, carries no such property, or the change concerns neither the property nor the label under build. eligible carries the same verdict the index's own Binding.Eligible would return for the node at this instant.
A resolver's calls are SERIALISED — this log's own mutex is held across every one — so an implementation need not itself be safe for concurrent use. It is nonetheless entered from whichever goroutine fans the change out, so it must assume no goroutine affinity, and it runs concurrently with graph writers.
A resolver is called by Manager.Apply and Manager.ApplyBatch, from every goroutine that fans a change out, while the manager's lock is held SHARED and this log's own mutex is held. It must therefore read the graph exactly as a registered subscriber's Apply already does at that point and take no lock of its own: the log's mutex is a leaf, and a resolver that acquired something a graph writer holds while waiting on the manager's lock would close a cycle.
A nil resolver is legitimate ONLY when nothing to be registered against the log resolves anything from the graph — an unbound index, or a test double. Every bound index build must supply one; without it Manager.FinishBuild falls back to Subscriber.Apply, which re-reads the graph at replay time and is exactly the defect rmp #2793 closed (see the file comment).
type Change ¶
type Change struct {
// OldValue and NewValue are present only for property changes.
// They are typed as any so this package stays generic across
// every PropertyValue kind without importing the lpg package.
OldValue any
NewValue any
Node graph.NodeID
Dst graph.NodeID // edge changes only
Property uint32 // 0 when not a property change
Label uint32 // 0 when not a label change
Op ChangeOp
}
Change describes a single mutation observed by the Manager. Each subscriber inspects the relevant fields and decides whether to update its own state.
Property and Label fields are interned identifiers from the owning graph's registries (lpg.PropertyKeyID / lpg.LabelID), surfaced as uint32 so this package does not import the lpg package and create a cycle.
A Change is delivered by value: Manager.Apply and Manager.ApplyBatch copy it into each Subscriber.Apply call and hold only a read lock, so several goroutines can be fanning changes out at the same time, each working on its own copy. Change is therefore safe for concurrent use. The one caveat is OldValue and NewValue: they carry lpg.PropertyValue values, which are immutable after construction except that their bytes and list variants expose slices aliasing the value's backing store, so a subscriber that retains such a slice must not mutate it.
func (Change) IsEdgeChange ¶
IsEdgeChange reports whether the change concerns an edge.
type ChangeOp ¶
type ChangeOp uint8
ChangeOp tags the shape of a Change. It is an immutable scalar with no methods, so it is safe for concurrent use.
type Manager ¶
type Manager struct {
// contains filtered or unexported fields
}
Manager owns the set of named indexes attached to a graph and fans out mutations to every subscriber.
Manager is safe for concurrent use.
Example ¶
ExampleManager shows the Manager lifecycle: register a concrete index under a name, list and count the registered indexes, and reject a duplicate registration with ErrIndexExists.
package main
import (
"errors"
"fmt"
"github.com/FlavioCFOliveira/GoGraph/graph/index"
"github.com/FlavioCFOliveira/GoGraph/graph/index/label"
)
func main() {
m := index.NewManager()
if err := m.CreateIndex("by_label", label.NewNodeIndex()); err != nil {
fmt.Println("unexpected:", err)
}
// Re-registering the same name is rejected.
err := m.CreateIndex("by_label", label.NewNodeIndex())
fmt.Println("duplicate is ErrIndexExists:", errors.Is(err, index.ErrIndexExists))
fmt.Println("count:", m.Count())
fmt.Println("names:", m.ListIndexes())
}
Output: duplicate is ErrIndexExists: true count: 1 names: [by_label]
func (*Manager) AbandonBuild ¶ added in v0.14.1
AbandonBuild retires l without registering anything, discarding the recording. It is idempotent and safe on a log Manager.FinishBuild has already retired, so it can be deferred unconditionally at the point the build starts.
func (*Manager) Apply ¶
Apply fans c out to every registered subscriber under a read lock so subscribers cannot be unregistered mid-update. The Manager itself does not enforce ordering across subscribers.
Ordering contract (what a subscriber may rely on). Changes are delivered in the order the write path emits them — the sole delivery path is an [IndexBuffer] appended in mutation order and drained through Manager.ApplyBatch; nothing sorts, coalesces, or parallelises the stream. A subscriber must be:
- idempotent (a replayed change produces no duplicate state), and
- order-independent across changes to DIFFERENT facets of a node — a property SET interleaved with a label add/remove converges to the same postings in either order, because inserts are gated on the node's final [Binding.Eligible]/[Binding.CurrentValue] state.
A subscriber need NOT be order-independent across MULTIPLE changes to the SAME property key: those carry old→new payloads and must be applied in mutation order (which the delivery path guarantees). Recovery does not replay this stream at all — it rebuilds each index from the live graph via BulkLoad — so no legal path ever delivers same-key changes out of mutation order.
Example ¶
ExampleManager_Apply shows the Manager fanning a change out to every registered subscriber. The label index observes OpAddNodeLabel events and can then be queried back through GetIndex for the NodeIDs that carry a given label.
package main
import (
"fmt"
"github.com/FlavioCFOliveira/GoGraph/graph"
"github.com/FlavioCFOliveira/GoGraph/graph/index"
"github.com/FlavioCFOliveira/GoGraph/graph/index/label"
)
func main() {
const labelPerson = uint32(7)
m := index.NewManager()
_ = m.CreateIndex("node_labels", label.NewNodeIndex())
// A mutation observed by the owning graph is fanned out to every
// subscriber. Here two nodes acquire the Person label.
m.Apply(index.Change{Op: index.OpAddNodeLabel, Node: graph.NodeID(1), Label: labelPerson})
m.Apply(index.Change{Op: index.OpAddNodeLabel, Node: graph.NodeID(4), Label: labelPerson})
// Recover the concrete index to run a query.
sub, _ := m.GetIndex("node_labels")
idx := sub.(*label.Index)
fmt.Println("kind:", idx.Kind())
fmt.Println("Person count:", idx.Count(labelPerson))
fmt.Println("Person members:", idx.Scan(labelPerson))
}
Output: kind: label Person count: 2 Person members: [1 4]
func (*Manager) ApplyBatch ¶
ApplyBatch fans an ordered slice of changes out to every subscriber in order. The whole batch is applied under one read lock; this is the substrate consumed by future transaction integration (Sprint 3).
func (*Manager) BeginBuild ¶ added in v0.14.1
func (m *Manager) BeginBuild(resolve BuildResolver) *BuildLog
BeginBuild starts recording every change the manager fans out, and returns the log to pass to Manager.FinishBuild.
resolve captures, as each change is recorded, the graph-dependent facts a bound index would otherwise re-read when the change is replayed; see BuildResolver for what those are and for the one case in which nil is correct.
Call it BEFORE the backfill scan reads anything. A change committed between this call and the scan is recorded AND seen by the scan, which is harmless (the replay is idempotent); a change committed before this call is seen by the scan alone, which is correct. What must not happen is a change committed after the scan has read its node and before the index is registered, and that is exactly what the recording catches.
Every BeginBuild must be retired by Manager.FinishBuild or Manager.AbandonBuild; a log left active makes every subsequent fan-out pay to record into it and never releases the memory. The idiom is a deferred AbandonBuild, which is a no-op once FinishBuild has retired the log.
func (*Manager) Count ¶
Count returns the number of currently registered indexes. It is safe to call on a nil Manager and returns 0 in that case.
func (*Manager) CreateIndex ¶
func (m *Manager) CreateIndex(name string, sub Subscriber) error
CreateIndex registers sub under name. Returns ErrIndexExists when the name is already taken.
func (*Manager) FinishBuild ¶ added in v0.14.1
func (m *Manager) FinishBuild(l *BuildLog, fn func(reg RegisterFunc) error) error
FinishBuild runs fn with the manager's lock held EXCLUSIVELY, so no change fan-out can interleave with anything fn does, and retires l afterwards whatever the outcome.
The RegisterFunc handed to fn replays l's recording into a subscriber before registering it, so an index built while those changes were being fanned out catches up on them at the instant it becomes reachable. Deriving the catch-up from the registration rather than taking it as a separate argument is deliberate: it makes it impossible to register an index and forget to catch it up.
The replay goes through ResolvedApplier.ApplyResolved whenever the subscriber implements it AND the log was given a BuildResolver, so the change is applied from the state captured when it was fanned out. Otherwise it goes through Subscriber.Apply, which resolves against the graph as it stands NOW — correct only for a subscriber that resolves nothing from the graph. See the file comment for the measurement behind that distinction (rmp #2793).
Registrations performed through that function are indivisible with respect to the fan-out — the property rmp #2703 established for an index and its numeric companion, here supplied by the manager's own lock rather than by a caller's barrier, so it holds on every path that builds an index.
FinishBuild returns ErrIndexBuildOverflow without running fn at all when the recording is incomplete (see MaxBuildLogChanges); nothing is registered and the caller's half-built index is simply discarded.
func (*Manager) GetIndex ¶
func (m *Manager) GetIndex(name string) (Subscriber, error)
GetIndex returns the subscriber registered under name. It is safe to call on a nil Manager and returns ErrIndexNotFound in that case.
func (*Manager) ListIndexes ¶
ListIndexes returns the names of every currently registered index in unspecified order. It is safe to call on a nil Manager and returns nil in that case.
type NodeSet ¶ added in v0.6.0
type NodeSet struct {
// contains filtered or unexported fields
}
NodeSet is the per-key node-set representation shared by the btree, hash, and label indexes. The zero value is a valid empty set. See the package-level nodeset.go documentation for the state machine and the query/serialization/GC-safety invariants.
Concurrency: a NodeSet is not safe for concurrent use on its own. It is embedded by value in an index (a btree leaf slot, a hash shard map, or the label-index map), and every read and mutation is serialised by that owning index's lock; a NodeSet is never shared across goroutines outside that discipline. Lookup paths that hand a set's contents to a caller copy out under the read lock, so the returned data is safe for concurrent use.
The two fields form a tagged union resolved solely by meta's low two bits (see the state* constants). ptr is GC-scanned and is always nil or a real Go pointer; it never carries tag bits.
func NodeSetFromBitmap ¶ added in v0.6.0
NodeSetFromBitmap returns the cheapest NodeSet representation of bm. A bitmap whose cardinality fits the inline small-set tier is down-converted (its few sorted ids extracted) so a sparse entry reloaded from a roaring image regains the memory win; a denser bitmap is kept on the bitmap tier WITHOUT extracting its (potentially huge) id array, so a dense label costs no transient O(cardinality) slice. Ownership of bm transfers to the set when it is kept; when down-converted, bm is no longer referenced.
It is the label index's deserialization adaptor: that index persists the roaring native image, so it reads back a bitmap and calls this to recover the tiered in-memory shape (sprint 206, #1585).
func NodeSetFromSorted ¶ added in v0.6.0
NodeSetFromSorted builds a NodeSet from an already strictly-ascending id slice. It is the deserialization constructor: the btree and hash readers parse the logical sorted NodeID list and hand it here, getting the cheapest representation for that cardinality (singleton/small/ bitmap) without re-sorting. The caller guarantees ids is sorted ascending with no duplicates.
func (*NodeSet) Add ¶ added in v0.6.0
Add inserts node into the set, preserving ascending order and promoting to a bitmap when the small tier would overflow. Adding a node already present is a no-op (set semantics). Returns true when the set was previously empty (so a caller maintaining a distinct-key count can detect a brand-new entry).
func (*NodeSet) AddRange ¶ added in v0.6.0
AddRange adds every id in [from, to] (inclusive) to the set. It always promotes to (or stays) a bitmap and uses roaring's run-container AddRange, so a contiguous band of NodeIDs is stored in O(1) space. This is the bulk-ingest fast path the label index relies on for dense labels; it is intentionally the ONLY entry point that can create a bitmap without first crossing smallSetMax, and a set that takes an AddRange is permanently a bitmap (#1585).
An INVERTED interval (from > to) names no ids and is a no-op: it does not promote, because promotion is one-way and a set that gained nothing must not pay for a tier it never needed (#2608).
func (*NodeSet) AppendTo ¶ added in v0.6.0
AppendTo appends every NodeID in strictly ascending order — the same order as ToArray — to dst and returns the extended slice, WITHOUT materialising a throwaway bitmap for the inline (singleton/small) states. It is the allocation-light way to drain a set into a caller-owned buffer under the index read lock: a singleton or small set appends straight from the inline fields, so a caller whose dst has spare capacity (e.g. a reused seek buffer) pays no heap allocation at all. Only the promoted bitmap state allocates a single iterator. The appended ids are an independent snapshot the caller may read after releasing the lock.
func (*NodeSet) Bitmap ¶ added in v0.6.0
Bitmap returns the set as a *roaring64.Bitmap. When the set is already in the bitmap state the live bitmap is returned (the caller must NOT mutate it); otherwise a fresh bitmap is materialised from the sorted ids. The materialised image is byte-identical under roaring's content-deterministic WriteTo to a bitmap that held the same ids all along, which is what keeps the label index's roaring-native on-disk format unchanged across this refactor (storage-engine-auditor, #1585).
shared reports whether the returned bitmap aliases the set's live bitmap (true only in the bitmap state); callers that need an independent copy must Clone when shared is true.
func (*NodeSet) CanonicalBitmap ¶ added in v0.12.0
CanonicalBitmap returns the set as a *roaring64.Bitmap whose serialized form is a function of the set's LOGICAL CONTENTS rather than of the container the set happens to hold, for sets of at most smallSetMax ids.
roaring picks a container encoding from construction history, not from content: NodeSet.AddRange builds a RUN container, while the same ids added one at a time build an ARRAY one, and the two encode identical membership in different bytes. That made a Serialize/Deserialize/Serialize cycle change the bytes for a label in the band the reader down-converts — measured 55 bytes in and 64 to 72 out at widths 4 to 8 — so a snapshot of unchanged data was not byte-reproducible (#2609).
The normalisation is BOUNDED at smallSetMax deliberately, and it normalises DOWN — towards the encoding the inline tiers already produce — rather than up.
Normalising an arbitrary set the other way requires cloning it first, because the bitmap tier hands back the live bitmap and roaring's run optimisation rewrites containers in place; measured, that costs a sparse 100 000-id label 6.55 to 90 microseconds and 1 289 to 218 065 bytes per serialize to produce a BYTE-IDENTICAL image. Normalising down needs no clone at all: every set that is NOT on the bitmap tier was already materialised from its sorted ids by NodeSet.Bitmap, so the only state that can be off-canonical is a bitmap-tier set at or below the bound — which only NodeSet.AddRange can create. Every other set takes a single cardinality check and nothing else.
The bound is smallSetMax because that is exactly the band NodeSetFromBitmap down-converts, and therefore the only band where a Serialize/Deserialize cycle can change the encoding. Above the bound the form is already stable across a cycle.
MEASURED, interleaved A/B over five pairs: a 100 000-id dense label and a 100 000-id sparse one are unchanged in time and IDENTICAL in allocations (15/op, every sample equal), and so is an Add-built label of 8 ids (26 allocs/op, every sample equal) — the common small label, which is on an inline tier and therefore short-circuits. Only an AddRange-built label at or below the bound pays anything, moving from 16 to 28 allocs/op: that is the path being normalised, and 28 is what the Add-built label of the same ids already cost. The normalisation makes the two paths do the same work rather than making either do more.
shared reports whether the returned bitmap aliases the set's live bitmap; callers that need an independent copy must Clone when shared is true. It is false whenever the set was normalised, since normalisation builds a new one.
func (*NodeSet) Cardinality ¶ added in v0.6.0
Cardinality returns the number of NodeIDs in the set.
func (*NodeSet) Contains ¶ added in v0.6.0
Contains reports whether node is in the set. O(1) for the singleton state, O(log n) for the small array, and roaring's container probe for the bitmap state.
func (*NodeSet) Minimum ¶ added in v0.6.0
Minimum returns the smallest NodeID in the set. The caller must ensure the set is non-empty; on an empty set it returns 0.
func (*NodeSet) OrInto ¶ added in v0.6.0
OrInto adds every NodeID in the set to dst (set union into dst), preserving dst's ascending order. It is the allocation-light way to fold a small set into a destination bitmap during a range scan: a singleton becomes a single Add, a small set an AddMany of the sorted ids (which hits roaring's batch-by-high-bits fast path), and a bitmap a roaring Or — never materialising a throwaway bitmap for the inline states (graph-theory-expert, #1584).
func (*NodeSet) Remove ¶ added in v0.6.0
Remove deletes node from the set. No-op when absent. A NodeSet never demotes: removing from a bitmap leaves it a bitmap even if its cardinality drops to one (promote-and-never-demote, #1584). Returns true when the set became EMPTY as a result (so a caller maintaining a distinct-key count can drop the key).
func (*NodeSet) RemoveRange ¶ added in v0.6.0
RemoveRange removes every id in [from, to] (inclusive). On an inline (non-bitmap) set it removes the few covered ids individually; on a bitmap it uses roaring's RemoveRange. A NodeSet never demotes, so a bitmap stays a bitmap. Returns true when the set became EMPTY.
func (*NodeSet) ToArray ¶ added in v0.6.0
ToArray returns the NodeIDs in strictly ascending order as a freshly allocated slice the caller owns. This is the canonical iteration order every index consumer relies on, and the exact sorted list the btree and hash on-disk formats serialize — so it is representation-independent.
type RegisterFunc ¶ added in v0.14.1
type RegisterFunc func(name string, sub Subscriber) error
RegisterFunc registers one already-built subscriber under name. It is supplied by Manager.FinishBuild to the closure it runs, and differs from Manager.CreateIndex in two ways that matter: it replays the build log into sub before registering it, and it runs under a lock the caller already holds, so several registrations performed through it are indivisible with respect to the change fan-out.
It returns the same errors Manager.CreateIndex does, so a caller can absorb ErrIndexExists for an IF NOT EXISTS statement exactly as before.
Concurrency: NOT safe for concurrent use, and valid only for the dynamic extent of the Manager.FinishBuild call that supplied it. It closes over the manager's exclusive lock hold and over the captured replay slice, so it carries no synchronisation of its own — that is deliberate, and is what makes several registrations through one instance indivisible with respect to the change fan-out. Calling it from another goroutine, or retaining it beyond the closure it was handed to, escapes the lock hold it assumes and races the manager's index map.
type ResolvedApplier ¶ added in v0.14.1
type ResolvedApplier interface {
Subscriber
ApplyResolved(c Change, current any, eligible bool)
}
ResolvedApplier is implemented by a Subscriber whose Apply resolves part of a Change against the graph rather than from the change alone, and which can instead be handed that resolution as data.
Why the interface exists ¶
A bound index answers two questions about the changed node that the Change does not carry: is the node currently ELIGIBLE for this index (live, and carrying the bound label), and what is its CURRENT value of the bound property (a label add/remove carries no property payload). On the live fan-out both are asked at the instant the change is fanned out, which is the instant the committing transaction's state is final — the state the index must converge to.
The build-log replay (Manager.FinishBuild) applies a change LATER, and asking those questions then answers about a different instant. rmp #2793 measured the consequence: a second transaction's eager, uncommitted mutation, opened after the change was recorded and still open at the replay, made the replay insert a value nothing had committed and suppress the value the graph did hold. ApplyResolved closes that by taking the two answers from the log, where they were captured at fan-out time by the log's BuildResolver.
current is the node's raw property value as of the recording, in whatever representation the subscriber's own value projection accepts, and is nil when the node was absent or carried no such property. eligible is the recorded answer to the eligibility question. A subscriber must apply c using exactly the rules its Apply uses, substituting these two values for its own reads, so that the replay produces precisely the effects the live fan-out would have produced.
Implementations must be safe for concurrent use on the same terms as Subscriber.Apply.
type Serializer ¶
Serializer is implemented by indexes that can persist and restore their internal state through an io.Writer / io.Reader pair. The Manager type-asserts every registered Subscriber to this interface during snapshot writes; subscribers that do not implement Serializer are silently skipped (rebuild-on-restart).
Implementations must:
- Write a fixed self-describing header (magic + format version) so a future format bump can be detected on read.
- Cover the entire on-disk payload with a CRC32C trailer (uint32 little-endian) so corruption surfaces as ErrIndexCorrupted.
- Be safe for concurrent reads from other goroutines while Serialize executes (typically by holding the index's own RLock for the duration of the write).
Deserialize replaces the receiver's state with the contents of r. On any structural problem or CRC mismatch the function returns a wrapped ErrIndexCorrupted and leaves the receiver in its previous state.
type Subscriber ¶
type Subscriber interface {
Apply(Change)
// Kind returns a short stable identifier of the underlying index
// implementation, used for introspection (e.g. "label", "hash",
// "btree").
Kind() string
}
Subscriber is implemented by every concrete index that wishes to receive change events from the Manager. The Apply method must be idempotent: replays of the same change must not produce duplicate state.
Implementations must be safe for concurrent use: the Manager fans changes out to Apply while query goroutines read the same index concurrently, so a concrete index synchronises its own state internally (the built-in hash and label indexes hold an RWMutex). The Manager itself does not serialise an index's reads against its Apply calls.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package btree provides an order-preserving property index over a constraints.Ordered value type, answering range predicates against the NodeIDs that carry each value.
|
Package btree provides an order-preserving property index over a constraints.Ordered value type, answering range predicates against the NodeIDs that carry each value. |
|
Package count holds the derived, non-durable relationship count-store that backs exact cardinality estimates for the Cypher planner (design docs/count-store-design.md, task #2082).
|
Package count holds the derived, non-durable relationship count-store that backs exact cardinality estimates for the Cypher planner (design docs/count-store-design.md, task #2082). |
|
Package hash provides a sharded hash index from arbitrary comparable property values to the set of NodeIDs that carry them, represented as a 64-bit Roaring bitmap.
|
Package hash provides a sharded hash index from arbitrary comparable property values to the set of NodeIDs that carry them, represented as a 64-bit Roaring bitmap. |
|
Package label provides a Roaring-bitmap-backed inverted index from label identifiers to the NodeIDs that carry them.
|
Package label provides a Roaring-bitmap-backed inverted index from label identifiers to the NodeIDs that carry them. |
|
Package stats holds the best-effort, approximate planner statistics that back the Cypher optimiser's cardinality estimates for single-column predicates (design docs/statistics-design.md, tasks #2097 / #2098).
|
Package stats holds the best-effort, approximate planner statistics that back the Cypher optimiser's cardinality estimates for single-column predicates (design docs/statistics-design.md, tasks #2097 / #2098). |