fusion

package
v0.1.0-alpha.4 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 11, 2026 License: MIT Imports: 17 Imported by: 0

README

pkg/fusion

A deterministic engine that answers one question about the knowledge graph with one response. You give it a query and a lens; it finds the matching entities, reads the structure around them, ranks and trims the result, and returns nodes keyed by what a human reads (a name, a path, a line), never by an entity ID. No language model is involved: the same graph and the same query give the same answer. The decision record is ADR-062.

A lens is the one piece a product writes. It says how a query is resolved (nl, symbol or prefix), which edges to walk and what to call them, and how to name, classify and locate an entity. A seed is an entity the resolve step returned; everything else in a response grows from the seeds.

Migrating from SemStreams' pkg/fusion: read MIGRATION.md.

Use

engine, err := fusion.NewEngine(client, fusion.NewBodyResolver(storeRegistry), logger,
    fusion.WithSignals(fusionvocab.New(cco.Parents)), // optional: ontology and salience ranking
    fusion.WithMetrics(registry),                     // optional: the two counters below
)
if err != nil {
    return err // a nil client or logger, or a registry that refused a counter
}
req := fusion.Request{Query: "ParseConfig", Want: []fusion.Want{fusion.WantBody, fusion.WantRelations}}
resp, err := engine.Fuse(ctx, req, lens)

client is any fusion.RetrievalClient: over NATS, the one fusionnats.New builds (below); in tests, a fake. lens is your fusion.Lens. NewBodyResolver may be given nil if you never ask for bodies; a node whose lens names a body then reports body_reason error.

What a response holds

Part Request it with Meaning
nodes always one per seed, ranked, within budget (max_nodes 20, max_bytes 60000 by default)
body WantBody (the default) the stored text of the node, read through the lens's storage handle
relations WantRelations (the default) refs to neighbours, by role the lens names, up to 12 per role
paths WantPaths up to 10 outgoing paths of names, up to 4 hops
impact WantImpact counts of entities and files that depend on the seeds (reverse closure, up to 200 nodes)
graph WantGraph typed facts, directed edges and their evidence, with caps of its own

Ranking starts from the order the resolve step returned, then moves a node by how well its name matches the query. If you pass WithSignals, a more specific ontology class and a heavier predicate weight move it a little more; a negative weight moves it down. Entities with equal scores keep resolve order.

The response says what it left out

An empty or short answer is never ambiguous. Each of these is in the response, and each is omitted when it is zero or false:

  • deferred with defer_reason: the engine withheld the answer. index_not_ready means a read replied that the index is still catching up; all_seeds_unhydrated means seeds resolved but none could be read (unhydrated lists them). A deferred response has no nodes and no misses: it is not "nothing found".
  • unhydrated: seeds that resolved but could not be read, each with a reason (not_found, error or unknown). The seeds that were read still answer.
  • misses: the graph answered and found nothing, with near-matches in did_you_mean. A miss does not prove the entity is absent.
  • unreadable_records and failed_records: a word search (nl resolve) counts the records it could not read and the records whose embedding failed. They ride on every response built after that search answered, including a miss. Symbol and prefix resolves never set them.
  • relations_truncated (on a node): a role had more than 12 relations, a relations read failed, or a target could not be read.
  • paths_truncated, and truncated on a path: more than 10 paths existed, or a path stopped at a node that could not be read. A path never names an unread node.
  • impact.truncated and graph.truncated: the cap was reached or a read was skipped.
  • body_reason (on a node): the body was asked for and could not be loaded (not_found or error). The node still ships.
  • truncated (top level): the budget cut nodes off.

contract_version is "2". When a Fuse call returns an error instead, either the context ended, or Resolve or Entities failed with something other than index_not_ready (a graph reset included). Over fusionnats, a seed whose stored value graph-ingest refuses does not fail Entities: it is listed in unhydrated (below). A seed read or a relations neighbour read that replies index_not_ready makes the whole response deferred. Every other failed read after the seeds (relations neighbours with any other error, relation targets, paths, impact, graph) never fails the call: it is skipped and shown as one of the labels above.

Logs and counters

Each skipped read adds one to semengine_fusion_skipped_reads_total{facet} (relations, paths, impact, graph), and each Fuse call logs at most one Warn line per facet that skipped, with the count and the first error. A body that cannot be loaded logs a Warn line and adds one to semengine_fusion_body_hydration_failures_total{reason}. Both counters exist only on the registry passed to WithMetrics; without it there are no counters and the logs and response labels remain.

Over NATS: fusionnats

client, err := fusionnats.New(natsClient, 0) // *natsclient.Client; 0 takes natsclient's DefaultRequestTimeout
if err != nil {
    return err // a nil requester or a negative timeout
}

New refuses a nil requester and a negative timeout. It starts nothing: the client holds no goroutine, no watch and no lock, so it has no Close, and it reads no readiness status. A read whose index is still catching up is refused with code index_not_ready, and the engine reads that reply. Each method sends one request to one of graph-query's graph.query.* verbs, with the subject read from graph.QueryVerbs():

Method Verb
Resolve, symbol byName
Resolve, prefix prefix
Resolve, nl semantic (with the request's scope when it is not empty)
Entity entity
Entities batch
Neighbors relationships
Names byName

A poisoned entity is one whose stored value graph-ingest refuses to read (code graph_state_reset_required). graph-ingest refuses a whole batch when one entity in it is poisoned, so on that refusal Entities reads each distinct requested ID with its own entity request: a poisoned seed is listed in unhydrated with reason error, the others hydrate, and the answer goes on. If every seed is poisoned the response is deferred with all_seeds_unhydrated. This costs one request per distinct ID, only while poison is present. A seed not found when read alone is listed with reason not_found. Any other error from those reads, including an ended context (checked before each read and after each refused one), fails the call. Entities refuses a nil context.

Two gaps remain, each a filed issue:

  • A prefix resolve whose page holds a poisoned entity is refused whole by graph-ingest, so that Fuse call fails (#578).
  • A stack with no graph-embedding still offers semantic: an nl question there fails with a transient internal refusal naming graph.embedding.query.search, which a caller may retry without success. Issue #579 asks graph-query to refuse it outright when the composition offers no embedder.

symbol and prefix questions need no graph-embedding: TestSmoke_NoEmbedderAnswers (internal/smoke) answers them over a stack of graph-ingest, graph-index and graph-query alone.

Packages

  • pkg/fusion: the engine, Lens, RetrievalClient, BodyResolver, the response types. It imports no NATS client and no ontology package.
  • pkg/fusion/fusionnats: New(requester, timeout) returns the RetrievalClient over NATS. It imports no NATS client itself: it takes anything with RequestClassified, which *natsclient.Client has.
  • pkg/fusion/fusionvocab: New(parents) returns the ranking signals. parents is a function from a class IRI to its ancestors; the BFO/CCO hierarchy lives in the semvocab module, which depends on SemEngine, so the caller passes semvocab's cco.Parents. Pass nil to rank with no ontology signal. Predicate weights come from this repository's vocabulary registry.

Not in this package

  • The SemStreams sub-query engine (engine.go, subquery.go, evidence.go, client.go) and lensregistry are not carried. A product creates its lens and passes it to Fuse.

Documentation

Overview

Package fusion is the deterministic lens-driven fusion engine (ADR-062): it resolves a query to seed entities, walks the structure around them, ranks and budgets the result, and returns an answer keyed by what a human reads, never by an entity ID. A product supplies a Lens (lens.go) for its domain and a RetrievalClient (retrieval.go) over the graph.

It is a near-leaf package: its domain dependencies are message (Triple in Entity, StorageReference as the Hydrate handle), storage (the body store interface), graph and pkg/errs (the classified codes it reads from replies), and metric (its counters register only on a registry the caller passes). It imports no NATS client and asks no readiness status: a read whose index is not ready says so in its reply (ADR-109 D1).

Index

Constants

View Source
const (
	GraphDirectionOutgoing = "outgoing" // the seed is the edge's subject (source)
	GraphDirectionIncoming = "incoming" // the seed is the edge's object (target)
)

GraphEdge.Direction values, relative to the projected seed node. Source and Target always carry the TRUE subject→object orientation regardless of direction — Direction only says which side of the edge the seed is on.

View Source
const (
	FacetRelations = Facet(WantRelations) // the per-node relations map (forward + reverse roles)
	FacetPaths     = Facet(WantPaths)     // the outgoing paths DFS
	FacetImpact    = Facet(WantImpact)    // the incoming impact BFS
)

The three edge-walk facets — values structurally pinned to their Want twins.

View Source
const ContractVersion = "2"

ContractVersion identifies the wire shape of Request/Response.

"2" removed the readiness envelope (`index`), the graph facet's `view_revision`, and the readiness gate's defer reasons: readiness is read from each reply (ADR-109 D1).

View Source
const DeferReasonAllSeedsUnhydrated = "all_seeds_unhydrated"

DeferReasonAllSeedsUnhydrated is the engine-level defer cause for a query whose seeds all resolved but none could be read: the reads answered and the hydration failed, so nothing was withheld for readiness. The other value Response.DeferReason takes is graph.ErrorCodeIndexNotReady.

Variables

This section is empty.

Functions

func SeedIDs

func SeedIDs(seeds []Seed) []string

SeedIDs projects seeds to their IDs, for the batch-hydration call and anywhere else that needs the identity set rather than the scores.

Types

type BodyReason

type BodyReason string

BodyReason is why a node's requested verbatim body (WantBody) could not be loaded. It is a DISTINCT type from UnhydratedReason — a body-hydration failure and a seed-hydration failure are different surfaces: an Unhydrated seed produced NO node at all, whereas a BodyReason rides on a node that exists and ranks; only its Body is absent (gh#616, #600). The set is CLOSED and shares UnhydratedReason's vocabulary value-for-value so the two failure surfaces read the same on the wire.

const (
	// BodyNotFound is a body whose reference did not resolve to a stored object:
	// the lens produced no hydrate handle for the entity, or the handle could not
	// be resolved. It does NOT license "the entity has no body" as a fact — only
	// that this lookup produced none.
	BodyNotFound BodyReason = "not_found"
	// BodyError is a genuine fault reading the stored body — the deref returned an
	// error. Distinct from BodyNotFound so a caller can tell a missing object from
	// a backend fault.
	BodyError BodyReason = "error"
)

The closed body-hydration-reason set. Kept in lockstep with the UnhydratedReason vocabulary (pinned by a test).

type BodyResolver

type BodyResolver struct {
	// contains filtered or unexported fields
}

BodyResolver dereferences a Lens.Hydrate handle to its verbatim bytes. It is the engine-side helper the assemble step uses to populate a node body from the handle a lens returned — backend-agnostic via the injected StoreResolver.

func NewBodyResolver

func NewBodyResolver(r StoreResolver) *BodyResolver

NewBodyResolver builds a BodyResolver over the given StoreResolver. A nil resolver is permitted (ResolveBody then errors on any non-nil handle) so a deployment with no verbatim-body stores still constructs cleanly.

func (*BodyResolver) ResolveBody

func (b *BodyResolver) ResolveBody(ctx context.Context, ref *message.StorageReference) ([]byte, error)

ResolveBody returns the verbatim body bytes for a Hydrate handle.

Key granularity (gh#376 coordination): the handle's Key addresses the EXACT verbatim body — one pre-sliced body blob per entity (keyed by entity ID or content hash) — so Get returns the body byte-for-byte with NO engine-side line math. The lens's Locator (path + line range) is for citation/display only; the body comes pre-sliced through the handle, not by trimming a whole file.

  • A nil ref means "no verbatim body" — returns (nil, nil), NOT an error. (Lens.Hydrate returns (nil, nil) for body-less entities; this preserves that signal end-to-end.)
  • A non-nil ref with an empty StorageInstance, or an instance with no registered store, is a wiring/producer fault — returns an error so the caller can degrade the node (the engine omits the body; hydration is best-effort and does not fail the fuse — see Lens.Hydrate).
  • The store's Get error is propagated wrapped.
  • A nil context is refused.

type Budget

type Budget struct {
	MaxNodes int `json:"max_nodes,omitempty"`
	MaxBytes int `json:"max_bytes,omitempty"`
}

Budget bounds a response. Zero fields take engine defaults.

type Direction

type Direction int

Direction selects edge traversal direction.

const (
	Outgoing Direction = iota
	Incoming
)

Outgoing follows a subject's predicates to targets; Incoming follows the reverse (who points at this entity).

type Edge

type Edge struct {
	Predicate string
	Target    string
}

Edge is a relationship from a subject entity to a target entity.

type EdgeSpec

type EdgeSpec struct {
	Predicate    string
	OutgoingRole string
	IncomingRole string  // "" to skip the reverse direction
	Facets       []Facet // nil/empty = all facets; else only the named facets
}

EdgeSpec declares one relationship predicate the engine should expand around a seed, with the role labels for its forward and (optional) reverse directions. For code: {Predicate: "code.relationship.calls", OutgoingRole: "callee", IncomingRole: "caller"}. An empty IncomingRole skips the reverse direction.

Facets optionally restricts which edge-walk facets this predicate feeds. An empty Facets means the edge participates in ALL three facets (relations, paths, impact) — the backward-compatible default. A non-empty Facets restricts the edge to exactly the named facets, so e.g. a containment edge can populate the relations map (Facets: {FacetRelations}) without inflating the impact walk.

type Engine

type Engine struct {
	// contains filtered or unexported fields
}

Engine runs the lens-driven deterministic fusion pipeline over a RetrievalClient, hydrating verbatim bodies through a BodyResolver. It is safe for concurrent Fuse calls: nothing in it changes after NewEngine.

func NewEngine

func NewEngine(graph RetrievalClient, body *BodyResolver, logger *slog.Logger, opts ...Option) (*Engine, error)

NewEngine builds a lens-driven engine. graph and logger are required. body may be nil — a node whose lens yields a body handle then reports BodyError. A counter registration the registry refuses is returned, and no engine.

func (*Engine) Fuse

func (e *Engine) Fuse(ctx context.Context, req Request, lens Lens) (Response, error)

Fuse resolves req against the graph through lens and returns the fused response.

The honesty rule is that an empty result is never ambiguous. Fuse asks no readiness status; it reads readiness from each reply (ADR-109 D1). Every failed read ends in one of these outcomes, checked in this order:

  • the context ended: Fuse returns the context's error, whichever read failed.
  • index_not_ready from Resolve, Entities or a relations Neighbors read: the response is deferred (graph.ErrorCodeIndexNotReady), with no nodes and no miss.
  • any other error from Resolve or Entities, fatal ones such as graph_state_reset_required included: Fuse returns it.
  • any other failed read (a relations Neighbors read, a relation target, the paths, impact and graph walks, index_not_ready included for the walks): the read is skipped and the answer goes on without it. A fatal walk read is skipped too, because poison refuses only the poisoned entities (ADR-109 D1) and the reply cannot tell that refusal from a reset.

A healthy graph that found nothing yields a miss with near-matches; seeds that resolved but could not be read yield Unhydrated, deferred, and NO miss.

type Entity

type Entity struct {
	ID      string
	Triples []message.Triple
}

Entity is the minimal projection a Lens reads: the entity's ID and its current triples. Kept deliberately small (no graph.EntityState, no NATS) so pkg/fusion stays a near-leaf — the Lens reads human-facing fields off the triples.

func (*Entity) First

func (e *Entity) First(predicate string) string

First returns the first object for predicate rendered as a string, or "". A convenience for lenses/ranker so they read triple objects without touching the triple shape.

func (*Entity) FirstInt

func (e *Entity) FirstInt(predicate string) int

FirstInt returns the first object for predicate as an int, or 0.

type Facet

type Facet string

Facet names one of the three edge-walk facets the engine derives from a lens's EdgeSpecs. It is a distinct type from Want (so WantBody, which is not edge-driven, is unpassable), but its values are DERIVED from the matching Want constants below so the two vocabularies cannot drift. (WantBody has no Facet.)

type GraphEdge

type GraphEdge struct {
	ID        string          `json:"id"`
	Source    string          `json:"source"`
	Target    string          `json:"target"`
	Predicate string          `json:"predicate"`
	Direction string          `json:"direction"`
	Evidence  []GraphEvidence `json:"evidence,omitempty"`
	Truncated bool            `json:"truncated,omitempty"`
}

GraphEdge is one directed relationship. Source and Target are node handles in TRUE subject→object orientation regardless of which side the seed is on; Direction reports the seed's side; when BOTH endpoints are seeds the edge is discovered outgoing-first, so Direction reads "outgoing" relative to the subject seed (deterministic first-discovery). Edge identity is (source, predicate, target) — rendered as the stable ID "<source>|<predicate>|<target>" — so parallel predicates between the same pair are distinct edges, opposite directions are distinct edges with swapped Source/Target, and multiple assertions of the same edge merge into ONE edge with multiple inspectable Evidence entries, listed in evidence order (source, timestamp, confidence, context). Truncated reports the per-edge evidence cap.

type GraphEvidence

type GraphEvidence struct {
	Source     string   `json:"source,omitempty"`
	Timestamp  string   `json:"timestamp,omitempty"`
	Confidence *float64 `json:"confidence,omitempty"`
	Context    string   `json:"context,omitempty"`
}

GraphEvidence is one stored assertion's provenance, projected verbatim: absent values are OMITTED from the wire, never defaulted or synthesized. Confidence is a pointer so an absent confidence is distinguishable from an asserted value — the stored float zero value is the unset default and is therefore omitted, never fabricated as a literal 0. Timestamp is RFC 3339 and omitted when the stored time is zero.

type GraphFact

type GraphFact struct {
	Predicate string          `json:"predicate"`
	Value     json.RawMessage `json:"value"`
	Datatype  string          `json:"datatype,omitempty"`
	Evidence  []GraphEvidence `json:"evidence,omitempty"`
	// Truncated reports the per-fact evidence cap was hit; the surviving
	// entries are the first in evidence order (source, timestamp, confidence,
	// context), whatever order the triples were stored in, and the
	// projection-level Truncated flag is set alongside.
	Truncated bool `json:"truncated,omitempty"`
}

GraphFact is one typed property fact: the ORIGINAL predicate verbatim, the value as typed JSON passthrough (no coercion, no stringification), the verbatim datatype (absent stays absent), and the contributing evidence. Multi-valued predicates produce multiple facts; duplicate assertions of the same (predicate, value) merge into one fact carrying one evidence entry per contributing triple.

type GraphNode

type GraphNode struct {
	Handle string      `json:"handle"`
	Facts  []GraphFact `json:"facts,omitempty"`
	// FactsTruncated reports the fact list was capped; FactsDropped counts the
	// distinct facts cut. Both are omitted when nothing was dropped.
	FactsTruncated bool `json:"facts_truncated,omitempty"`
	FactsDropped   int  `json:"facts_dropped,omitempty"`
}

GraphNode is one projected seed entity: the same opaque Handle as Node.Handle (never parse or construct it) plus the entity's typed property facts. FactsTruncated + FactsDropped report the per-node fact cap, so a consumer can always tell a small entity from a capped one.

type GraphProjection

type GraphProjection struct {
	Nodes []GraphNode `json:"nodes"`
	Edges []GraphEdge `json:"edges,omitempty"`
	// Truncated reports that ANY node- or edge-level cap below cut content
	// (facts, edges, or evidence), that an edge walk faulted and left the edge
	// set incomplete, or that an incoming counterpart's read failed and its
	// edge carries no evidence — a partial projection never reads as exact.
	Truncated bool `json:"truncated"`
}

GraphProjection is the optional structured graph facet carried on Response.Graph when the request Wants "graph". Nodes are the response's seed entities (same opaque handles as Node.Handle); Edges are the directed relationships around them, in true subject→object orientation. The projection is lossless but bounded: its caps are independent of the v1 node/body budget, and every cut is reported through FactsTruncated / FactsDropped / Truncated — never silently.

Its reads are not one snapshot: the engine assembles a projection from N independent reads with no read transaction, so nothing in it says the reads hit one indexed revision. A consumer that needs a coherent single-revision view must use pkg/graphview (ADR-081); this facet is best-effort ranked evidence.

type Hydration

type Hydration struct {
	// Entities are the hydrated entities IN REQUESTED ORDER.
	Entities []*Entity
	// Unhydrated names every requested ID absent from Entities. Nil when complete.
	Unhydrated []Unhydrated
}

Hydration is a batch fetch's outcome: the entities that loaded, in requested order, plus every requested ID that did not, with a reason.

The two lists together account for every requested ID exactly once. That totality is the point: before it, an ID whose read came back not-found was simply missing from a shorter slice, and no caller could tell which one — or whether anything had gone wrong at all (gh#597).

type Impact

type Impact struct {
	Nodes     int  `json:"nodes"`
	Files     int  `json:"files"`
	Truncated bool `json:"truncated"`
}

Impact summarizes the transitive reverse-relation closure of the response's seeds — the blast radius of changing them — carried on Response.Impact. It counts distinct affected nodes and files (a lens Location path); Truncated marks that the node cap was hit, a walk read was skipped, or an affected node's read for its file was skipped, so the counts are a lower bound.

type Lens

type Lens interface {
	// Name identifies the lens (e.g. "code", "docs"). Used as the registry key.
	Name() string

	// ResolveMode picks how to turn the raw query into seeds.
	ResolveMode(query string) ResolveMode

	// Edges are the relationship predicates to expand around each seed.
	Edges() []EdgeSpec

	// Label is the entity's human name (e.g. from dc.terms.title).
	Label(e *Entity) string

	// Kind is a short human kind for the entity (e.g. "function", "doc").
	Kind(e *Entity) string

	// Location returns the entity's domain-general place (file path or URL,
	// optional section fragment, optional line range).
	Location(e *Entity) Locator

	// Hydrate returns a HANDLE to the entity's verbatim body — never the bytes
	// themselves and never a filesystem read. The engine resolves the handle's
	// StorageInstance to a registered storage.Store and reads the body with
	// Get(Key) (ADR-062 hydration contract, increment 4). Returning a handle
	// (not a string) is what makes fusion deployment-independent: a remote
	// caller of a standalone fusion service (semsource ADR-0006) cannot read the
	// service's worktree, so bodies must be addressable through a backend-
	// agnostic store. Return (nil, nil) when the entity has no verbatim body.
	//
	// Error policy (engine contract): hydration is best-effort. A non-nil error
	// DEGRADES the response — the engine omits that node's body and continues —
	// it does NOT fail the fuse. This mirrors the engine's degrade-don't-fail
	// posture for sub-query failures (see Fuse). Lenses should return an error
	// only for genuine retrieval faults, not for "no body" (use (nil, nil)).
	Hydrate(ctx context.Context, e *Entity) (*message.StorageReference, error)
}

Lens supplies the only domain-specific parts of fusion (ADR-062). The engine owns resolve / expand / rank / budget / envelope; the lens declares which edges to walk, how to read an entity's human-facing fields, and how to hydrate its verbatim body.

type Locator

type Locator struct {
	Path     string // file path or URL
	Fragment string // section / anchor (docs)
	Lines    [2]int // line range (code); zero value means line-less
}

Locator is a domain-general place: a file path or URL, an optional section / anchor fragment (docs), and an optional line range (code). One of Lines or Fragment is typically set, not both.

type MapStoreResolver

type MapStoreResolver map[string]storage.Store

MapStoreResolver is a static StoreResolver over a name→Store map. The zero value (nil map) resolves nothing. Safe for concurrent reads (Go map reads are safe without writers; the map is build-once at wiring time).

func (MapStoreResolver) Store

func (m MapStoreResolver) Store(instance string) (storage.Store, bool)

Store implements StoreResolver.

type Miss

type Miss struct {
	Query      string   `json:"query"`
	DidYouMean []string `json:"did_you_mean,omitempty"`
}

Miss reports a query that resolved to nothing, with near-matches.

A Miss is NOT an absence proof and is no longer tied to Ready (ADR-084): it is reachable under ordinary lag now that healthy-but-behind indexes serve, and it was never evidence the entity does not exist — only that this lookup found nothing. Callers that must distinguish "found nothing" from "could not read" want Unhydrated, which is a statement about the read.

type Node

type Node struct {
	Name     string `json:"name"`
	Kind     string `json:"kind,omitempty"`
	Path     string `json:"path,omitempty"`
	Fragment string `json:"fragment,omitempty"`
	// Lines is [start,end] for code, nil for line-less domains (docs) — a slice
	// so omitempty actually omits it rather than emitting a spurious [0,0].
	Lines []int  `json:"lines,omitempty"`
	Body  string `json:"body,omitempty"`
	// BodyReason names why Body is absent when a body was requested (WantBody) but
	// could not be loaded — a bounded, closed reason (not_found / error) so a
	// missing body is a partial-result SIGNAL, never a silent empty string
	// (gh#616, #600). It is DISTINCT from Response.Unhydrated: a missing body is a
	// node that exists and ranks, so the reason rides on the node itself rather
	// than the top-level list. A failed body hydration never defers the response
	// or synthesizes a Miss — the node still ships.
	//
	// Omitted from the wire when the body hydrates, so a fully-hydrated response is
	// byte-unchanged for existing consumers.
	BodyReason BodyReason       `json:"body_reason,omitempty"`
	Relations  map[string][]Ref `json:"relations,omitempty"`
	// RelationsTruncated reports that Relations is missing a ref: a role had more
	// relations than its cap, a relations Neighbors read failed and was skipped,
	// or a relation's target read failed or found nothing. Omitted otherwise.
	RelationsTruncated bool `json:"relations_truncated,omitempty"`
	// Class is the BFO/CCO class IRI (provenance/debug; the agent ignores it).
	Class string `json:"class,omitempty"`
	// Handle is an opaque continuation token (internally the entity ID). Not an
	// addressing scheme: never parse or construct it.
	Handle string `json:"handle,omitempty"`
	// Rank is this node's 1-based position in RESOLVE order — where the index put it
	// before the engine re-ranked. Present only when the request set IncludeScores.
	//
	// Deliberately not the node's position in the response: that is the array index,
	// which the caller can already count. Ranking reorders (lexical, ontology,
	// salience), so the GAP between resolve rank and response position is the whole
	// signal — it is what makes "why did this come out third?" answerable.
	Rank int `json:"rank,omitempty"`
	// Similarity is the resolve mode's own relevance score, present only when
	// IncludeScores was set AND the mode reports one (semantic does; symbol and prefix
	// do not). Joined to this node by entity ID, never by position.
	//
	// A POINTER so the wire is self-describing: absent means the mode does not score,
	// present means this is the score — including a genuine 0.0, which a bare
	// `float64,omitempty` would erase into indistinguishability from "unavailable".
	// A separate has_similarity bool round-trips correctly in Go but forces every
	// non-Go consumer to learn that an absent key means zero rather than nothing.
	Similarity *float64 `json:"similarity,omitempty"`
}

Node is one fused result: verbatim body plus the structure around it. Domains differ only in which roles populate Relations (code: callers/callees; docs: links/sections).

type Option

type Option func(*Engine)

Option configures an Engine at construction. Nothing changes an engine after NewEngine returns it.

func WithMetrics

func WithMetrics(registry *metric.MetricsRegistry) Option

WithMetrics registers the engine's two counters on registry: semengine_fusion_body_hydration_failures_total{reason} and semengine_fusion_skipped_reads_total{facet}. Engines given the same registry share one series. Without it the engine has no counters; its log lines and the response's labels still declare what was left out.

func WithSignals

func WithSignals(s RankSignals) Option

WithSignals attaches the framework ranking signals — ontology specificity + predicate salience (ADR-062 increment 5, gh#396) — folded into ranking on top of resolve-order + lexical. nil (the default) keeps ranking at resolve-order + lexical.

type Path

type Path struct {
	Names     []string `json:"names"`
	Truncated bool     `json:"truncated,omitempty"`
}

Path is a bounded outgoing relation walk from a seed, rendered as a sequence of human names (never entity IDs), carried on Response.Paths. Truncated marks a path cut short at the depth cap, a cycle, a failed walk read, or a node that could not be read — the partial path is kept, not dropped.

type Provenance

type Provenance string

Provenance records how an answer was produced so callers can calibrate trust.

const (
	ProvenanceDeterministic Provenance = "deterministic" // exact lookup + structural walk
	ProvenanceEmbedding     Provenance = "embedding"     // seeds came from semantic search
	ProvenanceLLM           Provenance = "llm"           // an LLM reasoned over the result
)

The provenance tiers, in increasing order of uncertainty.

func ProvenanceForMode

func ProvenanceForMode(mode ResolveMode) Provenance

ProvenanceForMode maps a ResolveMode to the honesty-envelope provenance tier: symbol/prefix seeds are deterministic exact lookups; nl seeds come from embedding search. (The research_graph LLM sibling is the only "llm" source — the deterministic engine never sets it.) Exported so the engine's assemble step and any lens-driven caller stamp provenance consistently.

Deliberate divergence from semsource's provenanceFor (source/fusion/engine.go), whose catch-all is "deterministic" and only embedding is named: there, ResolveMode is a CLOSED int enum so "unknown" is unreachable. Here ResolveMode is an OPEN string type, so the catch-all must be the conservative tier — "embedding" — because provenance is a claim about HOW an answer was produced, and an unclassified seed has not earned the stronger claim. (The reasoning once rested on "only deterministic permits an authoritative not-found"; ADR-084 retired that license entirely — no provenance tier licenses an absence claim — but the conservative default stands on its own.) CAUTION: this means any NEW deterministic mode added to the enum MUST be added to the deterministic case below, or it silently degrades to embedding (no compile error — open string enum).

type RankSignals

type RankSignals interface {
	// ClassSpecificity scores how specific an ontology class IRI is: more
	// specific (deeper in the BFO/CCO subclass tree) = higher; 0 for an
	// unknown or unset class. A deeper class carries more information, so within
	// the same resolve/lexical tier a precisely-typed entity reorders ahead of a
	// vaguely-typed peer.
	ClassSpecificity(classIRI string) float64

	// PredicateSalience returns a predicate's stored salience weight, SIGNED:
	// positive boosts (an entity carrying the fact reorders ahead), negative
	// demotes (it reorders behind), 0 is neutral. Production reads vocabulary
	// PredicateMetadata.Weight. A negative weight is how a consumer down-ranks
	// structurally-identifiable noise (tests, generated code, mocks) that carries
	// the same boosted predicates as the real thing — additive boosting alone
	// cannot separate them (gh#441). The engine folds an entity's strongest boost
	// and strongest demotion together (see entitySalience), so a demotion is a
	// bounded secondary reordering, never an exclusion.
	PredicateSalience(predicate string) float64
}

RankSignals supplies the framework ranking signals the lens engine folds into rankEntities on top of resolve-order + lexical: ontology specificity and predicate salience (ADR-062 increment 5, gh#396). Injected through an interface so pkg/fusion stays a leaf (no vocabulary / bfo / cco imports); production wires the vocabulary registry + BFO/CCO subclass helper (see pkg/fusion/fusionvocab). A nil RankSignals leaves ranking at resolve-order + lexical — the increments 1–4 behavior — so attaching signals is purely additive to the pipeline (nil → unchanged). The salience signal itself is DIRECTIONAL: a predicate's weight is signed, so it can down-rank as well as boost (gh#441).

type Ref

type Ref struct {
	Name     string `json:"name"`
	Path     string `json:"path,omitempty"`
	Fragment string `json:"fragment,omitempty"`
	Line     int    `json:"line,omitempty"`
}

Ref points to a node by what a human reads — never the entity ID.

type Request

type Request struct {
	Query string `json:"query"`
	Want  []Want `json:"want,omitempty"`
	// Scope optionally constrains NL seed resolution to entities whose ID
	// matches at least one of these dot-delimited entity-ID prefixes
	// (OR-matched). Empty/absent means no filter — today's behavior. It lets a
	// lens instance over a shared embedding index retrieve only its own domain
	// so a smaller domain is not diluted by a larger co-resident one (ADR-071).
	// A list, not a scalar: one domain often spans several prefixes (semsource
	// "all code" = golang/python/ts/svelte). NL-only; ignored by symbol/prefix
	// resolve modes.
	Scope  []string `json:"scope,omitempty"`
	Budget Budget   `json:"budget,omitzero"`
	// IncludeScores asks for per-node rank and resolve similarity. Opt-in so the
	// default response is byte-unchanged: scores are debugging and calibration
	// signal, and most callers act on the ordering rather than the numbers.
	IncludeScores bool `json:"include_scores,omitempty"`
}

Request is the fused query, keyed by what an agent already knows — never an entity ID.

type Resolution

type Resolution struct {
	Seeds             []Seed
	UnreadableRecords int
	FailedRecords     int
}

Resolution is what Resolve found: the seeds, most relevant first, and, for a natural-language (semantic) search, the counts its reply reports of records it could not search: UnreadableRecords (a record it could not read) and FailedRecords (a record whose embedding failed). Symbol and prefix resolves leave both 0.

type ResolveMode

type ResolveMode string

ResolveMode picks how the engine turns a raw query string into seed entities. The Lens classifies the query; the engine maps the mode to a resolve strategy (symbol → graph.query.byName exact lookup, prefix → graph.query.prefix, nl → semantic search). Provenance follows from the mode: symbol/prefix are deterministic, nl is embedding.

const (
	// ResolveModeNL routes the query to semantic (embedding) search. Seeds are
	// embedding-ranked, so the response provenance is "embedding", not
	// "deterministic".
	ResolveModeNL ResolveMode = "nl"

	// ResolveModeSymbol routes to deterministic exact name/title lookup
	// (graph.query.byName). The provenance the byName index makes honest.
	ResolveModeSymbol ResolveMode = "symbol"

	// ResolveModePrefix routes to deterministic prefix lookup
	// (graph.query.prefix) — e.g. a path or namespace stem.
	ResolveModePrefix ResolveMode = "prefix"
)

type ResolveQuery

type ResolveQuery struct {
	Query string
	Mode  ResolveMode
	Scope []string
	Limit int
}

ResolveQuery is the argument set for RetrievalClient.Resolve. Mode selects the resolve strategy; Scope is honored only for ResolveModeNL (a filter on embedding candidates), where empty/nil means no filter.

type Response

type Response struct {
	Provenance Provenance `json:"provenance"`
	Nodes      []Node     `json:"nodes,omitempty"`
	Paths      []Path     `json:"paths,omitempty"`
	// PathsTruncated reports that Paths is missing a path: more paths existed than
	// the cap, or a path was not emitted because a node on it could not be read.
	// Omitted otherwise.
	PathsTruncated bool    `json:"paths_truncated,omitempty"`
	Impact         *Impact `json:"impact,omitempty"`
	// Graph is the structured graph projection (WantGraph): typed property
	// facts, explicit directed edges, and verbatim evidence, with its own caps
	// (its reads are independent, not one snapshot). Its truncation metadata is self-contained —
	// graph-facet truncation NEVER sets the top-level Truncated bit, and a
	// request without the graph want omits the field entirely (the v1 wire
	// shape is unchanged for non-requesting callers).
	Graph  *GraphProjection `json:"graph,omitempty"`
	Misses []Miss           `json:"misses,omitempty"`
	// Unhydrated names requested seeds that did not load — DISTINCT from Misses. A
	// Miss says "the graph was asked and had nothing"; an Unhydrated entry says "we
	// could not read this one", a statement about the read rather than about the
	// world. Neither licenses an absence conclusion (ADR-084), and the
	// all-seeds-unhydrated case deliberately synthesizes NO Miss: claiming a miss
	// there would assert exactly the thing the failed read left unknown.
	//
	// Omitted when everything hydrated, so a complete response is byte-unchanged.
	Unhydrated []Unhydrated `json:"unhydrated,omitempty"`
	// Deferred reports that the engine WITHHELD rather than answered: the empty
	// result is a refusal to look, not a finding. Check this to answer "did I get
	// an answer?".
	Deferred bool `json:"deferred,omitempty"`
	// DeferReason is why: graph.ErrorCodeIndexNotReady when a read's reply said the
	// index was not ready, or DeferReasonAllSeedsUnhydrated when seeds resolved and
	// none could be read. Empty when not deferred.
	DeferReason string `json:"defer_reason,omitempty"`
	// UnreadableRecords and FailedRecords are the natural-language search's counts
	// of records it could not read and records whose embedding failed, copied from
	// its reply onto every response built after it answered: nodes, a miss, or a
	// later deferral. A miss beside a non-zero count is not "nothing there". Each is
	// omitted when 0, and both always are after a symbol or prefix resolve.
	UnreadableRecords int    `json:"unreadable_records,omitempty"`
	FailedRecords     int    `json:"failed_records,omitempty"`
	Truncated         bool   `json:"truncated"`
	ContractVersion   string `json:"contract_version"`
}

Response is the fused answer. Nodes is the payload; Provenance and Deferred are the honesty envelope. Paths, Impact, and Graph are optional facets, present only when the request Wants them (WantPaths / WantImpact / WantGraph).

type RetrievalClient

type RetrievalClient interface {
	// Resolve maps a query to seeds, most relevant first, and reports what the
	// search could not read. Both its argument and its result are structs rather
	// than bare values, for the same reason: the NL-only Scope must not force
	// symbol/prefix callers to pass an ignored value, and a resolve dimension adds
	// a field instead of re-breaking every impl and fake (ADR-071).
	Resolve(ctx context.Context, q ResolveQuery) (Resolution, error)
	// Entity returns an entity by ID, or (nil, nil) if the read found nothing.
	// "Found nothing" is not proof of absence — see Hydration.
	Entity(ctx context.Context, id string) (*Entity, error)
	// Entities batch-fetches entities by ID and reports what it could not hydrate.
	// A non-nil error means a BACKEND failure, which callers MUST distinguish from
	// an entity that is simply absent.
	//
	// It returns a struct rather than a bare slice for the same reason Resolve takes
	// one (see above): partial hydration needed a second output, and a future one
	// should add a field instead of re-breaking every impl and fake.
	//
	// IMPLEMENTATIONS MUST return Entities in the REQUESTED ORDER. The engine's
	// resolve-rank base is position-derived, so hydration order is the ranking prior,
	// not a presentation detail — a transport that returns "whatever order was
	// convenient" silently reorders results by cache residency (graph-ingest's
	// batch verb answers cache hits first; Client.Entities in pkg/fusion/fusionnats
	// restores request order for exactly this reason).
	Entities(ctx context.Context, ids []string) (Hydration, error)
	// Neighbors returns edges from id along the given predicates in a direction.
	Neighbors(ctx context.Context, id string, predicates []string, dir Direction) ([]Edge, error)
	// Names suggests entity display names near a query (for a miss's did_you_mean).
	Names(ctx context.Context, query string, limit int) ([]string, error)
}

RetrievalClient is the resolve/expand surface the lens-driven Engine composes over (ADR-062 lens-driven entry): it maps a query to seeds and walks structure.

Production wraps NATS request/reply over the graph query verbs; tests use an in-memory fake. Keeping the engine behind this interface is what lets it stay deterministic and unit-testable without a live graph.

There is no readiness read: a read whose index is not ready returns a classified error with code graph.ErrorCodeIndexNotReady, and the engine reads readiness from that reply (ADR-109 D1).

type Seed

type Seed struct {
	// ID is the resolved entity ID.
	ID string
	// Similarity is the resolve mode's own relevance score, when it has one.
	// Semantic resolve reports cosine similarity; symbol and prefix report none.
	Similarity float64
	// HasSimilarity distinguishes "this mode scored it 0" from "this mode does not
	// score". Without it a prefix seed would surface as a perfect zero-relevance
	// match, which is a claim the prefix wire never made.
	HasSimilarity bool
}

Seed is one resolved candidate, most relevant first.

It carries Similarity because the graph.query.semantic wire already reports a per-result score and the resolve surface was throwing it away — the engine could rank only by the ORDER seeds arrived in, so a caller asking "how confident is this match?" had nothing to read. Modes that carry no score (symbol, prefix) leave it zero, which is why HasSimilarity exists rather than treating 0 as a value.

type StoreResolver

type StoreResolver interface {
	// Store returns the store registered under instance, and whether one exists.
	Store(instance string) (storage.Store, bool)
}

StoreResolver maps a StorageReference.StorageInstance name to the storage.Store that holds its data. Wiring supplies it — typically MapStoreResolver over the deployment's registered stores. Kept as an interface so the engine never assumes one backend and tests can substitute fakes.

Canonical key (gh#376 coordination with semsource): StorageInstance is the storage COMPONENT INSTANCE NAME (e.g. "objectstore", "filestore-media"), per StorageReference's own doc ("identifies which storage component holds the data … enables federation across multiple storage instances") — NOT the bucket name. Producers stamp the component instance name; wiring registers each store under that same name. (semstreams' own auto-stamp is inconsistent today — component.go uses the instance name, store.go uses the bucket — tracked separately; this helper, the first consumer, fixes the convention at instance-name.)

type Unhydrated

type Unhydrated struct {
	// Handle is the opaque token for the seed that did not load. Never parse or
	// construct it.
	Handle string           `json:"handle"`
	Reason UnhydratedReason `json:"reason"`
}

Unhydrated names one seed that did not load.

The field is Handle, not ID, deliberately: fusion's contract keys everything a consumer touches by what a human reads or by an opaque token, never by a parseable entity ID (see Node.Handle). The value is the same string the batch wire calls `id` — the layer boundary is the point. Naming it `id` on the product surface would invite consumers to parse and construct entity IDs, which is exactly what the handle convention exists to prevent.

type UnhydratedReason

type UnhydratedReason string

UnhydratedReason is why one requested ID did not hydrate. The set is CLOSED and fusion's own: it is the product-facing contract, named `unhydrated` on fusion's wire. UnhydratedUnknown has no counterpart in the batch reply's missing reasons.

const (
	// UnhydratedNotFound is a read that did not find the key. It does NOT license the
	// conclusion that the entity never existed — see the Response.Unhydrated docs.
	UnhydratedNotFound UnhydratedReason = "not_found"
	// UnhydratedError is a per-ID fault that did not fail the whole call, such as
	// an entity whose stored value its responder refuses (fusionnats lists a
	// poisoned seed this way) or an empty ID.
	UnhydratedError UnhydratedReason = "error"
	// UnhydratedUnknown is synthesized when a requested ID appears in neither the
	// handler's entity list nor its missing list — a handler that under-reported.
	// Naming it beats inventing not_found, which would assert something unobserved.
	UnhydratedUnknown UnhydratedReason = "unknown"
)

The closed unhydrated-reason set.

type Want

type Want string

Want enumerates the optional facets a caller can request. Empty defaults to body plus immediate relations.

const (
	WantBody      Want = "body"      // verbatim source/passage
	WantRelations Want = "relations" // callers/callees, links/sections
	WantPaths     Want = "paths"     // bounded outgoing relation paths from the seeds
	WantImpact    Want = "impact"    // transitive reverse-relation closure of the seeds
	WantGraph     Want = "graph"     // lossless structured projection: typed facts + directed edges + evidence
)

The requestable facets.

Directories

Path Synopsis
Package fusionnats is the NATS implementation of fusion.RetrievalClient (ADR-062): it maps the engine's resolve and expand calls onto graph-query's public verbs, so the fusion engine can run against a live graph.
Package fusionnats is the NATS implementation of fusion.RetrievalClient (ADR-062): it maps the engine's resolve and expand calls onto graph-query's public verbs, so the fusion engine can run against a live graph.
Package fusionvocab is the production implementation of fusion.RankSignals (ADR-062 increment 5, gh#396): it wires the lens engine's ranking signals onto the vocabulary registry (predicate salience) and an ontology's ancestor function (class specificity).
Package fusionvocab is the production implementation of fusion.RankSignals (ADR-062 increment 5, gh#396): it wires the lens engine's ranking signals onto the vocabulary registry (predicate salience) and an ontology's ancestor function (class specificity).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL