Help us shape the future of Go! Take the 2026 Go Developer Survey

query

package
v0.9.17 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 28, 2026 License: MIT Imports: 42 Imported by: 0

Documentation

Overview

Package query provides read-only vault diagnostics and search operations.

Index

Constants

View Source
const (
	ConfidenceStrong   = noisefloor.ConfidenceStrong
	ConfidenceModerate = noisefloor.ConfidenceModerate
	ConfidenceWeak     = noisefloor.ConfidenceWeak
	// ConfidenceNoMatch — in noise-floor mode, the top hit is at/below the
	// embedder's noise floor (nothing relevant). In the RRF-gap fallback it
	// means the top results are essentially tied (no clear winner). Both read
	// as "don't commit to top-1".
	ConfidenceNoMatch = noisefloor.ConfidenceNoMatch
)

Confidence tier strings. The noisefloor package is the single source of truth (Principle 7); these are aliases so the rest of the query package can keep referring to query.ConfidenceX without a second literal set that could silently drift from the labels Ask now writes via noisefloor.Relevance.

View Source
const (
	// BackupStateNoRepo — the vault is not inside a git repository at all.
	BackupStateNoRepo = "no_repo"
	// BackupStateNoUpstream — versioned, but the branch tracks nothing. Every
	// commit exists on exactly one disk.
	BackupStateNoUpstream = "no_upstream"
	// BackupStateBehind — an upstream exists and local work has not reached it.
	BackupStateBehind = "behind"
	// BackupStateCurrent — the upstream has everything local has.
	BackupStateCurrent = "current"
)

Backup states, most severe first.

View Source
const (
	WarnBackupNoRepo = "this vault is not in a git repository — nothing is versioning it, " +
		"and an accidental delete is unrecoverable"
	WarnBackupNoUpstream = "this vault's branch tracks no remote — every commit exists only on this machine. " +
		"Set one with: git remote add origin <url> && git push -u origin <branch>"
	WarnBackupBehindFmt = "%d commit(s) have never left this machine, the oldest %d day(s) ago — " +
		"push, or accept losing that work if the disk does"
)

Backup warnings. Phrased as what is AT RISK, not as what is misconfigured: the reader needs to know what they lose, not which setting is unset.

View Source
const (
	IndexStatusCurrent = "current"
	IndexStatusStale   = "stale"

	// IndexStatusUnknown is what the field says when the reconciliation could
	// not run at all — an unreadable vault root, a config that will not load.
	//
	// A third state rather than defaulting to "current", for the reason this
	// whole file exists: "checked, and it is current" and "could not check" are
	// different facts, and collapsing them into the optimistic one is how a
	// field becomes a constant success signal. DoctorResult.ValidationSummary
	// already draws exactly this line — nil means "validation not run", &{0,0}
	// means "ran, found nothing" — and the same distinction belongs here.
	IndexStatusUnknown = "unknown"
)

IndexStatusCurrent and IndexStatusStale are the values of the index_status field on doctor and vault status. Constants because two packages render them and consumers branch on the string.

View Source
const (
	// StatusMeshAuthenticated: pinned root verified the registry AND the running
	// signer proved possession of the resolved binding key. The ONLY green state.
	StatusMeshAuthenticated = "authenticated"
	// StatusMeshSelfConsistentUnpinned: no pin; the daemon-advertised root
	// verified its own registry for self-consistency ONLY — NOT authenticated.
	StatusMeshSelfConsistentUnpinned = "self-consistent-unpinned"
	// StatusMeshNotEnrolled: pinned + registry verified, but the member slug has
	// no live binding (enroll-add pending).
	StatusMeshNotEnrolled = "not-enrolled"
	// StatusMeshKeyMismatch: pinned + binding resolved, but the running signer
	// does NOT hold the binding's key (wrong signer/key).
	StatusMeshKeyMismatch = "key-mismatch"
	// StatusMeshUnverifiable: pinned, but the registry failed to verify
	// (bad-sig/stale/rollback) or could not be fetched.
	StatusMeshUnverifiable = "unverifiable"
	// StatusMeshNoSignal: no mesh signal present (caller should not surface the
	// section at all — see HasSignal).
	StatusMeshNoSignal = "no-signal"
)

Tier-2 status values (SSOT). Each is a distinct, machine-readable verdict.

View Source
const (
	// DaemonModePlaintext: the daemon serves no well-known root (404).
	DaemonModePlaintext = "plaintext"
	// DaemonModeAdvisoryConfigured: a well-known root is served (registry
	// configured). Advisory only — verdict-gating is a return-true no-op today.
	DaemonModeAdvisoryConfigured = "advisory-configured"
	// DaemonModeUnknown: the daemon was unreachable, so its mode is unknown.
	DaemonModeUnknown = ""
)

Tier-3 daemon-mode values (SSOT). FACTUAL served-state labels only — they do NOT imply message-signature protection (enforcement is a no-op today).

View Source
const (
	WarnMeshKeyMode      = "identity key file is not 0600 (custody mode is wrong)"
	WarnMeshKeySize      = "identity key file is not the expected ed25519 private-key size"
	WarnMeshUnpinned     = "registry self-consistent (daemon-advertised root), NOT authenticated — enroll persists a pin, or pass --mesh-root-pubkey"
	WarnMeshNotEnrolled  = "your slug is not in the network registry yet (enroll-add pending?)"
	WarnMeshKeyMismatch  = "your binding exists but the running signer does not hold its key (wrong signer/key?)"
	WarnMeshUnverifiable = "the network registry did not verify against your pinned root (bad signature, stale, or rolled back)"
	// WarnMeshRegistryNotSigned: the bytes are not a signed-registry envelope
	// at all. Live 2026-09-22: agents.yaml (the roster) passed as the registry
	// was reported as a bad signature, which sends the reader hunting tampering.
	// WarnMeshRegistryDueSoonFmt: inside the last week before the DECLARED
	// hub bound. When it lapses the hub drops signed posts; it does not queue.
	WarnMeshRegistryDueSoonFmt = "the registry has %d day(s) left before the hub starts dropping signed posts — re-sign and deploy it now"
	WarnMeshRegistryNotSigned  = "the registry file is not a readable signed registry (truncated, corrupt, or not a registry at all — " +
		"agents.yaml is the roster, not the registry); point --mesh-registry at the file its registry_path names, " +
		"or fetch <daemon_url>/.well-known/vaultmind-directory"
	WarnMeshNoRegistry         = "no registry available to verify (daemon unreachable and no --mesh-registry given)"
	WarnMeshEnforcementOff     = "message-signature enforcement is NOT YET active (advisory mode is a no-op today)"
	WarnMeshHeartbeatStale     = "wake-watcher heartbeat is stale — the watcher may be present-but-dead"
	WarnMeshFreshnessUnchecked = "registry freshness UNCHECKED — no registry to read (declare registry_path in agents.yaml); staleness is UNKNOWN, not OK"
	// WarnMeshRegistryUnreadable: a path WAS declared and could not be read.
	// A typo'd or rotated registry_path is a config error; degrading it to
	// silence gives the same output as never having declared one.
	WarnMeshRegistryUnreadable = "the declared registry file could not be read — check registry_path in agents.yaml"
	// WarnMeshSelfConsistencyFailed is the UNPINNED twin of Unverifiable. The
	// pinned wording blames "your pinned root", which on this path the
	// operator does not have — two warnings in one section, one saying you
	// have no pin and the other blaming it.
	WarnMeshSelfConsistencyFailed = "" /* 136-byte string literal not displayed */
	// WarnMeshRegistryAging: older than doctor's conservative default, with no
	// declared hub bound to check against. Deliberately advisory: doctor cannot
	// know when the hub refuses unless the operator declares its bound, and a
	// present-tense "sends are refused" that is false most days trains people
	// to filter the one day it is true.
	WarnMeshRegistryAging = "" /* 173-byte string literal not displayed */
	// WarnMeshRegistryStale: the registry verifies against its root but is
	// older than the freshness bound. Named separately from Unverifiable
	// because the fix is different (re-sign and redeploy, not investigate a
	// bad signature) and because the SYMPTOM is invisible from inside: reads,
	// long-poll and the wake-watcher keep working while every signed send is
	// refused. That is how the 2026-09-06 mesh-wide mute stayed unnoticed for
	// a day with every liveness check green.
	// WarnMeshFreshnessUnchecked: no registry was available to read, so the
	// freshness question was never asked. Named because "disabled" and
	// "healthy" printed identically — the same reasoning that produced
	// WarnMeshHeartbeatUnresolved: a check that cannot run is not a pass.
	WarnMeshRegistryStale = "the registry is past its freshness bound — signed SENDS will be refused while reads keep working; re-sign and redeploy it"
	// WarnMeshHeartbeatUnresolved: the check could not run. Rendering silence
	// (or a filesystem verdict) here is how a seven-day-dead watcher reported
	// "not found" — the checker had no path and never looked.
	WarnMeshHeartbeatUnresolved = "watcher heartbeat path is UNRESOLVED (no agent slug) — watcher liveness is UNKNOWN, not OK"
	// WarnMeshWatcherNotRearmed: lastwake is newer than lastarm. The watcher
	// fired, the session was re-invoked, and nobody armed a successor — every
	// message since is landing in silence. This is the one check that fires on
	// the real 2026-08-16 death.
	WarnMeshWatcherNotRearmed = "the watcher woke and was never re-armed — you are not being woken for messages"
)

Warning strings (SSOT) — each drives both the human Warnings slice and an envelope.AddWarning in the cmd layer.

View Source
const (
	HeartbeatUnresolved = "unresolved" // no path — the check could not run
	HeartbeatAbsent     = "absent"     // path known; no file (watcher never armed)
	HeartbeatStale      = "stale"      // file present; mtime outside the window
	HeartbeatFresh      = "fresh"
)

Heartbeat states. Three distinct non-green diagnoses with three different fixes; collapsing any two is how the last one hid.

View Source
const (
	RuleBrokenReference = "broken_reference"
	RuleMissingRequired = "missing_required_field"
	RuleUnknownType     = "unknown_type"
	RuleInvalidStatus   = "invalid_status"
)

Validate rule identifiers — the SSOT for rule strings used across the query layer. validate.go is the canonical definition site; doctor.go and any other caller must reference these constants instead of inlining the strings.

View Source
const ArcTypeName = "arc"

ArcTypeName is the note type de-duplication compares proposals against. A proposal can only duplicate an ARC; resembling a principle or a reference says nothing about whether this transformation is already recorded.

View Source
const BackupStaleDays = 3

BackupStaleDays is when "behind" becomes worth saying out loud.

Zero unpushed commits is silent, and a handful from today is normal working state. The failure this guards against is the SLOW one: the remote quietly stops receiving and nobody notices for weeks.

View Source
const DefaultNearestArcs = 3

DefaultNearestArcs is how many similar existing arcs de-duplication surfaces per proposal. Three reveals a near-tie — itself the signal that the covered/new judgement is hard — without turning the report into a ranking exercise. Locked by arcfinder_ssot_test.go per the scoring-constant SSOT rule in AGENTS.md.

View Source
const DefaultRRFK = 60

DefaultRRFK is the Reciprocal Rank Fusion smoothing constant from the original RRF paper. Applied when HybridRetriever.K is zero-value. Tuning this value shifts every hybrid retrieval's ranking; it lives here so there's one home to reason about.

View Source
const EmbeddingModelMixed = "mixed"

DoctorEmbeddings reports the vault's semantic-retrieval readiness. Surfaces which embedding lanes are populated so a user can diagnose a keyword-only fallback at a glance without running an ask query and hitting zero hits.

HasModalityImbalance flags the failure mode where a BGE-M3 vault has dense embeddings but some notes are missing sparse or colbert. Under hybrid RRF this silently compresses ranking: a partially-covered note at rank 1 in 2 lanes loses to a ubiquitous rank-3 note across 4 lanes. Dense-only vaults (MiniLM) are never flagged — sparse/colbert don't apply to that model. EmbeddingModelMixed is DoctorEmbeddings.Model for an index holding more than one embedding model.

View Source
const WarnNonAuthoritativeCitationFmt = "%s cites %s as evidence, but type %q is declared non-authoritative — " +
	"raw material, not a source. Distil it into a reviewed note and cite that."

WarnNonAuthoritativeCitation is emitted when evidence is not allowed to be evidence. It names both notes, because "you have 3 bad citations" is not something anyone can act on.

Variables

View Source
var ErrNoEmbedder = errors.New("no embedder available: run 'vaultmind index --embed' on the arcs vault first")

ErrNoEmbedder reports that de-duplication cannot run because the vault holds no embeddings.

It is a distinct sentinel because it is NOT a fault: `vaultmind index` without `--embed` is a valid, common configuration, and the aid is an optional enhancement over it. Callers surface it to the reader (who must not conclude "nothing resembles this") without flagging the run as degraded — a warning that fires on an ordinary setup is noise, and noise is how a report teaches people to stop reading its warnings.

Functions

func BuildAutoRetriever

func BuildAutoRetriever(ctx context.Context, db *index.DB) (retrieval.Retriever, func(), error)

BuildAutoRetriever returns a hybrid retriever if embeddings exist, otherwise keyword. Embedder initialization failure falls back to keyword silently.

func BuildRetriever

func BuildRetriever(ctx context.Context, mode string, db *index.DB) (retrieval.Retriever, func(), error)

BuildRetriever creates the appropriate retriever for the given search mode. Returns a cleanup function that must be deferred if non-nil.

func CloseSharedEmbedder added in v0.9.15

func CloseSharedEmbedder()

CloseSharedEmbedder releases the process's model, if one is loaded.

func CollectTypeBreakdown added in v0.1.11

func CollectTypeBreakdown(db *index.DB, cfg *vault.Config) (map[string]StatusTypeInfo, error)

CollectTypeBreakdown returns the per-type note counts together with each type's required fields and valid statuses, drawn from the vault config's type registry. It is the single source of truth for the per-type breakdown surfaced by both `vault status` (cold-start) and `doctor` (health hub). A nil cfg yields an empty (non-nil) map so callers can range safely.

func CosineSimilarity

func CosineSimilarity(a, b []float32) float64

CosineSimilarity is retained as the query-layer name for the shared implementation in internal/embedding. One definition, so the two layers that need it cannot drift apart (ADR-009 forbids distill importing query).

func FormatAsk

func FormatAsk(result *AskResult, w io.Writer) error

FormatAsk writes a human-readable text representation of an AskResult.

func FormatAskExplain

func FormatAskExplain(result *AskResult, w io.Writer) error

FormatAskExplain is like FormatAsk but prints per-hit lane breakdowns (which sub-retrievers scored the note, each lane's raw 1/(K+rank), and how many lanes went into the mean). Lets you see the fusion math on the command line instead of piping --json through jq/python — closes the diagnostic gap that had operators rebuilding ad-hoc tooling for every ranking investigation.

func FormatAskPointersOnly

func FormatAskPointersOnly(result *AskResult, w io.Writer) error

FormatAskPointersOnly is like FormatAsk but skips body content for both the target note and every context-pack item — output is title + id + type only. Used by the SessionStart hook so the body of "what matters most right now" is never preloaded; the agent has to query for it.

This converts the dogfood rule (use vaultmind ask before answering) from honor-system discipline (manifesto principle 9: discipline does not survive time pressure) to design: every body-read becomes an explicit, logged activation event the agent had to choose, rather than something the preload silently satisfied. Closes the trap documented in the plasticity-gap arc and the 2026-04-25 design signal under step 3 of the plasticity roadmap.

Retrieval is unchanged — search hits, context-pack assembly, and scoring all happen normally. Only the rendering omits bodies. The hint at the bottom names the next move (an explicit ask) so the agent knows the loop closes by querying, not by waiting for more context.

func FormatAskPreview

func FormatAskPreview(result *AskResult, w io.Writer) error

FormatAskPreview renders each ranked hit with a one-line snippet from the note body underneath the title — bridging the gap between pointers-only (titles, no body context) and full-body Ask (3000+ tokens of context pack). Closes the AX gap named in the felt- experience inventory: with pointers-only I see ids and titles but often can't tell what a note actually is until I query its body. The snippet was already being populated by every retriever; this just renders it.

func FormatAskRead

func FormatAskRead(result *AskResult, note *index.FullNote, w io.Writer) error

FormatAskRead renders the ask menu (search header + ranked hits) plus the body of one specific note inline — the single-command shortcut for the probe→read workflow when the agent already knows which hit from the menu they want. Backs `vaultmind ask --read N` and `vaultmind ask --read <id>`. The note argument is the resolved chosen hit's full body; the caller fetches it (cmd/ask.go) so this renderer stays in the format layer without taking a DB dependency.

FormatAskRead always renders without explain. To get per-lane RRF math under each hit when --read and --explain are combined, use FormatAskReadWithOptions.

func FormatAskReadWithOptions

func FormatAskReadWithOptions(result *AskResult, note *index.FullNote, w io.Writer, explain bool) error

FormatAskReadWithOptions is the explain-aware form of FormatAskRead. When explain is true, each ranked hit in the menu shows its per-lane RRF contribution underneath — matching what `vaultmind ask --explain` renders in default mode. Round-3 inter-agent review caught that `--read N --explain` was silently dropping --explain because runAskRead short-circuited before the explain path was read; this is the rendering side of the fix.

func FormatGitStatus

func FormatGitStatus(result *GitStatusResult, w io.Writer) error

FormatGitStatus writes a human-readable summary to w.

func FuzzyTitleMatches

func FuzzyTitleMatches(query string, titles []index.NoteTitle, n int) []index.NoteTitle

FuzzyTitleMatches returns up to n titles whose words overlap with the query's words, ordered by overlap count desc (ties broken by shorter title first — more specific titles win). Titles with zero token overlap are excluded — a silent return beats a misleading suggestion.

Intended as a zero-hit fallback on ask: when retrieval returns nothing, suggest the nearest titles as a user nudge, not as retrieval results.

func MeshSurfacedCounts added in v0.2.0

func MeshSurfacedCounts(m *DoctorMeshIdentity) (errs, warns int)

MeshSurfacedCounts returns the error/warning counts the mesh section contributes to the doctor rollup. Per M4 + doctor convention, mesh issues are WARNINGS (exit stays 0): every section Warnings entry counts as one warning.

func NoteSimilarities

func NoteSimilarities(ctx context.Context, queryText string, embedder embedding.Embedder, db *index.DB) (map[string]float64, error)

NoteSimilarities embeds the query and computes cosine similarity against all stored note embeddings. Returns noteID -> similarity (0.0-1.0). Returns nil map if embedder is nil (keyword-only mode).

func PruneShownLedgers added in v0.9.13

func PruneShownLedgers(dir string, retention time.Duration, now time.Time)

PruneShownLedgers removes ledgers in dir untouched for longer than retention. A ledger is only read within a short window, so a finished conversation's file is dead weight; best effort, never an error.

func ResultSurfacedIssueCounts added in v0.2.0

func ResultSurfacedIssueCounts(result *DoctorResult) (errs, warns int)

ResultSurfacedIssueCounts is the result-level SSOT rollup: it folds the vault-issue counts (SurfacedIssueCounts over result.Issues) together with the mesh-identity section's contribution (MeshSurfacedCounts). Mesh issues are WARNINGS only (doctor keeps exit-0-on-warning; no new exit code), so a not-authenticated mesh state bumps the warning rollup without ever turning it into an error. The cmd-layer rollup line uses this so the text "Issues: E errors, W warnings" reflects the mesh section too.

func RunLinks(db *index.DB, cfg LinksConfig, w io.Writer) error

RunLinks executes a single-direction ("out" or "in") links query and renders it to w. The `--both` path is handled by the cmd layer via CollectBoth so it can emit ONE envelope rather than two concatenated ones.

func RunResolve

func RunResolve(db *index.DB, input, vaultPath string, jsonOut bool, w io.Writer) error

RunResolve executes the resolve command logic.

func RunSelf

func RunSelf(db *index.DB, cfg SelfConfig, w io.Writer) error

RunSelf renders the agent's memory state — recent / hot / stale. Reads only existing schema columns (access_count, last_accessed_at, title, type). Three sections:

  • Recent: notes touched most recently, regardless of count.
  • Hot: notes ranked by approximate ACT-R activation (ln(1+count) - d*ln(elapsed_hours)), capturing both frequency and recency in one number.
  • Stale: accessed notes whose last_accessed_at is older than the stale threshold, sorted by activation desc.

Empty vault prints "no accesses recorded yet" so the caller can tell blank-slate from rendering failure.

func SearchAndJudge added in v0.8.0

func SearchAndJudge(
	ctx context.Context,
	ret retrieval.Retriever,
	emb embedding.Embedder,
	db *index.DB,
	floors FloorResolver,
	queryText string,
	limit int,
) (hits []retrieval.ScoredResult, verdict string, relevanceZ float64, err error)

SearchAndJudge searches a vault ONCE and judges its top hit against that vault's own noise floor.

The single search is the point. The first federation searched every vault twice — once for the hits, once again inside the verdict pass — and the two call sites each read as reasonable in isolation. Measured across three vaults that doubling cost 6.7 seconds, taking a federated query to 15.9s and past the recall hook's 15s timeout: the hook was killed and injected nothing, silently, exit 0. The feature worked at the CLI and was dead in the one place it was built to help.

Returns an empty verdict when the vault has no embedder. Unmeasured is not no_match: the gate keeps such a vault rather than reading a missing measurement as evidence of irrelevance.

func SurfacedIssueCounts added in v0.2.0

func SurfacedIssueCounts(issues DoctorIssues) (errors, warnings int)

SurfacedIssueCounts returns the error and warning counts for the issues a doctor run actually SURFACES — the items in DoctorIssues that the human report renders as detail lines and that the --json envelope represents. It is the single source of truth for the "Issues: E errors, W warnings" text rollup so that count can never again disagree with --json.

It deliberately does NOT fold in DoctorResult.ValidationSummary (the schema VALIDATION aggregate from SummarizeValidationIssues). That aggregate counts findings — unknown_type, invalid_status — that doctor surfaces only in the nested result.validation_summary JSON field, never as a per-item line in the text report. (missing_required_field and broken_reference ARE surfaced — via result.issues and, for broken_reference, a dedicated text line.) Counting it in the text rollup overstated warnings (e.g. "0 errors, 96 warnings") against a --json envelope that surfaced none. result.validation_summary stays in --json unchanged; this helper only governs the surfaced-set rollup.

Severity follows the doctor renderer's own framing: integrity violations that mean data is wrong are errors; advisories that mean the vault is degraded-but-usable are warnings.

func Truncate

func Truncate(s string, maxLen int) string

Truncate shortens a string to maxLen runes, appending "..." if truncated.

func WriteEmbedderDownNotice added in v0.9.0

func WriteEmbedderDownNotice(w io.Writer, err error) bool

WriteEmbedderDownNotice says, in the output the agent reads, that semantic search is down and why — when the vault HAS embeddings but the model would not load. Reports whether it wrote anything.

It exists because the fallback used to be silent apart from a log line, and the only visible text was the keyword-only hint claiming the vault had no embeddings and prescribing a re-embed — which fails for the same reason (live, 2026-09-23: a stale ONNX runtime next to the binary).

func WriteKeywordOnlyHint

func WriteKeywordOnlyHint(w io.Writer, retrievalMode string, hitCount int) bool

WriteKeywordOnlyHint writes a user-facing diagnostic to w when the ask retrieval ran in keyword-only mode AND returned zero hits. That combination signals a vault without embeddings — paraphrase queries cannot match and the user has no other feedback explaining the silence. Reports whether the hint was written.

Silent on hybrid mode (different problem — real zero-hit) and silent when keyword search actually found results (user got what they asked for).

func WriteTitleSuggestions

func WriteTitleSuggestions(w io.Writer, matches []index.NoteTitle) bool

WriteTitleSuggestions renders a human-readable "did you mean?" block for the given matches. Silent (returns false, writes nothing) on empty input so callers can compose with other zero-hit diagnostics without adding blank sections.

Types

type ActivationReranker

type ActivationReranker struct {
	Base   retrieval.Retriever
	Score  ActivationScoreFn
	Alpha  float64 // weight on RRF score, normalized to [0, 1]
	Beta   float64 // weight on activation score, normalized to [0, 1]
	FetchN int     // candidates to fetch from base; rerank operates on these
}

ActivationReranker is slice 5b” — the post-RRF rerank that replaces slice 5b's 5th-lane approach. It wraps a base retriever (typically the 4-way HybridRetriever), takes the top-N candidates, and reorders them by blending RRF score with activation score.

The structural property: activation operates only on candidates that already cleared the base retriever. It cannot introduce notes from outside the candidate set, so the activation drown-out that broke slice 5b' (mean-of-present treating activation-only single-lane as equivalent to multi-query-lane) is impossible by construction.

See the activation-rerank design decision for the full probe sequence and the candidate-fix analysis that produced this design.

Algorithm:

  1. base.Search(query, FetchN) → N candidates with RRF scores
  2. For each candidate c: rrf_norm[c] = c.Score / max_rrf activation_raw[c] = Score(c.ID) activation_norm[c] = activation_raw[c] / max_activation final[c] = Alpha * rrf_norm[c] + Beta * activation_norm[c]
  3. Sort candidates by final desc.
  4. Return top-K (= the limit passed to Search).

Zero-value Alpha/Beta/FetchN fall back to defaults that preserve the base retriever's behavior — a caller constructing the reranker with no explicit knobs gets safe behavior (rrf-only effective when β=0).

func (*ActivationReranker) Search

func (r *ActivationReranker) Search(ctx context.Context, query string, limit, offset int, filters index.SearchFilters) ([]retrieval.ScoredResult, int, error)

Search fetches FetchN candidates from the base retriever, blends RRF score with activation score using Alpha/Beta weights, sorts by the combined score, and returns the top `limit` (with `offset` applied after sorting).

Implements retrieval.Retriever so the reranker can slot into any pipeline that expects a Retriever — including being further wrapped by future cross-vault federation rerankers.

type ActivationRetriever

type ActivationRetriever struct {
	DB     *index.DB
	ExpDB  *experiment.DB
	Params experiment.ActivationParams
	// Now is the reference time for activation decay. Nil means time.Now().
	// Tests set a fixed instant so ranking is deterministic rather than
	// depending on sub-millisecond timing between access events and scoring.
	Now func() time.Time
}

ActivationRetriever ranks notes by their ACT-R activation score (base-level activation + decay). It implements retrieval.Retriever and is intended to be a 5th lane in HybridRetriever's RRF combine — the first principled wiring of the reinforcement signal that internal/experiment has been collecting since 2026-04-29.

The retriever is query-independent: every Search call returns the same ranking regardless of `query`, because activation is a function of access history, not text relevance. The mean-of-present RRF fusion in HybridRetriever then only boosts notes that ALSO appear in at least one query-dependent lane — recently-accessed notes that happen to match the query rise; recently-accessed notes that don't stay where the query ranks them.

Notes with access_count = 0 are not returned; the lane's coverage matches the substrate's coverage. Filters (type, tag) are honored because the agent's mental model of "activation" is per-type (asking about concepts shouldn't surface recently-touched sources).

func (*ActivationRetriever) Search

func (r *ActivationRetriever) Search(_ context.Context, _ string, limit, offset int, filters index.SearchFilters) ([]retrieval.ScoredResult, int, error)

Search returns up to `limit` notes ranked by activation score, descending. Score is normalized to [0, 1] so it composes cleanly with other retrievers; the absolute scale is irrelevant under RRF (only ranks matter), but normalization keeps Components reporting consistent with the other lanes.

Implementation:

  1. Pull all accessed-note IDs from the index (access_count > 0).
  2. Hydrate metadata (title, type, path, is_domain) in one batch query.
  3. Apply type/tag filters before scoring (cheaper than scoring then dropping).
  4. Call experiment.ComputeBatchScores — which uses the experiment DB's note_access events for the full access-time history that ACT-R math needs.
  5. Sort by score, normalize, return.

type ActivationScoreFn

type ActivationScoreFn func(ids []string) (map[string]float64, error)

ActivationScoreFn returns the activation score for each requested note id. Notes with no access events return 0 (or are absent from the map — the reranker treats both the same). Decoupling the scoring function from experiment.ComputeBatchScores at the type boundary keeps the reranker testable without spinning up an experiment DB and lets future scorers (e.g. activation-with-similarity, decay-only, count-only) compose without changing the reranker.

type ArcFinder added in v0.4.0

type ArcFinder struct {
	// contains filtered or unexported fields
}

Finder answers "which existing arcs does this resemble?" by cosine similarity against the arcs' stored dense embeddings.

Two deliberate choices, both learned the hard way by running the first version against the real vault:

  1. Cosine, not the hybrid retriever's fused score. The 4-lane RRF score is built for ORDERING within one query; its magnitude is not a similarity. The first implementation used it and every neighbour printed "0.02" — three ties, no discrimination, a number that looks like evidence and isn't. This project already has an arc about exactly that mistake (RRF fusion scores are not cosines); it recurred here because a plausible API was reached for without checking what its numbers mean.
  2. Load the arc vectors ONCE. The first version ran a full hybrid search per proposal — 27 proposals took 2m34s, which is not a tool anyone runs twice. One embed per proposal against a preloaded matrix is the whole job.

func NewArcFinder added in v0.4.0

func NewArcFinder(db *index.DB, embedder embedding.Embedder) (*ArcFinder, error)

NewArcFinder loads the arc vectors from an already-open index. The caller owns the DB and the embedder — this type borrows both and closes neither, which keeps the ownership obvious at the one place that opened them.

A vault with no embedded arcs yields a finder that answers nothing: correct, and still cheap.

func (*ArcFinder) NearestArcs added in v0.4.0

func (f *ArcFinder) NearestArcs(ctx context.Context, text string, limit int) ([]ArcMatch, error)

NearestArcs returns the arcs most similar to text, best first, scored by cosine similarity in [0,1] — a number the reader can actually compare across proposals, which is the entire point of showing it.

type ArcMatch added in v0.4.0

type ArcMatch struct {
	ID    string
	Title string
	Score float64
}

ArcMatch is an existing arc that resembles a proposal, with the cosine that says how much. It is evidence for a human judgement, never the judgement.

type AskConfig

type AskConfig struct {
	Query            string
	Budget           int
	MaxItems         int
	SearchLimit      int
	ActivationScores map[string]float64
	// Embedder is optional. When non-nil, raw cosine similarities are computed
	// and used for spreading activation scoring via ActivationFunc.
	Embedder embedding.Embedder
	// ActivationFunc optionally recomputes activation scores after similarities
	// are known. When provided and similarities are available, the returned
	// scores replace ActivationScores for context-pack sorting. This enables
	// spreading activation without coupling query to the experiment package.
	ActivationFunc func(similarities map[string]float64) map[string]float64
	// NoiseFloor is the vault's noise-floor N (the cosine an off-topic query gets
	// to any note). When HasNoiseFloor is true and cosine similarities are
	// available, the top-hit confidence is derived honestly from N and the
	// dispersion below (see internal/noisefloor) instead of the legacy RRF gap.
	NoiseFloor float64
	// NoiseFloorSigma is the vault's note-to-note cosine dispersion σ — the scale
	// for z = (top_cosine − N)/σ. The caller passes a measured, clamped σ; when
	// zero/unset, Ask falls back to the embedder's DefaultDispersion.
	NoiseFloorSigma float64
	HasNoiseFloor   bool
	// VaultLowContrast marks a tight vault (high note-to-note μ) where even
	// correct top hits read "weak". Surfaced on the result so the formatter can
	// explain a weak label rather than let an agent misread it as "irrelevant".
	VaultLowContrast bool
	// SuppressOnNoMatch is the recall floor: when the top hit lands at/below the
	// noise floor (no_match), Ask returns before the context pack and the access
	// fan-out — so an ambient recall on an off-domain prompt neither injects
	// noise nor reinforces the irrelevant note it happened to surface. Opt-in
	// (the recall hook sets it); interactive ask leaves it false.
	SuppressOnNoMatch bool
	// PointersOnly and JSONOutput describe the OUTPUT MODE, which is what
	// decides whether text actually reached the caller. Without them the access
	// ledger inferred delivery from `Target.Body != ""` — that is packHasText,
	// not delivery: under --pointers-only the pack is assembled and withheld at
	// render, so every row was marked delivered for a command that handed over
	// nothing. See AskResult.DeliveredTo.
	PointersOnly bool
	JSONOutput   bool

	// ExcerptTokens caps a per-item excerpt used when a whole note body will not
	// fit the remaining budget. 0 keeps the released all-or-nothing packing,
	// where such an item contributes no text and the pack still counts it —
	// which is how a hook reports "3 items" while delivering only titles.
	ExcerptTokens int

	// ShownBefore names notes whose text this conversation received recently.
	// They stay in the pack as titles (see memory.ContextItem.ShownBefore);
	// their text is not sent again and their access is not a delivered read.
	ShownBefore map[string]bool
}

AskConfig holds parameters for the Ask compound operation.

type AskResult

type AskResult struct {
	Query            string                    `json:"query"`
	TopHits          []retrieval.ScoredResult  `json:"top_hits"`
	Context          *memory.ContextPackResult `json:"context,omitempty"`
	RetrievalMode    string                    `json:"retrieval_mode,omitempty"`
	TopHitConfidence string                    `json:"top_hit_confidence,omitempty"`
	Similarities     map[string]float64        `json:"-"` // raw cosine similarities (not serialized)
	// Noise-floor relevance (populated when HasNoiseFloor + similarities exist).
	// TopHitCosine is the raw cosine of the query against the top hit; NoiseFloor
	// is N; NoiseFloorSigma is σ; RelevanceZ = (TopHitCosine − N)/σ is the label's
	// derivation; RelevanceR = TopHitCosine − N is the raw cosine margin, kept for
	// agents that want the un-normalized number. All surfaced so an agent sees the
	// real values, not just the tier word.
	// NoiseFloorApplied is true when TopHitConfidence was derived from the
	// noise-floor relevance (not the RRF-gap fallback). The formatter uses it to
	// print honest "nothing relevant" / "z above noise floor" labels, and JSON
	// consumers use it to know the floats below are meaningful (rather than
	// zero-because-unset). Because it disambiguates, the floats are NOT omitempty
	// — a MiniLM floor of 0.0 or an exact-boundary z of 0.0 must still serialize
	// rather than silently vanish.
	NoiseFloorApplied bool    `json:"noise_floor_applied,omitempty"`
	TopHitCosine      float64 `json:"top_hit_cosine,omitempty"`
	NoiseFloor        float64 `json:"noise_floor"`
	NoiseFloorSigma   float64 `json:"noise_floor_sigma"`
	RelevanceR        float64 `json:"relevance_r"`
	RelevanceZ        float64 `json:"relevance_z"`
	// LowContrastVault is true when the vault is tight (high note-to-note μ), so
	// a "weak" top hit often means the best available correct match, not
	// "irrelevant". The formatter renders a one-line hint when it's set.
	LowContrastVault bool `json:"low_contrast_vault,omitempty"`
	// VaultNoteCount is how many notes the vault holds, used to decide whether a
	// relevance judgement is meaningful at all. Below noisefloor.MinCalibNotes
	// the floor is a shipped default calibrated for a large corpus, so a "weak"
	// or "nothing relevant" verdict states a measurement the vault is too small
	// to support. Zero means "not measured" and preserves the original
	// behaviour for any caller that doesn't populate it.
	VaultNoteCount int `json:"vault_note_count,omitempty"`

	// Federated, when non-empty, is the cross-vault merged ranking. TopHits
	// stays the OWNING vault's own list: the two answer different questions
	// ("what did this vault rank?" vs "what did the federation rank?") and
	// collapsing them would hide which vault's floor produced the verdict.
	Federated []FederatedHit `json:"federated,omitempty"`
	// Vault names the vault that delivered the body, set only under
	// federation. Without it "here is a note" gives no way to answer "from
	// where?", which is the first question a three-vault agent asks.
	Vault string `json:"vault,omitempty"`
	// FederatedVaults reports every vault that was SEARCHED and what its own
	// floor said, so a reader can tell an empty vault from an unsearched one.
	FederatedVaults []FederatedVaultStatus `json:"federated_vaults,omitempty"`
	// contains filtered or unexported fields
}

AskResult is the combined output of a search + context-pack operation. RetrievalMode reports which retriever lane actually ran ("hybrid" when embeddings exist and were used, "keyword" when ask fell back to FTS-only). Set by the caller after Ask returns, since Ask itself does not know which retriever it was handed — the caller (cmd/ask.go) has that signal. Surfacing it in the JSON envelope lets agent consumers detect keyword-only fallback programmatically (the human-readable hint prints on stdout and isn't available via --json).

TopHitConfidence is one of "strong" | "moderate" | "weak" | "no_match" | "". Its PRIMARY derivation is the band-normalized relevance RelevanceZ = (TopHitCosine − NoiseFloor)/NoiseFloorSigma (see internal/noisefloor and NoiseFloorApplied below): z measures how many vault-σ the top hit clears the off-topic noise floor by — which is the question the agent is actually asking, expressed on a scale that transfers across vault tightness. When no noise floor is available (HasNoiseFloor false), it FALLS BACK to the RRF-gap heuristic (computeTopHitConfidenceRRFGap), which measures candidate separation, not relevance, and so never yields "no_match". Read NoiseFloorApplied to tell the two apart.

func Ask

func Ask(ctx context.Context, retriever retrieval.Retriever, resolver *graph.Resolver, db *index.DB, cfg AskConfig) (*AskResult, error)

Ask searches the vault for the query, computes raw cosine similarities (when an embedder is available), recomputes activation scores with spreading activation (via ActivationFunc), then packs token-budgeted context around the top hit. Context-pack failure is non-fatal.

func AskHits

func AskHits(ctx context.Context, retriever retrieval.Retriever, query string, searchLimit int) (*AskResult, error)

AskHits is the search-only equivalent of Ask — runs the retriever and computes top-hit confidence, but skips context-pack assembly, activation re-scoring, and the RecordNoteAccess fan-out that Ask fires on (target + N neighbors).

KNOWN DIVERGENCE (follow-up): AskHits does not compute cosine similarities, so it cannot derive noise-floor confidence — it uses the RRF-gap fallback label. The same query therefore reads as "relevance: <tier> (R=…)" under `ask` but "top-hit confidence: <tier>" under `ask --read N`, with different semantics behind the same tier word. The header phrasing differs so the derivation is at least visible. Unifying them means computing the top hit's cosine here too (an extra embed+score per --read); deferred until that cost is justified. Used by callers that want the menu without committing to the top hit, e.g. `vaultmind ask --read 2` reads hit #2's body — so packing context around hit #1 would mis-attribute access events to a note the agent didn't read. The caller is responsible for firing access on whatever it chooses to read.

func (*AskResult) BodyDecision added in v0.7.0

func (r *AskResult) BodyDecision(callerAsked bool) (delivered bool, reason string)

func (*AskResult) DeliveredIDs added in v0.9.13

func (r *AskResult) DeliveredIDs() []string

DeliveredIDs is the notes whose text this answer carries — what a session ledger records as shown.

func (*AskResult) DeliveredTo added in v0.7.0

func (r *AskResult) DeliveredTo(callerAsked, jsonOutput bool) (delivered bool, reason string)

BodyDecision reports whether an ask result will hand the caller note BODIES, and when it will not, which rule withheld them.

The decision itself is not new — formatAskWithOptions has always auto-degraded a weak or no-match result to pointers. What is new is that the decision is a VALUE instead of a side effect on a local, so the formatter and the telemetry read the same one. Recomputing it in two places is how a log grows to disagree with what the agent was actually shown, and a metric built on that log would measure the disagreement rather than the delivery.

callerAsked is the caller's own --pointers-only. It wins, and it is recorded separately: "the hook asked for ids" and "the tool judged the hit too weak to show" are different facts with opposite remedies. DeliveredTo reports whether note TEXT actually reached the caller, given the output mode — which is the question telemetry needs and BodyDecision does not answer.

BodyDecision describes what the TEXT formatter will render. The JSON envelope serializes bodies regardless of --pointers-only, so the two diverge there: `ask --pointers-only --json` was measured returning a 5,770-character target body while telemetry recorded body_delivered=false, and IsActivationSignal then discarded a genuine read — reopening the phantom loop from the other side.

It also refuses to credit a delivery when no pack exists. On a context-pack error the result carries a nil Context, and the previous call site credited delivery anyway.

type AutoRetrieverResult

type AutoRetrieverResult struct {
	Retriever retrieval.Retriever
	Embedder  embedding.Embedder // nil when keyword-only (no embeddings)
	Cleanup   func()             // always non-nil; safe to defer unconditionally
	// EmbedderErr is WHY the embedder did not load when the vault has
	// embeddings but the model could not start (e.g. an ONNX runtime too old
	// for the binary). nil when hybrid is up, or when the vault simply has no
	// embeddings. Callers must say it out loud: the fallback is keyword-only,
	// and it used to be reported as "this vault has no embeddings".
	EmbedderErr error
}

AutoRetrieverResult holds the retriever, embedder, and cleanup from BuildAutoRetriever. Retriever is always non-nil. Embedder is nil in keyword-only mode (no embeddings). Cleanup is always safe to call unconditionally (no-op in keyword-only mode).

func BuildAutoRetrieverFull

func BuildAutoRetrieverFull(ctx context.Context, db *index.DB) AutoRetrieverResult

BuildAutoRetrieverFull is like BuildAutoRetriever but also exposes the embedder for computing raw cosine similarities (spreading activation).

func BuildAutoRetrieverWithActivation

func BuildAutoRetrieverWithActivation(ctx context.Context, db *index.DB, expDB *experiment.DB) AutoRetrieverResult

BuildAutoRetrieverWithActivation is BuildAutoRetrieverFull plus the activation lane (slice 5b' from the plasticity roadmap). When expDB is non-nil and the underlying retriever is hybrid, an ActivationRetriever is appended as a 5th RRF lane named "activation". Notes with access_count = 0 simply don't appear in this lane, so cold-start vaults degrade cleanly to 4-way RRF.

IMPORTANT calibration obligation: enabling activation shifts the rank-1/rank-2 score gap distribution that step-4's TopHitConfidence thresholds (5% / 2%) were calibrated against. Callers turning this on for the default ask path must re-probe the gap distribution and update the threshold constants in internal/query/format.go, or the strong/moderate/weak labels silently miscalibrate. See the priority- order doc for the full step-4 ↔ step-5 coupling.

func BuildAutoRetrieverWithRerank

func BuildAutoRetrieverWithRerank(ctx context.Context, db *index.DB, expDB *experiment.DB, alpha, beta float64) AutoRetrieverResult

BuildAutoRetrieverWithRerank returns a 4-way HybridRetriever wrapped in an ActivationReranker (slice 5b”). The reranker takes the top FetchN candidates from 4-way and reorders them by blending RRF score with activation score using Alpha/Beta weights.

Replaces BuildAutoRetrieverWithActivation (slice 5b' — 5th-lane variant) for the default activation-aware path. The lane variant stays on disk, unwired, as the documented escalation that didn't pan out — see the activation-rerank design decision.

IMPORTANT calibration obligation (carried over from 5b'): enabling this reranker shifts the rank-1/rank-2 score gap distribution that step-4's TopHitConfidence thresholds (5% / 2%) were calibrated against. The blended score's gap distribution differs from the raw RRF gap; callers turning this on for the default ask path must re-probe and update the threshold constants in internal/query/format.go, or the strong/moderate/ weak labels silently miscalibrate.

Zero-value alpha/beta fall back to defaults pinned 2026-05-03 from the α/β probe across {0.5/0.5, 0.7/0.3, 0.9/0.1, 0.95/0.05} — see the activation-rerank design decision for the data and rationale. Current defaults: α=0.9 / β=0.1.

type BadCitation added in v0.8.0

type BadCitation struct {
	NoteID     string `json:"note_id"`
	SourceID   string `json:"source_id"`
	SourceType string `json:"source_type"`
}

BadCitation is one source_ids edge pointing at a type that may not be cited.

func FindNonAuthoritativeCitations added in v0.8.0

func FindNonAuthoritativeCitations(db *index.DB, cfg *vault.Config) ([]BadCitation, error)

FindNonAuthoritativeCitations returns every source_ids edge whose target is a type the vault declares non-authoritative.

Returns an empty slice when no type is restricted, so a vault that has not opted in does no work and sees no warnings.

type BankRate added in v0.7.0

type BankRate struct {
	WindowDays int
	Added      int
	// Total is every note in the vault, so a bank count reads against the size
	// of the thing it is supposedly feeding.
	Total int
}

BankRate counts notes added to a vault in a window — the write side of memory, and the half no surface has ever reported.

Retrieval has had metrics since the experiment DB existed. Banking has had none, and the absence shows: this vault's research half took 2 notes in 90 days, both from one bulk import, while the sessions that should have fed it produced verified findings that went into commit messages instead. A number nobody prints is a habit nobody has.

Counted from the notes table's `created` frontmatter date rather than git or file mtime. It is the vault's own claim about when a note came into being, it survives moves and re-indexing, and it is already indexed — no new state, no shelling out.

The limit worth stating: a note back-dated by hand counts on its stated day, and a bulk import of 200 sources all created "today" reads as a banner day. The number answers "is anything being written down", not "was it earned".

func BankRateSince added in v0.7.0

func BankRateSince(db *index.DB, windowDays int) (*BankRate, error)

BankRateSince counts notes whose `created` date falls within windowDays.

type BothResult added in v0.1.11

type BothResult struct {
	Out struct {
		SourceID string          `json:"source_id"`
		Links    []graph.OutLink `json:"links"`
	} `json:"out"`
	In struct {
		TargetID string         `json:"target_id"`
		Links    []graph.InLink `json:"links"`
	} `json:"in"`
}

BothResult is the combined payload for a `--both` query: it carries the outbound and inbound directions in ONE structure so the cmd layer can wrap it in a single envelope (out before in) instead of emitting two concatenated envelopes (invalid JSON). Inbound links use the same "links" key as the standalone --in payload; the object nesting (result.in vs result.out) already disambiguates direction.

func CollectBoth added in v0.1.11

func CollectBoth(db *index.DB, cfg LinksConfig) (BothResult, string, error)

CollectBoth resolves cfg.Input and returns the combined out+in payload WITHOUT rendering. Used by the cmd layer to build a single `--both` envelope.

type CandidateSearcher added in v0.9.16

type CandidateSearcher interface {
	SearchAmong(ctx context.Context, query string, limit int, filters index.SearchFilters, ids []string) ([]retrieval.ScoredResult, int, error)
}

CandidateSearcher is a lane that can score a given set of notes instead of the whole vault. The hybrid retriever hands it the other lanes' results.

type ColBERTEmbedFunc

type ColBERTEmbedFunc func(ctx context.Context, text string) ([][]float32, error)

ColBERTEmbedFunc produces per-token ColBERT vectors for a query string.

type ColBERTRetriever

type ColBERTRetriever struct {
	DB           *index.DB
	EmbedColBERT ColBERTEmbedFunc
	Dims         int
}

ColBERTRetriever searches by MaxSim scoring between query and stored ColBERT token matrices.

func (*ColBERTRetriever) Search

func (r *ColBERTRetriever) Search(ctx context.Context, query string, limit, offset int, filters index.SearchFilters) ([]retrieval.ScoredResult, int, error)

Search embeds the query as per-token ColBERT vectors, computes MaxSim scoring against all stored ColBERT embeddings, and returns the top results sorted by score descending.

func (*ColBERTRetriever) SearchAmong added in v0.9.16

func (r *ColBERTRetriever) SearchAmong(ctx context.Context, query string, limit int, filters index.SearchFilters, ids []string) ([]retrieval.ScoredResult, int, error)

SearchAmong scores only the notes named in ids, reading only their vectors. The hybrid retriever passes the other lanes' results (see CandidateSearcher).

type ContentDrift

type ContentDrift struct {
	NoteID      string `json:"note_id"`
	Path        string `json:"path"`
	CurrentHash string `json:"current_hash"` // sha256(file content)
	StoredHash  string `json:"stored_hash"`  // notes.hash from index DB
}

ContentDrift describes one note whose current file content hash differs from the indexer's stored hash.

func DetectContentDrift

func DetectContentDrift(db *index.DB, vaultPath string) ([]ContentDrift, error)

DetectContentDrift compares each domain note's current file content hash against the indexer's stored hash and returns the notes whose content has changed since the last `vaultmind index` pass.

Hash-based, NOT mtime-based: git checkouts, branch switches, and other VCS operations bump file mtime without touching content. The prior mtime-based detector produced ~95% false positives on real vaults (385 of 407 notes after a routine `git checkout main`). sha256 over the full file gives precise content identity — only real content edits trigger drift.

Deleted files are NOT drift — they are orphaned index entries, and ReconcileIndex reports them as such. Unreadable files are still skipped: a per-file IO problem must not abort a health run. ORDER BY path gives deterministic output (the experiment framework consumes this JSON; stable order avoids spurious diffs).

Kept as a named function delegating to ReconcileIndex so the drift contract and its tests stay addressable, with one implementation underneath.

type DoctorAllResult added in v0.1.11

type DoctorAllResult struct {
	Rollup DoctorRollup    `json:"rollup"`
	Vaults []*DoctorResult `json:"vaults"`
	Failed []FailedVault   `json:"failed,omitempty"`
}

DoctorAllResult is the combined, single-envelope payload for `doctor --all`. It carries a workspace-level Rollup plus the per-vault DoctorResults. Each entry in Vaults is a full DoctorResult (which already carries its own vault_path), so consumers get one machine-readable value for the whole workspace — NOT one envelope per vault.

Failed carries every discovered vault that could not be opened or diagnosed: rather than silently dropping a corrupt vault (which would hide it from the operator), we surface it here by path and reason. The field is omitempty so a clean workspace emits no `failed` key. Reporting failures in this combined envelope — instead of a per-vault error envelope — preserves the single-envelope contract.

type DoctorBackup added in v0.8.0

type DoctorBackup struct {
	State string `json:"state"`
	// Upstream is the tracked remote branch ("origin/main"), empty when none.
	Upstream string `json:"upstream,omitempty"`
	// Unpushed counts commits the upstream does not have.
	Unpushed int `json:"unpushed"`
	// OldestUnpushedDays is the age of the oldest such commit. The COUNT alone
	// under-reports the risk: 3 commits from this hour and 3 from last month
	// are very different amounts of unrecoverable work.
	OldestUnpushedDays int      `json:"oldest_unpushed_days"`
	Warnings           []string `json:"warnings,omitempty"`
}

DoctorBackup reports whether the vault's work exists anywhere else.

func CheckBackup added in v0.8.0

func CheckBackup(vaultPath string, now time.Time) (*DoctorBackup, error)

CheckBackup inspects the git repository containing vaultPath.

A vault that is not in a repository is reported, not skipped: "no answer" and "the answer is fine" must not look alike, which is the whole reason this check exists.

func (*DoctorBackup) AtRisk added in v0.8.0

func (b *DoctorBackup) AtRisk() bool

AtRisk reports whether anything would be lost with this disk.

type DoctorCallerUse added in v0.7.0

type DoctorCallerUse struct {
	Caller     string `json:"caller"`
	Injections int    `json:"injections"`
	Consumed   int    `json:"consumed"`
}

DoctorCallerUse is one hook's slice of the rollup.

type DoctorEmbeddings

type DoctorEmbeddings struct {
	TotalNotes           int    `json:"total_notes"`
	DenseCount           int    `json:"dense_count"`
	SparseCount          int    `json:"sparse_count"`
	ColBERTCount         int    `json:"colbert_count"`
	Model                string `json:"model"` // "bge-m3", "minilm", "mixed", or "" when no dense embeddings
	SemanticReady        bool   `json:"semantic_ready"`
	HasModalityImbalance bool   `json:"has_modality_imbalance"`
	// MixedModel is non-nil when the vault has notes embedded with more than
	// one model (e.g. mid-upgrade from MiniLM to BGE-M3). Each entry pairs a
	// model name with its row count. When set, Model == "mixed". Surfacing
	// this explicitly prevents the silent-failure shape where doctor reports
	// "bge-m3" while half the rows are still MiniLM. See vaultmind#22.
	MixedModel []DoctorModelBreakdown `json:"mixed_model,omitempty"`
	// RuntimeError is set when the embeddings exist but the runtime that
	// would load their model fails to start — semantic search is down while
	// every count above looks healthy. Populated by the cmd layer.
	RuntimeError string `json:"runtime_error,omitempty"`
}

type DoctorIssues

type DoctorIssues struct {
	DuplicateIDs              int                `json:"duplicate_ids"`
	BrokenReferences          int                `json:"broken_references"`
	MissingRequiredFields     int                `json:"missing_required_fields"`
	MalformedMarkers          int                `json:"malformed_markers"`
	UnresolvedLinks           int                `json:"unresolved_links"`
	NotesMissingIDOrType      int                `json:"notes_missing_id_or_type"`
	UnresolvedLinkDetails     []UnresolvedLink   `json:"unresolved_link_details,omitempty"`
	ObsidianIncompatibleLinks int                `json:"obsidian_incompatible_links"`
	IncompatibleLinkDetails   []IncompatibleLink `json:"incompatible_link_details,omitempty"`
	PathPseudoIDLinks         int                `json:"path_pseudo_id_links"`
	PathPseudoIDDetails       []UnresolvedLink   `json:"path_pseudo_id_details,omitempty"`

	// StaleIndex counts domain notes whose current file content hash
	// differs from the indexer's stored hash — i.e. notes edited AFTER
	// the last `vaultmind index` pass. The operator-visible signal that
	// downstream artifacts (index, embeddings, marker sections) are out
	// of sync with the source. See DetectContentDrift for the detection
	// contract.
	//
	// Replaced an earlier mtime-based detector that produced ~95% false
	// positives on real vaults — git checkouts/branch switches/pulls
	// bump mtime without touching content, and the prior signal could
	// not distinguish "edited" from "VCS-touched". Hash comparison is
	// precise: only actual content edits trigger drift.
	StaleIndex        int            `json:"stale_index"`
	StaleIndexDetails []ContentDrift `json:"stale_index_details,omitempty"`

	// OrphanedEntries counts index rows whose file is gone from disk. Until the
	// next incremental pass sweeps them they answer queries and cannot be
	// opened. Previously invisible: the drift detector `continue`d on the read
	// error, and its own test was named DeletedFileSilent.
	OrphanedEntries      int             `json:"orphaned_entries"`
	OrphanedEntryDetails []OrphanedEntry `json:"orphaned_entry_details,omitempty"`

	// UnindexedFiles counts *.md files on disk with no index row — notes that
	// are in no query result. total_files cannot reveal them: that count comes
	// from the database, so a never-indexed note is not even a mismatch.
	// Symlinks the scanner refused are NOT counted here; see IndexTruth.Skipped.
	UnindexedFiles       int      `json:"unindexed_files"`
	UnindexedFileDetails []string `json:"unindexed_file_details,omitempty"`

	// DuplicateIDDetails names each contested id and every file claiming it.
	// The counter above it used to come from a SQL query against a UNIQUE
	// column, which is structurally zero; this comes from the disk, which is
	// the only place the collision is visible.
	DuplicateIDDetails []DuplicateOnDisk `json:"duplicate_id_details,omitempty"`

	// HookDrift counts Claude Code hook scripts in the project's
	// `.claude/scripts/` whose bytes differ from the embedded canonical
	// in `internal/hookscripts/`. Surfaces "the foundation has rotted"
	// — copies were edited, or the binary was upgraded but old copies
	// linger. Resolution: `vaultmind hooks install --force <project>`.
	// Populated by cmd/doctor.go (project dir comes from there); query
	// layer keeps the type but doesn't import internal/hooks (business
	// layer isolation per ADR-009).
	HookDrift        int      `json:"hook_drift"`
	HookDriftDetails []string `json:"hook_drift_details,omitempty"`

	// LegacyHooksJSON is true when `.claude/hooks.json` exists at the
	// project root. That standalone file is no longer recognized by
	// Claude Code 2.1.129+ — projects with it have silently broken
	// hooks. The fix is to migrate the contents into
	// `.claude/settings.json` under a top-level `hooks` key.
	// Live evidence from companion project dogfood 2026-05-06/07.
	// Populated by cmd/doctor.go (project dir comes from there); query
	// layer keeps the type but doesn't import internal/hooks per
	// ADR-009 business-business isolation.
	LegacyHooksJSON bool `json:"legacy_hooks_json"`
}

DoctorIssues holds counts of vault health issues.

type DoctorMemoryUse added in v0.7.0

type DoctorMemoryUse struct {
	WindowDays      int `json:"window_days"`
	Injections      int `json:"injections"`
	BodiesDelivered int `json:"bodies_delivered"`
	Consumed        int `json:"consumed"`
	NotesBanked     int `json:"notes_banked"`
	VaultNotes      int `json:"vault_notes"`
	// PerCaller is each hook's own verdict. The aggregate hides which of the
	// three injection points is earning its tokens, and they are different bets:
	// session-start, mid-task, and compaction.
	PerCaller []DoctorCallerUse `json:"per_caller,omitempty"`
}

DoctorMemoryUse is the memory-usage rollup doctor prints. Populated by the cmd layer (which owns the experiment-DB handle) so query.Doctor's signature stays stable — the same arrangement HookDrift and MeshIdentity use.

func (DoctorMemoryUse) ConsumedRate added in v0.7.0

func (m DoctorMemoryUse) ConsumedRate() float64

ConsumedRate is consumed/injections, 0 when nothing was surfaced.

type DoctorMeshIdentity added in v0.2.0

type DoctorMeshIdentity struct {
	// Tier 1 — identity custody (local, keyless).
	KeyPresent      bool `json:"key_present"`
	KeyModeOK       bool `json:"key_mode_ok"`
	KeySizeOK       bool `json:"key_size_ok"`
	SignerReachable bool `json:"signer_reachable"` // INFO — signer is on-demand

	// Tier 2 — binding resolves in the live mesh (authenticated).
	Pinned          bool   `json:"pinned"`
	Authenticated   bool   `json:"authenticated"` // M4 top-level boolean
	NetworkID       string `json:"network_id,omitempty"`
	BindingResolves bool   `json:"binding_resolves"`
	HoldsBindingKey bool   `json:"holds_binding_key"` // selfVerify proof-of-possession
	Status          string `json:"status"`

	// Tier 3 — chat reachability (honest labels).
	DaemonReachable       bool   `json:"daemon_reachable"`
	DaemonMode            string `json:"daemon_mode,omitempty"`
	EnforcementActive     bool   `json:"enforcement_active"` // always false today
	WatcherHeartbeatFresh bool   `json:"watcher_heartbeat_fresh"`
	WatcherHeartbeatAge   int    `json:"watcher_heartbeat_age_secs"`
	// WatcherHeartbeatState is one of the Heartbeat* constants. Fresh==true iff
	// State=="fresh" (both kept for back-compat readers of the JSON).
	WatcherHeartbeatState string `json:"watcher_heartbeat_state"`
	// WatcherHeartbeatPath is WHERE the check looked. Always carried: a verdict
	// about a file whose location is undisclosed cannot be audited, and an
	// undisclosed path is exactly how a wrong directory stayed wrong for months.
	WatcherHeartbeatPath string `json:"watcher_heartbeat_path,omitempty"`
	// WatcherRearmed is false when lastwake postdates lastarm (see checkRearm).
	WatcherRearmed bool `json:"watcher_rearmed"`

	// Warnings each also drive an envelope.AddWarning in the cmd layer.
	Warnings []string `json:"warnings,omitempty"`
}

DoctorMeshIdentity is the JSON-serializable mesh-health section. It is a POINTER on DoctorResult (nil ⇒ absent from --json) so the section appears only when a mesh signal exists.

func BuildMeshIdentity added in v0.2.0

func BuildMeshIdentity(ctx context.Context, in MeshDoctorInput) (*DoctorMeshIdentity, error)

BuildMeshIdentity runs the 3-tier Contract-B identity health check and returns the populated section. It NEVER reads the private key file (tier-1 is Lstat stat-only; the binding-key check is a keyless proof-of-possession via the signer). Authenticated is true ONLY on the pinned + resolves + selfVerify-pass path (M1+M3); every other state leaves Authenticated false (M4).

func (*DoctorMeshIdentity) HasSignal added in v0.2.0

func (m *DoctorMeshIdentity) HasSignal() bool

HasSignal reports whether any mesh signal exists, so the cmd layer can decide whether to attach the section at all (nil ⇒ absent from --json).

type DoctorModelBreakdown

type DoctorModelBreakdown struct {
	Model string `json:"model"` // "bge-m3", "minilm", or "unknown"
	Count int    `json:"count"`
}

DoctorModelBreakdown is one entry in DoctorEmbeddings.MixedModel.

type DoctorResult

type DoctorResult struct {
	VaultPath         string `json:"vault_path"`
	TotalFiles        int    `json:"total_files"`
	DomainNotes       int    `json:"domain_notes"`
	UnstructuredNotes int    `json:"unstructured_notes"`
	IndexStatus       string `json:"index_status"`
	// IndexStatusReason is set only when IndexStatus is "unknown": what stopped
	// the disk reconciliation from running. Absent otherwise.
	IndexStatusReason string                    `json:"index_status_reason,omitempty"`
	Embeddings        *DoctorEmbeddings         `json:"embeddings,omitempty"`
	Types             map[string]StatusTypeInfo `json:"types,omitempty"`
	ValidationSummary *StatusIssuesSummary      `json:"validation_summary,omitempty"`
	Issues            DoctorIssues              `json:"issues"`

	// MemoryUse is the "is this vault actually used" section: notes surfaced to
	// an agent, how many carried a body, how many were then read, and how many
	// notes were banked. A POINTER with omitempty, on the same principle as
	// ValidationSummary — nil means "not measured" (no usage log on this
	// machine), which must stay distinguishable from a measured zero.
	//
	// It lives on doctor rather than only in `experiment memory` because a
	// number behind a command nobody remembers to run is the same failure as a
	// pointer nobody follows. This is the health hub; this is health.
	MemoryUse *DoctorMemoryUse `json:"memory_use,omitempty"`

	// MeshIdentity is the Contract-B identity-health section. It is a POINTER
	// with omitempty so it is ABSENT from --json unless a mesh signal exists
	// (identity key file present, an anchor exists, a --mesh-* flag passed, or
	// the daemon is reachable). Populated by the cmd layer (paths come from xdg +
	// flags + env). nil ⇒ no mesh substrate, the section is omitted entirely.
	MeshIdentity *DoctorMeshIdentity `json:"mesh_identity,omitempty"`

	// Backup is the "does this vault exist anywhere but this disk" section.
	// Populated by the cmd layer, same arrangement as MeshIdentity. A vault is
	// the one artifact here that cannot be rebuilt, so this is health.
	Backup *DoctorBackup `json:"backup,omitempty"`

	// BadCitations are source_ids edges pointing at types the vault declares
	// non-authoritative — evidence that is not allowed to be evidence. Empty
	// unless the vault opted in by declaring a type non-authoritative.
	BadCitations []BadCitation `json:"bad_citations,omitempty"`
}

DoctorResult is the JSON-serializable output of the doctor command.

Types and ValidationSummary carry the per-type breakdown and raw validation aggregate that `vault status` used to produce. doctor is now the single health hub, so the cold-start view (`doctor --summary`) and the read-only diagnosis (`doctor`) both surface them. They are populated by the cmd layer (which holds the vault config + schema registry) via the shared status.go helpers — the same place HookDrift is populated — so query.Doctor's signature stays stable.

ValidationSummary is a POINTER with omitempty so the unmeasured state (a raw query.Doctor call with no schema registry — e.g. the query-layer tests) omits the field entirely. The earlier non-pointer struct always serialized a false-zero {errors:0,warnings:0}, indistinguishable from a measured-healthy vault. nil now means "validation not run"; a non-nil &{0,0} means "ran, found nothing". Types likewise stays omitempty (empty map => omitted).

ValidationSummary (JSON: "validation_summary") holds the RAW schema-validation aggregate — all ValidateResult.Issues bucketed by severity. It may be larger than the surfaced result.issues set because it includes unknown_type / invalid_status findings the text report never renders as per-item lines. These are distinct labeled axes: validation_summary is the raw aggregate; result.issues is the surfaced/actionable set.

func Doctor

func Doctor(db *index.DB, vaultPath string, reg *schema.Registry) (*DoctorResult, error)

Doctor runs vault health diagnostics against the indexed database.

When reg is non-nil, the missing_required_fields counter is populated by running the schema validator and summing missing-field issues. When reg is nil, the counter stays 0 (caller hasn't loaded a registry — usually a misconfiguration). The signature was extended 2026-05-04 to close the silent-failure shape where MissingRequiredFields was a declared output that never got populated.

type DoctorRollup added in v0.1.11

type DoctorRollup struct {
	VaultCount int `json:"vault_count"`
	Discovered int `json:"discovered"`
	Diagnosed  int `json:"diagnosed"`
	Failed     int `json:"failed"`
	TotalNotes int `json:"total_notes"`
	// TotalErrors/TotalWarnings are the SURFACED set — the sum of each vault's
	// query.ResultSurfacedIssueCounts, the same axis each per-vault report
	// prints — so the rollup totals reconcile with summing the per-vault
	// reports. They deliberately do NOT count the raw schema-validation
	// aggregate (see TotalRawValidationFindings), which is a different axis.
	TotalErrors   int `json:"total_errors"`
	TotalWarnings int `json:"total_warnings"`
	// TotalRawValidationFindings is the RAW schema-validation aggregate summed
	// across vaults: each vault's ValidationSummary.Errors+Warnings (nil ⇒ 0).
	// It is a DIFFERENT axis from TotalErrors/TotalWarnings — it includes
	// unknown_type / invalid_status findings the per-vault text report never
	// surfaces as per-item lines. Kept under its own distinctly-labeled key so
	// the two axes are explicit rather than collapsed, mirroring how the
	// single-vault envelope keeps validation_summary alongside the surfaced set.
	TotalRawValidationFindings int `json:"total_raw_validation_findings"`

	// VaultsWithIssues lists the paths of vaults with errors or warnings, in the
	// order the vaults were diagnosed (already deterministic — discovery sorts
	// them). Always non-nil so JSON consumers see [] for a clean workspace
	// rather than null.
	VaultsWithIssues []string `json:"vaults_with_issues"`
	// contains filtered or unexported fields
}

DoctorRollup summarizes the health of every vault discovered under the root: how many vaults, the total note count, the combined error/warning counts, and the paths of vaults that have at least one error or warning. It is the top-of-output signal an operator scans before reading per-vault detail.

The count fields are kept honest so a vault that failed to open can never hide behind a lower number: Discovered is the total found (Diagnosed + Failed), Diagnosed is how many produced a full report, and Failed is how many could not be opened. VaultCount is retained as an alias of Discovered for backward compatibility with existing JSON consumers.

func BuildDoctorRollup added in v0.1.11

func BuildDoctorRollup(vaults []*DoctorResult, failed []FailedVault) DoctorRollup

BuildDoctorRollup aggregates per-vault DoctorResults into a workspace rollup. The headline TotalErrors/TotalWarnings sum each vault's SURFACED counts (ResultSurfacedIssueCounts — the same axis every per-vault report prints), so `doctor --all` totals reconcile with summing the per-vault reports. A vault counts as "having issues" iff its surfaced errors+warnings > 0, keyed on the same axis. The RAW schema-validation aggregate is not discarded: it is summed separately into TotalRawValidationFindings (each vault's ValidationSummary.Errors+Warnings; a nil ValidationSummary — a raw, un-validated result — contributes zero and never panics). The failed slice (vaults that could not be opened) is folded into the honest count breakdown — Discovered = diagnosed + failed — so the rollup never under-reports the number of vaults found. Pure aggregation: no I/O, no mutation of inputs.

func (DoctorRollup) RawValidationGap added in v0.2.0

func (r DoctorRollup) RawValidationGap() int

RawValidationGap reports how many RAW schema-validation findings are NOT surfaced as per-item text lines across the workspace — the workspace-level analogue of the single-vault gap line. It is TotalRawValidationFindings minus the surfaced validation findings (MissingRequiredFields + BrokenReferences), never the full surfaced headline (which folds in non-validation items and would under-report the gap). A non-positive result means nothing is hidden.

type DuplicateOnDisk added in v0.7.0

type DuplicateOnDisk struct {
	ID    string   `json:"id"`
	Paths []string `json:"paths"`
}

DuplicateOnDisk names one id and every file claiming it.

type EmbeddingRetriever

type EmbeddingRetriever struct {
	DB       *index.DB
	Embedder embedding.Embedder
}

EmbeddingRetriever searches by cosine similarity between query and stored note embeddings.

func (*EmbeddingRetriever) Search

func (r *EmbeddingRetriever) Search(ctx context.Context, query string, limit, offset int, filters index.SearchFilters) ([]retrieval.ScoredResult, int, error)

Search embeds the query, computes cosine similarity against all stored embeddings, and returns the top results sorted by score descending.

type FTSRetriever

type FTSRetriever struct {
	DB *index.DB
}

FTSRetriever wraps SQLite FTS5 search as a Retriever.

func (*FTSRetriever) Search

func (r *FTSRetriever) Search(_ context.Context, query string, limit, offset int, filters index.SearchFilters) ([]retrieval.ScoredResult, int, error)

Search runs FTS5 search and count, converting results to ScoredResult.

type FailedVault added in v0.1.11

type FailedVault struct {
	VaultPath string `json:"vault_path"`
	Error     string `json:"error"`
}

FailedVault names a discovered vault that could not be opened or diagnosed, together with the reason. It is the surfaced form of what `doctor --all` used to silently skip: an operator (human or JSON consumer) sees the path and the error instead of the vault vanishing.

type FederatedHit added in v0.8.0

type FederatedHit struct {
	retrieval.ScoredResult
	// Vault is the name of the vault this note was ranked highest in. One
	// owner per note: delivery, access recording and plasticity all follow it.
	Vault string `json:"vault"`
	// RRFScore is the fused rank score. Deliberately separate from Score,
	// which stays the ORIGINATING vault's own number — overwriting it would
	// hide the per-vault evidence behind a cross-vault artifact.
	RRFScore float64 `json:"rrf_score"`
	// Ranks records the 1-based rank this note achieved in each vault that
	// returned it, so an amplified hit can show its work.
	Ranks map[string]int `json:"ranks,omitempty"`
}

FederatedHit is a merged result carrying the vault it came from, so both the agent and the delivery path can answer "which vault owns this?".

func FederatedSearch added in v0.8.0

func FederatedSearch(ctx context.Context, sources []VaultSource, query string, limit int) ([]FederatedHit, error)

FederatedSearch fans a query out across sources and returns one merged ranked list. A source that ERRORS is reported, never silently skipped: a federation that quietly drops a vault answers "nothing relevant" while having failed to look, which is the exact failure this feature exists to end.

func MergeFederated added in v0.8.0

func MergeFederated(perVault map[string][]retrieval.ScoredResult, verdicts map[string]string, relevance map[string]float64) []FederatedHit

MergeFederated fuses per-vault results that the CALLER has already collected, gating by each vault's own verdict and breaking rank ties on relevance.

Taking collected results rather than live retrievers is not a style choice: the ORT/hugot embedder allows ONE session per process, so a federation that holds every vault open at once leaves all but the first on keyword search — silently, while reporting those vaults as "unmeasured". The caller searches one vault at a time and releases each embedder before opening the next.

type FederatedVaultStatus added in v0.8.0

type FederatedVaultStatus struct {
	Name        string `json:"name"`
	Verdict     string `json:"verdict,omitempty"`
	Contributed bool   `json:"contributed"`
	// NoHits marks a vault the search returned nothing from. It is kept apart
	// from an empty Verdict, which means unmeasured (no embedder): with --type
	// or --tag, a vault holding no such notes is ordinary, not broken.
	NoHits bool `json:"no_hits,omitempty"`
}

FederatedVaultStatus records what happened to one vault in a federated query: was it searched, what did its OWN floor say, did it contribute.

Reported because "federated across 1 vault" while searching three is a lie of omission: "we only looked in one" and "we looked in three and two had nothing" are different facts about your memory, and only one of them means you should go write something down.

type FilteredRetriever added in v0.9.11

type FilteredRetriever struct {
	Base    retrieval.Retriever
	Filters index.SearchFilters
}

FilteredRetriever applies fixed type and tag filters to every search. A command scopes its retriever once instead of threading filters through each path that searches — ask alone searches from the direct, --read and federated paths. A filter the caller passes explicitly takes precedence.

func (FilteredRetriever) Search added in v0.9.11

func (f FilteredRetriever) Search(ctx context.Context, query string, limit, offset int, filters index.SearchFilters) ([]retrieval.ScoredResult, int, error)

Search runs the base search with the fixed filters filling any the caller left empty.

type FloorResolver added in v0.8.0

type FloorResolver func(dims int) (floor, sigma float64)

FloorResolver supplies a vault's calibrated noise floor and dispersion for a given embedding dimensionality. Taken as a function because the calibration lives in the experiment DB, which is the caller's concern.

type GitStatusResult

type GitStatusResult struct {
	RepoDetected     bool     `json:"repo_detected"`
	Branch           string   `json:"branch"`
	Detached         bool     `json:"detached"`
	MergeInProgress  bool     `json:"merge_in_progress"`
	RebaseInProgress bool     `json:"rebase_in_progress"`
	WorkingTreeClean bool     `json:"working_tree_clean"`
	StagedFiles      []string `json:"staged_files"`
	UnstagedFiles    []string `json:"unstaged_files"`
	UntrackedFiles   []string `json:"untracked_files"`
}

GitStatusResult is the JSON response for the git status command.

func GitStatus

func GitStatus(detector git.RepoStateDetector, vaultPath string) (*GitStatusResult, error)

GitStatus detects git repository state and returns the result.

type HybridRetriever

type HybridRetriever struct {
	Retrievers []retrieval.NamedRetriever
	K          int // RRF smoothing constant; zero-value falls back to DefaultRRFK
}

HybridRetriever fuses results from N retrievers using Reciprocal Rank Fusion. Sub-retrievers are named so each note's Components map reports which sub-retriever contributed what — useful for studying the 4-way RRF contribution ("what did FTS add here vs dense?") during research.

Fusion uses mean-of-present RRF: a note's score is the mean of its 1/(K+rank) contributions across the lanes where it appeared. Absence from a lane is treated as missing data, not a zero score. This is the fix for the 2026-04-24 partial-coverage compression bug where newly- added notes missing sparse/colbert embeddings lost to ubiquitous- mediocre competitors even when they were rank 1 where they scored.

In small vaults (fetchLimit ≥ note count) this is unambiguously correct: absence from a lane's results means the modality wasn't computed for that note. In very large vaults (where fetchLimit truncates a lane's tail) absence could also mean "ranked below cutoff" — a different failure mode worth separate attention, not solved here.

func (*HybridRetriever) Search

func (h *HybridRetriever) Search(ctx context.Context, query string, limit, offset int, filters index.SearchFilters) ([]retrieval.ScoredResult, int, error)

Search runs all retrievers concurrently, then fuses their ranked lists via RRF.

type InResult added in v0.1.11

type InResult struct {
	TargetID string         `json:"target_id"`
	Links    []graph.InLink `json:"links"`
}

InResult is the JSON-serializable payload for an inbound-links (backlinks) query. TargetID is the queried note; Links are the edges pointing at it.

func CollectIn added in v0.1.11

func CollectIn(db *index.DB, noteID, edgeType string) (InResult, error)

CollectIn returns the inbound-links (backlinks) payload for noteID WITHOUT rendering.

type IncompatibleLink struct {
	SourceID     string `json:"source_id"`
	SourcePath   string `json:"source_path"`
	TargetRaw    string `json:"target_raw"`
	SuggestedFix string `json:"suggested_fix"`
}

IncompatibleLink describes a wikilink that resolves in VaultMind but not in Obsidian. Obsidian resolves [[target]] by matching target against filenames (without extension). If dst_raw uses a note's title instead of its filename stem, Obsidian won't find it.

type IndexTruth added in v0.7.0

type IndexTruth struct {
	// Drifted: row and file both exist, content differs. The note was edited
	// after the last index pass, so the index, the embeddings and any rendered
	// marker sections describe an older version of it.
	Drifted []ContentDrift `json:"drifted,omitempty"`

	// Orphaned: row exists, file does not. Until the next incremental pass
	// sweeps it, the note answers queries and cannot be opened.
	Orphaned []OrphanedEntry `json:"orphaned,omitempty"`

	// Unindexed: file exists, no row. Invisible to every query, and invisible
	// to total_files too — that count comes from the database, so a
	// never-indexed note does not even show up as a mismatch.
	Unindexed []string `json:"unindexed,omitempty"`

	// Duplicate: one id claimed by two or more files on disk. Not a database
	// question — notes.id is UNIQUE, so at most one of them is ever stored and
	// the SQL check is structurally incapable of seeing the collision. Only the
	// disk can answer it.
	Duplicate []DuplicateOnDisk `json:"duplicate,omitempty"`

	// Skipped: *.md symlinks the scanner refused to follow. Reported so they
	// are distinguishable from Unindexed — a symlink is a deliberate exclusion,
	// and counting it as staleness would make every vault holding one
	// permanently unhealthy. That is how a signal gets ignored, then disabled.
	Skipped []string `json:"skipped,omitempty"`
}

IndexTruth answers one question — does the index still describe what is on disk? — in every way it can be false.

One reconciliation rather than one check per symptom, because the symptoms were three separate checks that could not fail: index_status was a string literal in two constructors, drift detection swallowed the read error for a deleted file, and duplicate_ids ran GROUP BY … HAVING COUNT(*) > 1 against a column declared UNIQUE. Each was a constant success signal that automation branched on. Fixing them apart would have produced three more checks whose failure modes nobody reasoned about together.

func ReconcileIndex added in v0.7.0

func ReconcileIndex(db *index.DB, vaultPath string) (*IndexTruth, error)

ReconcileIndex compares the index against the vault on disk.

Every file is read exactly once — the content hash (drift) and the frontmatter id (duplicates) both come from that read.

func (IndexTruth) IsCurrent added in v0.7.0

func (t IndexTruth) IsCurrent() bool

IsCurrent reports whether the index describes the vault.

Skipped is deliberately not consulted: a refused symlink is a decision, not a discrepancy.

func (IndexTruth) Status added in v0.7.0

func (t IndexTruth) Status() string

Status renders IsCurrent as the index_status field's value.

type LinksConfig

type LinksConfig struct {
	Input      string
	Direction  string // "out", "in", or "both"
	EdgeType   string
	JSONOutput bool
	VaultPath  string
	IndexHash  string
}

LinksConfig holds parameters for links out/in operations.

type MeasuredCalibration

type MeasuredCalibration struct {
	NoiseFloor       float64
	NoiseFloorProbes int
	ProbeSetVersion  int
	NTNCosineMu      float64
	NTNCosineSigma   float64
	NTNSampleCount   int
	NoteCount        int
	EmbeddingDims    int
}

MeasuredCalibration is the raw per-vault calibration measured from the embedding space. The cmd layer stamps it with an id + timestamp and stores it as an experiment.CalibrationSnapshot.

func MeasureNoiseFloor

func MeasureNoiseFloor(ctx context.Context, embedder embedding.Embedder, db *index.DB) (*MeasuredCalibration, error)

MeasureNoiseFloor measures a vault's noise floor N and note-to-note cosine dispersion from its stored dense embeddings. N is the max cosine any of the fixed off-topic probe strings reaches against any note — the cosine an off-domain query gets, which the relevance metric R = top_cosine - N is measured against. Requires an embedder (for the probes) and embedded notes.

type MeshDaemonClient added in v0.2.0

type MeshDaemonClient interface {
	FetchRoot(ctx context.Context) (doctorclient.WellKnownRoot, error)
	FetchDirectory(ctx context.Context) ([]byte, error)
	Whoami(ctx context.Context) (bool, string, error)
}

MeshDaemonClient is the loopback-pinned daemon-probe seam. doctorclient.Client satisfies this.

type MeshDoctorInput added in v0.2.0

type MeshDoctorInput struct {
	// Tier-1 custody (STAT-ONLY — never read).
	KeyPath    string
	SocketPath string

	// Tier-2 authenticity.
	PinnedRootPub ed25519.PublicKey // nil ⇒ UNPINNED path (never green)
	NetworkID     string
	RegistryBytes []byte // offline registry override (--mesh-registry); else fetched
	// MaxStaleness is the DECLARED hub bound. Zero means undeclared, in which
	// case doctor falls back to its conservative default and says so rather
	// than asserting a refusal date it cannot know.
	MaxStaleness time.Duration
	// RegistryUnread carries a DECLARED registry path that could not be read,
	// so the cmd layer's failure reaches the section instead of vanishing.
	RegistryUnread string
	Slug           string
	Signer         MeshSigner // keyless proof-of-possession

	// Tier-3 reachability.
	Daemon        MeshDaemonClient
	HeartbeatPath string
	// LastwakePath / LastarmPath feed the re-arm check; either empty disables it.
	LastwakePath string
	LastarmPath  string

	// Clock seam.
	Now time.Time
}

MeshDoctorInput carries everything BuildMeshIdentity needs. The cmd layer resolves paths (xdg + flags + env), the anchor pin, the slug, and constructs the signer + loopback daemon client.

type MeshSigner added in v0.2.0

type MeshSigner interface {
	Sign(canonicalBytes []byte) ([]byte, error)
}

MeshSigner is the keyless signing seam: doctor asks the SIGNER (over the UDS) to sign a challenge; it NEVER reads the private key file itself. signer.Client satisfies this.

type MgetResult

type MgetResult struct {
	Notes    []index.FullNote `json:"notes"`
	NotFound []string         `json:"not_found"`
	Total    int              `json:"total"`
}

MgetResult is the response for batch note reads.

func Mget

func Mget(db *index.DB, ids []string, frontmatterOnly bool) (*MgetResult, error)

Mget fetches multiple notes by ID. If frontmatterOnly is true, body/headings/blocks are omitted from each note.

type NeighborEdge

type NeighborEdge struct {
	SourceID   string  `json:"source_id"`
	EdgeType   string  `json:"edge_type"`
	Confidence string  `json:"confidence"`
	Weight     float64 `json:"weight"`
}

NeighborEdge describes the edge by which a neighbor was reached.

type NeighborNode

type NeighborNode struct {
	ID       string        `json:"id"`
	Distance int           `json:"distance"`
	EdgeFrom *NeighborEdge `json:"edge_from,omitempty"`
}

NeighborNode represents a single node in the neighbors result.

type NeighborsResult

type NeighborsResult struct {
	StartID         string         `json:"start_id"`
	Nodes           []NeighborNode `json:"nodes"`
	MaxNodesReached bool           `json:"max_nodes_reached"`
}

NeighborsResult holds the result of a neighbors traversal query.

func Neighbors

func Neighbors(resolver *graph.Resolver, input string, depth int, minConfidence string, maxNodes int) (*NeighborsResult, error)

Neighbors performs a BFS traversal from the given input (resolved via the Resolver's entity resolution) and returns the result in a format suitable for JSON output.

type NoteGetConfig

type NoteGetConfig struct {
	Input           string
	FrontmatterOnly bool
	JSONOutput      bool
	VaultPath       string
}

NoteGetConfig holds parameters for the note get operation.

type NoteGetOutcome added in v0.7.0

type NoteGetOutcome struct {
	// NoteID is the RESOLVED id, empty when the input did not resolve.
	NoteID string
	// BodyDelivered is whether note text was actually rendered — false for
	// --frontmatter-only, for misses, and for ambiguous input.
	BodyDelivered bool
}

NoteGetOutcome reports what a note get actually did, so the caller can log it instead of guessing.

The caller used to log args[0] — the RAW INPUT — because RunNoteGet resolved internally and returned only an error. A title- or path-resolved read therefore landed on a key no lookup can match, and `note get` is the highest-signal event the tool emits. It also hardcoded "a body was delivered", which --frontmatter-only and every miss falsified.

func RunNoteGet

func RunNoteGet(db *index.DB, cfg NoteGetConfig, w io.Writer) (NoteGetOutcome, error)

RunNoteGet executes the note get logic and reports what it delivered.

type OrphanedEntry added in v0.7.0

type OrphanedEntry struct {
	NoteID string `json:"note_id"`
	Path   string `json:"path"`
}

OrphanedEntry names an index row whose file is gone from disk.

type OutResult added in v0.1.11

type OutResult struct {
	SourceID string          `json:"source_id"`
	Links    []graph.OutLink `json:"links"`
}

OutResult is the JSON-serializable payload for an outbound-links query. SourceID is the queried note; Links are the edges leaving it.

func CollectOut added in v0.1.11

func CollectOut(db *index.DB, noteID, edgeType string) (OutResult, error)

CollectOut returns the outbound-links payload for noteID WITHOUT rendering. Split from rendering so the cmd layer can aggregate directions into one envelope for the `--both` JSON path.

type SearchConfig

type SearchConfig struct {
	Query      string
	Limit      int
	Offset     int
	TypeFilter string
	TagFilter  string
	JSONOutput bool
	VaultPath  string
}

SearchConfig holds search parameters.

type SearchResult

type SearchResult struct {
	Query  string                   `json:"query"`
	Offset int                      `json:"offset"`
	Limit  int                      `json:"limit"`
	Hits   []retrieval.ScoredResult `json:"hits"`
	Total  int                      `json:"total"`
}

SearchResult is the JSON response for search.

func RunSearch

func RunSearch(retriever retrieval.Retriever, cfg SearchConfig, w io.Writer) (*SearchResult, error)

RunSearch executes the search command logic and returns the result for downstream use (e.g. experiment logging). Rendering is still written to w.

type SelfConfig

type SelfConfig struct {
	Limit          int
	StaleThreshold time.Duration
	DecayD         float64
	Now            time.Time
}

SelfConfig holds parameters for the self-introspection view.

func SelfDefaults

func SelfDefaults(cfg SelfConfig) SelfConfig

SelfDefaults fills zero-value SelfConfig fields with sensible defaults.

type ShownLedger added in v0.9.13

type ShownLedger struct {
	// contains filtered or unexported fields
}

ShownLedger records which notes' text one conversation received, so a note asked for again within a short window is sent as its title instead of its text a second time. Measured before it was built: 82% of the reach hook's delivered bodies in a conversation were repeats, 68% of them within five minutes.

Keyed on the harness conversation id. A subagent gets a fresh id, so it is never deduplicated against its parent; without an id nothing is recorded.

func OpenShownLedger added in v0.9.13

func OpenShownLedger(dir, sessionID string) ShownLedger

OpenShownLedger names the ledger for sessionID under dir. The id is reduced to a safe file name, so it cannot point outside dir.

func (ShownLedger) Recent added in v0.9.13

func (l ShownLedger) Recent(window time.Duration, now time.Time) map[string]bool

Recent returns the notes shown within window before now. A missing or partly unreadable ledger yields what can be read: deduplication is a saving, never a reason to fail an answer.

func (ShownLedger) Record added in v0.9.13

func (l ShownLedger) Record(ids []string, now time.Time) error

Record appends ids as shown at now.

type SparseEmbedFunc

type SparseEmbedFunc func(ctx context.Context, text string) (map[int32]float32, error)

SparseEmbedFunc produces a sparse vector for a query string.

type SparseRetriever

type SparseRetriever struct {
	DB          *index.DB
	EmbedSparse SparseEmbedFunc
}

SparseRetriever searches by sparse dot-product between query and stored sparse embeddings.

func (*SparseRetriever) Search

func (r *SparseRetriever) Search(ctx context.Context, query string, limit, offset int, filters index.SearchFilters) ([]retrieval.ScoredResult, int, error)

Search embeds the query as a sparse vector, computes dot-product similarity against all stored sparse embeddings, and returns the top results sorted by score descending.

type StatusIssuesSummary

type StatusIssuesSummary struct {
	Errors   int `json:"errors"`
	Warnings int `json:"warnings"`
}

StatusIssuesSummary holds aggregated issue counts.

func SummarizeValidationIssues added in v0.1.11

func SummarizeValidationIssues(db *index.DB, reg *schema.Registry) (StatusIssuesSummary, error)

SummarizeValidationIssues runs the schema validator and rolls its issues up into error/warning counts. Shared by `vault status` and `doctor` so both report the same rollup (SSOT). A nil reg yields a zero-value summary.

type StatusResult

type StatusResult struct {
	VaultPath         string `json:"vault_path"`
	TotalFiles        int    `json:"total_files"`
	DomainNotes       int    `json:"domain_notes"`
	UnstructuredNotes int    `json:"unstructured_notes"`
	IndexStatus       string `json:"index_status"`
	// IndexStatusReason is set only when IndexStatus is "unknown": what stopped
	// the disk reconciliation. Absent otherwise. Mirrors DoctorResult so the two
	// commands report an unavailable check the same way.
	IndexStatusReason string `json:"index_status_reason,omitempty"`
	// IndexStale is meaningful ONLY when IndexStatus is "current" or "stale".
	// A bool cannot express "not checked", so when IndexStatus is "unknown"
	// this field is false for lack of a third state, NOT because the index is
	// fresh. Branch on IndexStatus first.
	IndexStale    bool                      `json:"index_stale"`
	Types         map[string]StatusTypeInfo `json:"types"`
	IssuesSummary StatusIssuesSummary       `json:"issues_summary"`
}

StatusResult is the JSON response for vault status.

func VaultStatus

func VaultStatus(db *index.DB, vaultPath string, cfg *vault.Config, reg *schema.Registry) (*StatusResult, error)

VaultStatus combines doctor, schema, and validation into a single cold-start response.

type StatusTypeInfo

type StatusTypeInfo struct {
	Count    int      `json:"count"`
	Required []string `json:"required"`
	Statuses []string `json:"statuses"`
}

StatusTypeInfo describes a note type with its count and schema.

type UnresolvedLink struct {
	SourceID   string `json:"source_id"`
	SourcePath string `json:"source_path"`
	TargetRaw  string `json:"target_raw"`
}

UnresolvedLink describes a single unresolved link with source and target info.

type ValidateIssue

type ValidateIssue struct {
	Path     string `json:"path"`
	ID       string `json:"id"`
	Severity string `json:"severity"`
	Rule     string `json:"rule"`
	Message  string `json:"message"`
	Field    string `json:"field,omitempty"`
	Value    string `json:"value,omitempty"`
	// BrokenRefs carries the reference ids that did not resolve, for
	// RuleBrokenReference. It exists so a caller can group by TARGET rather than
	// by source file: eight broken references across two vaults turned out to be
	// one missing note referenced four times plus four unrelated problems, and
	// the per-file view made that look like eight separate things.
	BrokenRefs []string `json:"broken_refs,omitempty"`
}

ValidateIssue represents a single validation finding.

type ValidateResult

type ValidateResult struct {
	FilesChecked int             `json:"files_checked"`
	Valid        int             `json:"valid"`
	Issues       []ValidateIssue `json:"issues"`
}

ValidateResult is the JSON-serializable output of frontmatter validate.

func Validate

func Validate(db *index.DB, reg *schema.Registry) (*ValidateResult, error)

Validate runs all frontmatter validation rules against the indexed notes.

func ValidateLive

func ValidateLive(vaultPath string, reg *schema.Registry) (*ValidateResult, error)

ValidateLive walks vaultPath, parses each .md file's frontmatter, and runs schema rules against the live files on disk. It does NOT require an index.

Rules evaluated: unknown_type, missing_required_field, invalid_status. Unparseable frontmatter is reported as invalid_frontmatter. The broken_reference rule is skipped — it requires the full link graph, which only the indexer produces.

type VaultSource added in v0.8.0

type VaultSource struct {
	Name      string
	Retriever retrieval.Retriever
	// Verdict is this vault's OWN top-hit confidence, computed against its own
	// noise floor. Empty means unmeasured (no embedder), which is kept rather
	// than assumed irrelevant.
	Verdict string
	// RelevanceZ is this vault's own top-hit z, used only to break RANK TIES
	// between vaults. It is never compared as a score across vaults for
	// ordering — that is precisely what RRF exists to avoid.
	RelevanceZ float64
}

VaultSource is one vault participating in a federated query: a display name and a way to search it. Everything else about the vault (its floor, its DB, its resolver) stays with the caller, because delivery happens in the owning vault's own pipeline rather than through some merged pseudo-vault.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL