query

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 18, 2026 License: AGPL-3.0 Imports: 22 Imported by: 0

Documentation

Overview

Package query is the read path: open an index built by internal/build and serve semantic_search. It owns:

  • manifest validation on Open (dim + model identity must match the caller's Embedder)
  • over-fetch + threshold drop + citation enforcement
  • snippet density compression under a token budget

Designed to be importable by both `ckv query` (CLI) and the future MCP server (`ckv mcp`) without code duplication.

Index

Constants

View Source
const DefaultBudgetTokens = 4000

DefaultBudgetTokens is the response-side token budget. 4000 tokens ≈ 16K chars of snippet content across all hits.

View Source
const DefaultK = 10

DefaultK is the top-K when the caller omits it.

View Source
const DefaultSignatureContextLines = 5

DefaultSignatureContextLines is how many lines after the signature we keep in the SignatureWithContext density tier when the caller doesn't override Options.SignatureContextLines.

View Source
const DefaultThreshold = 0.4

DefaultThreshold drops hits whose normalized score is below this value. 0.4 is conservative — with the mock embedder it trims obvious noise without rejecting real matches.

View Source
const MinBudgetTokens = 20

MinBudgetTokens is the floor below which BudgetTokens can't even fit a single signature-density hit. Set so a one-line Go signature (~50-80 chars) rounds up to ~20 tokens with one hit; below this the engine returns ErrBudgetExceeded rather than truncate further.

Variables

View Source
var ErrBudgetExceeded = errors.New("query: budget exceeded")

ErrBudgetExceeded signals SearchOptions.BudgetTokens is too small for the engine to render even a minimum-density response. The engine degrades snippets to signature-only before raising; reaching this error means the budget is below the floor (MinBudgetTokens).

Caller guidance: raise BudgetTokens (defaults to 4000), or accept an empty response by passing BudgetTokens<0 to disable budgeting.

View Source
var ErrChunkNotFound = errors.New("chunk not found")

ErrChunkNotFound is returned when ExpandInFile / NarrowCandidates is asked about a chunk ID that does not exist in the index.

View Source
var ErrCitationNotFound = errors.New("query: citation not found")

ErrCitationNotFound signals catastrophic citation enforcement failure: every candidate that passed the threshold gate was dropped because its file could not be located under the recorded src_root. Almost always indicates the source tree was deleted or moved without rebuilding the index.

Caller guidance: re-run `ckv build --src <current path>`. Until then the index is unusable for retrieval — every hit would be unverifiable.

View Source
var ErrFreshnessStale = errors.New("query: index freshness stale")

ErrFreshnessStale signals the index's recorded IndexedHead is behind the source tree's current git HEAD. Returned by the freshness helpers (Engine.CheckFreshness, freshness.Check) when callers opt in to strict freshness; the on-the-wire Response keeps the soft Metadata.Fresh bool for callers that only need a hint.

Caller guidance: the result set is still usable for many cases (recent edits typically affect a small subset of files). Schedule a reindex when convenient. Prompt-time freshness sensitivity is the caller's taste.

View Source
var ErrIndexUnavailable = errors.New("query: index unavailable")

ErrIndexUnavailable signals that the on-disk index cannot be served by the supplied Embedder. Most commonly: the indexed model identity differs from the query-time model, the dimension is different, or the manifest is missing.

Caller guidance: surface to the operator with a "run `ckv build`" hint. Do not retry — the index will not change without a rebuild.

View Source
var ErrPolicyError = errors.New("query: policy error")

ErrPolicyError signals a policy or authorization check rejected the request. Typical raises: mTLS caller-cert SAN mismatch with envelope caller, intent flagged by content policy, or an internal-only tool requested through an external surface.

Caller guidance: this is a hard rejection — do not retry. Surface to the operator. Defined now for forward-compatible callers.

View Source
var ErrSanitizeFailed = errors.New("query: sanitize failed")

ErrSanitizeFailed signals the sanitize pipeline rejected the response payload. The sanitize module is the default-deny gate between raw retrieval and outbound delivery — secrets, PII, oversized blobs, or policy-flagged content land here.

Caller guidance: log the rejection (sanitize_report carries the reason), do not retry with the same intent. Defined now for forward-compatible callers.

Functions

func CosineSimilarity

func CosineSimilarity(a, b []float32) float64

CosineSimilarity is a small helper for the explain pathway in tests and other callers that want a direct vector-vs-vector distance instead of going through the store's KNN. Returns 0 when either vector is empty or differs in length.

func EnforceCitations

func EnforceCitations(hits []types.Hit, srcRoot string) (keep []types.Hit, dropped int)

EnforceCitations verifies every hit's citation against the on-disk source tree at srcRoot, returning the surviving hits and the count of dropped ones. Citation accuracy must be 100%, so any hit we can't verify is silently dropped (with a warning aggregated by the caller).

Verification:

  1. file exists at srcRoot/<rel>
  2. start_line ≥ 1 and end_line ≥ start_line

Commit-hash mismatch (stale citation) is NOT a drop — the file almost always still has useful content, just at a different commit than when it was indexed. Callers see the stale count returned alongside dropped and surface a warning. Use EnforceCitationsAt to enable the stale check.

When srcRoot is empty (no source tree available; we're running against a moved index), every hit is allowed through with a warning.

func EnforceCitationsAt

func EnforceCitationsAt(hits []types.Hit, srcRoot, currentHead string, docsRoots ...string) (keep []types.Hit, dropped int, stale int)

EnforceCitationsAt extends EnforceCitations with stale detection: when currentHead is non-empty, every surviving hit whose Chunk.CommitHash differs gets Hit.StaleCitation=true and counts toward the stale return. Useful diagnostic when callers can detect "the index is post-reindex-needed but still served." Empty currentHead disables the check; stale stays 0.

The surviving slice's order matches the input; only the metadata on each hit is touched. Dropped hits are removed as before — line validity and file existence remain hard requirements. docsRoots are additional roots (manifest.DocsRoots — the `ckv build --docs` corpus dirs) used to resolve doc/markdown chunk citations whose File is relative to a corpus dir rather than the code srcRoot. A citation is valid if its file exists under srcRoot OR any docsRoot; without this, every domain-corpus chunk is dropped because it never resolves under the code tree.

func ExpandQuery

func ExpandQuery(intent string, aliases AliasMap) string

ExpandQuery widens intent with any alias entry whose key appears as a substring of intent. Output shape:

"<intent> [aliases: <kw1>, <kw2>, ...]"

Returned intent is identical to input when:

  • aliases is nil or empty
  • no alias key appears in intent

The trailer is deterministic: alias keys are matched in sorted order and matched-keyword lists are deduplicated + sorted before joining. Determinism matters for the per-query footprint fingerprint (query.embed sub-span) so two runs of the same intent produce the same trace fingerprint.

Why substring match (not exact / token-aware): the intent is free-form natural language, often Korean with spaces / particles ("0번 블록의", "합의 알고리즘이"). Substring covers the common case; false positives are rare for the curated glossary we ship.

Types

type AliasFile

type AliasFile struct {
	Aliases AliasMap `yaml:"aliases"`
}

AliasFile is the on-disk YAML shape. We keep the top-level `aliases` key explicit so future additions (e.g., `metadata`, `version`) can land without breaking the loader.

type AliasMap

type AliasMap map[string][]string

AliasMap maps a "human" phrase (often Korean or domain-vague) to a list of "code" keywords the embedder is more likely to match.

Example:

"0번 블록"        → [genesis, genesis_block, GenesisAlloc]
"합의 알고리즘"   → [consensus, wbft, WBFT]
"시스템 컨트랙트" → [system contract, systemcontracts, SystemContracts]

Used by ExpandQuery to widen the intent string at query time. The raw intent stays first so semantic ordering isn't disturbed; aliases are appended in a bracket-tagged trailer so the embedder treats them as auxiliary context.

The map can be hand-curated (testdata/stablenet/glossary.yaml) or auto-generated by a glossary loader.

func LoadAliasMap

func LoadAliasMap(path string) (AliasMap, error)

LoadAliasMap reads a glossary YAML and returns the parsed map. Returns nil + nil error when path is empty — callers can pass the CLI flag through unconditionally and the no-flag case stays a no-op. Empty `aliases` block is treated the same as no file (nil return).

type AlignmentReport

type AlignmentReport struct {
	Status         AlignmentStatus `json:"status"`
	RecordedCommit string          `json:"recorded_commit,omitempty"`
	CurrentCommit  string          `json:"current_commit,omitempty"`
	RecordedDigest string          `json:"recorded_digest,omitempty"`
	CurrentDigest  string          `json:"current_digest,omitempty"`
	SchemaVersion  string          `json:"schema_version,omitempty"`
	CKGPath        string          `json:"ckg_path,omitempty"`
	Reason         string          `json:"reason,omitempty"`
	Warnings       []string        `json:"warnings,omitempty"`
}

AlignmentReport is the structured result of CheckAlignment. The authoritative keys are src_commit + graph_digest (design §3.1); the src_root path is NOT compared here (a different checkout of the same commit is legal — that concern belongs to the serving layer, as a warning).

func (AlignmentReport) Serviceable

func (r AlignmentReport) Serviceable() bool

Serviceable reports whether the alignment is trustworthy enough to serve canonical_id joins: ok / degraded / not_aligned are serviceable; only mismatch (commit or digest divergence) is not.

type AlignmentStatus

type AlignmentStatus string

AlignmentStatus classifies the CKG↔CKV alignment.

const (
	AlignmentOK         AlignmentStatus = "ok"          // commit + digest match
	AlignmentDegraded   AlignmentStatus = "degraded"    // commit matches, digest unverifiable / schema<1.19
	AlignmentMismatch   AlignmentStatus = "mismatch"    // commit or digest differ (join broken) — re-align needed
	AlignmentNotAligned AlignmentStatus = "not_aligned" // index built without --ckg
)

type BoostOptions

type BoostOptions struct {
	// SignatureMatch boosts hits whose symbol name or signature contains
	// query keywords. Default multiplier: 1.5
	SignatureMatch      bool
	SignatureMultiplier float64

	// DocMatch boosts hits whose leading comment block contains query
	// keywords. Default multiplier: 1.3
	DocMatch      bool
	DocMultiplier float64

	// RecentModified boosts hits whose commit_hash equals the indexed
	// HEAD (treated as "recently modified"). Default multiplier: 1.1
	// For real git log-based recency, the caller can pre-compute and
	// pass via a custom Score override.
	RecentModified   bool
	RecentMultiplier float64

	// PackageProximity boosts hits whose file path contains the given
	// package keyword. Default multiplier: 1.2
	PackageProximity  bool
	PackageKeyword    string
	PackageMultiplier float64

	// IndexedHead is used by RecentModified to compare chunk.CommitHash.
	// Set by the engine before calling Run.
	IndexedHead string
}

BoostOptions controls score boosting after vector + BM25 ranking. All multipliers are applied independently and combined multiplicatively.

func DefaultBoostOptions

func DefaultBoostOptions() BoostOptions

Defaults returns the recommended boost multipliers.

type BoostService

type BoostService struct{}

BoostService applies score boosting rules to candidate hits.

func (*BoostService) Run

func (s *BoostService) Run(hits []types.Hit, intent string, opts BoostOptions) []types.Hit

Run reorders hits by boosted scores. The hit's Score.Normalized is updated with the boosted value; the original raw distance is preserved. Stable sort: hits with equal boosted scores keep their input order.

func (*BoostService) RunContext

func (s *BoostService) RunContext(sc *SearchContext, indexedHead string)

RunContext applies boost to sc.RawHits when Options.EnableScoreBoost.

type BranchMatch

type BranchMatch struct {
	When     string         `json:"when"`
	Then     string         `json:"then"`
	At       string         `json:"at"`
	StepID   string         `json:"step_id"`
	FlowID   string         `json:"flow_id"`
	Symbol   string         `json:"symbol,omitempty"`
	Citation types.Citation `json:"citation"`
	Score    float64        `json:"score"`
}

BranchMatch is a failure branch surfaced for a symptom query.

type CommitSummary

type CommitSummary struct {
	Hash    string `json:"hash"`
	Author  string `json:"author"`
	Date    string `json:"date"`
	Subject string `json:"subject"`
}

CommitSummary is one git log entry attached to a hit.

type ConventionHit

type ConventionHit struct {
	ChunkID string         `json:"chunk_id"`
	File    string         `json:"file"`
	Package string         `json:"package"`
	Summary string         `json:"summary"`
	Stats   map[string]any `json:"stats"`
}

ConventionHit returns the raw ConventionStats plus the human summary the agent can present without further processing.

type DensityService

type DensityService struct{}

DensityService adjusts snippet length to fit within a token budget. Longer snippets are progressively compressed (full → signature+context → signature-only) until the total fits.

func (*DensityService) Run

func (s *DensityService) Run(_ context.Context, hits []types.Hit, budget int, maxDensity DensityTier, sigContextLines int) ([]Hit, int)

Run adjusts density for combined hits (primary + examples) under the given token budget. Returns formatted Hit slice and total tokens used.

func (*DensityService) RunContext

func (s *DensityService) RunContext(ctx context.Context, sc *SearchContext) error

RunContext reads sc.PrimaryHits + sc.ExampleHits, adjusts density, and writes sc.FinalHits + sc.FinalExamples + sc.TokensUsed.

type DensityTier

type DensityTier string

DensityTier names the three snippet shapes the engine can emit.

  • DensityFull — chunk's complete Text. Default when budget room allows.
  • DensitySignature5 — first non-blank line (the signature) plus the next N non-blank lines. Middle compression tier; useful for "what does this function start by doing".
  • DensitySignatureOnly — first non-blank line only (just the signature). Minimum useful representation; caller resolves the body via Citation if they need more.

Reported on every Hit (Hit.Density) so consumers can render progressively or count how aggressively the budget squeezed them.

const (
	DensityFull          DensityTier = "full"
	DensitySignature5    DensityTier = "signature+N"
	DensitySignatureOnly DensityTier = "signature_only"
)

type EmbedService

type EmbedService struct {
	// contains filtered or unexported fields
}

EmbedService converts text into a vector using the configured embedder. Can be called independently via Engine.Embed() or as part of the search pipeline.

func (*EmbedService) Run

func (s *EmbedService) Run(ctx context.Context, text string) ([]float32, error)

Run embeds the given text and returns the vector.

func (*EmbedService) RunContext

func (s *EmbedService) RunContext(ctx context.Context, sc *SearchContext) error

RunContext populates sc.QueryVec from sc.EmbedIntent.

type EmbedderInfo

type EmbedderInfo struct {
	Name      string `json:"name"`
	Dimension int    `json:"dimension"`
	// Status: "ready" | "stub" | "unavailable"
	//   ready       — embedder loaded, semantic signal expected
	//   stub        — placeholder embedder (mock, hash-based); recall not meaningful
	//   unavailable — embedder is nil or failed to attach
	Status string `json:"status"`
	// Provider is the execution backend tag (bgeonnx-specific):
	// "coreml" | "coreml-fallback-to-cpu" | "cpu" | "". Empty for
	// embedders that don't run a backend (mock).
	Provider string `json:"provider,omitempty"`
	// ModelDir is the on-disk model directory the embedder loaded from.
	// Empty for embedders that don't have one (mock).
	ModelDir string `json:"model_dir,omitempty"`
}

EmbedderInfo is the health-endpoint view of the embedder this Engine was opened with. Status answers "is this embedder useful for real semantic search?" — operators can distinguish "ckv alive but mock embedder" from "ckv ready with bgeonnx + model loaded" without inspecting the model name string.

Optional fields (Provider, ModelDir) are empty when the embedder implementation doesn't expose them — health consumers must tolerate the empty case.

type Engine

type Engine struct {
	// contains filtered or unexported fields
}

Engine is the long-lived query handler. Hold one per --out directory. Concurrency-safe: store.Search is read-only; we never mutate Engine state after Open (footprint Logger is sync-safe by contract).

func Open

func Open(outDir string, emb types.Embedder, opts ...OpenOption) (*Engine, error)

Open loads the index at outDir and validates the manifest against the supplied Embedder. Mismatches produce ErrIndexUnavailable with a human-readable cause; callers (CLI / MCP) surface a "ckv reindex" hint to the user. opts apply after the engine is constructed but before it is returned.

func (*Engine) CheckAlignment

func (e *Engine) CheckAlignment() AlignmentReport

CheckAlignment compares the recorded sources.ckg coordinates against the CKG graph currently at the recorded path. Returns not_aligned when the index was built without --ckg. Best-effort: an unreadable graph is a mismatch (the aligned-against graph is gone).

func (*Engine) CheckFreshness

func (e *Engine) CheckFreshness() error

CheckFreshness compares the loaded manifest's IndexedHead against the source tree's current git HEAD and returns ErrFreshnessStale (wrapped with the diff) when they differ. Returns nil when fresh, no error and no report when git is unavailable (callers who need the Report directly should use internal/freshness.Check).

Caller guidance for ErrFreshnessStale: stale results are usually still usable for most retrieval (recent edits affect a small subset of files), but the caller should schedule `ckv build` when convenient.

func (*Engine) Close

func (e *Engine) Close() error

Close releases the underlying store. Idempotent.

func (*Engine) Embed

func (e *Engine) Embed(ctx context.Context, text string) ([]float32, error)

Embed exposes the embedding service for independent use. Converts text into a vector without running the full search pipeline.

func (*Engine) Embedder

func (e *Engine) Embedder() types.Embedder

Embedder returns the underlying embedder. Callers (MCP index handler) reuse it to avoid re-initializing the embedder for build/reindex.

func (*Engine) EmbedderInfo

func (e *Engine) EmbedderInfo() EmbedderInfo

EmbedderInfo extracts metadata via duck typing — bgeonnx exposes Provider() and ModelDir(), the mock exposes neither. The returned struct is JSON-safe (no unset zero-value confusion for omitted fields) so health handlers can serialize it directly.

func (*Engine) ExpandFlow

func (e *Engine) ExpandFlow(ctx context.Context, stepID, direction string, hops int) (*ExpandResult, error)

ExpandFlow returns the steps adjacent to stepID up to `hops` away, following downstream `calls` (direction "down") or upstream `called_by` ("up"), plus the origin step's own failure branches.

func (*Engine) ExpandInFile

func (e *Engine) ExpandInFile(ctx context.Context, chunkID string, before, after int) ([]Hit, error)

ExpandInFile returns chunks immediately surrounding chunkID inside the same file. The window is [match - before, match + after] in chunk-index space (chunks ordered by start_line). When chunkID is unknown, returns ErrChunkNotFound. Score is zero on the returned hits — they are context, not ranked results.

before / after of 0 returns just the source chunk. before or after over the edge are silently clamped (no error for "ran out of chunks").

func (*Engine) ExplainMatch

func (e *Engine) ExplainMatch(ctx context.Context, chunkID, intent string) (*Explanation, error)

ExplainMatch reports why a particular chunk would have matched the intent: vector similarity computed fresh against the embedder, and BM25 details from the keyword index. When the chunk_id is unknown, returns ErrChunkNotFound.

The function is pure-read and does no caching beyond the keyword index already cached on the Engine. Cost is roughly:

  • 1 embed call (the intent vector)
  • 1 KNN-style cosine distance against the chunk's stored vector
  • 1 BM25 lookup against the in-memory index

func (*Engine) FindBranches

func (e *Engine) FindBranches(ctx context.Context, symptom string, k int) ([]BranchMatch, error)

FindBranches maps a symptom phrase to the failure branches of the flow steps most relevant to it (semantic search over flow chunks, whose embed text already includes each branch's "when"). Requires a real embedder.

func (*Engine) FindInvariants

func (e *Engine) FindInvariants(ctx context.Context, file, category string, tierMin int) ([]InvariantHit, error)

FindInvariants returns invariants matching the filter. tierMin (1, 2, or 3) drops anything below that confidence tier. The tier is read from the source chunk's InvariantRef back-pointer when the source is in the index; otherwise the invariant is included at its declared SymbolName-based tier where possible.

func (*Engine) FreshnessReport

func (e *Engine) FreshnessReport() (freshness.Report, error)

FreshnessReport returns the structured index-vs-HEAD comparison (IndexedHead, CurrentHead, ChangedFiles, Stale, Fresh, Warnings). Unlike CheckFreshness (which collapses the comparison to a single error), this exposes the full Report so the pkg/ckv facade can hand it to cks's cks.ops.freshness tool without re-deriving it. Git unavailability is reported via Report.Warnings (Fresh=false), not as a hard error — same contract as freshness.Check.

func (*Engine) GetConventions

func (e *Engine) GetConventions(ctx context.Context, packagePrefix string) ([]ConventionHit, error)

GetConventions returns convention chunks under the package prefix. Each Hit carries the deterministic prose summary plus the raw stats map so the agent can use whichever shape its prompt expects.

func (*Engine) GetFlow

func (e *Engine) GetFlow(ctx context.Context, sel FlowSelector) (*FlowView, error)

GetFlow lays out a flow's steps in call (topological) order. Selector must set exactly one of FlowID / EntryPoint / InvariantID.

func (*Engine) GetInvariantEnforcement

func (e *Engine) GetInvariantEnforcement(ctx context.Context, invID string) (*InvariantEnforcement, error)

GetInvariantEnforcement lists every (flow, step, loc) where a curated invariant is enforced.

func (*Engine) KeywordSearch

func (e *Engine) KeywordSearch(ctx context.Context, query string, k int, filter types.Filter) ([]Hit, error)

KeywordSearch returns the top-k BM25 hits for the natural-language query. The first call after Engine.Open builds the in-memory index (cached for subsequent calls). Filter is applied post-rank — we over-fetch by 3x when a filter is set, mirroring Store.Search.

func (*Engine) LookupPRsByFile

func (e *Engine) LookupPRsByFile(ctx context.Context, file string) ([]types.PRRef, error)

LookupPRsByFile delegates to the store's PR breadcrumb lookup.

func (*Engine) Manifest

func (e *Engine) Manifest() manifest.Manifest

Manifest returns a copy of the loaded manifest. Callers (ckv freshness) use this to compare on-disk identity without reopening.

func (*Engine) NarrowCandidates

func (e *Engine) NarrowCandidates(ctx context.Context, ids []string, filter types.Filter) ([]Hit, error)

NarrowCandidates loads the chunks for the given IDs and returns the subset that matches the filter. Score fields are zero-valued (this is a metadata refinement step, not a ranking step). Useful for multi-hop retrieval flows where an earlier semantic_search produced hit IDs and a follow-up wants to drop everything outside a category or path.

func (*Engine) OutDir

func (e *Engine) OutDir() string

OutDir returns the data directory the engine was opened from.

func (*Engine) Rerank

func (e *Engine) Rerank(ctx context.Context, candidates []types.Hit, intent string) ([]types.Hit, error)

Rerank reorders candidate hits using BM25 + RRF fusion.

func (*Engine) ResolvedVersion

func (e *Engine) ResolvedVersion() string

ResolvedVersion returns the concrete on-disk directory name the data path resolves to, following a `current` symlink. For a blue-green dataset served via <dataset>/current it reports the promoted version dir; for a plain data dir it reports that dir's name. Empty when the path can't be resolved. Lets health show which version is being served (reindex-migration-design §6).

func (*Engine) Search

func (e *Engine) Search(ctx context.Context, intent string, opts Options) (*Response, error)

Search runs the full pipeline (Facade): embed → vector search → rerank → threshold → citation → test split → density adjust.

For fine-grained control, use the individual service methods (Embed, VectorSearch, Rerank) and compose them manually.

func (*Engine) VectorSearch

func (e *Engine) VectorSearch(ctx context.Context, queryVec []float32, k int, filter types.Filter) ([]types.Hit, error)

VectorSearch runs ANN search against the store without reranking, threshold, or citation. Returns raw candidate hits.

func (*Engine) Warmup

func (e *Engine) Warmup(ctx context.Context) error

Warmup forces the embedder to pay its cold-start cost (ONNX session load, CoreML compile, tokenizer lazy init) before the first user-facing query lands. Runs a single dummy Embed call. Returns the embedder's error if it fails to attach — callers (health, CLI) should surface it so the operator can investigate before traffic starts.

Idempotent and cheap to call repeatedly: after the first call the embedder is already warm so subsequent calls just round-trip a short batch.

type EnrichService

type EnrichService struct{}

EnrichService attaches git history to each hit. Implements Phase 3 spec Step 3 (Metadata Enrichment). Per-file results are cached within a single Run to avoid duplicate git log calls when multiple hits share a file.

func (*EnrichService) Run

func (s *EnrichService) Run(ctx context.Context, hits []Hit, srcRoot string, maxCommits int) []Hit

Run executes `git log` for each unique file in hits and attaches the results. srcRoot is the repo root. maxCommits is the per-file history depth (5 matches the Phase 3 spec).

func (*EnrichService) RunContext

func (s *EnrichService) RunContext(ctx context.Context, sc *SearchContext, srcRoot string)

RunContext attaches git history to sc.FinalHits and sc.FinalExamples when Options.EnableMetadataEnrichment is set.

type ExpandResult

type ExpandResult struct {
	Origin         string         `json:"origin"`
	Direction      string         `json:"direction"`
	OriginBranches []types.Branch `json:"origin_branches,omitempty"`
	Neighbors      []FlowNeighbor `json:"neighbors"`
}

ExpandResult is the neighborhood of one step plus that step's own branches (the failure conditions at the origin).

type Explanation

type Explanation struct {
	ChunkID  string                      `json:"chunk_id"`
	Citation types.Citation              `json:"citation"`
	Vector   ExplanationVectorScore      `json:"vector_score"`
	Keyword  ExplanationKeywordScore     `json:"keyword_score"`
	Category string                      `json:"category,omitempty"`
	Guidance *types.ModificationGuidance `json:"guidance,omitempty"`
	Symbol   string                      `json:"symbol,omitempty"`
}

Explanation is the full explain_match response.

type ExplanationKeywordScore

type ExplanationKeywordScore struct {
	Score          float64  `json:"score"`
	MatchedTokens  []string `json:"matched_tokens"`
	QueryTokens    []string `json:"query_tokens"`
	ChunkTokenSize int      `json:"chunk_token_size"`
}

ExplanationKeywordScore captures BM25-side detail: the raw Okapi score over the chunk's tokens against the query, plus which tokens actually matched (intersection of query tokens and chunk tokens).

type ExplanationVectorScore

type ExplanationVectorScore struct {
	Normalized     float64 `json:"normalized"`
	CosineDistance float64 `json:"cosine_distance"`
}

ExplanationVectorScore is the vector-side of explain_match.

type FlowNeighbor

type FlowNeighbor struct {
	StepID   string         `json:"step_id"`
	Symbol   string         `json:"symbol,omitempty"`
	Citation types.Citation `json:"citation"`
	Relation string         `json:"relation"` // "calls" (downstream) | "called_by" (upstream)
}

FlowNeighbor is an adjacent step reached from an origin step.

type FlowSelector

type FlowSelector struct {
	FlowID      string
	EntryPoint  string
	InvariantID string // resolves to the first flow in the invariant's enforced_at
}

FlowSelector picks a flow by exactly one of its keys.

type FlowStepView

type FlowStepView struct {
	StepID     string         `json:"step_id"`
	Symbol     string         `json:"symbol,omitempty"`
	Citation   types.Citation `json:"citation"`
	Kind       string         `json:"kind,omitempty"`
	Calls      []string       `json:"calls,omitempty"`
	Reads      string         `json:"reads,omitempty"`
	Writes     string         `json:"writes,omitempty"`
	Emits      string         `json:"emits,omitempty"`
	Branches   []types.Branch `json:"branches,omitempty"`
	Invariants []string       `json:"invariants,omitempty"`
}

FlowStepView is one step in a laid-out flow.

type FlowView

type FlowView struct {
	FlowID     string         `json:"flow_id"`
	EntryPoint string         `json:"entry_point,omitempty"`
	Trigger    string         `json:"trigger,omitempty"`
	RootSymbol string         `json:"root_symbol,omitempty"`
	Links      []string       `json:"links,omitempty"`
	CalledBy   []string       `json:"called_by,omitempty"`
	Steps      []FlowStepView `json:"steps"`
}

FlowView is a flow's spine plus its steps in call (topological) order.

type HallucinationResult

type HallucinationResult struct {
	Verified bool   `json:"verified"`
	Reason   string `json:"reason,omitempty"` // empty when Verified=true
	// File-level diagnostics for false cases. Populated only when
	// Verified=false so the JSON payload stays compact for the common
	// case.
	ExpectedFile string `json:"expected_file,omitempty"`
}

HallucinationResult is the verdict for one hit-vs-source comparison. Verified=true means the snippet's content actually exists at the claimed file location; Verified=false flags a hallucination signal with a short Reason so log readers can debug without re-running.

The operational definition: a hit is *not* a hallucination if its snippet appears as a (whitespace-normalized) substring of the cited file's line range. This handles every density tier — DensitySignatureOnly is naturally a substring of a longer body, DensityFull is the full body itself.

func VerifyHit

func VerifyHit(h Hit, srcRoot string) HallucinationResult

VerifyHit reports whether h.Snippet aligns with the source file at srcRoot/<h.Citation.File>. Three failure modes are distinguished so log readers can route the response: "file_missing" (citation points nowhere), "out_of_range" (line numbers exceed file length), and "snippet_not_found" (content disagrees — the actual hallucination).

Whitespace normalization: tabs / spaces / newlines collapse to a single space and leading / trailing whitespace is trimmed. This matches how a reader would compare the snippet against the file (formatting differences are not hallucinations).

Empty srcRoot returns Verified=false with reason="no_src_root" — the caller asked for verification but didn't provide a tree to verify against.

func VerifyResponse

func VerifyResponse(resp *Response, srcRoot string) (verdicts []HallucinationResult, hallucinated int)

VerifyResponse runs VerifyHit across both Hits and Examples of the response and aggregates the verdicts. Returns the per-hit results (in response order: hits first, then examples) plus a count of non-verified hits. Convenient for eval pipelines that just want a hallucination rate.

type Hit

type Hit struct {
	ChunkID  string         `json:"chunk_id"`
	Citation types.Citation `json:"citation"`
	Snippet  string         `json:"snippet"`
	// Density names which 3-tier ladder rung this Snippet was rendered
	// at (DensityFull / DensitySignature5 / DensitySignatureOnly).
	// Useful for downstream UIs that want to badge compressed hits or
	// for eval pipelines counting how often the budget forced a
	// downgrade. Omitted when empty (e.g. callers building Hit by hand).
	Density DensityTier `json:"density,omitempty"`
	// StaleCitation propagates from types.Hit: the chunk's recorded
	// commit_hash disagrees with the current source-tree HEAD. The
	// citation file/lines still resolve — the content there may have
	// shifted since the index was built. Consumers can render a badge
	// or schedule a reindex. Omitted when false.
	StaleCitation bool             `json:"stale_citation,omitempty"`
	Score         types.HitScore   `json:"score"`
	Language      string           `json:"language"`
	IsTest        bool             `json:"is_test,omitempty"`
	Symbol        string           `json:"symbol,omitempty"`
	SymbolKind    types.SymbolKind `json:"symbol_kind,omitempty"`
	// ChunkKind classifies the chunk's origin strategy (symbol,
	// file_header, doc, invariant, convention, flow_*). Lets consumers
	// (cks knowledge quota) route knowledge chunks without re-querying.
	ChunkKind   types.ChunkKind `json:"chunk_kind,omitempty"`
	CanonicalID string          `json:"canonical_id,omitempty"` // ckg's import-path-qualified symbol id (ADR-0001); the stable key cks uses to FindByCanonicalID against ckg
	// Category and Guidance are populated by the policy loader at build
	// time. Category labels the chunk's domain ("consensus", "state",
	// ...); Guidance lists what the agent should also review, test, and
	// watch out for when modifying this code. Both omitted when the
	// chunk did not match any policy rule (or no policy was loaded).
	Category string                      `json:"category,omitempty"`
	Guidance *types.ModificationGuidance `json:"guidance,omitempty"`

	// GitHistory holds recent commits touching this hit's file.
	// Populated when Options.EnableMetadataEnrichment is set.
	GitHistory []CommitSummary `json:"git_history,omitempty"`
}

Hit is the response-shaped record: only what callers (LLM, CLI) need. We deliberately omit Chunk.Text — Snippet is the budget-adjusted view.

func DensityAdjust

func DensityAdjust(hits []types.Hit, budgetTokens int) (out []Hit, tokensUsed int)

DensityAdjust converts store hits into response Hits with snippets sized to fit budgetTokens. ctxLines tunes the SignatureWithContext tier (0 → DefaultSignatureContextLines). cap caps the maximum tier the engine will emit ("" → DensityFull, no cap).

Algorithm:

  1. Start every hit at `cap` (or DensityFull when no cap).
  2. If the total token count exceeds budget, downgrade hits one by one from the lowest-ranked (worst score) upward: full → signature+N → signature only.
  3. Stop as soon as the running total fits, or every hit is at the minimum.

Each Hit's Density field reports the final tier it landed in. Citation is *not* counted against the budget. Returns the response hits plus the final tokensUsed estimate.

func DensityAdjustWith

func DensityAdjustWith(hits []types.Hit, budgetTokens int, cap DensityTier, ctxLines int) (out []Hit, tokensUsed int)

DensityAdjustWith is the configurable variant exposing the density ceiling and context-line knob. DensityAdjust is the convenience wrapper that uses documented defaults.

type InvariantEnforcement

type InvariantEnforcement struct {
	InvID      string               `json:"inv_id"`
	Statement  string               `json:"statement,omitempty"`
	EnforcedAt []types.EnforcePoint `json:"enforced_at"`
}

InvariantEnforcement lists every place a curated invariant is enforced.

type InvariantHit

type InvariantHit struct {
	ChunkID     string                      `json:"chunk_id"`
	File        string                      `json:"file"`
	StartLine   int                         `json:"start_line"`
	EndLine     int                         `json:"end_line"`
	Marker      string                      `json:"marker"`             // e.g. "CRITICAL", "panic"
	Tier        types.InvariantTier         `json:"tier"`               // 1, 2, or 3
	Text        string                      `json:"text"`               // the invariant statement
	Category    string                      `json:"category,omitempty"` // inherited from source chunk's policy
	Guidance    *types.ModificationGuidance `json:"guidance,omitempty"` // ditto
	SourceChunk string                      `json:"source_chunk_id,omitempty"`
}

InvariantHit pairs a ChunkInvariant chunk with its parsed tier so MCP callers can filter without reparsing the back-reference.

type KeywordIndex

type KeywordIndex struct {
	// contains filtered or unexported fields
}

KeywordIndex is a lazily-built in-memory BM25 index over every chunk in the store. It rebuilds on first KeywordSearch after Engine.Open; subsequent searches reuse the cached index.

Rebuild on Open is the right trade-off for CKV's expected scale: rebuilding 50k chunks takes well under a second on modern hardware, and the index then serves arbitrary keyword queries with no per-call IO. For sustained high-write workloads a future revision can move this into the persistence layer.

Build serialization is handled by Engine.kwMu — KeywordIndex itself is read-only after construction so it needs no internal lock.

type OpenOption

type OpenOption func(*Engine)

OpenOption customizes Engine construction (functional options).

func WithFootprint

func WithFootprint(fp *footprint.Logger) OpenOption

WithFootprint attaches a logger to the Engine. Every Search emits a span (query.search.start / query.search.done) including intent hash, hit count, citation drops, and latency. Nil-safe.

type Options

type Options struct {
	K            int          // top-K (0 → DefaultK)
	Filter       types.Filter // pre-filter (lang / path / kind / commit)
	BudgetTokens int          // snippet density budget (0 → DefaultBudgetTokens)
	Threshold    float64      // min normalized score (0 → DefaultThreshold; <0 disables)
	SrcRoot      string       // absolute path used by citation enforcement;

	// ExamplesK splits test-file hits out of the main Hits slice into a
	// separate Examples slice in the response. Up to ExamplesK test
	// chunks pass through — distinct from K, which counts only the
	// non-test (primary implementation) hits. Defaults to 0 → no
	// separation, every hit goes through Hits as before.
	//
	// Why: an LLM coding agent gets cleaner signal when primary code
	// (the actual implementation it should mimic) is separate from
	// usage examples (tests that show how the code is called). With
	// them mixed, top-5 can be diluted by 2-3 test results that compete
	// for context window space.
	ExamplesK int

	// TraceID is a caller-supplied correlation ID propagated to the
	// footprint span and echoed in Response.Metadata.TraceID. Useful
	// when a single user action fans out into multiple Search calls
	// (CKS multiplex, retries) and the operator wants to grep all
	// related log entries together. Empty → engine generates one from
	// the intent hash + a monotonic suffix.
	TraceID string

	// DryRun, when true, validates the request (intent, embedder,
	// manifest identity) but skips embedding, store search, citation
	// enforcement, and density adjustment. The response carries
	// metadata only — Hits and Examples are empty. Useful for caller-
	// side budget / plan validation without paying the query cost
	// or polluting the footprint log with hot-path entries.
	DryRun bool

	// MaxDensity caps the snippet rendering tier (B3 ladder).
	// "" / DensityFull → no cap, downgrade only under budget pressure
	// (the documented default). DensitySignature5 → start every hit at
	// signature+N and downgrade to signature_only under pressure.
	// DensitySignatureOnly → emit signatures only, never bodies; useful
	// when the caller already has the chunk body cached and just wants
	// pointers (e.g. CLI list-mode).
	MaxDensity DensityTier

	// SignatureContextLines tunes the SignatureWithContext tier (number
	// of non-blank lines kept after the signature). 0 →
	// DefaultSignatureContextLines (5). Larger values keep more body
	// in the middle tier at the cost of fewer hits fitting the budget.
	SignatureContextLines int

	// Aliases is the vocabulary-bridge glossary applied to the intent
	// at the very start of Search. When non-nil, ExpandQuery
	// widens the intent with code-keyword aliases before embedding —
	// useful for Korean / domain-vague queries against an English code
	// corpus. Nil leaves the intent untouched (off by default).
	//
	// The expanded intent is what gets embedded and what appears in
	// the query.embed sub-span fingerprint; the *original* intent is
	// still echoed in the query.search top-level span for log triage.
	Aliases AliasMap

	// EnableBM25Rerank turns on the optional BM25 rerank pass between
	// store.search and threshold.drop. When true, the engine builds a
	// candidate-set BM25 over the vector hits (corpus = chunk.SymbolName
	// + first text line), scores each candidate against the *original*
	// intent (alias expansion is embed-side only), and reorders by RRF
	// fusion of vector + BM25 ranks. Default false — vector-only
	// behavior preserved as the baseline.
	//
	// The flag intentionally lives at the query-call level rather than
	// the Engine: comparison runs (`--bm25-rerank` vs no-flag) share
	// the same Engine + index, only the rerank step toggles.
	EnableBM25Rerank bool

	// EnableScoreBoost applies signature/doc/recent/package multipliers
	// to the rerank pass. Default false.
	EnableScoreBoost bool
	Boost            BoostOptions

	// EnableMetadataEnrichment attaches git log history to each hit.
	// Default false (calls git log per file, has IO cost).
	EnableMetadataEnrichment bool
	MaxHistoryCommits        int // 0 → 5
}

Options configure a single Search invocation. Zero values resolve to the documented defaults.

type RerankService

type RerankService struct{}

RerankService reorders candidate hits using BM25 + RRF fusion. Operates on the candidate set only (no global corpus), so cost is proportional to the number of candidates, not the index size.

func (*RerankService) Run

func (s *RerankService) Run(_ context.Context, candidates []types.Hit, intent string) ([]types.Hit, bm25.Stats)

Run reranks candidates by BM25 + RRF fusion against the given intent. Returns reordered hits and statistics.

func (*RerankService) RunContext

func (s *RerankService) RunContext(ctx context.Context, sc *SearchContext) error

RunContext reranks sc.RawHits if EnableBM25Rerank is set.

type Response

type Response struct {
	Hits []Hit `json:"hits"`
	// Examples holds test-file hits separated out from Hits when
	// Options.ExamplesK > 0. Nil/empty otherwise. The ranking inside
	// Examples follows the same score order as Hits — top of Examples
	// is the most-similar test result.
	Examples []Hit            `json:"examples,omitempty"`
	Warnings []string         `json:"warnings,omitempty"`
	Metadata ResponseMetadata `json:"metadata"`
}

Response is the full search response — hits plus diagnostics so MCP callers can report freshness/budget without an extra round trip.

type ResponseMetadata

type ResponseMetadata struct {
	TokensUsed     int    `json:"tokens_used"`
	IndexedHeadCKV string `json:"indexed_head_ckv"`
	Fresh          bool   `json:"fresh"`
	// TraceID echoes Options.TraceID (or the engine-generated fallback)
	// so callers can correlate this response with their footprint log
	// entries. Always set; non-empty.
	TraceID string `json:"trace_id,omitempty"`
	// DryRun mirrors Options.DryRun so callers reading only the
	// response can tell whether Hits actually came from the index.
	DryRun bool `json:"dry_run,omitempty"`
}

ResponseMetadata holds diagnostic metadata so the MCP layer can pass this through to LLM callers verbatim.

type SearchContext

type SearchContext struct {
	// Input (set by caller before pipeline starts)
	Intent      string
	EmbedIntent string // after alias expansion; same as Intent when no aliases
	Options     Options
	TraceID     string

	// After EmbedService
	QueryVec []float32

	// After StoreSearch
	RawHits []types.Hit

	// After Rerank + Threshold + Citation
	FilteredHits []types.Hit

	// After TestSplit
	PrimaryHits []types.Hit
	ExampleHits []types.Hit

	// After DensityAdjust
	FinalHits     []Hit
	FinalExamples []Hit
	TokensUsed    int

	// Accumulated warnings
	Warnings []string

	// Metadata
	DroppedCitations int
	StaleCitations   int
}

SearchContext carries the evolving state through the search pipeline. Each service reads its input fields, does its work, and writes its output fields. The Facade (Engine.Search) assembles the final Response from this context.

Individual services can also be called directly by passing a SearchContext with only the relevant fields populated.

type StoreSearchService

type StoreSearchService struct {
	// contains filtered or unexported fields
}

StoreSearchService runs approximate nearest neighbor search against the vector store. Over-fetches by overfetchFactor to give downstream reranking and filtering enough candidates.

func (*StoreSearchService) Run

func (s *StoreSearchService) Run(ctx context.Context, queryVec []float32, k int, filter types.Filter) ([]types.Hit, error)

Run performs vector search and returns raw candidate hits.

func (*StoreSearchService) RunContext

func (s *StoreSearchService) RunContext(ctx context.Context, sc *SearchContext) error

RunContext populates sc.RawHits from sc.QueryVec.

type ThresholdService

type ThresholdService struct{}

ThresholdService drops hits below the minimum normalized score.

func (*ThresholdService) Run

func (s *ThresholdService) Run(_ context.Context, hits []types.Hit, threshold float64) []types.Hit

Run filters hits below the threshold. Returns only passing hits.

func (*ThresholdService) RunContext

func (s *ThresholdService) RunContext(ctx context.Context, sc *SearchContext) error

RunContext applies threshold to sc.RawHits → sc.FilteredHits.

Directories

Path Synopsis
Package bm25 provides candidate-set BM25 rerank for CKV's query path.
Package bm25 provides candidate-set BM25 rerank for CKV's query path.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL