Documentation
¶
Overview ¶
Package query is the read path: open an index built by internal/build and serve semantic_search. It owns:
- manifest validation on Open (dim + model identity must match the caller's Embedder)
- over-fetch + threshold drop + citation enforcement
- snippet density compression under a token budget
Designed to be importable by both `ckv query` (CLI) and the future MCP server (`ckv mcp`) without code duplication.
Index ¶
- Constants
- Variables
- func CosineSimilarity(a, b []float32) float64
- func EnforceCitations(hits []types.Hit, srcRoot string) (keep []types.Hit, dropped int)
- func EnforceCitationsAt(hits []types.Hit, srcRoot, currentHead string, docsRoots ...string) (keep []types.Hit, dropped int, stale int)
- func ExpandQuery(intent string, aliases AliasMap) string
- type AliasFile
- type AliasMap
- type AlignmentReport
- type AlignmentStatus
- type BoostOptions
- type BoostService
- type BranchMatch
- type CommitSummary
- type ConventionHit
- type DensityService
- type DensityTier
- type EmbedService
- type EmbedderInfo
- type Engine
- func (e *Engine) CheckAlignment() AlignmentReport
- func (e *Engine) CheckFreshness() error
- func (e *Engine) Close() error
- func (e *Engine) Embed(ctx context.Context, text string) ([]float32, error)
- func (e *Engine) Embedder() types.Embedder
- func (e *Engine) EmbedderInfo() EmbedderInfo
- func (e *Engine) ExpandFlow(ctx context.Context, stepID, direction string, hops int) (*ExpandResult, error)
- func (e *Engine) ExpandInFile(ctx context.Context, chunkID string, before, after int) ([]Hit, error)
- func (e *Engine) ExplainMatch(ctx context.Context, chunkID, intent string) (*Explanation, error)
- func (e *Engine) FindBranches(ctx context.Context, symptom string, k int) ([]BranchMatch, error)
- func (e *Engine) FindInvariants(ctx context.Context, file, category string, tierMin int) ([]InvariantHit, error)
- func (e *Engine) FreshnessReport() (freshness.Report, error)
- func (e *Engine) GetConventions(ctx context.Context, packagePrefix string) ([]ConventionHit, error)
- func (e *Engine) GetFlow(ctx context.Context, sel FlowSelector) (*FlowView, error)
- func (e *Engine) GetInvariantEnforcement(ctx context.Context, invID string) (*InvariantEnforcement, error)
- func (e *Engine) KeywordSearch(ctx context.Context, query string, k int, filter types.Filter) ([]Hit, error)
- func (e *Engine) LookupPRsByFile(ctx context.Context, file string) ([]types.PRRef, error)
- func (e *Engine) Manifest() manifest.Manifest
- func (e *Engine) NarrowCandidates(ctx context.Context, ids []string, filter types.Filter) ([]Hit, error)
- func (e *Engine) OutDir() string
- func (e *Engine) Rerank(ctx context.Context, candidates []types.Hit, intent string) ([]types.Hit, error)
- func (e *Engine) ResolvedVersion() string
- func (e *Engine) Search(ctx context.Context, intent string, opts Options) (*Response, error)
- func (e *Engine) VectorSearch(ctx context.Context, queryVec []float32, k int, filter types.Filter) ([]types.Hit, error)
- func (e *Engine) Warmup(ctx context.Context) error
- type EnrichService
- type ExpandResult
- type Explanation
- type ExplanationKeywordScore
- type ExplanationVectorScore
- type FlowNeighbor
- type FlowSelector
- type FlowStepView
- type FlowView
- type HallucinationResult
- type Hit
- type InvariantEnforcement
- type InvariantHit
- type KeywordIndex
- type OpenOption
- type Options
- type RerankService
- type Response
- type ResponseMetadata
- type SearchContext
- type StoreSearchService
- type ThresholdService
Constants ¶
const DefaultBudgetTokens = 4000
DefaultBudgetTokens is the response-side token budget. 4000 tokens ≈ 16K chars of snippet content across all hits.
const DefaultK = 10
DefaultK is the top-K when the caller omits it.
const DefaultSignatureContextLines = 5
DefaultSignatureContextLines is how many lines after the signature we keep in the SignatureWithContext density tier when the caller doesn't override Options.SignatureContextLines.
const DefaultThreshold = 0.4
DefaultThreshold drops hits whose normalized score is below this value. 0.4 is conservative — with the mock embedder it trims obvious noise without rejecting real matches.
const MinBudgetTokens = 20
MinBudgetTokens is the floor below which BudgetTokens can't even fit a single signature-density hit. Set so a one-line Go signature (~50-80 chars) rounds up to ~20 tokens with one hit; below this the engine returns ErrBudgetExceeded rather than truncate further.
Variables ¶
var ErrBudgetExceeded = errors.New("query: budget exceeded")
ErrBudgetExceeded signals SearchOptions.BudgetTokens is too small for the engine to render even a minimum-density response. The engine degrades snippets to signature-only before raising; reaching this error means the budget is below the floor (MinBudgetTokens).
Caller guidance: raise BudgetTokens (defaults to 4000), or accept an empty response by passing BudgetTokens<0 to disable budgeting.
var ErrChunkNotFound = errors.New("chunk not found")
ErrChunkNotFound is returned when ExpandInFile / NarrowCandidates is asked about a chunk ID that does not exist in the index.
var ErrCitationNotFound = errors.New("query: citation not found")
ErrCitationNotFound signals catastrophic citation enforcement failure: every candidate that passed the threshold gate was dropped because its file could not be located under the recorded src_root. Almost always indicates the source tree was deleted or moved without rebuilding the index.
Caller guidance: re-run `ckv build --src <current path>`. Until then the index is unusable for retrieval — every hit would be unverifiable.
var ErrFreshnessStale = errors.New("query: index freshness stale")
ErrFreshnessStale signals the index's recorded IndexedHead is behind the source tree's current git HEAD. Returned by the freshness helpers (Engine.CheckFreshness, freshness.Check) when callers opt in to strict freshness; the on-the-wire Response keeps the soft Metadata.Fresh bool for callers that only need a hint.
Caller guidance: the result set is still usable for many cases (recent edits typically affect a small subset of files). Schedule a reindex when convenient. Prompt-time freshness sensitivity is the caller's taste.
ErrIndexUnavailable signals that the on-disk index cannot be served by the supplied Embedder. Most commonly: the indexed model identity differs from the query-time model, the dimension is different, or the manifest is missing.
Caller guidance: surface to the operator with a "run `ckv build`" hint. Do not retry — the index will not change without a rebuild.
var ErrPolicyError = errors.New("query: policy error")
ErrPolicyError signals a policy or authorization check rejected the request. Typical raises: mTLS caller-cert SAN mismatch with envelope caller, intent flagged by content policy, or an internal-only tool requested through an external surface.
Caller guidance: this is a hard rejection — do not retry. Surface to the operator. Defined now for forward-compatible callers.
var ErrSanitizeFailed = errors.New("query: sanitize failed")
ErrSanitizeFailed signals the sanitize pipeline rejected the response payload. The sanitize module is the default-deny gate between raw retrieval and outbound delivery — secrets, PII, oversized blobs, or policy-flagged content land here.
Caller guidance: log the rejection (sanitize_report carries the reason), do not retry with the same intent. Defined now for forward-compatible callers.
Functions ¶
func CosineSimilarity ¶
CosineSimilarity is a small helper for the explain pathway in tests and other callers that want a direct vector-vs-vector distance instead of going through the store's KNN. Returns 0 when either vector is empty or differs in length.
func EnforceCitations ¶
EnforceCitations verifies every hit's citation against the on-disk source tree at srcRoot, returning the surviving hits and the count of dropped ones. Citation accuracy must be 100%, so any hit we can't verify is silently dropped (with a warning aggregated by the caller).
Verification:
- file exists at srcRoot/<rel>
- start_line ≥ 1 and end_line ≥ start_line
Commit-hash mismatch (stale citation) is NOT a drop — the file almost always still has useful content, just at a different commit than when it was indexed. Callers see the stale count returned alongside dropped and surface a warning. Use EnforceCitationsAt to enable the stale check.
When srcRoot is empty (no source tree available; we're running against a moved index), every hit is allowed through with a warning.
func EnforceCitationsAt ¶
func EnforceCitationsAt(hits []types.Hit, srcRoot, currentHead string, docsRoots ...string) (keep []types.Hit, dropped int, stale int)
EnforceCitationsAt extends EnforceCitations with stale detection: when currentHead is non-empty, every surviving hit whose Chunk.CommitHash differs gets Hit.StaleCitation=true and counts toward the stale return. Useful diagnostic when callers can detect "the index is post-reindex-needed but still served." Empty currentHead disables the check; stale stays 0.
The surviving slice's order matches the input; only the metadata on each hit is touched. Dropped hits are removed as before — line validity and file existence remain hard requirements. docsRoots are additional roots (manifest.DocsRoots — the `ckv build --docs` corpus dirs) used to resolve doc/markdown chunk citations whose File is relative to a corpus dir rather than the code srcRoot. A citation is valid if its file exists under srcRoot OR any docsRoot; without this, every domain-corpus chunk is dropped because it never resolves under the code tree.
func ExpandQuery ¶
ExpandQuery widens intent with any alias entry whose key appears as a substring of intent. Output shape:
"<intent> [aliases: <kw1>, <kw2>, ...]"
Returned intent is identical to input when:
- aliases is nil or empty
- no alias key appears in intent
The trailer is deterministic: alias keys are matched in sorted order and matched-keyword lists are deduplicated + sorted before joining. Determinism matters for the per-query footprint fingerprint (query.embed sub-span) so two runs of the same intent produce the same trace fingerprint.
Why substring match (not exact / token-aware): the intent is free-form natural language, often Korean with spaces / particles ("0번 블록의", "합의 알고리즘이"). Substring covers the common case; false positives are rare for the curated glossary we ship.
Types ¶
type AliasFile ¶
type AliasFile struct {
Aliases AliasMap `yaml:"aliases"`
}
AliasFile is the on-disk YAML shape. We keep the top-level `aliases` key explicit so future additions (e.g., `metadata`, `version`) can land without breaking the loader.
type AliasMap ¶
AliasMap maps a "human" phrase (often Korean or domain-vague) to a list of "code" keywords the embedder is more likely to match.
Example:
"0번 블록" → [genesis, genesis_block, GenesisAlloc] "합의 알고리즘" → [consensus, wbft, WBFT] "시스템 컨트랙트" → [system contract, systemcontracts, SystemContracts]
Used by ExpandQuery to widen the intent string at query time. The raw intent stays first so semantic ordering isn't disturbed; aliases are appended in a bracket-tagged trailer so the embedder treats them as auxiliary context.
The map can be hand-curated (testdata/stablenet/glossary.yaml) or auto-generated by a glossary loader.
func LoadAliasMap ¶
LoadAliasMap reads a glossary YAML and returns the parsed map. Returns nil + nil error when path is empty — callers can pass the CLI flag through unconditionally and the no-flag case stays a no-op. Empty `aliases` block is treated the same as no file (nil return).
type AlignmentReport ¶
type AlignmentReport struct {
Status AlignmentStatus `json:"status"`
RecordedCommit string `json:"recorded_commit,omitempty"`
CurrentCommit string `json:"current_commit,omitempty"`
RecordedDigest string `json:"recorded_digest,omitempty"`
CurrentDigest string `json:"current_digest,omitempty"`
SchemaVersion string `json:"schema_version,omitempty"`
CKGPath string `json:"ckg_path,omitempty"`
Reason string `json:"reason,omitempty"`
Warnings []string `json:"warnings,omitempty"`
}
AlignmentReport is the structured result of CheckAlignment. The authoritative keys are src_commit + graph_digest (design §3.1); the src_root path is NOT compared here (a different checkout of the same commit is legal — that concern belongs to the serving layer, as a warning).
func (AlignmentReport) Serviceable ¶
func (r AlignmentReport) Serviceable() bool
Serviceable reports whether the alignment is trustworthy enough to serve canonical_id joins: ok / degraded / not_aligned are serviceable; only mismatch (commit or digest divergence) is not.
type AlignmentStatus ¶
type AlignmentStatus string
AlignmentStatus classifies the CKG↔CKV alignment.
const ( AlignmentOK AlignmentStatus = "ok" // commit + digest match AlignmentDegraded AlignmentStatus = "degraded" // commit matches, digest unverifiable / schema<1.19 AlignmentMismatch AlignmentStatus = "mismatch" // commit or digest differ (join broken) — re-align needed AlignmentNotAligned AlignmentStatus = "not_aligned" // index built without --ckg )
type BoostOptions ¶
type BoostOptions struct {
// SignatureMatch boosts hits whose symbol name or signature contains
// query keywords. Default multiplier: 1.5
SignatureMatch bool
SignatureMultiplier float64
// DocMatch boosts hits whose leading comment block contains query
// keywords. Default multiplier: 1.3
DocMatch bool
DocMultiplier float64
// RecentModified boosts hits whose commit_hash equals the indexed
// HEAD (treated as "recently modified"). Default multiplier: 1.1
// For real git log-based recency, the caller can pre-compute and
// pass via a custom Score override.
RecentModified bool
RecentMultiplier float64
// PackageProximity boosts hits whose file path contains the given
// package keyword. Default multiplier: 1.2
PackageProximity bool
PackageKeyword string
PackageMultiplier float64
// IndexedHead is used by RecentModified to compare chunk.CommitHash.
// Set by the engine before calling Run.
IndexedHead string
}
BoostOptions controls score boosting after vector + BM25 ranking. All multipliers are applied independently and combined multiplicatively.
func DefaultBoostOptions ¶
func DefaultBoostOptions() BoostOptions
Defaults returns the recommended boost multipliers.
type BoostService ¶
type BoostService struct{}
BoostService applies score boosting rules to candidate hits.
func (*BoostService) Run ¶
func (s *BoostService) Run(hits []types.Hit, intent string, opts BoostOptions) []types.Hit
Run reorders hits by boosted scores. The hit's Score.Normalized is updated with the boosted value; the original raw distance is preserved. Stable sort: hits with equal boosted scores keep their input order.
func (*BoostService) RunContext ¶
func (s *BoostService) RunContext(sc *SearchContext, indexedHead string)
RunContext applies boost to sc.RawHits when Options.EnableScoreBoost.
type BranchMatch ¶
type BranchMatch struct {
When string `json:"when"`
Then string `json:"then"`
At string `json:"at"`
StepID string `json:"step_id"`
FlowID string `json:"flow_id"`
Symbol string `json:"symbol,omitempty"`
Citation types.Citation `json:"citation"`
Score float64 `json:"score"`
}
BranchMatch is a failure branch surfaced for a symptom query.
type CommitSummary ¶
type CommitSummary struct {
Hash string `json:"hash"`
Author string `json:"author"`
Date string `json:"date"`
Subject string `json:"subject"`
}
CommitSummary is one git log entry attached to a hit.
type ConventionHit ¶
type ConventionHit struct {
ChunkID string `json:"chunk_id"`
File string `json:"file"`
Package string `json:"package"`
Summary string `json:"summary"`
Stats map[string]any `json:"stats"`
}
ConventionHit returns the raw ConventionStats plus the human summary the agent can present without further processing.
type DensityService ¶
type DensityService struct{}
DensityService adjusts snippet length to fit within a token budget. Longer snippets are progressively compressed (full → signature+context → signature-only) until the total fits.
func (*DensityService) Run ¶
func (s *DensityService) Run(_ context.Context, hits []types.Hit, budget int, maxDensity DensityTier, sigContextLines int) ([]Hit, int)
Run adjusts density for combined hits (primary + examples) under the given token budget. Returns formatted Hit slice and total tokens used.
func (*DensityService) RunContext ¶
func (s *DensityService) RunContext(ctx context.Context, sc *SearchContext) error
RunContext reads sc.PrimaryHits + sc.ExampleHits, adjusts density, and writes sc.FinalHits + sc.FinalExamples + sc.TokensUsed.
type DensityTier ¶
type DensityTier string
DensityTier names the three snippet shapes the engine can emit.
- DensityFull — chunk's complete Text. Default when budget room allows.
- DensitySignature5 — first non-blank line (the signature) plus the next N non-blank lines. Middle compression tier; useful for "what does this function start by doing".
- DensitySignatureOnly — first non-blank line only (just the signature). Minimum useful representation; caller resolves the body via Citation if they need more.
Reported on every Hit (Hit.Density) so consumers can render progressively or count how aggressively the budget squeezed them.
const ( DensityFull DensityTier = "full" DensitySignature5 DensityTier = "signature+N" DensitySignatureOnly DensityTier = "signature_only" )
type EmbedService ¶
type EmbedService struct {
// contains filtered or unexported fields
}
EmbedService converts text into a vector using the configured embedder. Can be called independently via Engine.Embed() or as part of the search pipeline.
func (*EmbedService) RunContext ¶
func (s *EmbedService) RunContext(ctx context.Context, sc *SearchContext) error
RunContext populates sc.QueryVec from sc.EmbedIntent.
type EmbedderInfo ¶
type EmbedderInfo struct {
Name string `json:"name"`
Dimension int `json:"dimension"`
// Status: "ready" | "stub" | "unavailable"
// ready — embedder loaded, semantic signal expected
// stub — placeholder embedder (mock, hash-based); recall not meaningful
// unavailable — embedder is nil or failed to attach
Status string `json:"status"`
// Provider is the execution backend tag (bgeonnx-specific):
// "coreml" | "coreml-fallback-to-cpu" | "cpu" | "". Empty for
// embedders that don't run a backend (mock).
Provider string `json:"provider,omitempty"`
// ModelDir is the on-disk model directory the embedder loaded from.
// Empty for embedders that don't have one (mock).
ModelDir string `json:"model_dir,omitempty"`
}
EmbedderInfo is the health-endpoint view of the embedder this Engine was opened with. Status answers "is this embedder useful for real semantic search?" — operators can distinguish "ckv alive but mock embedder" from "ckv ready with bgeonnx + model loaded" without inspecting the model name string.
Optional fields (Provider, ModelDir) are empty when the embedder implementation doesn't expose them — health consumers must tolerate the empty case.
type Engine ¶
type Engine struct {
// contains filtered or unexported fields
}
Engine is the long-lived query handler. Hold one per --out directory. Concurrency-safe: store.Search is read-only; we never mutate Engine state after Open (footprint Logger is sync-safe by contract).
func Open ¶
Open loads the index at outDir and validates the manifest against the supplied Embedder. Mismatches produce ErrIndexUnavailable with a human-readable cause; callers (CLI / MCP) surface a "ckv reindex" hint to the user. opts apply after the engine is constructed but before it is returned.
func (*Engine) CheckAlignment ¶
func (e *Engine) CheckAlignment() AlignmentReport
CheckAlignment compares the recorded sources.ckg coordinates against the CKG graph currently at the recorded path. Returns not_aligned when the index was built without --ckg. Best-effort: an unreadable graph is a mismatch (the aligned-against graph is gone).
func (*Engine) CheckFreshness ¶
CheckFreshness compares the loaded manifest's IndexedHead against the source tree's current git HEAD and returns ErrFreshnessStale (wrapped with the diff) when they differ. Returns nil when fresh, no error and no report when git is unavailable (callers who need the Report directly should use internal/freshness.Check).
Caller guidance for ErrFreshnessStale: stale results are usually still usable for most retrieval (recent edits affect a small subset of files), but the caller should schedule `ckv build` when convenient.
func (*Engine) Embed ¶
Embed exposes the embedding service for independent use. Converts text into a vector without running the full search pipeline.
func (*Engine) Embedder ¶
Embedder returns the underlying embedder. Callers (MCP index handler) reuse it to avoid re-initializing the embedder for build/reindex.
func (*Engine) EmbedderInfo ¶
func (e *Engine) EmbedderInfo() EmbedderInfo
EmbedderInfo extracts metadata via duck typing — bgeonnx exposes Provider() and ModelDir(), the mock exposes neither. The returned struct is JSON-safe (no unset zero-value confusion for omitted fields) so health handlers can serialize it directly.
func (*Engine) ExpandFlow ¶
func (e *Engine) ExpandFlow(ctx context.Context, stepID, direction string, hops int) (*ExpandResult, error)
ExpandFlow returns the steps adjacent to stepID up to `hops` away, following downstream `calls` (direction "down") or upstream `called_by` ("up"), plus the origin step's own failure branches.
func (*Engine) ExpandInFile ¶
func (e *Engine) ExpandInFile(ctx context.Context, chunkID string, before, after int) ([]Hit, error)
ExpandInFile returns chunks immediately surrounding chunkID inside the same file. The window is [match - before, match + after] in chunk-index space (chunks ordered by start_line). When chunkID is unknown, returns ErrChunkNotFound. Score is zero on the returned hits — they are context, not ranked results.
before / after of 0 returns just the source chunk. before or after over the edge are silently clamped (no error for "ran out of chunks").
func (*Engine) ExplainMatch ¶
ExplainMatch reports why a particular chunk would have matched the intent: vector similarity computed fresh against the embedder, and BM25 details from the keyword index. When the chunk_id is unknown, returns ErrChunkNotFound.
The function is pure-read and does no caching beyond the keyword index already cached on the Engine. Cost is roughly:
- 1 embed call (the intent vector)
- 1 KNN-style cosine distance against the chunk's stored vector
- 1 BM25 lookup against the in-memory index
func (*Engine) FindBranches ¶
FindBranches maps a symptom phrase to the failure branches of the flow steps most relevant to it (semantic search over flow chunks, whose embed text already includes each branch's "when"). Requires a real embedder.
func (*Engine) FindInvariants ¶
func (e *Engine) FindInvariants(ctx context.Context, file, category string, tierMin int) ([]InvariantHit, error)
FindInvariants returns invariants matching the filter. tierMin (1, 2, or 3) drops anything below that confidence tier. The tier is read from the source chunk's InvariantRef back-pointer when the source is in the index; otherwise the invariant is included at its declared SymbolName-based tier where possible.
func (*Engine) FreshnessReport ¶
FreshnessReport returns the structured index-vs-HEAD comparison (IndexedHead, CurrentHead, ChangedFiles, Stale, Fresh, Warnings). Unlike CheckFreshness (which collapses the comparison to a single error), this exposes the full Report so the pkg/ckv facade can hand it to cks's cks.ops.freshness tool without re-deriving it. Git unavailability is reported via Report.Warnings (Fresh=false), not as a hard error — same contract as freshness.Check.
func (*Engine) GetConventions ¶
GetConventions returns convention chunks under the package prefix. Each Hit carries the deterministic prose summary plus the raw stats map so the agent can use whichever shape its prompt expects.
func (*Engine) GetFlow ¶
GetFlow lays out a flow's steps in call (topological) order. Selector must set exactly one of FlowID / EntryPoint / InvariantID.
func (*Engine) GetInvariantEnforcement ¶
func (e *Engine) GetInvariantEnforcement(ctx context.Context, invID string) (*InvariantEnforcement, error)
GetInvariantEnforcement lists every (flow, step, loc) where a curated invariant is enforced.
func (*Engine) KeywordSearch ¶
func (e *Engine) KeywordSearch(ctx context.Context, query string, k int, filter types.Filter) ([]Hit, error)
KeywordSearch returns the top-k BM25 hits for the natural-language query. The first call after Engine.Open builds the in-memory index (cached for subsequent calls). Filter is applied post-rank — we over-fetch by 3x when a filter is set, mirroring Store.Search.
func (*Engine) LookupPRsByFile ¶
LookupPRsByFile delegates to the store's PR breadcrumb lookup.
func (*Engine) Manifest ¶
Manifest returns a copy of the loaded manifest. Callers (ckv freshness) use this to compare on-disk identity without reopening.
func (*Engine) NarrowCandidates ¶
func (e *Engine) NarrowCandidates(ctx context.Context, ids []string, filter types.Filter) ([]Hit, error)
NarrowCandidates loads the chunks for the given IDs and returns the subset that matches the filter. Score fields are zero-valued (this is a metadata refinement step, not a ranking step). Useful for multi-hop retrieval flows where an earlier semantic_search produced hit IDs and a follow-up wants to drop everything outside a category or path.
func (*Engine) Rerank ¶
func (e *Engine) Rerank(ctx context.Context, candidates []types.Hit, intent string) ([]types.Hit, error)
Rerank reorders candidate hits using BM25 + RRF fusion.
func (*Engine) ResolvedVersion ¶
ResolvedVersion returns the concrete on-disk directory name the data path resolves to, following a `current` symlink. For a blue-green dataset served via <dataset>/current it reports the promoted version dir; for a plain data dir it reports that dir's name. Empty when the path can't be resolved. Lets health show which version is being served (reindex-migration-design §6).
func (*Engine) Search ¶
Search runs the full pipeline (Facade): embed → vector search → rerank → threshold → citation → test split → density adjust.
For fine-grained control, use the individual service methods (Embed, VectorSearch, Rerank) and compose them manually.
func (*Engine) VectorSearch ¶
func (e *Engine) VectorSearch(ctx context.Context, queryVec []float32, k int, filter types.Filter) ([]types.Hit, error)
VectorSearch runs ANN search against the store without reranking, threshold, or citation. Returns raw candidate hits.
func (*Engine) Warmup ¶
Warmup forces the embedder to pay its cold-start cost (ONNX session load, CoreML compile, tokenizer lazy init) before the first user-facing query lands. Runs a single dummy Embed call. Returns the embedder's error if it fails to attach — callers (health, CLI) should surface it so the operator can investigate before traffic starts.
Idempotent and cheap to call repeatedly: after the first call the embedder is already warm so subsequent calls just round-trip a short batch.
type EnrichService ¶
type EnrichService struct{}
EnrichService attaches git history to each hit. Implements Phase 3 spec Step 3 (Metadata Enrichment). Per-file results are cached within a single Run to avoid duplicate git log calls when multiple hits share a file.
func (*EnrichService) Run ¶
Run executes `git log` for each unique file in hits and attaches the results. srcRoot is the repo root. maxCommits is the per-file history depth (5 matches the Phase 3 spec).
func (*EnrichService) RunContext ¶
func (s *EnrichService) RunContext(ctx context.Context, sc *SearchContext, srcRoot string)
RunContext attaches git history to sc.FinalHits and sc.FinalExamples when Options.EnableMetadataEnrichment is set.
type ExpandResult ¶
type ExpandResult struct {
Origin string `json:"origin"`
Direction string `json:"direction"`
OriginBranches []types.Branch `json:"origin_branches,omitempty"`
Neighbors []FlowNeighbor `json:"neighbors"`
}
ExpandResult is the neighborhood of one step plus that step's own branches (the failure conditions at the origin).
type Explanation ¶
type Explanation struct {
ChunkID string `json:"chunk_id"`
Citation types.Citation `json:"citation"`
Vector ExplanationVectorScore `json:"vector_score"`
Keyword ExplanationKeywordScore `json:"keyword_score"`
Category string `json:"category,omitempty"`
Guidance *types.ModificationGuidance `json:"guidance,omitempty"`
Symbol string `json:"symbol,omitempty"`
}
Explanation is the full explain_match response.
type ExplanationKeywordScore ¶
type ExplanationKeywordScore struct {
Score float64 `json:"score"`
MatchedTokens []string `json:"matched_tokens"`
QueryTokens []string `json:"query_tokens"`
ChunkTokenSize int `json:"chunk_token_size"`
}
ExplanationKeywordScore captures BM25-side detail: the raw Okapi score over the chunk's tokens against the query, plus which tokens actually matched (intersection of query tokens and chunk tokens).
type ExplanationVectorScore ¶
type ExplanationVectorScore struct {
Normalized float64 `json:"normalized"`
CosineDistance float64 `json:"cosine_distance"`
}
ExplanationVectorScore is the vector-side of explain_match.
type FlowNeighbor ¶
type FlowNeighbor struct {
StepID string `json:"step_id"`
Symbol string `json:"symbol,omitempty"`
Citation types.Citation `json:"citation"`
Relation string `json:"relation"` // "calls" (downstream) | "called_by" (upstream)
}
FlowNeighbor is an adjacent step reached from an origin step.
type FlowSelector ¶
type FlowSelector struct {
FlowID string
EntryPoint string
InvariantID string // resolves to the first flow in the invariant's enforced_at
}
FlowSelector picks a flow by exactly one of its keys.
type FlowStepView ¶
type FlowStepView struct {
StepID string `json:"step_id"`
Symbol string `json:"symbol,omitempty"`
Citation types.Citation `json:"citation"`
Kind string `json:"kind,omitempty"`
Calls []string `json:"calls,omitempty"`
Reads string `json:"reads,omitempty"`
Writes string `json:"writes,omitempty"`
Emits string `json:"emits,omitempty"`
Branches []types.Branch `json:"branches,omitempty"`
Invariants []string `json:"invariants,omitempty"`
}
FlowStepView is one step in a laid-out flow.
type FlowView ¶
type FlowView struct {
FlowID string `json:"flow_id"`
EntryPoint string `json:"entry_point,omitempty"`
Trigger string `json:"trigger,omitempty"`
RootSymbol string `json:"root_symbol,omitempty"`
Links []string `json:"links,omitempty"`
CalledBy []string `json:"called_by,omitempty"`
Steps []FlowStepView `json:"steps"`
}
FlowView is a flow's spine plus its steps in call (topological) order.
type HallucinationResult ¶
type HallucinationResult struct {
Verified bool `json:"verified"`
Reason string `json:"reason,omitempty"` // empty when Verified=true
// File-level diagnostics for false cases. Populated only when
// Verified=false so the JSON payload stays compact for the common
// case.
ExpectedFile string `json:"expected_file,omitempty"`
}
HallucinationResult is the verdict for one hit-vs-source comparison. Verified=true means the snippet's content actually exists at the claimed file location; Verified=false flags a hallucination signal with a short Reason so log readers can debug without re-running.
The operational definition: a hit is *not* a hallucination if its snippet appears as a (whitespace-normalized) substring of the cited file's line range. This handles every density tier — DensitySignatureOnly is naturally a substring of a longer body, DensityFull is the full body itself.
func VerifyHit ¶
func VerifyHit(h Hit, srcRoot string) HallucinationResult
VerifyHit reports whether h.Snippet aligns with the source file at srcRoot/<h.Citation.File>. Three failure modes are distinguished so log readers can route the response: "file_missing" (citation points nowhere), "out_of_range" (line numbers exceed file length), and "snippet_not_found" (content disagrees — the actual hallucination).
Whitespace normalization: tabs / spaces / newlines collapse to a single space and leading / trailing whitespace is trimmed. This matches how a reader would compare the snippet against the file (formatting differences are not hallucinations).
Empty srcRoot returns Verified=false with reason="no_src_root" — the caller asked for verification but didn't provide a tree to verify against.
func VerifyResponse ¶
func VerifyResponse(resp *Response, srcRoot string) (verdicts []HallucinationResult, hallucinated int)
VerifyResponse runs VerifyHit across both Hits and Examples of the response and aggregates the verdicts. Returns the per-hit results (in response order: hits first, then examples) plus a count of non-verified hits. Convenient for eval pipelines that just want a hallucination rate.
type Hit ¶
type Hit struct {
ChunkID string `json:"chunk_id"`
Citation types.Citation `json:"citation"`
Snippet string `json:"snippet"`
// Density names which 3-tier ladder rung this Snippet was rendered
// at (DensityFull / DensitySignature5 / DensitySignatureOnly).
// Useful for downstream UIs that want to badge compressed hits or
// for eval pipelines counting how often the budget forced a
// downgrade. Omitted when empty (e.g. callers building Hit by hand).
Density DensityTier `json:"density,omitempty"`
// StaleCitation propagates from types.Hit: the chunk's recorded
// commit_hash disagrees with the current source-tree HEAD. The
// citation file/lines still resolve — the content there may have
// shifted since the index was built. Consumers can render a badge
// or schedule a reindex. Omitted when false.
StaleCitation bool `json:"stale_citation,omitempty"`
Score types.HitScore `json:"score"`
Language string `json:"language"`
IsTest bool `json:"is_test,omitempty"`
Symbol string `json:"symbol,omitempty"`
SymbolKind types.SymbolKind `json:"symbol_kind,omitempty"`
// ChunkKind classifies the chunk's origin strategy (symbol,
// file_header, doc, invariant, convention, flow_*). Lets consumers
// (cks knowledge quota) route knowledge chunks without re-querying.
ChunkKind types.ChunkKind `json:"chunk_kind,omitempty"`
CanonicalID string `json:"canonical_id,omitempty"` // ckg's import-path-qualified symbol id (ADR-0001); the stable key cks uses to FindByCanonicalID against ckg
// Category and Guidance are populated by the policy loader at build
// time. Category labels the chunk's domain ("consensus", "state",
// ...); Guidance lists what the agent should also review, test, and
// watch out for when modifying this code. Both omitted when the
// chunk did not match any policy rule (or no policy was loaded).
Category string `json:"category,omitempty"`
Guidance *types.ModificationGuidance `json:"guidance,omitempty"`
// GitHistory holds recent commits touching this hit's file.
// Populated when Options.EnableMetadataEnrichment is set.
GitHistory []CommitSummary `json:"git_history,omitempty"`
}
Hit is the response-shaped record: only what callers (LLM, CLI) need. We deliberately omit Chunk.Text — Snippet is the budget-adjusted view.
func DensityAdjust ¶
DensityAdjust converts store hits into response Hits with snippets sized to fit budgetTokens. ctxLines tunes the SignatureWithContext tier (0 → DefaultSignatureContextLines). cap caps the maximum tier the engine will emit ("" → DensityFull, no cap).
Algorithm:
- Start every hit at `cap` (or DensityFull when no cap).
- If the total token count exceeds budget, downgrade hits one by one from the lowest-ranked (worst score) upward: full → signature+N → signature only.
- Stop as soon as the running total fits, or every hit is at the minimum.
Each Hit's Density field reports the final tier it landed in. Citation is *not* counted against the budget. Returns the response hits plus the final tokensUsed estimate.
func DensityAdjustWith ¶
func DensityAdjustWith(hits []types.Hit, budgetTokens int, cap DensityTier, ctxLines int) (out []Hit, tokensUsed int)
DensityAdjustWith is the configurable variant exposing the density ceiling and context-line knob. DensityAdjust is the convenience wrapper that uses documented defaults.
type InvariantEnforcement ¶
type InvariantEnforcement struct {
InvID string `json:"inv_id"`
Statement string `json:"statement,omitempty"`
EnforcedAt []types.EnforcePoint `json:"enforced_at"`
}
InvariantEnforcement lists every place a curated invariant is enforced.
type InvariantHit ¶
type InvariantHit struct {
ChunkID string `json:"chunk_id"`
File string `json:"file"`
StartLine int `json:"start_line"`
EndLine int `json:"end_line"`
Marker string `json:"marker"` // e.g. "CRITICAL", "panic"
Tier types.InvariantTier `json:"tier"` // 1, 2, or 3
Text string `json:"text"` // the invariant statement
Category string `json:"category,omitempty"` // inherited from source chunk's policy
Guidance *types.ModificationGuidance `json:"guidance,omitempty"` // ditto
SourceChunk string `json:"source_chunk_id,omitempty"`
}
InvariantHit pairs a ChunkInvariant chunk with its parsed tier so MCP callers can filter without reparsing the back-reference.
type KeywordIndex ¶
type KeywordIndex struct {
// contains filtered or unexported fields
}
KeywordIndex is a lazily-built in-memory BM25 index over every chunk in the store. It rebuilds on first KeywordSearch after Engine.Open; subsequent searches reuse the cached index.
Rebuild on Open is the right trade-off for CKV's expected scale: rebuilding 50k chunks takes well under a second on modern hardware, and the index then serves arbitrary keyword queries with no per-call IO. For sustained high-write workloads a future revision can move this into the persistence layer.
Build serialization is handled by Engine.kwMu — KeywordIndex itself is read-only after construction so it needs no internal lock.
type OpenOption ¶
type OpenOption func(*Engine)
OpenOption customizes Engine construction (functional options).
func WithFootprint ¶
func WithFootprint(fp *footprint.Logger) OpenOption
WithFootprint attaches a logger to the Engine. Every Search emits a span (query.search.start / query.search.done) including intent hash, hit count, citation drops, and latency. Nil-safe.
type Options ¶
type Options struct {
K int // top-K (0 → DefaultK)
Filter types.Filter // pre-filter (lang / path / kind / commit)
BudgetTokens int // snippet density budget (0 → DefaultBudgetTokens)
Threshold float64 // min normalized score (0 → DefaultThreshold; <0 disables)
SrcRoot string // absolute path used by citation enforcement;
// ExamplesK splits test-file hits out of the main Hits slice into a
// separate Examples slice in the response. Up to ExamplesK test
// chunks pass through — distinct from K, which counts only the
// non-test (primary implementation) hits. Defaults to 0 → no
// separation, every hit goes through Hits as before.
//
// Why: an LLM coding agent gets cleaner signal when primary code
// (the actual implementation it should mimic) is separate from
// usage examples (tests that show how the code is called). With
// them mixed, top-5 can be diluted by 2-3 test results that compete
// for context window space.
ExamplesK int
// TraceID is a caller-supplied correlation ID propagated to the
// footprint span and echoed in Response.Metadata.TraceID. Useful
// when a single user action fans out into multiple Search calls
// (CKS multiplex, retries) and the operator wants to grep all
// related log entries together. Empty → engine generates one from
// the intent hash + a monotonic suffix.
TraceID string
// DryRun, when true, validates the request (intent, embedder,
// manifest identity) but skips embedding, store search, citation
// enforcement, and density adjustment. The response carries
// metadata only — Hits and Examples are empty. Useful for caller-
// side budget / plan validation without paying the query cost
// or polluting the footprint log with hot-path entries.
DryRun bool
// MaxDensity caps the snippet rendering tier (B3 ladder).
// "" / DensityFull → no cap, downgrade only under budget pressure
// (the documented default). DensitySignature5 → start every hit at
// signature+N and downgrade to signature_only under pressure.
// DensitySignatureOnly → emit signatures only, never bodies; useful
// when the caller already has the chunk body cached and just wants
// pointers (e.g. CLI list-mode).
MaxDensity DensityTier
// SignatureContextLines tunes the SignatureWithContext tier (number
// of non-blank lines kept after the signature). 0 →
// DefaultSignatureContextLines (5). Larger values keep more body
// in the middle tier at the cost of fewer hits fitting the budget.
SignatureContextLines int
// Aliases is the vocabulary-bridge glossary applied to the intent
// at the very start of Search. When non-nil, ExpandQuery
// widens the intent with code-keyword aliases before embedding —
// useful for Korean / domain-vague queries against an English code
// corpus. Nil leaves the intent untouched (off by default).
//
// The expanded intent is what gets embedded and what appears in
// the query.embed sub-span fingerprint; the *original* intent is
// still echoed in the query.search top-level span for log triage.
Aliases AliasMap
// EnableBM25Rerank turns on the optional BM25 rerank pass between
// store.search and threshold.drop. When true, the engine builds a
// candidate-set BM25 over the vector hits (corpus = chunk.SymbolName
// + first text line), scores each candidate against the *original*
// intent (alias expansion is embed-side only), and reorders by RRF
// fusion of vector + BM25 ranks. Default false — vector-only
// behavior preserved as the baseline.
//
// The flag intentionally lives at the query-call level rather than
// the Engine: comparison runs (`--bm25-rerank` vs no-flag) share
// the same Engine + index, only the rerank step toggles.
EnableBM25Rerank bool
// EnableScoreBoost applies signature/doc/recent/package multipliers
// to the rerank pass. Default false.
EnableScoreBoost bool
Boost BoostOptions
// EnableMetadataEnrichment attaches git log history to each hit.
// Default false (calls git log per file, has IO cost).
EnableMetadataEnrichment bool
MaxHistoryCommits int // 0 → 5
}
Options configure a single Search invocation. Zero values resolve to the documented defaults.
type RerankService ¶
type RerankService struct{}
RerankService reorders candidate hits using BM25 + RRF fusion. Operates on the candidate set only (no global corpus), so cost is proportional to the number of candidates, not the index size.
func (*RerankService) Run ¶
func (s *RerankService) Run(_ context.Context, candidates []types.Hit, intent string) ([]types.Hit, bm25.Stats)
Run reranks candidates by BM25 + RRF fusion against the given intent. Returns reordered hits and statistics.
func (*RerankService) RunContext ¶
func (s *RerankService) RunContext(ctx context.Context, sc *SearchContext) error
RunContext reranks sc.RawHits if EnableBM25Rerank is set.
type Response ¶
type Response struct {
Hits []Hit `json:"hits"`
// Examples holds test-file hits separated out from Hits when
// Options.ExamplesK > 0. Nil/empty otherwise. The ranking inside
// Examples follows the same score order as Hits — top of Examples
// is the most-similar test result.
Examples []Hit `json:"examples,omitempty"`
Warnings []string `json:"warnings,omitempty"`
Metadata ResponseMetadata `json:"metadata"`
}
Response is the full search response — hits plus diagnostics so MCP callers can report freshness/budget without an extra round trip.
type ResponseMetadata ¶
type ResponseMetadata struct {
TokensUsed int `json:"tokens_used"`
IndexedHeadCKV string `json:"indexed_head_ckv"`
Fresh bool `json:"fresh"`
// TraceID echoes Options.TraceID (or the engine-generated fallback)
// so callers can correlate this response with their footprint log
// entries. Always set; non-empty.
TraceID string `json:"trace_id,omitempty"`
// DryRun mirrors Options.DryRun so callers reading only the
// response can tell whether Hits actually came from the index.
DryRun bool `json:"dry_run,omitempty"`
}
ResponseMetadata holds diagnostic metadata so the MCP layer can pass this through to LLM callers verbatim.
type SearchContext ¶
type SearchContext struct {
// Input (set by caller before pipeline starts)
Intent string
EmbedIntent string // after alias expansion; same as Intent when no aliases
Options Options
TraceID string
// After EmbedService
QueryVec []float32
// After StoreSearch
RawHits []types.Hit
// After Rerank + Threshold + Citation
FilteredHits []types.Hit
// After TestSplit
PrimaryHits []types.Hit
ExampleHits []types.Hit
// After DensityAdjust
FinalHits []Hit
FinalExamples []Hit
TokensUsed int
// Accumulated warnings
Warnings []string
// Metadata
DroppedCitations int
StaleCitations int
}
SearchContext carries the evolving state through the search pipeline. Each service reads its input fields, does its work, and writes its output fields. The Facade (Engine.Search) assembles the final Response from this context.
Individual services can also be called directly by passing a SearchContext with only the relevant fields populated.
type StoreSearchService ¶
type StoreSearchService struct {
// contains filtered or unexported fields
}
StoreSearchService runs approximate nearest neighbor search against the vector store. Over-fetches by overfetchFactor to give downstream reranking and filtering enough candidates.
func (*StoreSearchService) Run ¶
func (s *StoreSearchService) Run(ctx context.Context, queryVec []float32, k int, filter types.Filter) ([]types.Hit, error)
Run performs vector search and returns raw candidate hits.
func (*StoreSearchService) RunContext ¶
func (s *StoreSearchService) RunContext(ctx context.Context, sc *SearchContext) error
RunContext populates sc.RawHits from sc.QueryVec.
type ThresholdService ¶
type ThresholdService struct{}
ThresholdService drops hits below the minimum normalized score.
func (*ThresholdService) RunContext ¶
func (s *ThresholdService) RunContext(ctx context.Context, sc *SearchContext) error
RunContext applies threshold to sc.RawHits → sc.FilteredHits.