retrieval

package
v2.4.19 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 9, 2026 License: MIT Imports: 25 Imported by: 0

Documentation

Overview

Package retrieval implements hybrid, multi-signal, and adaptive retrieval pipelines for Cortex.

Package retrieval implements hybrid, multi-signal, and adaptive retrieval pipelines for Cortex.

Package retrieval implements hybrid, multi-signal, and adaptive retrieval pipelines for Cortex.

Package retrieval provides shared retrieval-pipeline orchestration that bridges the domain.VectorIndex port (W8.1, ADR-05) to the full search-result types that MCP, bench, CLI, and TUI consumers require.

The VectorIndex port returns lightweight VectorCandidate results carrying only an observation ID and a similarity score. Consumers need full observation data to format responses and to fuse vector results with FTS5 results via Reciprocal Rank Fusion. This package centralizes that post-fetch orchestration so it is NOT duplicated across consumer packages.

Dependency direction: this package imports ONLY internal/domain. It defines a narrow ObservationLookup interface (satisfied structurally by every concrete observation store) so it never reaches into a store package. Both internal/mcp and bench/locomo import this package; neither duplicates the helpers anymore.

The functions here are a pure extraction of logic that was previously duplicated verbatim in internal/mcp/tools_cortex.go and bench/locomo/runner.go. The extraction preserves byte-for-byte behavior: same RRF constant (k=60), same 1-based rank indexing, same tie-breaking (sort.Slice, NOT stable — matching the original), same soft-delete drop discipline.

Index

Constants

View Source 
const AgentCRAGMaxQueryRunes = 320
View Source 
const (
	// EnvHyDE is the feature-flag environment variable for HyDE (REQ-RET-108).
	// Default OFF: any value other than a truthy token keeps HyDE disabled.
	EnvHyDE = "CORTEX_RETRIEVAL_HYDE"
)

HyDE (Hypothetical Document Embeddings, arXiv 2212.10496) support for the adaptive retrieval pipeline (REQ-RET-108).

For a high-uncertainty query the generator asks the configured chat LLM to write a short hypothetical document that WOULD answer the query, embeds that document, and hands the resulting vector back to the composition layer as an additional hybrid-search input. Engagement is strictly gated:

  • Feature flag OFF by default. EnvHyDE must be explicitly enabled (HyDEFromEnv, same injectable-getenv convention as embedding.LeaseBatchFromEnv). When disabled, NO generation call is ever made.
  • Tier-gated (HyDEAllowedForTier in adaptive.go): only TierSemanticHybrid and TierMultiHopGraph qualify. TierDirectFactual and TierArchitecturalGlobal never trigger generation.
  • Cached: generated hypothetical documents live in a ScopedCache[string] keyed by embedding model + normalized query (QueryEmbeddingCacheKey) with a TTL, so repeated queries within the TTL reuse the document without a new LLM call.
  • Provider errors are NEVER cached: a failed generation or embedding is a miss and retrieval degrades to the normal (non-HyDE) path.

The composition layer (search entry points) is responsible for obtaining the two seams: generate (configured chat LLM) and embed (embedding service). Both seams must be non-nil for HyDE to engage; anything else degrades to false with zero side effects.

View Source 
const PostFilterPoolMultiplier = 3

PostFilterPoolMultiplier is the factor by which the retrieval pool is expanded when the adapter declares PostFilter or none. A multiplier of 3 gives in-engine filtering enough headroom to recover candidates the adapter's post-filter removed while keeping the pool bounded. This is a heuristic; the engine truncates to the requested limit after in-engine filtering.

Variables

View Source 
var ErrInvalidAgentCRAGScore = errors.New("agent CRAG score must be finite and normalized")
View Source 
var ErrInvalidAgentScore = errors.New("invalid agent retrieval score")

ErrInvalidAgentScore marks a score that cannot safely participate in retrieval ranking or confidence evaluation.

Functions

func ApplyRecognitionFilter added in v2.4.0 

func ApplyRecognitionFilter(results []*domain.SearchResult, cfg RecognitionConfig) []*domain.SearchResult

ApplyRecognitionFilter prunes post-PPR candidates below the recognition floors, then refills from the strongest pruned candidates so the requested TopK is always filled (and never exceeded), preserving descending-score order with ties broken by descending ID to mirror the RRF contract.

func ComputeMaxSimScore added in v2.3.0 

func ComputeMaxSimScore(queryTokens, docTokens []string) float64

ComputeMaxSimScore computes the ColBERT-style Late-Interaction MaxSim score between query and document.

MaxSim(Q, D) = sum_{q in Q} w(q) * max_{d in D} sim(q, d) / sum_{q in Q} w(q)

func ComputeWeightedMaxSimScore added in v2.3.8 

func ComputeWeightedMaxSimScore(queryTokens, docTokens []string) float64

ComputeWeightedMaxSimScore computes the specificity-weighted ColBERT MaxSim score.

func ExpandQuerySynonyms added in v2.3.0 

func ExpandQuerySynonyms(query string) string

ExpandQuerySynonyms returns an OR-expanded query string containing relevant synonyms.

func FuseResults 

func FuseResults(ftsResults []*domain.SearchResult, vecResults []*domain.VectorSearchResult, limit int) []*domain.SearchResult

FuseResults combines FTS5 full-text search results with vector similarity search results using Reciprocal Rank Fusion (k=60).

Preserves byte-for-byte behavior for all callers by delegating to FuseResultsWithOptions.

func FuseResultsWithNotes added in v2.4.0 

func FuseResultsWithNotes(ftsResults []*domain.SearchResult, vecResults []*domain.VectorSearchResult, noteRanked []*domain.SearchResult, opts FuseOptions) []*domain.SearchResult

FuseResultsWithNotes fuses FTS5, vector, and Chain-of-Note ranked lists (REQ-LME-001). The note-ranked list is a third accumulate input: its contribution is position-only — a note's Relevance score NEVER becomes an RRF rank input (score-as-rank pin, REQ-RET-002). Temporal decay stays a final multiplicative pass over the accumulated scores.

func FuseResultsWithOptions added in v2.3.8 

func FuseResultsWithOptions(ftsResults []*domain.SearchResult, vecResults []*domain.VectorSearchResult, opts FuseOptions) []*domain.SearchResult

FuseResultsWithOptions combines FTS5 full-text search results with vector similarity search results using Reciprocal Rank Fusion (k=60) with support for configurable exponential temporal decay (Time-Decayed RRF) and lexical/vector weighting.

func HyDEAllowedForTier added in v2.4.15 

func HyDEAllowedForTier(tier QueryTier) bool

HyDEAllowedForTier reports whether the retrieval tier may engage HyDE hypothetical-document generation (REQ-RET-108). Only high-uncertainty conceptual tiers qualify: TierSemanticHybrid and TierMultiHopGraph. TierDirectFactual (cheap lookups) and TierArchitecturalGlobal (community summaries) NEVER trigger a generation call, regardless of the feature flag.

func HyDEFromEnv added in v2.4.15 

func HyDEFromEnv(getenv func(string) string) bool

HyDEFromEnv reports whether EnvHyDE explicitly enables HyDE. Unset, empty, or unrecognized values are OFF (fail-closed default). Injected getenv keeps tests free of environment mutation.

func NormalizeQuery added in v2.3.0 

func NormalizeQuery(query string) string

NormalizeQuery cleans and corrects common typos in the search query.

func QueryEmbeddingCacheKey added in v2.4.14 

func QueryEmbeddingCacheKey(model, query string) string

QueryEmbeddingCacheKey builds the cache key for a query embedding: the embedding model id + normalized query text. Normalization collapses all whitespace runs to single spaces and case-folds, so queries differing only by whitespace/case share one entry. The \x00 separator keeps model/query boundaries unambiguous.

func ReRankWithLateInteraction added in v2.3.0 

func ReRankWithLateInteraction(query string, results []*domain.SearchResult) []*domain.SearchResult

ReRankWithLateInteraction re-ranks search results using ColBERT-inspired Late-Interaction MaxSim.

func RefineAgentCRAGQuery added in v2.3.0 

func RefineAgentCRAGQuery(query string) string

func RerankEndpoint added in v2.4.18 

func RerankEndpoint(baseURL string) string

post performs one batch request with 429 backoff (1s<<attempt, at most 3 retries) and paces every attempt against the provider rate budget (the shared bucket when one is composed, the private 60-rpm default otherwise). rerankEndpoint builds the provider URL (issue #123): operators configure the base URL in the same version-inclusive preset shape as the embedding provider (e.g. "https://host/v1", which the embedding client turns into /v1/embeddings). A base whose path already ends in a version segment gets only "/rerank" appended — appending "/v1/rerank" produced /v1/v1/rerank and a deterministic 404 that made the gate fail open on every search. A bare host keeps the documented "/v1/rerank" default. RerankEndpoint exposes the provider URL construction rule (issue #123) for diagnostics that probe the same contract (doctor --deep, conformance suites). Keeping one implementation prevents endpoint drift between the client and its probes.

func RevalidateCandidates 

func RevalidateCandidates(ctx context.Context, obs ObservationLookup, candidates []domain.VectorCandidate) []*domain.VectorSearchResult

RevalidateCandidates converts lightweight VectorCandidate results (ID + score from a domain.VectorIndex) into full VectorSearchResult entries by looking up the observation data via the provided ObservationLookup.

Candidates whose observation cannot be loaded (soft-deleted, missing, store error) are DROPPED — the same revalidation discipline the store-layer pipeline applies to fused candidates. A nil observation from the store is treated the same as an error: the candidate is dropped.

Batch fast path (VEC-01): when obs also implements BatchObservationLookup, the unique candidate IDs are hydrated with ONE GetByIDs call and the results are rebuilt by iterating the original candidate sequence. If the batch call fails, the error is swallowed and the unchanged per-ID loop runs instead — outputs are byte-equivalent either way.

The output preserves the INPUT ORDER of candidates (NOT re-sorted by score). Callers that need score-sorted output should sort the returned slice or rely on FuseResults, which re-sorts via RRF.

func SearchVectors 

func SearchVectors(ctx context.Context, idx domain.VectorIndex, q domain.VectorQuery, obs ObservationLookup) ([]*domain.VectorSearchResult, error)

SearchVectors executes a vector similarity search with capability-driven strategy selection. It reads idx.Capabilities, selects the appropriate filter strategy, retrieves candidates, revalidates them against the live observation store, applies in-engine filter safety-net when needed, and truncates to the requested limit.

Returns full VectorSearchResult entries (observation + similarity score). Soft-deleted, missing, or filter-mismatched candidates are dropped.

func TokenSimilarity added in v2.3.0 

func TokenSimilarity(a, b string) float64

TokenSimilarity calculates token-level string overlap similarity (Jaro-Winkler / trigram-like).

func TokenSpecificityWeight added in v2.3.8 

func TokenSpecificityWeight(token string) float64

TokenSpecificityWeight computes the information-theoretic importance weight of a token. Common stop words receive reduced weight, while technical identifiers, tokens containing underscores or digits, and longer terms receive elevated weights.

func TokenizeLateInteraction added in v2.3.0 

func TokenizeLateInteraction(text string) []string

TokenizeLateInteraction cleans and splits text into normalized word and subword tokens.

Types

type AdaptiveSearchOptions added in v2.3.0 

type AdaptiveSearchOptions struct {
	Mode string // "auto", "direct", "semantic", "multi_hop"
	// FusionScores are the RRF fusion scores of a prior retrieval stage. When
	// present and well-formed they drive SkewRoute-style tier routing in auto
	// mode; otherwise the heuristic classifier runs unchanged.
	FusionScores []float64
	// QueryVector is the query embedding for the multi-hop tier's dense leg.
	// When empty the tier seeds propagation from lexical evidence only and
	// omits the dense fusion input instead of failing.
	QueryVector []float32
	Project     string
	Scope       string
	Types       []string
	Limit       int
	GraphNodes  []graph.GraphAnalyticsNode
	GraphEdges  []graph.GraphAnalyticsEdge
	CRAGConfig  *CRAGConfig
}

AdaptiveSearchOptions controls the adaptive retrieval engine.

type AdaptiveSearchResult added in v2.3.0 

type AdaptiveSearchResult struct {
	Tier            QueryTier              `json:"tier"`
	Confidence      ConfidenceGrade        `json:"confidence"`
	ConfidenceScore float64                `json:"confidence_score"`
	NeedsRefinement bool                   `json:"needs_refinement"`
	Results         []*domain.SearchResult `json:"results"`
}

AdaptiveSearchResult represents the enriched search output with RAG metadata.

func ExecuteAdaptiveSearch added in v2.3.0 

func ExecuteAdaptiveSearch(
	ctx context.Context,
	query string,
	opts AdaptiveSearchOptions,
	lexicalSearch func(ctx context.Context, q domain.SearchOptions) ([]*domain.SearchResult, error),
	vectorSearch func(ctx context.Context, q domain.VectorQuery) ([]*domain.VectorSearchResult, error),
) (*AdaptiveSearchResult, error)

ExecuteAdaptiveSearch runs the adaptive RAG pipeline, dynamically selecting between direct lexical search, semantic hybrid vectors, and HippoRAG graph propagation.

type AgentScore added in v2.3.0 

type AgentScore struct {
	Signal       AgentScoreSignal
	Raw          *float64
	Normalized   float64
	SourceKind   string
	PublicHandle string
}

AgentScore carries one native retrieval signal and its stable public identity. Raw is a pointer so a structurally missing score cannot be confused with the valid value zero.

func NormalizeAgentScores added in v2.3.0 

func NormalizeAgentScores(scores []AgentScore) ([]AgentScore, error)

NormalizeAgentScores validates and normalizes scores, then orders them by descending normalized relevance. Bounded dense, MaxSim, PPR and summary inputs are clamped to one. Non-negative unbounded lexical inputs use x/(1+x) (expressed in an overflow-safe form). Missing, negative and non-finite values are rejected instead of receiving a confidence-bearing default.

Equal scores are ordered by source kind and public handle. Stable sorting preserves input order only when both public tie breakers are identical.

type AgentScoreSignal added in v2.3.0 

type AgentScoreSignal string

AgentScoreSignal identifies the native scale used by one retrieval signal. 

const (
	AgentScoreLexical AgentScoreSignal = "lexical"
	AgentScoreDense   AgentScoreSignal = "dense"
	AgentScoreMaxSim  AgentScoreSignal = "maxsim"
	AgentScorePPR     AgentScoreSignal = "ppr"
	AgentScoreSummary AgentScoreSignal = "summary"
)

type BatchObservationLookup 

type BatchObservationLookup interface {
	ObservationLookup
	GetByIDs(ctx context.Context, ids []int64) (map[int64]*domain.Observation, error)
}

BatchObservationLookup is the OPTIONAL batch-capable superset of ObservationLookup (VEC-01). RevalidateCandidates detects it via type assertion and, when hydration succeeds, replaces the per-candidate N+1 GetByID loop with a single GetByIDs call over the unique candidate IDs.

It is deliberately retrieval-local: it is NOT added to domain.ObservationRepository (which has multiple implementors), so stores opt in simply by exposing the method — *sqlite.Store does. A lookup that does not implement it keeps the exact legacy per-ID behavior.

GetByIDs contract:

  • Empty/nil ids MUST issue no SQL and return an empty map.
  • Live rows MUST be keyed by observation ID.
  • Soft-deleted and missing IDs MUST be absent from the map (or mapped to nil), which the engine treats as a drop — identical legacy semantics.
  • Rows for IDs that were not requested MAY be present and are ignored.

type CRAGConfig added in v2.3.0 

type CRAGConfig struct {
	HighThreshold float64 // typically 0.65
	LowThreshold  float64 // typically 0.30
	MinScoreFloor float64 // noise floor, results below this are stripped
	DynamicMode   bool    // enables dynamic score gap & relative elbow calibration (for RRF/dense distributions)
}

CRAGConfig defines thresholds for Corrective RAG evaluation and filtering.

func DefaultCRAGConfig added in v2.3.0 

func DefaultCRAGConfig() CRAGConfig

DefaultCRAGConfig returns standard CRAG evaluation parameters.

func DynamicCRAGConfig added in v2.3.8 

func DynamicCRAGConfig() CRAGConfig

DynamicCRAGConfig returns CRAG configuration optimized for dynamic/RRF score distributions.

type CRAGEvaluation added in v2.3.0 

type CRAGEvaluation struct {
	Grade           ConfidenceGrade        `json:"grade"`
	Confidence      float64                `json:"confidence"`
	NeedsRefinement bool                   `json:"needs_refinement"`
	FilteredResults []*domain.SearchResult `json:"filtered_results"`
}

CRAGEvaluation encapsulates the confidence evaluation of retrieved results.

func EvaluateCRAG added in v2.3.0 

func EvaluateCRAG(results []*domain.SearchResult, cfg CRAGConfig) CRAGEvaluation

EvaluateCRAG evaluates retrieved search results against CRAG confidence thresholds, filtering out noisy or irrelevant low-scoring candidates to protect downstream agents from hallucinations.

type CacheEntry added in v2.3.0 

type CacheEntry[T any] struct {
	// contains filtered or unexported fields
}

CacheEntry holds a cached value along with its expiration timestamp.

type ConfidenceGrade added in v2.3.0 

type ConfidenceGrade string

ConfidenceGrade represents the categorical confidence of retrieved context. 

const (
	ConfidenceGradeHigh   ConfidenceGrade = "high"
	ConfidenceGradeMedium ConfidenceGrade = "medium"
	ConfidenceGradeLow    ConfidenceGrade = "low"
)

func EvaluateAgentCRAG added in v2.3.0 

func EvaluateAgentCRAG(scores []float64, cfg CRAGConfig) (ConfidenceGrade, error)

type DefaultLateInteractionReranker added in v2.4.0 

type DefaultLateInteractionReranker struct{}

DefaultLateInteractionReranker is the zero-network local default. It delegates to ReRankWithLateInteraction so the revived 60/40 rank/MaxSim blend keeps exactly one source of ordering semantics.

func (DefaultLateInteractionReranker) Rerank added in v2.4.0 

func (DefaultLateInteractionReranker) Rerank(query string, docs []*domain.SearchResult) ([]*domain.SearchResult, error)

Rerank reorders docs with the late-interaction blend and never fails.

type FuseOptions added in v2.3.8 

type FuseOptions struct {
	// Limit is the maximum number of fused search results to return.
	Limit int
	// DecayHalfLifeDays specifies the half-life in days for exponential recency decay.
	// When <= 0, no temporal decay is applied (standard RRF).
	DecayHalfLifeDays float64
	// ReferenceTime is the anchor time for decay calculation. If zero, time.Now() is used.
	ReferenceTime time.Time
	// LexicalWeight specifies the multiplier for FTS5 lexical score (defaults to 1.0 if <= 0).
	LexicalWeight float64
	// VectorWeight specifies the multiplier for dense vector similarity score (defaults to 1.0 if <= 0).
	VectorWeight float64
	// NoteWeight specifies the multiplier for the Chain-of-Note ranked list
	// (defaults to 1.0 if <= 0). Note relevance never becomes an RRF input —
	// only the note-ranked list POSITION contributes (REQ-LME-001).
	NoteWeight float64
}

FuseOptions configures Reciprocal Rank Fusion and optional temporal decay.

type FusionRouteFeatures added in v2.4.0 

type FusionRouteFeatures struct {
	Skewness   float64 `json:"skewness"`
	Top1Margin float64 `json:"top1_margin"`
	Available  bool    `json:"available"`
}

FusionRouteFeatures holds training-free routing signals derived from an RRF fusion score distribution (SkewRoute style, REQ-ROUTE-001).

func ComputeFusionFeatures added in v2.4.0 

func ComputeFusionFeatures(scores []float64) FusionRouteFeatures

ComputeFusionFeatures derives training-free SkewRoute routing signals from an RRF fusion score distribution: Pearson skewness of the score distribution and the relative top-1 margin (s1 - s2) / s1. Scores are copied before sorting so the caller's slice is never reordered. Non-finite scores or fewer than two candidates yield unavailable features, which callers treat as "fall back to the heuristic".

type HyDEGenerator added in v2.4.15 

type HyDEGenerator struct {
	// contains filtered or unexported fields
}

HyDEGenerator produces and caches hypothetical document embeddings. Construct with NewHyDEGenerator; a nil generator is safe to call.

func NewHyDEGenerator added in v2.4.15 

func NewHyDEGenerator(enabled bool, model string, capacity int, ttl time.Duration, generate func(ctx context.Context, query string) (string, error), embed func(ctx context.Context, document string) ([]float32, error)) *HyDEGenerator

NewHyDEGenerator builds a HyDE generator. enabled comes from HyDEFromEnv (or composition config); generate is the configured chat LLM seam and embed the embedding seam. capacity/ttl <= 0 fall back to defaults. The returned generator is safe for concurrent use; if enabled is false or either seam is nil, VectorForQuery always returns false without side effects.

func (*HyDEGenerator) Calls added in v2.4.15 

func (g *HyDEGenerator) Calls() int64

Calls returns the number of chat-LLM generation attempts observed. A count of 0 proves no generation call was made (disabled or tier-gated paths).

func (*HyDEGenerator) Enabled added in v2.4.15 

func (g *HyDEGenerator) Enabled() bool

Enabled reports whether the generator will engage at all.

func (*HyDEGenerator) Hits added in v2.4.15 

func (g *HyDEGenerator) Hits() int64

Hits and Misses expose document-cache observability. All accessors are nil-receiver safe: a nil generator is a fully degraded, zero-call state.

func (*HyDEGenerator) Misses added in v2.4.15 

func (g *HyDEGenerator) Misses() int64

func (*HyDEGenerator) VectorForQuery added in v2.4.15 

func (g *HyDEGenerator) VectorForQuery(ctx context.Context, tier QueryTier, query string) ([]float32, bool)

VectorForQuery returns the embedding of the generated hypothetical document for the query, or false when HyDE does not engage (disabled, tier-gated, or provider failure). The tier gate runs FIRST so cheap tiers never incur even a cache lookup. Provider errors are not cached: the next call retries.

type ObservationLookup 

type ObservationLookup interface {
	GetByID(ctx context.Context, id int64) (*domain.Observation, error)
}

ObservationLookup is the observation-store subset needed for candidate revalidation. Every concrete observation store (*sqlite.Store, test fakes, any domain.ObservationRepository) satisfies this structurally. Defining the narrow interface here keeps this package free of any store import while remaining compatible with every backend.

type PassageNote added in v2.4.0 

type PassageNote struct {
	ObservationID int64
	Note          string
	Relevance     float64
	AnswerBearing bool
}

PassageNote is one Chain-of-Note reading note for a candidate passage.

func ChainOfNoteStage added in v2.4.0 

func ChainOfNoteStage(query string, candidates []*domain.SearchResult) ([]PassageNote, []*domain.SearchResult)

ChainOfNoteStage reads every candidate passage (Title + TopicKey + Content), scores each sentence against the query with the late-interaction MaxSim scorer, and returns the per-passage notes plus the candidates re-ordered by note relevance (Relevance DESC, ID ASC, stable). An empty or untokenizable query returns (nil, candidates) untouched — no notes, no reordering.

type QueryEmbeddingCache added in v2.4.14 

type QueryEmbeddingCache struct {
	// contains filtered or unexported fields
}

QueryEmbeddingCache is the ONE shared mechanism for caching query embeddings across search surfaces (REQ-RET-101). It wraps ScopedCache[[]float32] with the standard query-embedding keying: embedding model id + normalized query text.

A query embedding is a pure function of the query text and the embedding model that produced it. It NEVER depends on the corpus, tenant, workspace, project, or any search filter: those only decide what is searched *with* the vector, not the vector itself. Caching a query embedding and reusing it across searches (including searches scoped to different projects) is therefore semantically safe.

Contract:

  • Keyed by model id + normalized query text (QueryEmbeddingCacheKey): lowercased, whitespace-collapsed via strings.Fields join — matching the PR #128 hybrid-search normalization — so "Find Symbols" and " find symbols " share one entry.
  • Provider errors are NEVER cached: a failed embed is a miss and the provider is invoked again on retry. Empty vectors are likewise not cached.
  • TTL + LRU eviction are inherited from ScopedCache, so memory stays bounded under arbitrary query cardinality.
  • Hits and misses are exported through atomic counters for observability without changing any response shape.
  • Concurrency-safe via the underlying ScopedCache lock.

Reindex/document embedding paths MUST NOT use this cache (REQ-RET-101): document embeddings depend on mutable observation content, not just the text passed to the embedder.

func NewQueryEmbeddingCache added in v2.4.14 

func NewQueryEmbeddingCache(capacity int, ttl time.Duration) *QueryEmbeddingCache

NewQueryEmbeddingCache creates a QueryEmbeddingCache with the given capacity and TTL. Non-positive values fall back to the ScopedCache defaults (capacity 256, TTL 5m). A nil cache is safe to use: Embed degrades to a direct provider call.

func (*QueryEmbeddingCache) Embed added in v2.4.14 

func (c *QueryEmbeddingCache) Embed(ctx context.Context, model, query string, embed func(ctx context.Context, query string) ([]float32, error)) ([]float32, error)

Embed returns the cached vector for (model, query) or performs one provider round-trip through embed and caches the result. Provider errors and empty vectors are never cached. The embed function receives the ORIGINAL (not normalized) query text so providers see exactly what the caller intended.

func (*QueryEmbeddingCache) Hits added in v2.4.14 

func (c *QueryEmbeddingCache) Hits() int64

Hits returns the number of cache hits observed since construction.

func (*QueryEmbeddingCache) Len added in v2.4.14 

func (c *QueryEmbeddingCache) Len() int

Len returns the number of entries currently held in the cache.

func (*QueryEmbeddingCache) Misses added in v2.4.14 

func (c *QueryEmbeddingCache) Misses() int64

Misses returns the number of cache misses observed since construction.

type QueryTier added in v2.3.0 

type QueryTier string

QueryTier defines the complexity routing level in Adaptive-RAG. 

const (
	// TierDirectFactual represents direct lookups (symbols, IDs, exact keywords) ~1-5ms.
	TierDirectFactual QueryTier = "direct_factual"
	// TierSemanticHybrid represents conceptual queries combining lexical + dense vectors.
	TierSemanticHybrid QueryTier = "semantic_hybrid"
	// TierMultiHopGraph represents complex relational questions resolved via HippoRAG PPR.
	TierMultiHopGraph QueryTier = "multi_hop_graph"
	// TierArchitecturalGlobal represents macro architectural questions resolved via LightRAG Community Summaries.
	TierArchitecturalGlobal QueryTier = "architectural_global"
)

func ClassifyQueryComplexity added in v2.3.0 

func ClassifyQueryComplexity(query string) QueryTier

ClassifyQueryComplexity routes a query to the optimal retrieval tier in < 0.1ms.

func ClassifyQueryWithFusion added in v2.4.0 

func ClassifyQueryWithFusion(query string, features FusionRouteFeatures) QueryTier

ClassifyQueryWithFusion routes a query using fusion features when available, otherwise it falls back to the text heuristic unchanged.

type RecognitionConfig added in v2.4.0 

type RecognitionConfig struct {
	TopK          int     // requested final candidate count; <= 0 disables prune-only
	RelativeFloor float64 // keep candidates scoring at least RelativeFloor * top score
	MinScoreFloor float64 // absolute noise floor on Rank
}

RecognitionConfig configures the post-PPR recognition-memory filter that prunes weak candidates before final fusion while still filling the requested top-k.

func DefaultRecognitionConfig added in v2.4.0 

func DefaultRecognitionConfig() RecognitionConfig

DefaultRecognitionConfig returns standard recognition-memory parameters.

type RerankConfig added in v2.4.0 

type RerankConfig struct {
	Provider string
	Model    string
	BaseURL  string
	// Budget is the shared provider rate budget drawn by every subsystem
	// dialing the same provider base URL (rerank, embedding, agent chat).
	// When nil the reranker builds its own private budget at the provider's
	// advertised rpm, which is the historical per-subsystem behavior.
	Budget *ratelimit.Limiter
}

RerankConfig is the resolved rerank preset handed to the factories. It has no API-key field by design: the key resolves only from CORTEX_RERANK_API_KEY at construction (REQ-CFG-001), so it can never be serialized into a config file or passed in from an untrusted source.

type Reranker added in v2.4.0 

type Reranker interface {
	Rerank(query string, docs []*domain.SearchResult) ([]*domain.SearchResult, error)
}

Reranker orders one fused candidate list for a query before the final limit is applied (REQ-RNK-001). Implementations preserve the input set exactly — the same *domain.SearchResult pointers with the same multiplicity — and order by descending score with ties broken by descending observation ID (the RRF convention in fuseInputs). A non-nil error means the returned slice must be discarded wholesale: partially reranked output is never valid.

func NewReranker added in v2.4.0 

func NewReranker(cfg RerankConfig) (Reranker, error)

NewReranker builds the local-mode reranker. The disabled default returns (nil, nil) so the local composition constructs zero machinery, and the remote preset is refused fail-closed: only NewSecureReranker may ever build the HTTP reranker.

func NewSecureReranker added in v2.4.0 

func NewSecureReranker(cfg RerankConfig, policy embedding.OutboundPolicy) (Reranker, error)

NewSecureReranker is the sole construction path for the HTTP reranker: the base URL must be approved on the shared embedding OutboundPolicy, and the API key resolves only from CORTEX_RERANK_API_KEY (missing => construction error). Local composition never reaches this dial.

type ScopedCache added in v2.3.0 

type ScopedCache[T any] struct {
	// contains filtered or unexported fields
}

ScopedCache is a generic, thread-safe LRU cache with TTL expiration.

func NewScopedCache added in v2.3.0 

func NewScopedCache[T any](capacity int, ttl time.Duration) *ScopedCache[T]

NewScopedCache creates a new ScopedCache with a maximum capacity and TTL.

func (*ScopedCache[T]) Clear added in v2.3.0 

func (c *ScopedCache[T]) Clear()

Clear empties the cache completely.

func (*ScopedCache[T]) Get added in v2.3.0 

func (c *ScopedCache[T]) Get(key string) (T, bool)

Get retrieves a value from the cache if present and not expired.

func (*ScopedCache[T]) Len added in v2.3.0 

func (c *ScopedCache[T]) Len() int

Len returns the current number of items in the cache.

func (*ScopedCache[T]) PurgePrefix added in v2.3.0 

func (c *ScopedCache[T]) PurgePrefix(prefix string)

PurgePrefix removes all entries whose key starts with the given prefix.

func (*ScopedCache[T]) Set added in v2.3.0 

func (c *ScopedCache[T]) Set(key string, value T)

Set adds or updates a value in the cache.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.