Documentation
¶
Overview ¶
Package services provides retrieval services for the storage system.
Package services provides simplified retrieval services for the storage system. This service focuses on pure vector similarity search without complex weight calculations.
Index ¶
- func SetAllowedSynonymDir(dir string)
- type QueryPriorityConfig
- type ResultDebugInfo
- type RetrievalPlan
- type RetrievalService
- func (s *RetrievalService) GenerateDebugInfo(result *SearchResult, plan *RetrievalPlan) *ResultDebugInfo
- func (s *RetrievalService) Search(ctx context.Context, req *SearchRequest) ([]*SearchResult, error)
- func (s *RetrievalService) SetExperienceServices(distillationService *experience.DistillationService, ...)
- type RetrievalTrace
- type SearchRequest
- type SearchResult
- type SimpleRetrievalConfig
- type SimpleRetrievalService
- type SimpleSearchResult
- type WeightedQuery
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func SetAllowedSynonymDir ¶
func SetAllowedSynonymDir(dir string)
SetAllowedSynonymDir sets the allowed directory for synonym config files. This is a security measure to prevent path traversal attacks.
Types ¶
type QueryPriorityConfig ¶
type QueryPriorityConfig struct {
OriginalWeight float64 `json:"original_weight"` // Original query weight (default 1.0)
RuleRewriteWeight float64 `json:"rule_rewrite_weight"` // Rule-based rewrite weight (default 0.7)
LLMRewriteWeight float64 `json:"llm_rewrite_weight"` // LLM-based rewrite weight (default 0.5)
MaxQueries int `json:"max_queries"` // Maximum number of queries (default 3)
}
QueryPriorityConfig defines priority weights for different query types. This controls how much influence rewrites have on retrieval results.
func DefaultQueryPriorityConfig ¶
func DefaultQueryPriorityConfig() *QueryPriorityConfig
DefaultQueryPriorityConfig returns the default query priority configuration.
type ResultDebugInfo ¶
type ResultDebugInfo struct {
ID string `json:"id"` // Result ID
Score float64 `json:"score"` // Final score
Query string `json:"query"` // Query that matched this result
QueryWeight float64 `json:"query_weight"` // Weight of the query
Source string `json:"source"` // Result source (knowledge/experience/tool)
SubSource string `json:"sub_source"` // Sub-source (vector/keyword)
SourceWeight float64 `json:"source_weight"` // Weight from source
SubWeight float64 `json:"sub_weight"` // Weight from sub-source
Signals map[string]interface{} `json:"signals"` // Applied signals (success, reuse_count, etc.)
Breakdown map[string]float64 `json:"breakdown"` // Score breakdown for analysis
}
ResultDebugInfo contains detailed debugging information for a search result. This helps answer "why this result is ranked first?" and supports observability.
type RetrievalPlan ¶
type RetrievalPlan struct {
SearchKnowledge bool `json:"search_knowledge"` // Search in knowledge base
SearchExperience bool `json:"search_experience"` // Search in experiences
SearchTools bool `json:"search_tools"` // Search in tools
SearchTaskResults bool `json:"search_task_results"` // Search in task results
KnowledgeWeight float64 `json:"knowledge_weight"` // Weight for knowledge results (default 0.4)
ExperienceWeight float64 `json:"experience_weight"` // Weight for experience results (default 0.3)
ToolsWeight float64 `json:"tools_weight"` // Weight for tool results (default 0.2)
TaskResultsWeight float64 `json:"task_results_weight"` // Weight for task result results (default 0.1)
EnableQueryRewrite bool `json:"enable_query_rewrite"` // Enable query rewriting
EnableKeywordSearch bool `json:"enable_keyword_search"` // Enable keyword/BM25 search
EnableTimeDecay bool `json:"enable_time_decay"` // Enable time-based scoring decay
TopK int `json:"top_k"` // Maximum results per source
// Experience-specific configuration
ExperienceRankingEnabled bool `json:"experience_ranking_enabled"` // Enable experience ranking
ExperienceConflictResolve bool `json:"experience_conflict_resolve"` // Enable conflict resolution
ExperienceTopK int `json:"experience_top_k"` // Experience recall count (default 20)
}
RetrievalPlan defines the retrieval strategy for multi-source search.
func DefaultRetrievalPlan ¶
func DefaultRetrievalPlan() *RetrievalPlan
DefaultRetrievalPlan returns the default retrieval plan.
type RetrievalService ¶
type RetrievalService struct {
// contains filtered or unexported fields
}
RetrievalService provides intelligent retrieval across multiple data sources. It implements hybrid search (vector + keyword), query rewriting, and time-based decay.
func NewRetrievalService ¶
func NewRetrievalService( pool *postgres.Pool, embeddingClient *embedding.EmbeddingClient, llmClient *llm.Client, tenantGuard *postgres.TenantGuard, retrievalGuard *postgres.RetrievalGuard, kbRepo *repositories.KnowledgeRepository, expRepo *repositories.ExperienceRepository, toolRepo *repositories.ToolRepository, ) *RetrievalService
NewRetrievalService creates a new RetrievalService instance. Args: pool - database connection pool. embeddingClient - embedding service client for vector search. llmClient - LLM client for query rewriting (optional, can be nil). tenantGuard - tenant isolation guard. retrievalGuard - rate limiting and circuit breaker for retrieval. kbRepo - knowledge repository for data access. expRepo - experience repository for experience search. toolRepo - tool repository for tool search. Returns new RetrievalService instance.
func (*RetrievalService) GenerateDebugInfo ¶
func (s *RetrievalService) GenerateDebugInfo(result *SearchResult, plan *RetrievalPlan) *ResultDebugInfo
GenerateDebugInfo generates detailed debugging information for a search result. This helps answer "why this result is ranked first?" and supports observability. Args: result - search result to generate debug info for. plan - retrieval plan with weight configuration (optional, can be nil for default weights). Returns ResultDebugInfo with scoring breakdown and signals.
func (*RetrievalService) Search ¶
func (s *RetrievalService) Search(ctx context.Context, req *SearchRequest) ([]*SearchResult, error)
Search performs intelligent retrieval across multiple data sources. This implements the core retrieval pipeline with hybrid search, query rewriting, and time decay. Args: ctx - database operation context. req - search request with query and configuration. Returns search results or error if retrieval fails.
func (*RetrievalService) SetExperienceServices ¶
func (s *RetrievalService) SetExperienceServices( distillationService *experience.DistillationService, rankingService *experience.RankingService, conflictResolver *experience.ConflictResolver, )
SetExperienceServices sets the experience-specific services. Args: distillationService - experience distillation service. rankingService - experience ranking service. conflictResolver - experience conflict resolver.
type RetrievalTrace ¶
type RetrievalTrace struct {
OriginalQuery string `json:"original_query"`
RewrittenQuery string `json:"rewritten_query"`
RewriteUsed bool `json:"rewrite_used"`
VectorResults int `json:"vector_results"`
KeywordResults int `json:"keyword_results"`
FinalResults int `json:"final_results"`
ExecutionTime time.Duration `json:"execution_time"`
VectorError error `json:"vector_error,omitempty"`
SearchBreakdown map[string]int `json:"search_breakdown,omitempty"` // Results per source
}
RetrievalTrace contains debugging information for retrieval operations.
type SearchRequest ¶
type SearchRequest struct {
Query string `json:"query"` // Search query text
TenantID string `json:"tenant_id"` // Tenant ID for isolation
TopK int `json:"top_k"` // Number of results to return
MinScore float64 `json:"min_score"` // Minimum similarity score
Plan *RetrievalPlan `json:"plan"` // Retrieval strategy
EnableTrace bool `json:"enable_trace"` // Enable trace logging
Trace *RetrievalTrace `json:"trace,omitempty"` // Trace information
}
SearchRequest represents a search request with configuration.
type SearchResult ¶
type SearchResult struct {
ID string `json:"id"`
Content string `json:"content"`
Score float64 `json:"score"`
Source string `json:"source"` // knowledge, experience, tool, task_result
SubSource string `json:"sub_source"` // vector, keyword
Type string `json:"type"` // Result type for filtering
Metadata map[string]interface{} `json:"metadata"` // Additional metadata
CreatedAt time.Time `json:"created_at"`
// Query information for traceability and scoring
Query string `json:"query"` // Query that matched this result
QueryWeight float64 `json:"query_weight"` // Weight of the query (original=1.0, rule=0.7, llm=0.5)
}
SearchResult represents a single search result.
type SimpleRetrievalConfig ¶
type SimpleRetrievalConfig struct {
TopK int `json:"top_k"` // Number of results to return
MinScore float64 `json:"min_score"` // Minimum similarity score
QueryPrefix string `json:"query_prefix"` // Prefix for query embedding (e.g., "query:")
}
SimpleRetrievalConfig configuration for simple retrieval service
type SimpleRetrievalService ¶
type SimpleRetrievalService struct {
// contains filtered or unexported fields
}
SimpleRetrievalService provides pure vector similarity search This is inspired by ChromaDB's simple and direct approach: - Direct vector similarity search (1 - cosine_distance) - No complex weight calculations - No time decay - No query rewrites - Simple and effective for single knowledge base scenarios
func NewSimpleRetrievalService ¶
func NewSimpleRetrievalService( repo *repositories.KnowledgeRepository, embeddingClient *embedding.EmbeddingClient, config *SimpleRetrievalConfig, ) *SimpleRetrievalService
NewSimpleRetrievalService creates a new simple retrieval service
func (*SimpleRetrievalService) GetConfig ¶
func (s *SimpleRetrievalService) GetConfig() *SimpleRetrievalConfig
GetConfig returns the current configuration
func (*SimpleRetrievalService) Search ¶
func (s *SimpleRetrievalService) Search(ctx context.Context, tenantID, query string) ([]*SimpleSearchResult, error)
Search performs intelligent retrieval with precision mode support Returns results with cosine similarity score (1 - distance), where: - 1.0 = perfect match - 0.0 = orthogonal (no relation) - -1.0 = opposite meaning
func (*SimpleRetrievalService) SetConfig ¶
func (s *SimpleRetrievalService) SetConfig(config *SimpleRetrievalConfig)
SetConfig updates the retrieval configuration
type SimpleSearchResult ¶
type SimpleSearchResult struct {
Content string `json:"content"` // Content text
Source string `json:"source"` // Source document/file path
Score float64 `json:"score"` // Cosine similarity score (1 - distance)
}
SimpleSearchResult simple search result with only essential fields
type WeightedQuery ¶
type WeightedQuery struct {
Query string `json:"query"` // Query text
Weight float64 `json:"weight"` // Weight for scoring (original=1.0, rule=0.7, llm=0.5)
Source string `json:"source"` // Source: original, rewrite_rule, rewrite_llm
}
WeightedQuery represents a query with associated weight for retrieval. This enables controlling the impact of query rewrites on final results.
Source Files