Documentation
¶
Overview ¶
Package classifier provides query classification for tool forcing decisions.
The classifier determines whether a user query requires tool exploration (analytical query) versus simple conversation (non-analytical query). This enables the agent to force tool usage for questions that require codebase exploration.
Index ¶
- func ComputeToolsHash(toolNames []string) string
- func DescribeToolChoice(toolChoice *llm.ToolChoice) string
- func ExtractJSON(response string) ([]byte, error)
- func HasCitation(content string) bool
- func HasHedgingLanguage(content string) (bool, string)
- func LooksLikeOfferToHelp(content string) bool
- func NeedsToolCalls(toolChoice *llm.ToolChoice) bool
- func SuggestRetryToolChoice(original *llm.ToolChoice, suggestedTool string) *llm.ToolChoice
- type ClassificationCache
- func (c *ClassificationCache) Clear()
- func (c *ClassificationCache) Delete(query, toolsHash string)
- func (c *ClassificationCache) Get(query, toolsHash string) (*ClassificationResult, bool)
- func (c *ClassificationCache) HitRate() float64
- func (c *ClassificationCache) Hits() int64
- func (c *ClassificationCache) Misses() int64
- func (c *ClassificationCache) ResetMetrics()
- func (c *ClassificationCache) Set(query, toolsHash string, result *ClassificationResult)
- func (c *ClassificationCache) Size() int
- type ClassificationResult
- type ClassifierConfig
- type ClassifierType
- type FactoryConfig
- type LLMClassifier
- func (c *LLMClassifier) CacheStats() (hitRate float64, size int)
- func (c *LLMClassifier) Classify(ctx context.Context, query string) (*ClassificationResult, error)
- func (c *LLMClassifier) IsAnalytical(ctx context.Context, query string) bool
- func (c *LLMClassifier) SuggestTool(ctx context.Context, query string, available []string) (string, bool)
- func (c *LLMClassifier) SuggestToolWithHint(ctx context.Context, query string, available []string) (*ToolSuggestion, bool)
- type ParameterValidationResult
- type QualityConfig
- type QualityValidator
- type QueryClassifier
- type RegexClassifier
- func (c *RegexClassifier) IsAnalytical(ctx context.Context, query string) bool
- func (c *RegexClassifier) SuggestTool(ctx context.Context, query string, available []string) (string, bool)
- func (c *RegexClassifier) SuggestToolWithHint(ctx context.Context, query string, available []string) (*ToolSuggestion, bool)
- type ResponseValidator
- func (v *ResponseValidator) AddProhibitedPattern(pattern string) error
- func (v *ResponseValidator) HasProhibitedPattern(text string) (bool, string)
- func (v *ResponseValidator) Validate(response *llm.Response, toolChoice *llm.ToolChoice) ValidationResult
- func (v *ResponseValidator) ValidateWithToolRequirement(response *llm.Response, requireTools bool) (bool, string)
- type RetryableValidator
- func (r *RetryableValidator) GetRetryToolChoice(attempt int, original *llm.ToolChoice, suggestedTool string) *llm.ToolChoice
- func (r *RetryableValidator) MaxRetries() int
- func (r *RetryableValidator) ShouldRetry(result ValidationResult, attempt int) bool
- func (r *RetryableValidator) Validate(response *llm.Response, toolChoice *llm.ToolChoice) ValidationResult
- type SelectionResult
- type SelectorConfig
- type ToolChoiceSelector
- type ToolSuggestion
- type ToolValidationResult
- type ValidationResult
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ComputeToolsHash ¶
ComputeToolsHash creates a stable hash of tool definitions.
Description:
Computes a SHA-256 hash of tool names, sorted alphabetically, to create a stable hash that changes when available tools change. This enables cache invalidation when the tool set changes.
Inputs:
toolNames - List of tool names.
Outputs:
string - Hex-encoded hash of the tool names.
Thread Safety: This function is safe for concurrent use.
func DescribeToolChoice ¶
func DescribeToolChoice(toolChoice *llm.ToolChoice) string
DescribeToolChoice returns a human-readable description of the tool_choice.
func ExtractJSON ¶
ExtractJSON extracts JSON from potentially wrapped LLM responses.
Description:
Attempts to extract valid JSON from LLM responses that may be wrapped
in various formats. Handles these cases in order:
1. Clean JSON: {"is_analytical": true, ...}
2. Markdown wrapped: ```json\n{"is_analytical": true}\n```
3. Generic code block: ```\n{...}\n```
4. With preamble: "Here is the classification:\n{...}"
5. Multiple JSON objects: Takes first valid one
Inputs:
response - Raw LLM response text.
Outputs:
[]byte - Extracted JSON bytes. error - If no valid JSON found.
Example:
jsonBytes, err := ExtractJSON("```json\n{\"is_analytical\":true}\n```")
if err != nil {
return err
}
var result ClassificationResult
json.Unmarshal(jsonBytes, &result)
Thread Safety: This function is safe for concurrent use.
func HasCitation ¶
HasCitation checks if content contains a valid citation. This is a convenience method for external callers.
func HasHedgingLanguage ¶
HasHedgingLanguage checks if content contains hedging language. This is a convenience method for external callers.
func LooksLikeOfferToHelp ¶
LooksLikeOfferToHelp checks if response content appears to offer help rather than actually helping. This is a quick heuristic check.
func NeedsToolCalls ¶
func NeedsToolCalls(toolChoice *llm.ToolChoice) bool
NeedsToolCalls returns true if the tool_choice requires tool calls.
func SuggestRetryToolChoice ¶
func SuggestRetryToolChoice(original *llm.ToolChoice, suggestedTool string) *llm.ToolChoice
SuggestRetryToolChoice returns a stronger tool_choice for retry.
Description:
Given a failed validation result and the original tool_choice, suggests a stronger tool_choice to use for retry.
Inputs:
original - The tool_choice that was used originally. suggestedTool - A tool to force if available.
Outputs:
*llm.ToolChoice - Stronger tool_choice for retry.
Example:
if !result.Valid && result.Retryable {
retry := SuggestRetryToolChoice(original, "find_entry_points")
// retry.Type == "any" or "tool"
}
Types ¶
type ClassificationCache ¶
type ClassificationCache struct {
// contains filtered or unexported fields
}
ClassificationCache caches classification results with LRU eviction.
Description:
Provides a thread-safe LRU cache for classification results with TTL expiration. Cache keys are computed from query + toolsHash to ensure results are invalidated when available tools change.
Thread Safety: This type is safe for concurrent use.
func NewClassificationCache ¶
func NewClassificationCache(ttl time.Duration, maxSize int) *ClassificationCache
NewClassificationCache creates a cache with TTL and max size.
Description:
Creates a new LRU cache for classification results. The cache uses TTL-based expiration and LRU eviction when at capacity.
Inputs:
ttl - How long cached results are valid. Must be > 0. maxSize - Maximum number of entries before LRU eviction. Must be > 0.
Outputs:
*ClassificationCache - Ready-to-use cache.
Example:
cache := NewClassificationCache(10*time.Minute, 1000)
cache.Set("what tests exist?", toolsHash, result)
if cached, ok := cache.Get("what tests exist?", toolsHash); ok {
// Use cached result
}
Thread Safety: The returned cache is safe for concurrent use.
func (*ClassificationCache) Clear ¶
func (c *ClassificationCache) Clear()
Clear removes all entries from the cache.
Thread Safety: This method is safe for concurrent use.
func (*ClassificationCache) Delete ¶
func (c *ClassificationCache) Delete(query, toolsHash string)
Delete removes a specific entry from the cache.
Description:
Removes the cached result for the given query and toolsHash if it exists.
Inputs:
query - The original query string. toolsHash - Hash of available tool definitions.
Thread Safety: This method is safe for concurrent use.
func (*ClassificationCache) Get ¶
func (c *ClassificationCache) Get(query, toolsHash string) (*ClassificationResult, bool)
Get retrieves a cached result if valid (not expired).
Description:
Looks up a cached classification result by query and toolsHash. Returns nil if the entry doesn't exist, has expired, or the cache is empty.
Inputs:
query - The original query string. toolsHash - Hash of available tool definitions for cache key.
Outputs:
*ClassificationResult - The cached result with Cached=true, or nil. bool - True if a valid cached result was found.
Thread Safety: This method is safe for concurrent use.
func (*ClassificationCache) HitRate ¶
func (c *ClassificationCache) HitRate() float64
HitRate returns the cache hit rate (0.0-1.0).
Description:
Calculates the ratio of cache hits to total lookups. Returns 0 if no lookups have been performed.
Outputs:
float64 - Hit rate between 0.0 and 1.0.
Thread Safety: This method is safe for concurrent use.
func (*ClassificationCache) Hits ¶
func (c *ClassificationCache) Hits() int64
Hits returns the total number of cache hits.
Thread Safety: This method is safe for concurrent use.
func (*ClassificationCache) Misses ¶
func (c *ClassificationCache) Misses() int64
Misses returns the total number of cache misses.
Thread Safety: This method is safe for concurrent use.
func (*ClassificationCache) ResetMetrics ¶
func (c *ClassificationCache) ResetMetrics()
ResetMetrics resets the hit/miss counters to zero.
Thread Safety: This method is safe for concurrent use.
func (*ClassificationCache) Set ¶
func (c *ClassificationCache) Set(query, toolsHash string, result *ClassificationResult)
Set stores a classification result, evicting LRU if at capacity.
Description:
Stores or updates a classification result in the cache. If the cache is at capacity, the least recently used entry is evicted first.
Inputs:
query - The original query string. toolsHash - Hash of available tool definitions for cache key. result - The classification result to cache. Must not be nil.
Thread Safety: This method is safe for concurrent use.
func (*ClassificationCache) Size ¶
func (c *ClassificationCache) Size() int
Size returns the current number of entries in the cache.
Thread Safety: This method is safe for concurrent use.
type ClassificationResult ¶
type ClassificationResult struct {
// IsAnalytical indicates if the query requires tool exploration.
IsAnalytical bool `json:"is_analytical"`
// Tool is the recommended first tool to call.
// Empty if IsAnalytical is false.
// VALIDATED: Must exist in available tools list.
Tool string `json:"tool,omitempty"`
// Parameters are the recommended parameters for the tool.
// Keys match the tool's parameter schema.
// VALIDATED: Types and enum values checked against ParamDef.
Parameters map[string]any `json:"parameters,omitempty"`
// SearchPatterns are specific patterns to search for.
// Used for grep/glob operations within tools.
SearchPatterns []string `json:"search_patterns,omitempty"`
// Reasoning explains the classification decision.
// Useful for debugging and observability.
Reasoning string `json:"reasoning,omitempty"`
// Confidence is the model's confidence in this classification (0.0-1.0).
// Below ConfidenceThreshold triggers regex fallback.
Confidence float64 `json:"confidence,omitempty"`
// Cached indicates this result came from cache.
Cached bool `json:"-"`
// Duration is how long classification took.
Duration time.Duration `json:"-"`
// FallbackUsed indicates regex fallback was used.
FallbackUsed bool `json:"-"`
// ValidationWarnings contains any parameter validation warnings.
ValidationWarnings []string `json:"-"`
}
ClassificationResult contains the LLM's analysis of a query.
Thread Safety: This type is immutable after creation and safe for concurrent read.
func ExtractJSONWithFallback ¶
func ExtractJSONWithFallback(response string) (*ClassificationResult, bool)
ExtractJSONWithFallback tries to extract JSON and returns a default on failure.
Description:
Convenience wrapper around ExtractJSON that returns a default ClassificationResult (non-analytical) on extraction failure.
Inputs:
response - Raw LLM response text.
Outputs:
*ClassificationResult - Extracted result or non-analytical default. bool - True if extraction succeeded, false if using fallback.
Thread Safety: This function is safe for concurrent use.
func ParseClassificationResponse ¶
func ParseClassificationResponse(response string) (*ClassificationResult, error)
ParseClassificationResponse parses an LLM response into a ClassificationResult.
Description:
Complete parsing pipeline that extracts JSON from the response and unmarshals it into a ClassificationResult. Handles all common response formats from LLMs.
Inputs:
response - Raw LLM response text.
Outputs:
*ClassificationResult - Parsed result. error - If extraction or parsing failed.
Thread Safety: This function is safe for concurrent use.
func ValidateClassificationResult ¶
func ValidateClassificationResult( result *ClassificationResult, toolDefs map[string]tools.ToolDefinition, availableTools []string, ) (*ClassificationResult, bool)
ValidateClassificationResult validates a complete classification result.
Description:
Validates the tool name and parameters in a ClassificationResult against the available tools and their schemas. Returns a new result with validated parameters and any warnings.
Inputs:
result - The classification result to validate. toolDefs - Map of tool name to ToolDefinition. availableTools - List of available tool names.
Outputs:
*ClassificationResult - Validated result with warnings. bool - True if validation passed (tool found and usable).
Thread Safety: This function is safe for concurrent use.
func (*ClassificationResult) ToToolSuggestion ¶
func (r *ClassificationResult) ToToolSuggestion() *ToolSuggestion
ToToolSuggestion converts the classification result to a ToolSuggestion.
Description:
Converts the rich ClassificationResult into the simpler ToolSuggestion format used by the existing ToolForcingPolicy interface.
Outputs:
*ToolSuggestion - The tool suggestion, or nil if not analytical.
Thread Safety: This method is safe for concurrent use.
type ClassifierConfig ¶
type ClassifierConfig struct {
// Temperature for classification (0.0 = deterministic).
// Must be >= 0.0 and <= 1.0.
Temperature float64
// MaxTokens limits classification response length.
// Must be > 0.
MaxTokens int
// Timeout for each classification attempt.
// Must be > 0.
Timeout time.Duration
// MaxRetries before fallback to regex.
// 0 = no retries, fall back immediately on first failure.
MaxRetries int
// RetryBackoff is the base duration for exponential backoff.
// Retry N waits RetryBackoff * 2^N.
RetryBackoff time.Duration
// CacheTTL is how long to cache classification results.
// 0 = no caching.
CacheTTL time.Duration
// CacheMaxSize is maximum cache entries before LRU eviction.
// Must be > 0 if CacheTTL > 0.
CacheMaxSize int
// ConfidenceThreshold below which regex fallback is used.
// Must be >= 0.0 and <= 1.0.
ConfidenceThreshold float64
// FallbackToRegex enables regex fallback on LLM errors.
FallbackToRegex bool
// MaxConcurrent limits simultaneous classification calls.
// 0 = unlimited.
MaxConcurrent int
}
ClassifierConfig configures the LLM classifier behavior.
Thread Safety: This type should not be modified after passing to NewLLMClassifier.
func DefaultClassifierConfig ¶
func DefaultClassifierConfig() ClassifierConfig
DefaultClassifierConfig returns production defaults.
Description:
Returns a ClassifierConfig with sensible defaults for production use: - Temperature 0.0 for deterministic classification - 5 second timeout with 2 retries (100ms exponential backoff) - 10 minute cache TTL with 1000 max entries - 0.7 confidence threshold for regex fallback
Outputs:
ClassifierConfig - Ready-to-use configuration.
Thread Safety: This function is safe for concurrent use.
func (ClassifierConfig) Validate ¶
func (c ClassifierConfig) Validate() error
Validate checks that config values are within valid ranges.
Description:
Validates all configuration fields and returns an error describing all invalid fields if any are out of range.
Outputs:
error - Non-nil if any field is invalid, describing all issues.
Thread Safety: This method is safe for concurrent use.
type ClassifierType ¶
type ClassifierType string
ClassifierType specifies which classifier implementation to use.
const ( // ClassifierTypeRegex uses regex pattern matching (fast, no LLM calls). ClassifierTypeRegex ClassifierType = "regex" // ClassifierTypeLLM uses LLM-based classification (more accurate, requires LLM calls). ClassifierTypeLLM ClassifierType = "llm" )
type FactoryConfig ¶
type FactoryConfig struct {
// Type specifies which classifier to create.
// Defaults to ClassifierTypeRegex if empty.
Type ClassifierType
// LLMClient is required when Type is ClassifierTypeLLM.
// Ignored for ClassifierTypeRegex.
LLMClient llm.Client
// ToolDefinitions is required when Type is ClassifierTypeLLM.
// Ignored for ClassifierTypeRegex.
ToolDefinitions []tools.ToolDefinition
// LLMConfig configures the LLM classifier.
// If nil and Type is ClassifierTypeLLM, DefaultClassifierConfig() is used.
LLMConfig *ClassifierConfig
}
FactoryConfig configures the classifier factory.
type LLMClassifier ¶
type LLMClassifier struct {
// contains filtered or unexported fields
}
LLMClassifier implements QueryClassifier using the LLM client.
Description:
Uses an LLM to classify queries with higher accuracy than regex patterns. Implements caching, request coalescing, retry logic with exponential backoff, and falls back to RegexClassifier on errors.
Thread Safety: This type is safe for concurrent use after initialization.
func NewLLMClassifier ¶
func NewLLMClassifier( client llm.Client, toolDefs []tools.ToolDefinition, config ClassifierConfig, ) (*LLMClassifier, error)
NewLLMClassifier creates a classifier using the provided LLM client.
Description:
Creates an LLMClassifier with caching, request coalescing, and regex fallback. The classifier uses the same LLM client as the main agent loop.
Inputs:
client - LLM client for classification calls. Must not be nil. toolDefs - Available tool definitions. Must not be empty. config - Classifier configuration. Will be validated.
Outputs:
*LLMClassifier - Ready-to-use classifier. error - If client is nil, toolDefs empty, or config invalid.
Example:
classifier, err := NewLLMClassifier(llmClient, toolDefs, DefaultClassifierConfig())
if err != nil {
return err
}
result, err := classifier.Classify(ctx, "What tests exist?")
Thread Safety: The returned classifier is safe for concurrent use.
func (*LLMClassifier) CacheStats ¶
func (c *LLMClassifier) CacheStats() (hitRate float64, size int)
CacheStats returns cache statistics.
Description:
Returns the current cache hit rate and size. Returns zeros if caching is disabled.
Outputs:
hitRate - Cache hit rate (0.0-1.0). size - Current number of cached entries.
Thread Safety: This method is safe for concurrent use.
func (*LLMClassifier) Classify ¶
func (c *LLMClassifier) Classify(ctx context.Context, query string) (*ClassificationResult, error)
Classify analyzes a query and returns a classification result.
Description:
Performs LLM-based classification with caching, request coalescing, retry logic, and validation. Falls back to regex on failure.
Inputs:
ctx - Context for cancellation and timeout. Must not be nil. query - The user's question. Empty queries return non-analytical result.
Outputs:
*ClassificationResult - Classification with tool suggestion. error - Only if context cancelled; other errors trigger fallback.
Thread Safety: This method is safe for concurrent use.
func (*LLMClassifier) IsAnalytical ¶
func (c *LLMClassifier) IsAnalytical(ctx context.Context, query string) bool
IsAnalytical implements QueryClassifier interface.
Description:
Classifies the query and returns whether it's analytical. This is a convenience wrapper around Classify().
Thread Safety: This method is safe for concurrent use.
func (*LLMClassifier) SuggestTool ¶
func (c *LLMClassifier) SuggestTool(ctx context.Context, query string, available []string) (string, bool)
SuggestTool implements QueryClassifier interface.
Description:
Classifies the query and returns a suggested tool if the query is analytical. Returns empty string if not analytical.
Thread Safety: This method is safe for concurrent use.
func (*LLMClassifier) SuggestToolWithHint ¶
func (c *LLMClassifier) SuggestToolWithHint(ctx context.Context, query string, available []string) (*ToolSuggestion, bool)
SuggestToolWithHint implements QueryClassifier interface.
Description:
Classifies the query and returns a ToolSuggestion with the suggested tool and search hints. Falls back to regex if classification fails.
Thread Safety: This method is safe for concurrent use.
type ParameterValidationResult ¶
type ParameterValidationResult struct {
// ValidatedParams contains parameters that passed validation.
ValidatedParams map[string]any
// Warnings contains validation warnings for each parameter.
Warnings []string
// MissingRequired lists required parameters that are missing.
MissingRequired []string
}
ParameterValidationResult contains the outcome of parameter validation.
func ValidateParameters ¶
func ValidateParameters(params map[string]any, schema map[string]tools.ParamDef) ParameterValidationResult
ValidateParameters validates suggested parameters against tool schema.
Description:
Validates each parameter against the tool's ParamDef schema, checking: - Parameter exists in schema - Type matches expected type - Enum values are in allowed set - Required parameters are present Invalid parameters are removed and warnings are generated.
Inputs:
params - The parameters suggested by the LLM. schema - The tool's parameter schema.
Outputs:
ParameterValidationResult - Contains validated params and warnings.
Example:
schema := map[string]tools.ParamDef{
"type": {Type: tools.ParamTypeString, Required: true, Enum: []any{"test", "main"}},
}
result := ValidateParameters(map[string]any{"type": "test"}, schema)
Thread Safety: This function is safe for concurrent use.
type QualityConfig ¶
type QualityConfig struct {
// MinLengthForCitationRequirement is the minimum response length (chars)
// before citations become required. Default: 200.
MinLengthForCitationRequirement int
// EnableHedgingDetection enables detection of hedging language.
// Default: true.
EnableHedgingDetection bool
// EnableCitationRequirement enables citation requirement checking.
// Default: true.
EnableCitationRequirement bool
// StrictnessLevel controls validation strictness.
// 0 = disabled, 1 = warnings only (logged), 2 = soft fail (retryable), 3 = hard fail
// Default: 2 (soft fail).
StrictnessLevel int
}
QualityConfig configures the quality validator's strictness.
func DefaultQualityConfig ¶
func DefaultQualityConfig() QualityConfig
DefaultQualityConfig returns sensible defaults.
type QualityValidator ¶
type QualityValidator struct {
// contains filtered or unexported fields
}
QualityValidator validates response quality for analytical queries.
It checks for: - Hedging language that should be citations - Missing citations in long analytical responses - Proper evidence-based claims
Thread Safety: This type is safe for concurrent use after initialization.
func NewQualityValidator ¶
func NewQualityValidator(config *QualityConfig) *QualityValidator
NewQualityValidator creates a quality validator with the given config.
Inputs:
config - Configuration for strictness. Use nil for defaults.
Outputs:
*QualityValidator - Ready-to-use validator.
func (*QualityValidator) ValidateQuality ¶
func (v *QualityValidator) ValidateQuality(content string, isAnalytical bool) ValidationResult
ValidateQuality checks response content for quality issues.
Description:
Validates the response against quality rules: - Detects hedging language that should be evidence - Checks for citation presence in long responses - Returns validation result based on strictness level
Inputs:
content - The response content to validate. isAnalytical - Whether this is an analytical query requiring evidence.
Outputs:
ValidationResult - Contains validation status and reasoning.
Thread Safety: This method is safe for concurrent use.
type QueryClassifier ¶
type QueryClassifier interface {
// IsAnalytical determines if a query requires tool exploration.
//
// Description:
// Analyzes the query text to determine if it's asking about code
// structure, behavior, or properties that require tool exploration
// to answer accurately.
//
// Inputs:
// ctx - Context for tracing and cancellation. Must not be nil.
// query - The user's question text. Empty queries return false.
//
// Outputs:
// bool - True if the query requires tool exploration.
//
// Example:
// isAnalytical := classifier.IsAnalytical(ctx, "What tests exist?")
// // isAnalytical == true
//
// Thread Safety: This method is safe for concurrent use.
IsAnalytical(ctx context.Context, query string) bool
// SuggestTool recommends a starting tool based on query type.
//
// Description:
// Analyzes the query to suggest an appropriate first tool,
// validating that the suggested tool exists in the available set.
//
// Inputs:
// ctx - Context for tracing and cancellation. Must not be nil.
// query - The user's question text.
// available - List of available tool names to choose from.
//
// Outputs:
// string - The suggested tool name, or empty if no suggestion.
// bool - True if a valid suggestion was found.
//
// Example:
// tool, ok := classifier.SuggestTool(ctx, "What tests exist?",
// []string{"find_entry_points", "trace_data_flow"})
// // tool == "find_entry_points", ok == true
//
// Thread Safety: This method is safe for concurrent use.
SuggestTool(ctx context.Context, query string, available []string) (string, bool)
// SuggestToolWithHint recommends a tool with specific search instructions.
//
// Description:
// Enhanced version of SuggestTool that provides targeted search
// instructions. This enables Fix 2 (Targeted Exploration) by telling
// the LLM not just WHICH tool to use, but WHAT to search for.
//
// Inputs:
// ctx - Context for tracing and cancellation. Must not be nil.
// query - The user's question text.
// available - List of available tool names to choose from.
//
// Outputs:
// *ToolSuggestion - The suggestion with search instructions, or nil.
// bool - True if a valid suggestion was found.
//
// Example:
// suggestion, ok := classifier.SuggestToolWithHint(ctx, "What tests exist?",
// []string{"find_entry_points", "trace_data_flow"})
// // suggestion.ToolName == "find_entry_points"
// // suggestion.SearchHint == "Call find_entry_points with type='test'..."
// // suggestion.SearchPatterns == ["*_test.go", "func Test"]
//
// Thread Safety: This method is safe for concurrent use.
SuggestToolWithHint(ctx context.Context, query string, available []string) (*ToolSuggestion, bool)
}
QueryClassifier classifies user queries for tool forcing decisions.
Thread Safety: Implementations must be safe for concurrent use.
func MustNewClassifier ¶
func MustNewClassifier(config FactoryConfig) QueryClassifier
MustNewClassifier creates a QueryClassifier or panics on error.
Description:
Convenience wrapper around NewClassifier that panics on error. Use only for initialization code where errors are programming errors.
Inputs:
config - Factory configuration specifying the classifier type.
Outputs:
QueryClassifier - The created classifier.
Panics:
If configuration is invalid or classifier creation fails.
Thread Safety: This function is safe for concurrent use.
func NewClassifier ¶
func NewClassifier(config FactoryConfig) (QueryClassifier, error)
NewClassifier creates a QueryClassifier based on the factory configuration.
Description:
Factory function that creates either a RegexClassifier or LLMClassifier based on the configuration. This allows runtime selection of the classifier implementation.
Inputs:
config - Factory configuration specifying the classifier type.
Outputs:
QueryClassifier - The created classifier. error - Non-nil if configuration is invalid or classifier creation fails.
Example:
// Create regex classifier (default)
classifier, err := NewClassifier(FactoryConfig{Type: ClassifierTypeRegex})
// Create LLM classifier
classifier, err := NewClassifier(FactoryConfig{
Type: ClassifierTypeLLM,
LLMClient: client,
ToolDefinitions: toolDefs,
})
Thread Safety: This function is safe for concurrent use.
type RegexClassifier ¶
type RegexClassifier struct {
// contains filtered or unexported fields
}
RegexClassifier implements QueryClassifier using compiled regex patterns.
Thread Safety: This type is safe for concurrent use after initialization.
func NewRegexClassifier ¶
func NewRegexClassifier() *RegexClassifier
NewRegexClassifier creates a new RegexClassifier with compiled patterns.
Description:
Compiles all analytical query patterns into a single regex for efficient matching. The regex uses case-insensitive matching and word boundaries to reduce false positives.
Outputs:
*RegexClassifier - A new classifier ready for use.
Example:
classifier := NewRegexClassifier() isAnalytical := classifier.IsAnalytical(ctx, "What tests exist?")
Thread Safety: The returned classifier is safe for concurrent use.
func (*RegexClassifier) IsAnalytical ¶
func (c *RegexClassifier) IsAnalytical(ctx context.Context, query string) bool
IsAnalytical determines if a query requires tool exploration.
Description:
Uses precompiled regex with word boundaries to identify analytical queries. Creates a trace span for observability.
Inputs:
ctx - Context for tracing. Must not be nil. query - The user's question. Empty queries return false.
Outputs:
bool - True if the query matches analytical patterns.
Example:
classifier := NewRegexClassifier() result := classifier.IsAnalytical(ctx, "How does authentication work?") // result == true
Thread Safety: This method is safe for concurrent use.
func (*RegexClassifier) SuggestTool ¶
func (c *RegexClassifier) SuggestTool(ctx context.Context, query string, available []string) (string, bool)
SuggestTool recommends a starting tool based on query type.
Description:
Matches query against tool suggestion rules, then validates the suggestion exists in the available tools list.
Inputs:
ctx - Context for tracing. Must not be nil. query - The user's question text. available - List of available tool names. If nil/empty, returns no suggestion.
Outputs:
string - Suggested tool name, or empty if no valid suggestion. bool - True if a valid tool was suggested.
Example:
classifier := NewRegexClassifier()
tool, ok := classifier.SuggestTool(ctx, "What tests exist?",
[]string{"find_entry_points", "trace_data_flow"})
// tool == "find_entry_points", ok == true
Thread Safety: This method is safe for concurrent use.
func (*RegexClassifier) SuggestToolWithHint ¶
func (c *RegexClassifier) SuggestToolWithHint(ctx context.Context, query string, available []string) (*ToolSuggestion, bool)
SuggestToolWithHint recommends a tool with specific search instructions.
Description:
Enhanced version of SuggestTool that provides targeted search instructions. Matches query against targetedSearchRules to find the best tool AND specific search patterns the LLM should use.
Inputs:
ctx - Context for tracing. Must not be nil. query - The user's question text. available - List of available tool names. If nil/empty, returns nil.
Outputs:
*ToolSuggestion - The suggestion with search instructions, or nil. bool - True if a valid suggestion was found.
Example:
classifier := NewRegexClassifier()
suggestion, ok := classifier.SuggestToolWithHint(ctx, "What tests exist?",
[]string{"find_entry_points", "trace_data_flow"})
// suggestion.ToolName == "find_entry_points"
// suggestion.SearchHint contains specific instructions
Thread Safety: This method is safe for concurrent use.
type ResponseValidator ¶
type ResponseValidator struct {
// contains filtered or unexported fields
}
ResponseValidator checks LLM responses for quality and compliance.
The validator ensures responses: - Include tool calls when required - Don't contain prohibited patterns (e.g., "I'm ready to help") - Are not empty
Thread Safety: This type is safe for concurrent use after initialization.
func NewResponseValidator ¶
func NewResponseValidator() *ResponseValidator
NewResponseValidator creates a validator with default prohibited patterns.
Description:
Creates a validator configured to catch common unhelpful response patterns like "I'm ready to help" or "What would you like me to do".
Outputs:
*ResponseValidator - Ready-to-use validator.
Example:
validator := NewResponseValidator()
result := validator.Validate(response, toolChoice)
if !result.Valid && result.Retryable {
// Retry with stronger tool forcing
}
func (*ResponseValidator) AddProhibitedPattern ¶
func (v *ResponseValidator) AddProhibitedPattern(pattern string) error
AddProhibitedPattern adds a custom prohibited pattern.
Description:
Extends the validator with additional patterns to check. The pattern is compiled as a case-insensitive regex.
Inputs:
pattern - Regex pattern string to add.
Outputs:
error - Non-nil if pattern fails to compile.
Thread Safety: This method is NOT safe for concurrent use. Call only during initialization.
func (*ResponseValidator) HasProhibitedPattern ¶
func (v *ResponseValidator) HasProhibitedPattern(text string) (bool, string)
HasProhibitedPattern checks if text contains any prohibited patterns.
Description:
Standalone check for prohibited patterns without full validation. Useful for checking intermediate text before response is complete.
Inputs:
text - The text to check.
Outputs:
bool - True if a prohibited pattern was found. string - The matched pattern (empty if none).
Thread Safety: This method is safe for concurrent use.
func (*ResponseValidator) Validate ¶
func (v *ResponseValidator) Validate(response *llm.Response, toolChoice *llm.ToolChoice) ValidationResult
Validate checks if an LLM response meets quality requirements.
Description:
Validates the response based on the tool_choice that was used: - If tool_choice was "any" or "tool", requires tool calls - Checks for prohibited patterns in the response text - Checks for empty responses
Inputs:
response - The LLM response to validate. May be nil. toolChoice - The tool_choice that was used for the request.
Outputs:
ValidationResult - Contains validation status and reasoning.
Example:
result := validator.Validate(response, llm.ToolChoiceAny())
if !result.Valid {
log.Printf("Validation failed: %s", result.Reason)
}
Thread Safety: This method is safe for concurrent use.
func (*ResponseValidator) ValidateWithToolRequirement ¶
func (v *ResponseValidator) ValidateWithToolRequirement(response *llm.Response, requireTools bool) (bool, string)
ValidateWithToolRequirement checks if response has tool calls when required.
Description:
Simplified validation that only checks for tool call presence. Use this when you only care about whether tools were called.
Inputs:
response - The LLM response to check. requireTools - If true, response must have tool calls.
Outputs:
bool - True if validation passes. string - Reason for failure (empty if valid).
Thread Safety: This method is safe for concurrent use.
type RetryableValidator ¶
type RetryableValidator struct {
// contains filtered or unexported fields
}
WrapWithRetry creates a response validator that can retry on failure.
func NewRetryableValidator ¶
func NewRetryableValidator(maxRetries int) *RetryableValidator
NewRetryableValidator creates a validator with retry capabilities.
func (*RetryableValidator) GetRetryToolChoice ¶
func (r *RetryableValidator) GetRetryToolChoice(attempt int, original *llm.ToolChoice, suggestedTool string) *llm.ToolChoice
GetRetryToolChoice returns the tool_choice to use for retry.
func (*RetryableValidator) MaxRetries ¶
func (r *RetryableValidator) MaxRetries() int
MaxRetries returns the maximum number of retries configured.
func (*RetryableValidator) ShouldRetry ¶
func (r *RetryableValidator) ShouldRetry(result ValidationResult, attempt int) bool
ShouldRetry determines if a failed validation should be retried.
func (*RetryableValidator) Validate ¶
func (r *RetryableValidator) Validate(response *llm.Response, toolChoice *llm.ToolChoice) ValidationResult
Validate validates the response.
type SelectionResult ¶
type SelectionResult struct {
// ToolChoice is the selected tool_choice parameter.
ToolChoice *llm.ToolChoice
// SuggestedTool is the tool that was suggested (if any).
SuggestedTool string
// IsAnalytical indicates if the query was classified as analytical.
IsAnalytical bool
// Confidence is the classification confidence (0.0-1.0).
Confidence float64
// Reason explains the selection decision.
Reason string
}
SelectionResult contains the tool choice decision and reasoning.
type SelectorConfig ¶
type SelectorConfig struct {
// ForceThreshold sets the confidence level to force a specific tool.
ForceThreshold float64
// RequireThreshold sets the confidence level to require any tool.
RequireThreshold float64
}
SelectorConfig configures the ToolChoiceSelector.
func DefaultSelectorConfig ¶
func DefaultSelectorConfig() SelectorConfig
DefaultSelectorConfig returns sensible default thresholds.
type ToolChoiceSelector ¶
type ToolChoiceSelector struct {
// contains filtered or unexported fields
}
ToolChoiceSelector decides the tool_choice parameter based on query classification.
It uses confidence thresholds to determine whether to: - Force a specific tool (high confidence) - Force any tool (medium confidence) - Let the model decide (low confidence)
Thread Safety: This type is safe for concurrent use.
func NewToolChoiceSelector ¶
func NewToolChoiceSelector(classifier QueryClassifier, config *SelectorConfig) *ToolChoiceSelector
NewToolChoiceSelector creates a selector with the given classifier and config.
Description:
Creates a selector that uses the classifier to analyze queries and select appropriate tool_choice values based on confidence.
Inputs:
classifier - The query classifier to use. Must not be nil. config - Configuration for thresholds. Use nil for defaults.
Outputs:
*ToolChoiceSelector - Ready-to-use selector.
Example:
classifier := NewRegexClassifier() selector := NewToolChoiceSelector(classifier, nil) choice := selector.SelectToolChoice(ctx, "What tests exist?", tools)
func (*ToolChoiceSelector) SelectToolChoice ¶
func (s *ToolChoiceSelector) SelectToolChoice( ctx context.Context, query string, availableTools []string, ) SelectionResult
SelectToolChoice analyzes a query and returns the appropriate tool_choice.
Description:
Classifies the query and selects tool_choice based on confidence: - High confidence (>0.8) + valid tool → force specific tool - Medium confidence (>0.4) → require any tool - Low confidence → auto (model decides) - Non-analytical query → none (no tools)
Inputs:
ctx - Context for tracing. Must not be nil. query - The user's question. availableTools - List of available tool names.
Outputs:
SelectionResult - Contains the tool_choice and reasoning.
Example:
result := selector.SelectToolChoice(ctx, "What tests exist?",
[]string{"find_entry_points", "trace_data_flow"})
// result.ToolChoice.Type == "tool"
// result.ToolChoice.Name == "find_entry_points"
Thread Safety: This method is safe for concurrent use.
type ToolSuggestion ¶
type ToolSuggestion struct {
// ToolName is the suggested tool to call.
ToolName string
// SearchHint provides specific search instructions for the LLM.
// This tells the LLM WHAT to search for, not just which tool to use.
// Example: "Call find_entry_points with type='test' to find test functions"
SearchHint string
// SearchPatterns lists specific patterns the LLM should search for.
// These are concrete search terms relevant to the question.
// Example: ["*_test.go", "func Test", "t.Run"]
SearchPatterns []string
}
ToolSuggestion contains a tool recommendation with specific search instructions.
This enables targeted exploration by providing not just which tool to use, but HOW to use it with specific parameters and search patterns.
type ToolValidationResult ¶
type ToolValidationResult struct {
// Valid indicates if the tool name is valid.
Valid bool
// ToolName is the validated/matched tool name.
// May differ from input if fuzzy matching was used.
ToolName string
// FuzzyMatched indicates if fuzzy matching was used.
FuzzyMatched bool
// Warning contains any validation warning (e.g., fuzzy match used).
Warning string
// Error contains the error if validation failed.
Error error
}
ToolValidationResult contains the outcome of tool name validation.
func ValidateToolName ¶
func ValidateToolName(suggested string, available []string) ToolValidationResult
ValidateToolName validates the suggested tool against available tools.
Description:
Validates that the suggested tool name exists in the available tools list. First attempts exact match, then falls back to fuzzy matching using Levenshtein distance if no exact match is found.
Inputs:
suggested - The tool name suggested by the LLM. available - List of available tool names.
Outputs:
ToolValidationResult - Contains the matched tool or error.
Example:
result := ValidateToolName("find_tests", []string{"find_entry_points", "trace_data_flow"})
if result.Valid {
// result.ToolName == "find_entry_points" (fuzzy matched)
// result.FuzzyMatched == true
}
Thread Safety: This function is safe for concurrent use.
type ValidationResult ¶
type ValidationResult struct {
// Valid indicates if the response passed all checks.
Valid bool
// Reason explains why validation passed or failed.
Reason string
// Retryable indicates if a retry might succeed.
Retryable bool
// MatchedPattern contains the prohibited pattern that was matched (if any).
MatchedPattern string
}
ValidationResult contains the outcome of response validation.
Source Files