optimizers

package
v0.83.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 22, 2026 License: MIT Imports: 24 Imported by: 0

Documentation

Overview

Package optimizers implements the GEPA (Generative Evolutionary Prompt Adaptation) algorithm.

This file contains both the core functional GEPA implementation and advanced research-grade features including multi-level reflection systems, LLM-based self-critique, and sophisticated pattern analysis capabilities. Some advanced functions are currently unused but preserved for future integration and research purposes.

Index

Constants

This section is empty.

Variables

View Source
var AutoRunSettings = map[RunMode]struct {
	NumTrials int
	ValSize   int
}{
	LightMode:  {NumTrials: 7, ValSize: 100},
	MediumMode: {NumTrials: 25, ValSize: 300},
	HeavyMode:  {NumTrials: 50, ValSize: 1000},
}

AutoRunSettings defines default configurations for different run modes.

Functions

func CurrentCandidateID added in v0.80.0

func CurrentCandidateID(ctx context.Context) string

CurrentCandidateID returns the GEPA candidate ID attached to the context, or "unknown" when the context is outside an active GEPA evaluation.

func DefaultMultiObjectiveWeights added in v0.80.0

func DefaultMultiObjectiveWeights() map[string]float64

DefaultMultiObjectiveWeights returns a fresh copy of the baseline GEPA objective weights.

func RunEvolutionLoop added in v0.80.0

func RunEvolutionLoop(ctx context.Context, cfg EvolutionLoopConfig, hooks EvolutionLoopHooks) error

RunEvolutionLoop executes the shared GEPA-style generation loop once the caller provides initialization, evaluation, and evolution callbacks.

Types

type BootstrapFewShot

type BootstrapFewShot struct {
	Metric          func(example map[string]interface{}, prediction map[string]interface{}, ctx context.Context) bool
	MaxBootstrapped int
}

func NewBootstrapFewShot

func NewBootstrapFewShot(metric func(example map[string]interface{}, prediction map[string]interface{}, ctx context.Context) bool, maxBootstrapped int) *BootstrapFewShot

func (*BootstrapFewShot) Compile

func (b *BootstrapFewShot) Compile(ctx context.Context, program core.Program, dataset core.Dataset, metric core.Metric) (core.Program, error)

Compile implements the core.Optimizer interface.

func (*BootstrapFewShot) CompileLegacy added in v0.40.0

func (b *BootstrapFewShot) CompileLegacy(ctx context.Context, student, teacher core.Program, trainset []map[string]interface{}) (core.Program, error)

CompileLegacy provides backward compatibility for the old interface.

type COPRO added in v0.40.0

type COPRO struct {
	PromptModel     core.LLM // Optional model for generating prompts (if nil, uses default)
	Metric          core.Metric
	Breadth         int     // Number of prompt candidates to generate
	Depth           int     // Iterations of prompt refinement
	InitTemperature float64 // Randomness in prompt generation
	TrackStats      bool    // Optional performance tracking

	// LLM-assisted prompt generation components
	PromptGenerator  *LLMPromptGenerator
	CandidateHistory []PromptCandidate // Track previous attempts for learning
}

COPRO implements the Chain-of-Processing optimizer for prompt instruction and prefix optimization.

func NewCOPRO added in v0.40.0

func NewCOPRO(metric core.Metric, options ...COPROOption) *COPRO

NewCOPRO creates a new COPRO optimizer with enhanced LLM-assisted prompt generation.

func (*COPRO) Compile added in v0.40.0

func (c *COPRO) Compile(ctx context.Context, program core.Program, dataset core.Dataset, metric core.Metric) (core.Program, error)

Compile implements the core.Optimizer interface.

type COPROOption added in v0.40.0

type COPROOption func(*COPROOptions)

COPROOption is a functional option for configuring COPRO.

func WithBreadth added in v0.40.0

func WithBreadth(breadth int) COPROOption

WithBreadth sets the number of prompt candidates to generate.

func WithDepth added in v0.40.0

func WithDepth(depth int) COPROOption

WithDepth sets the number of refinement iterations.

func WithInitTemperature added in v0.40.0

func WithInitTemperature(temp float64) COPROOption

WithInitTemperature sets the randomness in prompt generation.

func WithPromptModel added in v0.1.0

func WithPromptModel(model core.LLM) COPROOption

WithPromptModel sets the model used for generating prompts.

func WithTrackStats added in v0.40.0

func WithTrackStats(track bool) COPROOption

WithTrackStats enables performance tracking.

type COPROOptions added in v0.40.0

type COPROOptions struct {
	PromptModel     core.LLM
	Breadth         int
	Depth           int
	InitTemperature float64
	TrackStats      bool
}

COPROOptions provides configuration options for COPRO.

type CandidateCaseObservation added in v0.80.0

type CandidateCaseObservation struct {
	Score             float64
	Passed            bool
	LatencyMS         float64
	TotalTokens       int64
	ToolCalls         int
	RobustnessFailure bool
}

type CandidateMetadata added in v0.39.0

type CandidateMetadata struct {
	// Individual performance metrics
	IndividualScores []float64 `json:"individual_scores"`
	DiversityScore   float64   `json:"diversity_score"`
	ImprovementDelta float64   `json:"improvement_delta"`

	// Multi-criteria scores
	MaxToMinGap float64 `json:"max_to_min_gap"`
	MaxScore    float64 `json:"max_score"`
	MaxToAvgGap float64 `json:"max_to_avg_gap"`

	// Selection tracking
	SelectionRank    int     `json:"selection_rank"`
	BucketAssignment int     `json:"bucket_assignment"`
	CompositeScore   float64 `json:"composite_score"`
}

CandidateMetadata contains detailed performance metrics for a candidate.

type CandidateMetrics added in v0.46.0

type CandidateMetrics struct {
	TotalEvaluations int                    `json:"total_evaluations"`
	SuccessCount     int                    `json:"success_count"`
	AverageFitness   float64                `json:"average_fitness"`
	BestFitness      float64                `json:"best_fitness"`
	ExecutionTimes   []time.Duration        `json:"execution_times"`
	ErrorCounts      map[string]int         `json:"error_counts"`
	Metadata         map[string]interface{} `json:"metadata"`
}

CandidateMetrics tracks detailed metrics for each candidate.

type CandidateResult added in v0.29.0

type CandidateResult struct {
	Program     core.Program       `json:"-"`
	Score       float64            `json:"score"`
	Step        int                `json:"step"`
	Temperature float64            `json:"temperature"`
	CreatedAt   time.Time          `json:"created_at"`
	Metadata    *CandidateMetadata `json:"metadata,omitempty"`
}

CandidateResult represents a candidate program and its performance.

type ContextAwarePerformanceTracker added in v0.46.0

type ContextAwarePerformanceTracker struct {
	// contains filtered or unexported fields
}

ContextAwarePerformanceTracker tracks performance metrics with context awareness.

func NewContextAwarePerformanceTracker added in v0.46.0

func NewContextAwarePerformanceTracker() *ContextAwarePerformanceTracker

NewContextAwarePerformanceTracker creates a new context-aware performance tracker.

type ContextPattern added in v0.46.0

type ContextPattern struct {
	PatternID         string                 `json:"pattern_id"`
	ContextFeatures   map[string]interface{} `json:"context_features"`
	SuccessFactors    []string               `json:"success_factors"`
	FailureFactors    []string               `json:"failure_factors"`
	OptimalCandidates []string               `json:"optimal_candidates"`
	Confidence        float64                `json:"confidence"`
	LastUpdated       time.Time              `json:"last_updated"`
}

ContextPattern represents learned patterns within execution contexts.

type ContextPerformanceMetrics added in v0.46.0

type ContextPerformanceMetrics struct {
	ContextType      string                 `json:"context_type"`
	ExecutionCount   int                    `json:"execution_count"`
	SuccessRate      float64                `json:"success_rate"`
	AverageLatency   time.Duration          `json:"average_latency"`
	QualityScore     float64                `json:"quality_score"`
	ErrorPatterns    map[string]int         `json:"error_patterns"`
	PerformanceTrend []float64              `json:"performance_trend"`
	LastUpdated      time.Time              `json:"last_updated"`
	Metadata         map[string]interface{} `json:"metadata"`
}

ContextPerformanceMetrics tracks performance within a specific context.

type ConvergenceStatus added in v0.46.0

type ConvergenceStatus struct {
	StagnationCount                int     `json:"stagnation_count"`
	PrematureConvergenceRisk       string  `json:"premature_convergence_risk"`
	ExplorationExploitationBalance string  `json:"exploration_exploitation_balance"`
	DiversityIndex                 float64 `json:"diversity_index"`
	IsConverged                    bool    `json:"is_converged"`
}

ConvergenceStatus tracks convergence indicators for the optimization.

type EmbeddingService added in v0.44.0

type EmbeddingService interface {
	GenerateEmbedding(ctx context.Context, text string) ([]float64, error)
	CosineSimilarity(vec1, vec2 []float64) float64
}

EmbeddingService defines the interface for generating context embeddings.

type EvolutionLoopConfig added in v0.80.0

type EvolutionLoopConfig struct {
	MaxGenerations int
	ReflectionFreq int
	PhaseName      string
}

EvolutionLoopConfig configures the shared GEPA-style generation loop.

type EvolutionLoopHooks added in v0.80.0

type EvolutionLoopHooks struct {
	Initialize         func(ctx context.Context) error
	BeforeGeneration   func(ctx context.Context, generation int) error
	EvaluateGeneration func(ctx context.Context, generation int) error
	// AfterEvaluation runs immediately after EvaluateGeneration for the same generation.
	// Callers can use it to consume state populated during evaluation, such as cached
	// fitness maps needed by archive or frontier updates.
	AfterEvaluation func(ctx context.Context, generation int) error
	Reflect         func(ctx context.Context, generation int) error
	OnNonFatalError func(ctx context.Context, stage string, generation int, err error)
	HasConverged    func(ctx context.Context, generation int) bool
	// Evolve is treated as fatal if it returns an error. Hooks that want retry or
	// best-effort behavior should absorb those failures internally.
	Evolve         func(ctx context.Context, generation int) error
	ReportProgress func(phase string, current, total int)
	Finalize       func(ctx context.Context) error
}

EvolutionLoopHooks injects domain-specific behavior into the shared loop.

type ExampleSelector added in v0.44.0

type ExampleSelector struct {
	Config         *MCPOptimizerConfig `json:"config"`
	SuccessHistory map[string][]bool   `json:"success_history"` // Track success history by pattern hash
	// contains filtered or unexported fields
}

ExampleSelector implements statistical weighting system for optimal example selection.

func (*ExampleSelector) RecordSuccess added in v0.44.0

func (es *ExampleSelector) RecordSuccess(interaction MCPInteraction, success bool)

RecordSuccess records the success/failure of an interaction pattern.

func (*ExampleSelector) SelectOptimalExamples added in v0.44.0

func (es *ExampleSelector) SelectOptimalExamples(ctx context.Context, candidates []MCPInteraction) ([]MCPInteraction, error)

SelectOptimalExamples selects the best examples based on statistical weighting.

type ExecutionPatterns added in v0.46.0

type ExecutionPatterns struct {
	SuccessRate         float64        `json:"success_rate"`
	SuccessCount        int            `json:"success_count"`
	TotalExecutions     int            `json:"total_executions"`
	AverageResponseTime time.Duration  `json:"average_response_time"`
	CommonFailures      []string       `json:"common_failures"`
	QualityIndicators   []string       `json:"quality_indicators"`
	RichTraceEvidence   []string       `json:"rich_trace_evidence"`
	ErrorDistribution   map[string]int `json:"error_distribution"`
	PerformanceTrends   []float64      `json:"performance_trends"`
}

ExecutionPatterns represents analyzed patterns from execution traces.

type ExecutionTrace added in v0.46.0

type ExecutionTrace struct {
	CandidateID string                 `json:"candidate_id"`
	ModuleName  string                 `json:"module_name"`
	Inputs      map[string]any         `json:"inputs"`
	Outputs     map[string]any         `json:"outputs"`
	Error       error                  `json:"error"`
	Duration    time.Duration          `json:"duration"`
	Success     bool                   `json:"success"`
	Timestamp   time.Time              `json:"timestamp"`
	ContextData map[string]interface{} `json:"context_data"`
}

ExecutionTrace tracks the execution of a module for reflection analysis.

type GEPA added in v0.46.0

type GEPA struct {
	// contains filtered or unexported fields
}

GEPA represents the main GEPA optimizer.

func NewGEPA added in v0.46.0

func NewGEPA(config *GEPAConfig) (*GEPA, error)

NewGEPA creates a new GEPA optimizer with the given configuration.

func (*GEPA) BootstrapPopulationFromSeed added in v0.80.0

func (g *GEPA) BootstrapPopulationFromSeed(ctx context.Context, seed *GEPACandidate) error

BootstrapPopulationFromSeed initializes a GEPA population from a single external seed candidate.

func (*GEPA) Compile added in v0.46.0

func (g *GEPA) Compile(ctx context.Context, program core.Program, dataset core.Dataset, metric core.Metric) (core.Program, error)

Compile implements the core.Optimizer interface for GEPA.

func (*GEPA) CurrentPopulation added in v0.80.0

func (g *GEPA) CurrentPopulation() *Population

CurrentPopulation returns the latest GEPA population.

func (*GEPA) EvolveCurrentPopulation added in v0.80.0

func (g *GEPA) EvolveCurrentPopulation(ctx context.Context) error

EvolveCurrentPopulation advances GEPA by one candidate-update step on the current population snapshot. Agent orchestration still decides how often to re-evaluate and reflect around those updates.

func (*GEPA) GetOptimizationState added in v0.46.0

func (g *GEPA) GetOptimizationState() *GEPAState

GetOptimizationState returns the current optimization state.

func (*GEPA) HasConverged added in v0.80.0

func (g *GEPA) HasConverged() bool

HasConverged returns whether GEPA's convergence criteria have been met.

func (*GEPA) ReflectCurrentPopulation added in v0.80.0

func (g *GEPA) ReflectCurrentPopulation(ctx context.Context, generation int) error

ReflectCurrentPopulation runs GEPA's current reflection logic for the given generation.

func (*GEPA) SetCurrentMultiObjectiveFitnessMap added in v0.80.0

func (g *GEPA) SetCurrentMultiObjectiveFitnessMap(fitnessMap map[string]*MultiObjectiveFitness)

SetCurrentMultiObjectiveFitnessMap records the current generation's multi-objective fitness map.

func (*GEPA) SetProgressReporter added in v0.46.0

func (g *GEPA) SetProgressReporter(reporter core.ProgressReporter)

SetProgressReporter sets a progress reporter for the optimizer.

func (*GEPA) UpdateBestCandidate added in v0.80.0

func (g *GEPA) UpdateBestCandidate(candidate *GEPACandidate)

UpdateBestCandidate records a new best candidate if it improves the optimizer state.

type GEPACandidate added in v0.46.0

type GEPACandidate struct {
	ID             string                 `json:"id"`
	ModuleName     string                 `json:"module_name"`
	Instruction    string                 `json:"instruction"`
	ComponentTexts map[string]string      `json:"component_texts,omitempty"`
	Demonstrations []core.Example         `json:"demonstrations"`
	Generation     int                    `json:"generation"`
	Fitness        float64                `json:"fitness"`
	ParentIDs      []string               `json:"parent_ids"`
	CreatedAt      time.Time              `json:"created_at"`
	Metadata       map[string]interface{} `json:"metadata"`
}

GEPACandidate represents a single prompt candidate in the GEPA population.

func CloneCandidate added in v0.80.0

func CloneCandidate(candidate *GEPACandidate) *GEPACandidate

CloneCandidate returns a deep copy of the candidate so callers can evaluate or mutate it without affecting the original lineage record.

type GEPAConfig added in v0.46.0

type GEPAConfig struct {
	// Evolutionary parameters
	PopulationSize int     `json:"population_size"` // Default: 20
	MaxGenerations int     `json:"max_generations"` // Default: 10
	MutationRate   float64 `json:"mutation_rate"`   // Default: 0.3
	CrossoverRate  float64 `json:"crossover_rate"`  // Default: 0.7
	ElitismRate    float64 `json:"elitism_rate"`    // Default: 0.1

	// Reflection parameters
	ReflectionFreq   int     `json:"reflection_frequency"` // Default: 2
	ReflectionDepth  int     `json:"reflection_depth"`     // Default: 3
	SelfCritiqueTemp float64 `json:"self_critique_temp"`   // Default: 0.7

	// Selection parameters
	TournamentSize     int    `json:"tournament_size"`     // Default: 3
	SelectionStrategy  string `json:"selection_strategy"`  // Default: "tournament" | "roulette" | "pareto" | "adaptive_pareto"
	ComponentSelection string `json:"component_selection"` // Default: "round_robin" | "random" | "all"

	// Convergence parameters
	ConvergenceThreshold float64 `json:"convergence_threshold"` // Default: 0.01
	StagnationLimit      int     `json:"stagnation_limit"`      // Default: 3

	// Performance parameters
	EvaluationBatchSize        int                   `json:"evaluation_batch_size"` // Default: 5
	ConcurrencyLevel           int                   `json:"concurrency_level"`     // Default: 3
	ValidationFrequency        int                   `json:"validation_frequency"`  // Default: 1
	ValidationSplit            float64               `json:"validation_split"`      // Default: 0.0 (disabled)
	ValidationExamples         []core.Example        `json:"-"`
	RandomSeed                 int64                 `json:"random_seed"`           // Default: 0 (time-based)
	MaxMergeInvocations        int                   `json:"max_merge_invocations"` // Default: 5
	MaxMetricCalls             int                   `json:"max_metric_calls"`      // Default: 0 (disabled)
	ScoreThreshold             float64               `json:"score_threshold"`       // Default: 0.0 (disabled)
	MaxRuntime                 time.Duration         `json:"max_runtime"`           // Default: 0 (disabled)
	RunDir                     string                `json:"run_dir,omitempty"`
	StopCallbacks              []GEPAStopper         `json:"-"`
	FeedbackEvaluator          GEPAFeedbackEvaluator `json:"-"`
	AddFormatFailureAsFeedback bool                  `json:"add_format_failure_as_feedback"`

	// LLM parameters
	GenerationModel string   `json:"generation_model"` // Default: uses core.GetDefaultLLM()
	ReflectionModel string   `json:"reflection_model"` // Default: uses core.GetTeacherLLM()
	GenerationLLM   core.LLM `json:"-"`
	ReflectionLLM   core.LLM `json:"-"`
	Temperature     float64  `json:"temperature"` // Default: 0.8
	MaxTokens       int      `json:"max_tokens"`  // Default: 500
}

GEPAConfig contains configuration options for GEPA optimizer.

func DefaultGEPAConfig added in v0.46.0

func DefaultGEPAConfig() *GEPAConfig

DefaultGEPAConfig returns the default configuration for GEPA.

type GEPAFeedback added in v0.80.0

type GEPAFeedback struct {
	Feedback        string                 `json:"feedback,omitempty"`
	TargetComponent string                 `json:"target_component,omitempty"`
	Metadata        map[string]interface{} `json:"metadata,omitempty"`
}

GEPAFeedback captures optional example-level guidance alongside the scalar GEPA score. Feedback is cached by whole-program component texts and example content, so evaluators should derive it from those inputs rather than candidate identity or generation-local state.

type GEPAFeedbackContext added in v0.80.0

type GEPAFeedbackContext struct {
	Candidate *GEPACandidate
	Example   core.Example
	Outputs   map[string]interface{}
	Err       error
}

GEPAFeedbackContext provides per-example execution context for optional feedback generation without changing the shared core.Metric contract.

type GEPAFeedbackEvaluator added in v0.80.0

type GEPAFeedbackEvaluator interface {
	EvaluateFeedback(ctx context.Context, expected, actual map[string]interface{}, info *GEPAFeedbackContext) *GEPAFeedback
}

GEPAFeedbackEvaluator optionally augments scalar scoring with example-level textual guidance that reflection and proposal prompts can consume.

type GEPAFeedbackEvaluatorFunc added in v0.80.0

type GEPAFeedbackEvaluatorFunc func(ctx context.Context, expected, actual map[string]interface{}, info *GEPAFeedbackContext) *GEPAFeedback

GEPAFeedbackEvaluatorFunc adapts a function into a GEPAFeedbackEvaluator.

func (GEPAFeedbackEvaluatorFunc) EvaluateFeedback added in v0.80.0

func (fn GEPAFeedbackEvaluatorFunc) EvaluateFeedback(ctx context.Context, expected, actual map[string]interface{}, info *GEPAFeedbackContext) *GEPAFeedback

type GEPAState added in v0.46.0

type GEPAState struct {
	CurrentGeneration        int                                  `json:"current_generation"`
	BestCandidate            *GEPACandidate                       `json:"best_candidate"`
	BestFitness              float64                              `json:"best_fitness"`
	BestValidationCandidate  *GEPACandidate                       `json:"best_validation_candidate,omitempty"`
	BestValidationFitness    float64                              `json:"best_validation_fitness"`
	PopulationHistory        []*Population                        `json:"population_history"`
	ReflectionHistory        []*ReflectionResult                  `json:"reflection_history"`
	ConvergenceStatus        *ConvergenceStatus                   `json:"convergence_status"`
	StartTime                time.Time                            `json:"start_time"`
	LastImprovement          time.Time                            `json:"last_improvement"`
	LastValidatedGeneration  int                                  `json:"last_validated_generation"`
	ExecutionTraces          map[string][]ExecutionTrace          `json:"execution_traces"`
	CandidateMetrics         map[string]*CandidateMetrics         `json:"candidate_metrics"`
	MultiObjectiveFitnessMap map[string]*MultiObjectiveFitness    `json:"multi_objective_fitness_map"`
	ValidationFrontier       map[int]*gepaValidationFrontierEntry `json:"validation_frontier,omitempty"`
	ValidationCoverage       map[string]int                       `json:"validation_coverage,omitempty"`
	MergeInvocations         int                                  `json:"merge_invocations,omitempty"`
	PerformedMerges          map[string]bool                      `json:"performed_merges,omitempty"`
	MetricCalls              int                                  `json:"metric_calls,omitempty"`
	StopReason               string                               `json:"stop_reason,omitempty"`
	StopMetadata             map[string]interface{}               `json:"stop_metadata,omitempty"`

	// Pareto archive for elite solution management
	ParetoArchive     []*GEPACandidate                  `json:"pareto_archive"`
	ArchiveFitnessMap map[string]*MultiObjectiveFitness `json:"archive_fitness_map"`
	MaxArchiveSize    int                               `json:"max_archive_size"`
	// contains filtered or unexported fields
}

GEPAState tracks the complete state of GEPA optimization.

func GetGEPAState added in v0.46.0

func GetGEPAState(ctx context.Context) *GEPAState

GetGEPAState retrieves GEPA state from context.

func NewGEPAState added in v0.46.0

func NewGEPAState() *GEPAState

NewGEPAState creates a new GEPA optimization state.

func (*GEPAState) AddMetricCalls added in v0.80.0

func (s *GEPAState) AddMetricCalls(count int)

func (*GEPAState) AddTrace added in v0.46.0

func (s *GEPAState) AddTrace(trace *ExecutionTrace)

AddTrace adds an execution trace to the state.

func (*GEPAState) BeginCandidateCaseRecording added in v0.80.0

func (s *GEPAState) BeginCandidateCaseRecording(candidateID string)

BeginCandidateCaseRecording resets the per-example observation buffer for a candidate so aggregate fitness can be derived from the current evaluation batch only.

func (*GEPAState) BestCandidateForApplication added in v0.82.0

func (s *GEPAState) BestCandidateForApplication() (*GEPACandidate, float64, bool)

BestCandidateForApplication returns the final candidate that should be surfaced to callers, preferring the best validation candidate when one exists and otherwise falling back to the best training candidate.

func (*GEPAState) FinalizeCandidateCaseFitness added in v0.80.0

func (s *GEPAState) FinalizeCandidateCaseFitness(candidate *GEPACandidate, averageScore float64) *MultiObjectiveFitness

FinalizeCandidateCaseFitness derives aggregate multi-objective fitness from the per-example observations recorded for a candidate and persists it once.

func (*GEPAState) GetCandidateEvaluation added in v0.80.0

func (s *GEPAState) GetCandidateEvaluation(candidateID string) *gepaCandidateEvaluation

GetCandidateEvaluation returns the cached evaluation for a candidate from the latest evaluated generation.

func (*GEPAState) GetCandidateReflection added in v0.80.0

func (s *GEPAState) GetCandidateReflection(candidateID string) *ReflectionResult

GetCandidateReflection returns the cached reflection for a candidate from the latest reflected generation.

func (*GEPAState) GetCandidateValidationEvaluation added in v0.80.0

func (s *GEPAState) GetCandidateValidationEvaluation(candidateID string) *gepaCandidateEvaluation

GetCandidateValidationEvaluation returns the cached validation evaluation for a candidate from the latest validation pass.

func (*GEPAState) GetEvaluationCaseCache added in v0.80.0

func (s *GEPAState) GetEvaluationCaseCache(key string) *gepaCachedEvaluationCase

func (*GEPAState) GetParetoArchive added in v0.46.0

func (s *GEPAState) GetParetoArchive() []*GEPACandidate

GetParetoArchive returns the current Pareto archive of elite solutions.

func (*GEPAState) GetTracesForCandidate added in v0.46.0

func (s *GEPAState) GetTracesForCandidate(candidateID string) []ExecutionTrace

GetTracesForCandidate returns all execution traces for a specific candidate.

func (*GEPAState) HasRecordedMerge added in v0.80.0

func (s *GEPAState) HasRecordedMerge(key string) bool

func (*GEPAState) MergeBudgetAvailable added in v0.80.0

func (s *GEPAState) MergeBudgetAvailable(maxInvocations int) bool

func (*GEPAState) MetricCallCount added in v0.80.0

func (s *GEPAState) MetricCallCount() int

func (*GEPAState) RecordCandidateCaseObservation added in v0.80.0

func (s *GEPAState) RecordCandidateCaseObservation(candidateID string, observation CandidateCaseObservation)

RecordCandidateCaseObservation stores one example-level observation for the currently evaluated candidate without overwriting aggregate candidate fitness.

func (*GEPAState) RecordCandidateFitness added in v0.80.0

func (s *GEPAState) RecordCandidateFitness(candidate *GEPACandidate, fitness *MultiObjectiveFitness, averageScore float64)

RecordCandidateFitness updates the stored fitness/metadata for a candidate.

func (*GEPAState) RecordMergeInvocation added in v0.80.0

func (s *GEPAState) RecordMergeInvocation(key string, maxInvocations int) bool

func (*GEPAState) RecordStopDecision added in v0.80.0

func (s *GEPAState) RecordStopDecision(decision *GEPAStopDecision)

func (*GEPAState) ResetEvaluationCaseCache added in v0.80.0

func (s *GEPAState) ResetEvaluationCaseCache()

func (*GEPAState) ResetRunControls added in v0.80.0

func (s *GEPAState) ResetRunControls(now time.Time)

ResetRunControls clears run-scoped stop/budget bookkeeping before a fresh Compile call. Merge dedup state is intentionally reset here too, because each Compile invocation is treated as an independent GEPA optimization run.

func (*GEPAState) SetCandidateEvaluations added in v0.80.0

func (s *GEPAState) SetCandidateEvaluations(evaluations map[string]*gepaCandidateEvaluation)

SetCandidateEvaluations replaces the latest candidate evaluation cache for the current generation. Reflection consumes this to build example-grounded prompts without re-running candidate evaluation.

func (*GEPAState) SetCandidateReflections added in v0.80.0

func (s *GEPAState) SetCandidateReflections(reflections map[string]*ReflectionResult)

SetCandidateReflections replaces the latest candidate reflection cache for the current generation. Proposal generation consumes this to turn reflection into concrete instruction updates without re-running reflection.

func (*GEPAState) SetCandidateValidationEvaluations added in v0.80.0

func (s *GEPAState) SetCandidateValidationEvaluations(evaluations map[string]*gepaCandidateEvaluation)

SetCandidateValidationEvaluations replaces the current validation-evaluation cache. Validation selection and final candidate choice consult this separately from minibatch/training evaluations.

func (*GEPAState) SetValidationFrontier added in v0.80.0

func (s *GEPAState) SetValidationFrontier(frontier map[int]*gepaValidationFrontierEntry, coverage map[string]int)

SetValidationFrontier replaces the current validation frontier and candidate coverage counts derived from the latest validation pass.

func (*GEPAState) UpdateParetoArchive added in v0.46.0

func (s *GEPAState) UpdateParetoArchive(candidates []*GEPACandidate, fitnessMap map[string]*MultiObjectiveFitness)

UpdateParetoArchive maintains elite Pareto-optimal solutions across generations.

func (*GEPAState) UpsertCandidateEvaluation added in v0.80.0

func (s *GEPAState) UpsertCandidateEvaluation(candidateID string, evaluation *gepaCandidateEvaluation)

UpsertCandidateEvaluation updates the cached evaluation for a single candidate without replacing the rest of the current evaluation cache.

func (*GEPAState) UpsertEvaluationCaseCache added in v0.80.0

func (s *GEPAState) UpsertEvaluationCaseCache(key string, cached *gepaCachedEvaluationCase)

func (*GEPAState) ValidationFrontierSnapshot added in v0.80.0

func (s *GEPAState) ValidationFrontierSnapshot() (map[int]*gepaValidationFrontierEntry, map[string]int)

ValidationFrontierSnapshot returns cloned validation-frontier state for selection and tests without exposing shared mutable maps.

type GEPAStopDecision added in v0.80.0

type GEPAStopDecision struct {
	Reason   string                 `json:"reason"`
	Metadata map[string]interface{} `json:"metadata,omitempty"`
}

GEPAStopDecision records why a GEPA run should stop.

type GEPAStopper added in v0.80.0

type GEPAStopper func(context.Context, *GEPA) *GEPAStopDecision

GEPAStopper allows callers to plug additional stopping logic into GEPA. Returning nil means the run should continue.

type IndividualReflectionInsights added in v0.46.0

type IndividualReflectionInsights struct {
	CommonStrengths        []string       `json:"common_strengths"`
	CommonWeaknesses       []string       `json:"common_weaknesses"`
	SuccessPatterns        []string       `json:"success_patterns"`
	FailurePatterns        []string       `json:"failure_patterns"`
	ImprovementThemes      []string       `json:"improvement_themes"`
	ConfidenceDistribution map[string]int `json:"confidence_distribution"`
}

type InstructionGenerator added in v0.28.0

type InstructionGenerator struct {
	PromptModel   core.LLM
	MaxCandidates int
	Temperature   float64
}

InstructionGenerator handles the generation of instruction candidates.

func (*InstructionGenerator) GenerateCandidates added in v0.28.0

func (g *InstructionGenerator) GenerateCandidates(
	ctx context.Context,
	program core.Program,
	demos []core.Example,
) (map[int][]string, error)

GenerateCandidates creates instruction candidates for each predictor.

type IntrospectionResult added in v0.29.0

type IntrospectionResult struct {
	Analysis             string   `json:"analysis"`
	Recommendations      []string `json:"recommendations"`
	Confidence           float64  `json:"confidence"`
	IdentifiedPatterns   []string `json:"identified_patterns"`
	SuggestedAdjustments []string `json:"suggested_adjustments"`
}

IntrospectionResult contains self-analysis and advice.

type LLMPromptGenerator added in v0.41.0

type LLMPromptGenerator struct {
	// contains filtered or unexported fields
}

LLMPromptGenerator handles sophisticated prompt generation using LLM assistance.

func NewLLMPromptGenerator added in v0.41.0

func NewLLMPromptGenerator(llm core.PromptModel, signature core.Signature) *LLMPromptGenerator

NewLLMPromptGenerator creates a new LLM-assisted prompt generator.

type MCPInteraction added in v0.44.0

type MCPInteraction struct {
	ID            string                 `json:"id"`
	Timestamp     time.Time              `json:"timestamp"`
	Context       string                 `json:"context"`                  // The user query or context that triggered this interaction
	ToolName      string                 `json:"tool_name"`                // Name of the MCP tool used
	Parameters    map[string]interface{} `json:"parameters"`               // Parameters passed to the tool
	Result        core.ToolResult        `json:"result"`                   // Result returned by the tool
	Success       bool                   `json:"success"`                  // Whether the interaction was successful
	ExecutionTime time.Duration          `json:"execution_time"`           // Time taken to execute
	ErrorMessage  string                 `json:"error_message,omitempty"`  // Error message if failed
	ContextVector []float64              `json:"context_vector,omitempty"` // Embedding vector for the context
	Metadata      map[string]interface{} `json:"metadata"`                 // Additional metadata
}

MCPInteraction represents a single MCP tool interaction with all relevant context.

type MCPMetrics added in v0.44.0

type MCPMetrics struct {
	Timestamp             time.Time `json:"timestamp"`
	ToolSelectionAccuracy float64   `json:"tool_selection_accuracy"`
	ParameterOptimality   float64   `json:"parameter_optimality"`
	ExecutionSuccessRate  float64   `json:"execution_success_rate"`
	AverageExecutionTime  float64   `json:"average_execution_time"`
	InteractionsProcessed int       `json:"interactions_processed"`
}

MCPMetrics represents performance metrics for MCP tool interactions.

type MCPOptimizer added in v0.44.0

type MCPOptimizer struct {
	core.BaseOptimizer
	PatternCollector  *PatternCollector
	SimilarityMatcher *SimilarityMatcher
	ExampleSelector   *ExampleSelector
	MetricsEvaluator  *MetricsEvaluator
	ToolOrchestrator  *ToolOrchestrator
	Config            *MCPOptimizerConfig
	// contains filtered or unexported fields
}

MCPOptimizer implements an optimizer specifically designed for MCP (Model Context Protocol) workflows. It follows the KNNFewShot + Statistical Weighting methodology to learn from successful MCP tool interactions.

func NewMCPOptimizer added in v0.44.0

func NewMCPOptimizer(embeddingService EmbeddingService) *MCPOptimizer

NewMCPOptimizer creates a new MCP optimizer with default configuration.

func NewMCPOptimizerWithConfig added in v0.44.0

func NewMCPOptimizerWithConfig(config *MCPOptimizerConfig, embeddingService EmbeddingService) *MCPOptimizer

NewMCPOptimizerWithConfig creates a new MCP optimizer with custom configuration.

func (*MCPOptimizer) Compile added in v0.44.0

func (m *MCPOptimizer) Compile(ctx context.Context, program core.Program, dataset core.Dataset, metric core.Metric) (core.Program, error)

Compile implements the core.Optimizer interface for MCP-specific optimization.

func (*MCPOptimizer) GetOptimizationStats added in v0.44.0

func (m *MCPOptimizer) GetOptimizationStats() map[string]interface{}

GetOptimizationStats returns current optimization statistics.

func (*MCPOptimizer) LearnFromInteraction added in v0.44.0

func (m *MCPOptimizer) LearnFromInteraction(ctx context.Context, interaction MCPInteraction) error

LearnFromInteraction learns from a new MCP interaction.

func (*MCPOptimizer) OptimizeInteraction added in v0.44.0

func (m *MCPOptimizer) OptimizeInteraction(ctx context.Context, context string, toolName string) (*MCPInteraction, error)

OptimizeInteraction optimizes a single MCP interaction using learned patterns.

type MCPOptimizerConfig added in v0.44.0

type MCPOptimizerConfig struct {
	MaxPatterns          int     `json:"max_patterns"`          // Maximum number of patterns to store
	SimilarityThreshold  float64 `json:"similarity_threshold"`  // Minimum similarity for pattern matching
	KNearestNeighbors    int     `json:"k_nearest_neighbors"`   // Number of neighbors for KNN matching
	SuccessWeightFactor  float64 `json:"success_weight_factor"` // Weight factor for successful patterns
	EmbeddingDimensions  int     `json:"embedding_dimensions"`  // Dimensions for context embeddings
	LearningEnabled      bool    `json:"learning_enabled"`      // Whether to learn from new interactions
	MetricsWindowSize    int     `json:"metrics_window_size"`   // Window size for performance metrics
	OptimizationInterval int     `json:"optimization_interval"` // Interval for optimization cycles (in interactions)
}

MCPOptimizerConfig holds configuration parameters for the MCP optimizer.

type MIPRO added in v0.1.0

type MIPRO struct {
	// contains filtered or unexported fields
}

MIPRO is the main optimizer implementing multi-step interactive prompt optimization.

func NewMIPRO added in v0.1.0

func NewMIPRO(
	metric func(example, prediction map[string]interface{}, ctx context.Context) float64,
	opts ...MIPROOption,
) *MIPRO

NewMIPRO creates a new MIPRO optimizer instance.

func (*MIPRO) Compile added in v0.1.0

func (m *MIPRO) Compile(
	ctx context.Context,
	program core.Program,
	dataset core.Dataset,
	metric core.Metric,
) (core.Program, error)

Compile implements the main optimization loop.

type MIPROConfig added in v0.28.0

type MIPROConfig struct {
	Mode           RunMode
	NumTrials      int
	ValSize        int
	MiniBatchSize  int
	AdaptiveParams bool
	ScalingFactors struct {
		TrialsPerVariable float64
		BatchSizeScaling  float64
	}
	TeacherSettings map[string]interface{}

	// TPE specific configuration
	TPEGamma        float64
	TPEGenerations  int
	Seed            int64
	NumModules      int // Number of modules to optimize (can be inferred from program)
	MaxLabeledDemos int // Maximum number of labeled demonstrations to use
}

MIPROConfig contains all configuration options for the optimizer.

type MIPROMetrics added in v0.28.0

type MIPROMetrics struct {
	TeacherPerformance  float64
	StudentPerformance  float64
	PromptEffectiveness map[string]float64
	OptimizationHistory []OptimizationStep
	TokenUsage          *core.TokenInfo
}

MIPROMetrics tracks comprehensive optimization metrics.

type MIPROOption added in v0.1.0

type MIPROOption func(*MIPRO)

MIPROOption defines a function type for configuring MIPRO.

func WithMaxLabeledDemos added in v0.1.0

func WithMaxLabeledDemos(maxDemos int) MIPROOption

WithMaxLabeledDemos sets the maximum number of labeled demos to use.

func WithMiniBatchSize added in v0.1.0

func WithMiniBatchSize(size int) MIPROOption

func WithMode added in v0.28.0

func WithMode(mode RunMode) MIPROOption

WithMode sets the optimization mode.

func WithModels added in v0.28.0

func WithModels(promptModel, taskModel core.LLM) MIPROOption

WithModels explicitly sets the prompt and task models for MIPRO.

func WithNumCandidates added in v0.1.0

func WithNumCandidates(num int) MIPROOption

func WithNumModules added in v0.28.0

func WithNumModules(numModules int) MIPROOption

WithNumModules explicitly sets the number of modules to optimize.

func WithNumTrials added in v0.1.0

func WithNumTrials(trials int) MIPROOption

WithNumTrials sets the number of optimization trials.

func WithRandomSeed added in v0.28.0

func WithRandomSeed(seed int64) MIPROOption

WithRandomSeed sets a specific random seed for reproducibility.

func WithSearchStrategy added in v0.28.0

func WithSearchStrategy(strategy SearchStrategy) MIPROOption

WithSearchStrategy sets a custom search strategy.

func WithTPEGamma added in v0.28.0

func WithTPEGamma(gamma float64) MIPROOption

WithTPEGamma sets the gamma parameter for the TPE optimizer.

func WithTPEGenerations added in v0.28.0

func WithTPEGenerations(generations int) MIPROOption

WithTPEGenerations sets the number of candidates to generate for each TPE optimization step.

func WithTeacherSettings added in v0.28.0

func WithTeacherSettings(settings map[string]interface{}) MIPROOption

WithTeacherSettings configures the teacher model settings.

type MetricsEvaluator added in v0.44.0

type MetricsEvaluator struct {
	Metrics []MCPMetrics        `json:"metrics"`
	Config  *MCPOptimizerConfig `json:"config"`
	// contains filtered or unexported fields
}

MetricsEvaluator provides MCP-specific performance metrics.

func (*MetricsEvaluator) GetAverageMetrics added in v0.44.0

func (me *MetricsEvaluator) GetAverageMetrics() *MCPMetrics

GetAverageMetrics calculates average metrics over the window.

func (*MetricsEvaluator) GetLatestMetrics added in v0.44.0

func (me *MetricsEvaluator) GetLatestMetrics() *MCPMetrics

GetLatestMetrics returns the most recent metrics.

func (*MetricsEvaluator) RecordMetrics added in v0.44.0

func (me *MetricsEvaluator) RecordMetrics(ctx context.Context, metrics MCPMetrics)

RecordMetrics records new performance metrics.

type MultiObjectiveFitness added in v0.46.0

type MultiObjectiveFitness struct {
	// Core objectives
	SuccessRate    float64 `json:"success_rate"`   // Objective 1: Basic success rate
	OutputQuality  float64 `json:"output_quality"` // Objective 2: Quality of outputs
	Efficiency     float64 `json:"efficiency"`     // Objective 3: Execution efficiency
	Robustness     float64 `json:"robustness"`     // Objective 4: Error handling capability
	Generalization float64 `json:"generalization"` // Objective 5: Cross-context performance

	// Meta-objectives
	Diversity  float64 `json:"diversity"`  // Population diversity contribution
	Innovation float64 `json:"innovation"` // Novel solution characteristics

	// Aggregated score for backward compatibility
	WeightedScore float64 `json:"weighted_score"`
}

MultiObjectiveFitness represents fitness across multiple objectives for Pareto-based selection.

func (*MultiObjectiveFitness) ComputeWeightedScore added in v0.80.0

func (f *MultiObjectiveFitness) ComputeWeightedScore(weights map[string]float64) float64

ComputeWeightedScore aggregates the objective vector using the supplied weights.

func (*MultiObjectiveFitness) ObjectiveValue added in v0.80.0

func (f *MultiObjectiveFitness) ObjectiveValue(objective string) float64

ObjectiveValue returns the named objective value from the fitness vector.

type ObjectiveStats added in v0.46.0

type ObjectiveStats struct {
	Mean   float64 `json:"mean"`
	StdDev float64 `json:"std_dev"`
	Min    float64 `json:"min"`
	Max    float64 `json:"max"`
	Range  float64 `json:"range"`
}

ObjectiveStats represents statistical information for a single objective.

type OptimizationState added in v0.28.0

type OptimizationState struct {
	SuccessfulPatterns []string
	PromptEvolution    []PromptVersion
	TeacherScores      map[string]float64
	CurrentIteration   int
	BestScore          float64
	Convergence        float64
}

OptimizationState tracks the progress of optimization.

type OptimizationStep added in v0.28.0

type OptimizationStep struct {
	Trial         int
	Performance   float64
	Improvements  []string
	FailurePoints []string
}

OptimizationStep represents a single step in the optimization process.

type ParetoFront added in v0.46.0

type ParetoFront struct {
	Candidates []*GEPACandidate `json:"candidates"`
	Rank       int              `json:"rank"` // Pareto rank (1 = best front)
	Size       int              `json:"size"`
}

ParetoFront represents a Pareto front for multi-objective optimization.

type PatternCollector added in v0.44.0

type PatternCollector struct {
	Patterns    []MCPInteraction    `json:"patterns"`
	IndexByCtx  map[string][]int    `json:"index_by_ctx"`  // Index patterns by context hash
	IndexByTool map[string][]int    `json:"index_by_tool"` // Index patterns by tool name
	Config      *MCPOptimizerConfig `json:"config"`
	// contains filtered or unexported fields
}

PatternCollector logs and stores successful MCP tool interactions with context.

func (*PatternCollector) AddInteraction added in v0.44.0

func (pc *PatternCollector) AddInteraction(ctx context.Context, interaction MCPInteraction) error

AddInteraction adds a new MCP interaction to the pattern collection.

func (*PatternCollector) GetPatternCount added in v0.44.0

func (pc *PatternCollector) GetPatternCount() int

GetPatternCount returns the total number of stored patterns.

func (*PatternCollector) GetPatternsByTool added in v0.44.0

func (pc *PatternCollector) GetPatternsByTool(toolName string) []MCPInteraction

GetPatternsByTool retrieves all patterns for a specific tool.

func (*PatternCollector) GetSimilarPatterns added in v0.44.0

func (pc *PatternCollector) GetSimilarPatterns(ctx context.Context, context string, toolName string) ([]MCPInteraction, error)

GetSimilarPatterns retrieves patterns similar to the given context.

type PerformanceContext added in v0.46.0

type PerformanceContext struct {
	// Execution context
	ExecutionID string `json:"execution_id"`
	CandidateID string `json:"candidate_id"`
	ModuleName  string `json:"module_name"`

	// Input characteristics
	InputTypes      []string `json:"input_types"`
	InputSize       int      `json:"input_size"`
	InputComplexity float64  `json:"input_complexity"`

	// System context
	SystemLoad      float64 `json:"system_load"`
	MemoryUsage     float64 `json:"memory_usage"`
	ConcurrentTasks int     `json:"concurrent_tasks"`

	// Generation context
	Generation        int     `json:"generation"`
	PopulationSize    int     `json:"population_size"`
	SelectionPressure float64 `json:"selection_pressure"`

	// Temporal context
	TimeOfDay      int    `json:"time_of_day"`     // Hour of day
	ExecutionPhase string `json:"execution_phase"` // init, evolution, reflection

	// Custom context data
	CustomData map[string]interface{} `json:"custom_data"`
}

PerformanceContext represents the context of a performance measurement.

type PerformanceLogger added in v0.46.0

type PerformanceLogger struct {
	// contains filtered or unexported fields
}

PerformanceLogger tracks performance metrics for GEPA optimization.

func NewPerformanceLogger added in v0.46.0

func NewPerformanceLogger() *PerformanceLogger

NewPerformanceLogger creates a new performance logger.

func (*PerformanceLogger) GetCandidateMetrics added in v0.46.0

func (pl *PerformanceLogger) GetCandidateMetrics(candidateID string) *CandidateMetrics

GetCandidateMetrics retrieves metrics for a candidate.

func (*PerformanceLogger) LogCandidateMetrics added in v0.46.0

func (pl *PerformanceLogger) LogCandidateMetrics(candidateID string, metrics *CandidateMetrics)

LogCandidateMetrics logs metrics for a candidate.

type PipelineChannels added in v0.39.0

type PipelineChannels struct {
	CandidateGeneration chan *PipelineStage
	BatchSampling       chan *PipelineStage
	CandidateEvaluation chan *PipelineStage
	Results             chan *PipelineStage
	Errors              chan error
	Done                chan struct{}
}

PipelineChannels contains channels for pipeline communication.

type PipelineResult added in v0.39.0

type PipelineResult struct {
	StepIndex      int
	BestProgram    core.Program
	BestScore      float64
	AllScores      []float64
	ProcessingTime time.Duration
	StageTimings   map[string]time.Duration
}

PipelineResult represents the result of a pipeline stage.

type PipelineStage added in v0.39.0

type PipelineStage struct {
	StepIndex  int
	Candidates []core.Program
	Batch      []core.Example
	Scores     []float64
	Timestamp  time.Time
	Error      error
}

PipelineStage represents a pipeline stage with candidates and associated data.

type Population added in v0.46.0

type Population struct {
	Candidates    []*GEPACandidate `json:"candidates"`
	Generation    int              `json:"generation"`
	BestFitness   float64          `json:"best_fitness"`
	BestCandidate *GEPACandidate   `json:"best_candidate"`
}

Population represents a generation of prompt candidates.

type PopulationInsights added in v0.46.0

type PopulationInsights struct {
	DiversityIndex         float64  `json:"diversity_index"`
	AverageFitness         float64  `json:"average_fitness"`
	BestFitness            float64  `json:"best_fitness"`
	WorstFitness           float64  `json:"worst_fitness"`
	FitnessVariance        float64  `json:"fitness_variance"`
	HighPerformingPatterns []string `json:"high_performing_patterns"`
	CommonWeaknesses       []string `json:"common_weaknesses"`
}

PopulationInsights contains analysis of the current population.

type PromptCandidate added in v0.40.0

type PromptCandidate struct {
	Instruction     string
	Prefix          string
	Score           float64 // Training score
	ValidationScore float64 // Validation score to prevent overfitting
	Generation      int     // Which depth iteration this was generated in
	Diversity       float64 // Semantic diversity score
	Rank            int     // Performance ranking
	AttemptID       string  // Unique identifier for tracking
}

PromptCandidate represents a candidate prompt configuration.

type PromptComponent added in v0.28.0

type PromptComponent struct {
	Type    string
	Content string
	Score   float64
}

PromptComponent represents a specific part of a prompt.

type PromptVersion added in v0.28.0

type PromptVersion struct {
	Template    string
	Performance float64
	Components  []PromptComponent
}

PromptVersion represents a specific version of a prompt template.

type ReflectionResult added in v0.46.0

type ReflectionResult struct {
	CandidateID     string    `json:"candidate_id"`
	Strengths       []string  `json:"strengths"`
	Weaknesses      []string  `json:"weaknesses"`
	Suggestions     []string  `json:"suggestions"`
	ConfidenceScore float64   `json:"confidence_score"`
	Timestamp       time.Time `json:"timestamp"`
	ReflectionDepth int       `json:"reflection_depth"`
}

ReflectionResult contains the results of reflecting on a prompt candidate.

type RunMode added in v0.28.0

type RunMode string

RunMode defines different optimization intensities for MIPRO.

const (
	LightMode  RunMode = "light"
	MediumMode RunMode = "medium"
	HeavyMode  RunMode = "heavy"
)

type SIMBA added in v0.29.0

type SIMBA struct {
	// contains filtered or unexported fields
}

SIMBA implements Stochastic Introspective Mini-Batch Ascent optimizer.

func NewSIMBA added in v0.29.0

func NewSIMBA(opts ...SIMBAOption) *SIMBA

NewSIMBA creates a new SIMBA optimizer.

func (*SIMBA) Compile added in v0.29.0

func (s *SIMBA) Compile(ctx context.Context, program core.Program, dataset core.Dataset, metric core.Metric) (core.Program, error)

Compile implements the core.Optimizer interface for SIMBA.

func (*SIMBA) GetConfig added in v0.29.0

func (s *SIMBA) GetConfig() SIMBAConfig

GetConfig returns the current configuration.

func (*SIMBA) GetState added in v0.29.0

func (s *SIMBA) GetState() SIMBAState

GetState returns the current optimization state (thread-safe).

type SIMBAConfig added in v0.29.0

type SIMBAConfig struct {
	// Mini-batch configuration
	BatchSize     int `json:"batch_size"`     // Default: 32
	MaxSteps      int `json:"max_steps"`      // Default: 8
	NumCandidates int `json:"num_candidates"` // Default: 6

	// Temperature controls
	SamplingTemperature float64 `json:"sampling_temperature"` // Default: 0.2

	// Introspective learning
	IntrospectionFrequency int `json:"introspection_frequency"` // Default: 2

	// Performance thresholds
	ConvergenceThreshold float64 `json:"convergence_threshold"` // Default: 0.001
	MinImprovementRatio  float64 `json:"min_improvement_ratio"` // Default: 0.05

	// Concurrency and resources
	MaxGoroutines  int `json:"max_goroutines"`  // Default: 10 (for non-LLM operations)
	LLMConcurrency int `json:"llm_concurrency"` // Default: 0 (unlimited for LLM calls)

	// Strategy configuration
	StrategyMode  string  `json:"strategy_mode"`  // Default: "both" (both, instruction_only, rule_only)
	StrategyRatio float64 `json:"strategy_ratio"` // Default: 0.5 (percentage of instruction perturbation when using both)

	// Bucket sorting configuration
	UseBucketSorting      bool      `json:"use_bucket_sorting"`      // Default: false
	BucketSortingCriteria []string  `json:"bucket_sorting_criteria"` // Default: ["max_to_min_gap", "max_score", "max_to_avg_gap"]
	BucketSortingWeights  []float64 `json:"bucket_sorting_weights"`  // Default: [0.4, 0.4, 0.2]

	// Pipeline processing configuration
	UsePipelineProcessing bool `json:"use_pipeline_processing"` // Default: false
	PipelineBufferSize    int  `json:"pipeline_buffer_size"`    // Default: 2

	// Early stopping configuration
	EarlyStoppingPatience  int     `json:"early_stopping_patience"`  // Default: 0 (disabled)
	EarlyStoppingThreshold float64 `json:"early_stopping_threshold"` // Default: 0.01

	// Fast mode configuration for Python compatibility
	FastMode                       bool `json:"fast_mode"`                        // Default: false
	DisableTrajectoryTracking      bool `json:"disable_trajectory_tracking"`      // Default: false
	DisableRuleGeneration          bool `json:"disable_rule_generation"`          // Default: false
	DisableInstructionPerturbation bool `json:"disable_instruction_perturbation"` // Default: false
}

SIMBAConfig contains configuration options for SIMBA optimizer.

type SIMBAOption added in v0.29.0

type SIMBAOption func(*SIMBA)

SIMBAOption defines functional options for SIMBA configuration.

func WithBucketSorting added in v0.39.0

func WithBucketSorting(enabled bool) SIMBAOption

WithBucketSorting enables or disables bucket sorting candidate selection.

func WithBucketSortingCriteria added in v0.39.0

func WithBucketSortingCriteria(criteria []string) SIMBAOption

WithBucketSortingCriteria sets the criteria for bucket sorting.

func WithBucketSortingWeights added in v0.39.0

func WithBucketSortingWeights(weights []float64) SIMBAOption

WithBucketSortingWeights sets the weights for bucket sorting criteria.

func WithFastMode added in v0.39.0

func WithFastMode(enabled bool) SIMBAOption

WithFastMode configures SIMBA for optimal speed with minimal features.

func WithLLMConcurrency added in v0.39.0

func WithLLMConcurrency(concurrency int) SIMBAOption

WithLLMConcurrency sets the concurrency limit for LLM calls.

func WithPipelineBufferSize added in v0.39.0

func WithPipelineBufferSize(size int) SIMBAOption

WithPipelineBufferSize sets the buffer size for pipeline channels.

func WithPipelineProcessing added in v0.39.0

func WithPipelineProcessing(enabled bool) SIMBAOption

WithPipelineProcessing enables or disables pipeline processing.

func WithSIMBABatchSize added in v0.29.0

func WithSIMBABatchSize(size int) SIMBAOption

WithSIMBABatchSize sets the mini-batch size.

func WithSIMBAMaxSteps added in v0.29.0

func WithSIMBAMaxSteps(steps int) SIMBAOption

WithSIMBAMaxSteps sets the maximum optimization steps.

func WithSIMBANumCandidates added in v0.29.0

func WithSIMBANumCandidates(num int) SIMBAOption

WithSIMBANumCandidates sets the number of candidate programs per iteration.

func WithSIMBAStrategyMode added in v0.39.0

func WithSIMBAStrategyMode(mode string) SIMBAOption

WithSIMBAStrategyMode sets the strategy mode (both, instruction_only, rule_only).

func WithSIMBAStrategyRatio added in v0.39.0

func WithSIMBAStrategyRatio(ratio float64) SIMBAOption

WithSIMBAStrategyRatio sets the ratio of instruction perturbation vs rule generation.

func WithSamplingTemperature added in v0.29.0

func WithSamplingTemperature(temperature float64) SIMBAOption

WithSamplingTemperature sets the sampling temperature.

type SIMBAState added in v0.29.0

type SIMBAState struct {
	CurrentStep      int
	BestScore        float64
	BestProgram      core.Program
	CandidateHistory []CandidateResult
	PerformanceLog   []StepResult
	IntrospectionLog []string
	StartTime        time.Time
	Trajectories     []Trajectory // Track execution trajectories for rule extraction
}

SIMBAState tracks optimization progress and history.

type SearchConfig added in v0.28.0

type SearchConfig struct {
	ParamSpace  map[string][]interface{}
	MaxTrials   int
	Seed        int64
	Constraints map[string]interface{}
}

SearchConfig contains configuration for search strategies.

type SearchStrategy added in v0.28.0

type SearchStrategy interface {
	SuggestParams(ctx context.Context) (map[string]interface{}, error)
	UpdateResults(params map[string]interface{}, score float64) error
	GetBestParams() (map[string]interface{}, float64)
	Initialize(config SearchConfig) error
}

SearchStrategy defines the interface for optimization search algorithms.

func NewTPEOptimizer added in v0.28.0

func NewTPEOptimizer(config TPEConfig) SearchStrategy

NewTPEOptimizer creates a new TPE optimizer instance.

type SimilarityMatcher added in v0.44.0

type SimilarityMatcher struct {
	Config *MCPOptimizerConfig `json:"config"`
	// contains filtered or unexported fields
}

SimilarityMatcher performs KNN-based context matching using embeddings.

func (*SimilarityMatcher) FindSimilarInteractions added in v0.44.0

func (sm *SimilarityMatcher) FindSimilarInteractions(ctx context.Context, targetContext string, patterns []MCPInteraction) ([]MCPInteraction, error)

FindSimilarInteractions finds the K most similar interactions to the given context.

type SimpleEmbeddingService added in v0.44.0

type SimpleEmbeddingService struct {
	// contains filtered or unexported fields
}

SimpleEmbeddingService provides a basic embedding service implementation.

func NewSimpleEmbeddingService added in v0.44.0

func NewSimpleEmbeddingService(dimensions int) *SimpleEmbeddingService

NewSimpleEmbeddingService creates a new simple embedding service.

func (*SimpleEmbeddingService) CosineSimilarity added in v0.44.0

func (s *SimpleEmbeddingService) CosineSimilarity(vec1, vec2 []float64) float64

CosineSimilarity calculates cosine similarity between two vectors.

func (*SimpleEmbeddingService) GenerateEmbedding added in v0.44.0

func (s *SimpleEmbeddingService) GenerateEmbedding(ctx context.Context, text string) ([]float64, error)

GenerateEmbedding generates a simple embedding based on text characteristics. This is a placeholder implementation - in production, use a proper embedding model.

type StepResult added in v0.29.0

type StepResult struct {
	Step            int           `json:"step"`
	BestScore       float64       `json:"best_score"`
	CandidateScores []float64     `json:"candidate_scores"`
	Temperature     float64       `json:"temperature"`
	BatchSize       int           `json:"batch_size"`
	Introspection   string        `json:"introspection,omitempty"`
	Duration        time.Duration `json:"duration"`
	Improvement     float64       `json:"improvement"`
}

StepResult captures metrics for each optimization step.

type StrategyType added in v0.39.0

type StrategyType string

StrategyType defines the optimization strategy type.

const (
	// InstructionPerturbation is the original strategy that modifies instructions.
	InstructionPerturbation StrategyType = "instruction_perturbation"
	// RuleGeneration is the new strategy that generates rules from trajectories.
	RuleGeneration StrategyType = "rule_generation"
)

type TPEConfig added in v0.28.0

type TPEConfig struct {
	// Gamma is the percentile split between good and bad observations (default: 0.25)
	Gamma float64
	// Seed is used for random number generation
	Seed int64
	// NumEIGenerations is the number of random points to evaluate EI on
	NumEIGenerations int
	// Prior distributions for each parameter (optional)
	PriorWeight float64
	// Kernel bandwidth factor
	BandwidthFactor float64
}

TPEConfig contains configuration for Tree-structured Parzen Estimators.

type TPEOptimizer added in v0.28.0

type TPEOptimizer struct {
	// contains filtered or unexported fields
}

TPEOptimizer implements the Tree-structured Parzen Estimator for Bayesian optimization.

func (*TPEOptimizer) GetBestParams added in v0.28.0

func (t *TPEOptimizer) GetBestParams() (map[string]interface{}, float64)

GetBestParams returns the best parameters found so far and their score.

func (*TPEOptimizer) Initialize added in v0.28.0

func (t *TPEOptimizer) Initialize(config SearchConfig) error

Initialize sets up the search space and constraints.

func (*TPEOptimizer) SuggestParams added in v0.28.0

func (t *TPEOptimizer) SuggestParams(ctx context.Context) (map[string]interface{}, error)

SuggestParams suggests the next set of parameters to try.

func (*TPEOptimizer) UpdateResults added in v0.28.0

func (t *TPEOptimizer) UpdateResults(params map[string]interface{}, score float64) error

UpdateResults updates the internal state with the results of the last trial.

type TeacherStudentOptimizer added in v0.28.0

type TeacherStudentOptimizer struct {
	Teacher         core.LLM
	Student         core.LLM
	TeacherSettings map[string]interface{}
	MaxExamples     int
	// contains filtered or unexported fields
}

TeacherStudentOptimizer handles the teacher-student learning dynamic.

func (*TeacherStudentOptimizer) GenerateDemonstration added in v0.28.0

func (t *TeacherStudentOptimizer) GenerateDemonstration(ctx context.Context, input core.Example) (core.Example, error)

GenerateDemonstration creates a high-quality demonstration using the teacher.

func (*TeacherStudentOptimizer) Initialize added in v0.28.0

func (t *TeacherStudentOptimizer) Initialize(ctx context.Context, program core.Program, dataset core.Dataset) error

Initialize sets up the teacher-student optimization.

type ToolOrchestrator added in v0.44.0

type ToolOrchestrator struct {
	Dependencies map[string][]string `json:"dependencies"` // Tool dependency mapping
	Workflows    []ToolWorkflow      `json:"workflows"`    // Recorded successful workflows
	Config       *MCPOptimizerConfig `json:"config"`
	// contains filtered or unexported fields
}

ToolOrchestrator optimizes multi-tool workflows and dependencies.

func (*ToolOrchestrator) GetDependencies added in v0.44.0

func (to *ToolOrchestrator) GetDependencies(toolName string) []string

GetDependencies returns the dependencies for a given tool.

func (*ToolOrchestrator) GetOptimalToolSequence added in v0.44.0

func (to *ToolOrchestrator) GetOptimalToolSequence(ctx context.Context, context string, availableTools []string) ([]string, error)

GetOptimalToolSequence suggests the optimal sequence of tools for a given context.

func (*ToolOrchestrator) GetWorkflowCount added in v0.44.0

func (to *ToolOrchestrator) GetWorkflowCount() int

GetWorkflowCount returns the total number of recorded workflows.

func (*ToolOrchestrator) RecordWorkflow added in v0.44.0

func (to *ToolOrchestrator) RecordWorkflow(ctx context.Context, workflow ToolWorkflow) error

RecordWorkflow records a successful multi-tool workflow.

type ToolWorkflow added in v0.44.0

type ToolWorkflow struct {
	ID        string                 `json:"id"`
	Steps     []WorkflowStep         `json:"steps"`
	Context   string                 `json:"context"`
	Success   bool                   `json:"success"`
	Timestamp time.Time              `json:"timestamp"`
	Metadata  map[string]interface{} `json:"metadata"`
}

ToolWorkflow represents a sequence of tool calls that achieved a successful outcome.

type Trajectory added in v0.39.0

type Trajectory struct {
	Example       core.Example
	Prediction    map[string]interface{}
	Score         float64
	Success       bool
	ProgramID     string // To track which program generated this
	ExecutionTime time.Duration
}

Trajectory represents an execution trajectory for rule extraction.

type WorkflowStep added in v0.44.0

type WorkflowStep struct {
	ToolName   string                 `json:"tool_name"`
	Parameters map[string]interface{} `json:"parameters"`
	Result     core.ToolResult        `json:"result"`
	Order      int                    `json:"order"`
	Duration   time.Duration          `json:"duration"`
}

WorkflowStep represents a single step in a tool workflow.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL