Documentation
¶
Overview ¶
Package evolution provides autonomous code evolution capabilities through iterative improvement cycles with safety checks and version control integration.
Architecture ¶
The evolution system is the core self-improvement engine of iterate. It operates in phases: plan, code, review, and merge. Each phase uses specialized tools and agents to safely modify the codebase.
Phase Overview ¶
Plan Phase: Analyzes the codebase, identifies bugs/issues, and creates a SESSION_PLAN.md with tasks. Uses read-only tools. See: Engine.RunPlanPhase(), BuildSystemPromptAider()
Code Phase: Executes tasks from the plan, generating unified diffs that fix bugs or add features. Supports parallel task execution via git worktrees to avoid conflicts. See: Engine.RunCodePhase(), ApplyUnifiedDiffs()
Review Phase: Validates changes through build, test, and vet checks. Rejects changes that break the codebase. See: Engine.RunReviewPhase(), verifyProtected()
Merge Phase: Commits and optionally creates pull requests with detailed descriptions of changes made. See: phases_pr.go, buildPRBody()
Diff Application ¶
The system uses unified diff format (git-style) for code changes, which is 3x more effective than custom formats. Three fallback strategies ensure robust application:
- Strategy 1: Exact match with context lines
- Strategy 2: Normalized whitespace matching
- Strategy 3: Block-level replacement without context
See: prompts_aider.go (ApplyUnifiedDiffs, applyDiffStrategy1-3)
Safety Mechanisms ¶
- Sandboxed execution: Commands validated against allowlists and blocked patterns (sandbox.go)
- Protected files: Git-related and config files cannot be modified (safety.go)
- Protected directories: vendor, node_modules, .git are excluded
- Verification gates: go build, go vet, go test run after changes (verify.go)
- Git snapshots: Automatic backup before changes (snapshot.go)
Multi-Agent Architecture ¶
Specialized agent types handle different aspects:
- Plan Agent: Analyzes codebase, creates plans (read-only tools)
- Build Agent: Implements code changes (full tool access)
- Review Agent: Validates changes (read + test tools)
- Test Agent: Writes and runs tests
See: agents.go for AgentType, AgentConfig
Provider Management ¶
Supports multiple AI model providers with automatic fallback on rate limits or failures.
See: provider_pool.go
Memory & Learning ¶
The system maintains memory of:
- Active learnings (memory/ACTIVE_LEARNINGS.md)
- Evolution journal (internal/evolution/journal.go)
- Failure patterns (failure_learning.go)
- Historical evolution data (memory.go)
Test-Driven Development ¶
TDD enforcement ensures code changes include tests:
- Write failing test → Write code fix → Verify test passes
See: tdd.go
MCP Integration ¶
Model Context Protocol support for external tool integration.
See: mcp.go
Repository Mapping ¶
Builds comprehensive maps of files, functions, and imports for better context during planning.
See: repo_map.go
Index ¶
- Variables
- func BuildRetryPrompt(attempt int, previousOutput string) string
- func BuildRetryPromptAider(attempt int, previousOutput string) string
- func BuildSystemPromptAider(repoPath, identity string) string
- func BuildUserMessageAider(repoPath, journal, issues string, mode string) string
- func CountUnifiedDiffs(output string) int
- func CreateMCPConfigFile(repoPath string, servers []MCPServerConfig) error
- func DetectUnifiedDiffs(output string) bool
- func FindMCPConfigFiles(repoPath string) []string
- func ValidateUnifiedDiffs(diffs []UnifiedDiff) []string
- type AgentConfig
- type AgentResult
- type AgentType
- type CodeAnalysis
- type CoverageConfig
- type DiffHunk
- type DryRunSandbox
- type Engine
- func (e *Engine) AnalyzeFailures() ([]*FailurePattern, error)
- func (e *Engine) ApplyUnifiedDiffs(diffs []UnifiedDiff) ([]string, error)
- func (e *Engine) BuildRepoMap() (*RepoMap, error)
- func (e *Engine) CallMCPTool(ctx context.Context, server, tool string, args map[string]interface{}) (string, error)
- func (e *Engine) CreateSnapshot(ctx context.Context, message string) (*Snapshot, error)
- func (e *Engine) GetAvoidanceGuidelines() string
- func (e *Engine) GetFailurePatternSummary() string
- func (e *Engine) GetMCPTools() []iteragent.Tool
- func (e *Engine) GetRepoMapForPrompt() (string, error)
- func (e *Engine) GetSnapshotList() []*Snapshot
- func (e *Engine) NewMultiAgent(agentType AgentType) *MultiAgentEngine
- func (e *Engine) RecordAndAnalyzeFailure(taskTitle, reason string) error
- func (e *Engine) RestoreSnapshot(ctx context.Context, snapshotID string) error
- func (e *Engine) Run(ctx context.Context, p iteragent.Provider, issues string) (*RunResult, error)
- func (e *Engine) RunCommunicatePhase(ctx context.Context, p iteragent.Provider) error
- func (e *Engine) RunImplementPhase(ctx context.Context, p iteragent.Provider) error
- func (e *Engine) RunMergePhase(ctx context.Context) error
- func (e *Engine) RunPRPhase(ctx context.Context) error
- func (e *Engine) RunPlanPhase(ctx context.Context, p iteragent.Provider, issues string) error
- func (e *Engine) RunReviewPhase(ctx context.Context, p iteragent.Provider) error
- func (e *Engine) RunSafetyChecks(ctx context.Context, prFiles []string) (bool, string, error)
- func (e *Engine) TraceID() string
- func (e *Engine) WithEventSink(sink chan<- iteragent.Event) *Engine
- func (e *Engine) WithThinking(level iteragent.ThinkingLevel) *Engine
- func (e *Engine) WriteLearningsToMemory(title, context, takeaway string) error
- type FailureAnalyzer
- type FailureEntry
- type FailurePattern
- type FileInfo
- type MCPClient
- func (m *MCPClient) CallTool(ctx context.Context, serverName, toolName string, args map[string]interface{}) (string, error)
- func (m *MCPClient) Close()
- func (m *MCPClient) GetAvailableTools() []iteragent.Tool
- func (m *MCPClient) HealthCheck() map[string]bool
- func (m *MCPClient) Init(ctx context.Context) error
- func (m *MCPClient) ListTools(serverName string) ([]MCPTool, error)
- func (m *MCPClient) LoadConfig(configPath string) error
- func (m *MCPClient) ReconnectServer(ctx context.Context, serverName string) error
- type MCPConfig
- type MCPConnection
- type MCPError
- type MCPMessage
- type MCPRequest
- type MCPServerConfig
- type MCPTool
- type MultiAgentEngine
- type PRState
- type ParallelAgent
- type ProviderConfig
- type ProviderPool
- func (p *ProviderPool) AddProvider(provider iteragent.Provider, config ProviderConfig)
- func (p *ProviderPool) GetProvider() iteragent.Provider
- func (p *ProviderPool) GetStats() map[string]ProviderStats
- func (p *ProviderPool) RecordFailure(providerName string, err error)
- func (p *ProviderPool) RecordSuccess(providerName string)
- type ProviderStats
- type ProviderWithStats
- type RepoMap
- type RunResult
- type SafetyCheck
- type Sandbox
- type SandboxConfig
- type SequentialAgent
- type Snapshot
- type SnapshotManager
- func (sm *SnapshotManager) Create(ctx context.Context, message string) (*Snapshot, error)
- func (sm *SnapshotManager) CreateBranch(ctx context.Context, branchName, baseSnapshotID string) error
- func (sm *SnapshotManager) Delete(ctx context.Context, snapshotID string) error
- func (sm *SnapshotManager) Diff(ctx context.Context, snapshotID string) (string, error)
- func (sm *SnapshotManager) DiscardUncommittedChanges(ctx context.Context) error
- func (sm *SnapshotManager) Get(snapshotID string) (*Snapshot, bool)
- func (sm *SnapshotManager) HasUncommittedChanges(ctx context.Context) (bool, error)
- func (sm *SnapshotManager) List() []*Snapshot
- func (sm *SnapshotManager) LoadSnapshots() error
- func (sm *SnapshotManager) Restore(ctx context.Context, snapshotID string) error
- func (sm *SnapshotManager) SaveSnapshotMetadata(snapshot *Snapshot) error
- type TDDEngine
- type TDDPhase
- type TDDResult
- type TDDVerification
- type UnifiedDiff
- type VerboseSandbox
- type VerificationResult
Constants ¶
This section is empty.
Variables ¶
var DefaultAgentConfigs = map[AgentType]AgentConfig{ AgentPlan: { Type: AgentPlan, Name: "planner", Description: "Analyzes codebase and creates implementation plans without making changes", Tools: []string{"glob", "grep", "read", "bash"}, MaxSteps: 20, Timeout: 5 * time.Minute, }, AgentBuild: { Type: AgentBuild, Name: "builder", Description: "Implements code changes based on plans", Tools: []string{"glob", "grep", "read", "write", "edit", "bash", "todo"}, MaxSteps: 50, Timeout: 15 * time.Minute, }, AgentReview: { Type: AgentReview, Name: "reviewer", Description: "Reviews code changes for quality and correctness", Tools: []string{"glob", "grep", "read", "bash"}, MaxSteps: 30, Timeout: 10 * time.Minute, }, AgentTest: { Type: AgentTest, Name: "tester", Description: "Writes and verifies tests", Tools: []string{"glob", "grep", "read", "write", "edit", "bash"}, MaxSteps: 40, Timeout: 10 * time.Minute, }, }
var DefaultSandboxConfig = SandboxConfig{ AllowedCommands: []string{ "go", "git", "ls", "cat", "grep", "find", "echo", "pwd", "mkdir", "rm", "cp", "mv", "chmod", "chown", "npm", "yarn", "pnpm", "pip", "python", "python3", "cargo", "rustc", "make", "cmake", "docker", "docker-compose", "curl", "wget", }, BlockedPatterns: []string{ `rm\s+-rf\s+/(?!\.)`, `dd\s+of=`, `mkfs`, `curl.*\|.*sh`, `wget.*\|.*sh`, `:(){ :|:& };:`, `fork\(\)`, `chmod\s+777`, `>\s*/dev/sd`, `curl\s+.+\s+-o\s+/etc`, `sudo\s+rm`, `git\s+push\s+--force`, `git\s+push\s+-f`, }, AllowedPaths: []string{ "/home/runner/work", "/tmp", }, Timeout: 5 * time.Minute, MaxOutputSize: 1024 * 1024, EnvWhitelist: []string{ "HOME", "PATH", "USER", "GITHUB_TOKEN", "GITHUB_REPOSITORY", "GO111MODULE", "GOPROXY", "GOSUMDB", }, }
var DefaultTDDVerification = TDDVerification{ RequireTest: true, RequireFailingTest: true, RequirePassingTest: true, }
var LanguageExtensions = map[string]string{
".go": "go",
".py": "python",
".js": "javascript",
".ts": "typescript",
".tsx": "typescript",
".jsx": "javascript",
".java": "java",
".c": "c",
".cpp": "cpp",
".h": "c",
".hpp": "cpp",
".rs": "rust",
".rb": "ruby",
".php": "php",
".cs": "csharp",
".swift": "swift",
".kt": "kotlin",
".scala": "scala",
".html": "html",
".css": "css",
".scss": "scss",
".sql": "sql",
".sh": "bash",
".yaml": "yaml",
".yml": "yaml",
".json": "json",
".md": "markdown",
}
var ProtectedFiles = []string{
"internal/evolution/engine.go",
"internal/evolution/*.go",
".github/workflows/evolve.yml",
".github/workflows/*.yml",
"cmd/iterate/repl.go",
"cmd/iterate/main.go",
".iterate/config.json",
"scripts/evolution/evolve.sh",
"scripts/social/social.sh",
}
ProtectedFiles defines patterns for files the agent MUST NOT edit during evolution. These are core infrastructure that could break the evolution system itself.
Functions ¶
func BuildRetryPrompt ¶
BuildRetryPrompt creates an escalating prompt for retry attempts
func BuildRetryPromptAider ¶
BuildRetryPromptAider creates an escalating retry prompt
func BuildSystemPromptAider ¶
BuildSystemPromptAider creates a prompt inspired by Aider's approach Uses unified diff format (like git diff) - this is 3X more effective than custom formats
func BuildUserMessageAider ¶
BuildUserMessageAider creates an aggressive, action-oriented user message
func CountUnifiedDiffs ¶
CountUnifiedDiffs returns the number of unified diff blocks
func CreateMCPConfigFile ¶
func CreateMCPConfigFile(repoPath string, servers []MCPServerConfig) error
func DetectUnifiedDiffs ¶
DetectUnifiedDiffs checks if output contains unified diffs
func FindMCPConfigFiles ¶
func ValidateUnifiedDiffs ¶
func ValidateUnifiedDiffs(diffs []UnifiedDiff) []string
ValidateUnifiedDiffs checks if diffs are well-formed - Lenient version
Types ¶
type AgentConfig ¶
type AgentResult ¶
type CodeAnalysis ¶
type CodeAnalysis struct {
TODOs []string // file:line: content
Hotspots []string // most changed files
NoTestPkgs []string // packages without test files
BuildOK bool
TestOK bool
}
CodeAnalysis holds analysis results for the codebase.
func AnalyzeCodebase ¶
func AnalyzeCodebase(repoPath string) CodeAnalysis
AnalyzeCodebase scans the repo for improvement opportunities.
func (CodeAnalysis) FormatAnalysis ¶
func (a CodeAnalysis) FormatAnalysis() string
FormatAnalysis returns a human-readable summary for the agent.
type CoverageConfig ¶
type CoverageConfig struct {
MinCoverage float64 // Minimum overall coverage percentage (0-100)
MinFileCoverage float64 // Minimum per-file coverage percentage (0-100)
MaxCoverageDrop float64 // Max allowed coverage drop from baseline
}
CoverageConfig defines coverage enforcement thresholds.
func DefaultCoverageConfig ¶
func DefaultCoverageConfig() CoverageConfig
DefaultCoverageConfig returns sensible coverage enforcement defaults.
type DryRunSandbox ¶
type Engine ¶
type Engine struct {
// contains filtered or unexported fields
}
Engine runs one evolution session.
func (*Engine) AnalyzeFailures ¶
func (e *Engine) AnalyzeFailures() ([]*FailurePattern, error)
func (*Engine) ApplyUnifiedDiffs ¶
func (e *Engine) ApplyUnifiedDiffs(diffs []UnifiedDiff) ([]string, error)
ApplyUnifiedDiffs applies unified diffs to files with flexible error recovery
func (*Engine) BuildRepoMap ¶
func (*Engine) CallMCPTool ¶
func (*Engine) CreateSnapshot ¶
func (*Engine) GetAvoidanceGuidelines ¶
func (*Engine) GetFailurePatternSummary ¶
func (*Engine) GetMCPTools ¶
func (*Engine) GetRepoMapForPrompt ¶
func (*Engine) GetSnapshotList ¶
func (*Engine) NewMultiAgent ¶
func (e *Engine) NewMultiAgent(agentType AgentType) *MultiAgentEngine
func (*Engine) RecordAndAnalyzeFailure ¶
func (*Engine) RestoreSnapshot ¶
func (*Engine) RunCommunicatePhase ¶
RunCommunicatePhase writes journal, posts issue comments, records learnings.
func (*Engine) RunImplementPhase ¶
RunImplementPhase reads SESSION_PLAN.md and executes tasks.
func (*Engine) RunMergePhase ¶
RunMergePhase merges the open PR, clears state, and returns to main.
func (*Engine) RunPRPhase ¶
RunPRPhase creates a feature branch from the current HEAD, pushes it, and opens a PR. It is designed to run after RunImplementPhase has already committed changes to main.
func (*Engine) RunPlanPhase ¶
RunPlanPhase runs the planning phase. Creates SESSION_PLAN.md via agent or fallback.
func (*Engine) RunReviewPhase ¶
RunReviewPhase runs an AI self-review of the open PR.
func (*Engine) RunSafetyChecks ¶
func (*Engine) WithEventSink ¶
WithEventSink sets a channel that receives live agent events during evolution.
func (*Engine) WithThinking ¶
func (e *Engine) WithThinking(level iteragent.ThinkingLevel) *Engine
WithThinking sets the extended thinking level for all agents spawned by this engine.
func (*Engine) WriteLearningsToMemory ¶
WriteLearningsToMemory is the public entry point for the synthesize workflow.
type FailureAnalyzer ¶
type FailureAnalyzer struct {
// contains filtered or unexported fields
}
func NewFailureAnalyzer ¶
func NewFailureAnalyzer(repoPath string, logger *slog.Logger) *FailureAnalyzer
func (*FailureAnalyzer) AnalyzeFailures ¶
func (fa *FailureAnalyzer) AnalyzeFailures() ([]*FailurePattern, error)
func (*FailureAnalyzer) GetAvoidanceGuidelines ¶
func (fa *FailureAnalyzer) GetAvoidanceGuidelines() string
func (*FailureAnalyzer) GetPatternSummary ¶
func (fa *FailureAnalyzer) GetPatternSummary() string
func (*FailureAnalyzer) SavePatterns ¶
func (fa *FailureAnalyzer) SavePatterns() error
type FailureEntry ¶
type FailurePattern ¶
type FileInfo ¶
type FileInfo struct {
Path string `json:"path"`
Name string `json:"name"`
Language string `json:"language"`
Size int64 `json:"size"`
Functions []string `json:"functions"`
Classes []string `json:"classes"`
Interfaces []string `json:"interfaces"`
Imports []string `json:"imports"`
Exports []string `json:"exports"`
}
type MCPClient ¶
type MCPClient struct {
// contains filtered or unexported fields
}
func (*MCPClient) GetAvailableTools ¶
func (*MCPClient) HealthCheck ¶
func (*MCPClient) LoadConfig ¶
type MCPConfig ¶
type MCPConfig struct {
Enabled bool `json:"enabled"`
Servers []MCPServerConfig `json:"servers"`
Timeout time.Duration `json:"timeout"`
RetryAttempts int `json:"retry_attempts"`
}
func DefaultMCPConfig ¶
func DefaultMCPConfig() MCPConfig
type MCPConnection ¶
type MCPConnection struct {
// contains filtered or unexported fields
}
type MCPMessage ¶
type MCPMessage struct {
JSONRPC string `json:"jsonrpc"`
ID int `json:"id,omitempty"`
Method string `json:"method,omitempty"`
Params json.RawMessage `json:"params,omitempty"`
Result json.RawMessage `json:"result,omitempty"`
Error *MCPError `json:"error,omitempty"`
}
type MCPRequest ¶
type MCPServerConfig ¶
type MCPServerConfig struct {
Name string `json:"name"`
Command string `json:"command"`
Args []string `json:"args"`
Env map[string]string `json:"env"`
Transport string `json:"transport"` // stdio, http
URL string `json:"url"`
AutoConnect bool `json:"auto_connect"`
}
func GetCommonMCPServers ¶
func GetCommonMCPServers() []MCPServerConfig
type MultiAgentEngine ¶
type MultiAgentEngine struct {
// contains filtered or unexported fields
}
func (*MultiAgentEngine) Execute ¶
func (ma *MultiAgentEngine) Execute(ctx context.Context, p iteragent.Provider, task string) *AgentResult
type PRState ¶
type PRState struct {
PRNumber int `json:"pr_number"`
PRURL string `json:"pr_url"`
Branch string `json:"branch"`
}
PRState persists PR info between phases (evolve.sh runs phases as separate CLI invocations).
type ParallelAgent ¶
type ParallelAgent struct {
// contains filtered or unexported fields
}
func NewParallelAgent ¶
func NewParallelAgent(e *Engine, agentTypes ...AgentType) *ParallelAgent
func (*ParallelAgent) Execute ¶
func (pa *ParallelAgent) Execute(ctx context.Context, p iteragent.Provider, tasks []string) []*AgentResult
type ProviderConfig ¶
type ProviderPool ¶
type ProviderPool struct {
// contains filtered or unexported fields
}
func CreateProviderPoolForEngine ¶
func CreateProviderPoolForEngine(logger *slog.Logger, modelName string) *ProviderPool
func NewProviderPool ¶
func NewProviderPool(logger *slog.Logger) *ProviderPool
func (*ProviderPool) AddProvider ¶
func (p *ProviderPool) AddProvider(provider iteragent.Provider, config ProviderConfig)
func (*ProviderPool) GetProvider ¶
func (p *ProviderPool) GetProvider() iteragent.Provider
func (*ProviderPool) GetStats ¶
func (p *ProviderPool) GetStats() map[string]ProviderStats
func (*ProviderPool) RecordFailure ¶
func (p *ProviderPool) RecordFailure(providerName string, err error)
func (*ProviderPool) RecordSuccess ¶
func (p *ProviderPool) RecordSuccess(providerName string)
type ProviderStats ¶
type ProviderWithStats ¶
type ProviderWithStats struct {
iteragent.Provider
Config ProviderConfig
Stats ProviderStats
}
type RepoMap ¶
type RepoMap struct {
RootPath string `json:"root_path"`
Files map[string]*FileInfo `json:"files"`
Functions map[string][]string `json:"functions"`
Classes map[string][]string `json:"classes"`
Imports map[string][]string `json:"imports"`
DepGraph map[string][]string `json:"dependency_graph"`
LastUpdate string `json:"last_update"`
// contains filtered or unexported fields
}
func LoadRepoMap ¶
func NewRepoMap ¶
func (*RepoMap) FindFilesByKeyword ¶
func (*RepoMap) FindRelatedFiles ¶
type RunResult ¶
type RunResult struct {
Status string
StartedAt time.Time
FinishedAt time.Time
PRNumber int
PRURL string
}
RunResult holds the outcome of a completed evolution run.
type SafetyCheck ¶
type SafetyCheck struct {
LintCheck bool
TestModificationCheck bool
SmokeTestCheck bool
RequireHumanReview bool
}
func DefaultSafetyCheck ¶
func DefaultSafetyCheck() *SafetyCheck
type Sandbox ¶
type Sandbox struct {
// contains filtered or unexported fields
}
func NewSandbox ¶
type SandboxConfig ¶
type SequentialAgent ¶
type SequentialAgent struct {
// contains filtered or unexported fields
}
func NewSequentialAgent ¶
func NewSequentialAgent(e *Engine, agentTypes ...AgentType) *SequentialAgent
func (*SequentialAgent) Execute ¶
func (sa *SequentialAgent) Execute(ctx context.Context, p iteragent.Provider, task string) *AgentResult
type SnapshotManager ¶
type SnapshotManager struct {
// contains filtered or unexported fields
}
func NewSnapshotManager ¶
func NewSnapshotManager(repoPath string, logger *slog.Logger) *SnapshotManager
func (*SnapshotManager) CreateBranch ¶
func (sm *SnapshotManager) CreateBranch(ctx context.Context, branchName, baseSnapshotID string) error
func (*SnapshotManager) Delete ¶
func (sm *SnapshotManager) Delete(ctx context.Context, snapshotID string) error
func (*SnapshotManager) DiscardUncommittedChanges ¶
func (sm *SnapshotManager) DiscardUncommittedChanges(ctx context.Context) error
func (*SnapshotManager) HasUncommittedChanges ¶
func (sm *SnapshotManager) HasUncommittedChanges(ctx context.Context) (bool, error)
func (*SnapshotManager) List ¶
func (sm *SnapshotManager) List() []*Snapshot
func (*SnapshotManager) LoadSnapshots ¶
func (sm *SnapshotManager) LoadSnapshots() error
func (*SnapshotManager) Restore ¶
func (sm *SnapshotManager) Restore(ctx context.Context, snapshotID string) error
func (*SnapshotManager) SaveSnapshotMetadata ¶
func (sm *SnapshotManager) SaveSnapshotMetadata(snapshot *Snapshot) error
type TDDEngine ¶
type TDDEngine struct {
// contains filtered or unexported fields
}
func NewTDDEngine ¶
func (*TDDEngine) ExecuteTask ¶
type TDDVerification ¶
type UnifiedDiff ¶
UnifiedDiff represents a parsed unified diff
func ParseUnifiedDiffs ¶
func ParseUnifiedDiffs(output string) []UnifiedDiff
ParseUnifiedDiffs extracts unified diffs from LLM output
type VerboseSandbox ¶
type VerificationResult ¶
type VerificationResult struct {
BuildPassed bool
TestPassed bool
VetPassed bool
Coverage float64
CoverageOK bool
CoveredFiles int
TotalFiles int
Output string
Error error
Violations []string
Duration time.Duration
}
VerificationResult holds the outcome of a build + test verification run.