tdg

package
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 11, 2026 License: AGPL-3.0 Imports: 20 Imported by: 0

Documentation

Overview

Package tdg provides Test-Driven Generation (TDG) for Trace.

TDG is an agent mode that forces correctness proofs through tests:

  1. UNDERSTAND - Agent analyzes the bug/feature request
  2. WRITE_TEST - Agent writes a test that should FAIL on current code
  3. VERIFY_FAIL - System runs test, must fail (proves bug exists)
  4. WRITE_FIX - Agent implements the fix
  5. VERIFY_PASS - System runs test, must pass (proves fix works)
  6. REGRESSION - System runs full suite (proves no breakage)
  7. DONE - Fix is proven correct

The TDG controller operates as an internal state machine, separate from the main agent loop states. It runs within the agent's EXECUTE phase when TDG mode is requested.

Multi-Language Support

TDG supports multiple languages through a configuration registry:

  • Go: go test -v -run {name}
  • Python: pytest -v -k {name}
  • TypeScript: npx jest --testNamePattern {name}

Iteration Limits

To prevent infinite loops, TDG enforces retry limits:

  • Max test generation attempts: 3
  • Max fix attempts: 5
  • Max regression fix attempts: 3

When limits are exceeded, TDG returns an error for user escalation.

Thread Safety

Controller instances are NOT safe for concurrent use. Each TDG session should use its own Controller instance. The LanguageConfigRegistry is safe for concurrent reads after initialization.

Example Usage

cfg := tdg.DefaultConfig()
runner := tdg.NewTestRunner(cfg, logger)
files := tdg.NewFileManager(projectRoot, logger)
gen := tdg.NewTestGenerator(llm, contextAsm, logger)

controller := tdg.NewController(cfg, runner, files, gen, logger)

result, err := controller.Run(ctx, &tdg.Request{
    BugDescription: "ValidateToken crashes when claims is nil",
    ProjectRoot:    "/path/to/project",
    Language:       "go",
})

if result.Success {
    fmt.Printf("Fixed! Test: %s\n", result.ReproducerTest.Name)
}

Index

Constants

This section is empty.

Variables

View Source
var (
	// ErrTestTimeout indicates test execution exceeded the timeout.
	ErrTestTimeout = errors.New("test execution timeout")

	// ErrSuiteTimeout indicates the full test suite exceeded the timeout.
	ErrSuiteTimeout = errors.New("test suite timeout")

	// ErrTotalTimeout indicates the entire TDG session exceeded the timeout.
	ErrTotalTimeout = errors.New("TDG session timeout")

	// ErrMaxTestRetries indicates max test generation retries exceeded.
	// This happens when the generated test keeps passing when it should fail.
	ErrMaxTestRetries = errors.New("max test generation retries exceeded")

	// ErrMaxFixRetries indicates max fix attempts exceeded.
	// This happens when the generated fix keeps failing the test.
	ErrMaxFixRetries = errors.New("max fix attempts exceeded")

	// ErrMaxRegressionFixes indicates max regression fix attempts exceeded.
	// This happens when fixing regressions keeps breaking other tests.
	ErrMaxRegressionFixes = errors.New("max regression fix attempts exceeded")

	// ErrRegressionDetected indicates the fix caused existing tests to fail.
	ErrRegressionDetected = errors.New("fix caused regression in existing tests")

	// ErrUnsupportedLanguage indicates no test configuration for the language.
	ErrUnsupportedLanguage = errors.New("no test configuration for language")

	// ErrTestWriteFailed indicates failure to write the test file to disk.
	ErrTestWriteFailed = errors.New("failed to write test file")

	// ErrPatchApplyFailed indicates failure to apply the code patch.
	ErrPatchApplyFailed = errors.New("failed to apply patch")

	// ErrPatchRollbackFailed indicates failure to rollback a patch.
	ErrPatchRollbackFailed = errors.New("failed to rollback patch")

	// ErrTestPassedUnexpectedly indicates the reproducer test passed when
	// it should have failed, meaning it doesn't actually reproduce the bug.
	ErrTestPassedUnexpectedly = errors.New("reproducer test passed unexpectedly")

	// ErrTestFailedUnexpectedly indicates the test failed after the fix
	// was applied, meaning the fix doesn't work.
	ErrTestFailedUnexpectedly = errors.New("test failed after fix applied")

	// ErrInvalidTestCase indicates the test case is malformed or incomplete.
	ErrInvalidTestCase = errors.New("invalid test case")

	// ErrInvalidPatch indicates the patch is malformed or incomplete.
	ErrInvalidPatch = errors.New("invalid patch")

	// ErrLLMGenerationFailed indicates the LLM failed to generate content.
	ErrLLMGenerationFailed = errors.New("LLM generation failed")

	// ErrContextAssemblyFailed indicates failure to assemble code context.
	ErrContextAssemblyFailed = errors.New("context assembly failed")

	// ErrNilContext indicates a nil context.Context was passed.
	ErrNilContext = errors.New("context must not be nil")

	// ErrEmptyRequest indicates an empty or invalid request.
	ErrEmptyRequest = errors.New("request must not be empty")

	// ErrAlreadyRunning indicates the controller is already running.
	ErrAlreadyRunning = errors.New("TDG controller already running")

	// ErrNotRunning indicates the controller is not running.
	ErrNotRunning = errors.New("TDG controller not running")
)
View Source
var DefaultLanguageConfigs = NewLanguageConfigRegistry()

DefaultLanguageConfigs is the shared language config registry.

Functions

func CountTestResults

func CountTestResults(output string, passPattern, failPattern *regexp.Regexp) (passed, failed int)

CountTestResults counts passed and failed tests from output.

Description:

A generic helper to count test results using patterns.

Inputs:

output - Test output string
passPattern - Pattern matching passed tests
failPattern - Pattern matching failed tests

Outputs:

passed - Number of passed tests
failed - Number of failed tests

func ExtractTestNames

func ExtractTestNames(output string, pattern *regexp.Regexp) []string

ExtractTestNames extracts test function names from test output.

Description:

A generic helper to extract test names that match common patterns.
Used when language-specific parsers need additional extraction.

Inputs:

output - Test output string
pattern - Regex pattern with capture group for test name

Outputs:

[]string - Extracted test names

func LanguageForFile

func LanguageForFile(filePath string) string

LanguageForFile is a convenience function using the default registry.

func RegisterTestOutputParser

func RegisterTestOutputParser(language string, parser TestOutputParser)

RegisterTestOutputParser registers a custom parser for a language.

Inputs:

language - The language identifier
parser - The parser function

Thread Safety: Safe for concurrent use.

Types

type Config

type Config struct {
	// MaxTestRetries is the maximum test generation attempts.
	// When exceeded, TDG returns ErrMaxTestRetries.
	// Default: 3
	MaxTestRetries int

	// MaxFixRetries is the maximum fix generation attempts.
	// When exceeded, TDG returns ErrMaxFixRetries.
	// Default: 5
	MaxFixRetries int

	// MaxRegressionFixes is the maximum regression fix attempts.
	// When exceeded, TDG returns ErrMaxRegressionFixes.
	// Default: 3
	MaxRegressionFixes int

	// TestTimeout is the timeout for a single test execution.
	// Default: 30s
	TestTimeout time.Duration

	// SuiteTimeout is the timeout for running the full test suite.
	// Default: 5m
	SuiteTimeout time.Duration

	// TotalTimeout is the timeout for the entire TDG session.
	// Default: 10m
	TotalTimeout time.Duration

	// MaxOutputBytes is the maximum test output to capture.
	// Output beyond this is truncated.
	// Default: 65536 (64KB)
	MaxOutputBytes int

	// EnableLintCheck enables lint checking on generated code (CB-25).
	// Default: true
	EnableLintCheck bool

	// WorkingDir overrides the working directory for test execution.
	// If empty, uses the project root from the request.
	WorkingDir string
}

Config holds configuration for a TDG session.

func DefaultConfig

func DefaultConfig() *Config

DefaultConfig returns a Config with sensible defaults.

Outputs:

*Config - Configuration with default values

func NewConfig

func NewConfig(opts ...Option) *Config

NewConfig creates a Config with the given options applied.

Inputs:

opts - Options to apply to the default config

Outputs:

*Config - Configuration with options applied

func (*Config) Validate

func (c *Config) Validate() error

Validate checks that the configuration is valid.

Outputs:

error - Non-nil if configuration is invalid

type Context

type Context struct {
	// SessionID uniquely identifies this TDG session.
	SessionID string

	// State is the current TDG state.
	State State

	// Request is the original request.
	Request *Request

	// ReproducerTest is the generated reproducer test.
	ReproducerTest *TestCase

	// ProposedPatches are the current proposed code changes.
	ProposedPatches []*Patch

	// AppliedPatches are patches that have been applied to disk.
	AppliedPatches []*Patch

	// TestRetries is the number of test generation attempts.
	TestRetries int

	// FixRetries is the number of fix generation attempts.
	FixRetries int

	// RegressionFixes is the number of regression fix attempts.
	RegressionFixes int

	// LastTestOutput is the output from the last test execution.
	LastTestOutput string

	// LastError is the last error encountered.
	LastError error

	// StartTime is when the TDG session started (Unix milliseconds UTC).
	StartTime int64

	// Metrics tracks execution metrics.
	Metrics *Metrics
}

Context holds the state during a TDG session.

This is the internal working state, not to be confused with context.Context.

func NewContext

func NewContext(sessionID string, req *Request) *Context

NewContext creates a new TDG context for a request.

func (*Context) Elapsed

func (c *Context) Elapsed() time.Duration

Elapsed returns the time since the session started.

type ContextAssembler

type ContextAssembler interface {
	// AssembleContext gathers relevant code for a query.
	AssembleContext(ctx context.Context, query string, tokenBudget int) (string, error)
}

ContextAssembler assembles code context for prompts.

type Controller

type Controller struct {
	// contains filtered or unexported fields
}

Controller orchestrates the TDG loop.

Thread Safety: NOT safe for concurrent use. Each TDG session should have its own Controller instance.

func NewController

func NewController(cfg *Config, runner *TestRunner, files *FileManager, gen *TestGenerator, logger *slog.Logger) *Controller

NewController creates a new TDG controller.

Inputs:

cfg - TDG configuration
runner - Test runner for executing tests
files - File manager for test/patch files
gen - Test generator for LLM interactions
logger - Logger for structured logging

Outputs:

*Controller - Configured controller

func (*Controller) GetContext

func (c *Controller) GetContext() *Context

GetContext returns the current TDG context (for inspection/debugging).

func (*Controller) IsRunning

func (c *Controller) IsRunning() bool

IsRunning returns true if the controller is currently running.

func (*Controller) Run

func (c *Controller) Run(ctx context.Context, req *Request) (*Result, error)

Run executes the full TDG loop.

Description:

Orchestrates the TDG state machine:
  1. Generate reproducer test
  2. Verify test fails
  3. Generate fix
  4. Verify test passes
  5. Check for regressions

Handles retries, timeouts, and rollback on failure.

Inputs:

ctx - Context for cancellation
req - The TDG request

Outputs:

*Result - TDG execution result
error - Non-nil on unrecoverable failure

type FileManager

type FileManager struct {
	// contains filtered or unexported fields
}

FileManager handles file operations for TDG including writing tests, applying patches, and rollback functionality.

Thread Safety: NOT safe for concurrent use on the same files. Each TDG session should have its own FileManager.

func NewFileManager

func NewFileManager(projectRoot string, logger *slog.Logger) *FileManager

NewFileManager creates a new file manager.

Inputs:

projectRoot - Root directory of the project
logger - Logger for structured logging

Outputs:

*FileManager - Configured file manager

func (*FileManager) ApplyPatch

func (m *FileManager) ApplyPatch(patch *Patch) error

ApplyPatch applies a code change and stores backup for rollback.

Description:

Reads the current file content (for rollback), then writes the new
content. Updates the patch with old content if not already set.
Uses atomic write for safety.

Inputs:

patch - The patch to apply

Outputs:

error - Non-nil on apply failure

Thread Safety: Uses internal locking.

func (*FileManager) BackupCount

func (m *FileManager) BackupCount() int

BackupCount returns the number of backed-up files.

Thread Safety: Safe for concurrent use.

func (*FileManager) Cleanup

func (m *FileManager) Cleanup() error

Cleanup removes test files that were created during TDG.

Description:

Similar to Rollback but only removes newly created files,
leaves modified files with their new content. Used after
successful TDG completion if test files shouldn't be kept.

Outputs:

error - Non-nil if any removal failed

Thread Safety: Uses internal locking.

func (*FileManager) CreatedCount

func (m *FileManager) CreatedCount() int

CreatedCount returns the number of created files.

Thread Safety: Safe for concurrent use.

func (*FileManager) HasBackups

func (m *FileManager) HasBackups() bool

HasBackups returns true if there are any backed-up files.

Thread Safety: Safe for concurrent use.

func (*FileManager) Rollback

func (m *FileManager) Rollback() error

Rollback restores all backed-up files to their original state.

Description:

Restores files that were modified by patches back to their original
content. Removes files that were created by TDG. Should be called
when TDG fails or is cancelled.

Outputs:

error - Non-nil if any rollback failed

Thread Safety: Uses internal locking.

func (*FileManager) WriteTest

func (m *FileManager) WriteTest(tc *TestCase) error

WriteTest writes a test file to disk.

Description:

Writes the test case content to the specified file path.
Creates parent directories if needed. Tracks the file for cleanup.
Uses atomic write (temp file + rename) for safety.

Inputs:

tc - The test case with file path and content

Outputs:

error - Non-nil on write failure

Thread Safety: Uses internal locking.

type HistoryEntry

type HistoryEntry struct {
	// Timestamp is when this entry was recorded (Unix milliseconds UTC).
	Timestamp int64 `json:"timestamp"`

	// State is the TDG state at this step.
	State State `json:"state"`

	// Action describes what happened.
	Action string `json:"action"`

	// Details contains additional information.
	Details string `json:"details,omitempty"`

	// Duration is how long this step took.
	Duration time.Duration `json:"duration,omitempty"`

	// Error contains any error from this step.
	Error string `json:"error,omitempty"`
}

HistoryEntry records a step in the TDG session.

type LLMClient

type LLMClient interface {
	// Generate produces text from a prompt.
	Generate(ctx context.Context, prompt string) (string, error)

	// GenerateWithSystem produces text with a system prompt.
	GenerateWithSystem(ctx context.Context, system, prompt string) (string, error)
}

LLMClient defines the interface for LLM interactions.

type LanguageConfig

type LanguageConfig struct {
	// Language is the language identifier (e.g., "go", "python").
	Language string

	// TestCommand is the test runner command (e.g., "go", "pytest").
	TestCommand string

	// TestArgs are arguments for running a single test.
	// Use {name} as placeholder for test name, {file} for file path.
	TestArgs []string

	// SuiteArgs are arguments for running the full test suite.
	// Use {package} as placeholder for package/directory path.
	SuiteArgs []string

	// TestFilePattern is the glob pattern for test files.
	TestFilePattern string

	// TestNameFlag is the flag for specifying test name (e.g., "-run").
	TestNameFlag string

	// Extensions are file extensions for this language.
	Extensions []string
}

LanguageConfig defines test execution settings for a language.

func GetLanguageConfig

func GetLanguageConfig(language string) (*LanguageConfig, bool)

GetLanguageConfig is a convenience function using the default registry.

type LanguageConfigRegistry

type LanguageConfigRegistry struct {
	// contains filtered or unexported fields
}

LanguageConfigRegistry manages test configurations for different languages.

Thread Safety: Safe for concurrent reads after initialization. Register operations should only be done during setup.

func NewLanguageConfigRegistry

func NewLanguageConfigRegistry() *LanguageConfigRegistry

NewLanguageConfigRegistry creates a registry with default configurations.

Outputs:

*LanguageConfigRegistry - Registry with Go, Python, TypeScript configs

func (*LanguageConfigRegistry) Get

func (r *LanguageConfigRegistry) Get(language string) (*LanguageConfig, bool)

Get returns the configuration for a language.

Inputs:

language - The language identifier

Outputs:

*LanguageConfig - The configuration, or nil if not found
bool - True if configuration exists

Thread Safety: Safe for concurrent use.

func (*LanguageConfigRegistry) GetByExtension

func (r *LanguageConfigRegistry) GetByExtension(ext string) (*LanguageConfig, bool)

GetByExtension returns the configuration for a file extension.

Inputs:

ext - The file extension including dot (e.g., ".go")

Outputs:

*LanguageConfig - The configuration, or nil if not found
bool - True if configuration exists

Thread Safety: Safe for concurrent use.

func (*LanguageConfigRegistry) LanguageForFile

func (r *LanguageConfigRegistry) LanguageForFile(filePath string) string

LanguageForFile returns the language for a file path.

Inputs:

filePath - The file path

Outputs:

string - The language identifier, or empty if unknown

func (*LanguageConfigRegistry) Languages

func (r *LanguageConfigRegistry) Languages() []string

Languages returns all registered language names.

Outputs:

[]string - Slice of language names

Thread Safety: Safe for concurrent use.

func (*LanguageConfigRegistry) Register

func (r *LanguageConfigRegistry) Register(cfg *LanguageConfig)

Register adds or updates a language configuration.

Inputs:

cfg - The configuration to register

Thread Safety: Safe for concurrent use, but should only be called during setup.

type Metrics

type Metrics struct {
	// TestAttempts is the number of test generation attempts.
	TestAttempts int `json:"test_attempts"`

	// FixAttempts is the number of fix generation attempts.
	FixAttempts int `json:"fix_attempts"`

	// RegressionFixes is the number of regression fix attempts.
	RegressionFixes int `json:"regression_fixes"`

	// TestsRun is the total number of test executions.
	TestsRun int `json:"tests_run"`

	// LLMCalls is the number of LLM API calls.
	LLMCalls int `json:"llm_calls"`

	// TotalTestDuration is the cumulative time spent running tests.
	TotalTestDuration time.Duration `json:"total_test_duration"`

	// ContextTokens is the total context tokens assembled.
	ContextTokens int `json:"context_tokens"`
}

Metrics tracks execution statistics for a TDG session.

type Option

type Option func(*Config)

Option is a function that modifies Config.

func WithLintCheck

func WithLintCheck(enabled bool) Option

WithLintCheck enables or disables lint checking.

func WithMaxFixRetries

func WithMaxFixRetries(n int) Option

WithMaxFixRetries sets the maximum fix generation retries.

func WithMaxOutputBytes

func WithMaxOutputBytes(n int) Option

WithMaxOutputBytes sets the maximum test output capture size.

func WithMaxRegressionFixes

func WithMaxRegressionFixes(n int) Option

WithMaxRegressionFixes sets the maximum regression fix retries.

func WithMaxTestRetries

func WithMaxTestRetries(n int) Option

WithMaxTestRetries sets the maximum test generation retries.

func WithSuiteTimeout

func WithSuiteTimeout(d time.Duration) Option

WithSuiteTimeout sets the test suite timeout.

func WithTestTimeout

func WithTestTimeout(d time.Duration) Option

WithTestTimeout sets the per-test timeout.

func WithTotalTimeout

func WithTotalTimeout(d time.Duration) Option

WithTotalTimeout sets the total TDG session timeout.

func WithWorkingDir

func WithWorkingDir(dir string) Option

WithWorkingDir sets the working directory for test execution.

type Patch

type Patch struct {
	// FilePath is the file to modify.
	FilePath string `json:"file_path"`

	// OldContent is the original file content (for rollback).
	OldContent string `json:"old_content,omitempty"`

	// NewContent is the new file content.
	NewContent string `json:"new_content"`

	// Applied indicates if the patch has been applied to disk.
	Applied bool `json:"applied"`
}

Patch represents a code change to apply.

func (*Patch) Validate

func (p *Patch) Validate() error

Validate checks that the patch is complete.

type Request

type Request struct {
	// BugDescription describes the bug to fix or feature to implement.
	BugDescription string `json:"bug_description"`

	// ProjectRoot is the root directory of the project.
	ProjectRoot string `json:"project_root"`

	// Language is the programming language (go, python, typescript).
	Language string `json:"language"`

	// GraphID is an optional existing Trace graph ID.
	GraphID string `json:"graph_id,omitempty"`

	// TargetFile is an optional hint for which file contains the bug.
	TargetFile string `json:"target_file,omitempty"`

	// TargetFunction is an optional hint for which function has the bug.
	TargetFunction string `json:"target_function,omitempty"`
}

Request contains the input for a TDG session.

func (*Request) Validate

func (r *Request) Validate() error

Validate checks that the request has required fields.

type Result

type Result struct {
	// Success indicates if TDG completed successfully.
	Success bool `json:"success"`

	// State is the final TDG state.
	State State `json:"final_state"`

	// ReproducerTest is the generated test that reproduces the bug.
	ReproducerTest *TestCase `json:"reproducer_test,omitempty"`

	// AppliedPatches are the code changes that were applied.
	AppliedPatches []*Patch `json:"applied_patches,omitempty"`

	// TestResults is the final test execution result.
	TestResults *TestResult `json:"test_results,omitempty"`

	// RegressionResults is the full suite test result.
	RegressionResults *TestResult `json:"regression_results,omitempty"`

	// Error contains the error message if TDG failed.
	Error string `json:"error,omitempty"`

	// Duration is how long the TDG session took.
	Duration time.Duration `json:"duration"`

	// Metrics contains execution metrics.
	Metrics *Metrics `json:"metrics"`
}

Result contains the outcome of a TDG session.

type RetryExhaustedError

type RetryExhaustedError struct {
	// Phase is the TDG phase where retries were exhausted.
	Phase string

	// Attempts is the number of attempts made.
	Attempts int

	// MaxAttempts is the configured maximum attempts.
	MaxAttempts int

	// LastError is the error from the last attempt.
	LastError error
}

RetryExhaustedError provides details when retries are exhausted.

func (*RetryExhaustedError) Error

func (e *RetryExhaustedError) Error() string

Error implements the error interface.

func (*RetryExhaustedError) Unwrap

func (e *RetryExhaustedError) Unwrap() error

Unwrap returns the last error.

type State

type State string

State represents a state in the TDG state machine.

const (
	// StateIdle is the initial state before TDG starts.
	StateIdle State = "idle"

	// StateUnderstand analyzes the bug/feature request.
	StateUnderstand State = "understand"

	// StateWriteTest generates a reproducer test.
	StateWriteTest State = "write_test"

	// StateVerifyFail runs the test to confirm it fails.
	StateVerifyFail State = "verify_fail"

	// StateWriteFix generates the fix.
	StateWriteFix State = "write_fix"

	// StateVerifyPass runs the test to confirm the fix works.
	StateVerifyPass State = "verify_pass"

	// StateRegression runs the full test suite.
	StateRegression State = "regression"

	// StateDone indicates successful completion.
	StateDone State = "done"

	// StateFailed indicates TDG failed (max retries, timeout, etc).
	StateFailed State = "failed"
)

func AllStates

func AllStates() []State

AllStates returns all valid TDG states.

func (State) IsActive

func (s State) IsActive() bool

IsActive returns true if the state allows continued execution.

func (State) IsTerminal

func (s State) IsTerminal() bool

IsTerminal returns true if the state is terminal (done or failed).

func (State) String

func (s State) String() string

String returns the string representation of the state.

type StateTransitionError

type StateTransitionError struct {
	From State
	To   State
}

StateTransitionError indicates an invalid state transition was attempted.

func (*StateTransitionError) Error

func (e *StateTransitionError) Error() string

Error implements the error interface.

type TestCase

type TestCase struct {
	// Name is the test function name (e.g., TestValidateToken_NilClaims).
	Name string `json:"name"`

	// FilePath is the path where the test file should be written.
	FilePath string `json:"file_path"`

	// Content is the full test file content.
	Content string `json:"content"`

	// Language is the programming language.
	Language string `json:"language"`

	// TestType is "unit" or "integration".
	TestType string `json:"test_type"`

	// PackagePath is the package/module path for running tests.
	PackagePath string `json:"package_path,omitempty"`
}

TestCase represents a generated test.

func (*TestCase) Validate

func (tc *TestCase) Validate() error

Validate checks that the test case is complete.

type TestExecutionError

type TestExecutionError struct {
	// TestName is the name of the test that failed.
	TestName string

	// FilePath is the path to the test file.
	FilePath string

	// ExitCode is the exit code from the test process.
	ExitCode int

	// Output is the captured stdout/stderr.
	Output string

	// Cause is the underlying error if any.
	Cause error
}

TestExecutionError provides details about a test execution failure.

func (*TestExecutionError) Error

func (e *TestExecutionError) Error() string

Error implements the error interface.

func (*TestExecutionError) Unwrap

func (e *TestExecutionError) Unwrap() error

Unwrap returns the underlying error.

type TestGenerator

type TestGenerator struct {
	// contains filtered or unexported fields
}

TestGenerator generates tests and fixes using an LLM.

Thread Safety: Safe for concurrent use if LLMClient and ContextAssembler are safe for concurrent use.

func NewTestGenerator

func NewTestGenerator(llm LLMClient, context ContextAssembler, logger *slog.Logger) *TestGenerator

NewTestGenerator creates a new test generator.

Inputs:

llm - LLM client for generation
context - Context assembler for code context
logger - Logger for structured logging

Outputs:

*TestGenerator - Configured generator

func (*TestGenerator) CallCount

func (g *TestGenerator) CallCount() int

CallCount returns the number of LLM calls made.

Thread Safety: Safe for concurrent use.

func (*TestGenerator) GenerateFix

func (g *TestGenerator) GenerateFix(ctx context.Context, req *Request, tc *TestCase, testOutput string) (*Patch, error)

GenerateFix creates a patch to fix the bug.

Description:

Uses the LLM to generate a code fix based on the reproducer test
and its failure output.

Inputs:

ctx - Context for cancellation
req - The original request
tc - The reproducer test
testOutput - Output from running the failing test

Outputs:

*Patch - The generated fix
error - Non-nil on generation failure

func (*TestGenerator) GenerateReproducerTest

func (g *TestGenerator) GenerateReproducerTest(ctx context.Context, req *Request) (*TestCase, error)

GenerateReproducerTest creates a test that should FAIL on current code.

Description:

Uses the LLM to generate a test that reproduces the described bug.
The test should fail on the current code and pass after the fix.

Inputs:

ctx - Context for cancellation
req - The TDG request with bug description

Outputs:

*TestCase - The generated test
error - Non-nil on generation failure

func (*TestGenerator) RefineFix

func (g *TestGenerator) RefineFix(ctx context.Context, req *Request, tc *TestCase, previousPatch *Patch, testOutput string) (*Patch, error)

RefineFix regenerates a fix after the test still failed.

Description:

Provides feedback that the fix didn't work and asks for a new fix.

Inputs:

ctx - Context for cancellation
req - The original request
tc - The reproducer test
previousPatch - The fix that didn't work
testOutput - Output showing the test still fails

Outputs:

*Patch - A new fix
error - Non-nil on generation failure

func (*TestGenerator) RefineTest

func (g *TestGenerator) RefineTest(ctx context.Context, req *Request, previousTest *TestCase, testOutput string) (*TestCase, error)

RefineTest regenerates a test after it passed when it should have failed.

Description:

Provides feedback to the LLM that the test didn't reproduce the bug
and asks for a new test that actually fails.

Inputs:

ctx - Context for cancellation
req - The original request
previousTest - The test that incorrectly passed
testOutput - The output showing the test passed

Outputs:

*TestCase - A new test
error - Non-nil on generation failure

func (*TestGenerator) ResetCallCount

func (g *TestGenerator) ResetCallCount()

ResetCallCount resets the call counter.

Thread Safety: Safe for concurrent use.

type TestOutputParser

type TestOutputParser func(output []byte) (passed bool, failedTests []string)

TestOutputParser parses test runner output to extract results.

Inputs:

output - Raw stdout/stderr from test execution

Outputs:

passed - True if all tests passed
failedTests - Names of failed tests

func GetTestOutputParser

func GetTestOutputParser(language string) TestOutputParser

GetTestOutputParser returns the parser for a language.

Inputs:

language - The language identifier

Outputs:

TestOutputParser - The parser function, or nil if not found

Thread Safety: Safe for concurrent use.

type TestResult

type TestResult struct {
	// Passed indicates if all tests passed.
	Passed bool `json:"passed"`

	// Output is the captured stdout/stderr.
	Output string `json:"output"`

	// Duration is how long test execution took.
	Duration time.Duration `json:"duration"`

	// ExitCode is the exit code from the test process.
	ExitCode int `json:"exit_code"`

	// FailedTests lists the names of failed tests.
	FailedTests []string `json:"failed_tests,omitempty"`

	// PassedTests lists the names of passed tests.
	PassedTests []string `json:"passed_tests,omitempty"`

	// TotalTests is the total number of tests run.
	TotalTests int `json:"total_tests"`

	// Truncated indicates if output was truncated due to size limits.
	Truncated bool `json:"truncated"`

	// TimedOut indicates if execution was killed due to timeout.
	TimedOut bool `json:"timed_out"`
}

TestResult contains the outcome of test execution.

type TestRunner

type TestRunner struct {
	// contains filtered or unexported fields
}

TestRunner executes tests and captures results.

Thread Safety: Safe for concurrent use. Each execution creates its own process.

func NewTestRunner

func NewTestRunner(cfg *Config, logger *slog.Logger) *TestRunner

NewTestRunner creates a new test runner.

Inputs:

cfg - TDG configuration
logger - Logger for structured logging

Outputs:

*TestRunner - Configured test runner

func (*TestRunner) RunSuite

func (r *TestRunner) RunSuite(ctx context.Context, language, packagePath string) (*TestResult, error)

RunSuite executes all tests in a package/directory.

Description:

Runs the full test suite for a package using the appropriate test
runner. Used for regression checking after a fix is applied.

Inputs:

ctx - Context for cancellation and timeout
language - The programming language
packagePath - The package or directory path

Outputs:

*TestResult - Suite execution result with all failures
error - Non-nil on execution failure

Thread Safety: Safe for concurrent use.

func (*TestRunner) RunTest

func (r *TestRunner) RunTest(ctx context.Context, tc *TestCase) (*TestResult, error)

RunTest executes a single test by name.

Description:

Executes the specified test using the appropriate test runner for
the language. Captures stdout/stderr and parses the result.

Inputs:

ctx - Context for cancellation and timeout
tc - The test case to execute

Outputs:

*TestResult - Test execution result
error - Non-nil on execution failure

Thread Safety: Safe for concurrent use.

func (*TestRunner) SetWorkingDir

func (r *TestRunner) SetWorkingDir(dir string)

SetWorkingDir sets the working directory for test execution.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL