Documentation
¶
Overview ¶
Package testing provides test collection and execution for AILANG programs. This implements the M-TESTING feature for property-based testing and inline tests.
Index ¶
- func BuildClusterTestHarness(cluster *PureCluster, tests []TestCase) core.CoreExpr
- func BuildInlineTestHarness(binding core.RecBinding, tests []TestCase) core.CoreExpr
- func ComputeSCCs(g *CallGraph) [][]string
- func FindSCCContaining(sccs [][]string, funcName string) []string
- func GetDependencyClosure(g *CallGraph, sccs [][]string, funcName string) []string
- func GetEffectNames(binding *core.RecBinding, coreTypeInfo types.CoreTypeInfo) []string
- func IsPure(binding *core.RecBinding, coreTypeInfo types.CoreTypeInfo) bool
- type ADTGenerator
- type ADTShrinker
- type BoolGenerator
- type CallGraph
- type Collector
- type CombinedResolver
- type ConstantGenerator
- type Executor
- func (e *Executor) CompareValues(actual, expected eval.Value) bool
- func (e *Executor) EvaluateExpression(expr ast.Expr) (eval.Value, error)
- func (e *Executor) EvaluateInlineTestsWithCluster(functionName string, tests []TestCase, coreProg *core.Program) (*eval.TupleValue, error)
- func (e *Executor) EvaluateInlineTestsWithHarness(binding core.RecBinding, tests []TestCase) (*eval.TupleValue, error)
- func (e *Executor) EvaluateLiteral(expr ast.Expr) (eval.Value, error)
- func (e *Executor) ExtractFunctionBinding(functionName string, sourceFile *ast.File) (*core.RecBinding, error)
- func (e *Executor) ExtractPureClusterForFunction(functionName string, sourceFile *ast.File) (*PureCluster, *core.Program, error)
- func (e *Executor) HasCrossFunctionDependencies(functionName string, coreProg *core.Program) bool
- func (e *Executor) SetDebug(debug bool)
- func (e *Executor) SetSourceFile(file *ast.File)
- type FilterGenerator
- type FloatGenerator
- type FloatShrinker
- type FrequencyGenerator
- type GenConfig
- type Generator
- type IntGenerator
- type IntShrinker
- type ListGenerator
- type ListShrinker
- type MapGenerator
- type NoOpShrinker
- type OneOfGenerator
- type OutputFormat
- type PropertyCase
- type PropertyResult
- type PropertyRunner
- func (pr *PropertyRunner) GenerateBool() bool
- func (pr *PropertyRunner) GenerateFloatInRange() float64
- func (pr *PropertyRunner) GenerateIntInRange() int
- func (pr *PropertyRunner) GenerateList(elemGen Generator, maxLen int) []eval.Value
- func (pr *PropertyRunner) GenerateString() string
- func (pr *PropertyRunner) GetConfig() GenConfig
- func (pr *PropertyRunner) GetRNG() *rand.Rand
- func (pr *PropertyRunner) ShrinkValue(original eval.Value, shrinker Shrinker, predicate func(eval.Value) bool) eval.Value
- type PureCluster
- type PurityError
- type RecordGenerator
- type Reporter
- type Runner
- type Shrinker
- type SizedGenerator
- type StringGenerator
- type StringShrinker
- type SuiteResult
- type TestCase
- type TestResult
- type TestStatus
- type TestSuite
- type TupleGenerator
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func BuildClusterTestHarness ¶
func BuildClusterTestHarness(cluster *PureCluster, tests []TestCase) core.CoreExpr
BuildClusterTestHarness creates a synthetic Core expression for testing a function with cross-function dependencies.
Given a pure cluster (function under test + all dependencies) and test cases, builds:
LetRec([f, g1, g2, ...],
Let("_test_1", App(f, arg_1),
Let("_test_2", App(f, arg_2),
Tuple([_test_1, _test_2])
)
)
)
All cluster bindings are in scope via the shared LetRec.
func BuildInlineTestHarness ¶
func BuildInlineTestHarness(binding core.RecBinding, tests []TestCase) core.CoreExpr
BuildInlineTestHarness creates a synthetic Core expression for evaluating inline tests.
Given a function binding and its test cases, builds a Core expression of the form:
LetRec("f", λ_f,
Let("_test_1", App(f, arg_1),
Let("_test_2", App(f, arg_2),
Tuple([_test_1, _test_2])
)
)
)
The function `f` is in scope for all test calls via the LetRec body. Returns a tuple of actual results for the test runner to compare against expected values.
Input:
- binding: Core LetRec binding for the function being tested
- tests: List of test cases (each contains input/expected tuple expressions)
Output: Core expression that evaluates all tests and returns tuple of actuals
func ComputeSCCs ¶
ComputeSCCs computes the strongly connected components of the call graph using Tarjan's algorithm. Returns SCCs in reverse topological order (dependencies come before dependents).
func FindSCCContaining ¶
FindSCCContaining returns the SCC that contains the given function name.
func GetDependencyClosure ¶
GetDependencyClosure returns all functions reachable from the given function, including the function itself and its SCC.
func GetEffectNames ¶
func GetEffectNames(binding *core.RecBinding, coreTypeInfo types.CoreTypeInfo) []string
GetEffectNames returns the effect names for a binding, or nil if pure.
func IsPure ¶
func IsPure(binding *core.RecBinding, coreTypeInfo types.CoreTypeInfo) bool
IsPure checks if a binding's type indicates it's a pure function. A function is pure if its effect row is nil or empty.
Types ¶
type ADTGenerator ¶
type ADTGenerator struct {
// contains filtered or unexported fields
}
ADTGenerator generates algebraic data type values (tagged unions).
func NewADTGenerator ¶
func NewADTGenerator(tag string, fieldGens []Generator, constructTag bool) *ADTGenerator
NewADTGenerator creates a generator for ADT values. If constructTag is true, wraps result in TaggedValue.
type ADTShrinker ¶
type ADTShrinker struct {
// contains filtered or unexported fields
}
ADTShrinker shrinks ADT values by shrinking their fields.
func NewADTShrinker ¶
func NewADTShrinker(fieldShrinkers []Shrinker) *ADTShrinker
NewADTShrinker creates a new ADT shrinker.
type BoolGenerator ¶
type BoolGenerator struct{}
BoolGenerator generates random booleans.
func NewBoolGenerator ¶
func NewBoolGenerator() *BoolGenerator
NewBoolGenerator creates a new bool generator.
type CallGraph ¶
type CallGraph struct {
// Nodes is the set of all function names in the module
Nodes map[string]bool
// Edges maps each function to the functions it calls
Edges map[string][]string
// Bindings maps function names to their RecBinding
Bindings map[string]*core.RecBinding
}
CallGraph represents dependencies between functions in a module. Nodes are function names, edges are directed "calls" relationships.
func BuildCallGraph ¶
BuildCallGraph constructs a call graph from a Core program. It identifies all top-level function bindings and their dependencies.
type Collector ¶
type Collector struct {
// contains filtered or unexported fields
}
Collector extracts tests and properties from an AST.
func NewCollector ¶
NewCollector creates a new test collector.
type CombinedResolver ¶
type CombinedResolver struct {
Builtins *runtime.BuiltinRegistry
Env *eval.Environment // Environment containing user-defined and imported functions
Modules map[string]*loader.LoadedModule // Loaded modules for module-qualified lookup
}
CombinedResolver resolves both builtin functions and user-defined functions from the environment. Used for inline test harness evaluation to support functions that depend on imports. It handles: - Builtin references (module="$builtin" or name starts with "_") - Module-qualified references (module="std/list" name="filter") - Local references (module="" or module matches current file)
func (*CombinedResolver) ResolveValue ¶
ResolveValue implements eval.GlobalResolver for combined resolution.
type ConstantGenerator ¶
type ConstantGenerator struct {
// contains filtered or unexported fields
}
ConstantGenerator always returns the same value.
func NewConstantGenerator ¶
func NewConstantGenerator(value eval.Value) *ConstantGenerator
NewConstantGenerator creates a generator that always returns the same value.
type Executor ¶
type Executor struct {
// contains filtered or unexported fields
}
Executor handles evaluation of test expressions through the AILANG pipeline.
func NewExecutor ¶
NewExecutor creates a new test executor.
func (*Executor) CompareValues ¶
CompareValues checks if two values are equal.
func (*Executor) EvaluateExpression ¶
EvaluateExpression evaluates a Surface AST expression through the pipeline. Uses ModeEval to properly handle function definitions and expression evaluation.
func (*Executor) EvaluateInlineTestsWithCluster ¶
func (e *Executor) EvaluateInlineTestsWithCluster( functionName string, tests []TestCase, coreProg *core.Program, ) (*eval.TupleValue, error)
EvaluateInlineTestsWithCluster evaluates inline tests for a function with cross-function dependencies.
func (*Executor) EvaluateInlineTestsWithHarness ¶
func (e *Executor) EvaluateInlineTestsWithHarness(binding core.RecBinding, tests []TestCase) (*eval.TupleValue, error)
EvaluateInlineTestsWithHarness evaluates inline tests using the test harness builder. This is the PREFERRED method for inline tests (fixes scoping issues in EvaluateExpression).
func (*Executor) EvaluateLiteral ¶
EvaluateLiteral converts an AST literal expression to an eval.Value.
func (*Executor) ExtractFunctionBinding ¶
func (e *Executor) ExtractFunctionBinding(functionName string, sourceFile *ast.File) (*core.RecBinding, error)
ExtractFunctionBinding extracts a Core LetRec binding for a function from source code.
func (*Executor) ExtractPureClusterForFunction ¶
func (e *Executor) ExtractPureClusterForFunction( functionName string, sourceFile *ast.File, ) (*PureCluster, *core.Program, error)
ExtractPureClusterForFunction extracts the pure dependency cluster for a function from source code.
func (*Executor) HasCrossFunctionDependencies ¶
func (e *Executor) HasCrossFunctionDependencies( functionName string, coreProg *core.Program, ) bool
HasCrossFunctionDependencies checks if a function has dependencies on other user-defined functions.
func (*Executor) SetSourceFile ¶
SetSourceFile sets the source file to provide context for test evaluation.
type FilterGenerator ¶
type FilterGenerator struct {
// contains filtered or unexported fields
}
FilterGenerator generates values that satisfy a predicate. Retries up to maxRetries times before giving up.
func NewFilterGenerator ¶
func NewFilterGenerator(source Generator, predicate func(eval.Value) bool, maxRetries int) *FilterGenerator
NewFilterGenerator creates a generator that filters values. maxRetries specifies how many attempts before giving up (0 = unlimited, dangerous!)
type FloatGenerator ¶
type FloatGenerator struct {
// contains filtered or unexported fields
}
FloatGenerator generates random floats.
func NewFloatGenerator ¶
func NewFloatGenerator(min, max float64) *FloatGenerator
NewFloatGenerator creates a new float generator with specified range.
type FloatShrinker ¶
type FloatShrinker struct{}
FloatShrinker shrinks floats toward zero.
func NewFloatShrinker ¶
func NewFloatShrinker() *FloatShrinker
NewFloatShrinker creates a new float shrinker.
type FrequencyGenerator ¶
type FrequencyGenerator struct {
// contains filtered or unexported fields
}
FrequencyGenerator chooses generators with weighted probability.
func NewFrequencyGenerator ¶
func NewFrequencyGenerator(weights []int, generators []Generator) *FrequencyGenerator
NewFrequencyGenerator creates a generator with weighted choice. weights and generators must have the same length.
type GenConfig ¶
type GenConfig struct {
Seed int64 // Seed for deterministic generation
MaxSize int // Maximum size for collections (lists, strings)
MinInt int // Minimum int value
MaxInt int // Maximum int value
MinFloat float64
MaxFloat float64
}
GenConfig configures generation parameters.
func DefaultConfig ¶
func DefaultConfig() GenConfig
DefaultConfig returns default generation configuration.
type Generator ¶
type Generator interface {
// Generate produces a random value using the provided RNG.
Generate(rng *rand.Rand) eval.Value
}
Generator produces random values of a specific type for property-based testing.
func OptionGenerator ¶
OptionGenerator generates Option type values (Some(x) | None).
func ResultGenerator ¶
ResultGenerator generates Result type values (Ok(x) | Err(e)).
type IntGenerator ¶
type IntGenerator struct {
// contains filtered or unexported fields
}
IntGenerator generates random integers.
func NewIntGenerator ¶
func NewIntGenerator(min, max int) *IntGenerator
NewIntGenerator creates a new int generator with specified range.
type IntShrinker ¶
type IntShrinker struct{}
IntShrinker shrinks integers toward zero.
func NewIntShrinker ¶
func NewIntShrinker() *IntShrinker
NewIntShrinker creates a new integer shrinker.
type ListGenerator ¶
type ListGenerator struct {
// contains filtered or unexported fields
}
ListGenerator generates random lists using an element generator.
func NewListGenerator ¶
func NewListGenerator(elemGen Generator, minLen, maxLen int) *ListGenerator
NewListGenerator creates a new list generator.
type ListShrinker ¶
type ListShrinker struct {
// contains filtered or unexported fields
}
ListShrinker shrinks lists by removing elements or shrinking elements.
func NewListShrinker ¶
func NewListShrinker(elemShrinker Shrinker) *ListShrinker
NewListShrinker creates a new list shrinker.
type MapGenerator ¶
type MapGenerator struct {
// contains filtered or unexported fields
}
MapGenerator transforms generated values using a mapping function.
func NewMapGenerator ¶
NewMapGenerator creates a generator that transforms values from source.
type NoOpShrinker ¶
type NoOpShrinker struct{}
NoOpShrinker is a shrinker that doesn't shrink (returns no shrinks). Useful for values that can't be simplified (e.g., booleans, unit).
func NewNoOpShrinker ¶
func NewNoOpShrinker() *NoOpShrinker
NewNoOpShrinker creates a new no-op shrinker.
type OneOfGenerator ¶
type OneOfGenerator struct {
// contains filtered or unexported fields
}
OneOfGenerator randomly chooses from multiple generators.
func NewOneOfGenerator ¶
func NewOneOfGenerator(generators ...Generator) *OneOfGenerator
NewOneOfGenerator creates a generator that randomly picks from options.
type OutputFormat ¶
type OutputFormat string
OutputFormat specifies the format for test output.
const ( FormatHuman OutputFormat = "human" // Human-readable colored output FormatJSON OutputFormat = "json" // Machine-readable JSON )
type PropertyCase ¶
type PropertyCase struct {
Name string // Property name
Property *ast.Property // The property specification (forall(...) => expr)
Location ast.Pos // Source location
IsInline bool // true for properties[...], false for property "name" {}
}
PropertyCase represents a property-based test extracted from the AST.
type PropertyResult ¶
type PropertyResult struct {
Name string // Property name
Status TestStatus // Pass/fail/skip
Duration time.Duration // Total execution time
TestsRun int // Number of test cases generated
FailingInput string // Minimal failing input (if failed)
Error string // Error message (if failed)
Location string // Source location
}
PropertyResult represents the outcome of a property-based test.
type PropertyRunner ¶
type PropertyRunner struct {
// contains filtered or unexported fields
}
PropertyRunner runs property-based tests with generators.
func NewPropertyRunner ¶
func NewPropertyRunner(config GenConfig, testRuns int) *PropertyRunner
NewPropertyRunner creates a new property runner with config.
func (*PropertyRunner) GenerateBool ¶
func (pr *PropertyRunner) GenerateBool() bool
GenerateBool generates a random boolean.
func (*PropertyRunner) GenerateFloatInRange ¶
func (pr *PropertyRunner) GenerateFloatInRange() float64
GenerateFloatInRange generates a random float in the configured range.
func (*PropertyRunner) GenerateIntInRange ¶
func (pr *PropertyRunner) GenerateIntInRange() int
GenerateIntInRange generates a random int in the configured range.
func (*PropertyRunner) GenerateList ¶
func (pr *PropertyRunner) GenerateList(elemGen Generator, maxLen int) []eval.Value
GenerateList generates a random list using the provided element generator.
func (*PropertyRunner) GenerateString ¶
func (pr *PropertyRunner) GenerateString() string
GenerateString generates a random string with length up to MaxSize.
func (*PropertyRunner) GetConfig ¶
func (pr *PropertyRunner) GetConfig() GenConfig
GetConfig returns the generation configuration.
func (*PropertyRunner) GetRNG ¶
func (pr *PropertyRunner) GetRNG() *rand.Rand
GetRNG returns the random number generator for this runner. Useful for custom generators that need direct RNG access.
func (*PropertyRunner) ShrinkValue ¶
func (pr *PropertyRunner) ShrinkValue( original eval.Value, shrinker Shrinker, predicate func(eval.Value) bool, ) eval.Value
ShrinkValue attempts to shrink a failing value to find a minimal counterexample. It uses the provided shrinker and predicate function. Returns the minimal shrunk value that still fails, or the original if no smaller failure found.
type PureCluster ¶
type PureCluster struct {
// FuncName is the function under test
FuncName string
// Bindings are all the RecBindings needed (in dependency order)
Bindings []core.RecBinding
// Names is the set of function names in the cluster
Names map[string]bool
}
PureCluster represents a set of pure functions that can be tested together.
func ExtractPureCluster ¶
func ExtractPureCluster( funcName string, g *CallGraph, sccs [][]string, coreTypeInfo types.CoreTypeInfo, ) (*PureCluster, error)
ExtractPureCluster extracts the pure cluster for a function. Returns error if any dependency has effects.
func (*PureCluster) DependencyNames ¶
func (pc *PureCluster) DependencyNames() []string
DependencyNames returns the names of dependencies (excluding the function under test).
func (*PureCluster) HasDependencies ¶
func (pc *PureCluster) HasDependencies() bool
HasDependencies returns true if the function has dependencies beyond itself.
type PurityError ¶
type PurityError struct {
// FuncUnderTest is the function being tested
FuncUnderTest string
// EffectfulFunc is the function that has effects
EffectfulFunc string
// Effects is the list of effects the function has
Effects []string
}
PurityError is returned when a function depends on an effectful function.
func (*PurityError) Error ¶
func (e *PurityError) Error() string
type RecordGenerator ¶
type RecordGenerator struct {
// contains filtered or unexported fields
}
RecordGenerator generates record values (field: value maps).
func NewRecordGenerator ¶
func NewRecordGenerator(fieldGens map[string]Generator) *RecordGenerator
NewRecordGenerator creates a generator for record values.
type Reporter ¶
type Reporter struct {
// contains filtered or unexported fields
}
Reporter handles formatting and outputting test results.
func NewReporter ¶
func NewReporter(format OutputFormat, writer io.Writer, colors bool) *Reporter
NewReporter creates a new test reporter.
func (*Reporter) Report ¶
func (r *Reporter) Report(result *SuiteResult) error
Report outputs a test suite result in the configured format.
type Runner ¶
type Runner struct {
// contains filtered or unexported fields
}
Runner executes tests and properties.
func (*Runner) RunSuite ¶
func (r *Runner) RunSuite(suite *TestSuite) *SuiteResult
RunSuite executes all tests in a test suite and returns aggregated results.
type Shrinker ¶
type Shrinker interface {
// Shrink returns a list of simpler values to try.
// The list should be ordered from "most simplified" to "least simplified".
// For example, IntShrinker.Shrink(100) might return [0, 50, 75, 88, 94, 97, 99]
Shrink(val eval.Value) []eval.Value
}
Shrinker defines the interface for shrinking values. Shrinking is the process of finding simpler counterexamples when a property fails. For example, if a property fails on input 100, we might shrink to 50, 25, ..., 0 to find the minimal failing case.
type SizedGenerator ¶
type SizedGenerator struct {
// contains filtered or unexported fields
}
SizedGenerator adjusts size based on a size parameter.
func NewSizedGenerator ¶
func NewSizedGenerator(size int, createFn func(size int) Generator) *SizedGenerator
NewSizedGenerator creates a size-aware generator.
func (*SizedGenerator) Generate ¶
func (g *SizedGenerator) Generate(rng *rand.Rand) eval.Value
Generate produces a value with the configured size.
func (*SizedGenerator) WithSize ¶
func (g *SizedGenerator) WithSize(size int) *SizedGenerator
WithSize returns a new generator with a different size.
type StringGenerator ¶
type StringGenerator struct {
// contains filtered or unexported fields
}
StringGenerator generates random strings.
func NewStringGenerator ¶
func NewStringGenerator(minLen, maxLen int, charset string) *StringGenerator
NewStringGenerator creates a new string generator with specified length range. If charset is empty, uses alphanumeric characters.
type StringShrinker ¶
type StringShrinker struct{}
StringShrinker shrinks strings toward empty string.
func NewStringShrinker ¶
func NewStringShrinker() *StringShrinker
NewStringShrinker creates a new string shrinker.
type SuiteResult ¶
type SuiteResult struct {
ModulePath string // Module path
Tests []TestResult // All test results
Properties []PropertyResult // All property results
TotalTests int // Total number of tests
PassedTests int // Number of passing tests
FailedTests int // Number of failing tests
SkippedTests int // Number of skipped tests
TotalDuration time.Duration // Total execution time
}
SuiteResult aggregates results from all tests in a test suite.
func NewSuiteResult ¶
func NewSuiteResult(modulePath string) *SuiteResult
NewSuiteResult creates a new empty suite result.
func RunTestsFromFile ¶
func RunTestsFromFile(filePath string, ast *ast.File) (*SuiteResult, error)
RunTestsFromFile is a convenience function that parses, collects, and runs tests from a file.
func (*SuiteResult) AddPropertyResult ¶
func (sr *SuiteResult) AddPropertyResult(result PropertyResult)
AddPropertyResult adds a property result and updates counters.
func (*SuiteResult) AddTestResult ¶
func (sr *SuiteResult) AddTestResult(result TestResult)
AddTestResult adds a test result and updates counters.
func (*SuiteResult) Success ¶
func (sr *SuiteResult) Success() bool
Success returns true if all tests passed.
func (*SuiteResult) Summary ¶
func (sr *SuiteResult) Summary() string
Summary returns a human-readable summary of the results.
type TestCase ¶
type TestCase struct {
Name string // Test name (from test "name" { ... })
Body []ast.Expr // Test body expressions
Location ast.Pos // Source location
IsInline bool // true for tests[...], false for test "name" {}
FunctionCtx string // Function name if this is an inline test
}
TestCase represents a single test case extracted from the AST.
type TestResult ¶
type TestResult struct {
Name string // Test name
Status TestStatus // Pass/fail/skip
Duration time.Duration // Execution time
Error string // Error message (if failed)
Location string // Source location (file:line:col)
}
TestResult represents the outcome of a single test execution.
type TestStatus ¶
type TestStatus string
TestStatus represents the outcome of a test execution.
const ( StatusPass TestStatus = "pass" StatusFail TestStatus = "fail" StatusSkip TestStatus = "skip" )
type TestSuite ¶
type TestSuite struct {
ModulePath string // Module path (e.g., "std/list")
Tests []TestCase // All test cases
Properties []PropertyCase // All property cases
}
TestSuite represents all tests and properties extracted from a module.
type TupleGenerator ¶
type TupleGenerator struct {
// contains filtered or unexported fields
}
TupleGenerator generates tuple values.
func NewTupleGenerator ¶
func NewTupleGenerator(elemGens []Generator) *TupleGenerator
NewTupleGenerator creates a generator for tuple values.