testrunner

package
v0.38.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 8, 2026 License: MIT Imports: 17 Imported by: 0

Documentation

Index

Constants

View Source
const (
	ExitOK      = 0
	ExitFailure = 1
	ExitError   = 2
)

Exit codes for the test command.

Variables

This section is empty.

Functions

func CombinationCount

func CombinationCount(boolVars []string, pinVars map[string]string) int

CombinationCount returns the number of combinations that would be generated for the given boolean vars and pin vars, without allocating the full slice.

func ComboLabel

func ComboLabel(c Combination, boolVars []string) string

ComboLabel returns a human-readable label for a combination.

func ExtractBooleanVars

func ExtractBooleanVars(cfg *tmplconfig.TemplateConfig, skipVars []string) []string

ExtractBooleanVars returns the sorted names of all boolean variables from the template config, excluding any in skipVars.

func PrintDryRun

func PrintDryRun(w io.Writer, plan *TestPlan)

PrintDryRun lists all combinations that would be tested.

func PrintJSONReport

func PrintJSONReport(w io.Writer, report Report) error

PrintJSONReport writes a JSON-formatted test report to w.

func PrintTextReport

func PrintTextReport(w io.Writer, report Report, boolVars []string, verbose bool)

PrintTextReport writes a human-readable test report to w.

func TruncateOutput

func TruncateOutput(output string, maxLen int) string

TruncateOutput limits output to roughly maxLen characters, keeping the last portion (which usually contains the error).

Types

type CaseResult

type CaseResult struct {
	CaseName    string        `json:"case_name"`
	Combination Combination   `json:"combination"`
	Status      CaseStatus    `json:"status"`
	Phase       string        `json:"phase,omitempty"`    // "scaffold", "validate:<cmd>", etc.
	Output      string        `json:"output,omitempty"`   // Captured stdout+stderr on failure
	Error       string        `json:"error,omitempty"`    // Error message
	KeptDir     string        `json:"kept_dir,omitempty"` // Path to preserved output dir
	Duration    time.Duration `json:"duration"`
}

CaseResult holds the outcome of testing a single combination.

type CaseStatus

type CaseStatus int

CaseStatus represents the outcome of a single test case.

const (
	CasePassed CaseStatus = iota
	CaseFailed
	CaseErrored
)

func (*CaseStatus) MarshalJSON

func (s *CaseStatus) MarshalJSON() ([]byte, error)

MarshalJSON encodes CaseStatus as a JSON string.

func (*CaseStatus) String

func (s *CaseStatus) String() string

String returns the human-readable name of the status.

func (*CaseStatus) UnmarshalJSON

func (s *CaseStatus) UnmarshalJSON(data []byte) error

UnmarshalJSON decodes a JSON string into CaseStatus.

type Combination

type Combination struct {
	Index int               `json:"index"`
	Vars  map[string]string `json:"vars"`
}

Combination represents a single set of boolean variable assignments.

func FilterCombinations

func FilterCombinations(combos []Combination, filter string) ([]Combination, error)

FilterCombinations applies a filter expression to narrow down combinations. Filter can be:

  • A numeric index (e.g., "7")
  • Comma-separated key=value pairs (e.g., "use_postgres=true,use_amqp=true")

Returns an error if the filter expression is malformed.

func GenerateCombinations

func GenerateCombinations(boolVars []string, pinVars map[string]string) []Combination

GenerateCombinations produces all 2^N boolean combinations for the given variable names. Pinned variables are excluded from permutation and added with their fixed value.

type Config

type Config struct {
	TemplateDir string            // Path to template directory
	Meta        map[string]string // Required string variable overrides
	ValuesFile  string            // Path to values JSON file
	SkipVars    []string          // Boolean vars to exclude from permutation (use default)
	PinVars     map[string]string // Vars to fix at a specific value (not permuted)
	RunCommands []string          // Validation commands (overrides template config)
	Filter      string            // Filter expression (index or key=value pairs)
	CaseName    string            // Run only the test case with this name
	Parallel    int               // Max concurrent test runs
	FailFast    bool              // Stop on first failure
	DryRun      bool              // List combinations without running
	KeepFailed  bool              // Keep scaffolded dirs on failure
	Timeout     time.Duration     // Per-command timeout
	MaxCases    int               // Safety limit for total combinations (0 = unlimited)
	Verbose     bool              // Show full output on failures
	AcceptHooks bool              // Opt-in hook and test command execution
	Format      string            // Output format: "text" or "json"
}

Config holds the full configuration for a matrix test run.

type Report

type Report struct {
	Cases       []CaseResult  `json:"cases"`
	Passed      int           `json:"passed"`
	Failed      int           `json:"failed"`
	Errored     int           `json:"errored"`
	TotalCases  int           `json:"total_cases"`
	Duration    time.Duration `json:"duration"`
	TemplateDir string        `json:"template_dir"`
}

Report aggregates all test case results.

func Execute

func Execute(ctx context.Context, plan *TestPlan, cfg Config) (Report, error)

Execute runs the test plan with the given configuration and returns the report.

func (Report) ExitCode

func (r Report) ExitCode() int

ExitCode returns the appropriate process exit code for the report.

type TestCasePlan

type TestCasePlan struct {
	Name     string
	Combos   []Combination
	Commands []string
}

TestCasePlan represents a single named test case with its filtered combinations and commands.

type TestPlan

type TestPlan struct {
	TemplateDir string
	BoolVars    []string
	Cases       []TestCasePlan
	Env         map[string]string
	ProjectName string
}

TestPlan holds the resolved plan for a matrix test run.

func Plan

func Plan(cfg Config) (*TestPlan, error)

Plan resolves the test configuration and generates the execution plan. It validates limits before allocating combinations to prevent OOM.

type ValidationResult

type ValidationResult struct {
	Command  string
	Output   string
	ExitCode int
	Duration time.Duration
	Err      error
}

ValidationResult holds the outcome of running a single validation command.

func RunValidationCommands

func RunValidationCommands(ctx context.Context, dir string, commands []string, env map[string]string, timeout time.Duration) *ValidationResult

RunValidationCommands executes each command sequentially in the given directory. Returns on the first failure if any command fails.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL