fuzz

package
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 9, 2026 License: MIT Imports: 13 Imported by: 0

Documentation

Overview

Package fuzz generates random, well-typed formulas with inputs and runs each one on the Go VM and on a SQL dialect, reporting any divergence.

It is the property-based half of "three evaluators, one behaviour" (ADR-0003): the conformance suite pins the behaviour somebody asked for, the generator searches for the behaviour nobody thought to ask about. Every formula it produces binds, so a bind error is a generator bug, and every divergence it finds can be pasted into the suite as a case.

The SQLite runner in internal/formula/sqlitefn and the DuckDB runner in the analytics module drive it; FADODB_FUZZ_N, FADODB_FUZZ_SEED, FADODB_FUZZ_WORKERS and FADODB_FUZZ_MAXTEXT size a run.

Index

Constants

View Source
const (
	DefaultSeed    = 20261106
	DefaultN       = 2000
	DefaultMaxText = 1024
)

Defaults.

View Source
const (
	MaxDepth = 4
)

MaxDepth bounds the nesting of a generated expression; nodeBudget bounds its size, so a wide call cannot explode.

View Source
const Tolerance = 1e-12

Tolerance is the relative difference allowed between the targets for a number that went through an aggregate or a math-library function (ADR-0019).

Variables

View Source
var Scope = func() *formula.Scope {
	s, err := formula.NewScope(Variables...)
	if err != nil {
		panic(err)
	}
	return s
}()

Scope is Variables as a scope.

View Source
var Variables = []formula.Variable{
	{Name: "n1", Type: formula.Scalar(formula.KindNumber)},
	{Name: "n2", Type: formula.Scalar(formula.KindNumber)},
	{Name: "n3", Type: formula.Scalar(formula.KindNumber)},
	{Name: "t1", Type: formula.Scalar(formula.KindText)},
	{Name: "t2", Type: formula.Scalar(formula.KindText)},
	{Name: "b1", Type: formula.Scalar(formula.KindBool)},
	{Name: "b2", Type: formula.Scalar(formula.KindBool)},
	{Name: "d1", Type: formula.Scalar(formula.KindDate)},
	{Name: "d2", Type: formula.Scalar(formula.KindDate)},
	{Name: "s1", Type: formula.Scalar(formula.KindDateTime)},
	{Name: "s2", Type: formula.Scalar(formula.KindDateTime)},
	{Name: "x1", Type: formula.DecimalType(2)},
	{Name: "x2", Type: formula.DecimalType(4)},
	{Name: "i1", Type: formula.DecimalType(0)},
	{Name: "m1", Type: formula.MoneyType(2)},
	{Name: "m2", Type: formula.MoneyType(2)},
	{Name: "j1", Type: formula.Scalar(formula.KindJSON)},
}

Variables is the scope every generated formula reads: three numbers, two texts, two bools, two dates, two datetimes, three exact numbers, two money values and a json value. The names are short so a report stays readable.

Functions

func Noisy

func Noisy(p *formula.Program) bool

Noisy reports whether the program's result may carry last-bit noise between the targets: a power, an exponential or a logarithm, which each engine's math library rounds its own way (ADR-0019). A number or a rendering of such a result compares to Tolerance; any other kind, a comparison or a date built from it, must still agree exactly, and a disagreement there is reported. SUM and AVERAGE add left to right on every evaluator and are exact.

Types

type Config

type Config struct {
	// Seed is the seed of the first formula; formula i uses Seed + i.
	Seed uint64
	// N is the number of formulas.
	N int
	// Workers is the number of goroutines, each with its own VM and
	// database connection.
	Workers int
	// MaxText is the text limit the formulas evaluate under (ADR-0021).
	// It is small by default, so that short inputs reach the boundary
	// often and a campaign never builds a long text.
	MaxText int
}

Config sizes a campaign.

func ConfigFromEnv

func ConfigFromEnv() (Config, error)

ConfigFromEnv reads FADODB_FUZZ_SEED, FADODB_FUZZ_N, FADODB_FUZZ_WORKERS and FADODB_FUZZ_MAXTEXT, with the defaults for what is unset.

type Formula

type Formula struct {
	// Seed reproduces the formula: New(Seed).Formula() generates it again.
	Seed   uint64
	Src    string
	Inputs []formula.Value
	Ctx    formula.Context
}

Formula is one generated formula with the inputs and the context it runs under. Inputs has one value per entry of Variables.

func (*Formula) CaseJSON

func (f *Formula) CaseJSON(result formula.Value, code string, t formula.Type) string

CaseJSON renders the formula with its inputs and context as a suite case whose expectation is the given value or error code, ready to paste into conformance/formula. The suite variables are those of Variables.

func (*Formula) DescribeContext

func (f *Formula) DescribeContext() string

DescribeContext renders the context.

func (*Formula) DescribeInputs

func (f *Formula) DescribeInputs() string

DescribeInputs renders the inputs as name=value pairs, blanks left out.

type Generator

type Generator struct {
	// contains filtered or unexported fields
}

Generator produces formulas from a seed. It is not safe for concurrent use; make one per goroutine.

Noise and quiet: a power, an exponential or a logarithm carries last-bit noise between the targets, which the runner tolerates in a number and in its rendering (Tolerance). It cannot tolerate it once a discontinuous operation has amplified it: a comparison, MOD, SIGN, rounding, a date part, a text position, a count, a conversion to an exact number. So those operations ask for quiet arguments, and inside a quiet subtree the generator never produces a noisy function. The conformance suite covers the shapes this leaves out, such as ROUND of a power, with values away from any rounding boundary. Exact arithmetic is never noisy.

func New

func New(seed uint64) *Generator

New makes a generator for a seed.

func (*Generator) Formula

func (g *Generator) Formula() *Formula

Formula generates the next formula. The first formula of a generator is a pure function of its seed; later ones continue the stream.

type Outcome

type Outcome struct {
	Prog *formula.Program
	// BindErr is set when the formula did not bind: a generator bug.
	BindErr error
	// VM is the VM's value, and VMCode its error code when it failed.
	VM     formula.Value
	VMCode string
	// SQL is the dialect's result; SQLErr is set when the statement
	// itself failed, which is a divergence too.
	SQL    formula.SQLResult
	SQLErr error
	// Reason says how the targets disagree; "" when they agree.
	Reason string
}

Outcome is one formula run on the VM and on a dialect.

func Run

func Run(f *Formula, vm *formula.VM, db *sql.DB, d formula.Dialect) *Outcome

Run binds f, runs it on the VM and through db on the dialect, and compares the two by the rules of the conformance suite: error codes must match and values must be equal, except that a Noisy program's number, or its rendering inside a text, may differ by Tolerance.

func (*Outcome) Agree

func (o *Outcome) Agree() bool

Agree reports whether the two targets produced the same result.

func (*Outcome) Report

func (o *Outcome) Report(f *Formula, d formula.Dialect) string

Report renders the outcome for a person: the formula, how to reproduce it, both results, the statement, and the formula as a suite case with the VM's result as the expectation.

type Stats

type Stats struct {
	Formulas int
	// VMErrors counts formulas the VM ended with an error code; the
	// rest produced a value.
	VMErrors int
	// Divergences counts formulas the targets disagreed on; Reports holds
	// the first few, rendered.
	Divergences int
	Reports     []string
	// Brief has one line per divergence, seed, reason and formula, for
	// the first maxBrief of them, to see the classes at a glance.
	Brief   []string
	Elapsed time.Duration
}

Stats is what a campaign found.

func Campaign

func Campaign(cfg Config, d formula.Dialect, open func() (*sql.DB, error)) (Stats, error)

Campaign generates cfg.N formulas and runs each on the VM and on the dialect under cfg.MaxText, over cfg.Workers goroutines; open gives each worker its own database. A formula that does not bind stops the campaign with an error, since the generator promises to bind.

func (Stats) Summary

func (s Stats) Summary(target string) string

Summary renders the counts in one line.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL