Documentation
¶
Overview ¶
Package fuzz generates random, well-typed formulas with inputs and runs each one on the Go VM and on a SQL dialect, reporting any divergence.
It is the property-based half of "three evaluators, one behaviour" (ADR-0003): the conformance suite pins the behaviour somebody asked for, the generator searches for the behaviour nobody thought to ask about. Every formula it produces binds, so a bind error is a generator bug, and every divergence it finds can be pasted into the suite as a case.
The SQLite runner in internal/formula/sqlitefn and the DuckDB runner in the analytics module drive it; FADODB_FUZZ_N, FADODB_FUZZ_SEED, FADODB_FUZZ_WORKERS and FADODB_FUZZ_MAXTEXT size a run.
Index ¶
Constants ¶
const ( DefaultSeed = 20261106 DefaultN = 2000 DefaultMaxText = 1024 )
Defaults.
const (
MaxDepth = 4
)
MaxDepth bounds the nesting of a generated expression; nodeBudget bounds its size, so a wide call cannot explode.
const Tolerance = 1e-12
Tolerance is the relative difference allowed between the targets for a number that went through an aggregate or a math-library function (ADR-0019).
Variables ¶
var Scope = func() *formula.Scope { s, err := formula.NewScope(Variables...) if err != nil { panic(err) } return s }()
Scope is Variables as a scope.
var Variables = []formula.Variable{ {Name: "n1", Type: formula.Scalar(formula.KindNumber)}, {Name: "n2", Type: formula.Scalar(formula.KindNumber)}, {Name: "n3", Type: formula.Scalar(formula.KindNumber)}, {Name: "t1", Type: formula.Scalar(formula.KindText)}, {Name: "t2", Type: formula.Scalar(formula.KindText)}, {Name: "b1", Type: formula.Scalar(formula.KindBool)}, {Name: "b2", Type: formula.Scalar(formula.KindBool)}, {Name: "d1", Type: formula.Scalar(formula.KindDate)}, {Name: "d2", Type: formula.Scalar(formula.KindDate)}, {Name: "j1", Type: formula.Scalar(formula.KindJSON)}, }
Variables is the scope every generated formula reads: three numbers, two texts, two bools, two dates and a json value. The names are short so a report stays readable.
Functions ¶
func Noisy ¶
Noisy reports whether the program's result may carry last-bit noise between the targets: a power, an exponential or a logarithm, which each engine's math library rounds its own way (ADR-0019). A number or a rendering of such a result compares to Tolerance; any other kind, a comparison or a date built from it, must still agree exactly, and a disagreement there is reported. SUM and AVERAGE add left to right on every evaluator and are exact.
Types ¶
type Config ¶
type Config struct {
// Seed is the seed of the first formula; formula i uses Seed + i.
Seed uint64
// N is the number of formulas.
N int
// Workers is the number of goroutines, each with its own VM and
// database connection.
Workers int
// MaxText is the text limit the formulas evaluate under (ADR-0021).
// It is small by default, so that short inputs reach the boundary
// often and a campaign never builds a long text.
MaxText int
}
Config sizes a campaign.
func ConfigFromEnv ¶
ConfigFromEnv reads FADODB_FUZZ_SEED, FADODB_FUZZ_N, FADODB_FUZZ_WORKERS and FADODB_FUZZ_MAXTEXT, with the defaults for what is unset.
type Formula ¶
type Formula struct {
// Seed reproduces the formula: New(Seed).Formula() generates it again.
Seed uint64
Src string
Inputs []formula.Value
Ctx formula.Context
}
Formula is one generated formula with the inputs and the context it runs under. Inputs has one value per entry of Variables.
func (*Formula) CaseJSON ¶
CaseJSON renders the formula with its inputs and context as a suite case whose expectation is the given value or error code, ready to paste into conformance/formula. The suite variables are those of Variables.
func (*Formula) DescribeContext ¶
DescribeContext renders the context.
func (*Formula) DescribeInputs ¶
DescribeInputs renders the inputs as name=value pairs, blanks left out.
type Generator ¶
type Generator struct {
// contains filtered or unexported fields
}
Generator produces formulas from a seed. It is not safe for concurrent use; make one per goroutine.
Noise and quiet: a power, an exponential or a logarithm carries last-bit noise between the targets, which the runner tolerates in a number and in its rendering (Tolerance). It cannot tolerate it once a discontinuous operation has amplified it: a comparison, MOD, SIGN, rounding, a date part, a text position, a count. So those operations ask for quiet arguments, and inside a quiet subtree the generator never produces a noisy function. The conformance suite covers the shapes this leaves out, such as ROUND of a power, with values away from any rounding boundary.
type Outcome ¶
type Outcome struct {
Prog *formula.Program
// BindErr is set when the formula did not bind: a generator bug.
BindErr error
// VM is the VM's value, and VMCode its error code when it failed.
VM formula.Value
VMCode string
// SQL is the dialect's result; SQLErr is set when the statement
// itself failed, which is a divergence too.
SQL formula.SQLResult
SQLErr error
// Reason says how the targets disagree; "" when they agree.
Reason string
}
Outcome is one formula run on the VM and on a dialect.
func Run ¶
Run binds f, runs it on the VM and through db on the dialect, and compares the two by the rules of the conformance suite: error codes must match and values must be equal, except that a Noisy program's number, or its rendering inside a text, may differ by Tolerance.
type Stats ¶
type Stats struct {
Formulas int
// VMErrors counts formulas the VM ended with an error code; the
// rest produced a value.
VMErrors int
// Divergences counts formulas the targets disagreed on; Reports holds
// the first few, rendered.
Divergences int
Reports []string
// Brief has one line per divergence, seed, reason and formula, for
// the first maxBrief of them, to see the classes at a glance.
Brief []string
Elapsed time.Duration
}
Stats is what a campaign found.