oracle

package
v0.18.46 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 5, 2026 License: AGPL-3.0 Imports: 11 Imported by: 0

Documentation

Overview

Package oracle is the optimization-invariance differential harness (#287): every corpus query runs with all registered optimizations enabled (baseline), then once per optswitch toggle with just that optimization disabled, then with all of them disabled — and every configuration must produce identical results. An optimization that changes answers is a bug by definition; a divergence names the toggle, which is the defect localization.

The harness is corpus-agnostic: callers supply the queries and a RunFunc, so the TPC-H and ClickBench suites (and later a query generator) plug in the same way.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CheckOrder

func CheckOrder(res *Result, keys []OrderKey) string

CheckOrder reports the first row that breaks the ordering the query asked for, or "" when the sequence holds. It is an ABSOLUTE check — no second engine — so it catches an ORDER BY that every comparison arm drops the same way. Keys must name columns the result projects.

NULL policy: PostgreSQL's, which is what wadjet implements and what the DuckDB arm CONFIGURES the reference engine to use (default_null_order='nulls_last_on_asc_first_on_desc') — NULLS LAST for ASC, NULLS FIRST for DESC. ADR-0012 makes PostgreSQL the authority on semantics, so the absolute checker has to encode the same rule the comparison arms are configured for.

It did not. Until this was corrected, compareRowsByKeys placed NULLs last in BOTH directions and explicitly refused to flip a NULL-involving pair under DESC, so a DESC result with NULLs FIRST — the correct answer — was reported as an ordering failure. Nothing caught it because TPC-H contains no NULLs at all, so the fixed corpus and the shape fuzzer over it never put a NULL in a sort key. The type-matrix fixture, whose every column nulls on its own stride, hit it on its second generated seed.

func Compare

func Compare(want, got *Result, spec CompareSpec) string

Compare returns "" when got matches want under spec, else a description of the first difference. want is the reference arm (DuckDB, the fast path, or the all-optimizations-on baseline).

func ParseCell

func ParseCell(s string, ref any) any

ParseCell converts one string cell (a CSV field from an external engine) into the Go type the reference result uses for that column, so canonical rendering compares like with like. ref is a non-nil sample value from the same column, or nil when the column was all-NULL on the reference side.

func RunDifferential

func RunDifferential(ctx context.Context, t *testing.T, queries []Query, run RunFunc)

RunDifferential drives the oracle. It temporarily forces every registered toggle on for the baseline (host env notwithstanding, so a stray kill switch can't silently weaken the oracle), restores prior state on return, and reports each divergence as a failed subtest named after the disabled toggle.

func TextCell

func TextCell(s, null string) any

TextCell types one cell of a reference engine's TEXT output (a CSV field, say) for FingerprintOf. Only a value carrying a fraction or an exponent becomes a float — that is where quantization is needed and where the two engines' last digits legitimately differ. Integer-looking text stays text, which is exactly how fingerprintCell renders an integer, so "1234567" digests identically whether it arrived as text, as an int64, or as a float64; and a string that merely looks integral ("007", a country code) keeps its own spelling.

null is the reference engine's spelling of NULL; text equal to it becomes a nil cell. An empty field is the empty STRING, never NULL — conflating the two is how a NULLed column reads as blank and passes.

Types

type Canon

type Canon struct {
	// contains filtered or unexported fields
}

Canon is a result in canonical comparable form. Exported so other differential harnesses (the standalone-vs-distributed oracle, #288) share the same comparison semantics as the kill-switch oracle.

Rows are rendered at two float precisions. Rounding floats can land a value exactly on a rounding boundary, where one ULP of accumulation- order noise between two CORRECT runs flips the rendered digit (observed: SUM at 14903.55 rendering 14903.5 vs 14903.6). A divergence therefore only counts when it survives BOTH precisions — real bugs (missing rows, nulls, dropped limits, wrong groups) differ at every precision, while a boundary hit at two independent quanta simultaneously is vanishingly rare. The cost: a float-only error smaller than the coarse quantum (~1e-4 relative) is absorbed; row-membership and integer errors are unaffected.

func Canonicalize

func Canonicalize(res *Result) *Canon

Canonicalize renders every row to a stable string: cells in Columns order, floats at 6 significant digits so accumulation-order noise between two correct runs doesn't register as divergence. Rows are sorted, so the comparison is order-insensitive.

func CanonicalizeOrdered

func CanonicalizeOrdered(res *Result) *Canon

CanonicalizeOrdered is Canonicalize without the sort: rows stay in the order the engine returned them. Use it when a top-level ORDER BY makes the row SEQUENCE part of the answer — comparing sorted canonical forms there would accept a result that lost its ordering.

func (*Canon) Diff

func (b *Canon) Diff(other *Canon, q Query) string

Diff returns "" when other matches, else a description of the first difference.

func (*Canon) Rows

func (b *Canon) Rows() int

Rows returns the number of canonical rows.

type CmpMode

type CmpMode int

CmpMode selects how strictly two results are compared.

const (
	// CmpUnordered compares the row multiset; row order is ignored.
	CmpUnordered CmpMode = iota
	// CmpOrdered compares rows positionally: the sequence is part of the answer.
	CmpOrdered
	// CmpCount compares row counts only.
	CmpCount
)

func (CmpMode) String

func (m CmpMode) String() string

type CompareSpec

type CompareSpec struct {
	Mode CmpMode
	// Limit, when > 0, is the trailing LIMIT both sides must respect.
	Limit int
	// OrderKeys are the projected ORDER BY keys, in ORDER BY sequence. Under
	// CmpUnordered they turn into a positional comparison of just those
	// columns, which is tie-immune. Empty when the keys are not projected.
	OrderKeys []OrderKey
}

CompareSpec is everything the harness needs to compare one query's results without producing a false positive.

type Fingerprint

type Fingerprint struct {
	// Rows is the row count; it is part of the identity, so a fingerprint
	// never matches a result of a different size.
	Rows int `json:"row_count"`
	// Fine and Coarse are the digests at 6 and 4 significant float digits.
	Fine   string `json:"fine"`
	Coarse string `json:"coarse"`
}

Fingerprint is a stored-comparable digest of a whole result: the row count plus one digest per precision of the canonical row rendering. It exists so a reference engine's answer can be committed to a file and compared later without that engine on the machine — the shape the DuckDB ground-truth gate needs.

Three properties the gate depends on, in the order they were learned the hard way:

  • Every column is covered, strings and NULLs included. A per-column numeric sum (internal/harness's value signature) skips string columns entirely, which is how a NULLed name column (#314) shipped green. Here a NULL renders "<null>", distinct from the empty string, and a string cell contributes its bytes.

  • Order sensitivity is the CALLER's decision, per query. With ordered set, the row sequence is part of the digest, so a dropped ORDER BY (#313/#316/#320) changes it; without, rows are sorted first, so an engine free to return them in any order is not held to one. Passing ordered for a query with no top-level ORDER BY would manufacture failures; passing it false for one that has an ORDER BY is the blind spot those three bugs walked through.

  • Float summation order does not move it. The rows are rendered at two precisions and a match at EITHER counts, which is the same dual-precision policy Canon.Diff applies and for the same reason (see the Canon doc comment): one ULP of accumulation noise can flip a rendered digit at a rounding boundary, but not at two independent quanta at once.

func FingerprintOf

func FingerprintOf(res *Result, ordered bool) Fingerprint

FingerprintOf digests res. ordered keeps the engine's row sequence in the digest; with it false the rendered rows are sorted first, so the digest is order-insensitive.

Cells render through fingerprintCell rather than canonCell: this digest is compared ACROSS engines, where integer-vs-float column typing is the reference engine's business and must not read as a divergence.

func (Fingerprint) Match

func (f Fingerprint) Match(got Fingerprint) (bool, string)

Match reports whether got is the same answer as f: identical row counts and agreement at either precision. The detail string names the first property that differs.

func (Fingerprint) String

func (f Fingerprint) String() string

type OrderKey

type OrderKey struct {
	Alias string
	Desc  bool
}

OrderKey names an output column that carries one ORDER BY key's value, together with the direction the query asked for.

type Query

type Query struct {
	Name string
	SQL  string
	// CountOnly relaxes the comparison to row counts within Tolerance.
	// Reserved for queries whose row MEMBERSHIP legitimately shifts with
	// float accumulation order (threshold comparisons over float
	// aggregates, the TPC-H Q02/Q22 class) — cell-exact comparison would
	// flake on borderline rows even between two correct runs.
	CountOnly bool
	Tolerance int
}

Query is one corpus entry.

func ExpandLimits

func ExpandLimits(queries []Query) []Query

ExpandLimits rewrites each LIMIT-ed query into two oracle entries: the stripped form (full-multiset compare — strictly stronger and tie-immune, same rationale as the DuckDB gate's stripLimit) and the verbatim form compared by row count only. At the LIMIT boundary any member of a tie group is admissible, so two correct runs can return different limited rows — but the row COUNT is deterministic, and running the verbatim query keeps LIMIT-dependent optimizations (top-N late materialization) exercised under the oracle. Queries already marked CountOnly keep that relaxation on the stripped form too.

type Result

type Result struct {
	Columns []string
	Rows    []map[string]any
	// RowValues is the same rows POSITIONALLY, cells aligned with Columns,
	// and it mirrors wadjet.QueryResult.RowValues: non-nil only when two
	// output columns share a name and the map therefore cannot hold both.
	// canonRowsWith reads it in preference to Rows when it is set.
	//
	// Without it a duplicate name silently SHRINKS the comparison: the
	// ClickBench corpus's Q30 is ninety columns all named `sum` — the map
	// holds one — so an oracle arm that moved eighty-nine of them would
	// have compared eighty-nine copies of the same surviving cell and
	// agreed with itself. Duplicate names became ordinary at 042f9852,
	// where an unaliased column started carrying the name PostgreSQL gives
	// it, and const-arith-agg and int-arith are exactly the toggles those
	// columns exercise.
	RowValues [][]any
}

Result is the minimal result shape the harness compares. Rows are column-name-keyed, matching wadjet.QueryResult.

type RunFunc

type RunFunc func(ctx context.Context, sql string) (*Result, error)

RunFunc executes one SQL query under the currently-set toggles.

Directories

Path Synopsis
Package collide is the COLLIDING-BARE-NAME fixture: three relations whose columns are all called c0, c1, c2, the way SQLancer names every schema it generates.
Package collide is the COLLIDING-BARE-NAME fixture: three relations whose columns are all called c0, c1, c2, the way SQLancer names every schema it generates.
Package dmlassign is the SET-value matrix for DML assignment casts, and the PostgreSQL answers it is checked against.
Package dmlassign is the SET-value matrix for DML assignment casts, and the PostgreSQL answers it is checked against.
Package multikey is the fixture and query corpus for correlated subqueries that correlate on MORE THAN ONE column.
Package multikey is the fixture and query corpus for correlated subqueries that correlate on MORE THAN ONE column.
Package shapegen generates random-but-valid SQL over a described schema, aimed at the shapes a BI client emits and the fixed benchmark corpus does not contain.
Package shapegen generates random-but-valid SQL over a described schema, aimed at the shapes a BI client emits and the fixed benchmark corpus does not contain.
Package sqlgen generates random-but-valid SQL queries over a described schema, for differential testing (#288).
Package sqlgen generates random-but-valid SQL queries over a described schema, for differential testing (#288).
Package typematrix is the fixture and query corpus for the type-coverage gates: one table carrying all 22 column types, and a generated corpus that pushes every type through every consumer that can retain, re-key, re-order or re-encode a value.
Package typematrix is the fixture and query corpus for the type-coverage gates: one table carrying all 22 column types, and a generated corpus that pushes every type through every consumer that can retain, re-key, re-order or re-encode a value.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL