shapegen

package
v0.18.49 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 5, 2026 License: AGPL-3.0 Imports: 8 Imported by: 0

Documentation

Overview

Package shapegen generates random-but-valid SQL over a described schema, aimed at the shapes a BI client emits and the fixed benchmark corpus does not contain.

It is a second generator alongside sqlgen, not a replacement: sqlgen targets the distributed-execution breakers (aggregate-free GROUP BY, dedup past a shuffle, scalar-subquery HAVING thresholds) with a flat clause model. shapegen targets NAME RESOLUTION and ORDER BY — table aliases, self-joins, quoted identifiers, star projections, aliases that shadow real columns, ordering by expressions/aliases/ordinals/hidden columns — which needs a query model that knows which output column carries which value. That model is also what lets every generated query declare how its result may be compared without flaking (see Query.CompareSpec).

Generation is fully determined by the seed: a failing seed IS the repro.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Column

type Column struct {
	Name string
	Kind Kind
	Lits []string
	// Bool marks a BOOLEAN column, which is a fact about the DOMAIN rather
	// than about the operators Kind selects: a BOOL is KindOpaque for every
	// arm that needs arithmetic or a string function, and is the ONE column
	// shape that can stand alone as a predicate. genBarePredicate is the only
	// reader.
	Bool bool
}

Column is one generatable column. Lits are rendered SQL literals drawn from the column's real domain, so predicates are selective without being empty.

type Edge

type Edge struct {
	LTable, LCol string
	RTable, RCol string
}

Edge is a joinable equality (FK pairs in practice).

type From

type From struct {
	Table   string // base table, or the CTE name
	Derived string // rendered subquery, for an inline derived table
	Alias   string
	Join    string // "JOIN", "LEFT JOIN", ","
	On      string
	// PK is a column set unique within this entry's rows, as reachable from
	// Alias. Empty when the entry exposes no unique key, which is what makes
	// an ordering non-total.
	PK []string
}

From is one FROM-clause entry. The first has an empty Join.

type Gen

type Gen struct {
	// contains filtered or unexported fields
}

Gen is a seeded generator. Identical (seed, schema) always yields the identical query.

func New

func New(seed int64, s *Schema) *Gen

New creates a generator.

func (*Gen) Query

func (g *Gen) Query() *Query

Query generates one query.

type Item

type Item struct {
	Expr string
	// Alias is the output column name. Empty only for Star items.
	Alias string
	// Star renders instead of Expr when set ("*" or "t.*").
	Star string
	// Exact marks a value both engines must produce bit-identically: a bare
	// column reference, or an integer-valued expression. Float arithmetic and
	// aggregates are not exact, so an ordering on them may legitimately break
	// ties differently.
	Exact bool
	Agg   bool
	// Opaque marks a value whose RENDERED form does not order the way the
	// value does: an IPv4 renders "10.0.0.9" and "10.0.0.10", which compare
	// lexicographically in the opposite order to the addresses. The absolute
	// order check reads rendered cells, so it cannot judge an ORDER BY over
	// one of these — see Query.OrderKeys.
	Opaque bool
}

Item is one select-list entry. Every generated item carries an explicit alias so output column names are unique and identical across engines — without that, results keyed by column name silently collapse duplicates and the two engines' default names for an expression differ.

type Kind

type Kind int

Kind classifies a column for operator, literal, and expression selection.

const (
	KindInt Kind = iota
	KindFloat
	KindText
	// KindDate is a text column holding ISO-8601 dates. Ordering and
	// comparison are identical to text, but date functions apply.
	KindDate
	// KindOpaque is a scalar whose DOMAIN the generator does not model: BYTES,
	// IPv4/IPv6/CIDR/MAC, UUID, TIMESTAMP, DURATION, PORT, PROTOCOL. The
	// generator projects it, groups by it, orders by it, joins on it, counts
	// it and MIN/MAXes it — everything it does to a column without knowing
	// what the values mean — and applies no arithmetic, no string function and
	// no date function to it.
	//
	// Nineteen of the engine's twenty-two types were unreachable by this
	// generator before this kind existed, because Kind was the whole type
	// universe and it had four members. A defect that needs a BYTES or an IPv4
	// column to fire could not be generated even in principle.
	KindOpaque
	// KindDecimal is exact-decimal numeric. Numeric for the arms that need a
	// number (SUM, AVG, arithmetic), but NOT interchangeable with float: its
	// ordering, its group-key encoding and its rendering all go through
	// separate code from FLOAT64's.
	KindDecimal
)

type Order

type Order struct {
	Expr string
	Desc bool
	// Key is the output column carrying this term's value, or "" when the
	// term orders by something the select list does not project.
	Key string
	// Exact marks a term both engines compute bit-identically (see Item).
	Exact bool
	// Opaque marks a term the absolute order check cannot judge (see Item).
	Opaque bool
}

Order is one ORDER BY term.

type Query

type Query struct {
	With     string // rendered "WITH x AS (...)" prefix, or ""
	Distinct bool
	Items    []Item
	From     []From
	Where    []string
	GroupBy  []string
	Having   string
	Order    []Order
	Limit    int
	Offset   int
	// LimitZero renders `LIMIT 0`, the boundary Limit cannot carry: zero is
	// how every other field here spells "no LIMIT", so the generator could
	// not emit the one value with a rule of its own (#487). `LIMIT 0` returns
	// no rows whatever the ORDER BY, on every engine, so it is the shape that
	// separates "the limit bound" from "the limit was ignored" — and it is
	// also the shape a paginating client sends to fetch a result's SHAPE
	// without its rows.
	LimitZero bool
	// TotalOrder records that the generator appended a uniqueness tiebreaker,
	// so no two output rows tie on the full ORDER BY list.
	TotalOrder bool
	// Shape tags the generator arm that produced this query, for coverage
	// reporting.
	Shape string
	Seed  int64
	// contains filtered or unexported fields
}

Query is a generated query in structured form, so a failure can be shrunk structurally instead of by text mutation.

func Shrink

func Shrink(s *Schema, q *Query, fails func(*Query) bool) *Query

Shrink reduces q to the smallest query for which fails still reports true ("still reproduces the divergence"). It applies structural removals — LIMIT, OFFSET, ORDER BY terms, HAVING, DISTINCT, WHERE conjuncts, select items, unreferenced trailing joins — accepting any removal that keeps the failure, until a full pass removes nothing.

Reporting the unshrunk query is close to useless: "some 8-table query differs" is not a defect report. The shrunk query is.

func (*Query) Clone

func (q *Query) Clone() *Query

Clone returns a deep copy of q.

func (*Query) CompareSpec

func (q *Query) CompareSpec() oracle.CompareSpec

CompareSpec derives how this query's results may be compared. This is the harness's trust boundary: a mismatch under the returned spec is always a defect, never an artifact of SQL's under-determination.

func (*Query) OrderKeys

func (q *Query) OrderKeys() []oracle.OrderKey

OrderKeys returns the ORDER BY terms whose values the result projects, in ORDER BY sequence. It stops at the first term the select list does not carry, since an unprojected key makes every later term unverifiable.

func (*Query) SQL

func (q *Query) SQL() string

SQL renders the query.

type Schema

type Schema struct {
	Tables []Table
	Edges  []Edge
}

Schema is the generation universe.

func TPCH

func TPCH() *Schema

TPCH is the generation universe for the TPC-H schema as Wadjet stores it: monetary columns are FLOAT64, date columns are ISO-8601 STRINGS (so ordering and comparison are lexicographic, which agrees with date semantics), and every table declares the column set that makes a row unique.

Literal pools are drawn from the real SF0.01 domains so predicates are selective without being always-empty — a generator whose every WHERE returns zero rows compares nothing.

func TypeMatrix

func TypeMatrix() *Schema

TypeMatrix is the generation universe over the type-matrix fixture (internal/oracle/typematrix): the same two tables that package's corpus uses, described so the generator can put EVERY wadjet type through the shapes it knows — projection, GROUP BY, ORDER BY, DISTINCT, joins, set operations, aggregates, window functions, predicates.

TPCH() spans three storage types, so nineteen of the engine's twenty-two were unreachable by this generator: a defect that needs a BYTES, IPv4, UUID or DECIMAL column to fire could not be generated even in principle. This schema closes that, and it is DERIVED from typematrix.Columns() rather than written out, so a type added there is generated here without a second edit.

The four container types (ARRAY, ROW, MAP, VECTOR) are deliberately NOT exposed as generatable columns: the generator would order and group by them, which is not defined, and the engine's answer to that is a separate question from the one this generator asks. The nested table is present for its id and group key, so joins across the two still happen.

type Table

type Table struct {
	Name string
	PK   []string
	Cols []Column
	// SelfJoin marks a table small enough that joining it to itself stays
	// bounded. genSelfJoin picks only from these; a schema with none gets the
	// single-table FROM instead.
	SelfJoin bool
}

Table is one generatable table. PK is a column set unique within the table — the generator appends it to ORDER BY to make an ordering total, which is what lets a LIMIT-ed or positionally-compared query be deterministic.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL