Documentation
¶
Overview ¶
Package factory is the producer half of the RFC-201 §5 generation factory: generate → execute → bless-or-file → dedup, with cmd/factory-run driving it and pkg/relational/conformance/factorycorpus owning the committed output.
The split matters. Nothing in the corpus package imports this one, so a committed test keeps running when the generator is broken, replaced or deleted — which is the entire argument for committing generated tests rather than regenerating them (§5, "Why commit rather than regenerate").
Generation is not grammar enumeration. It extends rowdiff's typed-spec generator: seeded PCG, a Case/Query/Pred/JoinSpec/AggSpec struct family, and renderers from those structs to SQL. That is the proven substrate in this tree, and it hands the factory something an ANTLR walk would not — the SPEC STRUCT IS THE FEATURE VECTOR. A grammar walker knows which productions it fired; the spec struct knows the query's whole shape, including facts no production carries (which columns are indexed, whether the join is an ON form or a comma form, whether a leaf is sargable), and those are the axes coverage has to be measured on.
Index ¶
- Constants
- Variables
- func FeatureVector(c *rowdiff.Case, q rowdiff.Query, projection []string) string
- func InapplicabilityLedger() []string
- func PinConn(ctx context.Context, db *sql.DB, disabledRules []string) (*sql.Conn, error)
- func SecondPlanRules() []string
- func Truncate(body string, limit int) string
- func WriteFindings(dir string, findings []*Finding) error
- func WriteManifest(path string, m Manifest) error
- type Batch
- type Candidate
- type CrossEngine
- type Finding
- type Manifest
- type OracleStats
- type Outcome
- type Runner
- type Sweep
Constants ¶
const FactoryRowCap = 24
FactoryRowCap bounds a case's row count.
rowdiff generates 20-120 rows because its comparison is in-memory and free. The factory's rows are WRITTEN INTO EVERY FILE, once as an INSERT and again as expectations, so the same number would make a thousand-file batch unreviewable — and an expectation nobody can read is not a reviewable proof, which is half the argument for committing them. The cap keeps the NULL rate (~1 in 6 per nullable column) and the duplicate-heavy 0..9 integer domain intact, which are the properties the predicates actually exercise; it costs the rarer boundary values (-1 and 2^62, ~1 in 20), which the un-capped rowdiff nightly still sweeps.
const GeneratorVersion = "rowdiff-gen/2"
GeneratorVersion identifies the candidate-derivation rules. It is recorded in every emitted file and MUST be bumped whenever a change here would make a seed produce a different candidate — otherwise a committed file claims a reproduction recipe that no longer reproduces it, and TestFactoryDeterminism is the thing that notices. gen/2: the emitted schema template dropped the scalar `NOT NULL` on the id column (rejected at CREATE since the DdlVisitor parity port — scalar non-nullability is unrepresentable in RecordMetaData), so every seed's DDL text changed; the committed corpus was rewritten in the same change.
const MaxExpectationRows = 80
MaxExpectationRows bounds how many rows a committed test may assert.
Cross-join shapes can return thousands of rows for a 24-row table, and a file holding a 4,000-row expectation is a diff nobody reviews and a regression nobody can localise. Over-large candidates are DROPPED with a counted reason, never truncated: a truncated expectation would be a silently weaker assertion wearing a frozen expectation's clothes.
const NestedGeneratorVersion = "rowdiff-nested/2"
NestedGeneratorVersion identifies the NESTED candidate-derivation rules.
It is a second version string rather than a bump of GeneratorVersion because the two case families are derived by different entry points over disjoint candidate namespaces: a change to the nested rules must not declare the hundreds of committed FLAT scenarios unreproducible, and vice versa. Every consumer that dispatches on a committed header's generator therefore reads this constant, never a substring of it.
It is also the ONLY declaration of it. There used to be a second, in rowdiff, and the two disagreed: the rowdiff copy had no readers at all — nothing dispatched on it, nothing stamped it — so a bump applied there changed no committed header and the corpus went on claiming a generator version the build did not have. A version constant with no reader cannot be wrong loudly, which is the only way a version constant is useful.
nested/2: the derivation rules changed — a recursive struct model (a struct inside a struct, so `n.dp.a` is three segments), table-derived column pools in place of literal lists, and projection variants reaching depth 3. Every nested seed therefore yields different candidates than nested/1 did, and the committed batch was re-stamped in the same change.
const PRBodyLimit = 65536
PRBodyLimit is GitHub's hard cap on a pull-request body, in characters. createPullRequest rejects anything longer with "Body is too long (maximum is 65536 characters)" — a GraphQL error, so it arrives AFTER the branch has been committed and pushed. The batch is already on the remote when the PR fails, which is why this has to be enforced before the call rather than retried after it.
Variables ¶
var TLPLabels = []string{"unfiltered", "p", "not-p", "p-is-null"}
TLPLabels names the four renderings, in TLPQueries order, for diagnostics.
Functions ¶
func FeatureVector ¶
FeatureVector serializes the structural identity of a candidate: everything about the query except its literal VALUES.
Erasing literals is the whole design. `a > 3` and `a > 7` are the same test; committing both is the "90k variants of one shape" §5.4 names as volume without coverage. What survives is what a coverage claim can be made about: the query family, the predicate tree's structure and leaf kinds, the sort and projection shape, and the index set the planner had available — that last one because the same predicate over an indexed and an unindexed column are genuinely different tests of the engine.
The rendering is stable and sorted wherever order is not semantic, so the same shape always produces the same string and the census can count on it.
func InapplicabilityLedger ¶
func InapplicabilityLedger() []string
InapplicabilityLedger renders the ledger for a run's manifest, so a batch says out loud which exemptions were available to it.
func PinConn ¶
PinConn returns a pinned connection with the given planner options applied. Pinned because the option lives on the connection: handing the query back to the pool would run it on a fresh, unconfigured one.
func SecondPlanRules ¶
func SecondPlanRules() []string
SecondPlanRules is the disabled-rule set the second-plan oracle pins.
func Truncate ¶
Truncate bounds a body at limit CHARACTERS, leaving an explicit marker rather than a silent cut.
GitHub counts characters, not bytes, so the budget is spent in runes; cutting by byte offset would also risk splitting a multi-byte rune and emitting invalid UTF-8. The marker is mandatory: a body that stops mid-sentence with no marker reads as a generator bug and sends the reader looking for a defect that is not there, while a marked cut sends them to the artifact.
func WriteFindings ¶
WriteFindings persists oracle disagreements. Each one is a potential engine bug and the run's exit code reports them; auto-minimization is not yet built, so the persisted record carries the full reproduction — seed, DDL, setup and every query — rather than a shrunk one.
func WriteManifest ¶
WriteManifest persists the manifest next to the batch.
Types ¶
type Batch ¶
type Batch struct {
// Dir is the corpus testdata directory family files are written into.
Dir string
// Quota caps committed scenarios per run. §5.2: per-run quotas keep PRs
// reviewable; the steady state is months of batches, not one dump.
Quota int
// contains filtered or unexported fields
}
Batch accumulates a run's committed output and enforces the dedup rule.
func NewBatch ¶
NewBatch prepares a batch against the corpus already committed in dir.
Seeding the dedup set from the EXISTING corpus is what makes the factory cumulative rather than repetitive: without it every run would re-commit the shapes the last run already covered, and a year of nightlies would produce one night's coverage a hundred times over.
func (*Batch) CountCandidate ¶
func (b *Batch) CountCandidate()
CountCandidate records a generated candidate.
type Candidate ¶
type Candidate struct {
Seed uint64
QueryIndex int
ProjIndex int
Case *rowdiff.Case
Query rowdiff.Query
Projection []string
// Nested marks a candidate derived by NestedCandidates: its case carries a
// struct column and its projections name dotted paths. It selects the
// generator version stamped into the emitted header, and it keeps the file
// NAME disjoint from the flat family so the two can share a seed number.
Nested bool
FeatureVector string
}
Candidate is one generation unit — a case, one query within it, one projection variant — that the TLP oracle can be built over.
func Candidates ¶
Candidates enumerates a seed's TLP-eligible candidates.
Eligibility is narrow, and every exclusion is a property of the ORACLE, not a limitation of the engine — the excluded shapes are covered by rowdiff's Oracle M and by the hand-authored corpus:
- No WHERE: nothing to partition on.
- Aggregate: the output is one row per group, not a row set the input partition maps onto. `COUNT(*)` over p, NOT p and p IS NULL sums to the unfiltered count, but SUM/MIN/MAX do not, and an oracle that holds for some functions is not an oracle.
- UNION and derived-table: they render through their own SQL paths, which the WHERE override does not reach. Silently rendering them unmodified would produce four identical queries and a partition that "holds" trivially — a tautological oracle, the exact instrument failure class RFC-201 §8.5 gates against.
- LIMIT or OFFSET: the partition is over the FULL result; truncating or skipping a prefix of each branch independently does not reassemble. OFFSET is guarded on its own and not left to ride on LIMIT. Today the generator only ever draws an offset alongside a limit, and the renderer only emits OFFSET inside a LIMIT clause, so an offset-only query is currently unreachable — but eligibility is a claim about whether the PARTITION PROPERTY HOLDS for a query spec, and it must not be true only because of what a renderer two files away happens to drop on the floor.
- DISTINCT: dedup does not distribute over a partition. A value appearing in both the p and the NOT-p branch survives once in each branch and once in the union — three rows on the left, one on the right.
func CandidatesForGenerator ¶
CandidatesForGenerator derives a seed's candidates under the generator a committed header names.
The determinism gate reads it. Dispatching there on the header's declared generator is what keeps the gate honest across two families: matching on the build's single GeneratorVersion would make every nested scenario report "generator drifted" — a red that says nothing about reproducibility — while ignoring the field would check a nested file against a flat candidate and report a drift that is really a lookup error. Unknown generator returns ok=false so a header naming a generator this build does not have FAILS rather than being silently regenerated by the wrong one.
func NestedCandidates ¶
NestedCandidates enumerates a seed's TLP-eligible NESTED candidates: the same derivation as Candidates, over a case whose table carries a struct column.
Eligibility is the SAME predicate, deliberately. A nested query is not a different oracle problem — the ternary-logic partition is a property of the WHERE, and a WHERE over `n.a` partitions exactly as one over `a` does — so giving the nested family its own eligibility rules would be inventing a second, less-tested oracle for no reason.
func (Candidate) GeneratorVersionOf ¶
GeneratorVersionOf is the generator string the candidate's header declares.
func (Candidate) Name ¶
Name is the candidate's stable file name: seed, query and projection, which together identify it uniquely and reproducibly.
func (Candidate) Ordered ¶
Ordered reports whether the candidate's query fixes a row order.
It is a method rather than an expression written twice because TWO decisions depend on it and they must never diverge: how the second-plan oracle compares the two plans' rows, and how the committed scenario asserts them (yamsql.Test.Unordered). Freezing an exact sequence that no oracle ever checked as a sequence is a frozen expectation with nothing behind it, and that is precisely what two independently written `len(q.OrderBy) > 0` checks would eventually produce.
It reads the candidate's ORIGINAL query, which is correct for all four TLP renderings: the WHERE override replaces only the predicate conjunct and never touches ORDER BY.
func (Candidate) TLPQueries ¶
TLPQueries returns the four renderings of the ternary-logic partition, in the order (base, p, NOT p, p IS NULL).
The triple is emitted FROM THE TYPED PREDICATE SPEC, not by rewriting parsed SQL. That is a deliberate choice and the cheaper one by a wide margin: a rewriter has to re-parse the query, find the WHERE, and reproduce operator precedence when it wraps — three places to be subtly wrong, each of which turns a correct engine into a reported bug. The generator already HAS the predicate as a tree; rendering it three more ways is the same renderer called three more times.
Composition with the rest of the WHERE is what makes this valid on more than toy queries. The override replaces only the predicate-tree conjunct, so a comma join's equality and an appended EXISTS stay identical across all four renderings and factor out of the partition exactly as the FROM clause does.
type CrossEngine ¶
type CrossEngine interface {
Rows(ctx context.Context, schemaTemplate string, setup []string, query string) ([][]any, error)
}
CrossEngine is the Java leg. plandiff.SetupRunner satisfies it; the indirection keeps this package free of a hard dependency on a reachable Java server, so a run without one degrades to metamorphic blessing rather than failing.
type Finding ¶
type Finding struct {
Oracle string `json:"oracle"`
Seed uint64 `json:"seed"`
Candidate string `json:"candidate"`
Detail string `json:"detail"`
DDL string `json:"ddl"`
Setup string `json:"setup"`
Queries []string `json:"queries"`
Left string `json:"left"`
Right string `json:"right"`
}
Finding is an oracle DISAGREEMENT: two things that must agree did not.
A finding is never committed as a test — freezing a disagreement would pin whichever side happens to be wrong. It is persisted with everything needed to reproduce it and it fails the run's exit code, under the standing rule that a red safety net is triaged, not filed.
type Manifest ¶
type Manifest struct {
Date string `json:"date"`
Generator string `json:"generator"`
SeedStart uint64 `json:"seed_start"`
Seeds uint64 `json:"seeds"`
Quota int `json:"quota"`
BlessingMode string `json:"blessing_mode"`
CandidatesGen int `json:"candidates_generated"`
CandidatesRun int `json:"candidates_executed"`
Blessed int `json:"blessed"`
Committed int `json:"committed"`
// DedupRejected counts blessed candidates dropped because their (feature
// vector, plan shape) point was already covered. A high number is HEALTHY
// — it is the frontier flattening — and a zero is the number to be
// suspicious of, because it means the key is discriminating on something
// that varies per candidate.
DedupRejected int `json:"dedup_rejected"`
// DedupPoints is the number of distinct points the committed corpus covers
// after this batch.
DedupPoints int `json:"dedup_points"`
// NameCollisions counts blessed candidates whose file name was already
// taken on disk. It is called out as its own field rather than left in the
// skip ledger because it is the one skip class that means a committed
// scenario was nearly destroyed: the file name and the dedup key are
// independent, so a planner change moves the key (dedup misses) without
// moving the name (the write would clobber). A non-zero value is a
// statement about the ENGINE, not about the batch.
NameCollisions int `json:"name_collisions"`
// SkipsByReason is the counted-skip ledger. Every candidate that did not
// become a test appears here under a reason class; a run whose skips do
// not account for its candidates is a run with a silent drop in it.
SkipsByReason map[string]int `json:"skips_by_reason"`
// SkipSamples holds a few verbatim messages per reason class. A class with
// a count and no example is a class nobody can triage: "second-plan infra
// 28" is indistinguishable between a transport hiccup and the disabled-rule
// option being accepted and ignored, and those are a shrug and an emergency.
SkipSamples map[string][]string `json:"skip_samples"`
Oracles OracleStats `json:"oracles"`
Blessings map[string]int `json:"blessings"`
// Exemptions names the structural-inapplicability families available to
// this run, each with the pin establishing it and the oracle that would
// retire it. A batch that blessed anything on fewer oracles than the rest
// says so here, in the same document as the counts — an exemption a
// reader has to go find in the source is one they will not.
Exemptions []string `json:"exemptions"`
// BlessedByFamily counts TLP-only blessings per exemption family, so the
// SIZE of each weakening is visible next to its justification.
BlessedByFamily map[string]int `json:"blessed_by_family"`
Findings int `json:"findings"`
Census factorycorpus.Census `json:"census_after"`
}
Manifest is a factory run's whole output ledger — the numbers RFC-201 §5.1 asks every stage to report and §5.5's PR description carries.
It is a ledger, not a summary: a run that commits nothing must still say what it saw and why nothing survived, because "the batch was empty" and "the generator produced nothing" and "everything deduplicated" are three different states and only the first is healthy.
func (Manifest) SummaryMarkdown ¶
SummaryMarkdown renders the batch's ledger as a bounded PR body.
WHY A SUMMARY AND NOT THE MANIFEST. The manifest embeds Census, and Census carries two maps with ONE ENTRY PER SCENARIO IN THE WHOLE COMMITTED CORPUS — ByFeature (a long feature-vector string per key) and ByKeyBlessing. That is whole-corpus state, not this batch's output, and it crossed GitHub's 65536 limit at roughly 500 scenarios. At 5000 scenarios the manifest is ~640KB, nearly ten times the cap, and it grows by up to one quota (default 1000) every night. Embedding it in a PR body was never going to keep working.
WHAT IS NOT DROPPED, which is the whole point. Every number RFC-201 §5.1 asks a stage to report survives: the seed range, all four funnel counts, the dedup stats, the name-collision count, the full skip ledger WITH its samples, the oracle stats, the blessing mix, the exemption families, the finding count, and the census SCALARS. What is left out is exactly the two per-scenario maps — and the reader is told so, by name, with the artifact that has them. A summary that quietly omitted a count would be worse than the failure it replaces, because a failed PR is loud and a missing number is not.
runURL should be the workflow run's URL so the pointer is actionable; an empty value renders the artifact name alone rather than a dangling link.
type OracleStats ¶
type OracleStats struct {
// TLPChecked counts candidates whose partition property was evaluated.
TLPChecked int `json:"tlp_checked"`
// TLPViolations counts partitions that did not reconstruct — each is a
// finding, not a skip.
TLPViolations int `json:"tlp_violations"`
// SecondPlanKept counts query executions where the second plan genuinely
// differed and the rows were compared.
SecondPlanKept int `json:"second_plan_kept"`
// SecondPlanSkipped counts executions where disabling the rule left the
// plan unchanged, so there was no second plan to compare against. Counted,
// never silent — and floored against SecondPlanKept at the run level,
// because "every case skipped" is exactly what the option being accepted
// and ignored would look like.
SecondPlanSkipped int `json:"second_plan_skipped"`
// SecondPlanViolations counts row disagreements between two plans for the
// same query — the strongest planner-bug signal this harness has.
SecondPlanViolations int `json:"second_plan_violations"`
// CrossEngineKept / CrossEngineViolations count the Java leg.
CrossEngineKept int `json:"cross_engine_kept"`
CrossEngineViolations int `json:"cross_engine_violations"`
// TLPOnlyBlessed counts candidates blessed on TLP alone because the
// second-plan oracle was STRUCTURALLY inapplicable to their shape. It is
// reported so the size of the exemption is visible rather than inferred
// from the difference between two other counters.
TLPOnlyBlessed int `json:"tlp_only_blessed"`
}
OracleStats counts what each oracle actually DID across a run.
Every field here is reported and floored by the caller. RFC-201 §8.5: an oracle whose instrument goes dark is indistinguishable from an oracle that passes, and three of the RFC-197 campaign's ledger entries were instrument failures rather than engine bugs.
type Outcome ¶
type Outcome struct {
Candidate Candidate
Blessed bool
Header factorycorpus.Header
Scenario *yamsql.Scenario
Skip string
Findings []*Finding
Stats OracleStats
// InapplicableFamily names the structural-inapplicability family that
// permitted a TLP-only blessing, or "" when every applicable oracle ran.
InapplicableFamily string
}
Outcome is one candidate's result. Exactly one of Blessed, Skip and Finding is meaningful; a candidate that is neither blessed nor skipped nor a finding is a bug in this file.
type Runner ¶
type Runner struct {
// DefaultConn plans with the full rule set.
DefaultConn *sql.Conn
// SecondPlanConn has secondPlanDisabledRule disabled, so the same query
// takes a different access path.
SecondPlanConn *sql.Conn
// Java, when non-nil, blesses cross-engine.
Java CrossEngine
// Date is the batch date stamped into every header. It is an INPUT, never
// read from the clock: §5.4 forbids wall-clock in the generation path, so
// a committed file can be regenerated byte-identically.
Date string
}
Runner executes candidates against a live schema.
type Sweep ¶
type Sweep struct {
// SetupDB executes the schema DDL; it must already point at an existing
// database path.
SetupDB *sql.DB
DBPath string
// ClusterFile is the path the per-schema query connections open.
ClusterFile string
// Java, when non-nil, adds the cross-engine oracle.
Java CrossEngine
// Date is stamped into every header.
Date string
// Filter, when non-nil, restricts the sweep to the candidates it accepts.
// It exists for re-emission (the format migration regenerates exactly the
// committed (seed, query, projection) triples): re-running a seed evaluates
// every candidate it derives, and the ones the original batches
// dedup-rejected must not be offered now, or a re-emission would grow the
// corpus as a side effect of rewriting it.
Filter func(Candidate) bool
// Nested sweeps the NESTED candidate family instead of the flat one: same
// seeds, a different derivation, a disjoint file namespace. It is a mode
// rather than an additional pass because a seed materializes ONE schema and
// the two families need different ones.
Nested bool
}
Sweep runs whole seeds: it materializes each seed's case once and evaluates every candidate derived from it against that one schema.
One schema per SEED, not per candidate. A seed's candidates share a table, a row set and an index set — they differ only in the query — so a schema per candidate would pay the CREATE SCHEMA TEMPLATE / CREATE SCHEMA / INSERT cost several times over for identical state, and that cost dominates a factory run (the queries themselves are milliseconds against a warm container).
func (Sweep) RunSeed ¶
RunSeed evaluates every candidate of one seed and offers each outcome to the batch. A seed with no TLP-eligible candidate is not an error — the generator emits aggregates, unions and LIMIT queries the partition oracle declines by construction — so it returns cleanly with nothing offered.