Documentation
¶
Overview ¶
Package explaindiff renders the PHYSICAL plan shape of every query in the yamsql conformance corpus into a stable, diffable text baseline, and diffs two such baselines entry-by-entry.
Why it exists (RFC-183 §6): removing the nil-inner "shell" plans from the Cascades planner changes plan hashes and costs, which can silently FLIP which plan wins. A flipped-but-still-correct plan passes every row-level test — the corpus runner, rowdiff, and the 1M stress all stay green while plan quality regresses. Diffing the planned shape across the whole corpus is the only check that can see it. `plan_contains` (yamsql.go) is a per-case substring assert on ONE query; it cannot.
Relationship to the sibling harnesses — this is not a parallel pipeline:
- `conformance/plandiff` diffs Go against JAVA over its OWN in-code corpus, and its Go side is the explain-only LOGICAL generator (embedded.NewExplainOnlyGeneratorWithSchema → buildLogicalPlanFor*). It answers "do the two engines agree?", never "did Go's physical plan shape move?", and cannot be retargeted at the physical planner without invalidating every golden in its corpus.
- `conformance/yamsql` owns the corpus and its loader. This package consumes that loader — it never re-parses the YAML itself.
- The planner entry point is `embedded.PlanPhysicalForTest`, the same no-FDB full-Cascades harness the planner's own tests and the RFC-182 rowdiff harness plan through. It runs the same Cascades planner the driver runs, but it is NOT the production driver path: the driver enters through the sqldriver/connection stack and supplies live table statistics, while this harness calls the planner directly with planner-default statistics. Two plans that differ only because of that are expected; what the baseline pins is that the SAME harness input keeps producing the SAME plan.
NO FDB REQUIRED. Planning is metadata-only: the corpus scenario's `schema_template` is compiled to in-memory RecordMetaData and the query is planned against it. `setup:` INSERTs are not replayed, so table statistics are the planner defaults rather than the live cardinalities the driver would fetch from FDB. That is a deliberate trade: the baseline is a REGRESSION key (same input → same output → any delta is a real planner change), not a prediction of the plan the live driver picks for a populated table.
Index ¶
- Constants
- func Collect(dir string) ([]Entry, Stats, error)
- func CollectWithReachability(dir string, reach *cascades.ReachabilityCollector) ([]Entry, Stats, error)
- func NormalizeAliases(s string) string
- func Render(entries []Entry, st Stats) string
- func RenderDiff(r DiffReport) string
- func ShapeOf(p plans.RecordQueryPlan) []string
- func ValidateDiffInputs(oldB, newB *Baseline) error
- type Baseline
- type Change
- type Delta
- type DiffReport
- type Entry
- type Stats
Constants ¶
const MaxEntryCountDrift = 0.10
MaxEntryCountDrift is the largest relative gap tolerated between the two baselines' entry counts, as a fraction of the larger side. A real corpus edit moves a handful of stanzas out of thousands; a dump that was killed or aborted loses a whole tail. 10% sits far above the former and far below the latter.
const MinBaselineEntries = 100
MinBaselineEntries is the floor below which a baseline is treated as broken rather than small. The yamsql corpus plans ~2400 queries; a dump holding a double-digit count means the glob, the loader, or the dump run itself failed, and letting that compare CLEAN is the single worst failure mode this harness has — it reports "no plan changed" because it looked at nothing.
Variables ¶
This section is empty.
Functions ¶
func Collect ¶
Collect walks dir/*.yaml, loads every scenario through the yamsql loader, and plans every query AND DELETE/UPDATE test in it.
Two plannable classes are pinned:
- SELECT / WITH / VALUES (yamsql.IsQuery) → the SELECT harness (embedded.PlanPhysicalForTest). Counted in Stats.Queries.
- DELETE / UPDATE (isDML) → the DML harness (embedded.PlanPhysicalDMLForTest). Counted in Stats.DML. This closes the DML-blind gap: a planner change that reads clean on every SELECT while corrupting the DELETE-WHERE-EXISTS path (RFC-184 W2) is invisible to a SELECT-only dump.
Everything else (`exec:` INSERT and other sequencing steps) has no interesting winning plan and is counted in Stats.NonQuery so the corpus total still reconciles (Queries + DML + NonQuery == tests).
Scenarios whose schema_template does not compile yield one `<PLAN-ERROR: …>` entry per query in the file rather than vanishing — a schema regression must not silently shrink the baseline.
The returned slice is sorted by (file, index) and is byte-stable across runs: no map iteration, no timestamps, no addresses.
func CollectWithReachability ¶
func CollectWithReachability(dir string, reach *cascades.ReachabilityCollector) ([]Entry, Stats, error)
CollectWithReachability is Collect with RFC-183's yield-time plan-reachability accounting routed into the caller's collector. nil = collect nothing.
func NormalizeAliases ¶
NormalizeAliases renumbers generated correlation identifiers densely from 0, in order of first appearance WITHIN this plan. Distinctness is preserved — two different correlations stay different, and the same correlation referenced twice stays the same — so the rendering still carries the plan's aliasing structure; only the global counter's value is erased. The prefix's case is preserved because `q$` and `Q$` come from different rendering paths and a swap between them is a real change.
It is exported because any comparison of two EXPLAIN texts needs it, not just this package's baseline dump: the factory's second-plan oracle decides whether the disabled rule produced a DIFFERENT plan by comparing two EXPLAIN strings, and raw counter values would make two identical plans look different and count a tautological row comparison as a real one.
func Render ¶
Render writes entries in the baseline text format.
The format is line-oriented and entry-keyed so both `diff -u` and Parse work on it:
# explain-baseline/v2 # files=334 queries=2407 dml=276 non_query=45 plan_errors=255 unexpected_errors=4 # === aggregate_expr.yaml#3 sql: SELECT COUNT(*) FROM T plan: StreamingAgg(keys=[], Scan(T)) shape: RecordQueryStreamingAggregationPlan shape: RecordQueryScanPlan === bad_column.yaml#0 expect-error=42703 sql: SELECT nope FROM T plan: <PLAN-ERROR: 42703: Unknown column NOPE›
Every payload line carries its own prefix, so no field can be confused with a structural line and a multi-node shape stays greppable.
func RenderDiff ¶
func RenderDiff(r DiffReport) string
RenderDiff formats a DiffReport for a human. The summary leads with the counts that decide RFC-183's P0 exit criteria: structural flips and plan-time regressions.
func ShapeOf ¶
func ShapeOf(p plans.RecordQueryPlan) []string
ShapeOf renders the plan's structural skeleton: the Go type of each node, indented two spaces per level, in GetChildren() order (documented stable). A nil child renders as `<nil>` — RFC-183's shells are exactly that state, so the baseline must be able to say it out loud.
Exported because it is the repo's one STABLE-BY-CONSTRUCTION plan fingerprint: it carries no aliases, no counters and no version-dependent cost numbers, so the same plan renders identically across runs and across releases. plans.PlanHash is explicitly NOT that (plan_hash.go documents its value as free to change across releases), which is why the RFC-201 factory dedups on this rendering instead. Type identity alone is COARSE — two plans differing only in which index or predicate they use collapse together — so a dedup key must pair it with a description of the query, never use it alone.
func ValidateDiffInputs ¶
ValidateDiffInputs rejects baseline pairs that cannot produce meaningful evidence, BEFORE Diff gets a chance to report CLEAN on them. Every check here exists because passing it silently is worse than failing loudly: this harness's whole job is to be the thing that says "nothing moved", so it must refuse to say that when it did not actually look.
Types ¶
type Baseline ¶
type Baseline struct {
// Version is the format tag from the first header line.
Version string
// Stats is the reconciliation header the dump wrote.
Stats Stats
// Entries are the per-query records, in file order.
Entries []Entry
// Path is where the baseline was read from, when it came from disk.
// ValidateDiffInputs uses it to reject a file diffed against itself.
Path string
}
Baseline is a parsed baseline file: its header plus its entries. The header is not decoration — the format version decides whether the two files are even comparable, and the stats line is the only way to tell a baseline that legitimately holds N entries from one whose dump was killed after N.
func LoadBaseline ¶
LoadBaseline reads and parses a baseline file from disk.
func Parse ¶
Parse reads back a rendered baseline. Round-tripping through Parse is what lets Diff report per-QUERY verdicts instead of per-line text hunks.
The header is PARSED, not skipped:
- a formatVersion other than the current one is refused outright, because a rendering change makes every entry differ for reasons that have nothing to do with the planner;
- the stats line must be present and well-formed;
- the stats line's `queries=` + `dml=` must equal the number of entries that follow (both classes produce one entry each). A dump interrupted mid-write keeps its header count and loses the tail, which is precisely the truncation this check catches.
type Delta ¶
type Delta struct {
// Key is the file#index the delta belongs to.
Key string
// Kind classifies the delta.
Kind Change
// SQL is the query text (new side's, falling back to old).
SQL string
// OldPlan / NewPlan are the two renderings; the missing side of an
// ADDED / REMOVED delta is empty.
OldPlan, NewPlan string
// OldShape / NewShape are the two structural skeletons.
OldShape, NewShape []string
// ShapeChanged distinguishes a STRUCTURAL flip (different operators or
// tree) from a rendering-only delta (same skeleton, different labels).
// RFC-183's risk is the former; separating them keeps a formatting
// churn from drowning a real flip. Only set when BOTH sides planned —
// a planned↔error transition is reported by RegressedToError /
// RecoveredFromError, and double-tagging it as a shape flip would
// inflate the count the exit gate reads.
ShapeChanged bool
// RegressedToError marks old-planned → new-fails. This is the single
// most important signal in the report: a query that stopped planning.
RegressedToError bool
// RecoveredFromError marks old-failed → new-plans.
RecoveredFromError bool
}
Delta is one differing key.
type DiffReport ¶
type DiffReport struct {
// Deltas are the differing keys, sorted by (file, index).
Deltas []Delta
// Same is the count of byte-identical entries.
Same int
// TotalOld / TotalNew are the two baselines' entry counts.
TotalOld, TotalNew int
// ShapeFlips counts deltas whose structural skeleton moved.
ShapeFlips int
// Regressions counts planned → error transitions.
Regressions int
// Recoveries counts error → planned transitions.
Recoveries int
// Mispaired counts file#index positions held by a DIFFERENT query on
// each side — the fingerprint of a corpus insertion, removal, or
// reorder. Non-zero means the ADDED/REMOVED noise below is corpus
// churn, not a planner change.
Mispaired int
}
DiffReport is the full verdict between two baselines.
func Diff ¶
func Diff(oldEntries, newEntries []Entry) DiffReport
Diff compares two baselines. Entries pair only when both their corpus position AND their SQL match; a corpus insertion, removal, or reorder therefore surfaces as ADDED/REMOVED (and is counted in Mispaired) instead of being misattributed as a plan change on an unrelated query.
func (DiffReport) Clean ¶
func (r DiffReport) Clean() bool
Clean reports whether the two baselines are identical.
type Entry ¶
type Entry struct {
// File is the corpus file's base name (e.g. "aggregate_expr.yaml").
File string
// Index is the 0-based position of the test within that file's
// `tests:` sequence. File+Index is the diff key: it points a reader
// straight at the exact stanza.
Index int
// SQL is the query text, whitespace-collapsed onto one line.
SQL string
// ErrorPin is the SQLSTATE the corpus stanza expects the query to fail
// with (`error_code:` / `error:`), empty when the stanza expects rows.
// It is what separates the two meanings of a failure marker: a pinned
// rejection is the corpus working as designed, an UNPINNED one is a
// query that should plan and doesn't.
ErrorPin string
// Plan is the recursive one-line Explain() rendering of the winning
// physical plan, or a `<PLAN-ERROR: …>` / `<PLAN-PANIC: …>` marker.
Plan string
// Shape is the plan's structural skeleton: one line per node, the Go
// plan type indented by depth. It survives label-format churn, so a
// pure rendering change shows up in Plan only, while a real structural
// flip moves Shape too. Empty when the query did not plan.
Shape []string
}
Entry is one corpus query's planned shape.
func (Entry) Failed ¶
Failed reports whether the entry is a plan failure marker rather than a plan. A query that STOPS planning is exactly the regression this harness exists to catch, so failures are recorded as entries — never skipped.
func (Entry) Panicked ¶
Panicked reports whether planning this query panicked. Always a bug: the planner's contract is an error, never a panic (design principle 4).
func (Entry) UnexpectedlyFailed ¶
UnexpectedlyFailed reports a query that failed to plan although its corpus stanza expects rows. This is the signal RFC-183 P0 must not regress: a query that stops planning.
type Stats ¶
type Stats struct {
// Files is the number of *.yaml scenarios walked.
Files int
// Queries is the number of planned SELECT/WITH/VALUES tests.
Queries int
// DML is the number of planned DELETE/UPDATE tests. Together with Queries
// it accounts for every entry: Queries + DML == len(entries).
DML int
// NonQuery is the number of non-plannable stanzas skipped (INSERT and
// other `exec:` sequencing steps).
NonQuery int
// PlanErrors is how many entries are failure markers.
PlanErrors int
// UnexpectedErrors is how many of those failures have no corpus error
// pin — statements that are supposed to plan and don't.
UnexpectedErrors int
}
Stats are the reconciliation counts for a Collect run. They are derived, deterministic, and printed in the baseline header so a diff of two headers immediately says whether the corpus itself moved.
func GenerateBaseline ¶
GenerateBaseline is the one-call entry point: walk dir, plan everything, render. Used by cmd/explain-differ and by the package's own tests, so the tool and the tests can never drift apart.
func GenerateBaselineWithReachability ¶
func GenerateBaselineWithReachability(dir string, reach *cascades.ReachabilityCollector) (string, Stats, error)
GenerateBaselineWithReachability is GenerateBaseline with RFC-183's yield-time plan-reachability accounting routed into the caller's collector.
The collector is a PARAMETER, so a corpus walk's tally is the corpus walk's alone. The reachability ratchet and several sibling tests in this package all plan this same corpus under t.Parallel; when the tally was package state in cascades they summed into one number and the ratchet read edges=53748 for a true 17916. nil = collect nothing.
Source Files