Documentation
¶
Overview ¶
Package sqlrows measures per-digest row efficiency — rows examined against rows sent — over a benchmark run, by sampling performance_schema.events_statements_summary_by_digest at both run boundaries and reporting the difference.
The counters in that table are cumulative since the server started, so a single reading says nothing about a run. sqlrows takes a full reading at the opening boundary, another at the closing boundary, and derives the interval afterwards, which is why it implements runctl.BaselineCollector rather than accumulating anything of its own.
Self-contamination ¶
The collector's own statements are recorded by performance_schema just like the application's. The connection it samples on therefore deliberately has no default database (see sqlstats connection hygiene), which makes MySQL attribute every statement issued here to a NULL schema, and the target schema is passed as a bound parameter (WHERE SCHEMA_NAME = ?) rather than through DATABASE(). A statement of this package can consequently never match the application schema's rows. Nothing in this package may use DATABASE().
That hygiene is verified, not assumed. The registry can only remove the default database from a DSN it is able to rebuild — a URL-form DSN reaches the driver unchanged and keeps the application's schema — so the first probe of every target reads the session's default database out of performance_schema.threads. A target whose connection has one, or whose connection cannot be checked, is skipped with CodeInspectorDefaultDB instead of measured: numbers that silently include the measurement's own statements are worse than no numbers, and the operator is told which target to re-register.
Because statements from a connection without a default database land on SCHEMA_NAME IS NULL, that condition alone does not identify the digest table's overflow row: the overflow row is the one where SCHEMA_NAME and DIGEST are *both* NULL. See TargetSample.Overflow.
Index ¶
- Constants
- Variables
- func Enabled() bool
- func Registration() runctl.Registration
- type Collector
- func (c *Collector) Budget() time.Duration
- func (c *Collector) CaptureBaseline(ctx context.Context, runID string, ep runctl.Epoch) (runctl.SampleResult, error)
- func (c *Collector) CaptureFinal(ctx context.Context, runID string, ep runctl.Epoch) (runctl.SampleResult, error)
- func (c *Collector) Collect(base, final runctl.BaselineHandle) (any, error)
- func (c *Collector) Name() string
- func (c *Collector) QuerySampleTextSupported(targetID string) (supported, known bool)
- func (c *Collector) Release(h runctl.BaselineHandle)
- type DBClock
- type DigestRow
- type DigestStat
- type HealthNote
- type InspectFunc
- type OverflowStat
- type Sample
- type Section
- type StatementKind
- type TargetSample
- type TargetSection
Constants ¶
const ( // CodeProbeSkip means capability probing ruled the target out // (performance_schema off, digest consumer disabled, columns missing). CodeProbeSkip = "probe-skip" // CodeNoSchema means TargetInfo.Schema is empty, so there is no value to // bind to WHERE SCHEMA_NAME = ?. Guessing one would measure a different // database. CodeNoSchema = "no-schema" // CodeInspectorDefaultDB means the connection handed to this collector // could not be proven free of a default database, so its own statements // would be recorded as digests of the schema it is measuring. The target is // skipped: contaminated numbers that look plausible are worse than none, // and the operator has to be told which target to re-register. CodeInspectorDefaultDB = "inspector-default-db" // CodeBudgetExhausted means the target's wave could not start inside the // boundary budget. Recorded rather than dropped: a silently missing target // is indistinguishable from a target with no traffic. CodeBudgetExhausted = "budget-exhausted" // CodeQueryError means a statement against the target failed. CodeQueryError = "query-error" // CodeUnpairedBoundary means only one of the two boundaries has the // target, so no interval exists for it. CodeUnpairedBoundary = "unpaired-boundary" // CodeDBRestart means the server changed identity or restarted between the // boundaries, so the counters do not share an origin. CodeDBRestart = "db-restart" // CodeCounterReset means the digest table was truncated or the counters // rewound between the boundaries. CodeCounterReset = "counter-reset" )
Reason codes for a target that carries no numbers. They are internal detail rather than wire values of runctl: a dropped boundary is reported to runctl as runctl.CodeNotCaptured, and these codes explain *why* inside the section.
const ( // AnomalyMissing means at least one of the four timestamps was not taken. AnomalyMissing = "clock-missing" // AnomalyBackwardsBaseline means the opening boundary's own two readings // are out of order. AnomalyBackwardsBaseline = "clock-backwards-baseline" // AnomalyBackwardsFinal means the closing boundary's two readings are out // of order. AnomalyBackwardsFinal = "clock-backwards-final" // AnomalyBackwardsInterval means the closing boundary started before the // opening boundary ended, which makes the measured interval empty or // inverted. AnomalyBackwardsInterval = "clock-backwards-interval" )
DBClock anomaly codes. The order below is the evaluation order: the first matching condition wins, so the value is deterministic when several hold.
const ( // HealthSkip reports a target ruled out by capability probing. HealthSkip = "sqlrows-skip" // HealthNoSchema reports a target with no schema to bind. HealthNoSchema = "sqlrows-no-schema" // HealthOverflow reports statements aggregated into the digest table's // overflow row, i.e. incomplete coverage. HealthOverflow = "sqlrows-overflow" // HealthDBRestart reports a server that restarted or changed identity // between the boundaries. HealthDBRestart = "sqlrows-db-restart" // HealthCounterReset reports counters that were truncated or rewound. HealthCounterReset = "sqlrows-counter-reset" // HealthClockAnomaly reports a database clock that stepped backwards. HealthClockAnomaly = "sqlrows-clock-anomaly" // HealthTargetDropped reports targets that produced no interval. HealthTargetDropped = "sqlrows-target-dropped" )
Health keys this package reports. The set is closed at seven: a new condition reuses one of these with a different message rather than adding a key, so a health snapshot stays readable.
const ( // Name is the collector name and the snapshot section key. Name = "sqlrows" // DigestTextFetchLimit bounds both how many digest texts are fetched at // the closing boundary and how many rows a target reports. The delta is // always computed over every digest — truncation happens after the // subtraction, never before it, because a digest that was outside the top // N at the opening boundary would otherwise contribute its whole // historical total to the interval. DigestTextFetchLimit = 200 // EnvFlag disables the collector when set to a false-ish value. EnvFlag = "ISUTOOLS_SQLROWS" )
const MissingQueryText = "(digest text unavailable)"
MissingQueryText stands in for a digest whose text was not fetched — it fell outside the fetched top, or the opening boundary was never taken. It is never classified as a statement kind, because guessing one from a placeholder would put a fabricated SELECT in front of a user.
Variables ¶
var ( // ErrNoTargetCaptured reports that every registered target failed to be // sampled (unreachable, permission denied, or out of budget). It is // returned together with a SampleResult whose Committed is false, which is // what tells runctl to degrade the run instead of trusting an empty // interval. Targets that were merely skipped — performance_schema off, no // schema — do not produce this error: having nothing to measure is not a // failure. ErrNoTargetCaptured = errors.New("sqlrows: no db target could be sampled") // ErrSampleType reports a handle that does not carry a *Sample. The // collector contract forbids panicking here: measurement must never break // the measured application. ErrSampleType = errors.New("sqlrows: baseline handle does not carry a sqlrows sample") )
Errors returned by the collector. They are sentinels so the run controller and tests can distinguish "nothing to measure" from "measurement broke".
Functions ¶
func Enabled ¶
func Enabled() bool
Enabled reports whether the collector should be wired in. Measurement of the measurement is what the flag is for: ISUTOOLS_SQLROWS=off removes every statement this package issues, which is how the ABBA overhead gate compares runs with and without it.
func Registration ¶
func Registration() runctl.Registration
Registration describes how sqlrows participates in a run. It is optional: a database without performance_schema must degrade the run to partial, not invalidate it.
Types ¶
type Collector ¶
type Collector struct {
// contains filtered or unexported fields
}
Collector samples the digest table at both run boundaries.
It holds no accumulating state: everything a snapshot needs travels inside the runctl.BaselineHandle, so Collect can derive an interval from two frozen samples without touching the database, the registry, or the fields below.
func New ¶
func New() *Collector
New returns a collector bound to the process-wide DB target registry.
The fan-out numbers are runctl's, not this package's: runctl is the single authority for time budgets, and a collector inventing its own would make "why was my target dropped?" unanswerable.
func (*Collector) Budget ¶
Budget declares the time one boundary of this collector needs, so a misconfigured budget table is rejected at registration instead of showing up as a truncated measurement.
The number is derived, not invented: sqlstats.MaxTargets (16) targets over runctl.BaselineConcurrency (8) is two waves, and each wave is bounded by runctl.PerTargetBudget.
func (*Collector) CaptureBaseline ¶
func (c *Collector) CaptureBaseline(ctx context.Context, runID string, ep runctl.Epoch) (runctl.SampleResult, error)
CaptureBaseline samples the opening boundary.
func (*Collector) CaptureFinal ¶
func (c *Collector) CaptureFinal(ctx context.Context, runID string, ep runctl.Epoch) (runctl.SampleResult, error)
CaptureFinal samples the closing boundary. It also fetches the digest texts of the rows that lead the interval, because Collect is not allowed to do I/O and the interesting digests are only known once both readings exist.
func (*Collector) Collect ¶
func (c *Collector) Collect(base, final runctl.BaselineHandle) (any, error)
Collect derives the interval from two frozen samples.
It reads nothing but the two handles: no database, no registry, not even the collector's own pending map. That is the whole point of carrying the sample inside the handle, and it is why a snapshot built minutes after the run still describes the run.
func (*Collector) QuerySampleTextSupported ¶
QuerySampleTextSupported reports whether the target's digest table has a QUERY_SAMPLE_TEXT column, and whether that is known at all.
The answer is a by-product of this package's capability probe, and query plan capture needs it: exposing it here is what keeps that consumer from paying for a second probe — and from probing a target sqlrows has already ruled out.
func (*Collector) Release ¶
func (c *Collector) Release(h runctl.BaselineHandle)
Release frees what a handle pins. It is idempotent, and safe on a zero handle, because runctl releases both edges of an interval even when only one of them was ever taken.
type DBClock ¶
type DBClock struct {
BaselineBefore time.Time `json:"baseline_before"`
BaselineAfter time.Time `json:"baseline_after"`
FinalBefore time.Time `json:"final_before"`
FinalAfter time.Time `json:"final_after"`
// Monotonic reports BaselineBefore <= BaselineAfter <= FinalBefore <=
// FinalAfter, all four readings present. When false, consumers must not
// judge freshness at all — neither fresh nor stale.
Monotonic bool `json:"monotonic"`
// Anomaly is the stable code of the first violated condition, empty when
// Monotonic is true.
Anomaly string `json:"anomaly,omitempty"`
}
DBClock carries the database's own UTC readings around both boundaries.
Plan 09 decides query-sample freshness from this interval, so a database clock that stepped backwards (NTP, a virtualization host, a manual date) must be visible rather than silently producing an empty or inverted window in which every sample looks stale.
type DigestRow ¶
type DigestRow struct {
CountStar uint64 `json:"count_star"`
TimerWait uint64 `json:"timer_wait"`
RowsExamined uint64 `json:"rows_examined"`
RowsSent uint64 `json:"rows_sent"`
RowsAffected uint64 `json:"rows_affected"`
CreatedTmpDiskTables uint64 `json:"created_tmp_disk_tables"`
SortMergePasses uint64 `json:"sort_merge_passes"`
NoIndexUsed uint64 `json:"no_index_used"`
NoGoodIndexUsed uint64 `json:"no_good_index_used"`
}
DigestRow is one row of events_statements_summary_by_digest. Every field is a cumulative unsigned counter; the interval value is the difference between two readings.
type DigestStat ¶
type DigestStat struct {
Digest string `json:"digest"`
// Query is the truncated DIGEST_TEXT, or MissingQueryText.
Query string `json:"query"`
Kind StatementKind `json:"kind"`
Count uint64 `json:"count"`
// TimerWaitPicos is the raw SUM_TIMER_WAIT delta; TotalTime is the same
// value as a duration, kept because picoseconds are unreadable and the
// raw number is needed to reproduce the ordering.
TimerWaitPicos uint64 `json:"timer_wait_picos"`
TotalTime time.Duration `json:"total_time"`
RowsExamined uint64 `json:"rows_examined"`
RowsSent uint64 `json:"rows_sent"`
RowsAffected uint64 `json:"rows_affected"`
// ExaminedPerSent is only defined for SELECT statements that returned at
// least one row. HasRatio distinguishes "not applicable" from "zero":
// treating a SELECT that sent nothing as a ratio of RowsExamined would
// invent the worst possible score for a query that simply found nothing.
ExaminedPerSent float64 `json:"examined_per_sent,omitempty"`
HasRatio bool `json:"has_ratio"`
// Index and sort quality signals, all interval values.
NoIndexUsed uint64 `json:"no_index_used"`
NoGoodIndexUsed uint64 `json:"no_good_index_used"`
CreatedTmpDiskTables uint64 `json:"created_tmp_disk_tables"`
SortMergePasses uint64 `json:"sort_merge_passes"`
}
DigestStat is one digest's interval value.
type HealthNote ¶
HealthNote is one grouped degradation message.
type InspectFunc ¶
type InspectFunc func(ctx context.Context, id string, purpose sqlstats.Purpose, fn func(context.Context, sqlstats.Querier) error) error
InspectFunc is the registry entry point the collector uses to reach a target. It matches sqlstats.Inspect and exists as a named type so tests can drive the collector without a database.
type OverflowStat ¶
type OverflowStat struct {
Detected bool `json:"detected"`
CountStar uint64 `json:"count_star,omitempty"`
// ReportedBy names the target carrying this server's overflow when this
// target is not the one reporting it.
ReportedBy string `json:"reported_by,omitempty"`
}
OverflowStat describes the digest table's overflow row.
That row is instance-global, so several targets on one server would otherwise report the same overflow several times. Detected is therefore set on the first target of a server only, and the others point at it.
type Sample ¶
type Sample struct {
Targets map[string]*TargetSample `json:"targets"`
}
Sample is one boundary's frozen reading of every target. It is built once and never mutated afterwards: handles are copied and shared, so a later mutation would silently change an interval that was supposed to be fixed.
type Section ¶
type Section struct {
// Targets is ordered by TargetID so two snapshots diff cleanly.
Targets []TargetSection `json:"targets"`
// Health carries this section's degradation notes, already grouped by
// key and reason.
Health []HealthNote `json:"health,omitempty"`
// Validity is the verdict this section contributes to the run. sqlrows is
// an optional collector, so it degrades a run to partial and never
// invalidates it: a database without performance_schema is a normal
// deployment, not a broken measurement.
Validity runctl.Validity `json:"validity"`
// Limit is the number of rows a target may show, recorded so a reader can
// tell a truncated table from a short one.
Limit int `json:"limit"`
}
Section is the snapshot section sqlrows contributes.
type StatementKind ¶
type StatementKind string
StatementKind separates the statement families a row can belong to. The examined/sent ratio only means something for SELECT, and DML is read through its affected-row count instead.
const ( // KindSelect covers SELECT, including a WITH ... SELECT common table // expression. KindSelect StatementKind = "select" // KindDML covers INSERT, UPDATE, DELETE and REPLACE. KindDML StatementKind = "dml" // KindOther covers everything else, including digests whose text is // unavailable. KindOther StatementKind = "other" )
func Classify ¶
func Classify(text string) StatementKind
Classify decides which statement family a digest text belongs to.
The family is what makes the numbers comparable: the examined-per-sent ratio is a SELECT diagnostic, while DML is read through affected rows. A WITH ... SELECT is classified as a SELECT — looking only at the first keyword would file every common table expression under "other" and hide the heaviest queries of an application that uses them.
type TargetSample ¶
type TargetSample struct {
TargetID string `json:"target_id"`
// Schema is the value bound to WHERE SCHEMA_NAME = ?. It is recorded so a
// snapshot shows which database the numbers describe.
Schema string `json:"schema"`
// ServerUUID and UptimeSec detect a server that restarted between the two
// boundaries, which would make the counters incomparable.
ServerUUID string `json:"server_uuid,omitempty"`
UptimeSec int64 `json:"uptime_sec,omitempty"`
// UTCBefore and UTCAfter bracket the digest read with the database's own
// clock. Plan 09 compares query-sample timestamps against this interval,
// so both edges are needed, not just one.
UTCBefore time.Time `json:"utc_before"`
UTCAfter time.Time `json:"utc_after"`
// Digests holds the rows whose SCHEMA_NAME matched the bound schema, keyed
// by hex DIGEST.
Digests map[string]DigestRow `json:"digests,omitempty"`
// Overflow is the row where SCHEMA_NAME and DIGEST are *both* NULL: the
// bucket MySQL aggregates statements into once the digest table is full.
// A NULL schema alone does not mean overflow — statements from any
// connection without a default database, this collector's included, also
// have a NULL schema but a real digest.
Overflow DigestRow `json:"overflow"`
HasOverflow bool `json:"has_overflow,omitempty"`
// Texts maps DIGEST to its truncated DIGEST_TEXT. Only the closing
// boundary fills it, and only for the digests that lead the interval.
Texts map[string]string `json:"texts,omitempty"`
// Captured reports that Digests is a real reading. When false, Code and
// Err say why and the target contributes no numbers.
Captured bool `json:"captured"`
Code string `json:"code,omitempty"`
Err string `json:"err,omitempty"`
}
TargetSample is one target's reading at one boundary.
type TargetSection ¶
type TargetSection struct {
TargetID string `json:"target_id"`
Schema string `json:"schema,omitempty"`
// Usable reports that Digests holds real interval values. When false the
// target contributes no numbers, and consumers that enrich rows — plan
// 09's EXPLAIN capture in particular — must skip this target entirely.
Usable bool `json:"usable"`
// Code and Reason explain an unusable target: probe-skip, no-schema,
// budget-exhausted, query-error, unpaired-boundary, db-restart or
// counter-reset.
Code string `json:"code,omitempty"`
Reason string `json:"reason,omitempty"`
// Digests holds the leading rows of the interval, ordered by total time.
Digests []DigestStat `json:"digests,omitempty"`
// Shown, Total and Dropped make truncation explicit. Total counts every
// digest that ran during the interval, not every digest in the table.
Shown int `json:"shown"`
Total int `json:"total"`
Dropped int `json:"dropped"`
// Overflow describes the digest table's overflow row for this target.
Overflow OverflowStat `json:"overflow"`
// DBClock carries the database-side interval. Consumers must not judge
// freshness when Monotonic is false.
DBClock DBClock `json:"db_clock"`
}
TargetSection is one target's interval.