sqlrows

package
v1.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 6, 2026 License: MIT Imports: 11 Imported by: 0

Documentation

Overview

Package sqlrows measures per-digest row efficiency — rows examined against rows sent — over a benchmark run, by sampling performance_schema.events_statements_summary_by_digest at both run boundaries and reporting the difference.

The counters in that table are cumulative since the server started, so a single reading says nothing about a run. sqlrows takes a full reading at the opening boundary, another at the closing boundary, and derives the interval afterwards, which is why it implements runctl.BaselineCollector rather than accumulating anything of its own.

Self-contamination

The collector's own statements are recorded by performance_schema just like the application's. The connection it samples on therefore deliberately has no default database (see sqlstats connection hygiene), which makes MySQL attribute every statement issued here to a NULL schema, and the target schema is passed as a bound parameter (WHERE SCHEMA_NAME = ?) rather than through DATABASE(). A statement of this package can consequently never match the application schema's rows. Nothing in this package may use DATABASE().

That hygiene is verified, not assumed. The registry can only remove the default database from a DSN it is able to rebuild — a URL-form DSN reaches the driver unchanged and keeps the application's schema — so the first probe of every target reads the session's default database out of performance_schema.threads. A target whose connection has one, or whose connection cannot be checked, is skipped with CodeInspectorDefaultDB instead of measured: numbers that silently include the measurement's own statements are worse than no numbers, and the operator is told which target to re-register.

Because statements from a connection without a default database land on SCHEMA_NAME IS NULL, that condition alone does not identify the digest table's overflow row: the overflow row is the one where SCHEMA_NAME and DIGEST are *both* NULL. See TargetSample.Overflow.

Index

Constants

View Source
const (
	// CodeProbeSkip means capability probing ruled the target out
	// (performance_schema off, digest consumer disabled, columns missing).
	CodeProbeSkip = "probe-skip"
	// CodeNoSchema means TargetInfo.Schema is empty, so there is no value to
	// bind to WHERE SCHEMA_NAME = ?. Guessing one would measure a different
	// database.
	CodeNoSchema = "no-schema"
	// CodeInspectorDefaultDB means the connection handed to this collector
	// could not be proven free of a default database, so its own statements
	// would be recorded as digests of the schema it is measuring. The target is
	// skipped: contaminated numbers that look plausible are worse than none,
	// and the operator has to be told which target to re-register.
	CodeInspectorDefaultDB = "inspector-default-db"
	// CodeBudgetExhausted means the target's wave could not start inside the
	// boundary budget. Recorded rather than dropped: a silently missing target
	// is indistinguishable from a target with no traffic.
	CodeBudgetExhausted = "budget-exhausted"
	// CodeQueryError means a statement against the target failed.
	CodeQueryError = "query-error"
	// CodeUnpairedBoundary means only one of the two boundaries has the
	// target, so no interval exists for it.
	CodeUnpairedBoundary = "unpaired-boundary"
	// CodeDBRestart means the server changed identity or restarted between the
	// boundaries, so the counters do not share an origin.
	CodeDBRestart = "db-restart"
	// CodeCounterReset means the digest table was truncated or the counters
	// rewound between the boundaries.
	CodeCounterReset = "counter-reset"
)

Reason codes for a target that carries no numbers. They are internal detail rather than wire values of runctl: a dropped boundary is reported to runctl as runctl.CodeNotCaptured, and these codes explain *why* inside the section.

View Source
const (
	// AnomalyMissing means at least one of the four timestamps was not taken.
	AnomalyMissing = "clock-missing"
	// AnomalyBackwardsBaseline means the opening boundary's own two readings
	// are out of order.
	AnomalyBackwardsBaseline = "clock-backwards-baseline"
	// AnomalyBackwardsFinal means the closing boundary's two readings are out
	// of order.
	AnomalyBackwardsFinal = "clock-backwards-final"
	// AnomalyBackwardsInterval means the closing boundary started before the
	// opening boundary ended, which makes the measured interval empty or
	// inverted.
	AnomalyBackwardsInterval = "clock-backwards-interval"
)

DBClock anomaly codes. The order below is the evaluation order: the first matching condition wins, so the value is deterministic when several hold.

View Source
const (
	// HealthSkip reports a target ruled out by capability probing.
	HealthSkip = "sqlrows-skip"
	// HealthNoSchema reports a target with no schema to bind.
	HealthNoSchema = "sqlrows-no-schema"
	// HealthOverflow reports statements aggregated into the digest table's
	// overflow row, i.e. incomplete coverage.
	HealthOverflow = "sqlrows-overflow"
	// HealthDBRestart reports a server that restarted or changed identity
	// between the boundaries.
	HealthDBRestart = "sqlrows-db-restart"
	// HealthCounterReset reports counters that were truncated or rewound.
	HealthCounterReset = "sqlrows-counter-reset"
	// HealthClockAnomaly reports a database clock that stepped backwards.
	HealthClockAnomaly = "sqlrows-clock-anomaly"
	// HealthTargetDropped reports targets that produced no interval.
	HealthTargetDropped = "sqlrows-target-dropped"
)

Health keys this package reports. The set is closed at seven: a new condition reuses one of these with a different message rather than adding a key, so a health snapshot stays readable.

View Source
const (
	// Name is the collector name and the snapshot section key.
	Name = "sqlrows"

	// DigestTextFetchLimit bounds both how many digest texts are fetched at
	// the closing boundary and how many rows a target reports. The delta is
	// always computed over every digest — truncation happens after the
	// subtraction, never before it, because a digest that was outside the top
	// N at the opening boundary would otherwise contribute its whole
	// historical total to the interval.
	DigestTextFetchLimit = 200

	// EnvFlag disables the collector when set to a false-ish value.
	EnvFlag = "ISUTOOLS_SQLROWS"
)
View Source
const MissingQueryText = "(digest text unavailable)"

MissingQueryText stands in for a digest whose text was not fetched — it fell outside the fetched top, or the opening boundary was never taken. It is never classified as a statement kind, because guessing one from a placeholder would put a fabricated SELECT in front of a user.

Variables

View Source
var (
	// ErrNoTargetCaptured reports that every registered target failed to be
	// sampled (unreachable, permission denied, or out of budget). It is
	// returned together with a SampleResult whose Committed is false, which is
	// what tells runctl to degrade the run instead of trusting an empty
	// interval. Targets that were merely skipped — performance_schema off, no
	// schema — do not produce this error: having nothing to measure is not a
	// failure.
	ErrNoTargetCaptured = errors.New("sqlrows: no db target could be sampled")

	// ErrSampleType reports a handle that does not carry a *Sample. The
	// collector contract forbids panicking here: measurement must never break
	// the measured application.
	ErrSampleType = errors.New("sqlrows: baseline handle does not carry a sqlrows sample")
)

Errors returned by the collector. They are sentinels so the run controller and tests can distinguish "nothing to measure" from "measurement broke".

Functions

func Enabled

func Enabled() bool

Enabled reports whether the collector should be wired in. Measurement of the measurement is what the flag is for: ISUTOOLS_SQLROWS=off removes every statement this package issues, which is how the ABBA overhead gate compares runs with and without it.

func Registration

func Registration() runctl.Registration

Registration describes how sqlrows participates in a run. It is optional: a database without performance_schema must degrade the run to partial, not invalidate it.

Types

type Collector

type Collector struct {
	// contains filtered or unexported fields
}

Collector samples the digest table at both run boundaries.

It holds no accumulating state: everything a snapshot needs travels inside the runctl.BaselineHandle, so Collect can derive an interval from two frozen samples without touching the database, the registry, or the fields below.

func New

func New() *Collector

New returns a collector bound to the process-wide DB target registry.

The fan-out numbers are runctl's, not this package's: runctl is the single authority for time budgets, and a collector inventing its own would make "why was my target dropped?" unanswerable.

func (*Collector) Budget

func (c *Collector) Budget() time.Duration

Budget declares the time one boundary of this collector needs, so a misconfigured budget table is rejected at registration instead of showing up as a truncated measurement.

The number is derived, not invented: sqlstats.MaxTargets (16) targets over runctl.BaselineConcurrency (8) is two waves, and each wave is bounded by runctl.PerTargetBudget.

func (*Collector) CaptureBaseline

func (c *Collector) CaptureBaseline(ctx context.Context, runID string, ep runctl.Epoch) (runctl.SampleResult, error)

CaptureBaseline samples the opening boundary.

func (*Collector) CaptureFinal

func (c *Collector) CaptureFinal(ctx context.Context, runID string, ep runctl.Epoch) (runctl.SampleResult, error)

CaptureFinal samples the closing boundary. It also fetches the digest texts of the rows that lead the interval, because Collect is not allowed to do I/O and the interesting digests are only known once both readings exist.

func (*Collector) Collect

func (c *Collector) Collect(base, final runctl.BaselineHandle) (any, error)

Collect derives the interval from two frozen samples.

It reads nothing but the two handles: no database, no registry, not even the collector's own pending map. That is the whole point of carrying the sample inside the handle, and it is why a snapshot built minutes after the run still describes the run.

func (*Collector) Name

func (c *Collector) Name() string

Name identifies the snapshot section this collector fills.

func (*Collector) QuerySampleTextSupported

func (c *Collector) QuerySampleTextSupported(targetID string) (supported, known bool)

QuerySampleTextSupported reports whether the target's digest table has a QUERY_SAMPLE_TEXT column, and whether that is known at all.

The answer is a by-product of this package's capability probe, and query plan capture needs it: exposing it here is what keeps that consumer from paying for a second probe — and from probing a target sqlrows has already ruled out.

func (*Collector) Release

func (c *Collector) Release(h runctl.BaselineHandle)

Release frees what a handle pins. It is idempotent, and safe on a zero handle, because runctl releases both edges of an interval even when only one of them was ever taken.

type DBClock

type DBClock struct {
	BaselineBefore time.Time `json:"baseline_before"`
	BaselineAfter  time.Time `json:"baseline_after"`
	FinalBefore    time.Time `json:"final_before"`
	FinalAfter     time.Time `json:"final_after"`
	// Monotonic reports BaselineBefore <= BaselineAfter <= FinalBefore <=
	// FinalAfter, all four readings present. When false, consumers must not
	// judge freshness at all — neither fresh nor stale.
	Monotonic bool `json:"monotonic"`
	// Anomaly is the stable code of the first violated condition, empty when
	// Monotonic is true.
	Anomaly string `json:"anomaly,omitempty"`
}

DBClock carries the database's own UTC readings around both boundaries.

Plan 09 decides query-sample freshness from this interval, so a database clock that stepped backwards (NTP, a virtualization host, a manual date) must be visible rather than silently producing an empty or inverted window in which every sample looks stale.

type DigestRow

type DigestRow struct {
	CountStar            uint64 `json:"count_star"`
	TimerWait            uint64 `json:"timer_wait"`
	RowsExamined         uint64 `json:"rows_examined"`
	RowsSent             uint64 `json:"rows_sent"`
	RowsAffected         uint64 `json:"rows_affected"`
	CreatedTmpDiskTables uint64 `json:"created_tmp_disk_tables"`
	SortMergePasses      uint64 `json:"sort_merge_passes"`
	NoIndexUsed          uint64 `json:"no_index_used"`
	NoGoodIndexUsed      uint64 `json:"no_good_index_used"`
}

DigestRow is one row of events_statements_summary_by_digest. Every field is a cumulative unsigned counter; the interval value is the difference between two readings.

type DigestStat

type DigestStat struct {
	Digest string `json:"digest"`
	// Query is the truncated DIGEST_TEXT, or MissingQueryText.
	Query string        `json:"query"`
	Kind  StatementKind `json:"kind"`
	Count uint64        `json:"count"`
	// TimerWaitPicos is the raw SUM_TIMER_WAIT delta; TotalTime is the same
	// value as a duration, kept because picoseconds are unreadable and the
	// raw number is needed to reproduce the ordering.
	TimerWaitPicos uint64        `json:"timer_wait_picos"`
	TotalTime      time.Duration `json:"total_time"`
	RowsExamined   uint64        `json:"rows_examined"`
	RowsSent       uint64        `json:"rows_sent"`
	RowsAffected   uint64        `json:"rows_affected"`
	// ExaminedPerSent is only defined for SELECT statements that returned at
	// least one row. HasRatio distinguishes "not applicable" from "zero":
	// treating a SELECT that sent nothing as a ratio of RowsExamined would
	// invent the worst possible score for a query that simply found nothing.
	ExaminedPerSent float64 `json:"examined_per_sent,omitempty"`
	HasRatio        bool    `json:"has_ratio"`
	// Index and sort quality signals, all interval values.
	NoIndexUsed          uint64 `json:"no_index_used"`
	NoGoodIndexUsed      uint64 `json:"no_good_index_used"`
	CreatedTmpDiskTables uint64 `json:"created_tmp_disk_tables"`
	SortMergePasses      uint64 `json:"sort_merge_passes"`
}

DigestStat is one digest's interval value.

type HealthNote

type HealthNote struct {
	Key     string `json:"key"`
	Message string `json:"message"`
}

HealthNote is one grouped degradation message.

type InspectFunc

type InspectFunc func(ctx context.Context, id string, purpose sqlstats.Purpose, fn func(context.Context, sqlstats.Querier) error) error

InspectFunc is the registry entry point the collector uses to reach a target. It matches sqlstats.Inspect and exists as a named type so tests can drive the collector without a database.

type OverflowStat

type OverflowStat struct {
	Detected  bool   `json:"detected"`
	CountStar uint64 `json:"count_star,omitempty"`
	// ReportedBy names the target carrying this server's overflow when this
	// target is not the one reporting it.
	ReportedBy string `json:"reported_by,omitempty"`
}

OverflowStat describes the digest table's overflow row.

That row is instance-global, so several targets on one server would otherwise report the same overflow several times. Detected is therefore set on the first target of a server only, and the others point at it.

type Sample

type Sample struct {
	Targets map[string]*TargetSample `json:"targets"`
}

Sample is one boundary's frozen reading of every target. It is built once and never mutated afterwards: handles are copied and shared, so a later mutation would silently change an interval that was supposed to be fixed.

type Section

type Section struct {
	// Targets is ordered by TargetID so two snapshots diff cleanly.
	Targets []TargetSection `json:"targets"`
	// Health carries this section's degradation notes, already grouped by
	// key and reason.
	Health []HealthNote `json:"health,omitempty"`
	// Validity is the verdict this section contributes to the run. sqlrows is
	// an optional collector, so it degrades a run to partial and never
	// invalidates it: a database without performance_schema is a normal
	// deployment, not a broken measurement.
	Validity runctl.Validity `json:"validity"`
	// Limit is the number of rows a target may show, recorded so a reader can
	// tell a truncated table from a short one.
	Limit int `json:"limit"`
}

Section is the snapshot section sqlrows contributes.

type StatementKind

type StatementKind string

StatementKind separates the statement families a row can belong to. The examined/sent ratio only means something for SELECT, and DML is read through its affected-row count instead.

const (
	// KindSelect covers SELECT, including a WITH ... SELECT common table
	// expression.
	KindSelect StatementKind = "select"
	// KindDML covers INSERT, UPDATE, DELETE and REPLACE.
	KindDML StatementKind = "dml"
	// KindOther covers everything else, including digests whose text is
	// unavailable.
	KindOther StatementKind = "other"
)

func Classify

func Classify(text string) StatementKind

Classify decides which statement family a digest text belongs to.

The family is what makes the numbers comparable: the examined-per-sent ratio is a SELECT diagnostic, while DML is read through affected rows. A WITH ... SELECT is classified as a SELECT — looking only at the first keyword would file every common table expression under "other" and hide the heaviest queries of an application that uses them.

type TargetSample

type TargetSample struct {
	TargetID string `json:"target_id"`
	// Schema is the value bound to WHERE SCHEMA_NAME = ?. It is recorded so a
	// snapshot shows which database the numbers describe.
	Schema string `json:"schema"`
	// ServerUUID and UptimeSec detect a server that restarted between the two
	// boundaries, which would make the counters incomparable.
	ServerUUID string `json:"server_uuid,omitempty"`
	UptimeSec  int64  `json:"uptime_sec,omitempty"`
	// UTCBefore and UTCAfter bracket the digest read with the database's own
	// clock. Plan 09 compares query-sample timestamps against this interval,
	// so both edges are needed, not just one.
	UTCBefore time.Time `json:"utc_before"`
	UTCAfter  time.Time `json:"utc_after"`
	// Digests holds the rows whose SCHEMA_NAME matched the bound schema, keyed
	// by hex DIGEST.
	Digests map[string]DigestRow `json:"digests,omitempty"`
	// Overflow is the row where SCHEMA_NAME and DIGEST are *both* NULL: the
	// bucket MySQL aggregates statements into once the digest table is full.
	// A NULL schema alone does not mean overflow — statements from any
	// connection without a default database, this collector's included, also
	// have a NULL schema but a real digest.
	Overflow    DigestRow `json:"overflow"`
	HasOverflow bool      `json:"has_overflow,omitempty"`
	// Texts maps DIGEST to its truncated DIGEST_TEXT. Only the closing
	// boundary fills it, and only for the digests that lead the interval.
	Texts map[string]string `json:"texts,omitempty"`
	// Captured reports that Digests is a real reading. When false, Code and
	// Err say why and the target contributes no numbers.
	Captured bool   `json:"captured"`
	Code     string `json:"code,omitempty"`
	Err      string `json:"err,omitempty"`
}

TargetSample is one target's reading at one boundary.

type TargetSection

type TargetSection struct {
	TargetID string `json:"target_id"`
	Schema   string `json:"schema,omitempty"`
	// Usable reports that Digests holds real interval values. When false the
	// target contributes no numbers, and consumers that enrich rows — plan
	// 09's EXPLAIN capture in particular — must skip this target entirely.
	Usable bool `json:"usable"`
	// Code and Reason explain an unusable target: probe-skip, no-schema,
	// budget-exhausted, query-error, unpaired-boundary, db-restart or
	// counter-reset.
	Code   string `json:"code,omitempty"`
	Reason string `json:"reason,omitempty"`
	// Digests holds the leading rows of the interval, ordered by total time.
	Digests []DigestStat `json:"digests,omitempty"`
	// Shown, Total and Dropped make truncation explicit. Total counts every
	// digest that ran during the interval, not every digest in the table.
	Shown   int `json:"shown"`
	Total   int `json:"total"`
	Dropped int `json:"dropped"`
	// Overflow describes the digest table's overflow row for this target.
	Overflow OverflowStat `json:"overflow"`
	// DBClock carries the database-side interval. Consumers must not judge
	// freshness when Monotonic is false.
	DBClock DBClock `json:"db_clock"`
}

TargetSection is one target's interval.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL