sqlstats

package
v1.7.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 15, 2026 License: MIT Imports: 24 Imported by: 0

Documentation

Overview

Package sqlstats wraps database/sql drivers with a measuring proxy and aggregates every query into an in-memory table. Works with any driver (MySQL/MariaDB via go-sql-driver, PostgreSQL via pgx stdlib or lib/pq, and SQLite drivers). Native clients that bypass database/sql are not wrapped.

Index

Constants

View Source 
const DriverSuffix = ":isutools"

DriverSuffix is appended to the original driver name on registration: Register("mysql") makes "mysql:isutools" available.

View Source 
const (
	// MaxTargets bounds the registry: reports, agent payloads and the
	// inspector handle budget all scale with it.
	MaxTargets = 16
)
View Source 
const RotateDrainBudget = generation.DefaultCompatWait

RotateDrainBudget bounds how long Rotate waits for the queries pinned to the generation it closes.

It is generation.DefaultCompatWait, which is runctl.DrainBudget: a rotation is the drain step of a run boundary expressed through the pre-generation API, and httpstats.ResetDrainBudget names the same authority for the same reason. TestRotateDrainBudgetIsTheSharedDrainBudget fails if they ever diverge.

View Source 
const SectionName = "sql"

SectionName is the snapshot section this collector fills. It is the key the existing snapshot and health output already use for SQL, so a run snapshot and a legacy snapshot name the same data the same way.

Variables

View Source 
var (
	// ErrForeignHandle rejects a handle minted by another collector or by
	// another instance of this one. Returning it is what keeps a mismatched
	// handle from being interpreted as a valid generation.
	ErrForeignHandle = errors.New("sqlstats: generation handle belongs to another collector")
	// ErrHandleReleased reports a handle whose data was already freed.
	ErrHandleReleased = errors.New("sqlstats: generation handle was released")
	// ErrNotDrained reports a generation whose rotation has not settled, so no
	// fixed value exists to collect yet.
	ErrNotDrained = errors.New("sqlstats: generation was not drained")
)

Handle errors. They are sentinels because the run controller maps a Collect failure onto a stable machine-readable code and must be able to tell "this handle is not mine" from "this generation has no data yet".

View Source 
var (
	// ErrInvalidTargetID means the explicit ID is empty, longer than
	// maxTargetIDLen, or contains a byte outside [A-Za-z0-9._-].
	ErrInvalidTargetID = errors.New("isutools: invalid target id")
	// ErrUnknownTarget means the ID was never registered. Consumer APIs
	// accept registered IDs only; they never create targets as a side effect.
	ErrUnknownTarget = errors.New("isutools: unknown target id")
	// ErrDuplicateTarget means the ID is taken by a different database, or
	// the database is already registered under another ID. Colliding targets
	// are rejected rather than merged, because merging would silently sum
	// two databases into one row of every report.
	ErrDuplicateTarget = errors.New("isutools: target id already in use")
	// ErrUnknownDriver means driverName is not registered with database/sql.
	ErrUnknownDriver = errors.New("isutools: driver is not registered")
	// ErrInvalidPurpose means the purpose is not valid for the call.
	ErrInvalidPurpose = errors.New("isutools: invalid purpose")
	// ErrDuplicatePurpose means (target, purpose) already has a credential.
	ErrDuplicatePurpose = errors.New("isutools: purpose already registered")
	// ErrInspectorTargetMismatch means a purpose credential points at a
	// different database server from the target's application credential.
	// The default database may differ because inspector connections remove it,
	// but driver, network and address must identify the same endpoint.
	ErrInspectorTargetMismatch = errors.New("isutools: inspector credential points at a different target")
	// ErrPurposeNotRegistered means the purpose has no credential and no
	// fallback is allowed for it.
	ErrPurposeNotRegistered = errors.New("isutools: purpose not registered")
	// ErrExecNotAllowed means Querier.ExecContext was handed a statement
	// outside the session-settings allowlist.
	ErrExecNotAllowed = errors.New("isutools: exec statement not allowed")
	// ErrUnparsedDSN means the DSN could not be parsed structurally, so the
	// connection hygiene rules cannot be applied to it.
	ErrUnparsedDSN = errors.New("isutools: dsn could not be parsed")
	// ErrUnsupportedDSNForm means the DSN parses, but not into a form the
	// inspector hygiene rules can be applied to. Only the
	// go-sql-driver/mysql form can be rebuilt without its default database,
	// which is the property the whole hygiene argument rests on.
	ErrUnsupportedDSNForm = errors.New("isutools: dsn form does not support inspector connection hygiene")
	// ErrTooManyTargets means the registry is at MaxTargets.
	ErrTooManyTargets = errors.New("isutools: too many db targets")
	// ErrDriverFailed reports a failure raised by the database driver itself.
	// The driver's own error is deliberately not wrapped: drivers routinely
	// echo the DSN they were opened with, and everything the registry returns
	// can end up in a health note, a published snapshot or an agent payload.
	// The identifying context is the TargetID, the Purpose and Display, all of
	// which are rebuilt from an allowlist and hold no credential.
	ErrDriverFailed = errors.New("isutools: database driver failed")
)

Registry errors. They are sentinels so callers can decide between "skip this target" and "fail startup" with errors.Is.

View Source 
var Default = NewStore(agg.DefaultMaxKeys)

Default is the generation-scoped store all proxied drivers report into.

Functions

func CloseDBInspectors added in v1.2.0 

func CloseDBInspectors()

CloseDBInspectors closes every pooled stats/explain connection. Nothing of ours should stay connected to the database between benchmark runs, so a shutdown path calls it; the registry stays usable and reopens on demand.

func FirstConn added in v0.2.0 

func FirstConn() (driverName, dsn string, ok bool)

FirstConn returns the base driver name and DSN of the first connection opened through a wrapped driver. dbinspect uses it to open its own raw connection for schema inspection without any extra integration code.

func Inspect added in v1.2.0 

func Inspect(ctx context.Context, id string, purpose Purpose, fn func(context.Context, Querier) error) error

Inspect runs fn on a dedicated connection of the target's purpose-specific credential.

Every call pins its own *sql.Conn: a pool limited to one connection is not a session guarantee, and session-local state (time zone, roles) would silently be lost across a reconnect. The connection, and any result set fn left open, are closed before Inspect returns.

PurposeStats falls back to the application credential when no stats credential is registered — normalized by the same hygiene rules — because the single-DSN setup must keep working. PurposeExplain never falls back and returns ErrPurposeNotRegistered instead.

func Notes added in v1.2.0 

func Notes() []string

Notes returns the registry's degradation notes (unparsed DSNs, ID collisions, exceeded limits). Registration is fail-open: it records a note and drops the target instead of breaking the instrumented application. Notes never contain a DSN or any error text taken from a driver.

func Register 

func Register(names ...string) error

Register wraps each named, already-registered driver and registers the measuring variant under name+DriverSuffix. Calling it again for the same name is a no-op.

func RegisterDBInspector added in v1.2.0 

func RegisterDBInspector(targetID string, purpose Purpose, driverName, dsn string) error

RegisterDBInspector attaches a purpose-specific credential to an existing target. Only PurposeStats and PurposeExplain are accepted: PurposeApp is the target's identity, and replacing it mid-run would change Display, Schema and the canonical tuple under the collectors' feet.

The DSN must use the go-sql-driver/mysql form, because that is the only form the connection hygiene rules can be applied to; a URL-form DSN is rejected with ErrUnsupportedDSNForm rather than opened without them.

func RegisterDBTarget added in v1.2.0 

func RegisterDBTarget(id, driverName, dsn string) error

RegisterDBTarget creates a logical target with an explicit, human-chosen ID and registers its PurposeApp credential. Call it before SQLDriverName so the proxy driver finds the target already named; otherwise the DSN is auto-registered under a derived ID and the explicit call fails.

func TargetIDForDSN added in v1.2.0 

func TargetIDForDSN(driverName, dsn string) (string, bool)

TargetIDForDSN reports the ID currently assigned to the DSN's canonical tuple. It never registers anything: it exists so callers can obtain an auto-derived ID instead of trying to spell out its hash suffix by hand. ok is false for an unparseable, unregistered or ambiguous DSN.

Types

type CommentTagPolicy added in v1.5.0 

type CommentTagPolicy string

CommentTagPolicy controls whether the first safe block comment participates in the aggregate key. All comments are scrubbed in both modes. 

const (
	CommentTagsOn  CommentTagPolicy = "on"
	CommentTagsOff CommentTagPolicy = "off"
	EnvCommentTags                  = "ISUTOOLS_SQL_COMMENT_TAGS"
)

func ResolveCommentTagPolicy added in v1.5.0 

func ResolveCommentTagPolicy(getenv func(string) string) (CommentTagPolicy, string)

ResolveCommentTagPolicy returns a secret-free reason code. Unknown values preserve the historical tag-on behavior.

type DSNFeatures added in v1.2.0 

type DSNFeatures struct {
	InterpolateParams bool
	ParseTime         bool
	MultiStatements   bool
}

DSNFeatures reports go-sql-driver/mysql DSN attributes of the application connection. Advisor checks read it instead of string-matching a DSN they are not allowed to see.

func Features added in v1.2.0 

func Features(id string) (DSNFeatures, bool)

Features reports the application DSN's attributes. known is false when the DSN could not be parsed as a go-sql-driver/mysql DSN, which callers must distinguish from "parsed, and the feature is off".

type EventObserver added in v1.4.0 

type EventObserver interface {
	SQLStart(time.Time) any
	SQLFinish(time.Time, any, string, time.Duration, bool)
	SQLCancel(time.Time, any)
}

EventObserver receives already-normalized SQL completion events. Observer panics are contained so diagnostics can never fail an application query.

type Frozen added in v0.2.0 

type Frozen struct {
	Generation int64
	Entries    []agg.Entry
	// CutShort reports that the rotation gave up on its bound with queries still
	// running in the generation it closed, so Entries may be missing their rows.
	// Callers publish such a generation as a partial section: a rotation that
	// truncates silently reports an under-counted SQL table as a whole one.
	CutShort bool
}

Frozen is a completed SQL generation.

type GenerationCollector added in v1.2.0 

type GenerationCollector struct {
	// contains filtered or unexported fields
}

GenerationCollector adapts the SQL store's generation machinery to runctl.GenerationCollector.

The store's own Rotate publishes the new generation and then waits for every query that started in the old one, which would put an in-flight query on the boundary path. This adapter splits the two: the boundary performs only the pointer swap, and the wait moves into Drain, where the contract allows a context to cancel it.

Nothing here waits for a query outside a caller's context. A query that never returns leaves its own generation unsettled and nothing else: the swap already happened, later boundaries are unaffected, and cancelling a Drain leaves no goroutine behind, because there is no goroutine — the wait belongs to the caller that asked for it.

func NewGenerationCollector added in v1.2.0 

func NewGenerationCollector(store *Store) *GenerationCollector

NewGenerationCollector wraps a store. A nil store means the package default, which is where every proxied driver reports.

func (*GenerationCollector) BeginBoundary added in v1.2.0 

func (c *GenerationCollector) BeginBoundary(ctx context.Context, runID string, ep runctl.Epoch) (runctl.BoundaryResult, error)

BeginBoundary swaps in a fresh store generation and returns a handle to the one it just closed. It moves a pointer, so it does not block on in-flight queries.

func (*GenerationCollector) Collect added in v1.2.0 

func (c *GenerationCollector) Collect(h runctl.GenerationHandle) (any, error)

Collect returns the frozen generation. It reads only what the rotation fixed and never touches the store's current table.

func (*GenerationCollector) Drain added in v1.2.0 

func (c *GenerationCollector) Drain(ctx context.Context, h runctl.GenerationHandle) error

Drain waits for the queries pinned to the handle's generation, then freezes it. The wait is the caller's own: it ends when every query has finished or when ctx says to stop, whichever comes first, and abandoning it leaves no goroutine that will later touch any generation.

func (*GenerationCollector) Freeze added in v1.2.0 

func (c *GenerationCollector) Freeze(ctx context.Context, runID string, ep runctl.Epoch) (runctl.BoundaryResult, error)

Freeze seals the running generation and returns its handle. Queries that start after it belong to the next generation, outside the run.

func (*GenerationCollector) Name added in v1.2.0 

func (c *GenerationCollector) Name() string

Name identifies the snapshot section this collector fills.

func (*GenerationCollector) Release added in v1.2.0 

func (c *GenerationCollector) Release(h runctl.GenerationHandle)

Release drops the frozen table. It is idempotent, and a handle this collector never minted is ignored rather than reported: Release has no error channel and must not panic into the caller.

func (*GenerationCollector) SetEventObserver added in v1.4.0 

func (c *GenerationCollector) SetEventObserver(observer EventObserver)

SetEventObserver forwards event observation to the underlying store.

type Purpose added in v1.2.0 

type Purpose string

Purpose selects which credential of a logical target a connection uses.

A TargetID names a logical database; a Purpose names one of the credentials that reach it. Keeping them separate is what lets a least-privilege EXPLAIN user be introduced without splitting the aggregation key: every consumer (row stats, pool stats, query plans, the multi-host agent) still joins on the TargetID alone. 

const (
	// PurposeApp is the application's own traffic connection. It is the only
	// source of Display, Schema and Features, and every target has one:
	// either the proxy driver observed it or RegisterDBTarget declared it.
	PurposeApp Purpose = "app"
	// PurposeStats is the connection used for SHOW STATUS/VARIABLES and
	// performance_schema digests.
	PurposeStats Purpose = "stats"
	// PurposeExplain is the least-privilege connection used for EXPLAIN.
	// It never falls back to the application credential: an implicit
	// downgrade to a credential holding DML rights would defeat the point
	// of running EXPLAIN under a restricted user.
	PurposeExplain Purpose = "explain"
)

type Querier added in v1.2.0 

type Querier interface {
	QueryContext(ctx context.Context, query string, args ...any) (Rows, error)
	QueryRowContext(ctx context.Context, query string, args ...any) Row
	// ExecContext runs statements that return no result set. It is
	// restricted to session settings (first token SET); anything else
	// returns ErrExecNotAllowed, because an inspection connection must
	// never be able to modify data.
	ExecContext(ctx context.Context, query string, args ...any) (sql.Result, error)
}

Querier is the restricted database handle passed to Inspect callbacks. It deliberately exposes no *sql.DB and no transaction control: a callback must not be able to outlive the call or open connections of its own.

type Row added in v1.2.0 

type Row interface {
	Scan(dest ...any) error
	Err() error
}

Row is the single-row view handed to Inspect callbacks.

type Rows added in v1.2.0 

type Rows interface {
	Next() bool
	Scan(dest ...any) error
	Columns() ([]string, error)
	Err() error
	Close() error
}

Rows is the result-set view handed to Inspect callbacks. It is a wrapper, not *sql.Rows, so the registry can force-close leaked result sets and keep the pinned connection reusable.

type Store added in v0.2.0 

type Store struct {
	// contains filtered or unexported fields
}

Store owns generation-scoped SQL aggregation tables.

func NewStore added in v0.2.0 

func NewStore(maxKeys int) *Store

NewStore constructs a generation-scoped SQL store.

func (*Store) CurrentGeneration added in v0.2.0 

func (s *Store) CurrentGeneration() int64

CurrentGeneration identifies the generation accepting new observations.

func (*Store) Observe added in v0.2.0 

func (s *Store) Observe(query string, duration time.Duration)

Observe adds an already-normalized query to the current generation.

func (*Store) ObserveResult added in v0.2.0 

func (s *Store) ObserveResult(query string, duration time.Duration, failed bool)

ObserveResult adds an already-normalized query result to the current generation.

func (*Store) Reset added in v0.2.0 

func (s *Store) Reset()

Reset starts a new generation and discards the frozen data. Prefer Rotate when the caller needs to retain the previous generation.

func (*Store) Rotate added in v0.2.0 

func (s *Store) Rotate() Frozen

Rotate publishes a new empty generation and freezes the previous one after all observations that started there have completed.

The wait is bounded by RotateDrainBudget, or by whatever SetRotateDrainBudget installed, so a query that never returns delays the rotation instead of parking it. A rotation that gave up returns Frozen.CutShort, which is the caller's cue to publish the generation as partial. Callers holding a request context should use RotateContext.

func (*Store) RotateContext added in v1.2.0 

func (s *Store) RotateContext(ctx context.Context) Frozen

RotateContext is Rotate bounded by the caller's context as well as by the drain budget, whichever ends first. A nil context means the budget alone.

It exists because Rotate runs on the /reset path while the handler holds the process-wide reset lock and the operation slot: without the caller's context a rotation waiting out a wedged query head-of-line-blocks every other admin endpoint for the whole budget, long after the request that asked for it is gone.

func (*Store) SetEventObserver added in v1.4.0 

func (s *Store) SetEventObserver(observer EventObserver)

SetEventObserver replaces the optional timeline/event observer.

func (*Store) SetRotateDrainBudget added in v1.2.0 

func (s *Store) SetRotateDrainBudget(budget time.Duration)

SetRotateDrainBudget bounds the wait performed by Rotate. A non-positive value restores RotateDrainBudget. It exists for callers whose own operation budget is tighter than the run controller's; the default is already bounded, so leaving it alone is safe.

func (*Store) Snapshot added in v0.2.0 

func (s *Store) Snapshot() []agg.Entry

Snapshot returns the current generation's SQL aggregates.

type TargetInfo added in v1.2.0 

type TargetInfo struct {
	// ID is the stable TargetID every collector joins on.
	ID string
	// Driver is the real driver name of the application connection (never
	// the proxied "<name>:isutools" variant).
	Driver string
	// Display is rebuilt from an allowlist of DSN fields, so no credential
	// can reach it even if a DSN carries unusual parameters.
	Display string
	// Schema is the application's default database name. It is not a
	// secret, and collectors bind it as a query parameter because their own
	// connections deliberately have no default database.
	Schema string
	// Purposes lists the registered credentials, always starting with
	// PurposeApp.
	Purposes []Purpose
}

TargetInfo is the public, credential-free description of one target.

func Target added in v1.2.0 

func Target(id string) (TargetInfo, bool)

Target returns one target by exact (byte-for-byte) ID match.

func Targets added in v1.2.0 

func Targets() []TargetInfo

Targets returns every registered target ordered by ID.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.