Documentation
¶
Overview ¶
Package dbpool reports database/sql connection-pool statistics for one measurement run.
The pool is the narrowest part of most ISUCON application stacks. When SetMaxOpenConns is smaller than the number of concurrent handlers, requests queue *inside* database/sql, where neither the SQL statistics nor the HTTP handler timings can see them: every individual query looks fast while the request that issued it is slow. Reporting WaitCount and WaitDuration over the run interval is what makes that invisible queue visible.
This version displays numbers only. There is deliberately no advisor threshold yet:
- WaitDuration is the sum of every goroutine's wait, so comparing it with the wall-clock length of the interval says nothing (64 concurrent waiters can accumulate 64x the interval without anything being wrong).
- "The pool was full at snapshot time" and "somebody waited earlier in the run" are two unrelated observations that cannot be chained into a cause.
- "Raise the limit" is not a safe conclusion: a larger pool can simply move the saturation into the database server.
Thresholds are to be derived from real measurements and added separately.
Collector implements runctl.BaselineCollector: it samples every watched pool at the run's opening and closing boundary and derives the interval from those two frozen samples alone. (*sql.DB).Stats does no I/O — it takes the handle's own mutex once — so a boundary sample costs microseconds even with MaxPools pools registered, and nothing whatsoever is measured in between.
Index ¶
- Constants
- Variables
- type Collector
- func (c *Collector) CaptureBaseline(ctx context.Context, runID string, ep runctl.Epoch) (runctl.SampleResult, error)
- func (c *Collector) CaptureFinal(ctx context.Context, runID string, ep runctl.Epoch) (runctl.SampleResult, error)
- func (c *Collector) Collect(base, final runctl.BaselineHandle) (any, error)
- func (c *Collector) Name() string
- func (c *Collector) Notes() []string
- func (c *Collector) Points() []Point
- func (c *Collector) Release(h runctl.BaselineHandle)
- func (c *Collector) Unwatch(targetID string) error
- func (c *Collector) Watch(targetID string, db *sql.DB) error
- func (c *Collector) Watched() []string
- type Entry
- type Point
- type PoolSample
- type Sample
Constants ¶
const ( // HealthNotRegistered reports that no pool was ever watched, which is why // the snapshot has no DB Pool section. HealthNotRegistered = "dbpool-not-registered" // HealthRegisteredMidRun reports a pool watched after the run's opening // boundary. It is measured from the next run onwards. HealthRegisteredMidRun = "dbpool-registered-mid-run" // HealthUnwatchedMidRun reports a pool unwatched before the run's closing // boundary. Its entry survives with a shortened interval. HealthUnwatchedMidRun = "dbpool-unwatched-mid-run" // HealthSampleFailed reports a pool whose Stats implementation panicked, // which drops it from that boundary rather than from the process. HealthSampleFailed = "dbpool-sample-failed" )
Health keys this package reports through Notes. They are stable strings: the admin UI and the multi-host agent match on them.
const ( // CodeCounterRewind means a cumulative counter went backwards between the // two samples, which happens when the application replaces the *sql.DB // under the same target. A delta across that boundary would be a negative // or wildly inflated number, so the entry reports the final absolute values // instead and flags itself. CodeCounterRewind = "counter-rewind" // CodeUnwatchedMidRun means UnwatchDBPool was called while the run was in // progress. The entry survives with the farewell sample taken at that // moment, so its interval is shorter than the run's. CodeUnwatchedMidRun = "unwatched-mid-run" )
Entry codes. An empty Code means the interval is a normal, complete one. The set is closed: transports and (future) advisors switch on these values, and they are dbpool's own codes, unrelated to runctl.CollectorBoundary.Code.
const MaxPools = sqlstats.MaxTargets
MaxPools bounds the watch set. It tracks the registry's target limit because a target has at most one application pool, so the registry normally reaches its own limit first; this bound exists so that a caller looping over dynamic configuration cannot grow the watch set without end.
const Name = "dbpool"
Name is the collector name and the key of its snapshot section.
Variables ¶
var ( // ErrNilDB means Watch was handed a nil handle. It is reported instead of // being silently ignored so that an argument bug does not masquerade as a // pool that simply never saw traffic. ErrNilDB = errors.New("isutools: WatchDBPool: db is nil") // ErrDuplicatePool means the target is already watched. Two handles for // one logical database would produce two rows that both claim to be it, so // the second registration is rejected; recreating a pool is // UnwatchDBPool followed by WatchDBPool. ErrDuplicatePool = errors.New("isutools: WatchDBPool: target already watched") // ErrTooManyPools means the watch set is at MaxPools. ErrTooManyPools = errors.New("isutools: WatchDBPool: too many pools (max 16)") )
Watch errors. They are sentinels so a caller can tell "my argument was wrong" from "this build has more pools than the toolkit supports" with errors.Is. An unregistered target ID is reported as sqlstats.ErrUnknownTarget rather than as a dbpool-specific alias: there is one target namespace, and a second name for the same condition would only make it harder to match on.
var Default = New()
Default is the process-wide collector that isutools registers with the run Controller and that WatchDBPool feeds. Tests build their own with New.
Functions ¶
This section is empty.
Types ¶
type Collector ¶
type Collector struct {
// contains filtered or unexported fields
}
Collector watches connection pools and turns two boundary samples into a per-pool interval report.
One mutex covers the watch set, the farewell samples and the run cache, so that a WatchDBPool racing an opening boundary produces a watch set that is either wholly before or wholly after the boundary — never half of each.
func (*Collector) CaptureBaseline ¶
func (c *Collector) CaptureBaseline(ctx context.Context, runID string, ep runctl.Epoch) (runctl.SampleResult, error)
CaptureBaseline samples every watched pool and freezes that set as the run's participants.
Freezing here is what implements deferred activation: a pool watched later in the run is absent from this sample and therefore absent from the run's report. The alternative — giving it a baseline taken mid-run — would report a slice of the interval next to entries covering all of it, in a table whose rows are meant to be comparable.
func (*Collector) CaptureFinal ¶
func (c *Collector) CaptureFinal(ctx context.Context, runID string, ep runctl.Epoch) (runctl.SampleResult, error)
CaptureFinal samples the run's frozen participants at the closing boundary. A participant unwatched mid-run contributes its farewell sample instead of a fresh read, so its interval ends where the pool did.
func (*Collector) Collect ¶
func (c *Collector) Collect(base, final runctl.BaselineHandle) (any, error)
Collect derives the run's per-pool report from two frozen samples.
It reads nothing but base.Sample() and final.Sample(): no (*sql.DB).Stats call, no registry lookup, not even the collector's own watch set. A snapshot is built after the run has closed, and a value read at that point would describe traffic the run never saw. Everything an Entry needs — Display and the per-pool timestamps included — therefore travels inside the sample.
The result is []Entry ordered by TargetID, so a snapshot is byte-stable and two runs can be diffed line by line.
func (*Collector) Notes ¶
Notes returns the degradation notes recorded so far. Each note starts with the health key it belongs to, so the caller can forward it verbatim.
func (*Collector) Points ¶ added in v1.4.0
Points returns the current watched-pool readings in TargetID order. A pool whose Stats callback panics is omitted; application diagnostics must never be able to panic the application.
func (*Collector) Release ¶
func (c *Collector) Release(h runctl.BaselineHandle)
Release drops the collector's own reference to a handle's sample. The handle keeps its copy — it is a value and Collect must keep working after a Release — so this only stops the collector from pinning a finished run. Idempotent, and a no-op for a zero handle or another collector's handle.
func (*Collector) Unwatch ¶
Unwatch removes a pool from the watch set.
It is idempotent: a registered target that is not watched is a no-op. An unregistered ID is still an error, because it means the caller is naming something that does not exist rather than undoing something it did.
If the run is in progress and this pool is part of it, Unwatch takes a farewell sample first. Dropping the entry instead would make the pool vanish from a report it genuinely contributed to; keeping it without a final sample would require reading a *sql.DB the application is about to close.
func (*Collector) Watch ¶
Watch adds a pool to the watch set under an existing TargetID.
Only IDs already present in the registry are accepted, compared byte for byte: no case folding, no trimming, no Unicode normalization. Watch never creates a target, because a typo that silently created a second target would split one database across two rows of every report. Auto-derived IDs end in a 26 character hash and cannot be spelled out by hand — obtain them from sqlstats.TargetIDForDSN, or name the target explicitly with RegisterDBTarget first.
The pool becomes part of the *next* run: see CaptureBaseline.
type Entry ¶
type Entry struct {
// TargetID is the registry's TargetID. Every other collector keys on the
// same value, byte for byte, which is what allows a reader (or the agent)
// to line up pool waits with row counts and query plans for one database.
TargetID string `json:"target_id"`
// Display is the credential-free endpoint description taken from the
// registry when the pool was watched.
Display string `json:"display"`
// MaxOpen is SetMaxOpenConns as observed at the closing boundary; 0 means
// unlimited.
MaxOpen int `json:"max_open"`
// Open is the number of established connections at the closing boundary.
Open int `json:"open"`
// InUse is the number of connections held by a caller at that moment.
InUse int `json:"in_use"`
// Idle is the number of pooled, unused connections at that moment.
Idle int `json:"idle"`
// WaitCount is how many times a caller had to wait for a connection during
// the interval. Any non-zero value means the pool limit, not the database,
// decided the latency of those calls.
WaitCount int64 `json:"wait_count"`
// WaitDuration is the summed wait of every waiting goroutine during the
// interval — not wall-clock time. It can legitimately exceed the length of
// the run, so it must always be displayed with that caveat and is best
// read through AverageWait.
WaitDuration time.Duration `json:"wait_duration_ns"`
// MaxIdleClosed counts connections closed during the interval because the
// idle pool was full (SetMaxIdleConns too small for the traffic).
MaxIdleClosed int64 `json:"max_idle_closed"`
// MaxIdleTimeClosed counts connections closed during the interval by
// SetConnMaxIdleTime.
MaxIdleTimeClosed int64 `json:"max_idle_time_closed"`
// MaxLifetimeClosed counts connections closed during the interval by
// SetConnMaxLifetime. A large value means the run spent its time
// reconnecting.
MaxLifetimeClosed int64 `json:"max_lifetime_closed"`
// BaselineAt and FinalAt are the measured ends of this entry's interval,
// on the same clock as runctl's boundary windows. They are per entry, not
// per run, because an entry unwatched mid-run ends early and has to be
// able to say so.
BaselineAt time.Time `json:"baseline_at"`
FinalAt time.Time `json:"final_at"`
// Partial marks an entry whose interval or values need a caveat; Code says
// which one.
Partial bool `json:"partial,omitempty"`
Code string `json:"code,omitempty"`
}
Entry is one pool's interval report. Point values describe the closing boundary; counters are deltas over the interval, because a cumulative counter that has been running since process start says nothing about the benchmark that just ran.
func (Entry) AverageWait ¶
AverageWait is the mean time one waiting caller spent queued for a connection. Unlike WaitDuration itself this is safe to compare with a query latency, because dividing the summed wait by the number of waits removes the concurrency factor without assuming anything about the distribution. It is zero when nobody waited.
type Point ¶ added in v1.4.0
type Point struct {
TargetID string
MaxOpen int
Open int
InUse int
Idle int
WaitCount int64
WaitDuration time.Duration
}
Point is one bounded, instantaneous pool reading for the optional timeline. Cumulative wait fields are converted to per-bucket deltas by the timeline collector; the remaining fields describe saturation at this instant.
type PoolSample ¶
type PoolSample struct {
// Stats is the value (*sql.DB).Stats returned at At.
Stats sql.DBStats
// At is when this particular pool was read. It is per pool rather than per
// boundary so that a farewell sample can carry the moment the pool left
// the watch set.
At time.Time
// Display is carried in the sample, not looked up later, so that
// Collect can build a complete Entry from frozen values alone.
Display string
// Unwatched marks a farewell sample: the pool was unwatched mid-run and
// this is the last observation that will ever exist for it.
Unwatched bool
}
PoolSample is one pool's frozen observation at a boundary.
sql.DBStats is a flat struct of numbers, so copying it copies it deeply and a caller holding this value can never observe the pool changing underneath.
type Sample ¶
type Sample map[string]PoolSample
Sample is the frozen value a runctl.BaselineHandle carries for this collector: one PoolSample per watched TargetID.
The plan sketched this as map[string]sql.DBStats. That type cannot work, because the same plan requires an entry to report Display and to end its interval at the moment of a mid-run UnwatchDBPool rather than at the run's closing boundary. Both are per pool, and neither is expressible in a bare sql.DBStats. Keeping them in the sample — instead of reading them back out of the collector during Collect — is what makes Collect a pure function of its two handles, which is the property the contract actually cares about.
Source Files