appdbwatch

package
v1.162.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 16, 2026 License: Apache-2.0 Imports: 17 Imported by: 0

Documentation

Overview

Package appdbwatch samples every application database on an interval (#2406, #2409): its health reduced to numbers, and its schema against what the applied migrations build. One replica samples at a time under an advisory lock; the samples are kept for the connection page's history and the gauges, and the leveled alerts are judged on them.

Index

Constants

View Source
const (
	ConditionTransactionIDAge  = "transaction_id_age"
	ConditionIdleInTransaction = "idle_in_transaction"
	ConditionConnections       = "connections"
	ConditionInvalidIndexes    = "invalid_indexes"
	ConditionDeadTuples        = "dead_tuples"
	ConditionDrift             = "drift"
)

The conditions an application database is alerted on, the last part of each alert's key.

View Source
const (
	DefaultIntervalSeconds    = 300
	DefaultHistoryDays        = 30
	DefaultLongRunningSeconds = 60
	// MinIntervalSeconds is the shortest interval a database is sampled at.
	MinIntervalSeconds = 10
	// MaxHistoryDays bounds how long samples are kept.
	MaxHistoryDays = 3650
)

Defaults.

View Source
const AlertFamily = "appdb"

AlertFamily is the levelalert family the application database alerts are recorded under.

View Source
const DefaultTick = 10 * time.Second

DefaultTick is how often each replica asks whether a pass is due; the pass itself runs once per the settings' interval.

View Source
const LockKey int64 = 4713210404

LockKey is the advisory lock one replica samples under. Storage holds 4713210402 and the webhook alerts 4713210403.

View Source
const SettingsSection = "appdb_health"

SettingsSection is the platform_settings section the health settings are kept in.

Variables

View Source
var (
	DefaultTransactionIDAge   = levelalert.Thresholds{Warning: 500_000_000, Critical: 1_000_000_000}
	DefaultIdleInTransaction  = levelalert.Thresholds{Warning: 300, Critical: 1800}
	DefaultConnectionsPercent = levelalert.Thresholds{Warning: 80, Critical: 95}
	DefaultInvalidIndexes     = levelalert.Thresholds{Warning: 1}
	DefaultDeadTuplePercent   = levelalert.Thresholds{Warning: 20, Critical: 50}
	DefaultDriftObjects       = levelalert.Thresholds{Warning: 1}
)

The default thresholds of each condition, a warning and a critical level.

Functions

func Assess added in v1.154.1

func Assess(r *appdbhealth.Report, s Settings)

Assess judges a report against the health settings (#2528): each condition an alert is posted for, at the level its thresholds give, one problem per session, index or table past it, then the advice no alert is posted for. The status is the most severe alert level, or healthy: it is the level the alert engine posts for the same numbers, so the card and the alerts agree. It writes Status and Findings into r.

func Bytes added in v1.154.1

func Bytes(n int64) string

Bytes writes a size in binary units.

func DriftOf

func DriftOf(ctx context.Context, kit appdbdrift.Kit, store appdbdrift.Migrations, connection string) []appdbdrift.Result

DriftOf checks every schema the platform keeps migrations for on connection.

func Duration added in v1.154.1

func Duration(secs float64) string

Duration writes seconds the way a person says them: "40 seconds", "14 minutes", "3 hours".

func Findings

func Findings(connection string, s Sample, settings Settings, link string) []levelalert.Finding

Findings judges one connection's sample against the settings. Titles and bodies carry counts and object names, never statement or session text.

Types

type Deps

type Deps struct {
	DB       *sql.DB
	Settings SettingsStore
	// Kit is the live postgres toolkit, nil when none is registered.
	Kit func() Kit
	// Migrations lists the platform's migrations, which name the schemas a
	// connection's drift is checked in.
	Migrations appdbdrift.Migrations
	// Engine routes the alerts; nil records nothing and sends nothing.
	Engine *levelalert.Engine
	// PageURL is the connection page an alert links to; nil omits it.
	PageURL func(connection string) string
	Tick    time.Duration
	Now     func() time.Time
}

Deps is what the service reads and sends through.

type Kit

type Kit interface {
	appdbhealth.Querier
	appdbdrift.Kit
	ListConnections() []toolkit.ConnectionDetail
}

Kit is the postgres toolkit as the loop reads it.

type Point

type Point struct {
	Connection string    `json:"connection"`
	SampledAt  time.Time `json:"sampled_at"`
	Sample     Sample    `json:"sample"`
}

Point is one stored sample.

type Sample

type Sample struct {
	appdbhealth.Summary
	// Reachable is false when the database did not answer the server read.
	Reachable bool `json:"reachable"`
	// DriftObjects is how many objects differ from what the applied
	// migrations build, across the schemas that could be checked, and
	// DriftUnchecked how many schemas the platform keeps migrations for could
	// not be (their Drift result says why).
	DriftObjects   int `json:"drift_objects"`
	DriftUnchecked int `json:"drift_unchecked"`
	// Drift is the check of each schema the platform keeps migrations for.
	Drift []appdbdrift.Result `json:"drift"`
}

Sample is one connection's reading as the history keeps it.

type Service

type Service struct {
	// contains filtered or unexported fields
}

Service samples the databases, keeps their history and evaluates the alerts.

func New

func New(d Deps) *Service

New returns the service, or nil without a database.

func (*Service) Gauges

Gauges is what the health gauges report: each connection's newest sample within three intervals. A read that fails reports nothing.

func (*Service) History

func (s *Service) History(ctx context.Context, connection string, days int) ([]Point, error)

History is connection's samples over the last days, oldest first.

func (*Service) Latest

func (s *Service) Latest(ctx context.Context, window time.Duration) ([]Point, error)

Latest is each connection's newest sample taken within the window.

func (*Service) Pass

func (s *Service) Pass(ctx context.Context) error

Pass is one iteration: sample every application database, keep the samples, drop the history past the settings' period, and evaluate the alerts. A pass within most of the interval of the last one is skipped: another replica, or this one, took it.

func (*Service) SaveSettings

func (s *Service) SaveSettings(ctx context.Context, v Settings, author string) error

SaveSettings validates and writes the settings.

func (*Service) Settings

func (s *Service) Settings(ctx context.Context) (View, error)

Settings reads the stored settings.

func (*Service) Start

func (s *Service) Start(ctx context.Context)

Start runs the sampling loop. Nil-safe.

func (*Service) Stop

func (s *Service) Stop()

Stop ends the loop and waits for it. Nil-safe; a second call is a no-op.

type Settings

type Settings struct {
	// Channels are the notification channels the alerts are sent to.
	Channels []string `json:"channels"`
	// IntervalSeconds is how often each application database is sampled.
	IntervalSeconds int `json:"interval_seconds"`
	HistoryDays     int `json:"history_days"`
	// LongRunningSeconds is how long a statement runs before its session is
	// listed as long-running.
	LongRunningSeconds int `json:"long_running_seconds"`
	// TransactionIDAge is the age of the oldest unfrozen transaction id.
	TransactionIDAge levelalert.Thresholds `json:"transaction_id_age"`
	// IdleInTransactionSeconds is how long the longest session has been idle
	// in a transaction.
	IdleInTransactionSeconds levelalert.Thresholds `json:"idle_in_transaction_seconds"`
	// ConnectionsPercent is the share of max_connections in use.
	ConnectionsPercent levelalert.Thresholds `json:"connections_percent"`
	InvalidIndexes     levelalert.Thresholds `json:"invalid_indexes"`
	// DeadTuplePercent is the highest dead tuple share of a table holding
	// 1,000 rows or more.
	DeadTuplePercent levelalert.Thresholds `json:"dead_tuple_percent"`
	// DriftObjects is how many objects differ from what the applied
	// migrations build.
	DriftObjects levelalert.Thresholds `json:"drift_objects"`
}

Settings is how application databases are sampled and what raises an alert (#2409). A threshold of 0 is off.

func Defaults

func Defaults() Settings

Defaults is the configuration before anything is written.

func (Settings) Interval

func (s Settings) Interval() time.Duration

Interval is how often a database is sampled.

func (Settings) Validate

func (s Settings) Validate() error

Validate refuses a setting the loop cannot run with.

type SettingsStore

type SettingsStore interface {
	Get(ctx context.Context) (settingsjson.Stored[Settings], error)
	Set(ctx context.Context, v Settings, author string) error
}

SettingsStore reads and writes the section.

func NewSettingsStore

func NewSettingsStore(db *sql.DB) SettingsStore

NewSettingsStore returns the section over db.

type View

type View struct {
	Settings
	UpdatedBy string `json:"updated_by,omitempty"`
	UpdatedAt string `json:"updated_at,omitempty"`
}

View is the read shape: the settings with who wrote them last.

func Read

func Read(ctx context.Context, store SettingsStore) (View, error)

Read returns the stored settings, or the defaults before any are written.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL