sources

package
v3.7.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 21, 2025 License: BSD-3-Clause Imports: 26 Imported by: 0

Documentation

Overview

Provides functionality to read monitored data from different sources.

Sources defines how to get the information for the monitored databases. At the moment, sources definitions support two storages: * PostgreSQL database * YAML file

* `postgres.go` files cover the functionality for the PostgreSQL database. * `yaml.go` files cover the functionality for the YAML file. * `resolver.go` implements continuous discovery from patroni and postgres cluster. * `types.go` defines the types and interfaces. * `sample.sources.yaml` is a sample configuration file.

Index

Constants

View Source
const (
	EnvUnknown       = "UNKNOWN"
	EnvAzureSingle   = "AZURE_SINGLE" //discontinued
	EnvAzureFlexible = "AZURE_FLEXIBLE"
	EnvGoogle        = "GOOGLE"
)

Variables

View Source
var (
	NewConn           = db.New
	NewConnWithConfig = db.NewWithConfig
)

NewConn and NewConnWithConfig are wrappers to allow testing

Functions

func VersionToInt added in v3.3.0

func VersionToInt(version string) (v int)

Types

type CmdOpts

type CmdOpts struct {
	Sources                      string   `` /* 160-byte string literal not displayed */
	Refresh                      int      `` /* 127-byte string literal not displayed */
	Groups                       []string `` /* 150-byte string literal not displayed */
	MinDbSizeMB                  int64    `` /* 184-byte string literal not displayed */
	MaxParallelConnectionsPerDb  int      `` /* 241-byte string literal not displayed */
	TryCreateListedExtsIfMissing string   `` /* 290-byte string literal not displayed */
	CreateHelpers                bool     `` /* 144-byte string literal not displayed */
}

SourceOpts specifies the sources related command-line options

type HostConfigAttrs

type HostConfigAttrs struct {
	DcsType                string   `yaml:"dcs_type"`
	DcsEndpoints           []string `yaml:"dcs_endpoints"`
	Scope                  string
	Namespace              string
	Username               string
	Password               string
	CAFile                 string                             `yaml:"ca_file"`
	CertFile               string                             `yaml:"cert_file"`
	KeyFile                string                             `yaml:"key_file"`
	LogsGlobPath           string                             `yaml:"logs_glob_path"`   // default $data_directory / $log_directory / *.csvlog
	LogsMatchRegex         string                             `yaml:"logs_match_regex"` // default is for CSVLOG format. needs to capture following named groups: log_time, user_name, database_name and error_severity
	PerMetricDisabledTimes []HostConfigPerMetricDisabledTimes `yaml:"per_metric_disabled_intervals"`
}

type HostConfigPerMetricDisabledTimes

type HostConfigPerMetricDisabledTimes struct {
	Metrics       []string `yaml:"metrics"`
	DisabledTimes []string `yaml:"disabled_times"`
	DisabledDays  string   `yaml:"disabled_days"`
}

type Kind

type Kind string
const (
	SourcePostgres           Kind = "postgres"
	SourcePostgresContinuous Kind = "postgres-continuous-discovery"
	SourcePgBouncer          Kind = "pgbouncer"
	SourcePgPool             Kind = "pgpool"
	SourcePatroni            Kind = "patroni"
	SourcePatroniContinuous  Kind = "patroni-continuous-discovery"
	SourcePatroniNamespace   Kind = "patroni-namespace-discovery"
)

func (Kind) IsValid

func (k Kind) IsValid() bool

type PatroniClusterMember

type PatroniClusterMember struct {
	Scope   string
	Name    string
	ConnURL string `yaml:"conn_url"`
	Role    string
}

type Reader

type Reader interface {
	GetSources() (Sources, error)
}

type ReaderWriter

type ReaderWriter interface {
	Reader
	Writer
}

func NewPostgresSourcesReaderWriter

func NewPostgresSourcesReaderWriter(ctx context.Context, connstr string) (ReaderWriter, error)

func NewPostgresSourcesReaderWriterConn

func NewPostgresSourcesReaderWriterConn(ctx context.Context, conn db.PgxPoolIface) (ReaderWriter, error)

func NewYAMLSourcesReaderWriter

func NewYAMLSourcesReaderWriter(ctx context.Context, path string) (ReaderWriter, error)

type RuntimeInfo added in v3.3.0

type RuntimeInfo struct {
	LastCheckedOn    time.Time
	IsInRecovery     bool
	VersionStr       string
	Version          int
	RealDbname       string
	SystemIdentifier string
	IsSuperuser      bool
	Extensions       map[string]int
	ExecEnv          string
	ApproxDbSize     int64
}

type Source

type Source struct {
	Name                 string             `yaml:"name" db:"name"`
	Group                string             `yaml:"group" db:"group"`
	ConnStr              string             `yaml:"conn_str" db:"connstr"`
	Metrics              map[string]float64 `yaml:"custom_metrics" db:"config"`
	MetricsStandby       map[string]float64 `yaml:"custom_metrics_standby" db:"config_standby"`
	Kind                 Kind               `yaml:"kind" db:"dbtype"`
	IncludePattern       string             `yaml:"include_pattern" db:"include_pattern"`
	ExcludePattern       string             `yaml:"exclude_pattern" db:"exclude_pattern"`
	PresetMetrics        string             `yaml:"preset_metrics" db:"preset_config"`
	PresetMetricsStandby string             `yaml:"preset_metrics_standby" db:"preset_config_standby"`
	IsEnabled            bool               `yaml:"is_enabled" db:"is_enabled"`
	CustomTags           map[string]string  `yaml:"custom_tags" db:"custom_tags"`
	HostConfig           HostConfigAttrs    `yaml:"host_config" db:"host_config"`
	OnlyIfMaster         bool               `yaml:"only_if_master" db:"only_if_master"`
}

Source represents a configuration how to get databases to monitor. It can be a single database, a group of databases in postgres cluster, a group of databases in HA patroni cluster. pgbouncer and pgpool kinds are purely to indicate that the monitored database connection is made through a connection pooler, which supports its own additional metrics. If one is not interested in those additional metrics, it is ok to specify the connection details as a regular postgres source.

func (*Source) Clone

func (s *Source) Clone() *Source

func (Source) Equal added in v3.5.0

func (s Source) Equal(s2 Source) bool

func (*Source) GetDatabaseName

func (s *Source) GetDatabaseName() string

func (Source) IsDefaultGroup added in v3.6.0

func (s Source) IsDefaultGroup() bool

func (Source) ResolveDatabases

func (s Source) ResolveDatabases() (SourceConns, error)

ResolveDatabases() return a slice of found databases for continuous monitoring sources, e.g. patroni

type SourceConn added in v3.2.0

type SourceConn struct {
	Source
	Conn       db.PgxPoolIface
	ConnConfig *pgxpool.Config
	RuntimeInfo
}

SourceConn represents a single connection to monitor. Unlike source, it contains a database connection. Continuous discovery sources (postgres-continuous-discovery, patroni-continuous-discovery, patroni-namespace-discovery) will produce multiple monitored databases structs based on the discovered databases.

func (*SourceConn) Connect added in v3.2.0

func (md *SourceConn) Connect(ctx context.Context, opts CmdOpts) (err error)

Connect will establish a connection to the database if it's not already connected. If the connection is already established, it pings the server to ensure it's still alive.

func (*SourceConn) DiscoverPlatform added in v3.2.0

func (md *SourceConn) DiscoverPlatform(ctx context.Context) (platform string)

TryDiscoverPlatform tries to discover the platform based on the database version string and some special settings that are only available on certain platforms. Returns the platform name or "UNKNOWN" if not sure.

func (*SourceConn) FetchApproxSize added in v3.3.0

func (md *SourceConn) FetchApproxSize(ctx context.Context) (size int64)

FetchApproxSize returns the approximate size of the database in bytes

func (*SourceConn) FetchRuntimeInfo added in v3.3.0

func (md *SourceConn) FetchRuntimeInfo(ctx context.Context, forceRefetch bool) (err error)

func (*SourceConn) FetchVersion added in v3.3.0

func (md *SourceConn) FetchVersion(ctx context.Context, sql string) (version string, ver int, err error)

func (*SourceConn) FunctionExists added in v3.2.0

func (md *SourceConn) FunctionExists(ctx context.Context, functionName string) (exists bool)

FunctionExists checks if a function exists in the database

func (*SourceConn) GetClusterIdentifier added in v3.3.0

func (md *SourceConn) GetClusterIdentifier() string

GetUniqueIdentifier returns a unique identifier for the host assuming SysId is the same for primary and all replicas but connection information is different

func (*SourceConn) GetDatabaseName added in v3.2.0

func (md *SourceConn) GetDatabaseName() string

GetDatabaseName returns the database name from the connection string

func (*SourceConn) GetMetricInterval added in v3.3.0

func (md *SourceConn) GetMetricInterval(name string) float64

GetMetricInterval returns the metric interval for the connection

func (*SourceConn) IsPostgresSource added in v3.2.0

func (md *SourceConn) IsPostgresSource() bool

func (*SourceConn) ParseConfig added in v3.2.0

func (md *SourceConn) ParseConfig() (err error)

ParseConfig will parse the connection string and store the result in the connection config

func (*SourceConn) Ping added in v3.2.0

func (md *SourceConn) Ping(ctx context.Context) (err error)

Ping will try to ping the server to ensure the connection is still alive

func (*SourceConn) SetDatabaseName added in v3.2.0

func (md *SourceConn) SetDatabaseName(name string)

SetDatabaseName sets the database name in the connection config for resolved databases

type SourceConns added in v3.2.0

type SourceConns []*SourceConn

SourceConn represents a single connection to monitor. Unlike source, it contains a database connection. Continuous discovery sources (postgres-continuous-discovery, patroni-continuous-discovery, patroni-namespace-discovery) will produce multiple monitored databases structs based on the discovered databases.

func ResolveDatabasesFromPatroni

func ResolveDatabasesFromPatroni(source Source) (SourceConns, error)

func ResolveDatabasesFromPostgres

func ResolveDatabasesFromPostgres(s Source) (resolvedDbs SourceConns, err error)

ResolveDatabasesFromPostgres reads all the databases from the given cluster, additionally matching/not matching specified regex patterns

func (SourceConns) GetMonitoredDatabase added in v3.2.0

func (mds SourceConns) GetMonitoredDatabase(DBUniqueName string) *SourceConn

type Sources

type Sources []Source

func (Sources) ResolveDatabases

func (srcs Sources) ResolveDatabases() (_ SourceConns, err error)

ResolveDatabases() updates list of monitored objects from continuous monitoring sources, e.g. patroni

func (Sources) Validate added in v3.1.0

func (srcs Sources) Validate() (Sources, error)

type Writer

type Writer interface {
	WriteSources(Sources) error
	DeleteSource(string) error
	UpdateSource(md Source) error
}

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL