Documentation
¶
Overview ¶
Provides functionality to read monitored data from different sources.
Sources defines how to get the information for the monitored databases. At the moment, sources definitions support two storages: * PostgreSQL database * YAML file
* `postgres.go` files cover the functionality for the PostgreSQL database. * `yaml.go` files cover the functionality for the YAML file. * `resolver.go` implements continuous discovery from patroni and postgres cluster. * `types.go` defines the types and interfaces. * `sample.sources.yaml` is a sample configuration file.
Index ¶
- Constants
- Variables
- func VersionToInt(version string) (v int)
- type CmdOpts
- type HostConfigAttrs
- type HostConfigPerMetricDisabledTimes
- type Kind
- type PatroniClusterMember
- type Reader
- type ReaderWriter
- type RuntimeInfo
- type Source
- type SourceConn
- func (md *SourceConn) Connect(ctx context.Context, opts CmdOpts) (err error)
- func (md *SourceConn) DiscoverPlatform(ctx context.Context) (platform string)
- func (md *SourceConn) FetchApproxSize(ctx context.Context) (size int64)
- func (md *SourceConn) FetchRuntimeInfo(ctx context.Context, forceRefetch bool) (err error)
- func (md *SourceConn) FetchVersion(ctx context.Context, sql string) (version string, ver int, err error)
- func (md *SourceConn) FunctionExists(ctx context.Context, functionName string) (exists bool)
- func (md *SourceConn) GetClusterIdentifier() string
- func (md *SourceConn) GetDatabaseName() string
- func (md *SourceConn) GetMetricInterval(name string) float64
- func (md *SourceConn) IsPostgresSource() bool
- func (md *SourceConn) ParseConfig() (err error)
- func (md *SourceConn) Ping(ctx context.Context) (err error)
- func (md *SourceConn) SetDatabaseName(name string)
- type SourceConns
- type Sources
- type Writer
Constants ¶
const ( EnvUnknown = "UNKNOWN" EnvAzureSingle = "AZURE_SINGLE" //discontinued EnvAzureFlexible = "AZURE_FLEXIBLE" EnvGoogle = "GOOGLE" )
Variables ¶
var ( NewConn = db.New NewConnWithConfig = db.NewWithConfig )
NewConn and NewConnWithConfig are wrappers to allow testing
Functions ¶
func VersionToInt ¶ added in v3.3.0
Types ¶
type CmdOpts ¶
type CmdOpts struct {
Sources string `` /* 160-byte string literal not displayed */
Refresh int `` /* 127-byte string literal not displayed */
Groups []string `` /* 150-byte string literal not displayed */
MinDbSizeMB int64 `` /* 184-byte string literal not displayed */
MaxParallelConnectionsPerDb int `` /* 241-byte string literal not displayed */
TryCreateListedExtsIfMissing string `` /* 290-byte string literal not displayed */
CreateHelpers bool `` /* 144-byte string literal not displayed */
}
SourceOpts specifies the sources related command-line options
type HostConfigAttrs ¶
type HostConfigAttrs struct {
DcsType string `yaml:"dcs_type"`
DcsEndpoints []string `yaml:"dcs_endpoints"`
Scope string
Namespace string
Username string
Password string
CAFile string `yaml:"ca_file"`
CertFile string `yaml:"cert_file"`
KeyFile string `yaml:"key_file"`
LogsGlobPath string `yaml:"logs_glob_path"` // default $data_directory / $log_directory / *.csvlog
LogsMatchRegex string `yaml:"logs_match_regex"` // default is for CSVLOG format. needs to capture following named groups: log_time, user_name, database_name and error_severity
PerMetricDisabledTimes []HostConfigPerMetricDisabledTimes `yaml:"per_metric_disabled_intervals"`
}
type Kind ¶
type Kind string
const ( SourcePostgres Kind = "postgres" SourcePostgresContinuous Kind = "postgres-continuous-discovery" SourcePgBouncer Kind = "pgbouncer" SourcePgPool Kind = "pgpool" SourcePatroni Kind = "patroni" SourcePatroniContinuous Kind = "patroni-continuous-discovery" SourcePatroniNamespace Kind = "patroni-namespace-discovery" )
type PatroniClusterMember ¶
type ReaderWriter ¶
func NewPostgresSourcesReaderWriter ¶
func NewPostgresSourcesReaderWriter(ctx context.Context, connstr string) (ReaderWriter, error)
func NewPostgresSourcesReaderWriterConn ¶
func NewPostgresSourcesReaderWriterConn(ctx context.Context, conn db.PgxPoolIface) (ReaderWriter, error)
func NewYAMLSourcesReaderWriter ¶
func NewYAMLSourcesReaderWriter(ctx context.Context, path string) (ReaderWriter, error)
type RuntimeInfo ¶ added in v3.3.0
type Source ¶
type Source struct {
Name string `yaml:"name" db:"name"`
Group string `yaml:"group" db:"group"`
ConnStr string `yaml:"conn_str" db:"connstr"`
Metrics map[string]float64 `yaml:"custom_metrics" db:"config"`
MetricsStandby map[string]float64 `yaml:"custom_metrics_standby" db:"config_standby"`
Kind Kind `yaml:"kind" db:"dbtype"`
IncludePattern string `yaml:"include_pattern" db:"include_pattern"`
ExcludePattern string `yaml:"exclude_pattern" db:"exclude_pattern"`
PresetMetrics string `yaml:"preset_metrics" db:"preset_config"`
PresetMetricsStandby string `yaml:"preset_metrics_standby" db:"preset_config_standby"`
IsEnabled bool `yaml:"is_enabled" db:"is_enabled"`
CustomTags map[string]string `yaml:"custom_tags" db:"custom_tags"`
HostConfig HostConfigAttrs `yaml:"host_config" db:"host_config"`
OnlyIfMaster bool `yaml:"only_if_master" db:"only_if_master"`
}
Source represents a configuration how to get databases to monitor. It can be a single database, a group of databases in postgres cluster, a group of databases in HA patroni cluster. pgbouncer and pgpool kinds are purely to indicate that the monitored database connection is made through a connection pooler, which supports its own additional metrics. If one is not interested in those additional metrics, it is ok to specify the connection details as a regular postgres source.
func (*Source) GetDatabaseName ¶
func (Source) ResolveDatabases ¶
func (s Source) ResolveDatabases() (SourceConns, error)
ResolveDatabases() return a slice of found databases for continuous monitoring sources, e.g. patroni
type SourceConn ¶ added in v3.2.0
type SourceConn struct {
Source
Conn db.PgxPoolIface
ConnConfig *pgxpool.Config
RuntimeInfo
}
SourceConn represents a single connection to monitor. Unlike source, it contains a database connection. Continuous discovery sources (postgres-continuous-discovery, patroni-continuous-discovery, patroni-namespace-discovery) will produce multiple monitored databases structs based on the discovered databases.
func (*SourceConn) Connect ¶ added in v3.2.0
func (md *SourceConn) Connect(ctx context.Context, opts CmdOpts) (err error)
Connect will establish a connection to the database if it's not already connected. If the connection is already established, it pings the server to ensure it's still alive.
func (*SourceConn) DiscoverPlatform ¶ added in v3.2.0
func (md *SourceConn) DiscoverPlatform(ctx context.Context) (platform string)
TryDiscoverPlatform tries to discover the platform based on the database version string and some special settings that are only available on certain platforms. Returns the platform name or "UNKNOWN" if not sure.
func (*SourceConn) FetchApproxSize ¶ added in v3.3.0
func (md *SourceConn) FetchApproxSize(ctx context.Context) (size int64)
FetchApproxSize returns the approximate size of the database in bytes
func (*SourceConn) FetchRuntimeInfo ¶ added in v3.3.0
func (md *SourceConn) FetchRuntimeInfo(ctx context.Context, forceRefetch bool) (err error)
func (*SourceConn) FetchVersion ¶ added in v3.3.0
func (*SourceConn) FunctionExists ¶ added in v3.2.0
func (md *SourceConn) FunctionExists(ctx context.Context, functionName string) (exists bool)
FunctionExists checks if a function exists in the database
func (*SourceConn) GetClusterIdentifier ¶ added in v3.3.0
func (md *SourceConn) GetClusterIdentifier() string
GetUniqueIdentifier returns a unique identifier for the host assuming SysId is the same for primary and all replicas but connection information is different
func (*SourceConn) GetDatabaseName ¶ added in v3.2.0
func (md *SourceConn) GetDatabaseName() string
GetDatabaseName returns the database name from the connection string
func (*SourceConn) GetMetricInterval ¶ added in v3.3.0
func (md *SourceConn) GetMetricInterval(name string) float64
GetMetricInterval returns the metric interval for the connection
func (*SourceConn) IsPostgresSource ¶ added in v3.2.0
func (md *SourceConn) IsPostgresSource() bool
func (*SourceConn) ParseConfig ¶ added in v3.2.0
func (md *SourceConn) ParseConfig() (err error)
ParseConfig will parse the connection string and store the result in the connection config
func (*SourceConn) Ping ¶ added in v3.2.0
func (md *SourceConn) Ping(ctx context.Context) (err error)
Ping will try to ping the server to ensure the connection is still alive
func (*SourceConn) SetDatabaseName ¶ added in v3.2.0
func (md *SourceConn) SetDatabaseName(name string)
SetDatabaseName sets the database name in the connection config for resolved databases
type SourceConns ¶ added in v3.2.0
type SourceConns []*SourceConn
SourceConn represents a single connection to monitor. Unlike source, it contains a database connection. Continuous discovery sources (postgres-continuous-discovery, patroni-continuous-discovery, patroni-namespace-discovery) will produce multiple monitored databases structs based on the discovered databases.
func ResolveDatabasesFromPatroni ¶
func ResolveDatabasesFromPatroni(source Source) (SourceConns, error)
func ResolveDatabasesFromPostgres ¶
func ResolveDatabasesFromPostgres(s Source) (resolvedDbs SourceConns, err error)
ResolveDatabasesFromPostgres reads all the databases from the given cluster, additionally matching/not matching specified regex patterns
func (SourceConns) GetMonitoredDatabase ¶ added in v3.2.0
func (mds SourceConns) GetMonitoredDatabase(DBUniqueName string) *SourceConn
func (SourceConns) SyncFromReader ¶ added in v3.2.0
func (mds SourceConns) SyncFromReader(r Reader) (newmds SourceConns, err error)
SyncFromReader will update the monitored databases with the latest configuration from the reader. Any resolution errors will be returned, e.g. etcd unavailability. It's up to the caller to proceed with the databases available or stop the execution due to errors.
type Sources ¶
type Sources []Source
func (Sources) ResolveDatabases ¶
func (srcs Sources) ResolveDatabases() (_ SourceConns, err error)
ResolveDatabases() updates list of monitored objects from continuous monitoring sources, e.g. patroni