Documentation
¶
Overview ¶
Package monitor is the alerting engine. It watches container state (Docker events), resource usage (polled stats), restart frequency, and log output, evaluates user-defined rules, and dispatches matches to the in-app feed and configured webhooks. It also maintains a stats snapshot for the Prometheus exporter.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func SendMail ¶
SendMail delivers one message via the configured SMTP server. It supports implicit TLS (cfg.TLS, e.g. port 465) and otherwise opportunistically upgrades with STARTTLS when the server offers it. Exported so the API can send a test mail.
ctx bounds the ENTIRE call — dial, TLS handshake, and the full SMTP dialog (EHLO/AUTH/MAIL/RCPT/DATA/QUIT) — not just the initial connect. Neither net/smtp nor crypto/tls's plain Dial takes a context, so this dials via net.Dialer/tls.Dialer's *Context variants and then applies ctx's deadline directly to the resulting connection; without that second step, a server that accepts the TCP/TLS handshake and then stalls before its greeting could still hang indefinitely once past the dial itself.
Types ¶
type ContainerStat ¶
type ContainerStat struct {
HostID int64
HostName string
ID string
Name string
State string
// Project is the container's compose project (stack) name, read from its
// com.docker.compose.project label — "" for a container not managed by
// Compose. Used to scope maintenance windows by stack; nothing else in
// the engine reads it today.
Project string
CPUPercent float64 // `docker stats` convention: 100% == one core
CPUCores float64 // cores the daemon reports, so CPUPercent can be normalised
MemBytes uint64
MemLimit uint64
MemPercent float64 // of the container's limit, not of host RAM
// Cumulative network counters, summed across the container's interfaces.
// Kept raw so history can derive rates at read time.
NetRx uint64
NetTx uint64
NetDrops uint64 // rx+tx dropped packets
NetErrors uint64 // rx+tx interface errors
// Rates derived from the previous poll. Zero until a second sample exists,
// and after a counter reset — a container that was just recreated has no
// meaningful rate yet, and inventing one would be worse than showing none.
NetRxRate float64 // bytes/s
NetTxRate float64 // bytes/s
// True only when applyNetRates actually computed the corresponding rate
// THIS poll — a failed sample, a first-ever sample, or a counter reset
// all leave the rate at its zero value but these at false. Without this,
// a netrx_rate/nettx_rate resource rule couldn't tell "measured, genuinely
// under threshold" from "couldn't measure it this poll", and the latter
// would read as a legitimate recovery (see evalResourceRules' unmeasured
// map) — turning one ongoing incident into a resolve/fire pair every time
// a sample was missed.
NetRxRateOK bool
NetTxRateOK bool
// Sampled is true only when this poll's SampleStats call for the
// container actually succeeded. A container stays in the snapshot (it's
// still running, per ListContainers) even when the stats call itself
// times out or errors — every numeric field above is then just its zero
// value, not a real reading. Without this flag that zero looked exactly
// like a legitimate counter reset to recordHistory/applyNetRates, so a
// single transient stats timeout on a busy container could read as "lost
// everything, then gained it all back" the moment sampling recovered —
// exactly the kind of spike the network rule type and Top Talkers are
// supposed to tell apart from a real incident.
Sampled bool
}
ContainerStat is the cached per-container snapshot used by the exporter.
type HostHealth ¶ added in v1.4.0
type HostHealth struct {
Reachable bool
Since time.Time
LastCheck time.Time
Err string // last ping error when unreachable
}
HostHealth is the engine's cached view of one host's reachability. Since is when the current reachable/unreachable state began, so the UI can show how long a host has been down (or up).
type Monitor ¶
type Monitor struct {
// contains filtered or unexported fields
}
Monitor is the long-running alert engine.
func (*Monitor) HostHealth ¶ added in v1.4.0
func (m *Monitor) HostHealth() map[int64]HostHealth
HostHealth returns a snapshot of every tracked host's reachability, keyed by host id, for the API to surface in the hosts list. Hosts that have not been probed yet are absent (callers treat them as reachable/unknown).
func (*Monitor) NotifySystem ¶ added in v1.7.0
func (m *Monitor) NotifySystem(ev *store.AlertEvent) error
NotifySystem records and delivers a system-generated alert event that has no per-rule webhook/email configuration to key off — the same shape fireHostAlert used for host-reachability alerts before this was pulled out so other system-triggered notifications (e.g. an available image update) can reuse it instead of hand-rolling InsertAlertEvent + a maintenance-window check + emailNotify themselves. ev.HostID/Project/ ContainerName/Severity are used to scope the maintenance-window lookup exactly like a rule-driven fire() does; ev.Suppressed/SuppressedBy are set on ev itself before it's inserted. Uses its own short-lived context rather than a caller-supplied one, so a slow/cancelled caller can't abort the write — the same reasoning fireHostAlert already relied on. NotifySystem's error return is nil once ev is durably recorded in the alert log (suppressed or not) — a caller with its own idempotency state keyed off "was this notified" (e.g. the image-update poller's dedup table) should only update that state once this returns nil; a non-nil error means the event was never recorded at all, so treating it as "notified" would permanently and silently lose the notification.
func (*Monitor) SetStatsInterval ¶ added in v1.4.0
SetStatsInterval overrides how often the stats sweep runs. Values ≤ 0 are ignored (the default stands). Call before Run; it is not safe to change once the loop is running.
func (*Monitor) Snapshot ¶
func (m *Monitor) Snapshot() []ContainerStat
Snapshot returns a copy of the latest per-container stats for the exporter.