health

package
v0.5.9 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 8, 2026 License: MIT Imports: 12 Imported by: 0

Documentation

Overview

Package health reports whether an osctrl deployment is working: the database and Redis it depends on, the services that make it up, and the background workers inside them.

Two collection strategies, because osctrl-api and osctrl-tls are separate processes in separate containers:

  • Anything osctrl-api can reach itself (DB ping, Redis PING, its own Go runtime) is computed when the operator asks for it.
  • osctrl-tls upserts one heartbeat row that osctrl-api reads. The database is the only channel between the two, the same conclusion pkg/servicecommands reached for restarts.

Index

Constants

View Source 
const (
	StatusOperational = "operational"
	StatusDegraded    = "degraded"
	StatusDown        = "down"
	StatusStale       = "stale"
	StatusUnknown     = "unknown"
)

Status values a component can report.

View Source 
const (
	// HeartbeatInterval is how often a service upserts its row. Fixed, not
	// a setting: the staleness threshold is derived from it and a second
	// knob would only let the two drift apart.
	HeartbeatInterval = 60 * time.Second
	// StaleAfter is when a heartbeat stops counting as live. Three missed
	// writes: one or two are scheduling noise, three mean something broke.
	StaleAfter = 3 * HeartbeatInterval
)

Variables

View Source 
var ErrNotReporting = errors.New("service has never reported health")

ErrNotReporting is returned when a service has no heartbeat row: it never ran with --health-enabled, or never started. It is deliberately distinct from "unhealthy" — the API renders it as unknown, not down.

Functions

This section is empty.

Types

type AlertsWorkerStats 

type AlertsWorkerStats struct {
	Enabled       bool   `json:"enabled"`
	QueueDepth    int    `json:"queue_depth"`
	QueueCapacity int    `json:"queue_capacity"`
	Matched       uint64 `json:"matched"`
	Dispatched    uint64 `json:"dispatched"`
	Collapsed     uint64 `json:"collapsed"`
	Dropped       uint64 `json:"dropped"`
	Failed        uint64 `json:"failed"`
}

AlertsWorkerStats mirrors alerts.WorkerSnapshot.

type Component 

type Component struct {
	ID      string         `json:"id"`
	Name    string         `json:"name"`
	Status  string         `json:"status"`
	Summary string         `json:"summary"`
	Details map[string]any `json:"details,omitempty"`
}

Component is one row on the health page.

func DatabaseComponent 

func DatabaseComponent(degraded bool, pingErr error, latency time.Duration) Component

DatabaseComponent reports the backend database. degraded comes from the existing pkg/backend health monitor; pingErr from a live ping.

func RedisComponent 

func RedisComponent(pingErr error, latency time.Duration) Component

RedisComponent reports Redis reachability from a PING.

func ServiceComponent 

func ServiceComponent(row ServiceStatus, err error, now time.Time, apiVersion string) Component

ServiceComponent turns a heartbeat row into a component. apiVersion is the version of the process serving the request, so skew between services is visible rather than silent.

func WorkersComponent 

func WorkersComponent(row ServiceStatus, err error, now time.Time) Component

WorkersComponent reports the background workers inside osctrl-tls, decoded from the heartbeat payload. It inherits staleness from the heartbeat: counters from a dead process say nothing about now.

type Manager 

type Manager struct {
	DB *gorm.DB
}

Manager owns the service_status table.

func NewManager 

func NewManager(db *gorm.DB) *Manager

NewManager initializes the manager and auto-migrates its table. It is only called when --health-enabled is set, so a deployment with the feature off never creates the table.

func (*Manager) Get 

func (m *Manager) Get(service string) (ServiceStatus, error)

Get returns one service's heartbeat, or ErrNotReporting.

func (*Manager) GetContext 

func (m *Manager) GetContext(ctx context.Context, service string) (ServiceStatus, error)

GetContext is Get with a caller-supplied context, so callers that need a bounded deadline (e.g. the health endpoint) are not left waiting forever on a saturated connection pool.

func (*Manager) Report 

func (m *Manager) Report(s ServiceStatus) error

Report upserts one service's heartbeat.

type RuntimeStats 

type RuntimeStats struct {
	UptimeSeconds int64  `json:"uptime_seconds"`
	Goroutines    int    `json:"goroutines"`
	Alloc         uint64 `json:"alloc"`
	TotalAlloc    uint64 `json:"total_alloc"`
	Sys           uint64 `json:"sys"`
	Lookups       uint64 `json:"lookups"`
	Mallocs       uint64 `json:"mallocs"`
	Frees         uint64 `json:"frees"`
	HeapAlloc     uint64 `json:"heap_alloc"`
	HeapSys       uint64 `json:"heap_sys"`
	HeapIdle      uint64 `json:"heap_idle"`
	HeapInuse     uint64 `json:"heap_inuse"`
	HeapReleased  uint64 `json:"heap_released"`
	HeapObjects   uint64 `json:"heap_objects"`
	StackInuse    uint64 `json:"stack_inuse"`
	StackSys      uint64 `json:"stack_sys"`
	MSpanInuse    uint64 `json:"mspan_inuse"`
	MSpanSys      uint64 `json:"mspan_sys"`
	MCacheInuse   uint64 `json:"mcache_inuse"`
	MCacheSys     uint64 `json:"mcache_sys"`
	BuckHashSys   uint64 `json:"buck_hash_sys"`
	GCSys         uint64 `json:"gc_sys"`
	OtherSys      uint64 `json:"other_sys"`
	NextGC        uint64 `json:"next_gc"`
	LastGC        int64  `json:"last_gc_unix_ms"`
	PauseTotalNs  uint64 `json:"pause_total_ns"`
	LastPauseNs   uint64 `json:"last_pause_ns"`
	NumGC         uint32 `json:"num_gc"`
}

RuntimeStats is the Go runtime detail shown per service. Field set mirrors what an operator expects from runtime.MemStats.

func SampleRuntimeMetrics 

func SampleRuntimeMetrics(startedAt time.Time) RuntimeStats

SampleRuntimeMetrics reports the runtime state without stopping the world. Use it anywhere sampling happens on a timer; use Snapshot on a request path where the exact MemStats numbers are worth a brief pause.

func Snapshot 

func Snapshot(startedAt time.Time) RuntimeStats

Snapshot reads the current runtime state.

runtime.ReadMemStats STOPS THE WORLD. The pause is short (tens of µs) and harmless when an operator loads a page, which is the only thing that calls this. Do not put it on a ticker inside osctrl-tls — the heartbeat carries NumGoroutine and existing atomics instead, and runtime/metrics (no STW) is the tool if periodic memory sampling is ever wanted.

type ServiceStatus 

type ServiceStatus struct {
	gorm.Model
	// Service is "tls" or "api". v1 only ever writes "tls": osctrl-api
	// reports live because it serves the request. The column accepts both
	// so a future API-side writer needs no migration.
	Service string `gorm:"uniqueIndex;size:16"`
	// Version is the build version of the reporting process, so the API can
	// spot version skew between services.
	Version string `gorm:"size:64"`
	// StartedAt is when the reporting process booted (uptime = now - this).
	StartedAt time.Time
	// ReportedAt is when this row was last written (liveness).
	ReportedAt time.Time
	// Goroutines is runtime.NumGoroutine() at write time — a plain load,
	// free to sample, unlike ReadMemStats which stops the world.
	Goroutines int
	// Payload is the JSON worker snapshot. See WorkerPayload.
	Payload string `gorm:"type:text"`
}

ServiceStatus is the heartbeat one service writes for the others to read. Upserted on Service, so the table holds one row per service forever and needs no retention sweep.

func (ServiceStatus) TableName 

func (ServiceStatus) TableName() string

TableName pins the table so a struct rename cannot silently orphan rows.

type UpgradeInfo 

type UpgradeInfo struct {
	Current   string    `json:"current"`
	Suggested string    `json:"suggested,omitempty"`
	Latest    string    `json:"latest,omitempty"`
	UpToDate  bool      `json:"up_to_date"`
	Checked   bool      `json:"checked"`
	CheckedAt time.Time `json:"checked_at,omitempty"`
	MoreInfo  string    `json:"more_information,omitempty"`
	// API and TLS expose the per-service versions behind Skew so the page
	// can name both sides of a mismatch.
	API  string `json:"api_version,omitempty"`
	TLS  string `json:"tls_version,omitempty"`
	Skew bool   `json:"skew"`
}

UpgradeInfo is the upgrade block on the health page.

type VersionCache 

type VersionCache struct {
	// contains filtered or unexported fields
}

VersionCache holds the last upstream version check.

version.RetrieveVersionData performs an external HTTP request, so it must never run on the request path. Refresh is called at boot and on a 24h ticker; Info only reads memory.

func NewVersionCache 

func NewVersionCache(current string) *VersionCache

NewVersionCache returns a cache for the running build version.

func (*VersionCache) Info 

func (c *VersionCache) Info(tlsVersion string) UpgradeInfo

Info renders the upgrade block. tlsVersion comes from the heartbeat row and may be empty when osctrl-tls is not reporting.

func (*VersionCache) Refresh 

func (c *VersionCache) Refresh(url string) error

Refresh fetches upstream version data. Errors are returned, not stored: a failed check leaves the previous answer in place rather than blanking the page because stats.osctrl.net was briefly unreachable.

func (*VersionCache) Set 

func (c *VersionCache) Set(data version.VersionData)

Set stores version data without fetching. Used by Refresh and by tests.

type WorkerPayload 

type WorkerPayload struct {
	Alerts *AlertsWorkerStats `json:"alerts,omitempty"`
	// Runtime is the reporting service's Go runtime state, sampled through
	// runtime/metrics rather than ReadMemStats — see SampleRuntimeMetrics for
	// why, and for the two fields that sampler cannot fill.
	Runtime *RuntimeStats `json:"runtime,omitempty"`
}

WorkerPayload is the JSON osctrl-tls writes in ServiceStatus.Payload. Everything in it is free to read — atomics, channel lengths, and runtime/metrics counters; nothing here stops the world. Subsystems that are disabled are omitted, so the page shows what the deployment actually runs.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.