Documentation
¶
Overview ¶
Package telemetry defines bounded-cardinality Prometheus telemetry for the reconciliation state machines of both resource families.
Index ¶
- func UnresolvedMigrationRecords(migrations []operatorv1alpha1.PtahMigration) []time.Time
- func UnresolvedSchemaRecords(schemas []operatorv1alpha1.PtahSchema) []time.Time
- type ApplyOutcome
- type ApprovalOutcome
- type CertificateCollector
- type DriftOutcome
- type FailureCategory
- type FailureStage
- type Metrics
- func (m *Metrics) ObserveApply(family ResourceFamily, outcome ApplyOutcome)
- func (m *Metrics) ObserveApproval(family ResourceFamily, outcome ApprovalOutcome)
- func (m *Metrics) ObserveDrift(engine operatorv1alpha1.DatabaseEngine, outcome DriftOutcome)
- func (m *Metrics) ObserveFailure(family ResourceFamily, stage FailureStage, category FailureCategory)
- func (m *Metrics) ObserveOperation(family ResourceFamily, operation Operation, outcome OperationOutcome, ...)
- func (m *Metrics) ObservePlan(family ResourceFamily, engine operatorv1alpha1.DatabaseEngine, ...)
- func (m *Metrics) ObserveReconciliation(family ResourceFamily, result ReconciliationResult)
- type Observer
- type Operation
- type OperationOutcome
- type PlanImpact
- type PlanStoreCollector
- type PlanStoreView
- type ReconciliationResult
- type ResourceFamily
- type SnapshotCollector
- type StateCollector
- type StateView
- type UnresolvedCollector
- type UnresolvedView
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func UnresolvedMigrationRecords ¶
func UnresolvedMigrationRecords(migrations []operatorv1alpha1.PtahMigration) []time.Time
UnresolvedMigrationRecords returns when each migration recorded a run nobody accounted for.
func UnresolvedSchemaRecords ¶
func UnresolvedSchemaRecords(schemas []operatorv1alpha1.PtahSchema) []time.Time
UnresolvedSchemaRecords returns, per schema owing proof, the instant the operator could first have settled it: a pending observation is settleable once its immutable horizon has passed, because until then a mutating Pod may still be running and no reading would be evidence. A migration record is settleable as soon as it exists, so the two families report the same quantity from different fields, which is why this lives here and not in the caller.
Types ¶
type ApplyOutcome ¶
type ApplyOutcome string
ApplyOutcome is a closed set of apply lifecycle observations.
const ( ApplyStarted ApplyOutcome = "started" ApplyCompleted ApplyOutcome = "completed" ApplyUncertain ApplyOutcome = "uncertain" ApplyStale ApplyOutcome = "stale" )
type ApprovalOutcome ¶
type ApprovalOutcome string
ApprovalOutcome is a closed set of approval lifecycle observations.
const ( ApprovalRequired ApprovalOutcome = "required" ApprovalAccepted ApprovalOutcome = "accepted" ApprovalStale ApprovalOutcome = "stale" )
type CertificateCollector ¶
type CertificateCollector struct {
// contains filtered or unexported fields
}
CertificateCollector publishes when the serving certificate this process answers admission with stops being valid.
Every replica serves admission, so every replica publishes the certificate it would present, and an alert takes the earliest. The certificate rotator replaces the Secret long before this instant; a value that keeps falling towards now is a rotation that is not happening, which is what the alert on it is for. It is read from the file on every scrape rather than cached, because the file is what the webhook server reloads.
func NewCertificateCollector ¶
func NewCertificateCollector(registerer prometheus.Registerer, path string) *CertificateCollector
NewCertificateCollector registers the gauge for the PEM certificate at path.
func (*CertificateCollector) Collect ¶
func (c *CertificateCollector) Collect(into chan<- prometheus.Metric)
Collect publishes the expiry, or nothing and a counted failure when the file cannot be read: an absent expiry is not a certificate that never expires.
The failure counter is emitted from here, after the read, so the scrape that failed is the one that reports it. Registered on its own, the registry could gather it before this ran and publish the count from before.
func (*CertificateCollector) Describe ¶
func (c *CertificateCollector) Describe(into chan<- *prometheus.Desc)
type DriftOutcome ¶
type DriftOutcome string
DriftOutcome is a closed set of database observation outcomes.
const ( DriftDetected DriftOutcome = "detected" DriftInSync DriftOutcome = "in_sync" )
type FailureCategory ¶
type FailureCategory string
FailureCategory identifies a bounded, operationally useful failure class.
const ( FailureInfrastructure FailureCategory = "infrastructure" FailureConfiguration FailureCategory = "configuration" FailureOperation FailureCategory = "operation" FailurePolicyChanged FailureCategory = "policy_changed" FailureStaleInput FailureCategory = "stale_input" FailureUncertain FailureCategory = "uncertain_outcome" )
type FailureStage ¶
type FailureStage string
FailureStage identifies the bounded state-machine stage that observed a failure. It deliberately excludes object identity and arbitrary error text.
const ( FailureStageController FailureStage = "controller" FailureStageResolve FailureStage = "resolve" FailureStageVerify FailureStage = "verify" FailureStageObserve FailureStage = "observe" FailureStagePlan FailureStage = "plan" FailureStageApply FailureStage = "apply" FailureStageHistory FailureStage = "history" )
func StageForMigrationOperation ¶
func StageForMigrationOperation(operation operatorv1alpha1.MigrationOperationType) FailureStage
StageForMigrationOperation maps the migration operation enum to a closed failure label. History is its own stage: a migration reads back what the revision table records, which no schema operation does.
func StageForOperation ¶
func StageForOperation(operation operatorv1alpha1.OperationType) FailureStage
StageForOperation maps the public operation enum to a closed failure label.
type Metrics ¶
type Metrics struct {
// contains filtered or unexported fields
}
Metrics implements Observer with Prometheus collectors.
func New ¶
func New(registerer prometheus.Registerer) *Metrics
New constructs and registers one independent collector set. Call it once per registry; tests should pass their own prometheus.Registry.
func (*Metrics) ObserveApply ¶
func (m *Metrics) ObserveApply(family ResourceFamily, outcome ApplyOutcome)
func (*Metrics) ObserveApproval ¶
func (m *Metrics) ObserveApproval(family ResourceFamily, outcome ApprovalOutcome)
func (*Metrics) ObserveDrift ¶
func (m *Metrics) ObserveDrift(engine operatorv1alpha1.DatabaseEngine, outcome DriftOutcome)
func (*Metrics) ObserveFailure ¶
func (m *Metrics) ObserveFailure(family ResourceFamily, stage FailureStage, category FailureCategory)
func (*Metrics) ObserveOperation ¶
func (m *Metrics) ObserveOperation(family ResourceFamily, operation Operation, outcome OperationOutcome, duration time.Duration)
func (*Metrics) ObservePlan ¶
func (m *Metrics) ObservePlan(family ResourceFamily, engine operatorv1alpha1.DatabaseEngine, impact PlanImpact)
func (*Metrics) ObserveReconciliation ¶
func (m *Metrics) ObserveReconciliation(family ResourceFamily, result ReconciliationResult)
type Observer ¶
type Observer interface {
ObserveReconciliation(ResourceFamily, ReconciliationResult)
ObserveDrift(operatorv1alpha1.DatabaseEngine, DriftOutcome)
ObservePlan(ResourceFamily, operatorv1alpha1.DatabaseEngine, PlanImpact)
ObserveApproval(ResourceFamily, ApprovalOutcome)
ObserveApply(ResourceFamily, ApplyOutcome)
ObserveOperation(ResourceFamily, Operation, OperationOutcome, time.Duration)
ObserveFailure(ResourceFamily, FailureStage, FailureCategory)
}
Observer is the controller-facing telemetry contract. Implementations must not derive labels from Kubernetes object identity, content digests, Secret data, or error strings.
type Operation ¶
type Operation string
Operation is the logical operation an observation belongs to, across both families. A migration reads its recorded history where a schema observes the live database, so the set is the union rather than either family's enum: mapping one family's operation onto the other's leaves a real operation labeled "unknown".
func OperationForMigration ¶
func OperationForMigration(operation operatorv1alpha1.MigrationOperationType) Operation
OperationForMigration maps the migration operation enum to the shared label.
func OperationForSchema ¶
func OperationForSchema(operation operatorv1alpha1.OperationType) Operation
OperationForSchema maps the schema operation enum to the shared label.
type OperationOutcome ¶
type OperationOutcome string
OperationOutcome is a closed set of logical operation outcomes.
const ( OperationSucceeded OperationOutcome = "success" OperationUncertain OperationOutcome = "uncertain" OperationStale OperationOutcome = "stale" OperationCanceled OperationOutcome = "canceled" )
type PlanImpact ¶
type PlanImpact string
PlanImpact says whether applying a plan can lose data.
It is three-valued because only a schema plan is examined for it. Nothing computes it for a migration, whose statements are written by hand, so a migration plan reports "unknown" rather than claiming a check that never ran. The two definite values keep the spelling a boolean label had.
const ( PlanDestructive PlanImpact = "true" PlanNonDestructive PlanImpact = "false" PlanImpactUnknown PlanImpact = "unknown" )
func DestructivePlan ¶
func DestructivePlan(destructive bool) PlanImpact
DestructivePlan maps a computed destructive flag onto the label.
type PlanStoreCollector ¶
type PlanStoreCollector struct {
// contains filtered or unexported fields
}
PlanStoreCollector publishes how many plans each family retains and how many bytes the schema plans hold in their chunks.
func NewPlanStoreCollector ¶
func NewPlanStoreCollector(registerer prometheus.Registerer, view PlanStoreView) *PlanStoreCollector
NewPlanStoreCollector registers the plan store gauges on one registry.
func (*PlanStoreCollector) Collect ¶
func (c *PlanStoreCollector) Collect(into chan<- prometheus.Metric)
Collect rebuilds the gauges. A view that has not synchronized, or a read that fails, publishes nothing rather than a store of zero.
func (*PlanStoreCollector) Describe ¶
func (c *PlanStoreCollector) Describe(into chan<- *prometheus.Desc)
type PlanStoreView ¶
type PlanStoreView interface {
Synced() bool
ListSchemaPlans(ctx context.Context) ([]operatorv1alpha1.PtahSchemaPlan, error)
ListMigrationPlans(ctx context.Context) ([]operatorv1alpha1.PtahMigrationPlan, error)
}
PlanStoreView lists the plans the store gauges are built from.
type ReconciliationResult ¶
type ReconciliationResult string
ReconciliationResult is a closed set of reconciliation outcomes.
const ( ReconciliationSucceeded ReconciliationResult = "success" ReconciliationFailed ReconciliationResult = "error" )
type ResourceFamily ¶
type ResourceFamily string
ResourceFamily says which family of resources an observation came from. Both families run the same lifecycle shape against the same database, and an alert on an uncertain outcome wants both; a reader who needs one asks for it by label.
const ( FamilySchema ResourceFamily = "schema" FamilyMigration ResourceFamily = "migration" )
type SnapshotCollector ¶
type SnapshotCollector struct {
// contains filtered or unexported fields
}
SnapshotCollector reads current API state once per leader scrape and shares it across the unresolved, resource-state and plan-store collectors. A warm informer cache can keep answering after its API access has failed; it cannot establish whether the state behind these alerts is still readable.
The reader must be the manager's APIReader, not its cached client. Lists do not request resourceVersion=0: an arbitrarily stale API cache is not enough. The four kinds are not an atomic cross-kind snapshot. Any failed list rejects the entire reading, so no other gauge presents a partial census as complete. Plan chunks and ConfigMap payloads are never listed.
func NewSnapshotCollector ¶
func NewSnapshotCollector(registerer prometheus.Registerer, reader client.Reader, now func() time.Time) *SnapshotCollector
NewSnapshotCollector registers the state metrics as one collection. Add the returned collector to the manager as a leader-elected Runnable as well.
func (*SnapshotCollector) Collect ¶
func (c *SnapshotCollector) Collect(into chan<- prometheus.Metric)
func (*SnapshotCollector) Describe ¶
func (c *SnapshotCollector) Describe(into chan<- *prometheus.Desc)
func (*SnapshotCollector) NeedLeaderElection ¶
func (*SnapshotCollector) NeedLeaderElection() bool
NeedLeaderElection keeps a healthy follower from hiding a lost leader scrape.
type StateCollector ¶
type StateCollector struct {
// contains filtered or unexported fields
}
StateCollector publishes the current population of resources by phase, the ones overdue for their own next reconciliation, the operations in flight and their age, and the lock releases still owed.
func NewStateCollector ¶
func NewStateCollector(registerer prometheus.Registerer, view StateView, now func() time.Time) *StateCollector
NewStateCollector registers the state gauges on one registry.
func (*StateCollector) Collect ¶
func (c *StateCollector) Collect(into chan<- prometheus.Metric)
Collect rebuilds the gauges. A view that has not synchronized, or a read that fails, publishes nothing: the unresolved collector reports that state and counts the failure, and the two read the same view.
func (*StateCollector) Describe ¶
func (c *StateCollector) Describe(into chan<- *prometheus.Desc)
type StateView ¶
type StateView interface {
Synced() bool
ListSchemas(ctx context.Context) ([]operatorv1alpha1.PtahSchema, error)
ListMigrations(ctx context.Context) ([]operatorv1alpha1.PtahMigration, error)
}
StateView lists the resources the state gauges are built from.
type UnresolvedCollector ¶
type UnresolvedCollector struct {
// contains filtered or unexported fields
}
UnresolvedCollector publishes the current population and age of unresolved mutating work.
func NewUnresolvedCollector ¶
func NewUnresolvedCollector(registerer prometheus.Registerer, view UnresolvedView, now func() time.Time) *UnresolvedCollector
NewUnresolvedCollector registers the gauges on one registry.
func (*UnresolvedCollector) Collect ¶
func (c *UnresolvedCollector) Collect(into chan<- prometheus.Metric)
Collect rebuilds the gauges from durable state. The failure counter is emitted from here, after the read, so a scrape that failed reports its own failure rather than the count from before it.
func (*UnresolvedCollector) Describe ¶
func (c *UnresolvedCollector) Describe(into chan<- *prometheus.Desc)
type UnresolvedView ¶
type UnresolvedView interface {
// Synced reports whether the view has caught up. A false answer means the
// counts below are not evidence of anything.
Synced() bool
// UnresolvedSchemas returns, for each PtahSchema owing proof of an Apply
// nobody accounted for, the instant the operator could first have settled
// it. A zero time means the resource is counted with no age.
UnresolvedSchemas(ctx context.Context) ([]time.Time, error)
// UnresolvedMigrations returns the same for each PtahMigration carrying a
// run nobody accounted for.
UnresolvedMigrations(ctx context.Context) ([]time.Time, error)
}
UnresolvedView lists the durable records the gauges are built from. It is an interface so the manager can pass one bounded API reading and a test can pass a fixture, and so the synchronized answer comes from whoever actually knows.