Documentation
¶
Overview ¶
Package worker provides background task scheduling for the control plane. Workers handle periodic operations like health checks, state reconciliation, garbage collection, and certificate renewal.
Index ¶
- type CertRenewer
- type GCConfig
- type GarbageCollector
- type HealthRunner
- type LeaderElector
- type Reconciler
- type Scheduler
- func (s *Scheduler) IsLeader() bool
- func (s *Scheduler) Register(w Worker)
- func (s *Scheduler) SetLeaderElector(e LeaderElector)
- func (s *Scheduler) SetMetrics(m gometrics.Metrics)
- func (s *Scheduler) Start(ctx context.Context) error
- func (s *Scheduler) Stop(ctx context.Context) error
- func (s *Scheduler) WorkerByName(name string) (WorkerInfo, bool)
- func (s *Scheduler) Workers() []WorkerInfo
- type TelemetryCollector
- type Worker
- type WorkerInfo
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type CertRenewer ¶
type CertRenewer struct {
// contains filtered or unexported fields
}
CertRenewer periodically checks for expiring TLS certificates and renews them.
func NewCertRenewer ¶
NewCertRenewer creates a new certificate renewer worker.
func (*CertRenewer) Interval ¶
func (c *CertRenewer) Interval() time.Duration
Interval returns how often the certificate renewer should run.
type GCConfig ¶ added in v1.5.1
type GCConfig struct {
// MaxInstancesPerTick caps how many instances the orphan-instance
// pass examines per tenant per tick. The remainder is picked up
// by the next tick. Default 500.
MaxInstancesPerTick int
// InstanceGracePeriod is the minimum age an instance must reach
// before it is eligible for reaping, even when its parent
// workload appears to be missing. Catches mid-Provision races
// where the workload row is being inserted right after the
// instance. Default 15 minutes.
InstanceGracePeriod time.Duration
}
GCConfig tunes the garbage-collector's per-tick behaviour.
Defaults are conservative: 500 instances + 15 minutes of grace is enough to keep up with a busy tenant without causing the worker to run for more than a few seconds, and the grace period is wide enough that a workload row insert that happens shortly after its first replica is created never gets reaped pre-maturely.
type GarbageCollector ¶
type GarbageCollector struct {
// contains filtered or unexported fields
}
GarbageCollector periodically removes orphaned resources from the store.
Orphan classes handled today:
- Instances whose ctrlplane.workload label points at a Workload that no longer exists. These are produced by:
- a workload.Service.Delete that crashed mid-cascade,
- workload rows force-removed via direct store access,
- backup-restore drift between the workloads and instances tables.
The GC is a backstop, not the primary cleanup path. The convergent instance.Service.Delete is the normal flow; the cron exists so operator-visible state catches up eventually when that flow has been short-circuited.
Future passes (out of scope for the current implementation but flagged for follow-up): orphan releases / deployments tied to vanished instances, orphan health-check / domain / route rows, orphan certificates whose domain no longer exists. Each of those requires the matching child store to grow either a Delete method (deploy.Store has none today) or a tenant-global list, neither of which is in scope here.
func NewGarbageCollector ¶
func NewGarbageCollector( tenants admin.Store, instances instance.Service, workloads workload.Store, events event.Bus, interval time.Duration, cfg GCConfig, ) *GarbageCollector
NewGarbageCollector creates a GC worker.
The worker calls instance.Service.Delete (not the underlying store) so the convergent provider Deprovision runs and the regular InstanceDeleted event fires for every reap — the same observability operators see for an interactive delete.
func (*GarbageCollector) Interval ¶
func (g *GarbageCollector) Interval() time.Duration
Interval returns how often the garbage collector should run.
func (*GarbageCollector) Name ¶
func (g *GarbageCollector) Name() string
Name returns the worker name.
func (*GarbageCollector) Run ¶
func (g *GarbageCollector) Run(ctx context.Context) error
Run executes one garbage-collection cycle.
One pass per tenant: list instances → for each, if it carries the ctrlplane.workload label and the named workload is missing AND the instance is older than the grace window, call Delete. A per-tenant error is logged via a gc.tenant_failed event and the sweep continues to the next tenant — one broken tenant must not blind-spot the rest.
func (*GarbageCollector) SetMetrics ¶ added in v1.8.0
func (g *GarbageCollector) SetMetrics(m gometrics.Metrics)
SetMetrics counts tenant sweeps and orphan deletes on m. A nil m records nothing. Call it before the scheduler starts.
type HealthRunner ¶
type HealthRunner struct {
// contains filtered or unexported fields
}
HealthRunner periodically executes health checks for all instances.
func NewHealthRunner ¶
NewHealthRunner creates a new health runner worker.
func (*HealthRunner) Interval ¶
func (h *HealthRunner) Interval() time.Duration
Interval returns how often the health runner should run.
type LeaderElector ¶ added in v1.8.0
type LeaderElector interface {
// IsLeader reports whether this replica holds leadership, trying to
// take it when it does not. The scheduler calls it before every worker
// run, so it should be cheap when leadership is already held.
IsLeader(ctx context.Context) (bool, error)
// Resign gives leadership up so another replica can take it. It is
// safe to call when this replica is not the leader.
Resign(ctx context.Context) error
}
LeaderElector decides which replica runs the scheduler's workers. When you run more than one replica of Ctrl Plane against the same store, give each one an elector over a shared lock so only the leader reconciles, collects garbage and renews certificates; without one every replica runs every worker.
The postgres store ships one built on a session-level advisory lock; see postgres.Store.LeaderElector.
type Reconciler ¶
type Reconciler struct {
// contains filtered or unexported fields
}
Reconciler is the platform's per-tick driver for state that lives outside any single user request:
- Datacenter bootstrap services — declarative + hook-contributed shared infrastructure that auto-deploys on every datacenter.
- (Future) instance drift correction — detect rows where the persisted state disagrees with the provider and pull them back into agreement. Stub today; the bootstrap path is the first concrete consumer.
The reconciler runs under synthesized system claims (no tenant scoping) so it can call services that gate on auth.RequireClaims.
func NewReconciler ¶
func NewReconciler( instances instance.Store, datacenters datacenter.Store, bootstraps bootstrap.Service, providers *provider.Registry, events event.Bus, interval time.Duration, ) *Reconciler
NewReconciler wires the reconciler. The instances store stays in the signature for the future drift-correction pass; the bootstrap service is the only consumer today.
func (*Reconciler) Interval ¶
func (r *Reconciler) Interval() time.Duration
Interval returns how often the reconciler should run.
func (*Reconciler) Run ¶
func (r *Reconciler) Run(ctx context.Context) error
Run executes one reconciliation cycle.
Walks every datacenter and asks the bootstrap service to drive its state forward. Per-datacenter errors are isolated — a broken datacenter does not blind-spot the others.
The list call uses an empty tenantID, which the datacenter store's hybrid-visibility query treats as cross-tenant. Bootstrap workloads are platform-owned, not tenant-scoped, so the reconciler operates on the union of all datacenters across tenants on every tick.
type Scheduler ¶
type Scheduler struct {
// contains filtered or unexported fields
}
Scheduler manages a set of workers, running each on its configured interval.
func NewScheduler ¶
func NewScheduler() *Scheduler
NewScheduler creates a new scheduler with an empty worker list.
func (*Scheduler) IsLeader ¶ added in v1.8.0
IsLeader reports whether the scheduler last found itself leader. It is always true when no elector is set.
func (*Scheduler) SetLeaderElector ¶ added in v1.8.0
func (s *Scheduler) SetLeaderElector(e LeaderElector)
SetLeaderElector makes the scheduler run workers only while e reports this replica as leader. Call it before Start. Without an elector every replica runs every worker, which is right for a single replica.
func (*Scheduler) SetMetrics ¶ added in v1.8.0
SetMetrics records how long each worker pass takes, which ticks were skipped for want of leadership, and the leader election answers on m. Call it before Start. A nil m records nothing. In a Forge app the ctrlplane extension passes app.Metrics() in for you.
func (*Scheduler) Start ¶
Start launches a goroutine for each periodic worker. It returns immediately.
The workers run on ctx without whatever span it carries: Forge starts extensions under a boot span, and a worker's spans and queries must not join that trace. Each worker starts a root span per unit of work.
func (*Scheduler) Stop ¶
Stop cancels all running workers, waits for them to finish and, when an elector is set, resigns leadership so another replica can take over without waiting for this one's lock to expire.
func (*Scheduler) WorkerByName ¶ added in v1.3.0
func (s *Scheduler) WorkerByName(name string) (WorkerInfo, bool)
WorkerByName returns runtime info for a specific worker.
func (*Scheduler) Workers ¶ added in v1.3.0
func (s *Scheduler) Workers() []WorkerInfo
Workers returns a snapshot of all registered workers with their runtime status.
type TelemetryCollector ¶
type TelemetryCollector struct {
// contains filtered or unexported fields
}
TelemetryCollector periodically gathers metrics and resource usage from providers.
func NewTelemetryCollector ¶
func NewTelemetryCollector(telemetry telemetry.Service, providers *provider.Registry, interval time.Duration) *TelemetryCollector
NewTelemetryCollector creates a new telemetry collector worker.
func (*TelemetryCollector) Interval ¶
func (t *TelemetryCollector) Interval() time.Duration
Interval returns how often the telemetry collector should run.
func (*TelemetryCollector) Name ¶
func (t *TelemetryCollector) Name() string
Name returns the worker name.
func (*TelemetryCollector) Run ¶
func (t *TelemetryCollector) Run(_ context.Context) error
Run executes one telemetry collection cycle. TODO: implement telemetry collection. Query each registered provider for current resource usage and metrics, then push the data through the telemetry service for storage and aggregation.
type Worker ¶
type Worker interface {
// Name identifies the worker.
Name() string
// Interval returns how often the worker should run.
// Return 0 for event-driven workers that do not run periodically.
Interval() time.Duration
// Run executes one cycle of the worker.
Run(ctx context.Context) error
}
Worker is a background task that runs periodically.
type WorkerInfo ¶ added in v1.3.0
type WorkerInfo struct {
Name string `db:"name" json:"name"`
Interval time.Duration `db:"interval" json:"interval"`
Running bool `db:"running" json:"running"`
LastRun *time.Time `db:"last_run" json:"last_run,omitempty"`
LastErr string `db:"last_error" json:"last_error,omitempty"`
RunCount int64 `db:"run_count" json:"run_count"`
}
WorkerInfo holds runtime information about a registered worker.
Source Files