autoscaler

Overview ¶

pkg/autoscaler/autoscale_cross_metrics.go

CrossMetricsRegistry — a shared registry of AutoMetrics instances, one per operatorbox. Enables autoscale conditions to reference another operatorbox's live metrics via the cross.metrics.* namespace.

Usage in Katalog:

# The database-backed-app operatorbox scales based on the database
# operatorbox's queue depth — if the DB is overwhelmed, slow down.
operatorBox:
  autoscale:
    conditions:
      when:
        - field: cross.managed-database.metrics.queueDepth
          greaterThan: "500"
    do:
      workers: 2   # back off while DB is under load

Resolution:

cross.<crd>.metrics.<field>
  → CrossMetricsRegistry.Get(<crd>).Get("metrics.<field>")

This is read-only. An operatorbox can observe another's metrics but cannot modify them. The same principle as cross-CRD CR state observation.

Thread safety: Register is called once at startup per operatorbox (under Kordinator init). Get is called on every autoscale tick (concurrent reads). sync.Map provides safe concurrent access with no lock contention on reads.

pkg/autoscaler/autoscale_metrics.go

AutoMetrics — live operatorbox metrics read directly from the runtime.

The autoscaler evaluates conditions against these metrics on every tick. All reads are atomic — no locks, no API calls, no informer lookups.

Metric fields are exposed under the "metrics.*" namespace in autoscale conditions and resolve through AutoMetrics.Get(field).

pkg/autoscaler/autoscale_semaphore.go

ResizableSemaphore — a weighted semaphore whose capacity can be changed at runtime.

The worker pool in Orkestra is gated by this semaphore rather than being a fixed goroutine count. All worker goroutines run continuously in a loop; the semaphore controls how many may enter the reconcile section simultaneously.

Increasing capacity: new goroutines can acquire immediately. Decreasing capacity: goroutines currently inside reconcile finish their current work. Goroutines blocked at Acquire wait until capacity is available. In-flight reconciles are never interrupted.

This is the correct primitive for operator autoscaling because:

• Resize is O(1) — no goroutine creation or cancellation
• In-flight reconciles complete cleanly after a scale-down
• The semaphore is the single source of truth for concurrency

pkg/autoscaler/autoscale_worker_info.go

WorkerInfo — the complete picture of an operatorbox's worker state. Returned by the CRD handler endpoint and shown in the Control Center.

The Control Center shows:

Configured: 4      ← CRD declared workers (always the baseline)
Effective:  12     ← current semaphore capacity (may be overridden)
InFlight:   7      ← goroutines currently reconciling
Idle:       5      ← effective - inFlight
Max:        12     ← highest workers ever from do: block (pre-allocated)
Autoscaler: ✓ enabled, override active

This replaces the misleading "N pending workers" display that showed all pre-allocated goroutines as pending regardless of autoscale state.

pkg/autoscaler/autoscaler.go

Operatorbox autoscaler — evaluates conditions on a ticker and applies or restores worker/queue/resync overrides.

One Autoscaler is created per operatorbox that declares autoscale: in its operatorBox: block. It runs a single goroutine for the lifetime of the operatorbox and stops cleanly when the context is cancelled.

The autoscaler has no persistent state. A restart always begins from the declared CRD baseline. Overrides applied before a restart are lost — the next evaluation tick will re-apply them if conditions are still met.