kordinator

package
v0.7.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 4, 2026 License: Apache-2.0 Imports: 34 Imported by: 0

README

pkg/kordinator

kordinator is the orchestration heart of every Orkestra operator. It decides when each CRD's workers start, in what order, under what conditions, and how they recover when the cluster changes while the operator is running.

Everything upstream of kordinator — informers, queues, the provider registry, the reconciler factory — produces inputs. Kordinator is where those inputs become running workers.

What kordinator does

  • Topological startup — reads the dependency graph from the Katalog and starts CRD workers in the correct order, without blocking on conditions that may take an arbitrary amount of time
  • Self-healing — monitors the cluster continuously and activates CRDs that appear after startup, deactivates and re-activates CRDs that are deleted and recreated at runtime
  • Health tracking — maintains per-CRD health state with atomic counters (total reconciles, failures, consecutive failures, worker utilization) and a separate aggregate operator health signal
  • Dependency gating — enforces condition: started and condition: healthy dependency requirements before activating a dependent CRD, without ever blocking the startup sequence
  • Autoscaler wiring — when autoscale: is declared on an operatorBox:, startCRDWorkers starts only baseline.workers goroutines, injects a spawnWorker callback so ResizeWorkers can add goroutines on scale-up, registers the CRD's AutoMetrics with GlobalCrossMetricsRegistry, and launches the autoscaler and resync goroutines in dedicated goroutines tied to the CRD's context
  • Rollback wiring — injects onTrigger / onClear callbacks into the reconciler so that rollback events update CRDHealth counters and are reflected in the /katalog/{crd} response and the Control Center
  • Runtime introspection — serves the /katalog, /katalog/{crd}, and /katalog/{crd}/health endpoints that power the Control Center

Where kordinator fits

Kordinator is the last component started in konstructRuntime. The startup sequence is:

loadProviders        — provider registry built
resourceKatalog      — CRD entries, informers, reconciler factories registered
informer.Factory     — informers created and started, caches warm
DependencyKordinator — workers start in dependency order  ← here

From this point on, every reconcile event flows through:

informer event
  └── queue.Add(key)             depth-limit enforced atomically
        └── runWorkerForGVK
              └── processItemForGVK
                    └── rec.Reconcile(ctx, req)
                          └── workerSem.Acquire()     autoscale concurrency gate
                                reconcileCore()
                              workerSem.Release()

For CRDs with autoscale: declared, three additional goroutines run alongside the worker pool for the lifetime of the CRD's context:

autoscaler goroutine   — evaluates conditions, calls ResizeWorkers / SetQueueDepthLimit / SetResyncInterval
resync goroutine       — re-enqueues all cached objects at do.resync when override is active

Developer documentation

Complete documentation is in docs/.

I want to understand… Go to
The CRD registry and what each entry holds 01 — ResourceKatalog
How health state is tracked per CRD 02 — CRDHealth
How workers start in dependency order 03 — Startup and dependency channels
How the operator recovers from missing or deleted CRDs 04 — Self-healing
How the worker loop runs and how shutdown drains cleanly 05 — Workers and drain
The runtime introspection HTTP handlers 06 — HTTP handlers

Key types at a glance

Type File Role
DependencyKordinator dependency_kordinator.go Startup, dependency gating, shutdown
Kontroller kontroller.go Base worker manager embedded by DependencyKordinator
ResourceKatalog kordinator_registry.go Per-GVK registry: informer, reconciler factory, CRD config
CRDHealth crd_health.go Per-CRD health counters, worker states, dependency status
OrkestraHealth crd_worker_health.go Aggregate operator health (ready / degraded)

Documentation

Overview

pkg/kordinator/cr_children.go

Child resource fetching and readiness for the CR detail endpoint.

pkg/kordinator/crd_health.go

pkg/kordinator/crd_health_handlers.go

pkg/kordinator/crd_rbac_health.go

pkg/kordinator/dependency_kordinator.go

╔═══════════════════════════════════════════════════════════════════════════════╗ ║ CRD Lifecycle Management Flow ║ ╚═══════════════════════════════════════════════════════════════════════════════╝

The retryMissingCRDs goroutine runs forever, handling both activation and deactivation of CRDs. This is the self‑healing core of Orkestra.

───────────────────────────────────────────────────────────────────────────────── SCENARIO 1: CRD MISSING AT STARTUP ─────────────────────────────────────────────────────────────────────────────────

Startup:

  • CRD A is missing from the cluster
  • Kordinate loop processes A → startedCh["A"] remains open
  • Kordinate continues (does NOT block) because missing CRDs are skipped
  • Retry loop starts in background

Retry loop (every postStartRetryInterval):

  • Phase 1: checks missing map
  • finds A is missing
  • calls utils.WaitForCRD() → false
  • remains in missing map
  • Phase 2: checks running CRDs (none)
  • Phase 3: checks deferred (not‑started) CRDs (A not ready due to missing)
  • Phase 4: allReady? false

Later:

  • User applies CRD A to cluster
  • Retry loop runs again
  • Phase 1: utils.WaitForCRD() → true
  • activateCRD(A) is called:
  • starts informer (entry.Informer.Run)
  • starts workers
  • closes startedCh["A"] ← UNBLOCKS dependents in Kordinate
  • Phase 3: allCRDsPresent() may still be false if other CRDs missing

Result: CRD A becomes operational without restarting Orkestra.

───────────────────────────────────────────────────────────────────────────────── SCENARIO 2: CRD DELETED AFTER STARTUP ─────────────────────────────────────────────────────────────────────────────────

Running state:

  • CRD A is active (workers running, informer watching)
  • Retry loop runs periodically

User deletes CRD A:

  • Informer reflector starts logging "failed to list... the server could not find..."
  • Retry loop runs:
  • Phase 1: missing map (A not there)
  • Phase 2: checks running CRDs
  • calls crdExists(A) → false
  • deactivateCRD(A) is called:
  • stopCRDWorkers(A) → stops workers, drains queue
  • removes from started map
  • marks as missing in informerFactory
  • health.SetStarted(false)
  • DOES NOT close startedCh[A]
  • Phase 4: allReady becomes false

Result: CRD A becomes degraded, workers stop, but dependents continue (degraded). No more reflector errors.

───────────────────────────────────────────────────────────────────────────────── SCENARIO 3: CRD REAPPEARS AFTER BEING DELETED ─────────────────────────────────────────────────────────────────────────────────

After deactivation:

  • CRD A is in missing map

  • Retry loop runs:

  • Phase 1: missing map contains A

  • utils.WaitForCRD() → true

  • activateCRD(A) is called (same as scenario 1)

  • Workers restart, informer starts

  • startedCh[A] is closed again (it was never closed during deactivation, but we create a new channel? No — startedCh persists. Actually we don't close it during deactivation, so it remains open. We need to be careful: during activation, we should close it regardless of whether it was closed before.)

    In activateCRD, we already handle this with select/default: select { case <-ch: // already closed default: close(ch) }

Result: CRD A becomes operational again without restarting Orkestra.

───────────────────────────────────────────────────────────────────────────────── SCENARIO 4: DEPENDENCY CHAIN WITH DELAYED ACTIVATION (FIXED BEHAVIOR) ─────────────────────────────────────────────────────────────────────────────────

StartupOrder: [A, B, C] where B depends on A:started, C depends on B:healthy

Initial state:

  • A: present
  • B: present
  • C: present

Kordinate loop:

  • A → starts, closes startedCh["A"]
  • B → dependenciesReady? true (A started) → starts, closes startedCh["B"]
  • C → dependenciesReady? false (B not yet healthy) → SKIPS (does NOT block)
  • Main loop finishes.

Retry loop (periodic):

  • Phase 3: checks not‑started CRDs
  • C: dependenciesReady? false (B still not healthy) → skip
  • Later, B becomes healthy → health checker closes healthyCh["B"]
  • Next retry tick:
  • Phase 3: C dependenciesReady? true → activateCRD(C)
  • Workers start, C becomes operational

Result: A and B start immediately; C starts only when B becomes healthy, without blocking the main goroutine or starving other CRDs.

───────────────────────────────────────────────────────────────────────────────── SCENARIO 5: DEPENDENCY CHAIN WITH DELETION IN THE MIDDLE ─────────────────────────────────────────────────────────────────────────────────

Running state:

  • A, B, C all active and healthy
  • D depends on C:started

User deletes CRD C:

  • Retry loop detects C missing
  • deactivateCRD(C):
  • stops workers
  • marks missing
  • DOES NOT close startedCh[C] (it's already closed from startup)

Result:

  • C is degraded, workers stopped
  • D continues running (degraded) because its dependency C is not ready
  • Health endpoints show C as degraded, D as degraded

Later, C is recreated:

  • activateCRD(C):
  • starts workers
  • attempts to close startedCh[C] (already closed, safe)
  • D's health becomes healthy again automatically

───────────────────────────────────────────────────────────────────────────────── KEY DESIGN DECISIONS ─────────────────────────────────────────────────────────────────────────────────

  1. startedCh is NEVER closed during deactivation. Reason: Dependents should continue running (degraded) rather than block. If we closed the channel, dependents would think the dependency is ready when it's actually missing.
  1. retry loop runs FOREVER, not just at startup. Reason: CRDs can be deleted at any time. We need continuous monitoring.
  1. activateCRD closes startedCh safely using select/default. Reason: startedCh may already be closed from initial startup or previous activation.
  1. deactivateCRD does NOT remove from startedCh map. Reason: The channel is still needed for future activations. The channel is never closed, so it remains in the map.
  1. health.SetStarted(false) on deactivation. Reason: Allows health endpoint to show the CRD as not started.
  1. Main Kordinate loop NEVER BLOCKS on dependency conditions. Reason: Dependencies with "healthy" requirement may take arbitrary time. Blocking would starve other CRDs that are ready to start. Instead, we skip CRDs whose dependencies aren't ready and rely on the retry loop to activate them later when conditions are satisfied.

╚═══════════════════════════════════════════════════════════════════════════════╝

Package kordinator is the orchestration heart of every Orkestra operator.

It sits between the informer factory and the reconcilers: it decides when each CRD's workers start, in what order, under what conditions, and how they recover when the cluster changes beneath them.

Architecture

The package is built in two layers.

The base layer is Kontroller — the worker manager. It holds the reconciler instances, the per-CRD context cancel functions, the WaitGroups, and the queue references. It knows how to start and stop workers for a GVK; it does not know anything about dependencies.

The coordination layer is DependencyKordinator, which embeds Kontroller and adds the dependency graph, the channel-based readiness signals, the self-healing retry loop, and the runtime health checker. It is the only component that calls Kordinate() — the method that blocks for the operator's lifetime and owns the startup → run → shutdown sequence.

ResourceKatalog

ResourceKatalog is the in-memory registry written once during konstructRuntime and read many times thereafter. Every GVK maps to a RegistryEntry that holds the informer, the reconciler factory closure, and the CRD configuration. Workers are created by calling entry.ReconcilerFactory() — the closure captures the provider registry, kube client, and provider stats so they never pass through the kordinator's own API.

CRD lifecycle

The DependencyKordinator drives every CRD through the following states: 

pending  → informer created, workers not yet started
started  → workers running, no successful reconcile yet
healthy  → at least one successful reconcile completed
degraded → consecutive failures exceeded threshold, or CRD missing

The transitions are recorded in CRDHealth using atomic operations — no locks in the hot path. State changes from workers (MarkWorkerProcessing, MarkWorkerIdle, RecordSuccess, RecordFailure) and from the health checker (SetDegraded, SetStarted) are all safe for concurrent callers.

Dependency channels

CRDs may declare dependencies with two possible conditions: 

dependsOn:
  other-crd:
    condition: started   # default — workers are running
  another-crd:
    condition: healthy   # first successful reconcile has completed

Each CRD gets two channels at startup: 

startedCh[gvk]  — closed when startCRDWorkers returns
healthyCh[gvk]  — closed by dependencyHealthChecker on first healthy state

dependenciesReady() checks these channels non-blocking using select/default. It returns false immediately if any channel is still open — it never blocks.

This is intentional. A dependency with condition: healthy may take minutes to satisfy. Blocking the startup loop on that condition would starve every CRD that appears later in the topological order, regardless of whether those CRDs depend on the slow one. The fix: skip and defer.

Non-blocking startup and deferred activation

Kordinate() iterates the topological order exactly once. For each CRD:

  • if dependenciesReady() returns false → skip, the retry loop handles it
  • if the CRD is missing from the cluster → skip, the retry loop handles it
  • otherwise → startCRDWorkers, close startedCh[gvk]

The background retryMissingCRDs goroutine runs for the operator's lifetime. On each tick it executes four phases in order: 

Phase 1 — Detect runtime disappearances: polls every registered GVK.
           When a CRD vanishes, deactivateCRD drains the workers,
           marks the CRD degraded, and propagates degradation to any
           dependent that requires condition: healthy.

Phase 2 — Re-activate missing CRDs: for every GVK in the missing map,
           checks whether the CRD has appeared in the cluster. If yes,
           activateCRD restarts the informer, launches workers, and
           closes startedCh[gvk] to unblock waiting dependents.

Phase 3 — Deferred activation: iterates the topological order looking
           for CRDs that were skipped at startup. When dependenciesReady()
           finally returns true, activateCRD is called and the CRD joins
           the running set. This is what handles condition: healthy.

Phase 4 — Aggregate health: if all CRDs are started and none are missing,
           sets katalog health to ready.

Self-healing invariants

startedCh is never closed during deactivation. This is load-bearing: if it were closed, dependents would believe the deactivated CRD is still ready and start processing against a resource that no longer exists. Keeping it open means dependents stay in their current state (degraded) and resume normally when activateCRD closes the channel again on re-activation.

activateCRD closes startedCh with a select/default guard: 

select {
case <-ch: // already closed from a previous activation — no-op
default:   close(ch)
}

Without this guard, re-activating a CRD that was previously started would panic with "close of closed channel".

Runtime health checker

dependencyHealthChecker runs on a ticker independent of the retry loop. It evaluates the health of every active CRD's dependencies and updates the DependencyStatus map inside CRDHealth — this feeds the Control Center and the /katalog/{crd} endpoint. It also closes healthyCh[gvk] exactly once when a CRD transitions to healthy for the first time, which unblocks any deferred CRDs waiting for condition: healthy.

Worker loop

runWorkerForGVK (in worker.go) is the inner loop for every worker goroutine: 

for {
    select {
    case <-ctx.Done(): return
    default:
        item, shutdown := wq.Queue.Get()
        if shutdown { return }
        health.MarkWorkerProcessing(workerID)
        func() {
            defer wq.Queue.Done(item)
            k.processItemForGVK(ctx, gvk, item)
        }()
        health.MarkWorkerIdle(workerID)
    }
}

wq.Queue.Done(item) is always called — deferred inside the closure — whether the reconcile succeeds or fails. Re-queuing with rate-limit backoff is handled inside processItemForGVK on error.

stopCRDWorkers cancels the CRD context and then calls wq.Queue.ShutDown() before waiting on the WaitGroup. The shutdown call is required: a worker blocking on wq.Queue.Get() waiting for its next item will never observe the context cancellation until it is unblocked by the queue shutdown.

HTTP surface

All three runtime introspection handlers are in crd_health_handers.go. 

BuildCRDHealthHandler  — /katalog/{crd}/health
BuildCRDInfoHandler    — /katalog/{crd}
BuildKatalogHandler    — /katalog

BuildCRDInfoHandler assembles a full CRD detail response on every request from live atomic reads — no caching. When the CRD declares provider blocks, it merges static ProviderBlocks metadata (declared kinds) with runtime ProviderStats (total calls, errors, error rate) into a providers array.

Resource count is len(inf.GetStore().List()) — the informer's local cache, updated in real time by the informer watch loop without any API calls.

pkg/kordinator/gateway_handlers.go

HTTP handlers served by the gateway process at /katalog and /katalog/{crd}.

The gateway /katalog response carries the same per-CRD shape as the runtime /katalog response but only populates the fields the gateway owns: admission, conversion, deletion-protection, and namespace-protection stats. Reconciler health fields (workers, queue depth, error rate, etc.) are omitted.

Control center merge strategy ────────────────────────────── The runtime /katalog response includes a "gatewayEndpoint" field. The control center reads that URL, fetches the gateway /katalog, then merges per-CRD by GVR string ("group/version/resource") — the canonical key used by both processes. Neither process pushes to the other; each is independently queryable.

pkg/config/pkg/kordinator/registry.go

pkg/kordinator/registry_cross.go

GetInformerByName — satisfies reconciler.KatalogRegistry.

Allows GenericReconciler to look up a sibling CRD's SharedIndexInformer for cross-CRD observation without importing pkg/kordinator directly (which would create an import cycle).

The lookup is by CRD name (the map key in spec.crds — lowercase). The GVK string stored in the registry is used internally; callers only need to know the short name.

Index

Constants

View Source 
const (
	// Dependency checks
	DefaultDependencyTimeout  = 60 * time.Second
	DefaultDependencyRetries  = 3
	DefaultDependencyInterval = 10 * time.Second

	PostStartBackoff              = 5 * time.Second
	PostStartBackoffMax           = 5 * time.Minute
	DependencyHealthCheckInterval = 10 * time.Second

	// Healthcheck
	RuntimeHealthCheckInterval = 5 * time.Second
)
View Source 
const (
	WorkerStateIdle       = "idle"
	WorkerStateProcessing = "processing"
	WorkerStateStopped    = "stopped"
)

Worker state constants

Variables

This section is empty.

Functions

func BuildCRDEnrichedHandler 

func BuildCRDEnrichedHandler(kat *katalog.Katalog, crdName string) http.HandlerFunc

───────────────────────────────────────────────────────────────────────────── CRD Enriched Handler Returns a single CRD definition with all enriched/default values applied. This endpoint powers the "View Enriched" button in the CRD detail view, allowing users to see exactly what Orkestra uses at runtime after applying default workers, default resync, enriched API types, and resolved dependencies.

The enriched view shows:

  • Fully resolved API types (group, version, plural enriched from discovery API)
  • Default workers applied when not specified (from DEFAULT_WORKERS env var)
  • Default resync applied when not specified (from DEFAULT_RESYNC env var)
  • Default queue depth applied when not specified (from QUEUE_DEPTH env var)
  • All validation and mutation rules merged from sources
  • Complete operatorBox: configuration after inheritance

Endpoint: /katalog/{crd}/enriched

Example response (user wrote only `kind: Service`): 

{
  "name": "service-watcher",
  "apiTypes": {
    "group": "",
    "version": "v1",
    "kind": "Service",
    "plural": "services"
  },
  "workers": 3,
  "resync": "30s",
  "maxDepth": 2000,
  "dependsOn": {
    "pod-manager": {
      "condition": "started"
    }
  }
}

This shows what Orkestra actually uses at runtime versus what the user wrote. ─────────────────────────────────────────────────────────────────────────────

func BuildCRDHealthHandler 

func BuildCRDHealthHandler(
	crd orktypes.CRDEntry,
	kfg *konfig.Konfig,
	inf cache.SharedIndexInformer,
	h *CRDHealth,
	o *OrkestraHealth,
) http.HandlerFunc

───────────────────────────────────────────────────────────────────────────── CRD Health Handler Returns the live health status of a single CRD reconciler. This endpoint is used by:

  • /katalog/<crd>/health
  • dashboards
  • readiness/liveness checks
  • operator self‑diagnostics

The handler exposes:

  • health state (healthy/degraded)
  • startup state
  • uptime
  • reconcile counters
  • last error
  • last reconcile timestamp

─────────────────────────────────────────────────────────────────────────────

func BuildCRDInfoHandler 

func BuildCRDInfoHandler(
	crd orktypes.CRDEntry,
	kfg *konfig.Konfig,
	inf cache.SharedIndexInformer,
	h *CRDHealth,
	o *OrkestraHealth,
	convStats *health.ConversionStats,
	admStats *health.AdmissionStats,
	protStats *health.DeletionProtectionStats,
	webhookControllerStats *health.WebhookStats,
	provStats *health.ProviderStats,
	nsStats *health.NamespaceProtectionStats,
	isDeletionProtected bool,
	isNamespaceProtected bool,
	isConversionEnabled bool,
	isAdmissionEnabled bool,
) http.HandlerFunc

───────────────────────────────────────────────────────────────────────────── CRD Info Handler Returns static + dynamic metadata about a CRD:

  • GVK/GVR
  • mode (dynamic/typed)
  • workers, resync, queue depth (with source: default or configured)
  • reconciler configuration (hooks, finalizers, constructor)
  • resource count from informer
  • health summary

This endpoint powers:

  • /katalog/<crd>
  • dashboards
  • operator introspection

─────────────────────────────────────────────────────────────────────────────

func BuildCRDRawHandler 

func BuildCRDRawHandler(m *merger.Merger, crdName string) http.HandlerFunc

───────────────────────────────────────────────────────────────────────────── CRD Raw Handler Returns a single CRD definition from the merged Katalog as JSON. This endpoint powers the "View Source" button in the CRD detail view, allowing users to see exactly how the CRD was defined in the Katalog.

Endpoint: /katalog/{crd}/raw

Example response: 

{
  "name": "website",
  "apiTypes": {
    "group": "demo.orkestra.io",
    "version": "v1alpha1",
    "kind": "Website",
    "plural": "websites"
  },
  "operatorBox:": {
    "onCreate": {
      "deployments": [
        {
          "image": "{{ .spec.image }}"
        }
      ]
    }
  }
}

─────────────────────────────────────────────────────────────────────────────

func BuildCRDetailAndEventsHandler 

func BuildCRDetailAndEventsHandler(
	crd orktypes.CRDEntry,
	inf cache.SharedIndexInformer,
	kube *kubeclient.Kubeclient,
	rc orktypes.OperatorBoxConfig,
) http.HandlerFunc

BuildCRDetailAndEventsHandler handles all sub-paths under /katalog/{crd}/cr/ Dispatches to detail or events based on whether the path ends with /events. Register at "/katalog/{crd}/cr/" (with trailing slash).

func BuildCRDetailHandler 

func BuildCRDetailHandler(
	crd orktypes.CRDEntry,
	inf cache.SharedIndexInformer,
	kube *kubeclient.Kubeclient,
	rc orktypes.OperatorBoxConfig,
) http.HandlerFunc

───────────────────────────────────────────────────────────────────────────── CR Detail Handler GET /katalog/{crd}/cr/{name} (cluster-scoped) GET /katalog/{crd}/cr/{namespace}/{name} (namespaced)

Returns the full CR status and its child resources. CR is read from the informer cache — no API server call. Children are read from the API server on demand.

rcMap maps GVK → OperatorBoxConfig so we know which resource types to look for as children (from onCreate/onReconcile templates). ─────────────────────────────────────────────────────────────────────────────

func BuildCREventsHandler 

func BuildCREventsHandler(
	crd orktypes.CRDEntry,
	kube *kubeclient.Kubeclient,
) http.HandlerFunc

───────────────────────────────────────────────────────────────────────────── CR Events Handler GET /katalog/{crd}/cr/{name}/events (cluster-scoped) GET /katalog/{crd}/cr/{namespace}/{name}/events (namespaced)

Returns recent Kubernetes events involving this CR. Events are listed from the API server using field selectors on involvedObject.name and involvedObject.namespace.

Includes events emitted by Orkestra (reason: Reconciled, ReconcileError, ValidationWarning, Deleting) and by Kubernetes itself.

Events are sorted newest-first and capped at 100. ─────────────────────────────────────────────────────────────────────────────

func BuildCRListHandler 

func BuildCRListHandler(
	crd orktypes.CRDEntry,
	inf cache.SharedIndexInformer,
) http.HandlerFunc

───────────────────────────────────────────────────────────────────────────── CR List Handler GET /katalog/{crd}/cr

Lists all CR instances from the informer cache. No API server calls — purely in-memory. Sort order: namespace/name ascending for deterministic output. ─────────────────────────────────────────────────────────────────────────────

func BuildEnrichedKatalogHandler 

func BuildEnrichedKatalogHandler(kat *katalog.Katalog) http.HandlerFunc

───────────────────────────────────────────────────────────────────────────── Enriched Katalog Handler Returns the complete enriched Katalog as JSON, including metadata and all CRDs with all default values and enrichments applied. This endpoint shows what Orkestra actually uses at runtime after processing the user's Katalog.

Endpoint: /katalog/enriched

The response contains the same structure as /katalog/raw, but with:

  • All defaults applied (workers, resync, queue depth)
  • All API types enriched (group, version, plural from discovery API)
  • All validation and mutation rules merged
  • Complete operatorBox: configuration after inheritance
  • Resolved dependencies with conditions

This endpoint is useful for:

  • Understanding what Orkestra actually does with your Katalog
  • Debugging why certain values are being used at runtime
  • Seeing the full effect of inheritance and defaults
  • Comparing user intent with runtime reality

Example response (user wrote minimal Katalog with just `kind: Pod`): 

{
  "apiVersion": "orkestra.orkspace.io/v1",
  "kind": "Katalog",
  "metadata": {
    "name": "platform-katalog",
    "description": "Production operator suite"
  },
  "spec": {
    "crds": [
      {
        "name": "pod-manager",
        "apiTypes": {
          "group": "",
          "version": "v1",
          "kind": "Pod",
          "plural": "pods"
        },
        "workers": 3,
        "resync": "30s",
        "maxDepth": 2000,
        "failureThreshold": 10,
        "namespaced": true,
        "operatorBox:": {
          "default": true
        }
      }
    ]
  }
}

─────────────────────────────────────────────────────────────────────────────

func BuildGatewayCRDHandler added in v0.4.9 

func BuildGatewayCRDHandler(name, gvk, gvrStr, gvrKey string, ws GatewayStatsProvider, kat *katalog.Katalog) http.HandlerFunc

BuildGatewayCRDHandler returns the handler for GET /katalog/{crd} on the gateway. name, gvk, gvrStr, and gvrKey are derived from the CRDEntry at registration time.

func BuildGatewayKatalogHandler added in v0.4.9 

func BuildGatewayKatalogHandler(kat *katalog.Katalog, ws GatewayStatsProvider) http.HandlerFunc

BuildGatewayKatalogHandler returns the handler for GET /katalog on the gateway. It iterates the enabled CRDs from the Katalog, looks up per-CRD stats from ws, and returns a GatewayKatalogResponse.

func BuildKatalogHandler 

func BuildKatalogHandler(
	kat *katalog.Katalog,
	kfg *konfig.Konfig,
	reg *ResourceKatalog,
	healthMap map[string]*CRDHealth,
	o *OrkestraHealth,
) http.HandlerFunc

───────────────────────────────────────────────────────────────────────────── Katalog Handler Returns a full list of CRDs in the running operator, including:

  • metadata (GVK, mode, namespace, dependencies)
  • resolved operational values (workers, resync, queue depth)
  • reconciler configuration summary
  • health summary (uptime, error rate, startedAt)
  • per‑CRD endpoints (/health, /info)

This is the top‑level endpoint for dashboards and operator UIs. ─────────────────────────────────────────────────────────────────────────────

func BuildNotifyHandler added in v0.4.9 

func BuildNotifyHandler(kat *katalog.Katalog) http.HandlerFunc

BuildNotifyHandler returns an http.HandlerFunc that receives a notification Event from the runtime and dispatches it to the configured team channels.

The gateway is the single point for SMTP/Slack dispatch — the runtime only POSTs pre-built events after its own throttle check.

func BuildRawKatalogHandler 

func BuildRawKatalogHandler(m *merger.Merger) http.HandlerFunc

───────────────────────────────────────────────────────────────────────────── Raw Katalog Handler Returns the complete merged Katalog as JSON, including metadata and all CRDs. This endpoint is used by the Control Center to display the source Katalog that defines the current operator.

Endpoint: /katalog/raw

The response contains:

  • apiVersion: The Orkestra API version
  • kind: Always "Katalog" at runtime (merged from sources)
  • metadata: Katalog metadata (name, description, version, author, license)
  • spec.crds: All merged CRD definitions from all sources

Example response: 

{
  "apiVersion": "orkestra.orkspace.io/v1",
  "kind": "Katalog",
  "metadata": {
    "name": "platform-katalog",
    "description": "Production operator suite",
    "version": "1.0.0",
    "author": "Platform Team"
  },
  "spec": {
    "crds": [
      {
        "name": "website",
        "apiTypes": { ... },
        "operatorBox:": { ... }
      }
    ]
  }
}

─────────────────────────────────────────────────────────────────────────────

func InvalidateChildrenCache added in v0.4.7 

func InvalidateChildrenCache(namespace, name string)

InvalidateChildrenCache removes the cached children for a CR. Called by the reconciler after a successful reconcile so the next endpoint request reflects the latest state immediately.

func RegisterCRRoutes 

func RegisterCRRoutes(
	mux *http.ServeMux,
	crd orktypes.CRDEntry,
	inf cache.SharedIndexInformer,
	kube *kubeclient.Kubeclient,
	rc orktypes.OperatorBoxConfig,
)

RegisterCRRoutes adds the CR list, detail, and events handlers to mux. Call this alongside BuildCRDHealthHandler and BuildCRDInfoHandler for each CRD.

Types

type AdmissionStatsResponse 

type AdmissionStatsResponse struct {
	WebhooksEnabled   bool    `json:"webhooksEnabled"`
	ValidationTotal   int64   `json:"validationTotal"`
	ValidationAllowed int64   `json:"validationAllowed"`
	ValidationDenied  int64   `json:"validationDenied"`
	ValidationWarned  int64   `json:"validationWarned"`
	ValAvgLatencyMs   float64 `json:"valAvgLatencyMs"`
	ValP95LatencyMs   float64 `json:"valP95LatencyMs"`
	ValMaxLatencyMs   float64 `json:"valMaxLatencyMs"`
	MutationTotal     int64   `json:"mutationTotal"`
	MutationApplied   int64   `json:"mutationApplied"`
	MutationSkipped   int64   `json:"mutationSkipped"`
	MutAvgLatencyMs   float64 `json:"mutAvgLatencyMs"`
	MutP95LatencyMs   float64 `json:"mutP95LatencyMs"`
	MutMaxLatencyMs   float64 `json:"mutMaxLatencyMs"`
}

type CRDHealth 

type CRDHealth struct {
	// contains filtered or unexported fields
}

CRDHealth tracks the runtime health of a single CRD's reconciler. It is fully concurrency‑safe and designed to be updated from multiple goroutines.

Fields tracked:

  • started: whether the reconciler has begun processing events

  • healthy: whether the reconciler is currently considered healthy

  • totalReconciles: total number of reconcile attempts (success + failure)

  • failedReconciles: number of failed reconciles

  • consecutiveFails: number of failures in a row (used for degradation)

  • lastError: last error message (string)

  • lastReconcile: timestamp of last reconcile attempt

  • startTime: timestamp when the reconciler first started

  • The activeWarnings: tracks current warn-mode violations per CR. It answers the question: "which CRs are currently violating advisory rules?"

    Map key: "namespace/name" — unique per CR Map value: slice of ActiveWarning, one per violated warn rule

This struct powers:

  • /katalog/<crd> endpoint
  • /katalog/<crd>/health endpoint
  • dashboard status
  • operator self‑diagnostics

func NewCRDHealth 

func NewCRDHealth(name string) *CRDHealth

NewCRDHealth initializes a CRDHealth tracker for a given CRD name. The reconciler starts in an "unhealthy" state until the first successful reconcile.

func (*CRDHealth) CRDExists 

func (h *CRDHealth) CRDExists() bool

CRDExists returns whether the CRD exists in the cluster.

func (*CRDHealth) ConsecutiveFails 

func (h *CRDHealth) ConsecutiveFails() int64

ConsecutiveFails returns the number of consecutive failed reconciles.

func (*CRDHealth) ErrorRate 

func (h *CRDHealth) ErrorRate() float64

ErrorRate returns the ratio of failed reconciles to total reconciles. If no reconciles have occurred, the error rate is 0.

func (*CRDHealth) ErrorRatePercent added in v0.1.9 

func (h *CRDHealth) ErrorRatePercent() float64

ErrorRatePercent returns the error rate as a percentage.

func (*CRDHealth) FailedReconciles 

func (h *CRDHealth) FailedReconciles() int64

FailedReconciles returns the number of failed reconcile attempts.

func (*CRDHealth) GetActiveWorkers 

func (h *CRDHealth) GetActiveWorkers() int32

GetActiveWorkers returns total workers (all are "active" if running)

func (*CRDHealth) GetAutoMetrics added in v0.1.9 

func (h *CRDHealth) GetAutoMetrics() map[string]interface{}

GetAutoMetrics returns the live AutoMetrics map, or nil when not available. Included in the /katalog/{crd} response as "metrics" so cross-binary autoscale conditions can observe this CRD's metrics via HTTP fallback.

func (*CRDHealth) GetDependencyStatuses 

func (h *CRDHealth) GetDependencyStatuses() map[string]DependencyStatus

GetDependencyStatuses returns a copy of all dependency statuses

func (*CRDHealth) GetIdleWorkers 

func (h *CRDHealth) GetIdleWorkers() int32

GetIdleWorkers returns number of workers waiting for work

func (*CRDHealth) GetProcessingWorkers 

func (h *CRDHealth) GetProcessingWorkers() int32

GetProcessingWorkers returns number of workers currently reconciling

func (*CRDHealth) GetTotalWorkers 

func (h *CRDHealth) GetTotalWorkers() int32

GetTotalWorkers returns the expected total workers

func (*CRDHealth) GetWorkerInfo added in v0.1.9 

func (h *CRDHealth) GetWorkerInfo() *ork_autoscaler.WorkerInfo

GetWorkerInfo returns the live WorkerInfo, or nil when not available.

func (*CRDHealth) GetWorkerStates 

func (h *CRDHealth) GetWorkerStates() map[string]string

GetWorkerStates returns a map of worker states for debugging

func (*CRDHealth) HasUnhealthyDependencies 

func (h *CRDHealth) HasUnhealthyDependencies() bool

HasUnhealthyDependencies returns true if any dependency is not satisfied

func (*CRDHealth) HealthAsMap added in v0.4.4 

func (h *CRDHealth) HealthAsMap() map[string]interface{}

HealthAsMap returns a snapshot of this CRD's health as a plain map, suitable for injection into the template resolver under the "health" key.

Available in conditions status.fields templates as: 

	{{ .health.healthy }}            — bool: reconciler is healthy
 {{ .health.state }}              — string: "healthy" / "degraded" / "pending" / "not started"
 {{ .health.status }}             — int: HTTP status code representing health state
	{{ .health.started }}            — bool: reconciler has started
	{{ .health.pending }}            — bool: reconciler is pending
	{{ .health.degraded }}           — bool: reconciler is degraded
	{{ .health.totalReconciles }}    — int64: total reconcile attempts
	{{ .health.failedReconciles }}   — int64: total failed reconciles
	{{ .health.consecutiveFails }}   — int64: current consecutive failure streak
	{{ .health.errorRatePercent }}   — float64: error rate as percentage
	{{ .health.lastReconcile }}      — string: RFC3339 timestamp of last reconcile
	{{ .health.uptime }}             — string: how long the reconciler has been running
	{{ .health.lastError }}          — string: most recent error message (empty if none)
	{{ .health.rollbackActive }}     — bool: rollback is currently blocking reconcile
	{{ .health.rollbackTotal }}      — int64: total rollback triggers since startup
	{{ .health.hasUnhealthyDeps }}   — bool: any dependency is unsatisfied

func (*CRDHealth) IsHealthy 

func (h *CRDHealth) IsHealthy() bool

IsHealthy reports whether the reconciler is currently considered healthy. Health is degraded after N consecutive failures.

func (*CRDHealth) IsMissing 

func (h *CRDHealth) IsMissing() bool

func (*CRDHealth) LastCRDCheck 

func (h *CRDHealth) LastCRDCheck() time.Time

LastCRDCheck returns when the CRD existence was last verified.

func (*CRDHealth) LastError 

func (h *CRDHealth) LastError() string

LastError returns the last recorded error message. If no error has occurred, it returns an empty string.

func (*CRDHealth) LastReconcile 

func (h *CRDHealth) LastReconcile() string

LastReconcile returns a human‑readable timestamp of the last reconcile. If the reconciler has started but not yet reconciled, it returns a placeholder.

func (*CRDHealth) MarkHealthySignaled 

func (h *CRDHealth) MarkHealthySignaled()

func (*CRDHealth) MarkWorkerIdle 

func (h *CRDHealth) MarkWorkerIdle(workerID string)

MarkWorkerIdle is called when a worker finishes processing

func (*CRDHealth) MarkWorkerProcessing 

func (h *CRDHealth) MarkWorkerProcessing(workerID string)

MarkWorkerProcessing is called when a worker starts processing an item

func (*CRDHealth) Name 

func (h *CRDHealth) Name() string

Name returns the CRD name associated with this health tracker.

func (*CRDHealth) Pending 

func (h *CRDHealth) Pending() bool

Pending reports the reconciler has started but yet not yet reconciled.

func (*CRDHealth) QueueDepth 

func (h *CRDHealth) QueueDepth(gvk string) int

QueueDepth returns the queue for this CRD

func (*CRDHealth) RecordFailure 

func (h *CRDHealth) RecordFailure(err error, degradeThreshold int)

RecordFailure marks a failed reconcile event. It increments failure counters, stores the error, and may degrade health if the number of consecutive failures exceeds the configured threshold.

func (*CRDHealth) RecordRollbackCleared added in v0.1.9 

func (h *CRDHealth) RecordRollbackCleared()

RecordRollbackCleared marks rollback inactive. Called by the reconciler when it clears the rollback annotation.

func (*CRDHealth) RecordRollbackTriggered added in v0.1.9 

func (h *CRDHealth) RecordRollbackTriggered()

RecordRollbackTriggered increments the rollback counter and marks rollback active. Called by the reconciler when it triggers a rollback.

func (*CRDHealth) RecordStartupFailure 

func (h *CRDHealth) RecordStartupFailure(err error, degradeThreshold int)

RecordStartupFailure is used when the reconciler fails before it has fully started. It does not affect total reconcile counts, only consecutive failure tracking.

func (*CRDHealth) RecordSuccess 

func (h *CRDHealth) RecordSuccess()

RecordSuccess marks a successful reconcile event. It resets the consecutive failure counter and updates timestamps.

func (*CRDHealth) ResetWorkerCounts 

func (h *CRDHealth) ResetWorkerCounts()

ResetWorkerCounts resets all worker counters (used during deactivation)

func (*CRDHealth) RollbackStats added in v0.1.9 

func (h *CRDHealth) RollbackStats() RollbackStats

RollbackStats returns a snapshot of rollback activity for this CRD.

func (*CRDHealth) SetAutoMetricsFn added in v0.1.9 

func (h *CRDHealth) SetAutoMetricsFn(fn func() map[string]interface{})

SetAutoMetricsFn stores the function that returns live AutoMetrics as a map. Called by startCRDWorkers when autoscale: is declared.

func (*CRDHealth) SetCRDExists 

func (h *CRDHealth) SetCRDExists(exists bool)

SetCRDExists records that the CRD exists in the cluster.

func (*CRDHealth) SetDegraded 

func (h *CRDHealth) SetDegraded()

SetDegraded marks the reconciler as degraded This is used when a CRD goes missing at runtime

func (*CRDHealth) SetDependencyHealth 

func (h *CRDHealth) SetDependencyHealth(depName string, status DependencyStatus)

SetDependencyHealth updates a single dependency's status

func (*CRDHealth) SetMissingAtRuntime 

func (h *CRDHealth) SetMissingAtRuntime()

func (*CRDHealth) SetNotStarted 

func (h *CRDHealth) SetNotStarted()

SetNotStarted marks the reconciler as not started.

func (*CRDHealth) SetStarted 

func (h *CRDHealth) SetStarted()

SetStarted marks the reconciler as started and records the start time. CompareAndSwap ensures the timestamp is only set once. pending is only set to true if the CRD has not yet had a successful reconcile — avoids flipping healthy CRDs back to pending on resync-triggered worker restarts.

func (*CRDHealth) SetTotalWorkers 

func (h *CRDHealth) SetTotalWorkers(count int32)

SetTotalWorkers sets the expected number of workers

func (*CRDHealth) SetWorkerInfoFn added in v0.1.9 

func (h *CRDHealth) SetWorkerInfoFn(fn func() *ork_autoscaler.WorkerInfo)

SetWorkerInfoFn stores the function that returns live WorkerInfo for this CRD. Called by startCRDWorkers after constructing the reconciler.

func (*CRDHealth) SignaledHealthy 

func (h *CRDHealth) SignaledHealthy() bool

func (*CRDHealth) Started 

func (h *CRDHealth) Started() bool

Started reports whether the reconciler has begun processing events.

func (*CRDHealth) StartedAt 

func (h *CRDHealth) StartedAt() string

StartedAt returns the timestamp when the reconciler first started. If not started, returns "not started". If starting, returns "starting".

func (*CRDHealth) StateAndStatus added in v0.4.4 

func (h *CRDHealth) StateAndStatus() (string, int)

StateAndStatus returns the reconciler's derived health state and the corresponding HTTP status code used by the /health endpoint.

State is one of: 

"not started", "pending", "degraded", "healthy"

Status is either: 

200 — healthy or pending
503 — degraded or not started

func (*CRDHealth) TotalReconciles 

func (h *CRDHealth) TotalReconciles() int64

TotalReconciles returns the total number of reconcile attempts.

func (*CRDHealth) UpdateDependencyStatus 

func (h *CRDHealth) UpdateDependencyStatus(depName string, status DependencyStatus)

Dependency tracking

func (*CRDHealth) Uptime 

func (h *CRDHealth) Uptime() string

Uptime returns how long the reconciler has been running. If not started, returns "not started".

type CRDHealthResponse 

type CRDHealthResponse struct {
	Name                     string                      `json:"name"`
	State                    string                      `json:"state"` // "not started", "pending", "started", "healthy", "degraded"
	Status                   int                         `json:"status"`
	IsKonductor              bool                        `json:"isKonductor"`
	Healthy                  bool                        `json:"healthy"`
	Started                  bool                        `json:"started"`
	Pending                  bool                        `json:"pending"`
	StartedAt                string                      `json:"startedAt"`
	Uptime                   string                      `json:"uptime"`
	QueueDepth               int                         `json:"queueDepth"`
	ErrorRate                float64                     `json:"errorRate"`
	ConsecutiveFails         int64                       `json:"consecutiveFails"`
	TotalReconciles          int64                       `json:"totalReconciles"`
	ResourceCount            int                         `json:"resourceCount"`
	LastError                string                      `json:"lastError"`
	LastReconcile            string                      `json:"lastReconcile"`
	HasUnhealthyDependencies bool                        `json:"hasUnhealthyDependencies"`
	Dependencies             map[string]DependencyStatus `json:"dependencies,omitempty"`
	Missing                  bool                        `json:"missing,omitempty"`
}

type CRDInfoResponse 

type CRDInfoResponse struct {
	Name                   string                           `json:"name"`
	Description            string                           `json:"description"`
	Mode                   string                           `json:"mode"`
	GVK                    string                           `json:"gvk"`
	GVR                    string                           `json:"gvr"`
	Namespaced             bool                             `json:"namespaced"`
	Namespace              string                           `json:"namespace"`
	DependsOn              []string                         `json:"dependsOn,omitempty"`
	IsKonductor            bool                             `json:"isKonductor"`
	Workers                int                              `json:"workers"`
	WorkersActive          int32                            `json:"workersActive"`
	WorkersIdle            int32                            `json:"workersIdle"`
	WorkersProcessing      int32                            `json:"workersProcessing"`
	WorkerDetails          map[string]string                `json:"workerDetails,omitempty"`
	WorkersSource          string                           `json:"workersSource"`
	Resync                 string                           `json:"resync"`
	ResyncSource           string                           `json:"resyncSource"`
	QueueDepth             int                              `json:"queueDepth"`
	MaxDepth               int                              `json:"maxDepth"`
	MaxDepthSource         string                           `json:"maxDepthSource"`
	ResourceCount          int                              `json:"resourceCount"`
	TotalReconciles        int64                            `json:"totalReconciles"`
	OperatorBox            OperatorBoxInfo                  `json:"operatorBox"`
	Healthy                bool                             `json:"healthy"`
	Started                bool                             `json:"started"`
	Pending                bool                             `json:"pending"`
	ErrorRate              float64                          `json:"errorRate"`
	Conversion             *ConversionStatsResponse         `json:"conversion,omitempty"`
	Admission              *AdmissionStatsResponse          `json:"admission,omitempty"`
	DeletionProtection     *DeletionProtectionStatsResponse `json:"deletionProtection,omitempty"`
	NamespaceProtection    *NamespaceProtectionResponse     `json:"namespaceProtection,omitempty"`
	WebhookControllerStats *WebhookControllerStats          `json:"webhookControllerStats,omitempty"`
	Providers              []ProviderInfoResponse           `json:"providers,omitempty"`
	RBAC                   RBACInfo                         `json:"rbac,omitempty"`
	AutoscalerEnabled      bool                             `json:"autoscalerEnabled,omitempty"`
	AutoscalerWorkers      *ork_autoscaler.WorkerInfo       `json:"autoscalerWorkers,omitempty"`
	Rollback               *RollbackStats                   `json:"rollback,omitempty"`
	// Metrics is the live AutoMetrics map for this operatorbox.
	// Populated only when autoscale: is declared. Used by cross-binary autoscale
	// conditions via the source.endpoint HTTP fallback — the remote autoscaler
	// calls this endpoint and reads "metrics.*" fields from the response.
	Metrics map[string]interface{} `json:"metrics,omitempty"`
}

type CRDSummaryResponse 

type CRDSummaryResponse struct {
	Name                     string             `json:"name"`
	Description              string             `json:"description"`
	Mode                     string             `json:"mode"`
	GVK                      string             `json:"gvk"`
	GVR                      string             `json:"gvr"`
	Namespaced               bool               `json:"namespaced"`
	Namespace                string             `json:"namespace"`
	DependsOn                []string           `json:"dependsOn,omitempty"`
	HasUnhealthyDependencies bool               `json:"hasUnhealthyDependencies"`
	Workers                  int                `json:"workers"`
	WorkersSource            string             `json:"workersSource"`
	WorkersActive            int32              `json:"workersActive"`
	Resync                   string             `json:"resync"`
	ResyncSource             string             `json:"resyncSource"`
	QueueDepth               int                `json:"queueDepth"`
	MaxDepth                 int                `json:"maxDepth"`
	MaxDepthSource           string             `json:"maxDepthSource"`
	ResourceCount            int                `json:"resourceCount"`
	OperatorBox              OperatorBoxSummary `json:"operatorBox"`
	Healthy                  bool               `json:"healthy"`
	State                    string             `json:"state"`
	Started                  bool               `json:"started"`
	Pending                  bool               `json:"pending"`
	StartedAt                string             `json:"startedAt"`
	Uptime                   string             `json:"uptime"`
	ErrorRate                float64            `json:"errorRate"`
	Endpoints                EndpointInfo       `json:"endpoints"`
	RBACCount                int                `json:"rbacCount,omitempty"`
	DeletionProtection       bool               `json:"deletionProtection"`
	ProviderCount            int                `json:"providerCount,omitempty"`
	KatalogNamespace         string             `json:"katalogNamespace,omitempty"`
}

type CRDetailResponse 

type CRDetailResponse struct {
	Name              string            `json:"name"`
	Namespace         string            `json:"namespace,omitempty"`
	Generation        int64             `json:"generation"`
	CreationTimestamp string            `json:"creationTimestamp"`
	Labels            map[string]string `json:"labels,omitempty"`
	Annotations       map[string]string `json:"annotations,omitempty"`

	// Ready reflects the Orkestra Ready condition.
	// true = operator reconciled successfully.
	// The CR may still be in a non-terminal phase (e.g. Running/build).
	Ready        bool   `json:"ready"`
	ReadyReason  string `json:"readyReason,omitempty"`
	ReadyMessage string `json:"readyMessage,omitempty"`

	// Status is the full status subresource as returned by the API server.
	// Includes phase, conditions, and any declared status fields.
	Status map[string]interface{} `json:"status,omitempty"`

	// Children holds the child resources created by Orkestra for this CR.
	// Keyed by lowercase kind (e.g. "deployment", "job", "cronjob").
	// Value is a single ChildSummary when one child exists, []ChildSummary when multiple.
	// Populated on demand from the API server — may be empty on first reconcile.
	Children map[string]interface{} `json:"children"`

	// EventsEndpoint is the URL to fetch recent events for this CR.
	EventsEndpoint string `json:"eventsEndpoint"`

	// Debug to know if it has templates
	HasTemplateBlocks bool `json:"hasTemplateBlocks"`
}

CRDetailResponse is returned by GET /katalog/{crd}/cr/{namespace}/{name}.

type CREvent 

type CREvent struct {
	Type      string `json:"type"` // Normal or Warning
	Reason    string `json:"reason"`
	Message   string `json:"message"`
	Source    string `json:"source"` // component that emitted the event
	Object    string `json:"object"` // involved object kind/name
	Count     int32  `json:"count"`
	FirstSeen string `json:"firstSeen"`
	LastSeen  string `json:"lastSeen"`
	Age       string `json:"age"`
}

CREvent is one Kubernetes event involving this CR or its children.

type CREventsResponse 

type CREventsResponse struct {
	Name      string    `json:"name"`
	Namespace string    `json:"namespace,omitempty"`
	Total     int       `json:"total"`
	Events    []CREvent `json:"events"`
}

CREventsResponse is returned by GET /katalog/{crd}/cr/{namespace}/{name}/events.

type CRListResponse 

type CRListResponse struct {
	CRD   string      `json:"crd"`
	GVK   string      `json:"gvk"`
	Total int         `json:"total"`
	Items []CRSummary `json:"items"`
}

CRListResponse is returned by GET /katalog/{crd}/cr.

type CRSummary 

type CRSummary struct {
	Name        string `json:"name"`
	Namespace   string `json:"namespace,omitempty"`
	Phase       string `json:"phase,omitempty"` // from status.phase if declared
	Ready       bool   `json:"ready"`           // from status.conditions[type=Ready]
	ReadyReason string `json:"readyReason,omitempty"`
	Age         string `json:"age"`
	Generation  int64  `json:"generation"`
}

CRSummary is one row in the CR list — equivalent to one line of kubectl get.

type ChildSummary 

type ChildSummary struct {
	Name      string                 `json:"name"`
	Namespace string                 `json:"namespace,omitempty"`
	Kind      string                 `json:"kind"`
	Status    map[string]interface{} `json:"status,omitempty"`
	Ready     bool                   `json:"ready"`
}

ChildSummary is a condensed view of one child resource. Only the fields that matter for observability are included — not the full object.

type ConstructorInfo 

type ConstructorInfo struct {
	Configured bool   `json:"configured"`
	Source     string `json:"source,omitempty"`
	Location   string `json:"location,omitempty"`
	Function   string `json:"function,omitempty"`
}

type ConversionStatsResponse 

type ConversionStatsResponse struct {
	Enabled      bool  `json:"enabled"`
	Total        int64 `json:"total"`
	Success      int64 `json:"success"`
	Failures     int64 `json:"failures"`
	AvgLatencyMs int64 `json:"avgLatencyMs"`
	P95LatencyMs int64 `json:"p95LatencyMs"`
}

type DeletionProtectionStatsResponse added in v0.1.9 

type DeletionProtectionStatsResponse struct {
	Enabled bool  `json:"enabled"`
	Total   int64 `json:"total"`   // total DELETE admission reviews received
	Blocked int64 `json:"blocked"` // DELETE requests denied
	Allowed int64 `json:"allowed"` // DELETE requests allowed through
}

DeletionProtectionStatsResponse exposes deletion protection status for the CRD detail view. All counts are cumulative since operator startup.

type DependencyKordinator 

type DependencyKordinator struct {
	*Kontroller
	// contains filtered or unexported fields
}

DependencyKordinator extends the base Kontroller with dependency‑aware startup. It ensures CRDs start in topological order and shut down in reverse order.

func NewDependencyKordinator 

func NewDependencyKordinator(
	kube *kubeclient.Kubeclient,
	factory *informer.Factory,
	katalog *ResourceKatalog,
	events *event.Event,
	hs domain.Health,
	queueRegistry *queue.QueueRegistry,
	defaultWorkqueue *queue.Workqueue,
	crdHealthMap map[string]*CRDHealth,
	orkHealth *OrkestraHealth,
	defaultWorkers int,
	depGraph *katalog.DependencyGraph,
	drainTimeout time.Duration,
) *DependencyKordinator

NewDependencyKordinator constructs a dependency‑aware kordinator. It embeds the base Kontroller and handles dependencies in the correct order.

func (*DependencyKordinator) GVKToCRD 

func (k *DependencyKordinator) GVKToCRD(gvk schema.GroupVersionKind) types.CRDEntry

GVKToCRD returns the CRD entry for a given gvk

func (*DependencyKordinator) Kordinate 

func (k *DependencyKordinator) Kordinate(ctx context.Context)

Kordinate starts CRDs in dependency order and blocks until leadership is lost. When leadership ends, it shuts down CRDs in reverse dependency order.

The startup loop is non‑blocking: if a CRD's dependencies are not yet satisfied (e.g., waiting for "healthy"), the CRD is skipped. The background retry loop will activate it later when dependencies become ready.

func (*DependencyKordinator) Name 

func (k *DependencyKordinator) Name() string

Name returns the name of the dependency kordinator

func (*DependencyKordinator) NameToCRD 

func (k *DependencyKordinator) NameToCRD(name string) types.CRDEntry

NameToCRD returns the CRD for a given name

func (*DependencyKordinator) NameToGVK 

func (k *DependencyKordinator) NameToGVK(name string) schema.GroupVersionKind

NameToGVK returns the GVK for a given name

func (*DependencyKordinator) NameToGVKMap 

func (k *DependencyKordinator) NameToGVKMap() map[string]string

NameToGVKMap returns a map of names to gvk string

type DependencyStatus 

type DependencyStatus struct {
	Name                string `json:"name"`
	State               string `json:"state"`               // "healthy", "degraded", "missing", "started"
	Condition           string `json:"condition"`           // "started", "healthy", "ready"
	AcceptableCondition string `json:"acceptableCondition"` // "started", "healthy", "ready"
	Satisfied           bool   `json:"satisfied"`
	LastCheck           string `json:"lastCheck,omitempty"`
}

type EndpointInfo 

type EndpointInfo struct {
	Health string `json:"health"`
	Info   string `json:"info"`
}

type FinalizersInfo 

type FinalizersInfo struct {
	Source string   `json:"source"`
	Values []string `json:"values"`
}

type GatewayCRDStatsResponse added in v0.4.9 

type GatewayCRDStatsResponse struct {
	Name                string                           `json:"name"`
	GVK                 string                           `json:"gvk"`
	GVR                 string                           `json:"gvr"` // merge key: "group/version/resource"
	Admission           *AdmissionStatsResponse          `json:"admission,omitempty"`
	Conversion          *ConversionStatsResponse         `json:"conversion,omitempty"`
	DeletionProtection  *DeletionProtectionStatsResponse `json:"deletionProtection,omitempty"`
	NamespaceProtection *NamespaceProtectionResponse     `json:"namespaceProtection,omitempty"`
}

GatewayCRDStatsResponse holds the gateway-owned stats for one CRD. The GVR field is the merge key used by the control center.

type GatewayKatalogResponse added in v0.4.9 

type GatewayKatalogResponse struct {
	Source  string `json:"source"` // always "gateway"
	Name    string `json:"name"`
	Version string `json:"version,omitempty"`

	// Security feature flags — mirrors Katalog configuration.
	AdmissionEnabled           bool `json:"admissionEnabled"`
	ConversionEnabled          bool `json:"conversionEnabled"`
	DeletionProtectionEnabled  bool `json:"deletionProtectionEnabled"`
	NamespaceProtectionEnabled bool `json:"namespaceProtectionEnabled"`
	StrictModeEnabled          bool `json:"strictModeEnabled"`

	// Per-CRD stats — only gateway-owned fields populated.
	CRDs []GatewayCRDStatsResponse `json:"crds"`

	// Process-level stats for events not attributable to a single CRD:
	// the webhook configuration itself and Orkestra infra resources.
	InfraProtection   *DeletionProtectionStatsResponse `json:"infraProtection,omitempty"`
	WebhookController *WebhookControllerStats          `json:"webhookController,omitempty"`
	GatewayVersion    string                           `json:"gatewayVersion"`
}

GatewayKatalogResponse is served at GET /katalog by the gateway process. It mirrors the top-level shape of KatalogResponse so control-center clients can pattern-match on the "source" field and merge stats by GVR key.

type GatewayStatsProvider added in v0.4.9 

type GatewayStatsProvider interface {
	AdmissionStatsFor(gvrKey string) *health.AdmissionStats
	ConversionStatsFor(gvrKey string) *health.ConversionStats
	ProtectionStatsFor(gvrKey string) *health.DeletionProtectionStats
	NamespaceStatsFor(gvrKey string) *health.NamespaceProtectionStats
	InfraProtectionStats() *health.DeletionProtectionStats
	WebhookControllerStats() *health.WebhookStats
}

GatewayStatsProvider is the subset of webhook.WebhookServer that the gateway /katalog handlers need. Defined here so pkg/kordinator does not import pkg/webhook (which would create a cycle).

type HooksInfo 

type HooksInfo struct {
	Configured bool   `json:"configured"`
	Source     string `json:"source,omitempty"`
	Location   string `json:"location,omitempty"`
	Function   string `json:"function,omitempty"`
}

type KatalogNamespaceSummary added in v0.6.5 

type KatalogNamespaceSummary struct {
	CRDs         []string     `json:"crds"`
	StatusCounts StatusCounts `json:"statusCounts"`
	Healthy      bool         `json:"healthy"`
	Description  string       `json:"description,omitempty"`
	Version      string       `json:"version,omitempty"`
	Workers      int          `json:"workers"`
	Resources    int          `json:"resources"`
}

KatalogNamespaceSummary groups CRDs that share the same katalog namespace. Namespaces are declared in katalog.metadata.namespace — "default" when not set.

type KatalogResponse 

type KatalogResponse struct {
	CRDs               []CRDSummaryResponse   `json:"crds"`
	Total              int                    `json:"total"`
	TotalEnabled       int                    `json:"totalEnabled"`
	OrkReady           bool                   `json:"OrkReady"`
	IsKonductor        bool                   `json:"isKonductor"`
	DeletionProtection bool                   `json:"deletionProtection"`
	Healthy            bool                   `json:"healthy"`
	Status             int                    `json:"status"`
	DegradedReason     string                 `json:"degradedReason,omitempty"`
	StatusCounts       StatusCounts           `json:"statusCounts"`
	Name               string                 `json:"name,omitempty"`
	Version            string                 `json:"version,omitempty"`
	CreatedBy          string                 `json:"createdBy,omitempty"`
	Author             string                 `json:"author,omitempty"`
	Description        string                 `json:"description,omitempty"`
	License            string                 `json:"license,omitempty"`
	RuntimeVersion     string                 `json:"runtimeVersion,omitempty"`
	ClusterName        string                 `json:"clusterName,omitempty"`
	Projects           map[string]interface{} `json:"projects,omitempty"`
	// Namespaces groups CRDs by katalog namespace. Always present — at minimum
	// contains "default" when no katalog declares an explicit namespace.
	Namespaces map[string]KatalogNamespaceSummary `json:"namespaces,omitempty"`
	// GatewayEndpoint is the HTTP base URL of the companion gateway process.
	// Set via ORK_GATEWAY_ENDPOINT on the runtime. The control center reads
	// this field and fetches gateway:/katalog to merge per-CRD webhook stats.
	// Empty when no gateway is paired with this runtime.
	GatewayEndpoint string `json:"gatewayEndpoint,omitempty"`
}

type Kontroller 

type Kontroller struct {
	// contains filtered or unexported fields
}

Every map has the same key GVK

func NewKontroller 

func NewKontroller(
	kube *kubeclient.Kubeclient,
	informerFactory *informer.Factory,
	katalog *ResourceKatalog,
	event *event.Event,
	hs domain.Health,
	crdHealthMap map[string]*CRDHealth,
	orkHealth *OrkestraHealth,
	queueRegistry *queue.QueueRegistry,
	defaultWorkqueue *queue.Workqueue,
	defaultWorkers int,
) *Kontroller

func (*Kontroller) Degraded 

func (k *Kontroller) Degraded(h domain.Health)

Set the controller to degraded

func (*Kontroller) MissingCRDs 

func (k *Kontroller) MissingCRDs() map[string]*informer.InformerEntry

MissingCRDs returns a map of missing crds keyed by gvk

func (*Kontroller) Name 

func (k *Kontroller) Name() string

Controller name

func (*Kontroller) SetReady 

func (k *Kontroller) SetReady(h domain.Health)

Set the controller ready

func (*Kontroller) Shutdown 

func (k *Kontroller) Shutdown(ctx context.Context)

Shutdown gracefully stops orkestra

func (*Kontroller) Start 

func (k *Kontroller) Start(ctx context.Context) error

func (*Kontroller) Started 

func (k *Kontroller) Started() bool

Healthy mark on startup

type NamespaceProtectionResponse added in v0.1.9 

type NamespaceProtectionResponse struct {
	Enabled              bool     `json:"enabled"`
	HasNamespaceRules    bool     `json:"hasNamespaceRules"`
	Total                int64    `json:"total"`
	Blocked              int64    `json:"blocked"`
	Allowed              int64    `json:"allowed"`
	AllowedNamespaces    []string `json:"allowedNamespaces,omitempty"`    // non-nil only when allowedNamespaces: is declared
	RestrictedNamespaces []string `json:"restrictedNamespaces,omitempty"` // non-nil only when restrictedNamespaces: is declared
}

NamespaceProtectionResponse exposes namespace protection status for the CRD detail view.

type OperatorBoxInfo 

type OperatorBoxInfo struct {
	Type        string                 `json:"type"`
	Finalizers  FinalizersInfo         `json:"finalizers"`
	Hooks       HooksInfo              `json:"hooks"`
	Constructor ConstructorInfo        `json:"constructor"`
	Templates   map[string]interface{} `json:"templates,omitempty"`
}

type OperatorBoxSummary 

type OperatorBoxSummary struct {
	Type           string `json:"type"`
	HasTemplates   bool   `json:"hasTemplates,omitempty"`
	HasHooks       bool   `json:"hasHooks,omitempty"`
	HasConstructor bool   `json:"hasConstructor,omitempty"`
}

type OrkestraHealth 

type OrkestraHealth struct {
	// contains filtered or unexported fields
}

func NewOrkestraHealth 

func NewOrkestraHealth() *OrkestraHealth

NewOrkestraHEalth initializes a CRDHealth tracker for Orkestra

func (*OrkestraHealth) IsKatalogReady 

func (h *OrkestraHealth) IsKatalogReady() bool

IsKatalogReady is used to track ready state of a katalog

func (*OrkestraHealth) IsKonductor added in v0.6.0 

func (h *OrkestraHealth) IsKonductor() bool

IsKonductor reports whether this pod is the current konductor (leader). The control center uses this to decide whether to trust this pod's CRD data.

func (*OrkestraHealth) IsOrkReady 

func (h *OrkestraHealth) IsOrkReady() bool

IsOrkReady is used to track ready state of orkestra

func (*OrkestraHealth) SetIsKonductor added in v0.6.0 

func (h *OrkestraHealth) SetIsKonductor(v bool)

SetIsKonductor marks whether this pod holds the leader election lease. Set to true at the start of Kordinate(), false when leadership is lost.

func (*OrkestraHealth) SetKatalogDegraded 

func (h *OrkestraHealth) SetKatalogDegraded()

SetKatalogDegraded marks a katalog as degraded

func (*OrkestraHealth) SetKatalogReady 

func (h *OrkestraHealth) SetKatalogReady()

SetKatalogReady marks a katalog as ready

func (*OrkestraHealth) SetOrkDegraded 

func (h *OrkestraHealth) SetOrkDegraded()

SetOrkDegraded marks orkestra engine as degraded

func (*OrkestraHealth) SetOrkReady 

func (h *OrkestraHealth) SetOrkReady()

SetOrkReady marks orkestra engine as ready

type ProviderInfoResponse 

type ProviderInfoResponse struct {
	Name      string   `json:"name"`
	Kinds     []string `json:"kinds"`     // declared resource kinds (static, from Katalog)
	Total     int64    `json:"total"`     // reconcile calls since startup
	Errors    int64    `json:"errors"`    // failed reconcile calls since startup
	ErrorRate float64  `json:"errorRate"` // errors / total, 0 when no calls yet
}

ProviderInfoResponse exposes per-provider metadata and error rate for one CRD. No auth, URLs, or credentials are exposed — metadata only.

type RBACInfo 

type RBACInfo struct {
	Rules      []RBACRule `json:"rules"`
	Summary    string     `json:"summary"`
	TotalRules int        `json:"totalRules"`
}

type RBACRule 

type RBACRule struct {
	APIGroups []string `json:"apiGroups,omitempty"`
	Resources []string `json:"resources,omitempty"`
	Verbs     []string `json:"verbs,omitempty"`
	// Human-readable description
	Description string `json:"description,omitempty"`
}

type RegistryEntry 

type RegistryEntry struct {
	CRD               orktypes.CRDEntry
	Informer          cache.SharedIndexInformer
	ReconcilerFactory func() domain.Reconciler // factory lives here
	FailureThreshold  int
}

type ResourceKatalog 

type ResourceKatalog struct {
	// contains filtered or unexported fields
}

func NewKordinatorRegistry 

func NewKordinatorRegistry() *ResourceKatalog

func (*ResourceKatalog) Entries 

func (r *ResourceKatalog) Entries() map[string]RegistryEntry

func (*ResourceKatalog) Get 

func (r *ResourceKatalog) Get(gvk string) (RegistryEntry, bool)

func (*ResourceKatalog) GetCrossAccessByName added in v0.6.5 

func (r *ResourceKatalog) GetCrossAccessByName(name string) *bool

GetInformerByLabelSelector returns the SharedIndexInformer for a CRD whose metadata.labelSelector contain the given key/value pair.

This enables cross‑CRD observation by semantic grouping rather than by CRD name. Platform teams can labelSelector CRDs (e.g. "tier=platform", "domain=payments") and application‑level logic can reference them without knowing the exact CRD name.

Lookup rules:

  • Match is case‑insensitive on both key and value
  • Returns the first CRD whose labelSelector contain key=value
  • Returns nil, false when no CRD matches

This is used by GenericReconciler via the KatalogRegistry interface to support cross‑context reads without importing pkg/kordinator directly (avoiding import cycles). GetCrossAccessByName returns the CrossAccess field of the named CRD. nil means readable (default). *false means the CRD has opted out of cross reads.

func (*ResourceKatalog) GetInformerByLabelSelector added in v0.1.9 

func (r *ResourceKatalog) GetInformerByLabelSelector(key, value string) (cache.SharedIndexInformer, bool)

func (*ResourceKatalog) GetInformerByName 

func (r *ResourceKatalog) GetInformerByName(name string) (cache.SharedIndexInformer, bool)

GetInformerByName returns the SharedIndexInformer for a CRD by its lowercase name (the spec.crds map key: "pipeline", "database", "website").

The registry stores entries keyed by GVK string (e.g. "demo.orkestra.io/v1alpha1, Kind=Pipeline"). This method searches by Kind name match (case-insensitive) so callers use simple names.

Returns nil, false when no CRD with that name is registered.

func (*ResourceKatalog) GetWorkers 

func (r *ResourceKatalog) GetWorkers(gvk string, defaultWorkers int) int

func (*ResourceKatalog) ListGVKs 

func (r *ResourceKatalog) ListGVKs() []string

func (*ResourceKatalog) Register 

func (r *ResourceKatalog) Register(
	gvk string,
	crd orktypes.CRDEntry,
	inf cache.SharedIndexInformer,
	rec func() domain.Reconciler,
)

func (*ResourceKatalog) Unregister 

func (r *ResourceKatalog) Unregister(gvk string)

type RollbackStats added in v0.1.9 

type RollbackStats struct {
	// TotalRollbacks is the number of times rollback was triggered since startup.
	TotalRollbacks int64 `json:"totalRollbacks"`
	// Active is true when rollback is currently blocking normal reconciliation.
	Active bool `json:"active"`
	// LastRollbackAt is the RFC3339 timestamp of the most recent rollback trigger.
	// Empty when no rollback has occurred.
	LastRollbackAt string `json:"lastRollbackAt,omitempty"`
}

RollbackStats is a snapshot of rollback activity for one CRD.

type StatusCounts 

type StatusCounts struct {
	Healthy  int `json:"healthy"`
	Degraded int `json:"degraded"`
	Started  int `json:"started"`
	Pending  int `json:"pending"`
}

type WebhookControllerStats 

type WebhookControllerStats struct {
	Reconciled int64 // total successful reconciliation cycles
	Failed     int64 // reconciliation attempts that encountered errors
}

WebhookControllerStats tracks reconciliation counters for the webhook controller.

type WorkerStats 

type WorkerStats struct {
	Workers           int32             `json:"workers"`
	WorkersActive     int32             `json:"workersActive"`
	WorkersIdle       int32             `json:"workersIdle"`
	WorkersProcessing int32             `json:"workersProcessing"`
	WorkerDetails     map[string]string `json:"workerDetails,omitempty"`
}

───────────────────────────────────────────────────────────────────────────── CRD Info Response ─────────────────────────────────────────────────────────────────────────────

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.