orchestrator

package
v0.3.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 7, 2026 License: AGPL-3.0 Imports: 32 Imported by: 0

Documentation

Overview

Package orchestrator wires the controllers together and runs the reconcile loop. It owns:

  • A central Store.
  • A task.Service tracking active reconciliations.
  • The Source, Kustomization, and HelmRelease controllers.
  • The loader that primes the Store with on-disk objects.

The lifecycle is:

o := orchestrator.New(orchestrator.Config{...})
if err := o.Bootstrap(ctx); err != nil { ... }
if err := o.Run(ctx); err != nil { ... }       // blocks until done
o.Stop()

Bootstrap loads YAML manifests from the configured path AND publishes a synthetic GitRepository pointing at the local working tree, so Kustomizations whose sourceRef resolves to the bootstrap repo can reconcile immediately.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Config

type Config struct {
	// StageCacheBytes caps the persistent kustomize stage cache. 0
	// disables eviction (unbounded growth — the GC subcommand still
	// handles age-based cleanup). The flag's expected unit is mebibytes
	// at the CLI layer; this field is bytes.
	StageCacheBytes int64

	// Path is the directory to scan for Flux objects.
	Path string
	// PathOrig, when non-empty, switches every command into
	// changed-only mode: only resources whose source files differ
	// (plus the sources they reference) get reconciled.
	PathOrig string

	// HelmOptions tunes templating (skip CRDs/secrets/tests, kube
	// version, etc.).
	HelmOptions helm.Options
	// WipeSecrets controls Secret cleartext placeholders.
	WipeSecrets bool
	// AllowMissingSecrets converts source auth-secret-not-found errors
	// into skips and omits HelmRelease valuesFrom Secret/ConfigMap refs
	// that cannot materialize in the offline tree. Skipped source
	// resources mark Ready with a "skipped:" reason; omitted valuesFrom
	// refs let HelmReleases render with the remaining values.
	AllowMissingSecrets bool

	// RegistryConfig is the docker config.json used for OCI auth.
	RegistryConfig string

	// CacheDir overrides the default on-disk cache root. The default
	// follows os.UserCacheDir (XDG_CACHE_HOME on Linux, Library/Caches
	// on macOS, %LocalAppData% on Windows) with a "flate" subdir, so
	// the cache survives reboots and OS tmpfs cleanups. Falls back to
	// os.TempDir()/flate-cache only when UserCacheDir errors.
	CacheDir string
	// SourceCache, when non-nil, is shared across orchestrators. The
	// `flate diff` flow constructs two orchestrators that point at the
	// same on-disk source-cache root; they MUST share one *Cache so the
	// internal mutex serializes concurrent slot allocation. When nil a
	// per-orchestrator cache is constructed (fine for single-orchestrator
	// commands like `build` / `get`).
	SourceCache *source.Cache
	// ExternalChanges, when non-nil, supplies the file-level diff so
	// the orchestrator skips its built-in change.Detect step. The
	// filter is still built from this set + the loaded SourceFiles
	// during Bootstrap.
	ExternalChanges *change.Set

	// Concurrency caps the number of active reconcile bodies running
	// in parallel. <= 0 means unbounded (every Kustomization / HR
	// reconciles on its own goroutine). Background watch loops are
	// unaffected. Sensible default for I/O-bound work is
	// runtime.NumCPU() * 4.
	Concurrency int

	// SourceRetry tunes the bounded, classified retry applied to every
	// source fetch (Git/OCI/Bucket) on transient network failures —
	// connection resets, refused connections, dial/IO timeouts. Permanent
	// errors (bad path, missing secret, not-found) fail fast. Attempts
	// <= 1 disables retry; the CLI defaults it to 3. See
	// source.RetryConfig / source.WithRetry.
	SourceRetry source.RetryConfig

	// GitDepth caps the shallow-clone history depth for GitRepository
	// sources (both the bare mirror and the legacy clone). 0 clones full
	// history; the CLI defaults it to 1 (opt-out via --git-depth=0).
	// Commit-pinned refs always full-clone regardless. See
	// git.Fetcher.Depth.
	GitDepth int

	// HelmTemplateCacheBytes caps the in-memory helm template-output
	// cache. Repeat HRs with identical effective inputs (chart
	// fingerprint, resolved values, render options) hit the cache and
	// skip action.Install.RunWithContext — the single largest CPU +
	// allocation consumer in the codebase. <= 0 disables the cache.
	// The CLI flag `--helm-template-cache-mb` exposes this in MB
	// units; embedders pass bytes directly.
	HelmTemplateCacheBytes int64

	// HelmRenderCacheBytes caps the persistent on-disk helm template-
	// output cache (Phase 3.4a). Cross-process reuse: repeat `flate
	// build` / `flate diff` invocations against the same checkout
	// short-circuit the helm render entirely when the chart
	// fingerprint + resolved values + opts tuple hits the disk layer.
	// <= 0 disables disk caching; the in-memory layer (sized by
	// HelmTemplateCacheBytes) continues to operate independently.
	// The CLI flag `--helm-render-cache-mb` exposes this in MB
	// units; embedders pass bytes directly.
	HelmRenderCacheBytes int64
}

Config carries everything the orchestrator needs.

type Orchestrator

type Orchestrator struct {
	// contains filtered or unexported fields
}

Orchestrator wires controllers and drives reconciliation.

func New

func New(cfg Config) (*Orchestrator, error)

New constructs an Orchestrator. It allocates the Store and TaskService but does not yet start any reconciliation — call Bootstrap then Run.

func (*Orchestrator) Bootstrap

func (o *Orchestrator) Bootstrap(ctx context.Context) error

Bootstrap discovers manifests, applies namespace inheritance, primes existence-only sources Ready, and prepares the change filter. Delegates the load / expand / alias phase to pkg/discovery; the remainder is dependency validation + change-filter construction.

Idempotent: a second call returns nil without re-running discovery. Bootstrap mutates orchestrator state (sourceFiles, parentOf, existence, depGraph, componentCache, filter); replaying it would rebuild the change.Filter from scratch, dropping any OnAdd hook already wired into the store's listener set. Centralizing the guard here means Render, embedders, and test harnesses all get the invariant for free. A partial-failure path that returns before flipping bootstrapped=true leaves the orchestrator eligible for a clean retry on the next call.

func (*Orchestrator) Filter

func (o *Orchestrator) Filter() *change.Filter

Filter returns the change filter (may be nil-but-non-active).

func (*Orchestrator) Render

func (o *Orchestrator) Render(ctx context.Context) (*Result, error)

Render is the structured embed-friendly entry point: Bootstrap + Run + collect everything an external caller needs to consume the reconcile result. CLI / Run() ergonomics remain unchanged; callers who want a single function that returns a typed Result use this.

The returned Result is non-nil even when err is non-nil — failures during reconcile populate Result.Failed without aborting collection, so the caller sees both the partial output and the failure list. An error from Bootstrap (the load phase) is fatal and returns (nil, err); errors from Run yield (result, err).

On any returned error, render defers Stop so embedders that receive (nil, err) — Bootstrap failure paths in particular — don't leak the staging cache tempdir and helm client until process exit. Stop is sync.Once-guarded so the deferred call composes safely with Run's own deferred Stop and any explicit caller Stop.

Idempotent: a second call returns the cached Result/err pair from the first. The controllers' Configure hooks panic if invoked after Start (reconcile-shaping config is frozen once dispatch begins), so a re-Run would panic. Caching at this boundary lets embedders retry Render without restarting controllers — pair with Bootstrap's same guarantee guard above.

func (*Orchestrator) Run

func (o *Orchestrator) Run(ctx context.Context) error

Run starts every controller, blocks until the task service drains, then aggregates and returns any failures. The post-drain reporting + error-string assembly lives in finalize so Run reads as a clean start → drain → finalize sequence.

func (*Orchestrator) Stop

func (o *Orchestrator) Stop()

Stop shuts the controllers down in reverse-construction order and releases the staging cache. Safe to call multiple times: each controller's Close is idempotent (drains the unsub slice once and nils it), and StagingCache.Close zeroes its stages map after the first cleanup. Wrapped in sync.Once so the bookkeeping reads cleanly even if a caller's defer runs after Run's defer.

Embedders who call only New + Bootstrap (without Run) MUST call Stop themselves — the staging cache holds a tempdir that would otherwise leak until process exit.

func (*Orchestrator) Store

func (o *Orchestrator) Store() *store.Store

Store returns the underlying object store.

func (*Orchestrator) WithFetcher

func (o *Orchestrator) WithFetcher(kind string, f source.Fetcher) *Orchestrator

WithFetcher installs (or replaces) a per-kind source.Fetcher on the internal source controller. Call BEFORE Bootstrap. Returns the orchestrator for chaining. Use this to embed flate as a library with a custom fetcher (in-memory test fixtures, additional source kinds, alternate verification logic) without forking the New() construction.

Passing a nil fetcher unregisters the kind — useful for stripping a default registration in tests.

Panics if called after Bootstrap: by then discovery has already run and any source CR discovered between New() and Bootstrap was reconciled by whichever fetcher was registered at that moment. A later swap silently misses those reconciles and the embedder gets inconsistent behavior across source CRs of the same kind. Bootstrap is the natural commit point for the controller wiring.

type Result

type Result struct {
	Manifests map[manifest.NamedResource][]map[string]any
	Failed    map[manifest.NamedResource]store.StatusInfo
	Orphans   map[manifest.NamedResource]string
}

Result is the structured output of Orchestrator.Render: the rendered manifests keyed by the originating Kustomization / HelmRelease, the set of resources that failed reconcile, and the orphans flate detected (sources sitting under a parent KS's spec.path but never emitted by that parent's render — real Flux would not reconcile them either).

Manifests is empty for an HR that had nothing to render or a KS whose render produced zero docs; Failed/Orphans are empty when everything reconciled cleanly.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL