controller

package
v0.9.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 30, 2026 License: Apache-2.0 Imports: 105 Imported by: 0

README

pkg/controller

Event-driven coordination layer for HAPTIC. This package is the only one in the tree that knows about the EventBus — it wraps the pure libraries (pkg/templating, pkg/k8s, pkg/dataplane, pkg/stores, pkg/webhook, pkg/httpstore) in event adapters and orchestrates startup, reconciliation, and shutdown.

Most day-to-day work in this package does not need this README; pkg/controller/CLAUDE.md has the detailed developer context and the source is the authoritative interface. This page is a short map for newcomers.

Mental Model

Three layers:

Pure library          Event adapter                         EventBus consumers
(pkg/templating,      (pkg/controller/renderer,             (commentator, metrics,
 pkg/dataplane,        pkg/controller/validator,             debug, pipeline)
 pkg/k8s, ...)         pkg/controller/deployer, ...)
  • Pure libraries expose plain Go APIs with no event dependencies.
  • Event adapters embed *component.Base (the shared event-loop scaffold) and translate events to pure-library calls and back.
  • Consumers (EventCommentator, metrics adapter, debug server) subscribe without publishing — they observe.

The sub-package tree in this directory mirrors those three layers. For the canonical list see docs/site/docs/development/design/package-structure.md.

Startup Sequence

The controller runs a reinitialization loop (iteration.go). Each iteration:

  1. Fetch and validate the HAProxyTemplateConfig and credentials on first start, or consume the previous iteration's accepted snapshot once. A failed replacement attempt fetches live state on retry. A schema re-resolution runs all four config validators before activation.
  2. Build a fresh EventBus and register every component via pkg/lifecycle.
  3. Start resource watchers and wait for initial sync.
  4. EventBus.Start() releases buffered events.
  5. Start reconciliation components (renderer, validator, deployer scheduler, drift monitor, metrics adapter, commentator).
  6. Observe reload requests throughout startup and steady state. A request cancels startup waits immediately and hands its exact config, discovery resolution, sources, and credentials to the next iteration.

This is why the docs consistently say "no pod restart on config change" — the CRD watcher triggers a fresh iteration inside the same process.

Key Sub-Packages

Purpose Package
Shared event-loop scaffold (embedded by nearly every component) component/
Observability (domain-aware logs, ring-buffered event history) commentator/, debug/, metrics/
Configuration ingestion (CRD + Secret loading) configloader/, credentialsloader/
Reconciliation pipeline (debounce → render → validate → publish) reconciler/, renderer/, validator/, pipeline/, rendercontext/, warmer/ (follower-side render that keeps the incremental graph warm)
Deployment orchestration (scheduler, executor, drift prevention) deployer/, discovery/, configpublisher/, statusapplier/
Webhook & validation bridges webhook/, dryrunvalidator/, proposalvalidator/, testrunner/
Leader election + leader-only gating leaderelection/, leadership/
Store management and overlay handling resourcestore/, resourcewatcher/, indextracker/
Event catalogue (≈50 domain events) events/

component/ is the biggest reusable abstraction: new components embed *component.Base, implement HandleEvent(event), and get subscribe-on-construction, single-flight dispatch, panic recovery, and ready/done signalling for free. See component/base.go and the existing consumers for examples.

Event Patterns

Two coordination modes via pkg/events:

  • Publish/Subscribe — fire-and-forget, buffered per subscriber. Carries resource changes, reconciliation triggers, and render/deployment results. The pipeline itself uses synchronous calls.
  • Request/Response (scatter-gather) — synchronous with timeout and expected-responder list. Used for config-set validation, where registered validators must approve the proposed configuration. Watched-resource admission uses the synchronous proposal pipeline.

Domain event types live in pkg/controller/events. pkg/events itself is domain-agnostic.

Writing a New Component

The short version:

  1. Decide whether it's a pure library (goes to a top-level pkg/<name>) or coordination (goes here as an event adapter).
  2. If it's coordination, embed *component.Base and implement HandleEvent.
  3. Subscribe to the events you need in the constructor — not in Start() — so buffered events aren't lost when EventBus.Start() releases them. Exception: leader-only components subscribe in Start() using SubscribeTypesLeaderOnly(), which suppresses the late-subscriber warning. The lifecycle registry only invokes Start() for those after leadership is held; all-replica components replay their last state on BecameLeaderEvent so the late-subscribing leader-only components still see current state. See reconciler/coordinator.go and configpublisher/component.go for the canonical leader-only pattern, and LEADER_ONLY_COMPONENTS.md for the contract.
  4. Register with pkg/lifecycle (mark leader-only, declare dependencies, add a health source).
  5. Add a log case to commentator/ for every new event type so it lands in the ring-buffered history.

pkg/controller/CLAUDE.md has the long version plus leadership-transition patterns (state replay on BecameLeaderEvent, cleanup on LostLeadershipEvent) that every new leader-only component must implement.

Testing

go test ./pkg/controller/...             # unit + adapter tests
go test ./pkg/controller/... -race       # race detector

Event adapters are typically tested by wiring up a real EventBus, publishing a trigger event, and asserting on the resulting events. See pkg/controller/configloader/loader_test.go for the canonical pattern (subscribes to a CRD-change event, publishes a parsed-config event); component.Base has its own unit tests in component/base_test.go.

End-to-end tests that actually spin up controllers against a Kind cluster live under tests/integration (build-tagged //go:build integration), not here. Run them with make test-integration or go test -tags=integration ./tests/integration/....

See Also

  • pkg/events — EventBus infrastructure
  • pkg/lifecycle — component registry, dependency ordering, leader-only gating
  • pkg/controller/CLAUDE.md — developer context, leadership-transition patterns, pitfalls
  • pkg/controller/LEADER_ONLY_COMPONENTS.md — checklist for leader-only components
  • docs/site/docs/development/design/package-structure.md — whole-repo orientation
  • docs/site/docs/development/design/sequence-diagrams.md — reconciliation and validation flows

License

Apache-2.0 — see root LICENSE.

Documentation

Overview

Package controller provides the main controller orchestration for the HAProxy template ingress controller.

The controller follows an event-driven architecture with a reinitialization loop: 1. Fetch and validate initial configuration 2. Create EventBus and components 3. Start components and watchers 4. Wait for configuration changes 5. Reinitialize on valid config changes

Index

Constants

View Source 
const (
	// RetryDelay is the duration to wait before retrying after an iteration failure.
	RetryDelay = 5 * time.Second
	// ConfigPollInterval is the interval for polling HAProxyTemplateConfig availability.
	ConfigPollInterval = 5 * time.Second
	// DebugEventBufferSize is the size of the event buffer for debug/introspection.
	DebugEventBufferSize = busevents.DebugSubscriberBuffer
	// ShutdownTimeout is the maximum time to wait for goroutines to finish during shutdown.
	// Set to 25s to allow clean exit before Kubernetes' default 30s terminationGracePeriodSeconds.
	ShutdownTimeout = 25 * time.Second
	// ShutdownProgressInterval is how often to log progress during shutdown.
	ShutdownProgressInterval = 5 * time.Second
	// ProcessShutdownTimeout leaves one second before the chart's SIGKILL deadline.
	ProcessShutdownTimeout = 89 * time.Second
)
View Source 
const (
	// ReadinessPath and LivenessPath judge the serving iteration; /healthz
	// keeps judging the newest one (ADR-0028).
	ReadinessPath = "/readyz"
	LivenessPath  = "/livez"
)

Variables

This section is empty.

Functions

func NewClusterSchemaFetcher 

func NewClusterSchemaFetcher(apiext apiextensionsclientset.Interface, d discovery.DiscoveryInterface) schemafetcher.Fetcher

NewClusterSchemaFetcher composes the apiextensions CRD getter and the discovery-backed OpenAPI v3 provider into the live-cluster schema fetcher. The controller's iteration wiring and the CLI commands that reach a live cluster both build their schema access through this entry point. A cached d (memory.NewMemCacheClient) answers every CRD's plural from one discovery fetch.

func ResolveLibraryRefs 

func ResolveLibraryRefs(
	ctx context.Context,
	k8sClient *client.Client,
	libraryGVR schema.GroupVersionResource,
	config *unstructured.Unstructured,
) (sources []*unstructured.Unstructured, unresolved []string, err error)

ResolveLibraryRefs fetches the HAProxyTemplateLibrary a config references, in merge order, and returns them followed by the config itself — the config is last so its inline content wins over every referenced snippet.

A reference resolves only when the object exists AND its spec.revision equals the revision the reference names. Both strings come from whoever wrote the objects; nothing here derives a revision from content, so an operator editing a snippet in place keeps a resolving reference while a half-applied set does not. Unresolved references are returned for the caller to report, not treated as an error.

func Run 

func Run(
	ctx context.Context,
	k8sClient *client.Client,
	crdName string,
	secretName, webhookCertDir string,
	webhookAdmissionTimeouts WebhookAdmissionTimeouts,
	debugPort int,
	agentTLS *transportsecurity.Source,
) error

Run is the main entry point for the controller.

It performs initial configuration fetching and validation, then enters a reinitialization loop where it responds to configuration changes by restarting with the new configuration.

The controller uses an event-driven architecture:

  • EventBus coordinates all components
  • SingleWatcher monitors HAProxyTemplateConfig CRD and Secret
  • Components react to events and publish results
  • ConfigChangeHandler detects validated config changes and signals reinitialization

Parameters:

  • ctx: Context for cancellation (SIGTERM, SIGINT, etc.)
  • k8sClient: Kubernetes client for API access
  • crdName: Name of the HAProxyTemplateConfig this controller serves (later wins); the last one is the primary, see primaryConfigName
  • secretName: Name of the Secret containing the HAPTIC agent credentials
  • webhookCertDir: Directory holding the webhook TLS cert (tls.crt/tls.key); empty disables the webhook
  • webhookAdmissionTimeouts: Controller-side admission deadlines. Zero values use the webhook component defaults.
  • debugPort: Port for debug HTTP server (0 to disable)

Returns:

  • Error if the controller cannot start or encounters a fatal error
  • nil if the context is cancelled (graceful shutdown)

func RunTypeBootstrap 

func RunTypeBootstrap(
	ctx context.Context,
	cfg *config.Config,
	fetcher schemafetcher.Fetcher,
	d discovery.DiscoveryInterface,
	logger *slog.Logger,
) (*typebootstrap.Result, error)

RunTypeBootstrap is the schema-source-agnostic core of the live type bootstrap: GVK resolution through the supplied discovery interface, schema acquisition through the supplied fetcher. The controller's iteration startup reaches it via runTypeBootstrap (which builds both from the iteration's K8s client); a CLI can call it directly with clients built from the operator's kubeconfig.

func SetBuildInfo 

func SetBuildInfo(version, haproxyVersion string)

SetBuildInfo configures version information exposed via the haptic_build_info Prometheus metric. Must be called before Run() to take effect.

Parameters:

  • version: Controller version (e.g., "0.1.0-alpha.10")
  • haproxyVersion: HAProxy version the controller was built for (e.g., "3.2")

func SetBuildSourceHash added in v0.7.0 

func SetBuildSourceHash(value string)

SetBuildSourceHash sets the immutable source identity before Run starts.

Types

type DiscoveryServedChecker 

type DiscoveryServedChecker struct {
	// contains filtered or unexported fields
}

DiscoveryServedChecker answers "is this plural resource served at this group/version?" from live apiserver discovery, memoizing one discovery call per distinct group/version. It doubles as the SchemaFieldChecker for RequiresFields stripping: the same discovery answer carries the Kind for each served plural, which the schema fetcher needs to probe the resolved schema's fields. Instances are single-use snapshots: build a fresh one per resolution so CRDs installed or upgraded between config loads are seen. Not safe for concurrent use (resolution is sequential).

Only an authoritative NotFound counts as "unserved". A TRANSIENT discovery error (apiserver blip, aggregated-API hiccup) is recorded in transientErr instead: silently treating it as unserved would strip optional features — and bounce the controller through a spurious reinit — on every blip. The caller checks TransientErr() after resolution and fails the whole resolution, which the iteration retry loop (or the next CRD event) re-attempts safely. Schema-fetch errors during FieldServed fail the resolution directly (the SchemaFieldChecker contract).

func NewDiscoveryServedChecker 

func NewDiscoveryServedChecker(ctx context.Context, d discovery.DiscoveryInterface, fetcher schemafetcher.Fetcher, logger *slog.Logger) *DiscoveryServedChecker

func (*DiscoveryServedChecker) FieldServed 

func (c *DiscoveryServedChecker) FieldServed(apiVersion, resources, fieldPath string) (bool, error)

FieldServed implements coreconfig.SchemaFieldChecker: it fetches the resolved schema for the resource (CRD openAPIV3Schema or aggregated OpenAPI v3, via the cluster fetcher) and walks it for the dot-path. Every error — transient fetch failure or a genuinely missing schema for a resource discovery says is served — fails the resolution rather than silently stripping, mirroring the fail-closed type-bootstrap contract.

func (*DiscoveryServedChecker) IsNamespaced added in v0.7.0 

func (c *DiscoveryServedChecker) IsNamespaced(apiVersion, resources string) bool

IsNamespaced implements coreconfig.NamespaceScopeChecker.

func (*DiscoveryServedChecker) IsServed 

func (c *DiscoveryServedChecker) IsServed(apiVersion, resources string) bool

IsServed implements coreconfig.ServedVersionChecker.

func (*DiscoveryServedChecker) TransientErr 

func (c *DiscoveryServedChecker) TransientErr() error

TransientErr returns the first non-NotFound discovery error observed, or nil when every answer was authoritative.

type InitialConfigBundle 

type InitialConfigBundle struct {
	Config      *coreconfig.Config
	CRD         *v1alpha1.HAProxyTemplateConfig
	Credentials *coreconfig.Credentials
	// ConfigVersion identifies the whole merged set, not just the primary
	// config — see conversion.CompositeVersion. The bootstrap guard compares
	// it for equality, so it has to change when ANY member changes.
	ConfigVersion      string
	CredentialsVersion string
	// Sources names the HAProxyTemplateConfig objects that receive status, at the
	// generation the merge observed. Libraries are a different kind and get none.
	Sources []events.ConfigSourceRef
}

fetchAndValidateInitialConfig fetches, parses, and validates the initial HAProxyTemplateConfig CRD and credentials Secret.

Returns the validated configuration and credentials, or an error if any step fails. InitialConfigBundle holds the parsed initial CRD / Secret values plus the resource versions of the underlying Secrets, so the iteration startup can wire bootstrap-version filtering for ConfigChangeHandler.

type StateCache 

type StateCache struct {
	*component.Base
	// contains filtered or unexported fields
}

StateCache caches controller state by subscribing to events.

This component implements the debug.StateProvider interface and provides thread-safe access to the controller's internal state for debug purposes.

It subscribes to key events and updates its cached state accordingly:

  • ConfigValidatedEvent → updates config cache
  • CredentialsUpdatedEvent → updates credentials cache
  • TemplateRenderedEvent → updates rendered config cache
  • ReconciliationTriggeredEvent → updates pipeline trigger state
  • RenderGateCompletedEvent/ValidationFailedEvent → updates validation state
  • DeploymentStartedEvent/CompletedEvent → updates deployment state
  • InstanceDeploymentFailedEvent → tracks failed endpoints

func NewStateCache 

func NewStateCache(eventBus *busevents.EventBus, resourceWatcher *resourcewatcher.ResourceWatcherComponent, logger *slog.Logger) *StateCache

NewStateCache creates a new state cache component.

The StateCache subscribes to the EventBus in the constructor (before EventBus.Start()) to ensure proper startup synchronization and receive all buffered startup events.

Usage: 

stateCache := NewStateCache(eventBus, resourceWatcher, logger)
go stateCache.Start(ctx)  // Process events in background
eventBus.Start()          // Release buffered events

func (*StateCache) GetAuxiliaryFiles 

func (sc *StateCache) GetAuxiliaryFiles() (*dataplane.AuxiliaryFiles, time.Time, error)

GetAuxiliaryFiles implements debug.StateProvider.

func (*StateCache) GetConfig 

func (sc *StateCache) GetConfig() (*coreconfig.Config, string, error)

GetConfig implements debug.StateProvider.

func (*StateCache) GetCredentials 

func (sc *StateCache) GetCredentials() (*coreconfig.Credentials, string, error)

GetCredentials implements debug.StateProvider.

func (*StateCache) GetErrors 

func (sc *StateCache) GetErrors() (*debug.ErrorSummary, error)

GetErrors implements debug.StateProvider.

func (*StateCache) GetPipelineStatus 

func (sc *StateCache) GetPipelineStatus() (*debug.PipelineStatus, error)

GetPipelineStatus implements debug.StateProvider.

func (*StateCache) GetRenderedConfig 

func (sc *StateCache) GetRenderedConfig() (string, time.Time, error)

GetRenderedConfig implements debug.StateProvider.

func (*StateCache) GetResourceCounts 

func (sc *StateCache) GetResourceCounts() (map[string]int, error)

GetResourceCounts implements debug.StateProvider.

func (*StateCache) GetResourcesByType 

func (sc *StateCache) GetResourcesByType(resourceType string) ([]any, error)

GetResourcesByType implements debug.StateProvider.

For `store: on-demand` types the result is the warm-cache subset only, not the full tracked set (see the partial-result contract on the interface declaration and listResources below). GetResourceCounts() remains the authoritative total.

func (*StateCache) GetValidatedConfig 

func (sc *StateCache) GetValidatedConfig() (*debug.ValidatedConfigInfo, error)

GetValidatedConfig implements debug.StateProvider.

func (*StateCache) HandleEvent 

func (sc *StateCache) HandleEvent(event busevents.Event)

HandleEvent implements component.EventHandler: it processes events and updates cached state.

type WebhookAdmissionTimeouts 

type WebhookAdmissionTimeouts struct {
	Resource time.Duration
}

WebhookAdmissionTimeouts contains the controller-side deadline for watched-resource admission. (HAProxyTemplateConfig admission no longer exists — ADR-0016.)

Source Files

View all Source files

Directories

Path Synopsis
Package buffers provides fixed buffer sizes for event subscriptions.
 Package buffers provides fixed buffer sizes for event subscriptions.
Package coalesce provides utilities for event coalescing in controller components.
 Package coalesce provides utilities for event coalescing in controller components.
Package commentator provides the Event Commentator pattern for domain-aware logging.
 Package commentator provides the Event Commentator pattern for domain-aware logging.
Package component provides event subscription, dispatch, panic recovery, and shutdown.
 Package component provides event subscription, dispatch, panic recovery, and shutdown.
Package configtest runs a HAProxyTemplateConfig's embedded validationTests against an already-built engine, bounded by a timeout.
 Package configtest runs a HAProxyTemplateConfig's embedded validationTests against an already-built engine, bounded by a timeout.
Package crdwatch reinitializes the controller when the CustomResourceDefinitions backing watched resources change.
 Package crdwatch reinitializes the controller when the CustomResourceDefinitions backing watched resources change.
Package debug provides controller-specific debug variable implementations.
 Package debug provides controller-specific debug variable implementations.
Package deployer applies a validated render to the HAProxy fleet through the HAPTIC agent that owns each pod's file tree.
 Package deployer applies a validated render to the HAProxy fleet through the HAPTIC agent that owns each pod's file tree.
Package diagnostics collects content-free evidence about a HAPTIC fleet.
 Package diagnostics collects content-free evidence about a HAPTIC fleet.
Package discovery provides the Discovery event adapter component.
 Package discovery provides the Discovery event adapter component.
Package dryrunvalidator implements the DryRunValidator that performs dry-run reconciliation for webhook validation.
 Package dryrunvalidator implements the DryRunValidator that performs dry-run reconciliation for webhook validation.
Package eventemitter publishes Kubernetes Events for rendered configuration, deployment failures, and rejected watched inputs.
 Package eventemitter publishes Kubernetes Events for rendered configuration, deployment failures, and rejected watched inputs.
Package events contains all domain event type definitions for the HAPTIC controller.
 Package events contains all domain event type definitions for the HAPTIC controller.
Package helpers provides shared utility functions for the controller layer.
 Package helpers provides shared utility functions for the controller layer.
Package httpstore provides the event adapter for HTTP resource fetching.
 Package httpstore provides the event adapter for HTTP resource fetching.
Package indextracker provides the IndexSynchronizationTracker that monitors resource watcher synchronization and publishes an event when all are synced.
 Package indextracker provides the IndexSynchronizationTracker that monitors resource watcher synchronization and publishes an event when all are synced.
Package inputisolation selects independently valid watched-resource updates.
 Package inputisolation selects independently valid watched-resource updates.
Package leadership provides utilities for handling leadership transitions in event-driven components.
 Package leadership provides utilities for handling leadership transitions in event-driven components.
Package names provides well-known string constants used across the controller layer.
 Package names provides well-known string constants used across the controller layer.
Package pipeline provides the render-validate pipeline for HAProxy configuration.
 Package pipeline provides the render-validate pipeline for HAProxy configuration.
Package pluggablevalidator implements the controller-side client for the pluggable-validator-sidecar wire protocol.
 Package pluggablevalidator implements the controller-side client for the pluggable-validator-sidecar wire protocol.
testutil
Package testutil provides a fixture unix-socket server for exercising the pluggablevalidator client without spinning up a real haproxy-spoa-hub sidecar.
 Package testutil provides a fixture unix-socket server for exercising the pluggablevalidator client without spinning up a real haproxy-spoa-hub sidecar.
Package proposalvalidator validates hypothetical configuration changes.
 Package proposalvalidator validates hypothetical configuration changes.
Package reconciler implements the Reconciler component that triggers reconciliation events on resource changes.
 Package reconciler implements the Reconciler component that triggers reconciliation events on resource changes.
Package rendercontext provides a centralized builder for template rendering contexts.
 Package rendercontext provides a centralized builder for template rendering contexts.
Package rendercycle binds one rendered output to every effect produced by the same controller reconciliation under an authenticated immutable root.
 Package rendercycle binds one rendered output to every effect produced by the same controller reconciliation under an authenticated immutable root.
internal/queryidentity
internal/resultauthority
Package rendergate runs the controller's own `haproxy -c` over each render, off the reconcile wall clock.
 Package rendergate runs the controller's own `haproxy -c` over each render, off the reconcile wall clock.
Package resourceapplier reconciles template-declared Kubernetes resources via Server-Side Apply (SSA).
 Package resourceapplier reconciles template-declared Kubernetes resources via Server-Side Apply (SSA).
Package resourcewatcher provides the ResourceWatcherComponent that creates and manages watchers for all Kubernetes resources defined in the controller configuration.
 Package resourcewatcher provides the ResourceWatcherComponent that creates and manages watchers for all Kubernetes resources defined in the controller configuration.
Package statusapplier applies template-driven status patches to Kubernetes resources.
 Package statusapplier applies template-driven status patches to Kubernetes resources.
Package testrunner implements validation test execution for HAProxyTemplateConfig.
 Package testrunner implements validation test execution for HAProxyTemplateConfig.
Package testutil provides shared test helpers for controller package tests.
 Package testutil provides shared test helpers for controller package tests.
Package throttle provides leading-edge refractory throttle helpers used by controller components that bound their write rate to slow downstream systems (e.g.
 Package throttle provides leading-edge refractory throttle helpers used by controller components that bound their write rate to slow downstream systems (e.g.
Package timeouts provides shared timeout constants used across controller sub-packages.
 Package timeouts provides shared timeout constants used across controller sub-packages.
Package timers provides safe timer management for event-driven controller components.
 Package timers provides safe timer management for event-driven controller components.
Package typebootstrap orchestrates the typed-watched-resources pipeline at controller startup.
 Package typebootstrap orchestrates the typed-watched-resources pipeline at controller startup.
Package validation provides pure validation services for HAProxy configuration.
 Package validation provides pure validation services for HAProxy configuration.
Package warmer keeps a follower's incremental render graph warm.
 Package warmer keeps a follower's incremental render graph warm.
Package webhook connects the admission server to controller validation.
 Package webhook connects the admission server to controller validation.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.