bundleapply

package
v0.14.29-dev Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 28, 2026 License: MIT Imports: 21 Imported by: 0

Documentation

Overview

Package bundleapply installs an app/bundle manifest set against a kubeconfig-addressed Kubernetes control plane — specifically a PEER-HOSTED one, with no cloudbox dependency anywhere on the execution path.

Why this exists

The cloudbox-hosted DKS plane already has a working bundle-apply path, but that path assumes a cloud-minted kubeconfig and a cloud-side orchestrator. A peer plane is just k3s on someone's box: the operator has a kubeconfig (client-cert auth, written by k3s at /etc/rancher/k3s/k3s.yaml and copied per-user to ~/.kube/outpost-control-plane/k3s.yaml) and nothing else. This package takes that kubeconfig PATH as a parameter and applies a bundle using only client-go — no access_token, no /api/v1 call, no overlay credential fetch.

The two kubeconfigs are never conflated

peer control plane : ~/.kube/outpost-control-plane/k3s.yaml
cloudbox plane     : ~/.kube/outpost.yaml

This package holds neither path as a default. The caller (the script, or admincore.BundleApply behind the MCP/CLI surfaces) passes the exact path; PeerControlPlaneKubeconfig / CloudboxKubeconfig are provided only as named references so no caller has to hardcode a literal and accidentally point the peer path at the cloudbox file.

Guarantees

  • Ordering: namespaces (and other cluster-scoped prerequisites) are applied before namespaced objects, so a bundle that ships its own Namespace plus workloads in one set applies cleanly on a fresh cluster.
  • Idempotence: apply uses server-side apply semantics (a stable field manager), so applying the same bundle twice converges instead of duplicating or erroring.
  • Readiness: an apply is NOT done when the apiserver accepts the object — it is done when the workload reports Ready. WaitForReady blocks up to an explicit timeout and returns a non-nil error (which the operator entry point turns into a non-zero exit) on timeout.

Evidence invariant (docs/fleet-evidence-invariant.md)

No success state is ever reached by the ABSENCE of evidence. An empty bundle, an unreachable apiserver, an object that never becomes Ready, a DaemonSet that schedules onto zero nodes — every one of these FAILS loudly. Default-deny is the rule on any unparseable or absent signal.

Wiring status

This package is deliberately NOT wired into main.go / conf / admincore / MCP / the CLI. It exposes a clean Go API plus the standalone script script/dks-peer-bundle-apply.sh. See docs/peer-dks-bundle-apply.md ("Deferred wiring") for the exact four-surface hooks a follow-up integration run should add.

Index

Constants

View Source
const (
	// AnnotationLifecycle marks how an object participates in the bundle
	// lifecycle. The only recognized value is LifecycleInstaller; any
	// other value is a hard error (an unparseable signal must fail, not
	// vanish).
	AnnotationLifecycle = "outpost.dhnt.io/lifecycle"
	// LifecycleInstaller marks a bootstrap object (typically a Job with
	// ttlSecondsAfterFinished) that is EXPECTED to complete and may then
	// be garbage-collected. Its absence alone proves nothing — the
	// declared outputs are the durable evidence.
	LifecycleInstaller = "installer"
	// AnnotationInstalls declares the durable post-install resources an
	// installer produces. REQUIRED on every LifecycleInstaller object —
	// without it, an absent installer is indistinguishable from one that
	// never ran, so the contract refuses the manifest outright. Entries
	// are separated by commas and/or whitespace, each of the form
	//
	//	<apiVersion>:<Kind>:<namespace>:<name>
	//
	// with an empty namespace for cluster-scoped resources, e.g.
	//
	//	apps/v1:Deployment:headlamp:headlamp,
	//	rbac.authorization.k8s.io/v1:ClusterRoleBinding::headlamp-admin
	AnnotationInstalls = "outpost.dhnt.io/installs"
)

Installer-style bundle lifecycle.

Some bundles bootstrap their product through an installer Job (often with ttlSecondsAfterFinished, so the completed Job is garbage-collected by design — the appstore headlamp built-in's install-headlamp Job is the canonical case). For such bundles, "is the Job present" is the wrong installed-ness question in both directions: a completed-then-reaped Job is not evidence of absence, and a surviving Namespace/RBAC skeleton after a failed install is not evidence of presence. The manifest contract below makes the durable product explicit so status and uninstall can reason about it — and fail closed when they cannot.

View Source
const CloudboxKubeconfig = "~/.kube/outpost.yaml"

CloudboxKubeconfig is the conventional per-user path to the cloudbox-hosted plane's kubeconfig. Named here purely so callers can assert "this is NOT the peer path" without hardcoding the literal.

View Source
const FieldManager = "outpost-bundleapply"

FieldManager is the server-side-apply field manager this package owns. A stable manager string is what makes repeated applies converge instead of fighting each other over field ownership.

View Source
const PeerControlPlaneKubeconfig = "~/.kube/outpost-control-plane/k3s.yaml"

PeerControlPlaneKubeconfig is the conventional per-user path to a peer-hosted (k3s) control plane's admin kubeconfig. It is NOT applied as a default anywhere in this package — callers pass an explicit path. It exists so a caller can reference the peer plane by name rather than retyping the literal and risking a typo that lands on the cloudbox file.

Variables

View Source
var (
	// ErrEmptyBundle is returned when a bundle path yields zero
	// applicable objects. An empty bundle is a failure, never a silent
	// success (evidence invariant): if the operator pointed us at a path,
	// something was expected to be there.
	ErrEmptyBundle = errors.New("bundleapply: bundle contains no Kubernetes objects")

	// ErrReadinessTimeout is returned by WaitForReady when at least one
	// object did not report Ready before the deadline. The caller must
	// exit non-zero on this error.
	ErrReadinessTimeout = errors.New("bundleapply: timed out waiting for objects to become Ready")

	// ErrNoKubeconfig is returned when the kubeconfig path is empty or the
	// file does not exist. There is no fallback to a default cluster — a
	// missing kubeconfig fails loudly.
	ErrNoKubeconfig = errors.New("bundleapply: kubeconfig path is empty or missing")

	// ErrCloudboxVenue is returned by the venue guard when the kubeconfig
	// path — after tilde expansion, absolutization, and symlink
	// resolution — is the CLOUDBOX kubeconfig. A peer bundle apply must
	// never land on the cloudbox plane, however the path was spelled.
	ErrCloudboxVenue = errors.New("bundleapply: refusing to apply a peer bundle against the cloudbox kubeconfig — the peer and cloudbox planes are never conflated")

	// ErrVenueUnresolvable is returned when the kubeconfig path cannot be
	// canonicalized (missing file, dangling symlink, unresolvable
	// component). An unresolvable venue FAILS — it is never "probably
	// fine".
	ErrVenueUnresolvable = errors.New("bundleapply: kubeconfig path cannot be canonicalized")

	// ErrNotFound marks a Get whose object does not exist on the cluster.
	// The apply loop uses it to decide whether THIS run created an object
	// (and may therefore roll it back) — every other Get failure stays a
	// hard error.
	ErrNotFound = errors.New("bundleapply: object not found")

	// ErrCRDNotReady is returned when a CustomResourceDefinition shipped
	// in the bundle did not reach the Established condition — or its
	// served types did not appear in apiserver discovery — before the
	// bounded wait expired. Applying the dependent custom resources
	// anyway would race the discovery cache (the pass-locally,
	// flake-in-the-field failure class), so this is a hard stop.
	ErrCRDNotReady = errors.New("bundleapply: CustomResourceDefinition not Established / not in discovery before the deadline")

	// ErrZeroDesiredWorkload is returned when a workload declares
	// spec.replicas: 0 and the caller did not opt in via
	// AllowScaleToZero. Zero desired replicas satisfying "all replicas
	// ready" is a readiness verdict resting on the absence of evidence —
	// exactly what this package refuses to green-light.
	ErrZeroDesiredWorkload = errors.New("bundleapply: workload declares zero desired replicas — trivially 'ready' with nothing running; opt in with allow-scale-to-zero if intended")
)
View Source
var ErrDeletionTimeout = errors.New("bundleapply: timed out waiting for objects to be removed")

ErrDeletionTimeout is returned by WaitForGone when at least one object is still present after the bounded wait. The caller must exit non-zero on this error — same contract as ErrReadinessTimeout on the apply side.

Functions

func CanonicalizePath

func CanonicalizePath(path string) (string, error)

CanonicalizePath expands a leading ~, makes the path absolute, and resolves every symlink. The path must exist — EvalSymlinks on a missing file is an error, which is exactly the evidence invariant: a venue we cannot positively resolve is a failure, never a pass.

func ExpandUser

func ExpandUser(path string) (string, error)

ExpandUser expands a leading ~ or ~/ in a path to the current user's home directory. It exists so callers can accept the conventional "~/.kube/..." kubeconfig paths without a shell doing the expansion.

func ResolveVenue

func ResolveVenue(path string) (string, error)

ResolveVenue canonicalizes a kubeconfig path and enforces the venue guard: the result must not be the cloudbox kubeconfig, however it was reached. Returns the canonical path to use.

func WaitForGone

func WaitForGone(ctx context.Context, client ResourceClient, objs []*unstructured.Unstructured, opts WaitOptions) (int, error)

WaitForGone blocks until every object in objs is confirmed absent (Get reports ErrNotFound) or the timeout elapses.

Evidence invariant: a Get that fails for any reason OTHER than ErrNotFound (apiserver unreachable, permission error) is a hard failure, never treated as "gone" — the absence of a successful read is not evidence the object was removed.

func WaitForReady

func WaitForReady(ctx context.Context, client ResourceClient, objs []*unstructured.Unstructured, opts WaitOptions) (int, error)

WaitForReady blocks until every object reports Ready or the timeout elapses. It returns the count of objects confirmed Ready and a non-nil error on timeout (ErrReadinessTimeout) or on a terminal failure (e.g. a Pod that entered phase Failed, or a zero-desired workload without the explicit opt-in).

Evidence invariant: a Get that fails (apiserver unreachable, object vanished) is a hard error, never treated as "ready". Readiness must be affirmatively observed on a live object.

Types

type Bundle

type Bundle struct {
	// Objects in apply order: cluster-scoped prerequisites (Namespaces,
	// CRDs, RBAC) first, then namespaced workloads.
	Objects []*unstructured.Unstructured
}

Bundle is an ordered set of Kubernetes objects decoded from a manifest path. Objects are already sorted into apply order (see applyRank).

func LoadBundle

func LoadBundle(path string) (*Bundle, error)

LoadBundle reads a bundle from a path — either a single manifest file or a directory of *.yaml / *.yml / *.json files (recursively) — splits multi-document YAML, decodes each document into an unstructured object, and returns them in apply order.

An empty result is ErrEmptyBundle, never a silent success: pointing at a path that yields nothing is a failure per the evidence invariant.

type DeleteOptions

type DeleteOptions struct {
	// Client is the Kubernetes surface to delete against (required).
	Client ResourceClient
	// Timeout bounds an optional wait for the deleted objects to actually
	// vanish (finalizers, garbage collection). Zero/negative skips the
	// wait entirely — the objects are deleted and the call returns
	// without confirming they are gone.
	Timeout time.Duration
	// PollInterval is how often the gone-check is re-run. Defaults to 2s.
	PollInterval time.Duration
	// Log receives progress lines. Defaults to no-op.
	Log Logf
}

DeleteOptions controls one uninstall run.

type DeleteResult

type DeleteResult struct {
	// Deleted lists the objects (kind ns/name) this run removed — in
	// reverse apply order.
	Deleted []string
	// Failed lists objects that could NOT be deleted (with the reason).
	// A non-empty Failed always accompanies a non-nil error.
	Failed []string
	// Gone is the count confirmed absent after the wait (only meaningful
	// when Options.Timeout > 0).
	Gone int
}

DeleteResult reports one uninstall run.

func DeleteBundle

func DeleteBundle(ctx context.Context, b *Bundle, opts DeleteOptions) (DeleteResult, error)

DeleteBundle removes every object in the bundle, in reverse apply order (dependents before the Namespace/CRDs/RBAC they depend on), best-effort across all objects — one delete failure does not stop the rest from being attempted. When Options.Timeout > 0 it then waits, bounded, for the deleted objects to actually disappear.

This is the uninstall counterpart of ApplyBundle: it reuses the exact deletion mechanics (deleteReverse) that ApplyBundle's failure path uses to roll back a partial apply, so an operator-initiated uninstall and an apply-triggered rollback can never drift in ordering or accounting shape.

Installer lifecycle (see lifecycle.go): the delete set is the bundle objects PLUS every installer's declared outputs — the actual installed product, not just the bootstrap manifest. Namespaced outputs would fall with their Namespace anyway (the explicit delete just confirms it), but cluster-scoped outputs (a ClusterRoleBinding an installer created) survive a Namespace deletion and are only removed because they are declared. Deleting an already-gone object is success, so re-running an uninstall — or uninstalling after a TTL-reaped installer — converges.

type Logf

type Logf func(format string, args ...any)

Logf is the logging sink the Applier writes progress to. Defaults to a no-op; the CLI wires it to stderr. Kept as a plain func so this package imports no logging framework.

type ObjectStatus

type ObjectStatus struct {
	Kind      string `json:"kind"`
	Namespace string `json:"namespace,omitempty"`
	Name      string `json:"name"`
	// Exists reports whether the object is present on the cluster at all.
	Exists bool `json:"exists"`
	// Ready reuses the exact rollout evaluation ApplyBundle's readiness
	// wait uses (evalReadiness) — a status Ready=true means the same
	// thing an apply's confirmed-ready count means.
	Ready bool `json:"ready"`
	// Reason is a short human string: why not-ready, why not-exists, or
	// the confirming detail when ready.
	Reason string `json:"reason,omitempty"`
	// Installer marks an object annotated outpost.dhnt.io/lifecycle=installer.
	// Its absence alone never decides installed-ness — the declared
	// outputs do (see the lifecycle contract in lifecycle.go).
	Installer bool `json:"installer,omitempty"`
	// DeclaredOutput marks a row synthesized from an installer's
	// outpost.dhnt.io/installs declaration rather than decoded from the
	// manifest — the durable post-install evidence being asserted.
	DeclaredOutput bool `json:"declared_output,omitempty"`
}

ObjectStatus is one bundle object's live state — a pure read, no apply or wait involved.

type Options

type Options struct {
	// Client is the Kubernetes surface to apply against (required).
	Client ResourceClient
	// Timeout bounds the readiness wait for the WHOLE bundle. Must be > 0
	// — a zero/negative timeout is rejected rather than treated as
	// "wait forever" (an unbounded wait can mask a stuck rollout).
	Timeout time.Duration
	// PollInterval is how often readiness is re-checked. Defaults to 2s
	// when unset.
	PollInterval time.Duration
	// CRDWaitTimeout bounds, per CRD, the wait for Established + the
	// served types appearing in discovery before dependent custom
	// resources are applied. Defaults to 60s when unset. Timing out is a
	// hard failure (ErrCRDNotReady), never "apply the CR and hope".
	CRDWaitTimeout time.Duration
	// AllowScaleToZero is the EXPLICIT opt-in that lets a workload with
	// zero desired replicas count as rolled out. Without it a
	// spec.replicas: 0 workload is a terminal failure — zero desired
	// satisfying "all replicas ready" is an absence-of-evidence green.
	AllowScaleToZero bool
	// DisableRollback keeps everything this run created in place on
	// failure, instead of the default best-effort cleanup. The result
	// still reports exactly what was created and left behind.
	DisableRollback bool
	// RollbackTimeout bounds the best-effort cleanup pass. Defaults to
	// 60s. The cleanup runs on a context detached from the caller's (a
	// cancelled apply must still get its bounded cleanup chance).
	RollbackTimeout time.Duration
	// Log receives progress lines. Defaults to no-op.
	Log Logf
}

Options controls one apply run.

type ResourceClient

type ResourceClient interface {
	// Apply performs an idempotent server-side apply of obj (create if
	// absent, reconcile if present) and returns the applied object.
	Apply(ctx context.Context, obj *unstructured.Unstructured) (*unstructured.Unstructured, error)
	// Get fetches the live object matching obj's GVK / namespace / name.
	// A missing object is reported as an error wrapping ErrNotFound so
	// the apply loop can tell "does not exist yet" (this run will create
	// it) from every other failure, which stays hard.
	Get(ctx context.Context, obj *unstructured.Unstructured) (*unstructured.Unstructured, error)
	// Delete removes the live object matching obj's GVK / namespace /
	// name. Deleting an object that is already gone is a success — the
	// rollback path must converge, not fight over who removed it.
	Delete(ctx context.Context, obj *unstructured.Unstructured) error
	// ServesGVK reports whether the apiserver's discovery currently
	// serves the given GroupVersionKind. Used to gate custom resources
	// behind their CRD actually being discoverable — an Established CRD
	// whose type has not reached the discovery cache still 404s.
	ServesGVK(ctx context.Context, gvk schema.GroupVersionKind) (bool, error)
}

ResourceClient is the minimal Kubernetes surface bundleapply needs. It is an interface so the apply/wait logic can be driven by a deterministic in-memory fake in tests — no apiserver, no kubeconfig — while production uses the dynamic-client implementation below.

func NewDynamicClient

func NewDynamicClient(kubeconfigPath string) (ResourceClient, error)

NewDynamicClient builds a ResourceClient from a kubeconfig PATH. The path is required and must exist — there is NO fallback to an in-cluster config or a default cluster (a missing kubeconfig fails loudly, per the no-silent-fallback rule).

The venue guard is enforced HERE, in the Go API: the path is canonicalized (tilde expansion, absolutization, symlink resolution) and refused when it lands on the cloudbox kubeconfig — so a caller using this package directly cannot bypass the guard the operator script also applies. A path that cannot be canonicalized is an error.

Nothing on this path touches cloudbox: it reads the kubeconfig file and dials the apiserver named in it, nothing more.

type Result

type Result struct {
	Applied int
	Ready   int
	// Created lists the objects (kind ns/name) that did not exist before
	// this run applied them — the rollback set. Pre-existing objects this
	// run reconciled are never in here and are never deleted.
	Created []string
	// RolledBack lists the created objects the failure cleanup deleted.
	RolledBack []string
	// CleanupFailed lists created objects the cleanup could NOT delete
	// (with the reason) — these were left behind and the operator must
	// remove them by hand.
	CleanupFailed []string
}

Result summarizes an apply run — including, on failure, the precise transactional accounting: what this run created, what the rollback removed, and what it could NOT remove. A partial apply is never left behind silently.

func ApplyBundle

func ApplyBundle(ctx context.Context, b *Bundle, opts Options) (Result, error)

ApplyBundle applies every object in the bundle in order, then waits for the workloads to become Ready. It is the single top-level entry point.

Ordering: objects are applied in applyRank order, so Namespaces (and other cluster-scoped prerequisites) land before namespaced objects.

CRD gate: when the bundle ships a CustomResourceDefinition together with custom resources of the type it defines, each such CR is applied only after the CRD reports Established AND the type appears in apiserver discovery — bounded by Options.CRDWaitTimeout, hard failure on timeout. Applying immediately would race the discovery cache and fail intermittently.

Idempotence: each Apply is a server-side apply under a stable field manager, so calling ApplyBundle twice with the same bundle converges.

Transactionality: the run records which objects it CREATED (as opposed to reconciled). On any failure — apply error, CRD gate timeout, readiness timeout, terminal workload state — those created objects are deleted again in reverse apply order, best-effort and bounded, and the result + error report precisely what was and was not cleaned. Objects that existed before this run are never deleted.

Readiness: after all objects are applied, WaitForReady blocks up to Options.Timeout. A timeout returns ErrReadinessTimeout — the operator entry point turns that into a non-zero exit.

type StatusOptions

type StatusOptions struct {
	// Client is the Kubernetes surface to read (required).
	Client ResourceClient
	// AllowScaleToZero mirrors Options.AllowScaleToZero: a workload with
	// spec.replicas: 0 reports Ready (scaled down on purpose) instead of
	// the terminal ErrZeroDesiredWorkload reason. Without it, a
	// zero-desired workload's Reason names the same terminal condition
	// ApplyBundle would have failed on — status never waits, it only
	// reports what a wait would have seen.
	AllowScaleToZero bool
}

StatusOptions controls one status read.

type StatusResult

type StatusResult struct {
	Objects []ObjectStatus
	// Installed is true only when every non-installer object exists on
	// the cluster AND every installer's declared durable outputs exist.
	// An installer's own absence never decides it (lifecycle contract).
	Installed bool
	// AllReady is true only when every asserted object — non-installer
	// bundle objects plus declared installer outputs — exists AND reports
	// Ready. A garbage-collected installer does not count against it; a
	// present-but-unready (e.g. failed) installer does.
	AllReady bool
}

StatusResult is the bundle-wide status snapshot.

func StatusBundle

func StatusBundle(ctx context.Context, b *Bundle, opts StatusOptions) (StatusResult, error)

StatusBundle reports the live state of every object in the bundle without applying or deleting anything. It is a single pass — no polling, no bounded wait — built from the same evidence primitives ApplyBundle's readiness wait uses (Client.Get + evalReadiness), so "installed and ready" here means exactly what a successful ApplyBundle would have confirmed.

Evidence invariant: a Get failure that is NOT "object not found" (an unreachable apiserver, a permission error) is a hard failure, never reported as "not installed" — the absence of a successful read is not evidence the object is absent.

Installer lifecycle (see lifecycle.go): an object annotated outpost.dhnt.io/lifecycle=installer is judged by its DECLARED OUTPUTS, not by its own presence. A completed installer Job reaped by ttlSecondsAfterFinished does not make the bundle "not installed" as long as every declared durable output exists; conversely, a bundle is NEVER installed while any declared output is missing — a surviving Namespace/RBAC skeleton after a failed install stays installed=false.

type WaitOptions

type WaitOptions struct {
	Timeout          time.Duration
	PollInterval     time.Duration
	AllowScaleToZero bool
	Log              Logf
}

WaitOptions bounds one readiness wait.

Directories

Path Synopsis
Command bundleapply is the standalone runner behind script/dks-peer-bundle-apply.sh: it applies an app/bundle manifest set against a PEER-HOSTED control plane addressed purely by a kubeconfig path.
Command bundleapply is the standalone runner behind script/dks-peer-bundle-apply.sh: it applies an app/bundle manifest set against a PEER-HOSTED control plane addressed purely by a kubeconfig path.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL