Documentation
¶
Overview ¶
Package bundleapply installs an app/bundle manifest set against a kubeconfig-addressed Kubernetes control plane — specifically a PEER-HOSTED one, with no cloudbox dependency anywhere on the execution path.
Why this exists ¶
The cloudbox-hosted DKS plane already has a working bundle-apply path, but that path assumes a cloud-minted kubeconfig and a cloud-side orchestrator. A peer plane is just k3s on someone's box: the operator has a kubeconfig (client-cert auth, written by k3s at /etc/rancher/k3s/k3s.yaml and copied per-user to ~/.kube/outpost-control-plane/k3s.yaml) and nothing else. This package takes that kubeconfig PATH as a parameter and applies a bundle using only client-go — no access_token, no /api/v1 call, no overlay credential fetch.
The two kubeconfigs are never conflated ¶
peer control plane : ~/.kube/outpost-control-plane/k3s.yaml cloudbox plane : ~/.kube/outpost.yaml
This package holds neither path as a default. The caller (the script, or admincore.BundleApply behind the MCP/CLI surfaces) passes the exact path; PeerControlPlaneKubeconfig / CloudboxKubeconfig are provided only as named references so no caller has to hardcode a literal and accidentally point the peer path at the cloudbox file.
Guarantees ¶
- Ordering: namespaces (and other cluster-scoped prerequisites) are applied before namespaced objects, so a bundle that ships its own Namespace plus workloads in one set applies cleanly on a fresh cluster.
- Idempotence: apply uses server-side apply semantics (a stable field manager), so applying the same bundle twice converges instead of duplicating or erroring.
- Readiness: an apply is NOT done when the apiserver accepts the object — it is done when the workload reports Ready. WaitForReady blocks up to an explicit timeout and returns a non-nil error (which the operator entry point turns into a non-zero exit) on timeout.
Evidence invariant (docs/fleet-evidence-invariant.md) ¶
No success state is ever reached by the ABSENCE of evidence. An empty bundle, an unreachable apiserver, an object that never becomes Ready, a DaemonSet that schedules onto zero nodes — every one of these FAILS loudly. Default-deny is the rule on any unparseable or absent signal.
Wiring status ¶
This package is deliberately NOT wired into main.go / conf / admincore / MCP / the CLI. It exposes a clean Go API plus the standalone script script/dks-peer-bundle-apply.sh. See docs/peer-dks-bundle-apply.md ("Deferred wiring") for the exact four-surface hooks a follow-up integration run should add.
Index ¶
- Constants
- Variables
- func CanonicalizePath(path string) (string, error)
- func ExpandUser(path string) (string, error)
- func ResolveVenue(path string) (string, error)
- func WaitForGone(ctx context.Context, client ResourceClient, objs []*unstructured.Unstructured, ...) (int, error)
- func WaitForReady(ctx context.Context, client ResourceClient, objs []*unstructured.Unstructured, ...) (int, error)
- type Bundle
- type DeleteOptions
- type DeleteResult
- type Logf
- type ObjectStatus
- type Options
- type ResourceClient
- type Result
- type StatusOptions
- type StatusResult
- type WaitOptions
Constants ¶
const ( // AnnotationLifecycle marks how an object participates in the bundle // lifecycle. The only recognized value is LifecycleInstaller; any // other value is a hard error (an unparseable signal must fail, not // vanish). AnnotationLifecycle = "outpost.dhnt.io/lifecycle" // LifecycleInstaller marks a bootstrap object (typically a Job with // ttlSecondsAfterFinished) that is EXPECTED to complete and may then // be garbage-collected. Its absence alone proves nothing — the // declared outputs are the durable evidence. LifecycleInstaller = "installer" // AnnotationInstalls declares the durable post-install resources an // installer produces. REQUIRED on every LifecycleInstaller object — // without it, an absent installer is indistinguishable from one that // never ran, so the contract refuses the manifest outright. Entries // are separated by commas and/or whitespace, each of the form // // <apiVersion>:<Kind>:<namespace>:<name> // // with an empty namespace for cluster-scoped resources, e.g. // // apps/v1:Deployment:headlamp:headlamp, // rbac.authorization.k8s.io/v1:ClusterRoleBinding::headlamp-admin AnnotationInstalls = "outpost.dhnt.io/installs" )
Installer-style bundle lifecycle.
Some bundles bootstrap their product through an installer Job (often with ttlSecondsAfterFinished, so the completed Job is garbage-collected by design — the appstore headlamp built-in's install-headlamp Job is the canonical case). For such bundles, "is the Job present" is the wrong installed-ness question in both directions: a completed-then-reaped Job is not evidence of absence, and a surviving Namespace/RBAC skeleton after a failed install is not evidence of presence. The manifest contract below makes the durable product explicit so status and uninstall can reason about it — and fail closed when they cannot.
const CloudboxKubeconfig = "~/.kube/outpost.yaml"
CloudboxKubeconfig is the conventional per-user path to the cloudbox-hosted plane's kubeconfig. Named here purely so callers can assert "this is NOT the peer path" without hardcoding the literal.
const FieldManager = "outpost-bundleapply"
FieldManager is the server-side-apply field manager this package owns. A stable manager string is what makes repeated applies converge instead of fighting each other over field ownership.
const PeerControlPlaneKubeconfig = "~/.kube/outpost-control-plane/k3s.yaml"
PeerControlPlaneKubeconfig is the conventional per-user path to a peer-hosted (k3s) control plane's admin kubeconfig. It is NOT applied as a default anywhere in this package — callers pass an explicit path. It exists so a caller can reference the peer plane by name rather than retyping the literal and risking a typo that lands on the cloudbox file.
Variables ¶
var ( // ErrEmptyBundle is returned when a bundle path yields zero // applicable objects. An empty bundle is a failure, never a silent // success (evidence invariant): if the operator pointed us at a path, // something was expected to be there. ErrEmptyBundle = errors.New("bundleapply: bundle contains no Kubernetes objects") // ErrReadinessTimeout is returned by WaitForReady when at least one // object did not report Ready before the deadline. The caller must // exit non-zero on this error. ErrReadinessTimeout = errors.New("bundleapply: timed out waiting for objects to become Ready") // ErrNoKubeconfig is returned when the kubeconfig path is empty or the // file does not exist. There is no fallback to a default cluster — a // missing kubeconfig fails loudly. ErrNoKubeconfig = errors.New("bundleapply: kubeconfig path is empty or missing") // ErrCloudboxVenue is returned by the venue guard when the kubeconfig // path — after tilde expansion, absolutization, and symlink // resolution — is the CLOUDBOX kubeconfig. A peer bundle apply must // never land on the cloudbox plane, however the path was spelled. ErrCloudboxVenue = errors.New("bundleapply: refusing to apply a peer bundle against the cloudbox kubeconfig — the peer and cloudbox planes are never conflated") // ErrVenueUnresolvable is returned when the kubeconfig path cannot be // canonicalized (missing file, dangling symlink, unresolvable // component). An unresolvable venue FAILS — it is never "probably // fine". ErrVenueUnresolvable = errors.New("bundleapply: kubeconfig path cannot be canonicalized") // ErrNotFound marks a Get whose object does not exist on the cluster. // The apply loop uses it to decide whether THIS run created an object // (and may therefore roll it back) — every other Get failure stays a // hard error. ErrNotFound = errors.New("bundleapply: object not found") // ErrCRDNotReady is returned when a CustomResourceDefinition shipped // in the bundle did not reach the Established condition — or its // served types did not appear in apiserver discovery — before the // bounded wait expired. Applying the dependent custom resources // anyway would race the discovery cache (the pass-locally, // flake-in-the-field failure class), so this is a hard stop. ErrCRDNotReady = errors.New("bundleapply: CustomResourceDefinition not Established / not in discovery before the deadline") // ErrZeroDesiredWorkload is returned when a workload declares // spec.replicas: 0 and the caller did not opt in via // AllowScaleToZero. Zero desired replicas satisfying "all replicas // ready" is a readiness verdict resting on the absence of evidence — // exactly what this package refuses to green-light. ErrZeroDesiredWorkload = errors.New("bundleapply: workload declares zero desired replicas — trivially 'ready' with nothing running; opt in with allow-scale-to-zero if intended") )
var ErrDeletionTimeout = errors.New("bundleapply: timed out waiting for objects to be removed")
ErrDeletionTimeout is returned by WaitForGone when at least one object is still present after the bounded wait. The caller must exit non-zero on this error — same contract as ErrReadinessTimeout on the apply side.
Functions ¶
func CanonicalizePath ¶
CanonicalizePath expands a leading ~, makes the path absolute, and resolves every symlink. The path must exist — EvalSymlinks on a missing file is an error, which is exactly the evidence invariant: a venue we cannot positively resolve is a failure, never a pass.
func ExpandUser ¶
ExpandUser expands a leading ~ or ~/ in a path to the current user's home directory. It exists so callers can accept the conventional "~/.kube/..." kubeconfig paths without a shell doing the expansion.
func ResolveVenue ¶
ResolveVenue canonicalizes a kubeconfig path and enforces the venue guard: the result must not be the cloudbox kubeconfig, however it was reached. Returns the canonical path to use.
func WaitForGone ¶
func WaitForGone(ctx context.Context, client ResourceClient, objs []*unstructured.Unstructured, opts WaitOptions) (int, error)
WaitForGone blocks until every object in objs is confirmed absent (Get reports ErrNotFound) or the timeout elapses.
Evidence invariant: a Get that fails for any reason OTHER than ErrNotFound (apiserver unreachable, permission error) is a hard failure, never treated as "gone" — the absence of a successful read is not evidence the object was removed.
func WaitForReady ¶
func WaitForReady(ctx context.Context, client ResourceClient, objs []*unstructured.Unstructured, opts WaitOptions) (int, error)
WaitForReady blocks until every object reports Ready or the timeout elapses. It returns the count of objects confirmed Ready and a non-nil error on timeout (ErrReadinessTimeout) or on a terminal failure (e.g. a Pod that entered phase Failed, or a zero-desired workload without the explicit opt-in).
Evidence invariant: a Get that fails (apiserver unreachable, object vanished) is a hard error, never treated as "ready". Readiness must be affirmatively observed on a live object.
Types ¶
type Bundle ¶
type Bundle struct {
// Objects in apply order: cluster-scoped prerequisites (Namespaces,
// CRDs, RBAC) first, then namespaced workloads.
Objects []*unstructured.Unstructured
}
Bundle is an ordered set of Kubernetes objects decoded from a manifest path. Objects are already sorted into apply order (see applyRank).
func LoadBundle ¶
LoadBundle reads a bundle from a path — either a single manifest file or a directory of *.yaml / *.yml / *.json files (recursively) — splits multi-document YAML, decodes each document into an unstructured object, and returns them in apply order.
An empty result is ErrEmptyBundle, never a silent success: pointing at a path that yields nothing is a failure per the evidence invariant.
type DeleteOptions ¶
type DeleteOptions struct {
// Client is the Kubernetes surface to delete against (required).
Client ResourceClient
// Timeout bounds an optional wait for the deleted objects to actually
// vanish (finalizers, garbage collection). Zero/negative skips the
// wait entirely — the objects are deleted and the call returns
// without confirming they are gone.
Timeout time.Duration
// PollInterval is how often the gone-check is re-run. Defaults to 2s.
PollInterval time.Duration
// Log receives progress lines. Defaults to no-op.
Log Logf
}
DeleteOptions controls one uninstall run.
type DeleteResult ¶
type DeleteResult struct {
// Deleted lists the objects (kind ns/name) this run removed — in
// reverse apply order.
Deleted []string
// Failed lists objects that could NOT be deleted (with the reason).
// A non-empty Failed always accompanies a non-nil error.
Failed []string
// Gone is the count confirmed absent after the wait (only meaningful
// when Options.Timeout > 0).
Gone int
}
DeleteResult reports one uninstall run.
func DeleteBundle ¶
func DeleteBundle(ctx context.Context, b *Bundle, opts DeleteOptions) (DeleteResult, error)
DeleteBundle removes every object in the bundle, in reverse apply order (dependents before the Namespace/CRDs/RBAC they depend on), best-effort across all objects — one delete failure does not stop the rest from being attempted. When Options.Timeout > 0 it then waits, bounded, for the deleted objects to actually disappear.
This is the uninstall counterpart of ApplyBundle: it reuses the exact deletion mechanics (deleteReverse) that ApplyBundle's failure path uses to roll back a partial apply, so an operator-initiated uninstall and an apply-triggered rollback can never drift in ordering or accounting shape.
Installer lifecycle (see lifecycle.go): the delete set is the bundle objects PLUS every installer's declared outputs — the actual installed product, not just the bootstrap manifest. Namespaced outputs would fall with their Namespace anyway (the explicit delete just confirms it), but cluster-scoped outputs (a ClusterRoleBinding an installer created) survive a Namespace deletion and are only removed because they are declared. Deleting an already-gone object is success, so re-running an uninstall — or uninstalling after a TTL-reaped installer — converges.
type Logf ¶
Logf is the logging sink the Applier writes progress to. Defaults to a no-op; the CLI wires it to stderr. Kept as a plain func so this package imports no logging framework.
type ObjectStatus ¶
type ObjectStatus struct {
Kind string `json:"kind"`
Namespace string `json:"namespace,omitempty"`
Name string `json:"name"`
// Exists reports whether the object is present on the cluster at all.
Exists bool `json:"exists"`
// Ready reuses the exact rollout evaluation ApplyBundle's readiness
// wait uses (evalReadiness) — a status Ready=true means the same
// thing an apply's confirmed-ready count means.
Ready bool `json:"ready"`
// Reason is a short human string: why not-ready, why not-exists, or
// the confirming detail when ready.
Reason string `json:"reason,omitempty"`
// Installer marks an object annotated outpost.dhnt.io/lifecycle=installer.
// Its absence alone never decides installed-ness — the declared
// outputs do (see the lifecycle contract in lifecycle.go).
Installer bool `json:"installer,omitempty"`
// DeclaredOutput marks a row synthesized from an installer's
// outpost.dhnt.io/installs declaration rather than decoded from the
// manifest — the durable post-install evidence being asserted.
DeclaredOutput bool `json:"declared_output,omitempty"`
}
ObjectStatus is one bundle object's live state — a pure read, no apply or wait involved.
type Options ¶
type Options struct {
// Client is the Kubernetes surface to apply against (required).
Client ResourceClient
// Timeout bounds the readiness wait for the WHOLE bundle. Must be > 0
// — a zero/negative timeout is rejected rather than treated as
// "wait forever" (an unbounded wait can mask a stuck rollout).
Timeout time.Duration
// PollInterval is how often readiness is re-checked. Defaults to 2s
// when unset.
PollInterval time.Duration
// CRDWaitTimeout bounds, per CRD, the wait for Established + the
// served types appearing in discovery before dependent custom
// resources are applied. Defaults to 60s when unset. Timing out is a
// hard failure (ErrCRDNotReady), never "apply the CR and hope".
CRDWaitTimeout time.Duration
// AllowScaleToZero is the EXPLICIT opt-in that lets a workload with
// zero desired replicas count as rolled out. Without it a
// spec.replicas: 0 workload is a terminal failure — zero desired
// satisfying "all replicas ready" is an absence-of-evidence green.
AllowScaleToZero bool
// DisableRollback keeps everything this run created in place on
// failure, instead of the default best-effort cleanup. The result
// still reports exactly what was created and left behind.
DisableRollback bool
// RollbackTimeout bounds the best-effort cleanup pass. Defaults to
// 60s. The cleanup runs on a context detached from the caller's (a
// cancelled apply must still get its bounded cleanup chance).
RollbackTimeout time.Duration
// Log receives progress lines. Defaults to no-op.
Log Logf
}
Options controls one apply run.
type ResourceClient ¶
type ResourceClient interface {
// Apply performs an idempotent server-side apply of obj (create if
// absent, reconcile if present) and returns the applied object.
Apply(ctx context.Context, obj *unstructured.Unstructured) (*unstructured.Unstructured, error)
// Get fetches the live object matching obj's GVK / namespace / name.
// A missing object is reported as an error wrapping ErrNotFound so
// the apply loop can tell "does not exist yet" (this run will create
// it) from every other failure, which stays hard.
Get(ctx context.Context, obj *unstructured.Unstructured) (*unstructured.Unstructured, error)
// Delete removes the live object matching obj's GVK / namespace /
// name. Deleting an object that is already gone is a success — the
// rollback path must converge, not fight over who removed it.
Delete(ctx context.Context, obj *unstructured.Unstructured) error
// ServesGVK reports whether the apiserver's discovery currently
// serves the given GroupVersionKind. Used to gate custom resources
// behind their CRD actually being discoverable — an Established CRD
// whose type has not reached the discovery cache still 404s.
ServesGVK(ctx context.Context, gvk schema.GroupVersionKind) (bool, error)
}
ResourceClient is the minimal Kubernetes surface bundleapply needs. It is an interface so the apply/wait logic can be driven by a deterministic in-memory fake in tests — no apiserver, no kubeconfig — while production uses the dynamic-client implementation below.
func NewDynamicClient ¶
func NewDynamicClient(kubeconfigPath string) (ResourceClient, error)
NewDynamicClient builds a ResourceClient from a kubeconfig PATH. The path is required and must exist — there is NO fallback to an in-cluster config or a default cluster (a missing kubeconfig fails loudly, per the no-silent-fallback rule).
The venue guard is enforced HERE, in the Go API: the path is canonicalized (tilde expansion, absolutization, symlink resolution) and refused when it lands on the cloudbox kubeconfig — so a caller using this package directly cannot bypass the guard the operator script also applies. A path that cannot be canonicalized is an error.
Nothing on this path touches cloudbox: it reads the kubeconfig file and dials the apiserver named in it, nothing more.
type Result ¶
type Result struct {
Applied int
Ready int
// Created lists the objects (kind ns/name) that did not exist before
// this run applied them — the rollback set. Pre-existing objects this
// run reconciled are never in here and are never deleted.
Created []string
// RolledBack lists the created objects the failure cleanup deleted.
RolledBack []string
// CleanupFailed lists created objects the cleanup could NOT delete
// (with the reason) — these were left behind and the operator must
// remove them by hand.
CleanupFailed []string
}
Result summarizes an apply run — including, on failure, the precise transactional accounting: what this run created, what the rollback removed, and what it could NOT remove. A partial apply is never left behind silently.
func ApplyBundle ¶
ApplyBundle applies every object in the bundle in order, then waits for the workloads to become Ready. It is the single top-level entry point.
Ordering: objects are applied in applyRank order, so Namespaces (and other cluster-scoped prerequisites) land before namespaced objects.
CRD gate: when the bundle ships a CustomResourceDefinition together with custom resources of the type it defines, each such CR is applied only after the CRD reports Established AND the type appears in apiserver discovery — bounded by Options.CRDWaitTimeout, hard failure on timeout. Applying immediately would race the discovery cache and fail intermittently.
Idempotence: each Apply is a server-side apply under a stable field manager, so calling ApplyBundle twice with the same bundle converges.
Transactionality: the run records which objects it CREATED (as opposed to reconciled). On any failure — apply error, CRD gate timeout, readiness timeout, terminal workload state — those created objects are deleted again in reverse apply order, best-effort and bounded, and the result + error report precisely what was and was not cleaned. Objects that existed before this run are never deleted.
Readiness: after all objects are applied, WaitForReady blocks up to Options.Timeout. A timeout returns ErrReadinessTimeout — the operator entry point turns that into a non-zero exit.
type StatusOptions ¶
type StatusOptions struct {
// Client is the Kubernetes surface to read (required).
Client ResourceClient
// AllowScaleToZero mirrors Options.AllowScaleToZero: a workload with
// spec.replicas: 0 reports Ready (scaled down on purpose) instead of
// the terminal ErrZeroDesiredWorkload reason. Without it, a
// zero-desired workload's Reason names the same terminal condition
// ApplyBundle would have failed on — status never waits, it only
// reports what a wait would have seen.
AllowScaleToZero bool
}
StatusOptions controls one status read.
type StatusResult ¶
type StatusResult struct {
Objects []ObjectStatus
// Installed is true only when every non-installer object exists on
// the cluster AND every installer's declared durable outputs exist.
// An installer's own absence never decides it (lifecycle contract).
Installed bool
// AllReady is true only when every asserted object — non-installer
// bundle objects plus declared installer outputs — exists AND reports
// Ready. A garbage-collected installer does not count against it; a
// present-but-unready (e.g. failed) installer does.
AllReady bool
}
StatusResult is the bundle-wide status snapshot.
func StatusBundle ¶
func StatusBundle(ctx context.Context, b *Bundle, opts StatusOptions) (StatusResult, error)
StatusBundle reports the live state of every object in the bundle without applying or deleting anything. It is a single pass — no polling, no bounded wait — built from the same evidence primitives ApplyBundle's readiness wait uses (Client.Get + evalReadiness), so "installed and ready" here means exactly what a successful ApplyBundle would have confirmed.
Evidence invariant: a Get failure that is NOT "object not found" (an unreachable apiserver, a permission error) is a hard failure, never reported as "not installed" — the absence of a successful read is not evidence the object is absent.
Installer lifecycle (see lifecycle.go): an object annotated outpost.dhnt.io/lifecycle=installer is judged by its DECLARED OUTPUTS, not by its own presence. A completed installer Job reaped by ttlSecondsAfterFinished does not make the bundle "not installed" as long as every declared durable output exists; conversely, a bundle is NEVER installed while any declared output is missing — a surviving Namespace/RBAC skeleton after a failed install stays installed=false.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Command bundleapply is the standalone runner behind script/dks-peer-bundle-apply.sh: it applies an app/bundle manifest set against a PEER-HOSTED control plane addressed purely by a kubeconfig path.
|
Command bundleapply is the standalone runner behind script/dks-peer-bundle-apply.sh: it applies an app/bundle manifest set against a PEER-HOSTED control plane addressed purely by a kubeconfig path. |