Documentation
¶
Overview ¶
Package applyset provides server-side apply and pruning with ApplySet membership tracking. This is a simplified implementation based on KEP-3659. https://git.k8s.io/enhancements/keps/sig-cli/3659-kubectl-apply-prune
Package applyset implements KEP-3659 ApplySet for kro's instance controller.
Responsibilities ¶
ApplySet: server-side apply resources, label them for membership, return metadata. Controller: convert metadata to labels/annotations, call Prune separately, decide which metadata to write based on prune outcome.
Workflow ¶
1. Project() computes union metadata (batch + parent annotations). Error = requeue. 2. Controller patches parent with union metadata (before apply/prune) 3. Apply() runs SSA and returns batch-only metadata 4. Prune() deletes orphans using scope from Project().PruneScope() 5. If prune completes, controller shrinks parent annotations to batch metadata
ApplySet is stateless - all methods are pure functions of their inputs.
Metadata Semantics ¶
Apply() returns batch-only metadata: just the GKs and namespaces of this batch. Project() returns union metadata: batch + parent annotations (the "memory"). Controller chooses which to write based on prune outcome.
Prune Safety ¶
IMPORTANT: Prune only runs if ALL applies succeeded. If any apply failed, the failed resource's UID won't be in keepUIDs, so pruning would delete a potentially working resource. Use Errors() to check safety.
SkipApply ¶
Resources with SkipApply=true are NOT applied (no SSA call). They are also excluded from the current reconcile's GK/namespace set. However, if the resource was previously applied, its GK remains in the parent annotation ("memory"), and prune will find and delete it. This is how includeWhen=false resources get cleaned up: they were applied before, now they're skipped, and the parent annotation provides prune scope from prior reconciles.
ApplySet ID ¶
Computed from parent GKNN: applyset-<base64(sha256(name.namespace.kind.group))>-v1 Stable across reconciles, unique per instance.
Parent annotations (controller applies from Metadata) ¶
- applyset.kubernetes.io/tooling: identifies kro as the manager
- applyset.kubernetes.io/contains-group-kinds: GKs of managed resources (shrinks/expands dynamically)
- applyset.kubernetes.io/additional-namespaces: namespaces where managed resources live
Child labels (applied during SSA) ¶
- applyset.kubernetes.io/part-of: links child to parent's ApplySet ID
Index ¶
- Constants
- Variables
- func ID(parent interface{ ... }) string
- func ToolingID() string
- type ApplyMode
- type ApplyResult
- type ApplyResultItem
- type ApplySet
- func (a *ApplySet) Apply(ctx context.Context, resources []Resource, mode ApplyMode) (*ApplyResult, Metadata, error)
- func (a *ApplySet) DeleteOrphan(ctx context.Context, candidate OrphanCandidate) (DeleteOrphanResult, error)
- func (a *ApplySet) ListOrphans(ctx context.Context, opts PruneOptions) ([]OrphanCandidate, error)
- func (a *ApplySet) Project(resources []Resource) (Metadata, error)
- type ApplySetConflictError
- type Config
- type DeleteOrphanResult
- type Interface
- type Metadata
- type OrphanCandidate
- type PruneOptions
- type PruneResultItem
- type PruneScope
- type Resource
Constants ¶
const ( // ApplySetToolingAnnotation is the key of the label that indicates which tool is used to manage this ApplySet. // Tooling should refuse to mutate ApplySets belonging to other tools. // The value must be in the format <toolname>/<semver>. // Example value: "kubectl/v1.27" or "helm/v3" or "kro/v1.0.0" ApplySetToolingAnnotation = "applyset.kubernetes.io/tooling" // ApplySetAdditionalNamespacesAnnotation lists namespaces beyond the parent's own namespace. // The parent namespace is implicitly included and must NOT be listed here. // Value: comma-separated namespace names, or empty if all resources are in parent namespace. // Example: "kube-system,ns1,ns2" (parent namespace is NOT listed). ApplySetAdditionalNamespacesAnnotation = "applyset.kubernetes.io/additional-namespaces" // ApplySetGKsAnnotation is the standard KEP annotation for group-kinds. // Format: comma-separated "Kind.group" entries (Kind only for core resources). // Example value: "ConfigMap,Deployment.apps,Service" ApplySetGKsAnnotation = "applyset.kubernetes.io/contains-group-kinds" // ApplySetParentIDLabel is the key of the label that makes object an ApplySet parent object. // Its value MUST use the format specified in V1ApplySetIdFormat below. ApplySetParentIDLabel = "applyset.kubernetes.io/id" // V1ApplySetIdFormat is the format required for the value of ApplySetParentIDLabel (and ApplysetPartOfLabel). // The %s segment is the unique ID of the object itself, which MUST be the base64 encoding // (using the URL safe encoding of RFC4648) of the hash of the GKNN of the object it is on, in the form: // base64(sha256(<name>.<namespace>.<kind>.<group>)). V1ApplySetIdFormat = "applyset-%s-v1" // ApplySetIDPartDelimiter is the delimiter used to separate the parts of the ApplySet ID. ApplySetIDPartDelimiter = "." // ApplysetPartOfLabel is the key of the label which indicates that the object is a member of an ApplySet. // The value of the label MUST match the value of ApplySetParentIDLabel on the parent object. ApplysetPartOfLabel = "applyset.kubernetes.io/part-of" )
Label and annotation keys from the ApplySet specification. https://git.k8s.io/enhancements/keps/sig-cli/3659-kubectl-apply-prune#design-details-applyset-specification
const (
// FieldManager is the field manager name used for server-side apply.
FieldManager = "kro.run/applyset"
)
Internal constants for ApplySet implementation.
Variables ¶
var ErrApplySetConflict = errors.New("resource belongs to a different ApplySet")
ErrApplySetConflict is returned when a resource already belongs to a different ApplySet. This indicates the resource is managed by another controller/instance and should not be overwritten without explicit action.
var ErrDuplicateResource = errors.New("found resources with conflicts")
ErrDuplicateResource is returned when multiple resource IDs target the same Kubernetes object.
Functions ¶
Types ¶
type ApplyMode ¶
type ApplyMode struct {
Concurrency int // 0 = len(resources)
}
ApplyMode controls Apply behavior.
type ApplyResult ¶
type ApplyResult struct {
Applied []ApplyResultItem
}
ApplyResult contains outcomes for all resources.
func (*ApplyResult) ByID ¶
func (r *ApplyResult) ByID() map[string]ApplyResultItem
ByID returns a map of results keyed by resource ID for easy lookup.
func (*ApplyResult) Errors ¶
func (r *ApplyResult) Errors() error
Errors returns combined errors from apply operations, or nil if none. Use this to check if it's safe to prune (don't prune if applies failed).
func (*ApplyResult) HasClusterMutation ¶
func (r *ApplyResult) HasClusterMutation() bool
HasClusterMutation returns true if any apply operation changed the cluster.
func (*ApplyResult) ObservedUIDs ¶
func (r *ApplyResult) ObservedUIDs() sets.Set[types.UID]
ObservedUIDs returns the UIDs of all successfully applied resources.
type ApplyResultItem ¶
type ApplyResultItem struct {
ID string // same as input Resource.ID
Desired *unstructured.Unstructured // what we sent
Observed *unstructured.Unstructured // cluster state after apply (nil if error)
Changed bool // resourceVersion changed
Error error
}
ApplyResultItem is the outcome of applying a single resource.
type ApplySet ¶
type ApplySet struct {
// contains filtered or unexported fields
}
ApplySet implements Interface for server-side apply with membership tracking.
func New ¶
New creates an ApplySet for a specific parent (instance). Parent GKNN (name, namespace, kind, group) is used to generate the ApplySet ID per KEP-3659. Namespaces for pruning are derived from resources passed to Apply.
func (*ApplySet) Apply ¶
func (a *ApplySet) Apply(ctx context.Context, resources []Resource, mode ApplyMode) (*ApplyResult, Metadata, error)
Apply runs SSA on all resources and returns batch-only metadata. Caller should call Prune separately after Apply succeeds.
func (*ApplySet) DeleteOrphan ¶ added in v0.9.2
func (a *ApplySet) DeleteOrphan(ctx context.Context, candidate OrphanCandidate) (DeleteOrphanResult, error)
DeleteOrphan deletes a single orphan candidate using a UID precondition to avoid deleting a resource that was recreated since listing.
func (*ApplySet) ListOrphans ¶ added in v0.9.2
func (a *ApplySet) ListOrphans(ctx context.Context, opts PruneOptions) ([]OrphanCandidate, error)
ListOrphans discovers orphaned resources (applyset members not in KeepUIDs) without deleting them. The caller can order the returned candidates before issuing deletes via DeleteOrphan.
type ApplySetConflictError ¶
type ApplySetConflictError struct {
ResourceName string
ResourceNamespace string
ResourceGVK string
CurrentApplySetID string
DesiredApplySetID string
}
ApplySetConflictError provides details about an ApplySet membership conflict.
func (*ApplySetConflictError) Error ¶
func (e *ApplySetConflictError) Error() string
func (*ApplySetConflictError) Unwrap ¶
func (e *ApplySetConflictError) Unwrap() error
type Config ¶
type Config struct {
Client dynamic.Interface
RESTMapper meta.RESTMapper
Log logr.Logger
ParentNamespace string // fallback namespace for namespaced resources without namespace set
}
Config for creating an ApplySet.
type DeleteOrphanResult ¶ added in v0.9.2
type DeleteOrphanResult struct {
// Pruned is non-nil when the delete was accepted by the API server.
Pruned *PruneResultItem
// Conflict is true when the UID precondition failed (resource was recreated).
Conflict bool
}
DeleteOrphanResult describes the outcome of a single orphan deletion.
type Interface ¶
type Interface interface {
// Project computes metadata as union of current resources + parent annotations.
// Returns GKs/namespaces from both batch AND parent memory (for prune scope).
// Returns error if RESTMapping fails for any resource.
Project(resources []Resource) (Metadata, error)
// Apply runs SSA on resources and returns batch-only metadata.
// Batch metadata contains only GKs/namespaces from THIS apply (not parent memory).
Apply(ctx context.Context, resources []Resource, mode ApplyMode) (*ApplyResult, Metadata, error)
// ListOrphans discovers orphaned resources without deleting them.
// Returns candidates that the caller can order (e.g. reverse-topological)
// before issuing deletes via DeleteOrphan.
ListOrphans(ctx context.Context, opts PruneOptions) ([]OrphanCandidate, error)
// DeleteOrphan deletes a single orphan candidate using a UID precondition.
DeleteOrphan(ctx context.Context, candidate OrphanCandidate) (DeleteOrphanResult, error)
}
Interface defines server-side apply and pruning operations with KEP-3659 ApplySet membership tracking. It is implemented by ApplySet struct and is intentionally stateless: each method is a pure function of its inputs, with no internal state carried between calls. The caller is responsible for orchestrating the workflow correctly: calling Project() to get union metadata, patching the parent object with that metadata before apply/prune, passing the correct PruneScope from Project() to Prune(), and shrinking parent annotations after successful prune. Hopefully this design will make it easier for callers to implement custom workflows around the core functionality.
type Metadata ¶
type Metadata struct {
ID string
Tooling string
GroupKinds sets.Set[schema.GroupKind]
AdditionalNamespaces sets.Set[string] // excludes parent namespace per KEP-3659
}
Metadata contains the computed ApplySet state. Controller decides how to store it (annotations, labels, status, etc).
func (Metadata) Annotations ¶
Annotations returns the KEP-3659 parent annotations.
func (Metadata) GroupKindsString ¶
GroupKindsString returns GKs as comma-separated "Kind.group" for KEP-3659 annotation.
func (Metadata) NamespacesString ¶
NamespacesString returns namespaces as comma-separated for KEP-3659 annotation.
func (Metadata) PruneScope ¶
func (m Metadata) PruneScope() *PruneScope
PruneScope returns a PruneScope from this Metadata for use with Prune(). Note: This only includes AdditionalNamespaces. Prune() will fall back to parent namespace if the scope is empty.
type OrphanCandidate ¶ added in v0.9.2
type OrphanCandidate struct {
Object *unstructured.Unstructured
GVR schema.GroupVersionResource
}
OrphanCandidate is a resource discovered during orphan listing that is a member of the applyset but not in the keep set. Callers use this to apply ordering (e.g. reverse-topological) before issuing deletes.
type PruneOptions ¶
type PruneOptions struct {
// KeepUIDs are UIDs of resources that should NOT be pruned.
// Typically from ApplyResult.ObservedUIDs().
KeepUIDs sets.Set[types.UID]
// Scope defines GKs and namespaces to prune from (required).
// Use Metadata.PruneScope() to get the scope from Project() output.
// Pass the superset scope (union of batch + parent) to ensure
// prune finds all orphans.
Scope *PruneScope
}
PruneOptions controls ListOrphans behavior.
type PruneResultItem ¶
type PruneResultItem struct {
Object *unstructured.Unstructured
}
PruneResultItem is a successfully pruned resource.
type PruneScope ¶
type PruneScope struct {
GroupKinds sets.Set[schema.GroupKind]
Namespaces sets.Set[string] // required for namespace-scoped RBAC compatibility
}
PruneScope defines the search space for orphan detection.
type Resource ¶
type Resource struct {
// ID is a stable identifier provided by the controller (e.g., "deployment" or "workers-0").
ID string
// Object is the desired state to apply (GVK/ns/name must be set correctly by the caller).
Object *unstructured.Unstructured
// Current optionally carries the live object fetched by the controller.
// This allows conflict detection without extra GETs inside Apply().
Current *unstructured.Unstructured
// SkipApply excludes the resource from SSA and from the current GK/namespace set.
// Prune relies on the parent annotation "memory" from previous reconciles to
// delete these resources if they were previously applied. Use for includeWhen=false.
SkipApply bool
}
Resource is an input to Apply.