Documentation
¶
Overview ¶
Package credentialuse holds the Secrets, and the objects that lead to them, that a managed resource reads to reach its external system, for as long as losing them would be unrecoverable. A namespace deletion then can't wedge on a managed resource whose credentials were deleted first.
The design is docs/design-credential-use-finalizer.md, and the build follows docs/plan-credential-use-implementation.md. Section numbers (§) in this package's comments are the design's.
What is here so far ¶
This is the vocabulary the protocol is built from (plan, PR 1):
- Ref and ID, the refs a holder reads, and the record that stores them on the holder (§4);
- FinalizerName, the marker label and the epoch: the keys a hold writes on the protected object (§3, §10.1);
- Classify and ClassifyJSONPatch, which read an API error by its code, reason, message and causes, never by IsForbidden or IsInvalid alone (§6.2);
- the patch builders: an acquiring write (A2), a release (R3) and the label janitor's patch (§6.2, §10.1);
- Decide and the user predicates: §5's table and §6.1's rules, as pure functions.
Nothing here talks to an API server, and nothing in provider-runtime uses this package yet.
Dependencies ¶
The package depends on client-go and apimachinery only, never on controller-runtime: runtimes without a manager use it too (§7.1). A test checks that.
Index ¶
- Constants
- Variables
- func EncodeRecord(ids []ID) (string, error)
- func FinalizerName(holder schema.GroupKind, namespace string) (string, error)
- func FinalizerPrefix(holder schema.GroupKind) (string, error)
- func GraceEnd(deletionTimestamp, firstSeen time.Time, grace time.Duration) time.Time
- func IsCredentialUseFinalizer(f string) bool
- func IsDeletingUser(holder metav1.Object, p ID, ownerFinalizers []string) bool
- func IsEstablished(holder metav1.Object, ownerFinalizers []string) bool
- func IsMarkerLabel(key string) bool
- func IsUser(holder metav1.Object, p ID, ownerFinalizers []string) bool
- func MarkerLabel(holder schema.GroupKind) string
- func NewEpoch() (string, error)
- type Class
- type Decision
- type ID
- type Patch
- func AcquireGuardedPatch(current metav1.Object, holder schema.GroupKind, namespace, epoch string) (Patch, error)
- func AcquireStrategicPatch(holder schema.GroupKind, namespace, epoch string) (Patch, error)
- func JanitorPatch(read metav1.Object, holder schema.GroupKind) (Patch, error)
- func ReleasePatch(read metav1.Object, holder schema.GroupKind, namespace string) (Patch, error)
- func RestoreEpochPatch(read metav1.Object, holder schema.GroupKind, namespace, epoch string) (Patch, error)
- type Record
- type Ref
- type ReleaseInput
Constants ¶
const ( // AnnotationRefs is the record: the protected objects a holder reads // (§4). AnnotationRefs = Domain + "/refs.v1" // AnnotationUnheld lists the refs a holder reads that aren't held, with // why (§11.3). AnnotationUnheld = Domain + "/unheld" )
Keys on holders.
const ( // LabelHeld scopes the protected-object caches: an object carries it // while any kind holds it (§10.1). LabelHeld = Domain + "/held" // AnnotationAllowHoldersFrom is a target's consent to be held by holders // in other namespaces (§11.3). AnnotationAllowHoldersFrom = Domain + "/allow-holders-from" )
Keys on protected objects and their Namespaces.
const ( // AnnotationDisown is the Disown switch (§7.4). AnnotationDisown = Domain + "/disown" // AnnotationHeldCount is Disown's live count (§7.4). AnnotationHeldCount = Domain + "/held-count" // AnnotationDisownSettled is set once Disown has finished (§7.4). AnnotationDisownSettled = Domain + "/disown-settled" // AnnotationConfig holds the settings every process of a kind shares // (§7.3). AnnotationConfig = Domain + "/config" // AnnotationIndex locates a kind's index in PerName and PerNamePoll // modes (§10.6.1). AnnotationIndex = Domain + "/index" )
Keys on a holder kind's CustomResourceDefinition.
const ClusterScopedName = "cluster-scoped"
ClusterScopedName is the name part of a cluster-scoped holder kind's finalizer (§3). A GroupKind is either namespaced or cluster-scoped, so it can't collide with a namespace of that name.
const Domain = "credential-use.krateo.io"
Domain is the prefix of every label and annotation key this package owns.
Variables ¶
var ( // ErrNotHeld: the object doesn't carry the finalizer a release would // remove, so it is already released (§6.2 R1). ErrNotHeld = errors.New("credential-use: the object doesn't carry the finalizer") // ErrNoEpoch: the object carries the finalizer but not its epoch. Only a // third party causes that, and R1 then sets a fresh epoch and requeues. ErrNoEpoch = errors.New("credential-use: the object carries the finalizer but no epoch") // ErrNothingToClean: the janitor found no label to remove. ErrNothingToClean = errors.New("credential-use: no label to remove") )
Functions ¶
func EncodeRecord ¶
EncodeRecord returns the AnnotationRefs value for ids: a JSON array, sorted and deduplicated by identity (§4). No refs encode as "[]".
func FinalizerName ¶
FinalizerName returns the finalizer of holder kind K in holder namespace N, credential-use.<lowercase kind>.<group>/<namespace> (§3, U6). An empty namespace means a cluster-scoped holder kind, whose name part is ClusterScopedName. The same string is the key of the epoch annotation.
Only a pathological group makes the name too long; setup refuses to start then (§7.3).
func FinalizerPrefix ¶
FinalizerPrefix returns the prefix every finalizer of holder kind K carries, credential-use.<lowercase kind>.<group>/ (§3).
func GraceEnd ¶
GraceEnd returns when the grace for a protected object being deleted runs out: grace after the later of its deletionTimestamp and the time this process first saw that timestamp (§5, L4). So neither a server clock behind the local one nor a late delivery can shorten it.
firstSeen is a local time.Now() reading; the comparison with the server's timestamp is on the wall clock.
func IsCredentialUseFinalizer ¶
IsCredentialUseFinalizer reports whether f is a credential-use finalizer of any holder kind. The held label goes with the last of them (§6.2 R3).
func IsDeletingUser ¶
IsDeletingUser reports whether holder is a user of p that is being deleted: it has a deletionTimestamp and still carries an owner finalizer (§6.1).
func IsEstablished ¶
IsEstablished reports whether holder carries any of the owner finalizers: the runtime's own finalizer, which it writes only after the first Connect and Observe succeed (§2, §7.1). An established holder fails open on an Acquire error; a new one fails closed (§8).
func IsMarkerLabel ¶
IsMarkerLabel reports whether key is some holder kind's marker label.
func IsUser ¶
IsUser reports whether holder, which exists, uses protected object p (§6.1): its record names p, or can't be parsed, and it is not being deleted without an owner finalizer. A holder that is being deleted and lacks every owner finalizer is excluded: the runtime removes the record in the same write as its owner finalizer, and the exclusion covers every other way a stale record is left behind, and breaks foreground-deletion cycles (§11.8).
func MarkerLabel ¶
MarkerLabel returns holder kind K's marker label key, credential-use.krateo.io/k-<hash>, where <hash> is the first 16 hex digits of the SHA-256 of <lowercase kind>.<group> (§10.1, L3). It lets one kind's holds be selected. A collision is harmless: every reader also checks the finalizer prefix.
Types ¶
type Class ¶
type Class int
A Class is what an API error means to the protocol (§6.2). It is read from the status's code, reason, message and causes, never from IsForbidden or IsInvalid alone, because two different 422s and three different 403s exist.
const ( // ClassNone is a nil error. ClassNone Class = iota // ClassOther is any error none of the classes below describes, // including errors that carry no API status. ClassOther // ClassDeleting, class (a): 422 Invalid with a FieldValueForbidden cause // on metadata.finalizers, whose message is ValidateNoNewFinalizers'. P is being deleted and lacks the finalizer, so // no new finalizer can be added. The only class that is the Deleting // outcome. ClassDeleting // ClassPolicyDenied, class (b): a ValidatingAdmissionPolicy refused the // write. A 403 when the validation sets reason Forbidden, as ours does // (§10.6.1), and a 422 when it sets none. ClassPolicyDenied // ClassPolicyConfig, class (c): a policy or its binding couldn't be // configured, for example a missing parameter object under // parameterNotFoundAction Deny. Always a 422, and never Deleting. ClassPolicyConfig // ClassRBAC, class (d): the authorizer refused the request. Recognised // by the authorizer's own message (IP1). The only class PerNamePoll // retries as RBAC propagation (§10.7.2). ClassRBAC // ClassAdmissionDenied, class (e): any other 403, such as a webhook // denial, NamespaceLifecycle refusing a create in a terminating // namespace, or a 403 whose reason isn't Forbidden. An Acquire error at // once (IP1), and counted, so an RBAC message the pattern stops matching // shows up (L3 of the review of revision 16). ClassAdmissionDenied // ClassNotFound: the object doesn't exist (the Missing outcome of A2). ClassNotFound // ClassConflict: a resourceVersion precondition failed. ClassConflict // ClassAlreadyExists: a create found the object there already. ClassAlreadyExists // ClassStateChanged: a JSON patch's test op failed, or one of its ops // didn't apply, because the object changed since it was read. Returned // only by ClassifyJSONPatch (§6.2, "JSON patches are classified apart"). ClassStateChanged )
func Classify ¶
Classify returns the class of an error from an acquiring write (A2), an index write (§10.6) or a live read in the consent gate (§6.2). Errors from JSON patches (R3, Disown's strips, the label janitor) go through ClassifyJSONPatch instead.
func ClassifyJSONPatch ¶
ClassifyJSONPatch returns the class of an error from a JSON patch with test ops: R3, Disown's strips and the label janitor's patch.
A failed test op is a 422 Invalid with the generic message and no causes: the patch handler builds it with NewGenericServerResponse, which drops json-patch's own text (k8s.io/apiserver@v0.37.2 pkg/endpoints/handlers/patch.go:423-426). A remove at an index that has moved looks the same. Both mean the object changed since it was read, so such a 422, with exactly that message, is ClassStateChanged; any other causeless error is ClassOther. A 422 that carries a policy's text is ClassPolicyDenied or ClassPolicyConfig, and is never read as a state change. The Deleting outcome doesn't apply to a patch that only removes finalizers, so ClassDeleting is never returned.
type Decision ¶
type Decision int
A Decision is what §5's table says to do with one finalizer of K for N on a protected object.
func Decide ¶
func Decide(in ReleaseInput) Decision
Decide applies §5's table (U5):
P not being deleted: kept while any user exists
P being deleted, no user at all: released at once
P being deleted, its namespace deleting: kept while any user exists
P being deleted, its namespace live: the grace runs; after it, kept
while any user is being deleted
A cluster-scoped P has no namespace and takes the last row.
type ID ¶
type ID struct {
Group string `json:"group"`
Kind string `json:"kind"`
Namespace string `json:"namespace"`
Name string `json:"name"`
}
An ID is the identity of a protected object: its group, kind, namespace and name. Two keys of one Secret are one ID. The JSON form is the record's (§4), so every field is always present, the empty group included.
func SortedIDs ¶
SortedIDs returns ids sorted and deduplicated, as the record keeps them (§4). It doesn't modify ids.
type Patch ¶
A Patch is a patch body and the type to send it as.
func AcquireGuardedPatch ¶
func AcquireGuardedPatch(current metav1.Object, holder schema.GroupKind, namespace, epoch string) (Patch, error)
AcquireGuardedPatch builds A2 for a custom resource (§6.2) from the object a live GET just returned: a merge patch guarded by its resourceVersion that writes the whole finalizer list with the finalizer added, the epoch and both labels. A merge patch replaces the list, so the guard is what keeps another writer's change from being lost: a conflict GETs and retries.
func AcquireStrategicPatch ¶
AcquireStrategicPatch builds A2 for a built-in kind such as a Secret or a ConfigMap (§6.2): one strategic merge patch, sent with no precondition. It merges the finalizer into metadata.finalizers as a set, sets the epoch, and sets the held label and K's marker label. Finalizers are tagged patchStrategy:"merge", so the patch can't drop anyone else's, and never adds a duplicate. It does deduplicate the list: an entry someone else had twice comes out once.
Custom resources don't accept strategic merge patches: use AcquireGuardedPatch for them.
func JanitorPatch ¶
JanitorPatch builds the label janitor's patch for holder kind K (§10.1): on an object a third party stripped of its finalizers but not its labels, it removes K's marker label when no finalizer of K is left, and the held label when no credential-use finalizer of any kind is left. It tests the object's resourceVersion as read, which implies the design's test that the finalizer array is unchanged, so it can't race a new acquiring write: any write in between is ClassStateChanged.
It returns ErrNothingToClean when no label is to be removed.
func ReleasePatch ¶
ReleasePatch builds R3 for holder kind K in holder namespace N, from the object as R1 read it (§6.2):
test metadata.uid is the incarnation R1 read test the epoch annotation is the one R1 read test metadata.finalizers[i] is the finalizer test metadata.finalizers is the whole array R1 read remove metadata.finalizers[i] remove the epoch annotation
When the finalizer is K's last on the object, the patch also removes K's marker label, and when it is the last credential-use finalizer of any kind, the held label, each tested first with its value as read. A label that is already absent is left alone, so that its remove can't fail. So the patch tests every value it removes, and applies to exactly the state it was built from.
The whole array is always tested (L1 of the review of #53, the user's decision), not only when a label goes. Then no two releasers can both apply from one read: say K's releasers for namespaces a and b each read [Fa, Fb], so neither is last, and both remove their finalizer and leave the labels. With the array tested, the second fails, re-reads [Fb], is last, and removes the labels. The cost is a retry whenever the array changed since R1, for any reason; finalizer arrays change rarely, and the retries are bounded (§6.2, "JSON patches are classified apart").
The epoch test is what makes a concurrent A2 break the release: every A2 writes a fresh epoch (§6.4). A failed test is ClassStateChanged (ClassifyJSONPatch), and the releaser goes back to R1.
It returns ErrNotHeld when the object lacks the finalizer, and ErrNoEpoch when it has the finalizer but no epoch, or an empty one.
func RestoreEpochPatch ¶
func RestoreEpochPatch(read metav1.Object, holder schema.GroupKind, namespace, epoch string) (Patch, error)
RestoreEpochPatch builds R1's repair for an object that carries holder kind K's finalizer for namespace N but not its epoch (§6.2 R1): it sets a fresh epoch. It tests the object's resourceVersion as read, so it lands only on the state that was read: any change in between is ClassStateChanged, and R1 reads again.
type Record ¶
type Record struct {
// Present reports whether the holder carries the annotation at all.
Present bool
// Unparseable reports a present annotation that can't be read. Such a
// holder counts as a user of every protected object its namespace's
// check considers: it fails safe (§4).
Unparseable bool
// IDs are the recorded refs, sorted and deduplicated. Empty when the
// record is absent or unparseable.
IDs []ID
}
A Record is what a holder's AnnotationRefs says: the protected objects it reads (§4). Release reads only records, never RefsFunc.
func ReadRecord ¶
ReadRecord reads obj's record.
A record is read strictly, and anything it can't read exactly fails safe: the holder then counts as a user of everything (§4). A record is a JSON array of objects, each with exactly the four keys group, kind, namespace and name, spelled exactly, each once, each a JSON string, kind and name non-empty, and nothing after the array. So a missing or null field, which a lenient decoder would read as an empty one and so as another object (a missing namespace reads as cluster-scoped), is unparseable. A record of a later format goes under a new key (refs.v2), so an unknown key here is damage, not a newer format.
type Ref ¶
type Ref struct {
GroupKind schema.GroupKind // {"" "Secret"} for a Secret
Namespace string // empty for a cluster-scoped object
Name string
// Next, when set, names the refs found inside this object. The tracker
// calls it after the object is held, with the object it read (§9).
Next func(obj *unstructured.Unstructured) ([]Ref, error)
// Guaranteed declares that the object's own controller already keeps it
// alive while the holder exists (§9.3). It is neither recorded nor held;
// its Next is still followed.
Guaranteed bool
}
A Ref names an object a holder reads to reach its external system: a Secret, or an intermediate object that leads to one (§9).
type ReleaseInput ¶
type ReleaseInput struct {
// Deleting: P has a deletionTimestamp.
Deleting bool
// ClusterScoped: P has no namespace, so it always takes the last row.
ClusterScoped bool
// NamespaceDeleting: P's Namespace has a deletionTimestamp, read live.
NamespaceDeleting bool
// GraceOver: the grace has run out (GraceEnd).
GraceOver bool
// Users and DeletingUsers count the holders of K in N that use P, and
// those of them being deleted (§6.1).
Users, DeletingUsers int
}
ReleaseInput is what §5's table reads for one protected object P and one (K, N). Every field comes from the last reads before the release patch: R1's live GET of P, R2's live GET of P's Namespace and R2's consistent list of K in N (§5, §6.2).