Documentation
¶
Overview ¶
Package registryprune implements in-use-aware garbage collection of the fleet's container registry.
Index ¶
- Constants
- Variables
- func Deletable(c Class, p Policy) bool
- func ValidateRegistryHost(raw string) error
- type BlobGC
- type Class
- type Config
- type Handler
- type HostEnumerator
- type InUseSet
- type InventorySource
- type ManifestResolver
- type Metrics
- type Payload
- type PodRunner
- type Policy
- type Result
- type Scheduler
- type SpecSource
Constants ¶
const JobKind = "registry-prune"
JobKind is this handler's key in the job registry.
const TickInterval = time.Minute
TickInterval is how often the scheduler re-evaluates whether a run is due. It is the granularity of the interval and backoff gates, not the run cadence itself (that is Scheduler.Interval).
Variables ¶
var ErrRunInFlight = errors.New("a registry prune run is already queued or running")
ErrRunInFlight is returned by EnqueueNow when a run is already queued or running. Two concurrent runs would each classify the whole catalog from a listing the other is mutating.
var ErrUnsafeToPrune = errors.New("unsafe to prune")
ErrUnsafeToPrune aborts a prune run. Every ambiguity in the in-use computation resolves to this error rather than to a smaller protected set: a digest wrongly believed unused is deleted forever and takes an instance's recreatability with it (#64 — 11 of 24 live Engine instances lost the manifests they would have needed to be rebuilt). A run that refuses to start costs a cycle; a run that under-protects costs a customer.
Functions ¶
func Deletable ¶
Deletable reports whether a Class is eligible for deletion under p. Deletable classes are exactly ClassShaOrphan and ClassFeatStale, plus ClassUnclassified only when p.DeleteUnrecognised is set (the #66 fix). Nothing else is ever deletable — this function, not Classify's caller, is the single place that decision is made.
func ValidateRegistryHost ¶
ValidateRegistryHost reports whether raw is a spelling BuildInUseSet will accept, so a caller can fail at startup instead of discovering it a whole interval later. It is the same check BuildInUseSet applies; nothing here is a second, looser opinion.
Types ¶
type BlobGC ¶
type BlobGC struct {
Podman PodRunner
// HostID is the host carrying the registry.
HostID string
// RegistryPod is the podman pod name of the managed registry instance.
// Lifecycle is native podman, not systemd: podman-api speaks libpod over
// an SSH tunnel and has no shell on the far end, and the quadlet's
// Restart=always would have systemd restart the registry mid-GC anyway.
RegistryPod string
// RegistryContainer is the container to measure storage size in. Empty
// disables sizing; sizing never fails the job either way.
RegistryContainer string
// StoragePath is the registry storage directory ON THE HOST, hostPath-
// mounted into the GC pod. Must be absolute.
StoragePath string
// MountPath is where StoragePath is mounted inside the GC POD. It is not
// where the registry container mounts it — see SizePath, which is measured
// in a different container and is deliberately a separate field. The two
// coincide only because both default to /var/lib/registry.
MountPath string
// SizePath is the storage directory as seen inside the REGISTRY container
// (its `rootdirectory`), used only for `du`. Defaults to
// /var/lib/registry — NOT to MountPath, which would silently re-couple
// the two and make sizing measure a path that does not exist in the
// registry container the moment MountPath is configured.
SizePath string
// ConfigPath, Image, PodName and Timeout all default; see the default*
// constants.
ConfigPath string
Image string
// PodName is the one-shot GC pod's name. It must NOT be the registry's own
// pod name; see validate.
PodName string
Timeout time.Duration
}
BlobGC is Stage B: reclaiming the blobs that Stage A's manifest deletions only unlinked.
Deleting a manifest through the registry API unlinks it; the bytes stay on disk until the registry's own `garbage-collect` runs, and that must not run against a live registry (a push mid-GC can have its freshly-written blob swept). So the sequence is stop → GC → start, and the start is deferred.
func (*BlobGC) Reclaim ¶
func (g *BlobGC) Reclaim(ctx context.Context, jc *jobs.JobContext, p Payload) (res Result, err error)
Reclaim performs Stage B. It is a no-op (and never touches the registry) on a dry run or when the payload sets SkipBlobGC.
The named returns are load-bearing: the deferred restart runs during a panic unwind, fills in the after-size, AND can turn "GC succeeded but the registry did not come back" into a job failure.
type Class ¶
type Class string
Class is the outcome of classifying one digest (an imgregistry.TagGroup — every tag currently pointing at that digest, within one repo) against a Policy. Deletion happens by digest, so a TagGroup is the unit of classification: one class per digest, covering every tag that shares it.
const ( // ClassProtected: at least one tag in the group is an exact name from // Policy.ProtectedExact (e.g. "latest", "main"). ClassProtected Class = "protected" // ClassCalVer: at least one tag matches Policy.CalVer. ClassCalVer Class = "calver" // ClassRepoProtected: at least one tag matches this repo's entry in // Policy.ExtraPerRepo. ClassRepoProtected Class = "repo-protected" // its digest is — the same content is also reachable through a protected // tag (in this repo or another; digests are shared across repos, and // InUseSet/protectedDigests are keyed that way deliberately). ClassSharesProtected Class = "shares-protected" // ClassInUse: the digest is in the fleet-wide InUseSet (running somewhere, // or pinned by a stored spec). Checked before every delete rule, so a // dry-run's report of "why did this survive" is never wrong. ClassInUse Class = "in-use" // ClassFeatRecent: a "feat-*" tag younger than the retention window. ClassFeatRecent Class = "feat-recent" // ClassShaIsLatest: a bare-hex tag sharing its digest with the repo's own // "latest" tag (both land in the same TagGroup, since TagGroup already // groups by digest). Distinguished from ClassProtected so a report reads // "this is latest's own digest, under an extra alias" rather than the // generic "protected" — same disposition (kept), more specific reason. ClassShaIsLatest Class = "sha-is-latest" // ClassUnclassified: no rule recognises any tag in the group. KEPT by // default — see the #66 comment on Deletable. ClassUnclassified Class = "unclassified" // ClassShaOrphan: every tag in the group is bare-hex-form and nothing // above protects it. Deletable. ClassShaOrphan Class = "sha-orphan" // ClassFeatStale: a "feat-*" tag at or past the retention window (or of // unknown age — see the comment in Classify). Deletable. ClassFeatStale Class = "feat-stale" )
func Classify ¶
func Classify(repo string, tg imgregistry.TagGroup, inUse InUseSet, protectedDigests map[string]struct{}, p Policy, now time.Time) Class
Classify decides the Class of one digest (tg) within repo, given the fleet-wide in-use set, a precomputed set of digests already known to be protected (e.g. via a protected tag in another repo — digests are shared across repos, so protection must be too), the policy, and the current time (a parameter, not time.Now, so this stays a pure function callers can test exhaustively).
PRECONDITION — the caller MUST invoke this once per digest, i.e. once per imgregistry.Tags() entry (a TagGroup already carries every tag sharing that digest), and never once per individual tag. ClassShaIsLatest depends on this: it fires only because a bare-hex tag sharing digest with the repo's own "latest" arrives in tg.Tags alongside "latest" itself. A caller that flattens tags and calls Classify per-tag will never observe that combination — "latest" and its hex alias would be classified separately — so ClassShaIsLatest becomes permanently unreachable. That is not unsafe (the hex tag still falls through to ClassProtected and stays kept), but it is a silent behaviour loss: fix the call site, not this function, if that happens.
Precedence, first match wins:
- in-use — checked before every delete rule (KEPT), so a dry-run's reported reason for survival is never wrong.
- sha-is-latest — a bare-hex tag sharing digest with this repo's own "latest" (both are in tg.Tags, since TagGroup already groups by digest). More specific than the generic "protected" below.
- protected (exact name match)
- calver
- repo-protected (per-repo extra)
- shares-protected (digest, not name, matches a protected tag elsewhere)
- feat-* aging rules (feat-recent / feat-stale)
- sha-orphan — every tag in the group is bare-hex and nothing above fired
- unclassified — the default, KEPT unless Policy.DeleteUnrecognised
type Config ¶
type Config struct {
// RegistryHosts are every spelling of the registry being pruned as it may
// appear in an image reference (e.g. "reg.example:5000", "reg.example",
// "100.64.0.23:5000"). They are what separate our refs from foreign ones;
// an empty list aborts, because without one a host-qualified foreign ref
// cannot be told from one of ours.
RegistryHosts []string
// MaxSnapshotAge bounds how stale a host's inventory snapshot may be. Must
// be positive — a zero value is treated as unconfigured and aborts, rather
// than silently disabling the staleness check.
MaxSnapshotAge time.Duration
// Now is a test seam; defaults to time.Now.
Now func() time.Time
}
Config parameterises BuildInUseSet.
type Handler ¶
type Handler struct {
// Registry is the full client, including Delete. Note that the in-use
// pass is handed only the read-only ManifestResolver slice of it, so no
// bug in the protection computation can reach a delete.
Registry imgregistry.Client
Hosts HostEnumerator
Inventory InventorySource
Specs SpecSource
Config Config
Metrics Metrics // optional
// BlobGC is Stage B (blob reclamation). nil disables it entirely, which is
// Stage A only: manifests unlinked, blobs left recoverable. See blobgc.go.
BlobGC *BlobGC
// contains filtered or unexported fields
}
Handler implements jobs.Handler for the "registry-prune" kind.
func (*Handler) Run ¶
Run executes one prune. The order is non-negotiable:
- fetch the catalog ONCE,
- build the in-use set from it,
- classify EVERY repo,
- apply the per-repo tripwire,
- only then delete.
Step 3 preceding step 5 is the #64 fix: a per-repo delete-as-you-go reclaims blobs still needed by a repo that has not been classified yet.
type HostEnumerator ¶
HostEnumerator yields the fleet's CURRENT host set. BuildInUseSet takes this rather than a []string because a partial host list under-protects silently: the hosts that were passed still produce a non-zero fleet-wide count, so no fail-closed rule fires, and every digest pinned only on a missing host is deleted. That is the #64 failure mode through a door the rules do not watch.
INVARIANT: an implementation must return every configured host or fail. It must be the same live view the server reloads on SIGHUP (*instance.Service satisfies it), never a snapshot captured by the caller.
type InUseSet ¶
type InUseSet struct {
// Valid is true only on a set BuildInUseSet returned without error. The
// zero value, and every aborted return, is false. Check it before acting:
// an aborted set answers Has() false for every digest in the registry, so
// a caller that logs the error and carries on would delete all of it.
Valid bool
Digests map[string]struct{}
// NonDigestObserved names running containers whose observed image was not
// digest-form, i.e. podman gave us an image ID rather than a manifest
// digest and the entry we derived from it can never match anything in the
// registry. Such a container is protected only by its ImageTag and by its
// spec, so the gap must be recorded (a job step) rather than left silent.
// Entries are "host/slug/container: <image>".
NonDigestObserved []string
// ForeignSkipped lists the image references skipped as belonging to some
// other registry, deduped. A run that skips everything is a misconfigured
// registry host, and must be visible rather than silent.
ForeignSkipped []string
}
InUseSet is the set of manifest digests that must never be deleted, keyed by digest across ALL repositories. Blobs are shared between repos, so a digest protected anywhere is protected everywhere — deliberately broader than necessary, because that is the conservative direction.
func BuildInUseSet ¶
func BuildInUseSet(ctx context.Context, hostSrc HostEnumerator, inv InventorySource, specs SpecSource, reg ManifestResolver, cfg Config) (InUseSet, error)
BuildInUseSet computes the fleet-wide set of digests that must be protected from deletion. It is the union of two sources:
- observed: every running container's image AND its image tag, from the warm inventory. Both are needed: podman reports a manifest digest when it has one and an image ID when it does not, and only the tag can be resolved back to a manifest in the latter case;
- desired: every image-bearing value in every stored spec's parameters ("image", "pg_image", any "*_image"), resolved tag->digest through the registry. This half is what makes a stopped instance — or one whose spec pins something nothing is currently running — still recreatable. The legacy registry-gc.sh has no equivalent.
Any doubt aborts with ErrUnsafeToPrune and an empty set; see the doc on that error.
type InventorySource ¶
type InventorySource interface {
ListAllInstancesWithMeta(ctx context.Context, host string) ([]instance.Observed, instance.Freshness, error)
}
InventorySource is the observed half: the warm inventory cache, satisfied by *instance.Service.
type ManifestResolver ¶
type ManifestResolver interface {
Catalog(ctx context.Context) ([]string, error)
Manifest(ctx context.Context, repo, ref string) (imgregistry.Manifest, error)
}
ManifestResolver is the read-only slice of imgregistry.Client this package needs. It deliberately excludes Delete: the component that decides what to protect must not be able to remove anything, so no bug here can turn into a deletion.
type Metrics ¶
type Metrics interface {
// RunDone records a terminal run outcome ("succeeded", "failed",
// "dry-run", "aborted").
RunDone(result string)
// ManifestsDeleted records manifests actually removed from repo.
ManifestsDeleted(repo string, n int)
// RepoSkipped records a repo skipped by the per-repo tripwire, with the
// number of delete candidates that tripped it. This is the counter that
// would have surfaced the 2026-08-02 incident within a tick instead of a
// week.
RepoSkipped(repo string, candidates int)
// BytesReclaimed records the bytes Stage B's blob GC freed on disk. It is
// called ONLY when the measurement is real (BlobGC.Result.Measured): an
// unmeasured run must record nothing rather than zero, because "we could
// not size the registry" and "the GC freed nothing" are different facts
// and only one of them is a reason to look at the GC.
BytesReclaimed(bytes int64)
}
Metrics records prune outcomes. nil-safe via Handler.metric().
type Payload ¶
type Payload struct {
Policy Policy `json:"policy"`
// DryRun performs the full classification and records every job step, and
// issues ZERO deletes.
DryRun bool `json:"dry_run"`
// SkipBlobGC is the "--no-gc" equivalent: manifest deletion only, leaving
// the blobs recoverable. Stage B (blob GC) is a separate component; this
// field is the payload half of its gate.
SkipBlobGC bool `json:"skip_blob_gc"`
}
Payload is the job-args shape the scheduler enqueues and the handler reads. It carries a snapshot of the resolved policy, so a config reload mid-flight cannot change a running job's behaviour.
type PodRunner ¶
type PodRunner interface {
PlayKube(ctx context.Context, hostID, yaml string, replace bool, networks ...string) error
WaitForPodCompletion(ctx context.Context, hostID, podName string, timeout time.Duration) (int, error)
PodStop(ctx context.Context, hostID, name string) error
PodStart(ctx context.Context, hostID, name string) error
PodRemove(ctx context.Context, hostID, name string, force bool) error
ContainerExec(ctx context.Context, hostID, container string, cmd []string) (podman.ExecResult, error)
}
PodRunner is the slice of podman.Client Stage B needs. Narrow on purpose: this stage stops the fleet's only container registry, and the smaller the surface it holds, the smaller the set of things a bug here can do.
type Policy ¶
type Policy struct {
// ProtectedExact is a list of tag names that are always kept, matched
// case-sensitively and exactly.
ProtectedExact []string
// CalVer matches calendar-versioned release tags (e.g. "2024.01.01").
// Nil means no calver protection.
CalVer *regexp.Regexp
// ExtraPerRepo adds repo-specific protected-tag patterns on top of
// ProtectedExact/CalVer. A repo with no entry gets no extra protection.
ExtraPerRepo map[string]*regexp.Regexp
// RetentionDays is how long a "feat-*" tag survives before it is
// eligible for deletion, aged from the digest's config-blob .created.
RetentionDays int
// MaxDeletesPerRepo is the per-repo tripwire threshold. Not used by
// Classify/Deletable directly — it is Task 5's concern — but it lives on
// Policy because it is part of the same reviewable ruleset.
MaxDeletesPerRepo int
// DeleteUnrecognised opts a run into deleting ClassUnclassified tags. The
// #66 fix: false by default, because some repos have no CI-enforced tag
// vocabulary and ~13% of their tags are legitimate developer-branch
// names indistinguishable, by pattern alone, from garbage.
DeleteUnrecognised bool
}
Policy is the classification ruleset. Kept as data, not code, so it stays reviewable and testable — ported verbatim from registry-gc.sh's defaults (see DefaultPolicy).
func DefaultPolicy ¶
func DefaultPolicy() Policy
DefaultPolicy returns the ruleset ported verbatim from registry-gc.sh.
type Result ¶
type Result struct {
Before int64
After int64
Reclaimed int64
// Measured is true only when BOTH sizes were read. A false Measured means
// the byte fields say nothing — they must not be reported as zero bytes
// reclaimed.
Measured bool
}
Result is Stage B's measured outcome, returned as data rather than only as formatted job steps: the bytes-reclaimed metric has no other data path, and re-parsing a step string is not one.
Reclaimed is a FLOOR, not a measurement. The "after" size is read once the registry is serving pushes again, so anything written in between counts against the reclaim; a negative difference is clamped to zero.
type Scheduler ¶
type Scheduler struct {
Store store.JobStore
// Interval is the run cadence. Zero or negative disables the scheduler
// entirely (it never enqueues), rather than meaning "every tick".
Interval time.Duration
// Payload is evaluated once per enqueue. (The server does not re-parse
// flags on SIGHUP — only hosts and the operator file are reloaded — so this
// is a seam for a future reload, not one today.)
Payload func() Payload
Now func() time.Time
// contains filtered or unexported fields
}
Scheduler enqueues registry-prune jobs on a schedule. Store/Now are injected so the tick logic is unit-testable without real time.
Unlike prune.Scheduler there is no per-host dimension: there is one registry, so dedup, backoff and the interval gate are all global.
func (*Scheduler) EnqueueNow ¶
EnqueueNow enqueues a run immediately, bypassing the interval and backoff gates but NOT the in-flight check. It is the on-demand trigger behind POST /registry/prune.
Skipping the interval gate is the entire point. That gate is backed by PERSISTED job history — the newest succeeded run being younger than Interval returns early — so it survives a restart, and without this method the only way to get a second run inside 24h is to edit -registry-prune-interval and restart, which is precisely the knob nobody should be improvising with on the day of the first real delete. #220's rollout is a sequence of on-demand runs (dry run, read the steps, Stage-A-only real run, verify, then Stage B) and each step must not cost a day.
The in-flight check is NOT skipped, and is taken under the same lock the ticker uses: a manual enqueue that raced past it would put two runs on the same catalog, each reasoning from a listing the other is mutating.