Documentation
¶
Overview ¶
Package sweeper reclaims the storage Dispatch owns.
It touches only ephemeral artifacts — the ones Dispatch itself created on a handler's behalf. Durable artifacts, which are the application's uploads merely tracked here, are unreachable from every code path in this package. That guarantee is enforced twice: the store's sweep queries constrain themselves to ephemeral with a literal, and the sweeper re-checks each artifact before acting on it.
Deletion is two-phase. A sweep marks an artifact deleted and stops serving it; a later purge removes the bytes once a grace period has passed. A mistaken sweep is therefore observable and reversible for the length of that window rather than instantly destructive, and both phases are idempotent under retry.
Sweeping runs on the elected leader only, so a fleet of workers does not race to delete the same objects.
Index ¶
- Constants
- type Observer
- type Option
- func WithBatchSize(n int) Option
- func WithDryRun(dry bool) Option
- func WithEnabled(enabled bool) Option
- func WithInterval(d time.Duration) Option
- func WithLeaderCheck(fn func() bool) Option
- func WithLogger(l log.Logger) Option
- func WithObserver(o Observer) Option
- func WithPurgeGrace(d time.Duration) Option
- func WithRetention(d time.Duration) Option
- func WithTelemetry(t *telemetry.Telemetry) Option
- type Result
- type Sweeper
- func (s *Sweeper) PurgeOnce(ctx context.Context) (Result, error)
- func (s *Sweeper) RunOnce(ctx context.Context) (res Result, err error)
- func (s *Sweeper) Start(ctx context.Context) error
- func (s *Sweeper) Stop(ctx context.Context) error
- func (s *Sweeper) SweepOnce(ctx context.Context) (Result, error)
Constants ¶
const ( DefaultRetention = 168 * time.Hour DefaultPurgeGrace = 24 * time.Hour DefaultInterval = 15 * time.Minute DefaultBatchSize = 500 )
Defaults for a sweeper built without options.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Observer ¶
type Observer interface {
ArtifactSwept(ctx context.Context, a *artifact.Artifact)
ArtifactPurged(ctx context.Context, a *artifact.Artifact)
}
Observer is notified for each artifact a pass acts on, so lifecycle events reach the extension registry without this package depending on it.
type Option ¶
type Option func(*Sweeper)
Option configures a Sweeper.
func WithBatchSize ¶
WithBatchSize caps how many artifacts one pass may touch.
func WithDryRun ¶
WithDryRun reports what would be swept without changing anything.
func WithEnabled ¶
WithEnabled is the kill switch. A disabled sweeper's passes are no-ops, so reclamation can be stopped without redeploying.
func WithInterval ¶
WithInterval sets how often the background loop runs.
func WithLeaderCheck ¶
WithLeaderCheck restricts sweeping to the elected leader. Without one, every worker in a fleet would sweep concurrently.
func WithObserver ¶
WithObserver receives a callback per artifact acted on.
func WithPurgeGrace ¶
WithPurgeGrace sets how long a soft-deleted artifact's bytes survive. This is the window in which a mistaken sweep can still be caught.
func WithRetention ¶
WithRetention sets how long an ephemeral artifact survives after its last owner reaches a terminal state.
func WithTelemetry ¶
WithTelemetry sets where the sweeper records its spans and metrics: a root "sweep artifacts" span per pass that does work, and the swept, purged and bytes reclaimed counters. The Forge extension passes the engine's own.
type Result ¶
type Result struct {
// Swept is how many artifacts were marked deleted.
Swept int
// Purged is how many had their bytes removed.
Purged int
// BytesReclaimed counts the bytes freed by purging.
BytesReclaimed int64
// Skipped counts artifacts a pass declined to act on, which is where
// a backend failure shows up: the next pass retries them.
Skipped int
}
Result reports what one pass did.
type Sweeper ¶
type Sweeper struct {
// contains filtered or unexported fields
}
Sweeper reclaims ephemeral artifacts whose owners have finished.
func New ¶
New creates a Sweeper. It is enabled by default; use WithEnabled(false) to build one that does nothing until switched on.
func (*Sweeper) PurgeOnce ¶
PurgeOnce removes the bytes of artifacts soft-deleted longer ago than the grace period, then deletes their rows.
A backend failure skips that artifact rather than aborting the pass, so one unreachable object cannot stall reclamation of everything else. The next pass retries it.
func (*Sweeper) RunOnce ¶
RunOnce performs a sweep followed by a purge.
A pass that does work (this instance leads and the sweeper is enabled) runs under one "sweep artifacts" span, a child of ctx's span if it has one. The background loop gives it none, so each pass is its own trace. A follower's pass does nothing and starts no span.
Source Files