Documentation
¶
Overview ¶
Package cleanup is the retention center: a registry of cleanup targets, a scheduler that runs them under per-target modes, and a run history.
Index ¶
- Constants
- Variables
- func AuditWindows(general, security int) (int, int)
- func ModeDefinition(m Meta) runtimesettings.Definition
- func ModeKey(id string) string
- func Register(t Target)
- func RegisterBuiltins(r *Registry, d BuiltinDeps)
- func SweepByCutoff(ctx context.Context, oldest, cutoff time.Time, slice time.Duration, ...) (rows int64, more bool, err error)
- func WriteRunsCSV(w io.Writer, runs []RunRecord) error
- type AlertHistory
- type AuditSaver
- type BuiltinDeps
- type Category
- type Center
- func (c *Center) Get(ctx context.Context, id string) (TargetView, error)
- func (c *Center) List(ctx context.Context) (Overview, error)
- func (c *Center) ModeOf(ctx context.Context, m Meta) Mode
- func (c *Center) Preview(ctx context.Context, id string, days *int) (Impact, error)
- func (c *Center) PreviewCount(ctx context.Context) int
- func (c *Center) Run(ctx context.Context, id string, req Request) (RunRecord, Result, error)
- func (c *Center) Runs(ctx context.Context, f RunsFilter) (RunsPage, error)
- func (c *Center) Start(ctx context.Context)
- func (c *Center) Tick(ctx context.Context)
- type Config
- type FloorError
- type Impact
- type JanitorSource
- type Limits
- type Managed
- type ManagedState
- type Meta
- type Mode
- type Origin
- type Overview
- type Registry
- type Request
- type Result
- type RunOptions
- type RunRecord
- type RunStore
- type RunsFilter
- type RunsPage
- type SchedulerView
- type Settings
- type SweepStore
- type Target
- type TargetView
- type TokenSource
- type Trigger
- type Usage
- type UsageView
Constants ¶
const ( TargetAuditLog = "audit.log" TargetEmailLog = "messaging.email_log" TargetWebhookDelivery = "messaging.webhook_deliveries" TargetEventWebhook = "messaging.event_webhook_deliveries" TargetAlertHistory = "observability.alert_history" TargetAPITokens = "auth.api_tokens" //nolint:gosec // target id, not a credential )
Built-in target ids.
const ( EnvMaxRows = "APP_CLEANUP_MAX_ROWS" EnvSlice = "APP_CLEANUP_SLICE" EnvStagger = "APP_CLEANUP_STAGGER" EnvStartupDelay = "APP_CLEANUP_STARTUP_DELAY" EnvHistoryDays = "APP_CLEANUP_HISTORY_DAYS" EnvUsageTTL = "APP_CLEANUP_USAGE_TTL" )
Environment variables that bound a run.
const ( EventRun = "cleanup.run" EventFailed = "cleanup.failed" )
Event types emitted through the event webhook mechanism.
const AuditActionRun = "cleanup_run"
AuditActionRun is the audit action written for every run, dry runs included.
Variables ¶
var ( ErrUnknownTarget = errors.New("cleanup: unknown target") ErrBusy = errors.New("cleanup: this target is already running") ErrReadOnly = errors.New("cleanup: this target is managed elsewhere") ErrNoRetention = errors.New("cleanup: retention is set to keep forever, pass days to clean anyway") ErrInvalidDays = errors.New("cleanup: days must be a positive number") )
Errors the center returns; handlers map them to HTTP statuses.
var Default = NewRegistry(runtimesettings.Register)
Default is the process-wide registry whose mode settings join the runtime settings registry.
var RunsCSVHeader = []string{"id", "target", "started_at", "finished_at", "days", "dry_run", "rows", "bytes", "trigger", "actor", "error"}
RunsCSVHeader is the column order of the CSV export.
Functions ¶
func AuditWindows ¶
AuditWindows applies the audit floors and the rule that security entries outlive the general window. 0 means keep forever; a general window of 0 keeps everything whatever the security value says.
func ModeDefinition ¶
func ModeDefinition(m Meta) runtimesettings.Definition
ModeDefinition is the runtime setting for m's mode.
func RegisterBuiltins ¶
func RegisterBuiltins(r *Registry, d BuiltinDeps)
RegisterBuiltins registers the targets that wrap existing sweepers, the token cleanup and the janitor stages.
func SweepByCutoff ¶
func SweepByCutoff(ctx context.Context, oldest, cutoff time.Time, slice time.Duration, maxRows int64, del func(ctx context.Context, cutoff time.Time) (int64, error)) (rows int64, more bool, err error)
SweepByCutoff removes rows older than cutoff by calling del with a cutoff that advances one slice at a time from the oldest row, so each statement covers a bounded span and the row cap can stop the run between slices. del must delete rows strictly older than the cutoff it gets; nothing at or after cutoff is touched.
Types ¶
type AlertHistory ¶
type AlertHistory interface {
HistoryStats(ctx context.Context) (rows int64, oldest time.Time, bytes int64, err error)
CountHistoryBefore(ctx context.Context, cutoff time.Time) (int64, error)
PruneHistory(ctx context.Context, cutoff time.Time) (int64, error)
}
AlertHistory is the alert_history access. *alerting.DB satisfies it.
type AuditSaver ¶
type AuditSaver interface {
SaveAuditEntry(ctx context.Context, e store.AuditEntry) error
}
AuditSaver writes audit rows. *store.DB satisfies it.
type BuiltinDeps ¶
type BuiltinDeps struct {
Store SweepStore
Alerts AlertHistory
Tokens TokenSource
Janitor JanitorSource
Settings Settings
Now func() time.Time
}
BuiltinDeps are the collaborators of the built-in targets. A nil Alerts, Tokens or Janitor skips those targets.
type Category ¶
type Category string
Category groups targets in the dashboard.
const ( CategoryAuth Category = "auth" CategoryAudit Category = "audit" CategoryDeploys Category = "deploys" CategoryDocker Category = "docker" CategoryData Category = "data" CategoryObservability Category = "observability" CategoryMessaging Category = "messaging" CategoryOther Category = "other" )
Target categories.
type Center ¶
type Center struct {
// contains filtered or unexported fields
}
Center runs targets, keeps history and drives the scheduler.
func (*Center) Preview ¶
Preview reports what a run with days (or the saved policy when nil) would remove, writing nothing.
func (*Center) PreviewCount ¶
PreviewCount is how many targets only measure now, for the first-run review. Read-through targets count when the janitor is in dry-run, which is a real preview state.
func (*Center) Run ¶
Run executes a run on demand and records it. A returned error is a pre-flight refusal; a failure of the run itself is in Result.Err and in history.
func (*Center) Runs ¶
Runs lists history. It reads one extra row to know whether another page exists.
type Config ¶
type Config struct {
Registry *Registry
Store RunStore
Settings Settings
Audit AuditSaver
// Emit hands an event to the event webhook queue; nil drops events.
Emit func(eventType string, data map[string]any)
// Notify sends a plain-text notice to the alert channels; nil skips notices.
Notify func(ctx context.Context, text string) error
Limits Limits
Logger *slog.Logger
Now func() time.Time
}
Config wires a Center.
type FloorError ¶
FloorError means a requested window is shorter than the target's minimum.
func (*FloorError) Error ¶
func (e *FloorError) Error() string
type JanitorSource ¶
type JanitorSource interface {
janitor.PolicyStore
ListJanitorRuns(ctx context.Context, limit int) ([]store.JanitorRun, error)
}
JanitorSource reads the janitor policy and its recorded passes. *store.DB satisfies it.
type Limits ¶
type Limits struct {
// MaxRows caps the rows one run removes; the run reports more_remaining when it stops there.
MaxRows int64
// Slice is the time span of one delete batch, so no single DELETE covers an unbounded range.
Slice time.Duration
// Stagger spaces targets within one scheduler tick.
Stagger time.Duration
// StartupDelay keeps the scheduler quiet right after boot.
StartupDelay time.Duration
// HistoryDays is how long run history is kept.
HistoryDays int
// UsageTTL is how long a measured usage is reused by the list endpoint.
UsageTTL time.Duration
}
Limits bound how much one run and the scheduler do.
type Managed ¶
type Managed interface {
State(ctx context.Context) (ManagedState, error)
}
Managed is implemented by read-through targets that report their own state.
type ManagedState ¶
ManagedState is the policy a read-through target shows.
type Meta ¶
type Meta struct {
// ID is dotted, for example "audit.log".
ID string
Category Category
// Title and Description are English fallbacks for the i18n keys.
TitleKey, Title string
DescKey, Description string
// PolicySetting is the runtime setting holding the retention days.
PolicySetting string
// Floor is the shortest allowed retention in days, 0 for none.
Floor int
Destructive bool
Origin Origin
// Presets lists the day counts the dashboard offers; empty means the standard presets.
Presets []int
// ReadOnly marks a target whose policy and runs live elsewhere (the janitor); Link says where.
ReadOnly bool
Link string
}
Meta is a target's static description.
type Mode ¶
type Mode string
Mode is what the scheduler does with a target.
func DefaultMode ¶
DefaultMode is the mode a target starts in: unchanged behavior for existing sweepers, preview for new ones.
type Origin ¶
type Origin string
Origin says whether a target replaces an existing sweeper or covers data that was never bounded.
type Overview ¶
type Overview struct {
Targets []TargetView `json:"targets"`
Scheduler SchedulerView `json:"scheduler"`
}
Overview is the body of GET /api/v1/cleanup.
type Registry ¶
type Registry struct {
// contains filtered or unexported fields
}
Registry holds the registered targets in registration order.
func NewRegistry ¶
func NewRegistry(registerSetting func(runtimesettings.Definition)) *Registry
NewRegistry builds a registry. registerSetting receives each target's mode setting; nil skips registration, for tests.
type Request ¶
type Request struct {
// Days overrides the saved policy when set.
Days *int
DryRun bool
Opts RunOptions
}
Request is one manual or API run.
type RunOptions ¶
type RunOptions struct {
DryRun bool
Trigger Trigger
Actor string
// ActorType, ActorID and RemoteAddr fill the audit row; empty means a system actor.
ActorType string
ActorID string
RemoteAddr string
// MaxRows caps one run; Slice is the time span of one delete batch. The center fills both from its limits.
MaxRows int64
Slice time.Duration
}
RunOptions say how one run behaves.
type RunRecord ¶
type RunRecord struct {
ID int64 `json:"id"`
Target string `json:"target"`
StartedAt time.Time `json:"started_at"`
FinishedAt time.Time `json:"finished_at"`
Days int `json:"days"`
DryRun bool `json:"dry_run"`
Rows int64 `json:"rows"`
Bytes int64 `json:"bytes"`
Trigger string `json:"trigger"`
Actor string `json:"actor"`
Error string `json:"error"`
}
RunRecord is one history row on the wire.
type RunStore ¶
type RunStore interface {
InsertCleanupRun(ctx context.Context, r store.CleanupRun) (int64, error)
ListCleanupRuns(ctx context.Context, f store.CleanupRunFilter) ([]store.CleanupRun, error)
LatestCleanupRuns(ctx context.Context) ([]store.CleanupRun, error)
CleanupFailedSince(ctx context.Context, target string, since time.Time) (bool, error)
PruneCleanupRuns(ctx context.Context, before time.Time) (int64, error)
}
RunStore persists run history. *store.DB satisfies it.
type RunsFilter ¶
type RunsFilter struct {
Target string
Since time.Time
Until time.Time
DryRun *bool
Status string
Limit int
Cursor int64
}
RunsFilter narrows Runs.
type SchedulerView ¶
SchedulerView is the scheduler state on the wire.
type Settings ¶
type Settings interface {
Int(ctx context.Context, key string) int
String(ctx context.Context, key string) string
Bool(ctx context.Context, key string) bool
Duration(ctx context.Context, key string) time.Duration
}
Settings resolves runtime settings. *runtimesettings.Service satisfies it, and a nil one yields env then defaults.
type SweepStore ¶
type SweepStore interface {
CleanupStats(ctx context.Context, t store.CleanupTable) (store.CleanupTableStats, error)
CleanupCountOlder(ctx context.Context, t store.CleanupTable, cutoff time.Time) (int64, error)
CountAuditEntriesTiered(ctx context.Context, cutoff, securityCutoff time.Time) (int64, error)
DeleteAuditEntriesTiered(ctx context.Context, cutoff, securityCutoff time.Time) (int64, error)
PruneEmailDeliveries(ctx context.Context, before time.Time) (int64, error)
DeleteWebhookDeliveriesOlderThan(ctx context.Context, cutoff time.Time) (int64, error)
DeleteEventDeliveriesOlderThan(ctx context.Context, cutoff string) (int64, error)
}
SweepStore is the table access the built-in sweeper targets use. *store.DB satisfies it.
type Target ¶
type Target interface {
Meta() Meta
Measure(ctx context.Context) (Usage, error)
Preview(ctx context.Context, days int) (Impact, error)
// Run removes data older than days. With opts.DryRun it must not write.
Run(ctx context.Context, days int, opts RunOptions) Result
}
Target is one kind of data the center can clean.
type TargetView ¶
type TargetView struct {
ID string `json:"id"`
Category string `json:"category"`
Title string `json:"title"`
TitleKey string `json:"title_key,omitempty"`
Description string `json:"description"`
DescriptionKey string `json:"description_key,omitempty"`
PolicyKey string `json:"policy_key"`
Days int `json:"days"`
Source string `json:"source"`
Mode string `json:"mode"`
Origin string `json:"origin"`
Destructive bool `json:"destructive"`
Floor int `json:"floor"`
Presets []int `json:"presets"`
Usage UsageView `json:"usage"`
UsageError string `json:"usage_error,omitempty"`
LastRun *RunRecord `json:"last_run"`
NextRun *time.Time `json:"next_run"`
ReadThrough bool `json:"read_through"`
Link string `json:"link,omitempty"`
DryRun *bool `json:"dry_run,omitempty"`
}
TargetView is one target on the wire.
type TokenSource ¶
type TokenSource = janitor.TokenPurger
TokenSource deletes inactive API tokens. *store.DB satisfies it.