auditretention

package
v0.0.6 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 16, 2026 License: MIT Imports: 13 Imported by: 0

Documentation

Overview

Package auditretention seals and prunes the audit trail (Task 20218).

The audit_events table had no retention path: nothing pruned it, pkg/compact did not touch it, and on this project's own hub it reached 1,096,198 rows and 2.44 GB. Giving it a DELETE was not enough, because the table is hash-chained and Task 20167 ships a verifier — an unexplained truncation is indistinguish- able from tampering, and correctly reported as such.

So a prune here is three things that must all happen or none of them:

  1. SEAL. The prefix is streamed to a JSONL file, chain-verified row by row on the way out, and digested. Nothing is deleted before that file is on disk and fsynced.
  2. TRUNCATE. The rows are deleted.
  3. ANCHOR. A row records the boundary hash, the surviving id, and the export's digest and location, so the verifier can walk across the gap.

Steps 2 and 3 are one transaction inside statedb; step 1 completes before either. A crash anywhere leaves a state that is either "not pruned" or "pruned and explained", never "pruned and unexplained".

Why the seal is always JSONL, whatever the operator's SIEM prefers: it is the only lossless format pkg/auditexport offers — CEF truncates payloads and CSV cannot be appended in batches without a second header — and an archive that has lost bytes cannot answer the question it was kept for. Converting a seal into CEF or CSV afterwards is `cloop audit-log export`; converting the other way is impossible.

And why it is gzipped by default: an audit archive is written once, read rarely, and enormously repetitive — successive rows differ in a few fields of the same JSON shape. The prefix this project's own hub needed to seal was 1.81 GB of JSONL on a filesystem with 1.9 GB free; compressed it is a small fraction of that. The digest covers the file as written, so a compressed seal is verified exactly the same way. --no-compress is there for an operator who would rather grep the archive than gunzip it.

Index

Constants

View Source
const DefaultExportDirName = "audit-archive"

DefaultExportDirName is where seals land under a project's .cloop directory when the operator has not chosen somewhere else.

Variables

This section is empty.

Functions

func ResolveCutoff

func ResolveCutoff(retentionDays int, now time.Time) time.Time

ResolveCutoff turns a retention window in days into an instant.

Types

type Options

type Options struct {
	// Before is the cutoff: rows with a timestamp strictly older are eligible.
	// Required — a zero value is rejected rather than silently interpreted as
	// "now", which would prune the entire trail.
	Before time.Time

	// ExportDir is the directory the seal is written to. Created if missing.
	ExportDir string

	// Actor is recorded on the anchor. Defaults to "system".
	Actor string

	// DryRun resolves and reports the candidate without writing or deleting
	// anything — not even the export.
	DryRun bool

	// BatchSize is how many rows are read and written per round trip. Zero
	// selects a sane default. Bounded internally.
	BatchSize int

	// Uncompressed writes plain JSONL instead of gzip. The zero value
	// compresses, which is the right default for a write-once, read-rarely,
	// highly repetitive archive.
	Uncompressed bool

	// Hold is how far the audit sinks forwarding this chain have all got
	// (Task 20426): the newest row every live cursor has acknowledged, from
	// auditforward.RetentionHold. Nil means no sink forwards the chain.
	// Unless Force is set the prune stops at it, so retention never removes a
	// row a configured sink has not received.
	Hold *statedb.AuditForwardHold

	// Force prunes past Hold. Rows a sink has not acknowledged are sealed and
	// removed anyway, and reach that sink only through the archive; the
	// forwarder records the loss as an audit.forward.discontinuity row.
	Force bool

	// Now is injected by tests. Production leaves it nil.
	Now func() time.Time
}

Options configures a prune.

type Report

type Report struct {
	DryRun bool
	Cutoff time.Time

	// Candidate is the resolved prefix. Count == 0 means there was nothing
	// older than the cutoff and no other field is meaningful.
	Candidate statedb.AuditPruneCandidate

	ExportPath   string
	ExportSHA256 string
	ExportBytes  int64

	// Anchor is the row written. Zero when DryRun or when nothing was pruned.
	Anchor statedb.AuditAnchor

	// Pruned reports that rows were actually deleted.
	Pruned bool

	// Clamped reports that the cutoff was clamped to Options.Hold: rows
	// older than the cutoff that a sink has not acknowledged were kept.
	// HeldBy names the slowest sink, HeldThroughID the newest row it has
	// acknowledged, and Requested is the prefix the cutoff alone would have
	// removed. Candidate is what was removed instead.
	Clamped       bool
	HeldBy        string
	HeldThroughID int64
	Requested     statedb.AuditPruneCandidate

	// ForcedPast names the sinks a forced prune removed unacknowledged rows
	// from under, in name order.
	ForcedPast []string
}

Report describes what a prune did, or would have done.

func Prune

func Prune(db *statedb.DB, opts Options) (*Report, error)

Prune seals and removes the audit prefix older than opts.Before.

It is safe to run against a live hub: the seal reads in bounded batches that release the database mutex between them, and the truncation is a single transaction. It does not VACUUM — reclaiming the freed pages is `cloop db maintain`, which has to care about peer hubs in a way this does not. See cmd/hub_audit_cmd.go.

type SealStatus

type SealStatus struct {
	Anchor statedb.AuditAnchor
	// Present is false when the file is missing — which is not necessarily
	// wrong (archives get moved to cold storage on purpose) but is always
	// worth saying out loud.
	Present bool
	OK      bool
	Actual  string
	Err     error
}

SealStatus is the result of re-checking one anchor's export against the digest recorded when it was written.

func VerifySeals

func VerifySeals(db *statedb.DB) ([]SealStatus, error)

VerifySeals re-hashes every anchor's export and compares it to the digest in the anchor.

This is the check that reaches outside the database. VerifyAuditChain can only prove the surviving rows are consistent with what the anchors claim, and an attacker able to rewrite audit_events can rewrite audit_anchors too. The seal files are the part of the record that can be copied somewhere the hub cannot write, and this is what compares the two.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL