amber-bench

Overview ¶

amber-bench: an ingest → delete → gc benchmark for amber-store.

The workload is 1000 references over ~50 GiB of unique data, interleaved into two classes so that deleting 700 of them leaves ~30 GiB: "kept" refs (i%10 < 3) carry 100 MiB of fresh random data each, "deleted" refs (i%10 >= 3) carry 30 MiB. Every ref i > 0 also contains whole-file copies (reflinks, so they cost no disk) of ~1/3 of its fresh size taken from ref i-1, so ~25 % of the ingested bytes are duplicates already in the store — and, because kept refs copy from deleted neighbours too, the bytes the deleted refs "own" and the bytes a collection can actually reclaim differ. Files are 256 KiB–4 MiB; content is seeded, so a rerun produces the identical dataset and pack layout.

Phases (-phase, default all): gen writes the dataset; ingest stores every ref in-process the way the CLI and a daemon do (ingest.Dir, then the collector's PrepareRef) and times each; delete removes the 700 through ReleaseRef; gc runs the real CLI — `gc run` under policy, then a forced `--garbage 0` pass — with `gc status` and du snapshots around each step; verify runs fstree.CheckComplete on every surviving ref and restores a sample through the CLI, comparing it byte for byte with the source; report prints the summary from the results file. Each phase appends to -out, so phases can be rerun individually.

The dataset plus the store need ~2× the unique size on disk. At full scale the run is a few minutes on an SSD; -refs and -scale shrink it (-refs 30 -scale 0.1 -segment 4194304 is a seconds-long smoke test that still seals and reaps packs).

go build -o /tmp/amber-store ./cmd/amber-store
go run ./cmd/amber-bench -data /tmp/bench/data -store /tmp/bench/store \
    -restore /tmp/bench/restore -bin /tmp/amber-store -out /tmp/bench/results.json