hopperbench

Overview ¶

Command hopperbench measures hopper's hot paths against a real database and prints machine-readable results, one JSON object per scenario.

hopperbench -database-url postgres://... [-scenarios throughput,insert,copy,latency] [-jobs 50000] [-clients 4]
hopperbench -compare base.jsonl,head.jsonl [-threshold 0.10]

It works in a schema of its own ("hopperbench"), which it creates and drops, so it can point at any database. Numbers depend on the hardware, the Postgres configuration and the network between them; compare runs made on the same setup, such as a PR branch against canary. The -compare mode does that: it reads two files of results, averages the faster half of the runs of each scenario, and exits non-zero if the second is slower than the first by more than the threshold on any scenario. Noise on a shared machine only ever slows a run down, so the faster half is the better estimate; averaging it, rather than taking the single best run, keeps one lucky run from deciding the outcome.

Scenarios:

throughput  no-op jobs preloaded, then drained by -clients clients
mixed       as throughput, with all four priorities interleaved
scheduled   as throughput, with jobs scheduled up to 500ms in the future
retry       as throughput, with every job failing its first attempt
insert      InsertMany in batches of 100 (the unnest statement)
copy        InsertMany in batches of 5000 (the COPY path)
latency     commit-to-Work on an idle queue, local wake-up
loaded      commit-to-Work while -clients clients work a stream of inserts at -rate jobs/s for -seconds;
            with -report N it also prints one "loaded-window" line every N seconds (for soaks)

-history preloads that many rows of history first, to measure the live table's independence from accumulated history.