tune

Overview ¶

Package main implements `tune` — a bandit tuner over the live bench knobs.

go run ./bench/tune -exit <pk> -out bench/<date>/<commit> -pins $S/pins \
    -runner run-mux.sh -args '2 http://127.0.0.1:18080 "<order>"' \
    -objective mux-tunnels-2/50up:ratio,mux-tunnels-2/10up:ratio \
    -knobs 'upload.chunk_bytes=1MiB,2MiB,4MiB;upload.concurrency=2,4,8' \
    -rounds 12 -trials 2 -env 'TUNNELS=2 LEGS='

WHY A BANDIT AND NOT A SWEEP. bench/run-sweep.sh spends one complete runner invocation on every value of one knob, and every value pays the same price whether it looked hopeless after its first run or not. A rig minute is the scarce thing here — a 2-trial 10/50 MB down+up set is about four minutes — and the measurement is NOISY: the paired ratios of the 2026-09-16/17 campaign swing +-30 % between trials of the same shape, and the unpaired bar swings 2x inside an hour. A full sweep of three knobs at three values each is 9 runs that say almost nothing about the INTERACTION between them, and a Bayesian optimizer over a continuous space would spend its budget learning a surface nobody asked for.

So: one arm per (knob, value) — a discrete grid, the same grid a sweep would have used — and Thompson sampling with a Gaussian posterior per arm. Every round spends exactly ONE runner invocation, on the value its own posterior says is most likely to be the best, so a value that lost twice stops being paid for while a value that is merely uncertain keeps being probed. The knob under test rotates round-robin, and the OTHER knobs are held at the incumbent — each knob's posterior-mean-best value so far — so a round is a real measurement of one knob against the current best-known configuration rather than against the compiled defaults.

Nothing here decides anything permanent: it writes tune.tsv (every round, its settings and its objective) and incumbent.txt (a SETTINGS line to paste into a campaign). The verdict on the incumbent is still a full run of the campaign under bench/verdict.sh — this only says where to point it.