nova-sandbox

Overview ¶

What the wall denied, said out loud.

The failure this file is about was measured on 2026-09-18, dogfooding `nova-sandbox run` on a real card step: a `go build` inside the wall died and the ONLY symptom was Go's own sentence — `go: cannot find GOROOT directory: 'go' binary is trimmed and GOROOT is not set`. Nothing in that sentence says "sandbox", and nothing the tool printed said which path had been refused. A wall that denies silently costs the reader the whole investigation.

The root cause of that particular one is fixed where it belonged, in the optional roots' ancestors (pkg/sandbox/policy.go). This file is the class: when a contained command exits non-zero, ASK THE OPERATING SYSTEM what it refused and print one line per path, with the flag that would have allowed it.

THE HONEST LIMIT, measured on this machine (macOS 26, arm64, 2026-09-18) and stated here because a reader will otherwise think this is broken:

• macOS does report seatbelt violations to the unified log, under the subsystem `com.apple.sandbox.reporting`, category `violation`, and the parser below reads that exact shape.
• It does NOT report them for a profile applied with `sandbox-exec -p`. A denial produced by this tool is absent from `log show` at every level, `--info` and `--debug` included, while other processes' violations are in the same window.
• The two ways to ask for them do not exist on this OS either: `(deny default (with report))` is refused by the compiler — "report modifier does not apply to deny action" — and `(trace "<file>")` aborts sandbox-exec with SIGABRT, exit 134.

So on this macOS these lines are usually silent, and the NOTE that goes with a non-zero exit is what speaks instead. The reader is built and tested anyway, because it costs one process on a failing run, it is right where the OS does report, and the alternative — leaving the tool with no way at all to say what it denied — is the thing being fixed.

No seatbelt, no seatbelt violations. The parser and the line are platform-independent and tested everywhere; the reader is darwin's, and here it answers nothing rather than guessing.

The egress verbs are the card's OUTBOUND wall: `plan` renders one run's nftables ruleset from the reviewed allowlist in git, `apply` hands it to nft, `check` reads a plan back and asserts its invariants, and `drop` takes the wall away when the run is over. The rules themselves live in pkg/sandbox (egress.go); this file is the verb, the flags and the three seams that touch the machine.

The wall is on the BENCH, not in the card: a friend, 2026-09-18, "Not an env list the worker applies (the worker is the adversary)". A card cannot see this ruleset, cannot name a host for it and cannot take it down; it can only find out that a destination is denied.

apply and drop are LINUX's, because nftables is. On darwin the outbound wall is the seatbelt profile this binary already generates, and both verbs refuse there rather than pretending a plan was enforced.

The handoff's descriptor layer on unix: every open is an openat from a descriptor already checked, with O_NOFOLLOW so no component is a link and O_NONBLOCK so a FIFO cannot hold the open, and every descriptor is fstat'd for its type and its device before anything reads it.

nova-sandbox runs ONE command with its filesystem reach cut down by the operating system: the OS and toolchain roots and every --read directory are readable, every --write directory is readable and writable, and everything else on disk is denied to it by the kernel. docs/SPEC-SANDBOX.md is normative and this binary is the darwin half of it; on every other platform the wrap REFUSES, because a sandbox that silently does nothing is the failure the tool exists to close.

Exit codes are env(1)'s, not SPEC.md's 0/1/2, and the reason is in the spec: the exec path's status belongs to the wrapped command. 0-124 the command's own, 125 the tool said NO before it ran, 126 it could not be executed, 127 it was on no PATH entry, 128+N killed by signal N. The probe, check and version verbs are not wrappers and use SPEC.md's grammar unchanged.

The reap verb's three questions about the process table, answered on unix. Each is one bounded call to a tool macOS ships, or one signal — and each is behind a var in reapverb.go, so the verb's logic is tested with none of them.

The reap verb clears what a SIGKILL left behind, which is the one hole in the run verb's contract that the run verb cannot close from inside.

Measured on the bench in a 20-run soak, 2026-09-18. `nova-sandbox run` deletes its volume on every path out — a clean exit, an error, a signal it can catch, a --timeout — and a delete that fails prints SANDBOX LEAK. A SIGKILL is none of those: the tool is gone between one instruction and the next, so there is no path out to take and no line to print. Both halves of the containment then leak. The volume stays mounted, and the contained command's own `sleep 60` is reparented to PID 1 with its working directory ON that volume, which holds it open against every unmount — so the leak is not even one a later `diskutil apfs deleteVolume` clears by itself. `nova-sandbox check` says nothing about it: check asks what the backend can enforce, not what this machine is still holding.

So: a verb that names the debt and clears it. It touches nothing it did not make — `nova-` is the whole of its authority, exactly as the run verb's delete is — and it never takes a volume from a LIVE run, which is what the owner marker at the volume root is for. A reaper that cannot tell a working card from an orphan is a reaper nobody dares to run, and a reaper nobody runs is the same as no reaper at all.

The run verb gives one command a DISPOSABLE place to work and then takes the place away. On darwin that place is an APFS volume of its own in the boot container, quota'd by --size, mounted at /Volumes/nova-<name>: it is the only directory the wall lets the command write to, the command runs in a process group of its own, and on exit — normal, error, signal or --timeout — the group is killed and the volume is UNMOUNTED AND DELETED. Nothing of the run survives on the boot volume, so there is no cleanup step to forget, no half-cleaned job directory to inherit, and no quota that leaks into the next card's disk.

the coordinator, 2026-09-18: "build our own minimal isolation and hygiene sandboxes on Mac" — the Mac equivalent of a container, built out of the two things macOS already has: the seatbelt wall this binary already applies, and an APFS volume that costs nothing to make and nothing to throw away.

A failure to delete is the one thing this verb may not do quietly: it prints SANDBOX LEAK with the disk and the one command that removes it, and exits 3. A leak that nobody is told about is exactly the cleanup debt the verb exists to abolish.

Everything the verb reaches the machine through is a package-level var — the volume manager (diskutil), the executor (a contained child in a process group of its own) and the clock — so run_test.go replaces all three and the logic is tested without touching a disk: create, run, ALWAYS delete, report a leak, refuse a duplicate name, kill the group on a timeout.

The run verb's executor where there is no disposable volume to run in. run.go refuses the verb before this is reached; this exists so the verb's platform-independent logic compiles and is unit-tested on every platform the repository builds on.

The run verb's WINDOWS half: docs/SPEC-SANDBOX.md, "Windows — the disposable place", rules W1..W12.

The contract does not change. `nova-sandbox run --name <n> ...` on windows is the same verb with the same five steps, the same receipt, the same leak line and the same exit codes as the darwin section: look, create, run, KILL, delete; `SANDBOX DONE name=<n> exit=<code> wall=<s> freed=<bytes>` on the way out; `SANDBOX LEAK ...` and exit 3 when the place could not be removed; 124 on --timeout; 125 for every refusal of the tool's own. What differs is named here and nowhere else.

WHAT THE PLACE IS (W1): a Job Object plus a per-run scratch directory, and the two are ONE unit. The job is the windows answer to the darwin section's process group -- step 4, the kill -- and the scratch is the disposable place itself. A job with no scratch leaves the run's files on the profile; a scratch with no job leaves a survivor holding a handle to the directory the tool is about to delete. So they are made together and, when the second of them fails, the first is unmade before the verb returns.

WHY THIS FILE HAS NO BUILD TAG. Everything the verb reaches Windows through is the `winPlacer` interface below, and the SEQUENCE over that interface is the contract: create the job before the process, put the child in the job at creation, close the job before the delete, delete on every path out. That sequence is what runwin_test.go asserts, with a fake placer, ON ANY HOST -- there is no Windows bench in the estate (2026-09-18), and a contract that could only be tested where it runs would be tested nowhere. The Win32 calls themselves are in runwin_windows.go behind `//go:build windows`, and runwin_other.go is the same interface refusing off windows so this file compiles everywhere.

WHAT IS NOT PROVEN HERE, and is not claimed to be: that CreateJobObjectW, the extended limit information, PROC_THREAD_ATTRIBUTE_JOB_LIST and WindowsSandbox.exe behave on a real Windows machine as the rules say. The first Windows bench proves that. This file proves the shape that machine will be asked for.

Every platform that is not windows has no Job Object and no per-run scratch, and this file is that absence STATED rather than stubbed. The refusal a caller sees on linux is run.go's noDisposableBody, printed before anything is attempted; on darwin the verb never reaches this placer at all, because runVerb dispatches on runGOOS. The placer here exists so the windows half's LOGIC -- which is platform-independent and unit-tested with a fake in runwin_test.go -- still compiles everywhere.

It does NOT reach for WSL, for a container, or for an ordinary directory. W11: a build that reached for WSL when AppContainer, the Job Object or Windows Sandbox was unavailable must refuse instead, and the same holds for a host that is not windows at all.

Every platform that is not darwin has no disposable-volume body, and this file is that absence stated rather than stubbed. The refusal a caller sees is run.go's, printed before anything is attempted and naming where the disposable place is on that platform instead; the manager here exists so the verb's LOGIC — which is platform-independent and unit-tested with a fake — still compiles everywhere.

The worktree verb materialises one pull request's exact head in a scratch tree of its own. It is not a wrapper and builds no wall: it uses SPEC.md's 0/1/2 grammar (0 the verb ran, 2 could not run), reads the repository through git on PATH, and reads the pull request through a forge client seam so a test can put a fake there. Every seam the verb reaches the outside world through is a package-level var, because the tests in worktree_test.go replace each one and no test touches a network, a real forge, the real clock or a real process.