Documentation
¶
Overview ¶
Package fuzzwatch bounds a fuzz target's watchdog by SCHEDULED time rather than by wall-clock time.
The problem ¶
Three fuzz targets in this repository assert that a call terminates, using the same shape:
select {
case v := <-done:
...
case <-time.After(watchdog):
t.Fatalf("did not terminate")
}
The property being asserted is real and worth asserting: `go test -fuzz` has no per-input deadline, the interpreter's own step and allocation budgets are blind to a builtin that loops inside Go, and two of the three defects FuzzApplyStdlib found were exactly that shape. But `time.After` measures the wall clock, and the wall clock counts time during which this process was not running at all. On a contended CI runner a call that would finish in a millisecond can be descheduled for seconds, and the watchdog cannot tell that from a genuine hang. It fails, no input is really to blame, and the board goes red for a reason nobody can reproduce -- the failure mode this repository has spent a lot of effort eliminating.
Simply raising the watchdog does not fix it. The headroom is already enormous: measured on a 4-core box, FuzzEval averages 0.33ms per input against a 30s watchdog (~90,000x), FuzzApplyStdlib 0.73ms against 15s (~20,000x) and FuzzSchemaValidate 0.91ms against 20s (~22,000x). A bound four orders of magnitude above the mean is not too tight; it is measuring the wrong thing.
What this measures instead ¶
A single heartbeat goroutine per process ticks at a fixed interval and records how much wall clock passed during which it did NOT run on schedule. That is a direct measurement of "this process was not being given the CPU", and it is exactly the quantity a wall-clock watchdog wrongly charges to the code under test.
A Budget then spends only SCHEDULED time. When the caller's timer fires, Budget.Check answers one of three things:
- Continue: the process was descheduled for part of the window, so the budget is not spent. Wait the returned amount longer.
- Hung: the full budget of scheduled time elapsed with the process running normally. The call under test did not terminate; fail.
- Inconclusive: the hard wall-clock cap was reached and the process was starved throughout. Nothing can be concluded about this input, so nothing is asserted about it.
On a healthy machine no stall is ever recorded, Check returns Hung at exactly the configured budget, and every target detects precisely what it detected before. Detection is reduced only on a machine that is not running us -- where the alternative is not detection but a coin flip.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Budget ¶
type Budget struct {
// contains filtered or unexported fields
}
Budget is a watchdog budget denominated in scheduled time.
Create one before starting the work, arm an ordinary timer for Budget.Total, and call Budget.Check when it fires.
type Report ¶
type Report struct {
// Wall is the total wall-clock time since the budget was created.
Wall time.Duration
// Lost is the part of Wall during which this process was demonstrably not
// being scheduled.
Lost time.Duration
// LongestStall is the longest single heartbeat gap in the window.
LongestStall time.Duration
}
Report describes what the wall clock was doing during a watchdog window. It exists to be printed in a failure message: "did not terminate in 15s" is a much weaker claim than "did not terminate in 15s of scheduled time, during which the process was never descheduled by more than 120ms".
func (Report) Scheduled ¶
Scheduled is the part of the window during which the process was actually running -- the quantity a watchdog should be spending.
type Verdict ¶
type Verdict int
Verdict is what a fired watchdog timer actually means.
const ( // Continue means the process was descheduled for part of the window, so // the budget of scheduled time is not spent. Wait longer. Continue Verdict = iota // Hung means the budget of scheduled time elapsed while the process was // running normally. The call under test did not terminate. Hung // Inconclusive means the hard wall-clock cap was reached with the process // starved throughout. Whether the call would have terminated is unknown, // so nothing may be asserted about this input. Inconclusive )
Source Files