Documentation
¶
Overview ¶
Package testdeadline bounds tests that assert termination.
Several hardening regressions assert that an operation which used to hang or die now FINISHES. The assertion is "this terminates", not "this terminates within N seconds", so the budget only has to be large enough to never fire on a working build and small enough to catch a hang.
The budget is CPU time, not wall time (#789). Wall time counts every moment the process waits for a CPU, so a wall-clock bound fails a correct build on a starved runner (-cpu=1 beside a CPU hog), and no margin is safe from that. Within and Watch bound the CPU time of the goroutine under test; RunChild and Guard bound the CPU time of a child process.
A hang that blocks uses no CPU, so no CPU bound sees it. Backstop gives such a wait a context that ends just before the binary's -timeout.
Instrumentation still costs CPU: the race detector slows a program by roughly an order of magnitude, and a budget picked for an ordinary build would then fail `make race` on correct code. Scale returns the budget to use, and every bound in this package applies it. Callers pass the ordinary-build value.
Index ¶
- Variables
- func Backstop(t TB) (context.Context, context.CancelFunc)
- func Guard(p *os.Process, budget time.Duration) func() bool
- func RunChild(cmd *exec.Cmd, budget time.Duration) ([]byte, error)
- func Scale(d time.Duration) time.Duration
- func Watch(name string, d time.Duration, maxHeap uint64, fn func()) time.Duration
- func Within(budget time.Duration, fn func()) (time.Duration, bool)
- type TB
Constants ¶
This section is empty.
Variables ¶
var ErrOverBudget = errors.New("over its CPU budget")
ErrOverBudget is the error RunChild wraps when it kills a child for using its CPU budget.
Functions ¶
func Backstop ¶ added in v1.75.0
func Backstop(t TB) (context.Context, context.CancelFunc)
Backstop returns a context for a wait whose regression is a hang that uses no CPU: a goroutine or child process blocked on input or on a signal. CPU bounds (Within, Watch, RunChild, Guard) cannot see such a hang.
The context ends a margin before t's deadline, the binary's -timeout. The margin is backstopMargin or a quarter of the time left, whichever is less, so a short -timeout (-timeout=20s) still gets a live context (#791). A correct run that reaches it would have hit the -timeout anyway, so starvation alone cannot fail a test that the -timeout would pass. What it adds is the failure message: the hung test fails by name instead of the binary panicking with every goroutine's stack. If the binary has no -timeout, or its deadline has already passed, the context ends with the test.
func Guard ¶ added in v1.75.0
Guard kills the started process p once it has used Scale(budget) of CPU time, summed over its threads. Call the returned stop function after cmd.Wait returns; it reports whether Guard killed p.
It is RunChild for a test that drives the child itself (pipes, signals). As with RunChild, a child that blocks without using CPU is not bounded: pair Guard with Backstop for that.
func RunChild ¶ added in v1.75.0
RunChild runs cmd and returns its combined output, as cmd.CombinedOutput does. It kills the child once the child has used Scale(budget) of CPU time, summed over its threads, and then returns an error that wraps ErrOverBudget.
It is for tests that re-execute the test binary to contain a regression that does not return. A wall-clock deadline on the child counts process start-up and every moment the child waits for a CPU, so it fails a correct build on a starved runner; CPU time does not grow while the child waits. A child that blocks without using CPU is not bounded here: callers keep exec.CommandContext(t.Context(), ...) so the child ends with the test. On a platform where this package cannot read process CPU time (any OS but Linux), RunChild bounds wall time instead.
func Scale ¶
Scale returns d adjusted for the instrumentation this binary was built with. It never returns less than d.
func Watch ¶ added in v1.70.0
Watch runs fn and PANICS -- ending the test binary -- if fn has used Scale(d) of CPU time without returning, or if the heap grows by more than maxHeap bytes while it runs.
It is for regressions whose failure mode is a walk that does not come back: exponential work inside one builtin step, which ignores a context deadline and the step budget, and may allocate as it goes. Such a walk cannot be interrupted from outside, so a test that merely waited for it would hang the suite, and one that waited on a timer and then returned would leave it running -- and allocating -- behind the next test until the host ran out of memory. Ending the binary is the only bounded failure. A working build returns from fn in milliseconds of CPU, so neither bound fires.
The budget is CPU time on fn's thread, not wall time (#789): see Within. A walk that does not come back keeps using CPU, so it still reaches the budget; a correct walk on a starved process does not, however long it waits for a CPU.
Watch returns the CPU time fn used.
func Within ¶ added in v1.75.0
Within runs fn on its own OS thread and waits until fn returns or has used Scale(budget) of CPU time. It returns the CPU time fn used, and ok=false if fn was still running when the budget was spent. On ok=false fn's goroutine is left running: the callers use Within for code that cannot be interrupted, which is the defect they test for.
It bounds CPU time, not wall time, because a test asserts what the code does, and a starved process does the same work in more wall time. A process at 0.1% CPU share (-cpu=1 beside a CPU hog) can take minutes of wall time over milliseconds of work, so no wall-clock bound is safe from it. CPU time is charged only while fn's thread runs, so it does not grow while the process waits for a CPU.
Within bounds only work done on fn's goroutine. It does not bound time fn spends blocked: a call that waits forever uses no CPU, and the `go test` -timeout is the backstop for it. On a platform where this package cannot read thread CPU time (any OS but Linux), it bounds wall time instead.
Source Files