Documentation
¶
Overview ¶
Package crashinject provides a subprocess-based crash-injection harness for deterministic crash-safety testing of WAL, snapshot, and checkpoint write paths.
Architecture ¶
Crash-injection tests use a parent–child model:
- The test parent calls Run with a named scenario.
- Run spawns cmd/crashinject-helper as a child process, passing GOGRAPH_CRASH_AT=<scenario> and GOGRAPH_CRASH_DIR=<dir>.
- The helper runs the scenario and calls Breakpoint at a precisely chosen execution point.
- Breakpoint sends SIGKILL to itself, terminating the child abruptly at that exact state.
- Run returns an Out value describing how the child exited, and the caller inspects the artefacts left in dir.
Breakpoint registration ¶
Library code (e.g. store/wal, store/snapshot) calls Breakpoint at any point where a crash should be injected. A typical call site:
crashinject.Breakpoint("wal.mid-frame")
This is a no-op in production (GOGRAPH_CRASH_AT is not set) and self-kills the process when running under the crash harness.
Concurrency ¶
Breakpoint reads an environment variable set once at process startup — it is safe to call concurrently with no locking. Run is safe to call from multiple goroutines (each invocation spawns an independent child process); the package-level binary cache is guarded by a sync.Once.
Index ¶
Constants ¶
const EnvCrashAt = crashpoint.EnvCrashAt
EnvCrashAt is the environment variable read by Breakpoint to decide which named point should trigger a crash. It is an alias for crashpoint.EnvCrashAt; the canonical definition lives in the dependency-light crashpoint package so production code can embed breakpoints without importing this test-harness package.
const EnvCrashDir = crashpoint.EnvCrashDir
EnvCrashDir is the environment variable that tells the helper binary where to place its artefacts (WAL files, temp data). Alias for crashpoint.EnvCrashDir.
Variables ¶
This section is empty.
Functions ¶
func Breakpoint ¶
func Breakpoint(name string)
Breakpoint is a thin re-export of crashpoint.Breakpoint so existing callers of crashinject.Breakpoint keep working. New production call sites should import internal/crashpoint directly to avoid pulling the testing package into their binaries.
func HelperBinaryDir ¶ added in v0.12.0
func HelperBinaryDir() string
HelperBinaryDir reports the temporary directory holding this process's cached crashinject-helper binary, or "" when no helper has been built (or after RemoveHelperBinary has run).
It exists so a test can assert that the directory the process-exit hook will delete is the directory the build actually created — the link between "the removal works" and "the removal is aimed at the right path". It carries the same concurrency restriction as RemoveHelperBinary.
func RemoveHelperBinary ¶ added in v0.12.0
func RemoveHelperBinary()
RemoveHelperBinary deletes the temporary directory holding this process's cached crashinject-helper binary. It is a no-op when no helper was ever built, so every package may call it unconditionally.
Why the hook is process-scoped, and not t.Cleanup ¶
[buildHelperOnce] caches the binary path in a sync.Once for the whole test process, precisely so that N crash scenarios pay for one 15 MB `go build` instead of N. A t.Cleanup registered by the test that happened to trigger that build would delete the binary out from under every later test in the same process, turning the cache into a use-after-free. The correct scope is therefore the process, and TestMain is the only process-scoped hook the testing package offers.
How to wire it ¶
Every package whose tests call Run must install this in its TestMain. A bare `defer RemoveHelperBinary()` does NOT work alongside goleak.VerifyTestMain, because goleak calls os.Exit and deferred functions do not run through os.Exit. Use goleak's own cleanup hook, which replaces that os.Exit call and is therefore responsible for exiting:
func TestMain(m *testing.M) {
goleak.VerifyTestMain(m, goleak.Cleanup(func(exitCode int) {
crashinject.RemoveHelperBinary()
os.Exit(exitCode)
}))
}
TestHelperCleanup_WiredInEveryCallerPackage enforces this wiring across the module, so a new package that calls Run cannot silently reintroduce the leak.
Concurrency ¶
RemoveHelperBinary is NOT safe for concurrent use with Run. It reads the cached directory without synchronisation and deletes the binary that Run executes, so it must be called only after every test in the process has returned — which is exactly what the TestMain hook above guarantees.
Types ¶
type Opts ¶
type Opts struct {
// Dir is the crash artefact directory forwarded to the helper via
// GOGRAPH_CRASH_DIR. If empty, [Run] creates a fresh t.TempDir()
// and the caller finds artefacts there after Run returns.
Dir string
// Env holds additional KEY=VALUE pairs appended to the child
// environment (after GOGRAPH_CRASH_AT and GOGRAPH_CRASH_DIR).
Env []string
// Timeout caps the child execution. Zero defaults to 30 s.
Timeout time.Duration
}
Opts configures a Run invocation.
type Out ¶
type Out struct {
// Signal is the signal that terminated the child, or nil.
Signal os.Signal
// Dir is the crash artefact directory used by the child. Callers
// inspect artefacts left there after Run returns.
Dir string
// Stdout and Stderr hold the child's captured output streams.
Stdout []byte
Stderr []byte
// ExitCode is the numeric exit status. Meaningful only when
// Killed is false and the child exited voluntarily.
ExitCode int
// Killed reports whether the child was terminated by SIGKILL at a
// [Breakpoint] (i.e. a genuine crash-injection self-kill).
// It is false when the child was killed by a context timeout or
// cancellation; use [Out.TimedOut] to distinguish that case.
Killed bool
// TimedOut reports whether the context deadline elapsed before the
// child exited. When true, Killed is false even if the child was
// ultimately terminated by SIGKILL (the kill was issued by
// exec.CommandContext, not by a crashpoint self-kill).
TimedOut bool
}
Out captures the observable outcome of a helper child process spawned by Run.
func Run ¶
Run builds (lazily) and spawns the cmd/crashinject-helper binary with GOGRAPH_CRASH_AT=scenario. It waits for the child to exit and returns the captured output and exit status.
The caller should inspect Out.Killed to confirm the child was terminated by SIGKILL, and then examine the artefacts in Out.Dir.
Source Files