README
¶
sandboxtest
sandboxtest is a reusable, standard-library-only conformance suite for
sandbox executors. External backends implement the small structural SUT
interface rather than importing this module's internal packages:
type SUT interface {
RunCommand(context.Context, string, string) ([]byte, int, error)
Level() uint8
GuaranteeBits() uint64
}
ArgvSUT is an optional shell-free extension used by platform probes when an
operation has a direct executable form. RunSuite accepts a factory that
returns a fresh SUT for each check, so state and planted environment values do
not leak between cases.
One-way assertions
The suite tests claimed implications, not mechanism names. A set guarantee bit must survive its negative probes, and every negative probe needs an unconfined positive control. A missing bit does not require permissive behavior: a backend may enforce defense in depth without claiming an end-to-end guarantee.
The base suite checks in-policy reads/writes, claimed read/write boundaries,
environment scrubbing, and level/bit consistency. CheckClaimedImplications
adds scenario-specific process, network, address, target, and resource probes.
A claimed bit without a supplied probe fails.
Platform command helpers
The package keeps shell details behind platform files. Unix helpers use the
platform shell only where a direct argv operation is unavailable. Windows
helpers prefer ArgvSUT and otherwise construct explicit cmd.exe commands
with Windows quoting. Consumers call RunSuite and implication probes; they do
not need to select a shell themselves.
Reuse from another module
func TestBackendConformance(t *testing.T) {
sandboxtest.RunSuite(t, "backend", func(t *testing.T, workspace string) sandboxtest.SUT {
sut := newBackendForTest(t, workspace)
t.Cleanup(func() { _ = sut.Close() })
return sut
})
}
Mirror the exported guarantee-bit positions exactly and add a drift test if your package also exposes named constants. Supply a fresh writable workspace, ensure the outside positive-control targets are usable, and clean up every process, listener, grant, ACL, and machine object your factory owns. Never convert an unavailable requested mechanism into a passing skip.
Documentation
¶
Overview ¶
Package sandboxtest is a reusable conformance suite for sandbox executors, modelled on the storekit `storetest` pattern: a consumer supplies a factory that builds an executor, and RunSuite asserts the core sandbox invariants hold against it. It is the executor analogue of storetest — one suite any backend (null / seatbelt / the Linux ladder) is run through, so a new or downstream backend proves the load-bearing security contract without re-deriving the assertions.
What it asserts (the contract, not a mechanism) ¶
Every assertion is gated on what the executor REPORTS (Guarantees bits / Level), never on the host platform. The SAME suite therefore passes against the null backend (LevelNone, no OS enforcement), a rung-2 Linux executor (LevelDegraded), a rung-1 executor (LevelFull), and Seatbelt — each is held only to the guarantees it actually claims:
- Write boundary — a write inside a policy-writable root succeeds. When the executor claims WriteBoundary, every covered write outside every writable root is denied. An executor that withholds the bit may still deny writes; absent claims never require permissive behavior.
- Read boundary — a read inside the workspace succeeds. When ReadBoundary is claimed, a host-readable file outside the workspace is denied.
- Env scrub — a secret planted in the parent environment is absent from a spawned child whenever the executor claims EnvScrub. This is the harness secret-leak boundary and holds independently of any OS mechanism.
- Self-consistency — the reported guarantees and Level are internally coherent and fail-secure: LevelNone claims no OS-enforcement bit beyond EnvScrub; address-scoped networking implies a network boundary; a write boundary implies at least a degraded level; LevelFull implies a write boundary. An incoherent posture (a set bit with no honest backing) is the signal the auto-approval interlock must never trust.
Scenario-specific process, network, and resource behavior is reusable through CheckClaimedImplications. Platform suites provide the setup callbacks; this package owns strict bit gating and positive-control validation.
Dependency posture ¶
This package deliberately imports ONLY the standard library. The executor is consumed through the minimal structural interface SUT, and the guarantee-bit and level constants are mirrored from the sandbox package's stdlib-only seam (SPEC §6: "Bit positions are exported constants; ... the consumer builds each posture's required mask from them" — designed for probing without importing the package). Keeping sandboxtest import-free of sandbox is what lets the sandbox package's own tests drive this suite against internal backends (e.g. the null backend, reachable only through an unexported seam) with no import cycle. A drift guard in the sandbox package asserts these mirrored constants stay equal to the originals.
Index ¶
- Constants
- func CheckClaimedImplications(t *testing.T, sut SUT, probes ImplicationProbes)
- func RequireLiveGate(t testing.TB, gate LiveGate)
- func RunSuite(t *testing.T, name string, newSUT Factory)
- type ArgvSUT
- type Factory
- type ImplicationProbe
- type ImplicationProbes
- type ImplicationResult
- type LiveGate
- type SUT
Constants ¶
const ( GuaranteeProcessBoundary uint64 = 1 << iota GuaranteeWriteBoundary GuaranteeReadBoundary GuaranteeEnvScrub GuaranteeNetworkBoundary GuaranteeAddressNetwork GuaranteeResourceLimits GuaranteeTargetNetwork )
Guarantee bits mirror the sandbox package's seam-facing bitmask (SPEC §6, Bit order matches sandbox.Guarantee* exactly; a drift guard in the sandbox package pins the correspondence. They are the machine-readable posture the suite gates every assertion on.
const ( LevelNone uint8 = iota LevelDegraded LevelFull )
Isolation levels mirror the sandbox package's achieved-isolation rollup (SPEC §6). The zero value LevelNone is fail-closed.
Variables ¶
This section is empty.
Functions ¶
func CheckClaimedImplications ¶
func CheckClaimedImplications(t *testing.T, sut SUT, probes ImplicationProbes)
CheckClaimedImplications runs exactly the probes whose guarantee bits are claimed. A claimed guarantee without a probe is a conformance failure; an unclaimed guarantee never executes its probe and imposes no permissiveness requirement.
func RequireLiveGate ¶
RequireLiveGate enforces a fail-closed live-test contract. It returns only when the caller opted in and both worker support and required evidence have been proven.
func RunSuite ¶
RunSuite runs the full conformance suite against newSUT under a named subtest. A consumer typically calls it once per backend they can construct, e.g.:
sandboxtest.RunSuite(t, "live", func(t *testing.T, ws string) sandboxtest.SUT {
profile, err := sandbox.NewProfile(sandbox.ProfileConfig{
WorkspaceRoot: ws, WorkspaceRead: sandbox.Allow,
WorkspaceWrite: sandbox.Allow, HostWrite: sandbox.Deny,
})
if err != nil { t.Fatalf("NewProfile: %v", err) }
set, err := sandbox.NewExecutorSet(profile,
sandbox.WithScratchRoot(t.TempDir()), sandbox.WithMaxExecutors(1))
if err != nil { t.Fatalf("NewExecutorSet: %v", err) }
t.Cleanup(func() { _ = set.Close() })
e, err := set.For("conformance")
if err != nil { t.Fatalf("ExecutorSet.For: %v", err) }
return e
})
Sub-tests use t.Setenv (env-scrub) and therefore do not run in parallel.
Types ¶
type ArgvSUT ¶
type ArgvSUT interface {
RunArgv(ctx context.Context, dir string, argv []string) ([]byte, int, error)
}
ArgvSUT is the optional shell-free execution surface used by platform probe helpers whenever the operation has a direct executable form. Executor implementations should expose it; the smaller SUT remains supported so downstream conformance adapters are not forced to emulate argv execution.
type Factory ¶
Factory builds a fresh, WRITE-CONFINING executor for the given workspace. The contract the suite relies on: the workspace is a writable root, the process's $HOME is NOT writable, and the environment is scrubbed (non-inherit). The factory is invoked once per sub-test — AFTER the suite plants any environment it needs — because an executor snapshots the environment at construction, so a later-planted secret must be visible when the factory builds the executor for the env-scrub check to be meaningful.
type ImplicationProbe ¶
type ImplicationProbe func(context.Context, SUT) (ImplicationResult, error)
ImplicationProbe exercises one property against sut. Implementations own any scenario-specific setup (nested processes, listeners, or requested limits) and must clean it up before returning.
type ImplicationProbes ¶
type ImplicationProbes struct {
Read ImplicationProbe
Process ImplicationProbe
Network ImplicationProbe
AddressNetwork ImplicationProbe
TargetNetwork ImplicationProbe
Resource ImplicationProbe
}
ImplicationProbes supplies the scenario-dependent behavioral checks that a generic executor surface cannot construct by itself. This dependency-inverted seam lets platform suites reuse the claim gating and positive-control rules.
type ImplicationResult ¶
ImplicationResult is the outcome of one end-to-end guarantee probe. A useful negative probe always includes an unconfined positive control, preventing a missing tool or unreachable target from masquerading as enforcement.
type LiveGate ¶
type LiveGate struct {
OptInEnv string
Description string
Supported func() (bool, string)
Evidence func() (bool, string)
}
LiveGate describes an opt-in disposable-worker requirement. A gate that was not requested is reported as explicitly unrun. Once requested, an unsupported worker or missing evidence is a hard failure so CI can never turn a skipped security matrix into a green acceptance result.
type SUT ¶
type SUT interface {
// RunCommand runs a shell command string in dir under the executor's policy
// and returns combined output, the process exit code, and an error that is
// non-nil only when the process did not complete normally (spawn/setup
// failure, signal, or context cancellation) — a ran-but-nonzero command
// returns a nil error and the real code.
RunCommand(ctx context.Context, dir, command string) ([]byte, int, error)
// Level reports the achieved isolation level (LevelNone..LevelFull).
Level() uint8
// GuaranteeBits reports the per-property guarantee bitmask.
GuaranteeBits() uint64
}
SUT is the minimal structural surface the conformance suite exercises on an executor. *sandbox.Executor satisfies it. It is deliberately narrow (interface segregation): the suite spawns commands and reads the achieved posture, nothing more, so any conforming executor — present or future — can be run through it without this package importing the sandbox package.
Source Files