Documentation
¶
Overview ¶
Package adversarial provides offline, deterministic fault-injection / adversarial test packs that ship inside the waza binary.
Packs are simple bundles of YAML task files plus on-disk fixtures, embedded via go:embed. The `waza adversarial` command extracts the selected packs to a temporary directory, synthesizes an EvalSpec, and reuses the normal orchestration pipeline so adversarial runs share the exact same engine adapters, graders, snapshots, gate semantics, and JSON output schema as every other eval.
Each pack ships at least one task and every task is marked golden so that `waza gate` treats unsafe outcomes as hard failures (exit code 2).
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ErrUnknownPack = errors.New("unknown adversarial pack")
ErrUnknownPack is returned when a caller asks for a pack that does not exist among the built-in packs.
var PackFS embed.FS
PackFS is the embedded filesystem holding every built-in pack under data/<pack-name>/.
Functions ¶
Types ¶
type Manifest ¶
type Manifest struct {
Name string `yaml:"name"`
Title string `yaml:"title,omitempty"`
Description string `yaml:"description,omitempty"`
Version int `yaml:"version"`
Tags []string `yaml:"tags,omitempty"`
Tasks []string `yaml:"tasks"`
}
Manifest is the on-disk shape of a pack's pack.yaml.
type Pack ¶
type Pack struct {
Manifest Manifest
// Root is the path inside PackFS where this pack lives, e.g.
// "data/prompt-injection".
Root string
}
Pack is a loaded adversarial pack: its parsed manifest plus the embedded path it lives at. Pack content is read lazily through Extract.
func LoadPack ¶
LoadPack parses the manifest of the named pack and returns it. It does not extract any fixtures or task files to disk; use Extract for that.
func (*Pack) Extract ¶
Extract copies the pack's task files and fixtures into dst. The layout inside dst is:
dst/<pack-name>/pack.yaml dst/<pack-name>/tasks/... dst/<pack-name>/fixtures/...
Returns the absolute pack root under dst.
func (*Pack) TaskRelPaths ¶
TaskRelPaths returns the manifest's task-file paths joined to the extracted pack root. The returned paths use OS separators.
Source Files