redteam

package
v0.7.9-dev.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 8, 2026 License: MIT Imports: 11 Imported by: 0

Documentation

Overview

Package redteam attacks the registry's own S3/S4 gate: it takes the golden attack corpus, rewrites each attack fixture with the paraphrase and obfuscation tricks red-team harnesses use (garak/PyRIT- style, without a model in the loop so a run is reproducible), scans the rewrites with the same rule bundle `agentry validate` and the pipeline use, and reports which rewrites the expected rule no longer catches. Report-only by construction: nothing here gates anything — the output feeds rule development.

Index

Constants

This section is empty.

Variables

View Source
var DefaultCorpusDirs = []string{"/srv/agentry/attack-corpus", "testdata/attack-corpus"}

DefaultCorpusDirs are tried in order when no corpus directory is configured: the image ships the corpus at the first, a source checkout has it at the second.

Functions

func FindCorpus

func FindCorpus(configured string) (string, error)

FindCorpus resolves the corpus directory.

Types

type Evasion

type Evasion struct {
	Fixture   string   `json:"fixture"`
	Technique string   `json:"technique,omitempty"`
	Mutator   string   `json:"mutator"`
	Missing   []string `json:"missing"`
	Fired     []string `json:"fired"`
}

Evasion is one rewrite an expected rule no longer catches.

type Mutator

type Mutator struct {
	// Name is the stable id reported per case.
	Name string
	// Family groups mutators the way harness taxonomies do.
	Family string
	// Apply rewrites content; a mutator that cannot apply returns ok=false
	// so the case is not counted.
	Apply func(content string) (out string, ok bool)
}

Mutator is one deterministic rewrite of a prose file.

func Mutators

func Mutators() []Mutator

Mutators is the fixed suite, in report order.

type MutatorSummary

type MutatorSummary struct {
	Mutator string `json:"mutator"`
	Family  string `json:"family"`
	Cases   int    `json:"cases"`
	Evaded  int    `json:"evaded"`
}

MutatorSummary is the per-mutator tally.

type Report

type Report struct {
	RulesetSemver string `json:"rulesetSemver"`
	BundleDigest  string `json:"bundleDigest"`
	CorpusDir     string `json:"corpusDir"`
	StartedAt     string `json:"startedAt"`
	DurationMs    int64  `json:"durationMs"`
	// Fixtures is the number of attack fixtures with S3/S4 expectations.
	Fixtures int `json:"fixtures"`
	// Cases is fixtures × applicable mutators; Evaded counts cases where at
	// least one expected rule stopped firing.
	Cases  int `json:"cases"`
	Evaded int `json:"evaded"`
	// ByMutator summarizes per rewrite.
	ByMutator []MutatorSummary `json:"byMutator"`
	// Evasions lists every evaded case (what to look at first).
	Evasions []Evasion `json:"evasions"`
}

Report is one run's outcome.

func Run

func Run(bundle *rules.Bundle, corpusDir string) (*Report, error)

Run scans every attack fixture's rewrites with bundle and reports.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL