rules

package
v0.7.9-dev.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 8, 2026 License: MIT Imports: 21 Imported by: 0

Documentation

Overview

Package rules is the embedded scanner engine that powers both the server-side publication pipeline stages S3 and S4 and the offline `agentry validate` command. Server and CLI run the same code over the same versioned rule bundle, so a local scan is a faithful subset of what the registry runs at publish time: "local/remote parity".

The package has two scanners:

  • S3 (ScanS3) is a set of deterministic detectors for hidden and obfuscated content: zero-width characters, bidi controls (Trojan Source), the Unicode tag block (ASCII smuggling), variation selectors outside a variation sequence (emoji smuggling), private-use runs, mixed-script confusable tokens, HTML comments, CSS-hidden markup, oversized data: URIs, entropy-gated base64/hex blobs, trailing whitespace columns, and NFC-normalization deltas. No configuration: the rules are the code.

  • S4 (Bundle) is a signature engine that loads a versioned YAML rule bundle (see the repo's /rules directory, shipped as ruleset releases) and evaluates regex + suppression rules over prose and prompt files. Rules carry an id, severity, confidence, references, and match / non-match fixtures so every rule ships with tests.

Scan runs both scanners over a file set. Every finding carries the OWASP taxonomy ids (LLM01 prompt injection, ASI06 memory & context poisoning).

Nothing in this package executes, decodes, or renders untrusted content: it only inspects bytes and reports. That property is what lets it run inside the sandboxed scanner task profile and unchanged in the CLI.

Index

Constants

View Source
const (
	// DeclarationUndeclared: the text uses a host, a secret or a program the
	// manifest does not declare. High.
	DeclarationUndeclared = "undeclared"
	// DeclarationInternet: the text reaches internet hosts the manifest
	// declares in network.egress or in its browser's hosts, or names none
	// and egress or a browser host is declared. Medium.
	DeclarationInternet = "declared-internet"
	// DeclarationLocalExec: the text names a declared program outside the
	// base commands, which a hosted run does not provide. Medium.
	DeclarationLocalExec = "declared-local-exec"
	// DeclarationUnnamed: the text names no secret or program, and the
	// manifest declares the capability; what it reaches cannot be told from
	// the text, so it is not rated as a hosted run's. Medium.
	DeclarationUnnamed = "declared-unnamed"
	// DeclarationHosted: every secret, program or server the text names is
	// declared and one a hosted run provides (declared secrets, the base
	// commands, declared MCP servers). Low.
	DeclarationHosted = "declared-hosted"
)

The ratings by declaration a finding can carry in its DeclarationDetail, from the most to the least severe.

View Source
const (
	// TaxonomyLLM01 is OWASP Top 10 for LLM Applications 2025: Prompt Injection.
	TaxonomyLLM01 = "LLM01"
	// TaxonomyASI06 is OWASP Top 10 for Agentic Applications 2026:
	// Memory & Context Poisoning.
	TaxonomyASI06 = "ASI06"
)

OWASP taxonomy ids attached to findings. S3 and S4 both map to prompt injection (LLM01) and memory/context poisoning (ASI06).

View Source
const DeclarationDetail = "declaration"

DeclarationDetail is the Detail key a rated finding carries: what the rating by declaration decided, one of the Declaration* values.

Variables

This section is empty.

Functions

func BrowserHostsAdmit

func BrowserHostsAdmit(hosts []string, host string) bool

BrowserHostsAdmit reports whether a browser's declared hosts admit host: an exact entry, or a "*." entry that host is a name below (V15's grammar; the runner's browse proxy matches them the same way). A name with an empty label (".cdn.example", "a..cdn.example") is no host and is admitted by none.

func IsCapabilityFamily

func IsCapabilityFamily(family string) bool

IsCapabilityFamily reports whether a family's hits are rated by declaration rather than by the bundle.

func ShannonEntropy

func ShannonEntropy(b []byte) float64

ShannonEntropy returns the Shannon entropy (bits per symbol) of b. Exported because internal/scan's S2 secret detector needs the same measure: two implementations of a scoring function that gates findings are two thresholds waiting to drift apart.

func UntarBundle

func UntarBundle(data []byte, write func(name string, content []byte) error) ([]string, error)

UntarBundle reads a Tar() archive and hands each entry to write.

It accepts only flat, regular YAML entries: no directories, no symlinks, no path separators, no "..", and no name that is not a plain file name on Windows too (unportableName). A rule bundle has no directory structure to preserve, so an archive that contains structure did not come from Tar() — it is refused rather than interpreted, which is what keeps an untrusted response from writing outside the directory the caller chose.

Returns the entry names in archive order.

Types

type Bundle

type Bundle struct {
	// contains filtered or unexported fields
}

Bundle is a loaded, compiled, versioned rule set (the S4 engine input). It is the ruleset release artifact, shared byte-for-byte between the server pipeline and the CLI.

func LoadBundle

func LoadBundle(fsys fs.FS, dir string) (*Bundle, error)

LoadBundle loads and compiles a rule bundle from dir within fsys. dir must contain a bundle.yaml manifest and one or more family YAML files, each a document with a top-level `rules:` list. LoadBundle validates every rule (id shape, closed severity/confidence vocab, compilable patterns, id uniqueness, and the presence of the mandatory fixtures) and computes the reproducible digest. It performs no rule evaluation and executes nothing.

func (*Bundle) Digest

func (b *Bundle) Digest() string

Digest returns the bundle's content digest: the hex-encoded SHA-256 over the sorted, byte-exact rule files (including bundle.yaml). Two loads of the same files always yield the same digest, so it is a stable bundle identity for recording on a scan_run and for cache keys.

func (*Bundle) MatchingRules

func (b *Bundle) MatchingRules(text string) []string

MatchingRules returns the ids of the rules that fire on the given text, with the "S4/" prefix. It is used to escalate S3 HTML-comment findings whose hidden text is itself an injection payload and is handy for callers that only need a yes/no per rule (e.g. fixture checks).

func (*Bundle) Notes

func (b *Bundle) Notes() string

Notes returns the bundle's release notes.

func (*Bundle) Rules

func (b *Bundle) Rules() []Rule

Rules returns the authored (uncompiled) rules in load order. The returned slice is a copy; callers may not mutate the bundle.

func (*Bundle) Scan

func (b *Bundle) Scan(path string, content []byte) []Finding

Scan evaluates every rule in the bundle over one prose file and returns the S4 findings, sorted deterministically. Content is treated as data: patterns are matched, never executed. Matching is effectively line-oriented (patterns compile with the multiline flag), and a candidate hit is suppressed when any of the rule's notPatterns matches the same line(s), or any of the bundle's notPatterns matches the containing paragraph (see suppressed) — this is how benign discussion of an attack (security docs, this very engine's own rules) is kept out of the results.

func (*Bundle) Tar

func (b *Bundle) Tar() ([]byte, error)

Tar serializes the bundle as an uncompressed tar archive. Output is deterministic: entries appear in digest order with fixed mode, zero ownership, and a fixed timestamp, so two calls produce identical bytes and the archive itself can be digested.

func (*Bundle) Version

func (b *Bundle) Version() string

Version returns the bundle's semantic version from bundle.yaml.

type Confidence

type Confidence string

Confidence expresses how sure the detector is that the match is a true positive. Low-confidence findings never gate alone; they surface to reviewers.

const (
	// ConfidenceLow marks heuristic matches that need a human look.
	ConfidenceLow Confidence = "low"
	// ConfidenceMedium marks matches with a reasonable false-positive rate.
	ConfidenceMedium Confidence = "medium"
	// ConfidenceHigh marks matches with no legitimate explanation in prose.
	ConfidenceHigh Confidence = "high"
)

Confidence levels.

type Declared

type Declared struct {
	// Egress is the hosts network.egress names, lowercased.
	Egress []string
	// EgressAny is wildcard egress ("*"), which names no host.
	EgressAny bool
	// ServerHosts are the hosts of the remote MCP servers the manifest
	// declares, lowercased: what the release reaches through a declared
	// server, which is low, as every declared server is.
	ServerHosts []string
	// MCPServers reports any declared MCP server, packaged or remote.
	MCPServers bool
	// BrowserHosts are the hosts capabilities.browser.hosts names,
	// lowercased: exact names, or "*." suffixes that admit the names
	// below one and not the bare name (V15). The browser reaches them
	// through the runner's browse proxy. The person's site, named at
	// run time, is not among them: the author does not control it, so a
	// browser of sites alone is no way out for a text that names no
	// host.
	BrowserHosts []string
	// BrowserAny is a browser whose scope is any public site, which names
	// no host and admits any, addresses included, as EgressAny does (the
	// browse proxy dials public addresses only).
	BrowserAny bool
	// Secrets are the declared secret names.
	Secrets []string
	// Exec is the declared commands; nil when exec is none or absent.
	Exec []string
	// FileWrite reports a filesystem.write that declares more than none.
	FileWrite bool
	// BaseCommands are the commands a hosted run provides to every
	// release (manifest.BaseExecCommands). A declared command outside
	// them is one only a run on the person's own computer can provide.
	BaseCommands []string
}

Declared is what a release's manifest declares, as the rating by declaration reads it. The zero value declares nothing.

type Destination

type Destination struct {
	Target string
	Kind   DestinationKind
}

Destination is one place text names: a host (a URL's, an IP address, a bare name) or an email address, lower-cased, without a trailing dot.

func Destinations

func Destinations(text string) []Destination

Destinations returns the destinations text names, each once, in the order URLs, email addresses, IPv4 addresses, bare host names. A URL or an address is taken out of the text before the later patterns run, so its parts are not named again (the userinfo of a disguised URL included).

type DestinationKind

type DestinationKind string

DestinationKind is how a destination was written.

const (
	DestinationURL   DestinationKind = "url"
	DestinationEmail DestinationKind = "email"
	DestinationIP    DestinationKind = "ip"
	DestinationHost  DestinationKind = "host"
)

The ways a destination is written.

type File

type File struct {
	// Path is the repo-relative, slash-separated file path.
	Path string
	// Content is the exact file bytes to scan.
	Content []byte
}

File is one file handed to Scan: a repo-relative path and its raw bytes.

type Finding

type Finding struct {
	// RuleID is the stable identifier of the detector or signature that
	// fired, e.g. "S3/unicode-tag-block" or "S4/instruction-override-001".
	RuleID string
	// Scanner is the pipeline stage that produced the finding: "S3" or "S4".
	Scanner string
	// Severity is the gate-facing severity.
	Severity Severity
	// Confidence is the detector's confidence in the match.
	Confidence Confidence
	// Title is a short human-readable summary.
	Title string
	// FilePath is the repo-relative path of the scanned file.
	FilePath string
	// LineStart is the 1-based first line of the match (0 if not line-scoped).
	LineStart int
	// LineEnd is the 1-based last line of the match (== LineStart for a
	// single-line match).
	LineEnd int
	// Excerpt is a capped, single-line rendering of the offending text with
	// invisible and control characters shown as <U+XXXX> escapes so the hit
	// is legible in reports and terminals.
	Excerpt string
	// Detail carries structured, detector-specific context (counts, decoded
	// code points, matched pattern, and so on). Always non-nil on emitted
	// findings.
	Detail map[string]string
	// Taxonomy holds the OWASP ids for the finding.
	Taxonomy []string
}

Finding is one detector hit on one file. It is the unit the pipeline persists (findings rows) and the CLI prints. Findings are value types with no pointers into scanned content, so they are safe to copy and sort.

func RateByDeclaration

func RateByDeclaration(findings []Finding, d Declared) []Finding

RateByDeclaration re-rates the S4 hits of the capability families against what d declares, and returns findings (the same slice, changed in place). Every other finding is left as the bundle rated it. A rated finding names its rating in Detail[DeclarationDetail], and its Severity is the rating's. Where the text's lines name something the extraction cannot place, it is rated as undeclared: the rating fails toward holding.

func Scan

func Scan(bundle *Bundle, files []File) []Finding

Scan runs the S3 deterministic detectors and the S4 signature engine over a set of files and returns all findings, sorted deterministically. It is the convenience entry point used by both the server pipeline and `agentry validate`.

Scanner reach:

  • S3 runs on every text file.
  • S4 runs only on prose-ish files (.md .markdown .txt .text .yaml .yml .json and extensionless files).

After both scanners run, Scan escalates any S3 HTML-comment finding whose hidden text matches an S4 signature from low to medium severity: a comment that hides an injection payload is worse than one that merely hides prose.

bundle may be nil, in which case only S3 runs (useful for tests and for the hidden-content pass in isolation).

func ScanS3

func ScanS3(path string, content []byte) []Finding

ScanS3 runs every S3 deterministic detector over a single text file and returns the findings sorted deterministically. Non-text (binary) content is skipped — binaries are S1's concern. ScanS3 is pure: same input, same output, no I/O.

type Fixtures

type Fixtures struct {
	// Match holds strings that MUST trigger the rule (recall).
	Match []string `yaml:"match"`
	// NonMatch holds benign look-alike strings that MUST NOT trigger
	// the rule (precision: benign lookalikes).
	NonMatch []string `yaml:"nonMatch"`
}

Fixtures are the required per-rule test strings: every rule ships with match and non-match fixtures. The fixture-runner test asserts that every Match string produces the rule's finding and every NonMatch string does not.

type Manifest

type Manifest struct {
	// Version is the bundle's semantic version, e.g. "0.1.0".
	Version string `yaml:"version"`
	// Notes is free-form release notes for the bundle.
	Notes string `yaml:"notes"`
	// NotPatterns are suppressions applied to EVERY rule in the bundle,
	// in addition to each rule's own. They exist for cross-cutting
	// precision concerns, most importantly descriptive framing: prose
	// that catalogues an attack ("prompts to echo secrets", "look for
	// exfiltration directives") rather than issuing it. An agent
	// *about* prompt-injection defense must remain publishable, and
	// repeating the same alternation in every rule file guarantees it
	// drifts.
	//
	// Unlike per-rule notPatterns, which see only the matched line(s),
	// these are scoped to the containing paragraph (contiguous
	// non-blank lines, a Markdown heading being a paragraph of its
	// own), because descriptive prose wraps and its marker can sit on
	// the line before the match. A blank line or a heading ends the
	// paragraph, so a single descriptive phrase cannot whitelist a
	// whole file.
	NotPatterns []string `yaml:"notPatterns"`
}

Manifest is the bundle.yaml document at the root of a rule bundle. It gives the bundle a semantic version that is recorded on every scan_run and pinned in the CLI release.

type Rule

type Rule struct {
	// ID is the rule identifier, "family-nnn" (e.g. "instruction-override-001").
	// Findings use "S4/<ID>".
	ID string `yaml:"id"`
	// Family groups related rules (one of the six S4 families).
	Family string `yaml:"family"`
	// Title is a short human-readable summary.
	Title string `yaml:"title"`
	// Severity is the finding severity (info|low|medium|high|critical).
	Severity Severity `yaml:"severity"`
	// Confidence is the finding confidence (low|medium|high).
	Confidence Confidence `yaml:"confidence"`
	// Description explains what the rule detects and why.
	Description string `yaml:"description"`
	// References are external citations (OWASP ids, advisories, docs).
	References []string `yaml:"references"`
	// Remediation, where a rule gives one, says what a publisher can change
	// so it no longer fires. It is read for reports only; matching ignores
	// it.
	Remediation string `yaml:"remediation,omitempty"`
	// Patterns are Go regular expressions; a line/text matching any of them is
	// a candidate hit. Case-insensitive by default (see compileRule).
	Patterns []string `yaml:"patterns"`
	// NotPatterns suppress a candidate hit when any of them matches the same
	// line(s) — used to exempt benign discussion of an attack.
	NotPatterns []string `yaml:"notPatterns"`
	// Fixtures are the mandatory match/non-match test strings.
	Fixtures Fixtures `yaml:"fixtures"`
}

Rule is one signature as authored in a YAML family file.

type Severity

type Severity string

Severity ranks how dangerous a finding is. It mirrors the pipeline gate vocabulary.

const (
	// SeverityInfo is advisory only; never gates a publish on its own.
	SeverityInfo Severity = "info"
	// SeverityLow is a minor concern surfaced on the report.
	SeverityLow Severity = "low"
	// SeverityMedium routes a version to human review.
	SeverityMedium Severity = "medium"
	// SeverityHigh routes a version to human review (platform queue if public).
	SeverityHigh Severity = "high"
	// SeverityCritical auto-rejects a version.
	SeverityCritical Severity = "critical"
)

Severity levels, ordered least to most severe.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL