sealed

package
v0.134.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 25, 2026 License: Apache-2.0 Imports: 25 Imported by: 0

Documentation

Overview

Package sealed builds what the sealed coverage tier runs against: advisory databases generated from one file of our own text, a container with no network, the expectations a scenario states, and the comparison between those and what a scan produced.

The tier exists to assert exact results. A test against the live databases can only assert structure, because an advisory published tomorrow changes today's answer. Against a database generated from test/integration/testdata/ecosystems/advisories.yaml, a scan either produces the findings the scenario names or it has a bug, and running it with no network makes any fetch the design did not account for fail rather than drift.

Index

Constants

View Source
const Cleared = "<cleared>"

Cleared is what a normalized field is replaced with.

View Source
const FailingToolMessage = "sealed fixture: this program failed on purpose"

FailingToolMessage is what a program replaced by Fail writes before exiting 2.

View Source
const FixtureSuffix = ".fixture"

FixtureSuffix marks a dependency manifest stored under another name. A requirements.txt, go.mod or package-lock.json anywhere in this repository is read by the forge's dependency graph and reported as a dependency of Draugr, vulnerabilities included; stored with the suffix it is a file nothing parses, and Prepare restores its name in the copy a scan reads.

View Source
const Image = "ubuntu:24.04@sha256:008173c23f95b170204355c12626cb5a965d779a7e1283b09e9cffbb1bf33ca3"

Image is the container the sealed tier runs in, pinned by digest. It supplies only the root of a filesystem: /usr, /etc and /opt are the host's, mounted read-only, so the tools, interpreters and libraries inside are the ones installed on the machine running the test, and the only thing the container takes away is the network.

View Source
const TrivySchemaVersion = 2

TrivySchemaVersion is the database schema the pinned Trivy reads. A Trivy that moves to another fails to open this database rather than reading it wrongly; bump it with the tool and regenerate.

Variables

This section is empty.

Functions

func AWSAccessKeyID

func AWSAccessKeyID() (string, error)

AWSAccessKeyID returns a random string in the shape of an AWS access key id, which no AWS account issued. Regenerated until its entropy clears the threshold secret scanners apply, so a low-entropy draw cannot make a scenario fail by chance.

func Check

func Check(exp Expected, anns []Annotation, got []Finding) ([]string, error)

Check compares what a scan reported with what the scenario expects, and returns one line per difference: an expected result that is missing, a result nothing expected, and a result on a line an annotation marks as one its rule must not report.

func CheckErrors

func CheckErrors(exp []ErrorExpectation, report []byte) ([]string, error)

CheckErrors compares the controls a report says could not run with the ones the scenario expects to fail, one line per difference. A control that errors unexpectedly is a difference, and so is an expected failure that ran, or failed for another reason.

func CheckInit

func CheckInit(exp, got InitExpectation) []string

CheckInit compares what `draugr init` wrote with what the scenario expects, one line per difference.

func Diff

func Diff(want, got string) string

Diff renders the lines that differ between two documents, as removals and additions around the longest run of lines they share, capped at a screenful.

func RunReplacements

func RunReplacements(work string, start time.Time) map[string]string

RunReplacements are the strings particular to one sealed run: its directory and the dates the run could have stamped.

func ValidateSARIF

func ValidateSARIF(raw []byte) error

ValidateSARIF checks a document against the SARIF 2.1.0 schema.

The published schema declares JSON Schema draft-04, which the validator does not read. The two differences that matter are its "$schema" and its top-level "id", renamed "$id" in later drafts; the schema uses no other draft-04 construct, so both are rewritten on load and nothing else is.

func WriteGoVulnDB

func WriteGoVulnDB(home string, advs Advisories, fetchedAt time.Time) error

WriteGoVulnDB writes the Go advisories in advs as a Go vulnerability database in the offline layout govulncheck's -db reads (index/db.json, index/modules.json, index/vulns.json, ID/<id>.json) into home/.draugr/feeds/govulndb, and records it in the feed manifest as fetched at fetchedAt, which is where a scan looks for it.

func WriteRetireRepo

func WriteRetireRepo(home string, advs Advisories, fetchedAt time.Time) error

WriteRetireRepo writes the js advisories in advs as a retire.js repository at home/.draugr/data/retirejs/jsrepository.json, with the cache index naming it, which is the copy Draugr hands retire.js with --jsrepo when offline. Each library is identified by its file name and by a banner comment.

func WriteTrivyDB

func WriteTrivyDB(cacheDir string, advs Advisories) error

WriteTrivyDB writes a Trivy vulnerability database holding advs into cacheDir/db, the layout `trivy --cache-dir <cacheDir> --skip-db-update` reads. Advisories in an ecosystem Trivy does not cover (js) are left out.

Types

type Advisories

type Advisories struct {
	Advisories []Advisory `yaml:"advisories"`
}

Advisories is the parsed advisories file.

func LoadAdvisories

func LoadAdvisories(path string) (Advisories, error)

LoadAdvisories reads and checks an advisories file.

func (Advisories) For

func (a Advisories) For(ecosystem string) []Advisory

For returns the advisories in one ecosystem, in file order.

type Advisory

type Advisory struct {
	// ID is the identifier a scanner reports: a CVE where there is one.
	ID string `yaml:"id"`
	// GoID is the Go vulnerability database's own identifier, for an advisory govulncheck reads.
	GoID string `yaml:"goID,omitempty"`
	// Ecosystem is the package ecosystem: pip, npm, go, rubygems, cargo, composer, nuget, maven,
	// or js for a library retire.js identifies by its file.
	Ecosystem string `yaml:"ecosystem"`
	// Package is the name the ecosystem gives the package.
	Package string `yaml:"package"`
	// Fixed is the first version without the vulnerability; every earlier version is affected.
	Fixed string `yaml:"fixed"`
	// Severity is CRITICAL, HIGH, MEDIUM or LOW.
	Severity string `yaml:"severity"`
	// Title is a one-line summary, written for the fixture.
	Title string `yaml:"title"`
	// Symbols maps a Go package path to the functions the vulnerability is in, which is what
	// govulncheck follows to decide whether it is reachable.
	Symbols map[string][]string `yaml:"symbols,omitempty"`
	// CWE is the weakness, which retire.js requires on every entry.
	CWE string `yaml:"cwe,omitempty"`
}

Advisory is one vulnerability the sealed databases carry, in the terms every generator needs.

type Annotation

type Annotation struct {
	// File is relative to the repository, with any fixture suffix removed.
	File string
	// Line is the line the annotation is about: the first line after it that is neither blank nor
	// another annotation.
	Line int
	Rule string
	// Want is true for `ruleid:` and false for `ok:`.
	Want bool
}

Annotation is an expectation written in a fixture's own code, the way Semgrep tests its rules: a comment naming a rule that must (`ruleid:`) or must not (`ok:`) report the next line of code.

func ReadAnnotations

func ReadAnnotations(dir string) ([]Annotation, error)

ReadAnnotations collects the annotations in every file under dir, skipping vendor/ and node_modules/, which hold other people's code.

type ComponentExpectation

type ComponentExpectation struct {
	Name string `yaml:"name"`
	// Repositories are the repository urls, relative to the descriptor.
	Repositories []string `yaml:"repositories"`
	// Paths and Ignore are the scope its repositories declare. Empty means the whole repository.
	Paths  []string `yaml:"paths"`
	Ignore []string `yaml:"ignore"`
}

ComponentExpectation is one component `draugr init` declares.

type Container

type Container struct {
	// Work is the one writable directory, mounted at the same path inside. The run's HOME is
	// Work/home, so every cache and dataset Draugr reads is the one the test put there.
	Work string
	// HostHome is the invoking user's home directory, mounted read-only for the tools installed
	// under it.
	HostHome string
	// Path is the PATH inside the container.
	Path string
	// Extra are further read-only mounts, such as a GOROOT outside the directories already mounted.
	Extra []string
	// Env is added to the environment inside the container.
	Env map[string]string
	// UID and GID are who the container runs as, so what it writes belongs to the invoking user.
	UID, GID int
}

Container describes one sealed run.

func HostContainer

func HostContainer(work string) (Container, error)

HostContainer describes a sealed run over work using the invoking user's tools: PATH as it is, behind the tools Draugr installed and the directory of the semgrep on PATH.

Semgrep's CLI runs a second program, pysemgrep, by looking it up on PATH. Beside the semgrep binary is the one from the same installation; a stale one elsewhere on PATH belongs to another interpreter, which the host may find through its own home directory and the container cannot.

func (Container) Args

func (c Container) Args(dir string, argv ...string) []string

Args is the docker command line that runs argv sealed, starting in dir.

func (Container) Command

func (c Container) Command(dir string, argv ...string) *exec.Cmd

Command is Args as a command ready to run.

func (*Container) Fail

func (c *Container) Fail(tool string) error

Fail replaces tool inside the container with a program that writes FailingToolMessage to stderr and exits 2.

func (*Container) Hide

func (c *Container) Hide(tool string) error

Hide removes tool from PATH inside the container. Each directory on PATH that holds it is replaced by a copy made of links to everything else in it, so every other program stays where it was.

func (Container) Home

func (c Container) Home() string

Home is the HOME a sealed run sees.

type ErrorExpectation

type ErrorExpectation struct {
	Control string `yaml:"control"`
	// Contains is text the control's error must include.
	Contains string `yaml:"contains"`
}

ErrorExpectation is one control that must fail to run.

type Expected

type Expected struct {
	// Init is what `draugr init` writes for the repository.
	Init InitExpectation `yaml:"init"`
	// Secrets are credentials the harness writes into the repository before it is committed.
	// Each must be reported by the secrets control.
	Secrets []SecretExpectation `yaml:"secrets"`
	// Findings are every other result the scan must report, and together with the secrets and the
	// fixture's inline sast annotations, every result it may report.
	Findings []FindingExpectation `yaml:"findings"`
	// Sealed changes what the container provides, for a scenario about a failure.
	Sealed RunOptions `yaml:"sealed"`
	// Errors are the controls that must fail to run, each with text its error must contain. Every
	// other control must run.
	Errors []ErrorExpectation `yaml:"errors"`
}

Expected is a scenario's expected.yaml: what `draugr init` proposes for the repository and what a sealed scan of it reports.

func (Expected) LeavesFieldsUnwritten added in v0.134.0

func (e Expected) LeavesFieldsUnwritten() bool

LeavesFieldsUnwritten reports whether a scan of the scenario leaves out fields the normalizers clear: a control that failed to start writes no scanner version, and a scan that finds nothing writes no result to take a fingerprint from.

type Finding

type Finding struct {
	Control      string
	Tool         string
	Rule         string
	File         string
	Line         int
	Package      string
	Reachability string
}

Finding is one SARIF result, reduced to what an expectation can name.

func Observe

func Observe(sarif []byte) ([]Finding, error)

Observe reads the findings in a SARIF document.

func (Finding) String

func (f Finding) String() string

type FindingExpectation

type FindingExpectation struct {
	Control string `yaml:"control"`
	// Tool is the scanner that reported it, as the SARIF names it.
	Tool string `yaml:"tool"`
	Rule string `yaml:"rule"`
	// Location is file:line relative to the repository, or the file alone for a result about a
	// whole file.
	Location string `yaml:"location"`
	// Package is "ecosystem name version", for a finding about a dependency.
	Package string `yaml:"package,omitempty"`
	// Reachability is the verdict reachability analysis reached, where it ran.
	Reachability string `yaml:"reachability,omitempty"`
}

FindingExpectation is one result a scan must report.

type InitExpectation

type InitExpectation struct {
	// Controls are the controls it enables, in any order.
	Controls []string `yaml:"controls"`
	// Scanners are the scanners it enables inside a control beyond the control's default, as
	// control.scanner (sast.gosec, sca.retirejs), in any order. Empty means none.
	Scanners []string `yaml:"scanners"`
	// Reachability are the reachability analyzers it turns on. Empty means none.
	Reachability []string `yaml:"reachability"`
	// Components are the components it declares.
	Components []ComponentExpectation `yaml:"components"`
	// PerDirectory is what `draugr init --per-directory` writes, for a scenario whose tree has
	// directories with their own dependency files. Unset skips that run.
	PerDirectory *InitExpectation `yaml:"perDirectory"`
}

InitExpectation is the descriptor `draugr init` writes.

func ObserveInit

func ObserveInit(path string) (InitExpectation, error)

ObserveInit reads the descriptor `draugr init` wrote, through the loader a scan uses, as the controls and scanners it enables and the components it declares.

type Normalizer

type Normalizer struct {
	// Clear are paths to fields replaced with Cleared. "*" matches every element of an array or
	// every value of an object. Each path must match at least one field.
	Clear [][]string
	// Sort are paths to arrays sorted by their elements' JSON.
	Sort [][]string
	// Replace maps a string to what it becomes, in every string value.
	Replace map[string]string
	// AllowMissing accepts a Clear path that matches nothing. For a run where a control failed to
	// start, which leaves no scanner version and no finding for that control to clear.
	AllowMissing bool
}

Normalizer makes a report comparable across runs: it clears the fields that vary by machine or by moment, replaces run-specific strings wherever they appear, and sorts the arrays whose order depends on which job finished first.

A path that matches nothing is an error rather than a no-op. A field that moved would otherwise stop being cleared without anybody noticing, and the golden would start pinning a value that changes on the next run, or would stop pinning anything at the old path.

func ReportNormalizer

func ReportNormalizer(replace map[string]string) Normalizer

ReportNormalizer is the normalizer for Draugr's report.json.

func SARIFNormalizer

func SARIFNormalizer(replace map[string]string) Normalizer

SARIFNormalizer is the normalizer for Draugr's results.sarif.

func (Normalizer) Apply

func (n Normalizer) Apply(raw []byte) ([]byte, error)

Apply normalizes a JSON document and returns it indented, with a trailing newline.

type RunOptions

type RunOptions struct {
	// WithoutTool is a program removed from PATH inside the container.
	WithoutTool string `yaml:"withoutTool"`
	// FailingTool is a program replaced by one that writes FailingToolMessage and exits 2.
	FailingTool string `yaml:"failingTool"`
	// WithoutTrivyDB leaves Trivy's vulnerability database unwritten.
	WithoutTrivyDB bool `yaml:"withoutTrivyDB"`
	// GoVulnDBAge is how long before the run the local Go vulnerability database is recorded as
	// fetched, as a Go duration. Empty means at the start of the run.
	GoVulnDBAge string `yaml:"goVulnDBAge"`
}

RunOptions take something away from a sealed run, so a scenario can assert that its absence is reported rather than passed.

type Scenario

type Scenario struct {
	// Name is the directory name, which is also the component and the repository's directory.
	Name string
	// Dir is the scenario directory.
	Dir string
	// Expected is the parsed expected.yaml.
	Expected Expected
}

Scenario is one directory under ecosystems/: a repository to scan, the descriptor to scan it with, and what the scan must produce.

func LoadScenario

func LoadScenario(dir string) (Scenario, error)

LoadScenario reads the scenario in dir.

func (Scenario) Prepare

func (s Scenario) Prepare(work string) (string, error)

Prepare copies the scenario's repository to work/<name>, restores the names of its manifests, writes the secrets the scenario asks for, and commits the result. It returns the repository directory.

The secrets are generated here and never committed to this repository: a credential-shaped string in a public tree is refused by push protection and reported by every scanner that reads it, including the one being tested.

type SecretExpectation

type SecretExpectation struct {
	// File is where the harness writes it, relative to the repository. It is written on line 1.
	File string `yaml:"file"`
	// Rules are the rules that must report it.
	Rules []string `yaml:"rules"`
}

SecretExpectation is one generated credential.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL