Documentation
¶
Overview ¶
Package check judges a declared check against a source tree.
A check is a declared expectation — what must hold, how it is judged (sh, tool, llm, runtime), how severe a failure is, and whether it is about a state of the tree or a transition one change made. Evaluating one needs the declaration, a source tree and three capabilities: run a command in a container, ask a model a checklist, probe a URL. None of that is a project, an adoption, a Change or a review, so none of it is AppTown's: this package knows nothing of AppTown and could be lifted into another program (design/architecture/layout.md, "When something becomes a library"); scripts/shape-check.py's standalone rule holds it to that.
AppTown subsystem internal/apptown/conformance wraps records and decisions around it: which declaration a project adopted, whose exception excuses a finding, what the gate decides over head and base, and what a review recorded.
Index ¶
- Constants
- Variables
- func BundleHash(s string) string
- func Evaluator(c Declaration) string
- func Evaluators() []string
- func FailsAt(failOn, severity string) bool
- func FindingsSummary(tool string, fs []Finding) string
- func Fingerprint(id Identity, check, file string) string
- func IsPostureSh(c Declaration) bool
- func IsRuntime(c Declaration) bool
- func JudgedAtLanding(c Declaration) bool
- func JudgedWhere(c Declaration) string
- func LLMEvaluatorVersion(c Declaration) string
- func ModeFor(c Declaration) string
- func NakedRefusal(code, expect int, summary string) string
- func OccurrenceLines(out string) []string
- func PackEvidence(src string, globs []string) (bundle string, files []string)
- func PageRefusal(status int, path string) string
- func PostureScript(c Declaration) (string, bool)
- func ProbePage(ctx context.Context, p PageProbe) (status int, path string, err error)
- func SARIFMessageField(text, label string) string
- func Scope(c Declaration) string
- func SeverityRank(s string) int
- func TemplateHash() string
- func ToolSummary(tool, output string) string
- func Validate(checks []Declaration) error
- type BrowserExpectation
- type ChecklistMeta
- type Completer
- type CompleterFunc
- type Completion
- type Declaration
- type Declared
- type Evaluation
- type Evidence
- type Fetcher
- type FetcherFunc
- type Finding
- type Identity
- type Item
- type Judge
- func (j *Judge) Checklist(ctx context.Context, items []Item, diff string) (map[int]Verdict, *ChecklistMeta, error)
- func (j *Judge) Evaluate(ctx context.Context, checks []Declared, src Source, mode Mode) ([]Evaluation, error)
- func (j *Judge) EvaluateSh(checks []Declared, src Source, mode Mode, namePrefix string) []Evaluation
- func (j *Judge) EvaluateTool(d Declared, src Source, mode Mode, namePrefix string) Evaluation
- type Judged
- type Location
- type Mode
- type Outcome
- type PackageRef
- type PageProbe
- type ProbeSpec
- type ProbeVerdict
- type Prober
- type ProberFunc
- type Provenance
- type Scanner
- type Scanners
- type ScannersFunc
- type Source
- type Tokens
- type Verdict
Constants ¶
const ( ScopeTree = "tree" ScopeDiff = "diff" )
Check scopes.
const BrowserEnvPath = "/_apptown/env.js"
BrowserEnvPath is the file a static part's server answers with the values AppTown gives its page (window.__APPTOWN_ENV__).
const ChecklistSystem = `` /* 550-byte string literal not displayed */
ChecklistSystem is the prompt template. Its hash is part of the evaluator identity, so an edited template is a changed evaluator.
const DefaultMemoryWithin = 90
DefaultMemoryWithin is memory-within's share when the probe names none.
const PackingVersion = "pack-v2"
PackingVersion changes whenever the packing algorithm below changes — exclusions, caps, ordering, truncation markers. It is part of every AI verdict's evaluator identity.
const ScanTimeout = 5 * time.Minute
ScanTimeout bounds one scanner run.
const ShImage = "alpine/git:v2.47.2"
ShImage is the pinned image every sh check runs in: alpine with git, because the reference for "what does this repository ignore" is git's own matcher (repo/no-build-artifacts asks it of the committed tree) and a tree question is a shell question. The image's ENTRYPOINT is git; runs clear it (NoEntrypoint) so `sh -c` is the command.
const ShTimeout = 60 * time.Second
ShTimeout bounds one check's script.
Variables ¶
var HTTPFetcher = FetcherFunc(func(ctx context.Context, url string) (int, string, error) { resp, err := probeClient.Get(url) if err != nil { return 0, "", err } defer resp.Body.Close() body, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20)) return resp.StatusCode, string(body), err })
HTTPFetcher is the real fetcher: a plain GET, redirects unfollowed, the body read up to a megabyte.
var HTTPProber = ProberFunc(func(_ context.Context, url string) (int, error) { resp, err := probeClient.Get(url) if err != nil { return 0, err } resp.Body.Close() return resp.StatusCode, nil })
HTTPProber is the real prober: a plain GET, redirects unfollowed.
var ProbeTransport http.RoundTripper = trust.Lazy(trust.Deployments)
ProbeTransport is the transport every probe of an app uses: a pool of its own, so a boot's checks still reuse one connection, but one that nothing else in the process closes. A connection pooled on http.DefaultTransport is closed under the probe whenever anything calls CloseIdleConnections on it (every httptest.Server.Close does), and the probe read that as an app that gave no answer. It trusts the deployments use (internal/trust): a preview's or a deployment's address may be signed by a company's proxy or by a desktop trial's own authority.
Functions ¶
func BundleHash ¶
func Evaluator ¶
func Evaluator(c Declaration) string
Evaluator is the single default rule used by validation, descriptions and execution. Explicit declarations win; inferred declarations retain publication precedence (script, rubric, scanner, runtime probe).
func Evaluators ¶
func Evaluators() []string
Evaluators is every evaluator kind a declaration may name. Validate admits exactly these, so a caller that judges declared checks can be held to judging each kind.
func FailsAt ¶
FailsAt says whether a finding of severity fails a scan whose failOn is failOn: every finding when failOn is empty, otherwise one ranked at or above it. It is the one meaning of a scan's failOn; a policy ceiling (wow.FailOnLooseness) orders thresholds by it.
func FindingsSummary ¶
FindingsSummary names the top findings so a reader sees WHAT, not a count.
func Fingerprint ¶
Fingerprint is the stable finding identity shared by gates and review exceptions. Dispositions on a live instance match on it: it does not change.
func IsPostureSh ¶
func IsPostureSh(c Declaration) bool
IsPostureSh / IsRuntime classify an expectation's evaluation mode.
func IsRuntime ¶
func IsRuntime(c Declaration) bool
func JudgedAtLanding ¶
func JudgedAtLanding(c Declaration) bool
JudgedWhere says, in words, where a check is judged — the catalog shows it. JudgedAtLanding says whether a landing can judge the check, so a change can be accepted on it. A runtime check answered only by a standing deployment's steady use is skipped at every landing: no change can prove it, and it is never a change's acceptance.
func JudgedWhere ¶
func JudgedWhere(c Declaration) string
func LLMEvaluatorVersion ¶
func LLMEvaluatorVersion(c Declaration) string
LLMEvaluatorVersion is the evaluator identity of an AI check: packing version, template hash, and the hash of the rubric and its file globs. A changed rubric, file list, template or packing is a changed evaluator.
func ModeFor ¶
func ModeFor(c Declaration) string
func NakedRefusal ¶
NakedRefusal names the cause a header-less probe most likely hit, so the finding tells the fixing agent what to exempt. A redirect is the HTTPS redirect; a 403 is a host allow-list refusing a request that arrived by address (Rails config.hosts, Django ALLOWED_HOSTS); anything else is reported as what it is.
func OccurrenceLines ¶
OccurrenceLines splits a check's output into its occurrences: one per non-blank line, trimmed, duplicates kept (two identical lines are two occurrences — the baseline matches them one for one).
func PackEvidence ¶
PackEvidence collects the files matching globs under src, deterministically (sorted relative paths; excluded dirs, except files a glob names exactly, symlinks, binaries and minified files skipped; per-file and per-rubric caps with explicit markers), and returns the bundle text plus the list of files it contains.
func PageRefusal ¶
PageRefusal says why a front-page answer means a visitor cannot use the app, or "" when it does not: any server error, and the two ways a host allow-list refuses a host it was never told (Django's DisallowedHost is 400, Rails host authorization 403). Any other answer means the app answered: a 401 or a redirect to sign in is a working front page, and a 404 is an API-only app with nothing at its root (greeting-api, pulse-api).
func PostureScript ¶
func PostureScript(c Declaration) (string, bool)
PostureScript returns an expectation's committed-tree sh script and whether it supports posture mode at all. An llm-only rubric has no posture mode.
func ProbePage ¶
ProbePage GETs the app's root as a visitor would and returns the status it ends on and the path that answered. Redirects on the same origin are followed (to the same address, with the same host and headers); one that leaves the origin (a sign-in provider) is the answer as it stands.
func SARIFMessageField ¶
SARIFMessageField: "Label: value" out of a multi-line SARIF message.
func Scope ¶
func Scope(c Declaration) string
Scope resolves a check's scope with the defaults of §2.1.
func SeverityRank ¶
SeverityRank orders severities for failOn thresholds. Unknown ranks as high: a finding nobody scored is not a finding you may silently ship. A note was scored, as the lowest level a check declaration may take.
func TemplateHash ¶
func TemplateHash() string
func ToolSummary ¶
ToolSummary trims a tool's raw output to a one-glance detail line.
func Validate ¶
func Validate(checks []Declaration) error
Validate normalizes and checks a set of declarations, filling the defaults (evaluator, scope, severity) in place.
Types ¶
type BrowserExpectation ¶
type BrowserExpectation struct {
Static bool
// Attached is what the page must be given: the integration settings as
// they are now.
Attached map[string]string
// Installed is what the running container was started with, read from the
// container itself; nil when that could not be established.
Installed map[string]string
}
BrowserExpectation is what the caller knows from trusted sources, not from the response being judged: whether the app's contract serves a static site, what the project's integrations give its page now, and what the running container was given when it started.
type ChecklistMeta ¶
type ChecklistMeta struct {
Served string // the model the provider actually served
PromptHash string
Items int
Cost float64
Tokens Tokens
ParseError bool
Retried bool // the answer was asked for twice: the first was not JSON
}
ChecklistMeta is what a call records about itself.
type Completer ¶
type Completer interface {
Complete(ctx context.Context, system, user string) (Completion, error)
}
Completer asks a model for one answer.
type CompleterFunc ¶
type CompleterFunc func(ctx context.Context, system, user string) (Completion, error)
CompleterFunc adapts a function to Completer.
func (CompleterFunc) Complete ¶
func (f CompleterFunc) Complete(ctx context.Context, system, user string) (Completion, error)
type Completion ¶
type Completion struct {
Text string
Model string // the model the provider actually served
Tokens Tokens
Cost float64
}
Completion is one answer from the model, as the caller's adapter observed it. What it cost and whose budget it lands on stay with the caller.
type Declaration ¶
type Declaration struct {
Name string `json:"name"`
Title string `json:"title,omitempty"` // authored action title for a remediation; identity remains Name
Expects string `json:"expects"` // one plain sentence: what conformance looks like
Severity string `json:"severity,omitempty"` // note|low|medium|high (default note)
Evaluator string `json:"evaluator,omitempty"` // sh|llm|runtime|tool (default sh when sh set, llm when rubric set, tool when tool set)
Sh string `json:"sh,omitempty"`
Rubric string `json:"rubric,omitempty"`
Probe string `json:"probe,omitempty"` // runtime evaluator: a probe against the booted app
Tool string `json:"tool,omitempty"` // tool evaluator: names a scanner binding to run (design/subsystems/conformance.md §13)
// Scope says what the check is ABOUT (design/subsystems/conformance-surface.md §2):
// "tree" — a state of the committed tree, judged at the gate AND in every
// fleet review; "diff" — a transition made by one change, judged at the gate
// only. sh/tool/runtime are always tree. An llm rubric defaults to diff (the
// meaning every existing block version has) and may declare tree, in which
// case Files names the evidence the rubric is judged on.
Scope string `json:"scope,omitempty"`
Files []string `json:"files,omitempty"`
// Occurrences says how the check reports WHAT it found: "lines" — one
// occurrence per output line, so a gate can tell a pre-existing finding
// from a new one (design/subsystems/ways-of-working-versions.md §4.2). Unset: the
// check is boolean; failing on both trees is "regression unknown".
Occurrences string `json:"occurrences,omitempty"`
}
Declaration is what must hold and how it is judged. The JSON tags are the authored form: a way of working carries its checks in this shape.
func ParseDeclarations ¶
func ParseDeclarations(raw string) ([]Declaration, error)
ParseDeclarations reads authored check declarations (a JSON array) and validates them: a key a declaration does not have is refused, so a misspelt field is an error, never a check silently judged without it. Stored declarations were read this way when saved.
type Declared ¶
type Declared struct {
Identity
Declaration
}
Declared is one declaration with the identity that declares it.
type Evaluation ¶
type Evaluation struct {
Declared
Mode Mode
Outcome Outcome
Detail string
Findings []Finding
Occurrences []string
Evidence Evidence
// Fault: the judge failed, not the tree. A change must never be sent back
// to its author over one.
Fault bool
}
Evaluation is one declaration against one source in one mode.
func Judgement ¶
func Judgement(it Item, v Verdict, answered bool, meta *ChecklistMeta, mode Mode) Evaluation
Judgement is what one checklist answer means for the item it answered: the verdict words are the checklist's, the outcome is the judge's. answered says the checklist returned anything at all for this item — an item it passed over is cannot-assess, since silence is not a pass. meta, when given, records what the call cost and what prompt it was.
type Evidence ¶
type Evidence struct {
EvaluatorVersion string // the identity a verdict keys on; unchanged strings
Hash string // hash of the evidence bundle judged
PromptHash string // model calls only
Report string // the report a tool wrote and the judge read; "" = none
Image string // the image (by ID) an exec-image check ran in; "" = none
Files []string
Output string
ExitCode *int
Duration time.Duration
Tokens Tokens
}
Evidence is what the judge recorded about HOW it judged: the evaluator's own identity, the hash of the exact material it looked at, and its output.
type FetcherFunc ¶
FetcherFunc adapts a function to Fetcher.
type Finding ¶
type Finding struct {
Fingerprint string `json:"fingerprint"`
Kind string `json:"kind"` // vuln | secret | sast | license
Severity string `json:"severity"` // critical|high|medium|low|unknown
Identifiers []string `json:"identifiers,omitempty"`
Package *PackageRef `json:"package,omitempty"`
Location *Location `json:"location,omitempty"`
Title string `json:"title"`
Detail string `json:"detail,omitempty"`
URL string `json:"url,omitempty"`
Source string `json:"source"` // binding name
// Reached: osv-scanner call analysis — does the program actually call the
// vulnerable code? nil = no data. VEX-grade context: an unreached vuln is
// the textbook not_affected justification. Never changes the verdict by
// itself; dispositions stay a human act.
Reached *bool `json:"reached,omitempty"`
// Items: the occurrences a conformance check reported (one per line when
// the check declares occurrences: lines), so a gate can subtract what
// already fails on the base tree and count only what is new.
Items []string `json:"items,omitempty"`
}
type Identity ¶
Identity names whoever declared a check, opaquely: AppTown passes the block and the version the project adopted. The library only prints it.
type Item ¶
type Item struct {
Declared
Scope string // tree | diff
Evidence string // packed files for tree; "" for diff (the shared diff is attached once)
Files []string
Hash string // hash of the exact evidence this item is judged on
}
Item is one rubric with the evidence it is judged on.
type Judge ¶
type Judge struct {
// Run executes one container. The caller supplies it rather than a bare
// execution.Runner because a host adds its own cache mounts, labels and
// ownership to every run it owns.
Run func(execution.Request) execution.Outcome
// Complete asks a model for one answer. nil: rubrics cannot be assessed.
Complete Completer
// Probe makes one HTTP request. nil: runtime checks cannot be assessed.
Probe Prober
// Scanners resolves a tool check's name to the tool it names. nil: tool
// checks cannot be assessed.
Scanners Scanners
// RunImage runs a command INSIDE an already-built image, with nothing
// mounted: the exec-image self-check, which must see the app as the app
// sees itself. nil: those checks cannot be assessed.
RunImage func(image string, argv, env []string) (exitCode int, output string)
// Readiness resolves what path a `readiness` probe means for this source —
// the app's own declaration, which only the caller can read. nil or false:
// the check reports does-not-conform, as an app that is served while
// declaring nothing about when it is ready has not met the way of working.
Readiness func(src Source) (string, bool)
// NamePrefix names the containers a run owns, so a host can find them.
NamePrefix string
// Now is the judge's clock, for how long an evaluation took; nil is the
// real clock.
Now func() time.Time
}
A Judge holds the three capabilities evaluating a check can need: run a command in a container, ask a model a checklist, probe a URL. A capability it was not given is not an error — the checks that need it report cannot-assess with the reason.
func (*Judge) Checklist ¶
func (j *Judge) Checklist(ctx context.Context, items []Item, diff string) (map[int]Verdict, *ChecklistMeta, error)
Checklist makes the ONE call for a set of items and returns verdicts by item number (1-based). diff is attached once when any item is diff-scoped. A failed call returns err; the caller records the items as skipped with the reason — conformance never depends on a model being up.
func (*Judge) Evaluate ¶
func (j *Judge) Evaluate(ctx context.Context, checks []Declared, src Source, mode Mode) ([]Evaluation, error)
Evaluate judges every declaration against one source, in one mode.
sh: one container run per check against the staged tree. llm: every rubric in the batch into ONE checklist call, prompt hash and evidence hash recorded. runtime: probe the source's app through both doors. tool: the scanner the check names, from the catalogue the caller gave the judge.
A capability the judge or the source lacks is cannot-assess with the reason, never an error: the tree is not at fault for what the judge could not do.
func (*Judge) EvaluateSh ¶
func (j *Judge) EvaluateSh(checks []Declared, src Source, mode Mode, namePrefix string) []Evaluation
EvaluateSh judges every sh check in the batch against the staged tree, one container run each.
func (*Judge) EvaluateTool ¶
EvaluateTool judges one tool check: the named scanner against the source, or inside the image the source names.
Evaluation.Findings are what the TOOL reported, itemised and normalized — not a roll-up of the check. A caller that excuses individual findings (a recorded exception, a verified false positive) applies that to them and decides what the check then says.
type Judged ¶
type Judged struct {
Evaluation
Provenance Provenance
Own []Finding
PreExisting []Finding
}
Judged is one head evaluation with its provenance: the findings it carries as the base leaves them (an introduced finding names only what is new), split into what this tree introduced and what the base already reported. A finding that can be neither — a boolean check failing on both trees, with no occurrences to compare — is in neither list, and the provenance says Undecided: a reader judges the diff.
func Classify ¶
func Classify(head, base []Evaluation, baseKnown bool) []Judged
Classify judges every head evaluation against what the base tree reported. One rule serves both kinds of finding, because the difference is in what a fingerprint identifies:
A DECLARED CHECK's finding stands for the whole check, so the same fingerprint on the base says only "this check fails there too". The occurrences decide: the ones not on the base are this tree's, and a check with no occurrences to compare is Undecided.
Any other finding — a scanner's — identifies the problem itself, so the base reporting it is proof the problem was already there. Matching prefers the scanner's own stable fingerprint and falls back to the loose key, so a line shift alone never turns carried debt into "new".
A check that judges one property of the whole app and reports no occurrences — a probe of the running app, a model's verdict on a rubric — is PreExisting when the base fails it too: the property was already broken there, and there is nothing to count that the change could have added to. (A shell check without occurrences stays Undecided: its author could have asked it to list what it found.)
A declared check the base could not be judged on (a base evaluation of it that is cannot-assess: main's deployment runs another commit, the model did not answer) leaves its head finding Undecided: nothing says whether the change introduced it, and the caller names why.
baseKnown says the base was evaluated at all: without it nothing can be subtracted and every finding is the head's (Introduced). What the base commit is, whether it is on the protected branch, and which checks are judged absolutely whatever the base says, are the caller's facts — established before the base evaluations are handed over.
type Mode ¶
type Mode int
Mode is what the evaluation is asked to say: a Gate judges one change's proposed tree, a Posture sweep judges a committed tree as it stands.
type Outcome ¶
type Outcome int
Outcome is what a judge can say on its own. A person's decisions over an outcome — excepted, not applicable — are the caller's, never the library's.
func JudgePage ¶
JudgePage asks the front page through the front door: refused is a finding, no answer is the review's inability to look, not the app's fault.
func ProbeBothDoors ¶
func ProbeBothDoors(ctx context.Context, p Prober, frontDoor, naked string, ps ProbeSpec) (state Outcome, detail string)
ProbeBothDoors judges a path through the front door and naked. Both must answer the expected status: the front door because that is what serves, naked because that is what any header-less probe sees (§5.2).
type PackageRef ¶
type PageProbe ¶
PageProbe is one front-page request: the address the app is asked at, and the host name and headers a visitor's request carries through the front door. An empty Host leaves the address's own.
type ProbeSpec ¶
type ProbeSpec struct {
Path string
Expect int
Readiness bool // resolve the path from the contract's declared readiness
BrowserEnv bool // judge the page against AppTown's /_apptown/env.js
Page bool // judge the front page through the front door (JudgePage)
// MemoryDeclared judges the contract, not the running app.
MemoryDeclared bool
// MemoryWithin is the share of its applied limit, in percent, a part of
// the standing deployment must stay under; 0 when the probe is another.
MemoryWithin int
}
ProbeSpec is a parsed `probe` field.
func ParseProbe ¶
type ProbeVerdict ¶
ProbeVerdict is a probe's verdict and, when the verdict is unknown, whether AppTown is answerable for it: Fault means AppTown could not supply or establish what the check depends on, so no change to the app could make it pass. A change must never be sent back to its author over one.
func JudgeBrowserEnv ¶
func JudgeBrowserEnv(ctx context.Context, f Fetcher, base string, want BrowserExpectation) ProbeVerdict
JudgeBrowserEnv judges a running app at base: a static site whose integrations give its page values must have been started with exactly those, must serve exactly what it was started with in /_apptown/env.js, and its HTML must load that file by the supported convention. A server app is not judged. Conforming says the values are served and the markup loads them first; it does not say a browser ran the script (a content security policy can still stop it) or that reporting works.
Why: a static app can adopt error reporting, pass review and ship, and never report, because nothing put the DSN in its page. The judgment is strict because looser ones were fooled: a mention of the path, a <base href>, a template and nomodule must not pass for a loader, and an AppTown delivery fault must be cannot-assess, never a pass or rework for the app.
type ProberFunc ¶
ProberFunc adapts a function to Prober.
type Provenance ¶
type Provenance int
Provenance: how a finding on the head tree reads against the base tree the change forked from (design/subsystems/ways-of-working-versions.md §4.2). A change is judged on what IT did; what already failed on the base is the estate's. Which commit is the base, and whether it is on the protected branch, are facts the caller establishes and passes in.
const ( Introduced Provenance = iota // the change introduced it, or the base cannot say otherwise PreExisting // every occurrence was already on the base Undecided // fails on both trees with no occurrence evidence, or the base could not be judged: unknowable here )
func (Provenance) String ¶
func (p Provenance) String() string
type Scanner ¶
type Scanner struct {
Name string `yaml:"name" json:"name"`
Description string `yaml:"description" json:"description"`
Image string `yaml:"image" json:"image"`
Command []string `yaml:"command" json:"command"`
Report string `yaml:"report" json:"report"`
Network bool `yaml:"network" json:"network"`
// Format enables the structured-findings layer (findings.md): osv | sarif.
// Empty = exit-code contract only. Kind labels SARIF findings (secret|sast|license).
Format string `yaml:"format" json:"format"`
Kind string `yaml:"kind" json:"kind"`
// ArtifactKind overrides the report artifact's kind (default scan-report);
// e.g. "sbom" for syft — evidence of what shipped, not findings.
ArtifactKind string `yaml:"artifactKind" json:"artifactKind"`
// Target: what the scanner looks at — "source" (the committed tree,
// default), "image" (the app's built container image, handed over
// as a saved tarball at /work/.apptown/image.tar; use ${IMAGE_TAR} in
// the command), or "exec-image" (run the command INSIDE the app's built
// image — a framework self-check that must import settings;
// design/subsystems/conformance.md §13.5). Scan POINTS in system workflows
// (`with: {on: image}`) expand to every attached scanner with the matching
// target (design/decisions/workflows-as-lanes.md).
Target string `yaml:"target" json:"target"`
// HistoryCommand: a source scanner that can read Git history checks,
// at a landing, every commit the landing brings to the default branch,
// not only its final tree: this command runs instead of Command, with
// the project's repository mounted read-only at /history and
// ${HISTORY_RANGE} naming those commits (target..candidate). A secret
// added in one commit and removed in the next still lands in history,
// which only a history scan sees.
HistoryCommand []string `yaml:"historyCommand" json:"historyCommand"`
// Manual: this scanner is invoked ONLY by name (a way-of-working `tool`
// check, design/subsystems/conformance.md §13), never auto-expanded at a scan point.
// It keeps a way-of-working-specific tool (a Django linter) from firing
// fleet-wide the way a universal scanner (gitleaks) does.
Manual bool `yaml:"manual" json:"manual"`
}
type Scanners ¶
Scanners is the catalogue a tool check's name is looked up in. Resolving a name to an attached, configured tool is the caller's: the judge only asks.
type ScannersFunc ¶
ScannersFunc makes a catalogue out of one lookup.
type Source ¶
type Source struct {
Tree string // a read-only checkout of the exact commit; the caller stages it
SHA string
Diff string // for diff-scoped rubrics; "" in a review
Image string // built app image reference for exec-image tools; "" = not available
// Env is what an exec-image tool runs with: the environment of a deployed
// app, derived by the caller from what the app declares.
Env []string
App string // base URL of a booted app for runtime probes; "" = not available
// Naked: the same app reached by address with no front-door signal. "" when
// only one door is reachable.
Naked string
}
Source is the material an evaluation judges. A capability the source lacks is cannot-assess with the reason, never an error.
type Tokens ¶
Tokens is what a model call cost, as the judge observed it. What that means in money, and whose budget it lands on, is the caller's business.
type Verdict ¶
type Verdict struct {
Item int `json:"item"`
Verdict string `json:"verdict"`
Evidence string `json:"evidence"`
Note string `json:"note"`
}
Verdict is one item's parsed answer.
func ParseChecklist ¶
ParseChecklist reads the judge's answer as a JSON array of verdicts, tolerating a code fence around it.