canaryscan

package
v0.3.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 31, 2026 License: Apache-2.0 Imports: 24 Imported by: 0

Documentation

Overview

CANARY: REQ=CP-268; FEATURE="MermaidRefs"; ASPECT=Engine; STATUS=TESTED; TEST=TestCANARY_CBIN_202_ExtractDiagramRefs,TestCANARY_CBIN_202_ScanDiagramRefsSkipsOversizedFile; UPDATED=2026-08-30

CANARY: REQ=ENG-4325; FEATURE="MigrateNotes"; ASPECT=Engine; STATUS=TESTED; TEST=TestCANARY_CBIN_301_ExtractMigrateNotes; UPDATED=2026-08-29

CANARY: REQ=CP-277; FEATURE="StalenessConfig"; ASPECT=Engine; STATUS=TESTED; TEST=TestCANARY_CBIN_304_StaleDaysFromConfig,TestCANARY_CBIN_304_UpdateStaleReportsEvidenceCurrency,TestCANARY_CBIN_304_UpdateStaleMutatesNothing; UPDATED=2026-08-30

Index

Constants

View Source 
const (
	MaxFileBytes  = 16 << 20 // 16 MiB per file
	MaxLineBytes  = 1 << 20  // 1 MiB per logical line
	MaxFieldBytes = 1024     // per token field value, UTF-8 bytes
	// ParserSchemaVersion identifies the token grammar/semantics. Bumped to
	// 2 when STATUS became a pure declaration (no lexical promotion).
	ParserSchemaVersion = 2
)

Scanner and token-grammar limits. These bound what a single scan will read and what a single token field may hold, so a hostile or accidental input (a multi-gigabyte artifact, a minified one-line bundle) cannot exhaust memory or wedge a scan.

View Source 
const (
	IssueBinary       = "binary"
	IssueFileTooLarge = "file_too_large"
	IssueLineTooLarge = "line_too_large"
	IssueReadError    = "read_error"
	IssueParseError   = "parse_error"
)

Scan issue reasons.

View Source 
const DefaultStaleDays = config.DefaultStaleDays

DefaultStaleDays is the fallback staleness window (in days) when neither Config.StaleDays nor .canary/project.yaml's verification.staleness_days is set. It is pkg/config's constant under canaryscan's historical name.

View Source 
const EvidenceFile = ".canary/evidence.json"

EvidenceFile is the evidence store's path relative to a project root.

Variables

View Source 
var UndeclaredKey = evidence.FeatureKey{Feature: "*", Aspect: "*"}

UndeclaredKey is the stand-in feature/aspect used for a claimed requirement that declares no tokens at all. Such a claim can never be satisfied by evidence — there is nothing for evidence to attest to — so it is reported as one missing entry rather than passing vacuously.

Functions

func AnnotateSources 

func AnnotateSources(rep *Report, reg *sources.Registry)

AnnotateSources stamps each requirement with its source name and ticket URL.

func Claims added in v0.3.3 

func Claims(gapPath string, reg *sources.Registry) ([]string, error)

Claims returns the requirement IDs claimed in gapPath, in first-seen order with duplicates removed. The claim grammar is reg.ClaimPattern(): "✅ <KEY>-<digits>"; a nil registry means the default (CBIN).

func DeclaredFeatures added in v0.3.3 

func DeclaredFeatures(rep Report, reqID string) []evidence.FeatureKey

DeclaredFeatures returns rep's declared feature/aspect keys for reqID, deduplicated. Two tokens differing only in OWNER or UPDATED describe one thing to prove, so the key is required once, not twice. An id with no declarations returns nil.

func DefaultSkipRegex 

func DefaultSkipRegex() *regexp.Regexp

DefaultSkipRegex returns the default skip path regex.

func HeadCommit added in v0.3.3 

func HeadCommit(root string) (string, error)

HeadCommit returns root's current commit SHA. Evidence binds to a commit, so a repository whose HEAD cannot be read cannot be verified.

func LoadCanaryIgnore 

func LoadCanaryIgnore(root string) (*ignore.GitIgnore, error)

LoadCanaryIgnore loads .canaryignore from root. Returns nil if file missing.

func RefTimeFromEnv 

func RefTimeFromEnv() time.Time

RefTimeFromEnv returns time from CANARY_TEST_TIMESTAMP (RFC3339) for tests; zero if unset or invalid.

func ReportEvidenceCurrency added in v0.3.3 

func ReportEvidenceCurrency(rep Report, staleDiags []string, recs []evidence.Record, projectID, commit string) []string

ReportEvidenceCurrency answers, for every requirement named by staleDiags, whether it has passing evidence at the current commit.

This replaces the old --update-stale behavior, which rewrote UPDATED= dates in source. Rewriting a date made a stale claim *look* fresh without any new proof — the exact failure mode evidence-backed verification exists to prevent. Nothing is mutated here: the report is the whole output, one line per requirement, sorted by ID: 

CANARY_UPDATE_STALE req=<id> evidence=current
CANARY_UPDATE_STALE req=<id> evidence=missing

"current" means every feature/aspect the requirement declares has a PASS record for this project at this commit; anything less is "missing".

func RequiredFeatures added in v0.3.3 

func RequiredFeatures(rep Report, claimed []string) map[string][]evidence.FeatureKey

RequiredFeatures maps each claimed requirement to the feature/aspect pairs it declares in rep. A claimed requirement rep knows nothing about maps to a single UndeclaredKey entry so it is still checked (and fails), never silently skipped.

func Run 

func Run(cfg Config, stdout, stderr io.Writer) (exitCode int)

Run runs the full scan pipeline: load config, scan, optional update-stale, write outputs, verify, strict. It writes to stdout/stderr via the given writers (use os.Stdout, os.Stderr from CLI). Returns exit code: 0 success, 2 verify/staleness failure, 3 parse/IO error.

func RunFromArgs 

func RunFromArgs(root, out, csv, verifyPath, skipExpr string, strict, updateStale, projectOnly bool, staleDays int) int

RunFromArgs is a convenience that builds Config from flag-like args and runs with os.Stdout/os.Stderr. Used by tools/canary main. skipExpr is the regex string; if empty, default is used. staleDays overrides the staleness window in days; 0 means "use config/default".

func ScanDiagramRefs 

func ScanDiagramRefs(root string, skip *regexp.Regexp, reg *sources.Registry, ignorePatterns *ignore.GitIgnore) ([]DiagramRef, []ScanIssue, error)

ScanDiagramRefs walks root for .md/.markdown/.mmd files and extracts all requirement-ID mentions inside mermaid diagrams. Paths in the result are root-relative with forward slashes. ignorePatterns (from LoadCanaryIgnore) is honored the same way Scan honors it: relative path matched first, dirs skipped via SkipDir; nil means no .canaryignore patterns apply.

A file that cannot be read is reported as a ScanIssue rather than silently skipped, so callers can tell an empty result from an incomplete one.

func ScanDigest added in v0.3.3 

func ScanDigest(files []FileDigest) string

ScanDigest folds a file-digest set into one digest over the sorted "path:hash" lines, so two scans of the same tree produce the same value and any change to any indexed file changes it.

func ScanMigrateNotes added in v0.3.0 

func ScanMigrateNotes(root string, skip *regexp.Regexp, ignorePatterns *ignore.GitIgnore, reg *sources.Registry) ([]MigrateNote, []ScanIssue, error)

ScanMigrateNotes walks root and extracts CANARY:MIGRATE guidance notes from every non-binary file the scanner would otherwise read. Paths in the result are root-relative with forward slashes. ignorePatterns (from LoadCanaryIgnore) is honored the same way Scan honors it: relative path matched first, dirs skipped via SkipDir; nil means no .canaryignore patterns apply. Files larger than 1MB are skipped.

A file that cannot be read is reported as a ScanIssue rather than silently skipped, so callers can tell an empty result from an incomplete one.

func ScanSummaryLine 

func ScanSummaryLine(rep Report) string

ScanSummaryLine returns a single parseable line for rep so agents get metrics without reading status.json.

func ScanTokenRecords added in v0.3.3 

func ScanTokenRecords(root string, skip *regexp.Regexp, projectFilter *regexp.Regexp, ignorePatterns *ignore.GitIgnore, reg *sources.Registry) ([]TokenRecord, []FileDigest, []ScanIssue, error)

ScanTokenRecords walks root and returns every CANARY token it finds, one record per token, alongside the digest of every file that contributed at least one token and the issues the scan hit. skip, projectFilter, ignorePatterns and reg may be nil.

Issues are returned rather than swallowed: a caller that is building an index (as opposed to printing a report) can refuse to commit anything when the tree contains a token the parser rejects.

func SerializeToken added in v0.3.3 

func SerializeToken(fields []Field) (string, error)

SerializeToken renders canonical form: `// CANARY: K=V; K="quoted"`. Values that could otherwise be misread are quoted with `\`→`\\` and `"`→`\"` escapes; `;` is legal only inside quotes. Control characters (C0 and DEL), oversized values, unknown ASPECT/STATUS enum members and malformed REQ ids are refused.

func Stale 

func Stale(rep Report, maxAge time.Duration, refTime time.Time) []string

Stale returns diagnostics for TESTED/BENCHED tokens older than maxAge. If refTime is zero, time.Now().UTC() is used.

func StateSkipRegex added in v0.3.3 

func StateSkipRegex() *regexp.Regexp

StateSkipRegex returns the default skip path regex extended with canary's own `.canary` state directory. See stateSkipPattern.

func VerifiedRequirements added in v0.3.3 

func VerifiedRequirements(rep Report, recs []evidence.Record, projectID, commit string) []string

VerifiedRequirements returns, sorted, the IDs of every requirement in rep whose declared features all have passing evidence for projectID at commit. It asks pkg/evidence.Complete -- the single completion function -- once per requirement, so a peer's export and this project's own `canary verify` answer the same question the same way.

The result is always non-nil (an empty slice when nothing is verified) so a caller can hand it straight to Report.Verified, where nil carries the distinct meaning "not checked".

func VerifyClaims 

func VerifyClaims(rep Report, gapPath string, reg *sources.Registry, recs []evidence.Record, projectID, commit string) []string

VerifyClaims reads the GAP file and returns one diagnostic per claimed requirement that lacks passing evidence at commit. The decision is made by evidence.Complete — the single completion function — never by inspecting declared STATUS values: a token saying TESTED proves nothing.

An empty claims file yields no diagnostics here (allowEmpty), preserving `scan --verify`'s historical contract; `canary verify` is where an empty claims file is itself a failure. CANARY: REQ=ENG-4322; FEATURE="TicketSources"; ASPECT=Engine; STATUS=TESTED; TEST=TestCANARY_CBIN_201_VerifyClaimsTicketSource; UPDATED=2026-08-30 CANARY: REQ=CP-236; FEATURE="EvidenceVerify"; ASPECT=Engine; STATUS=TESTED; TEST=TestVerifyClaims_NoEvidenceFails,TestVerifyClaims_CurrentEvidencePasses,TestVerifyClaims_UndeclaredClaimFails,TestVerifyClaims_EmptyClaimsFileIsSilent,TestVerifyClaims_UnreadableGapFileReportsParseError; UPDATED=2026-08-30

func WriteCSV 

func WriteCSV(path string, rep Report) error

WriteCSV writes rep to path as CSV, atomically, one row per (requirement, feature, file). A feature's tests and benches are joined with "|" inside single cells rather than zipped column-wise against the file list: the old zip paired unrelated files, tests, and benches by shared row index, asserting adjacencies that never existed. Every cell is passed through csvGuard so no value can be interpreted as a spreadsheet formula, and encoding/csv quotes any value containing a comma, quote, or newline so the output is decodable by an independent reader.

func WriteJSON 

func WriteJSON(path string, rep Report) error

WriteJSON writes rep to path as JSON, atomically. The encode targets an in-memory buffer (which cannot short-write), and the file is committed by writeOut, whose error -- a failed create, write, or close -- is returned: a partial or unwritable report never passes silently.

Types

type Config 

type Config struct {
	Root        string
	Out         string
	CSV         string
	VerifyPath  string
	Strict      bool
	SkipRegex   *regexp.Regexp
	UpdateStale bool
	ProjectOnly bool
	// StaleDays overrides the staleness window (in days) used by --strict and
	// --update-stale. When <= 0, the effective threshold falls back to
	// .canary/project.yaml's verification.staleness_days, then to 30 days.
	StaleDays int
}

Config holds scanner run options. Used by both the standalone tools/canary binary and the canary scan CLI subcommand so scan works from any CWD.

type DiagramRef 

type DiagramRef struct {
	ReqID string
	File  string
	Line  int // 1-based
}

DiagramRef records one requirement-ID mention inside a mermaid diagram.

func ExtractDiagramRefs 

func ExtractDiagramRefs(relPath, content string, reg *sources.Registry) []DiagramRef

ExtractDiagramRefs finds requirement IDs inside ```mermaid fenced blocks. relPath ending in .mmd is treated as a whole-file mermaid diagram (no fences required). IDs are normalized through reg (flatfile padding); reg nil means sources.Default() (the legacy CBIN series).

type Feature 

type Feature struct {
	Feature string   `json:"feature"`
	Aspect  string   `json:"aspect"`
	Status  string   `json:"status"`
	Files   []string `json:"files"`
	Tests   []string `json:"tests"`
	Benches []string `json:"benches"`
	Owner   string   `json:"owner,omitempty"`
	Updated string   `json:"updated"`
	// Priority carries the token's declared PRIORITY (1 = highest). It is a
	// pointer so "declared nothing" stays distinguishable from "declared 0":
	// a consumer that needs a number applies its own default (5, the one
	// `canary index` uses) only when this is nil, rather than inventing one
	// for a token that did declare a value. An unparsable PRIORITY reads as
	// absent, matching `canary index`'s leniency.
	//
	// Additive: absent from a token means absent here.
	Priority *int `json:"priority,omitempty"`
	// DependsOn carries the token's DEPENDS_ON requirement IDs verbatim, so
	// verification can resolve external/peer dependency state without
	// re-parsing the tree. Additive: absent from a token means absent here.
	DependsOn []string `json:"depends_on,omitempty"`
}

Feature is one feature/aspect/status aggregation.

type Field added in v0.3.3 

type Field struct{ Key, Value string }

Field is one key/value pair of a CANARY token, in source order. Values are held decoded: quoting and escaping belong to the wire form only.

func ParseTokenLine added in v0.3.3 

func ParseTokenLine(line string) (fields []Field, ok bool, err error)

ParseTokenLine parses one line; ok=false when the line holds no CANARY token. CANARY:MIGRATE guidance lines are free text, not KV tokens, and so report ok=false as well. A malformed token returns ok=true with err set.

type FileDigest added in v0.3.3 

type FileDigest struct {
	Path string
	Hash string
}

FileDigest pairs a scanned file with the digest of its contents.

type MigrateNote added in v0.3.0 

type MigrateNote struct {
	File   string   `json:"file"`
	Line   int      `json:"line"` // 1-based
	Text   string   `json:"text"`
	ReqIDs []string `json:"req_ids,omitempty"` // registry-pattern matches inside Text, normalized
}

MigrateNote is one CANARY:MIGRATE guidance comment.

func ExtractMigrateNotes added in v0.3.0 

func ExtractMigrateNotes(relPath, content string, reg *sources.Registry) []MigrateNote

ExtractMigrateNotes finds CANARY:MIGRATE guidance comments in content. relPath is recorded verbatim on each note. reg nil means sources.Default() (the legacy CBIN series). A MIGRATE line is free text, not a KV token, and is never fed to parseKV — this is what keeps a MIGRATE line from ever aborting a scan.

type Report 

type Report struct {
	GeneratedAt    string        `json:"generated_at"`
	Requirements   []Requirement `json:"requirements"`
	Summary        Summary       `json:"summary"`
	MigrationNotes []MigrateNote `json:"migration_notes,omitempty"`
	Issues         []ScanIssue   `json:"issues,omitempty"`
	// Verified lists, sorted, the requirement IDs every one of whose declared
	// features had passing evidence at the scanned commit. It is the export a
	// peer project reads to answer "is this requirement done?" -- declarations
	// alone (STATUS=TESTED) never answer that.
	//
	// It is a pointer so absence is distinguishable from emptiness: nil means
	// this scan could not check evidence at all (no store, or no commit to
	// bind it to) and the key is omitted, which a reader must treat as
	// unknown. A non-nil empty slice is emitted as "verified": [] and means
	// the check ran and nothing passed.
	Verified *[]string `json:"verified,omitempty"`
}

Report is the scan output (JSON/CSV shape). Fields are only ever added: status.json is a cross-package wire contract.

func Scan 

func Scan(root string, skip *regexp.Regexp, projectFilter *regexp.Regexp, ignorePatterns *ignore.GitIgnore, reg *sources.Registry) (Report, error)

Scan walks root and returns a Report. skip, projectFilter, ignorePatterns and reg may be nil. Files the scanner cannot fully process are recorded in Report.Issues and skipped; only walk errors abort.

type Requirement 

type Requirement struct {
	ID        string    `json:"id"`
	Source    string    `json:"source,omitempty"`
	TicketURL string    `json:"ticket_url,omitempty"`
	Diagrams  []string  `json:"diagrams,omitempty"` // filled by Task 4
	Features  []Feature `json:"features"`
}

Requirement groups features by requirement ID.

type ScanIssue added in v0.3.3 

type ScanIssue struct {
	Path   string `json:"path"`
	Reason string `json:"reason"` // "binary" | "file_too_large" | "line_too_large" | "read_error" | "parse_error"
	Detail string `json:"detail,omitempty"`
}

ScanIssue records a file the scanner could not fully process. Issues never abort a scan: the file is skipped and the rest of the tree still scans.

type Summary 

type Summary struct {
	ByStatus           map[string]int `json:"by_status"`
	ByAspect           map[string]int `json:"by_aspect"`
	TotalTokens        int            `json:"total_tokens"`
	UniqueRequirements int            `json:"unique_requirements"`
}

Summary holds aggregate counts.

type TokenRecord added in v0.3.3 

type TokenRecord struct {
	// File is the path the token was read from, relative to the scan root.
	File string
	// Line is the 1-based line the token sits on.
	Line int
	// Raw is the token body verbatim, without the "CANARY:" marker.
	Raw string
	// ReqID is the registry-normalized requirement ID.
	ReqID string
	// Fields holds every declared key, upper-cased, values decoded.
	Fields map[string]string
	// ContentHash is the hex SHA-256 of the whole file this token came from.
	ContentHash string
}

TokenRecord is one CANARY token exactly where it was found. Scan folds tokens into per-requirement aggregates, which is the right shape for a report and the wrong shape for an index: the index needs the line each token sits on, its verbatim text, and every field it declared. Both views come out of the same parser, so the index can never disagree with the scan about what a token says.

func (TokenRecord) Field added in v0.3.3 

func (r TokenRecord) Field(key string) string

Field returns the named field, or "" when the token did not declare it. It never invents a value: an absent UPDATED reads as "" so a caller cannot mistake today's date for something the author wrote.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.