scan

package
v0.0.7 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 28, 2026 License: MIT Imports: 16 Imported by: 0

Documentation

Overview

Package scan implements Pasture's read-only Goldmark inventory and classification of native harness syntax across canonical Markdown source roots (github.com/dayvidpham/pasture issue #47).

The scanner never rewrites source and never infers semantics. It:

  1. independently walks a code-owned closed list of canonical source roots (see CanonicalRoots), rejecting symlinked owners and applying an explicit, closed exclusion list for generated/vendor/test-fixture content (see isExcludedPath and excludedPathSegments);
  2. reconciles that independently discovered file set against a checked-in owner manifest that records an explicit active/dead disposition for every owner (see OwnerManifest) — the manifest is reconciliation input, never the discovery source, so an unlisted active file, a stale manifest entry, or a missing disposition is a hard scan failure;
  3. parses every active owner through the same Goldmark configuration used by internal/codegen/ir (goldmark.New(), no extensions) and reports every candidate occurrence of a closed, code-owned pattern registry (see PatternID) — across prose, inline code, fenced/indented code, block HTML (including HTML comments), and inline raw HTML — with its owner, file, AST node context, exact byte/source range, and exact snippet (see Candidate);
  4. classifies every candidate against a checked-in classification manifest (see ClassificationManifest) into one of the closed Classification values, or explicitly reports it unclassified — there is no implicit default — and conversely fails the scan if any checked- in classification entry matches no real candidate (see RequireNoOrphanedClassifications), so the manifest can drift stale in neither direction without a scan failure; and
  5. hashes the canonical tree before and after scanning and fails if a single byte changed (see HashTree), proving the scan is read-only. This proof covers every byte of every non-excluded file under a canonical root; it does not cover the content of excluded segments (testdata/vendor/.git/.opencode — a scanner bug that wrote there would not be caught) or file permission bits (see HashTree's own doc comment).

The resulting Inventory is the input #46 (process/Git/filesystem effects), #43 (task effects), and #40 (runtime contracts) consume before freezing their own closed sets, and the input #42's strict migration gate consumes via RequireZeroUnclassified.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CanonicalRoots

func CanonicalRoots() []string

CanonicalRoots returns a fresh defensive copy of the closed, code-owned canonical source-root list every production scan must use. Tests may scan a smaller synthetic root set to isolate one behavior, but any call in the production code path — including ScanCanonical — uses exactly this set.

func Discover

func Discover(baseDir string, roots []string) ([]string, error)

Discover independently walks baseDir/root for every root in roots and returns the sorted, deduplicated set of relative (to baseDir, slash-separated) Markdown owner paths. It is independent of any OwnerManifest by construction — Discover never reads a manifest, so the manifest can never become the discovery source (see ReconcileOwners).

Discover rejects a symlinked root, directory, or file with an actionable diagnostic (pasture#47 requires "rejecting symlinked owners"), and it excludes every path segment named in the closed excludedPathSegments list. Only files with a ".md" extension are reported: figures/*.yaml, skills/install-cli/.gitkeep, and every other non-Markdown file under a canonical root are silently not owners, not silently excluded content.

func HashTree

func HashTree(baseDir string, roots []string) (string, error)

HashTree computes a deterministic content digest over every non-excluded file under every canonical root (see discoverAllFiles), independent of file extension. Calling HashTree before and after a scan and comparing the two results is pasture#47's required byte-for-byte read-only proof: any change to any byte, any added/removed/renamed file, or any permission- affecting rewrite that changes content under a canonical root changes the digest.

Boundary: this proof covers exactly the same tree discoverAllFiles walks — every non-excluded file under a canonical root — and nothing else:

  • content of an excluded segment (testdata/vendor/.git/.opencode) is never read or hashed, so a scan bug that wrote into skills/.opencode/* or skills/testdata/* during ScanWithManifests would not change the digest and would not be caught by this proof;
  • file mode/permission bits are not part of the digest — only file content and path are — so a permission-only change (with identical content) is likewise invisible to this proof.

Both are acceptable for this proof's purpose (detecting the scanner mutating canonical *source* content), but a caller must not read "hash every canonical source tree byte-for-byte" as covering excluded content or permission bits.

func ModuleRoot

func ModuleRoot() (string, error)

ModuleRoot walks upward from the current working directory until it finds go.mod, returning that directory. It mirrors tools/codegen/main.go's unexported moduleRoot helper so any caller — test or future CLI — can locate the real repository root ScanCanonical needs as baseDir without re-deriving this walk.

func ReconcileOwners

func ReconcileOwners(discovered []string, manifest OwnerManifest) error

ReconcileOwners compares discovered (Discover's independent result) against manifest and returns a *ReconcileError naming every problem:

  • an unlisted active file: discovered but absent from the manifest;
  • a stale manifest entry: manifested but no longer discovered; and
  • (structurally impossible once NewOwnerManifest has validated manifest) a missing/invalid disposition.

A nil error means every discovered path has exactly one manifested disposition and every manifested path was actually discovered.

func RequireNoOrphanedClassifications

func RequireNoOrphanedClassifications(inv Inventory) error

RequireNoOrphanedClassifications aggregates every orphaned classification- manifest entry into one actionable error (mirroring ReconcileError's everything-in-one-report design), naming each entry's owner/pattern/ section/ordinal and, where cheap, a closest-miss reason: how many real occurrences of that entry's (owner, pattern, content-window, section) scope the current scan actually found, so a maintainer can immediately see whether the entry's ordinal simply ran past the real occurrence count. ScanWithManifests calls this between Classify and its own return, so the production pipeline fails on classification-manifest drift exactly as it already fails on owner-manifest drift (see ReconcileOwners).

func RequireZeroUnclassified

func RequireZeroUnclassified(inv Inventory) error

RequireZeroUnclassified is the strict-gate check pasture#42's migration gate consumes: a nonzero unclassified candidate count is a hard prerequisite failure that must block strict-mode activation. It reports every unclassified candidate's owner/file/section/range/ordinal and pattern so a maintainer can classify each one without re-running the scanner to find them.

Types

type Candidate

type Candidate struct {
	// contains filtered or unexported fields
}

Candidate is one reported occurrence of a closed PatternID inside one active owner's Goldmark AST. Candidate is immutable and can only be constructed by this package's own scanning (see scanFileCandidates); a caller can inspect one but never fabricate one, so a Candidate handed to Classify always names a real, exact source occurrence.

func ScanCandidates

func ScanCandidates(baseDir string, discovered []string, owners OwnerManifest) ([]Candidate, error)

ScanCandidates reads and parses every active (non-dead, per owners) owner named in discovered, in the given (already deterministic) order, and returns every candidate found across all of them. discovered must already have been reconciled against owners (see ReconcileOwners) — ScanCandidates still defensively errors if it encounters a discovered path owners does not know about, rather than silently skipping it.

func (Candidate) ASTNode

func (c Candidate) ASTNode() string

ASTNode returns the Goldmark node-kind context the candidate was found in.

func (Candidate) ContentWindow

func (c Candidate) ContentWindow() string

ContentWindow returns the enclosing source line around the match — the wider "reviewed content" a maintainer actually looked at when classifying this occurrence. Classify keys on this (plus Section, to further scope disambiguation — see ClassificationEntry) rather than on Snippet, which is frequently identical across many unrelated occurrences of the same pattern (e.g. every "Skill(/" call shares the same Snippet) and would otherwise degrade the classification key to raw encounter order.

func (Candidate) IsValid

func (c Candidate) IsValid() bool

IsValid reports whether every Candidate invariant holds.

func (Candidate) Location

func (c Candidate) Location() ir.Location

Location returns the candidate's canonical owner/file/section and exact byte source range.

func (Candidate) Pattern

func (c Candidate) Pattern() PatternID

Pattern returns the closed pattern identity that matched.

func (Candidate) Snippet

func (c Candidate) Snippet() string

Snippet returns the exact matched source text (e.g. "Skill(/"). This is the precise, narrow match reported for display/diagnostics; it is not the classification-manifest matching key — see ContentWindow.

type Classification

type Classification string

Classification is the closed, exhaustive disposition a maintainer assigns to one discovered Candidate through the checked-in ClassificationManifest.

There is deliberately no "unclassified" member of this type: unclassified is not a value a candidate can be classified *as*, it is the absence of a matching classification-manifest entry (see Inventory.UnclassifiedCount and ClassifiedCandidate.Classified). Modeling it as a value would let a zero Classification silently mean "no fallback assigns meaning" one day and "explicitly reviewed, no meaning applies" the next — the two states this package must keep visibly distinct per pasture#47's acceptance criteria ("every candidate is explicitly classified or visibly unclassified; no fallback assigns meaning").

const (
	// ClassificationOrchestration is native team/assignment orchestration
	// syntax (e.g. TeamCreate, SendMessage, Skill invocation) that #38's
	// IR represents as a typed orchestration SemanticOperation.
	ClassificationOrchestration Classification = "orchestration"
	// ClassificationUserDecision is native user-interaction syntax (e.g.
	// AskUserQuestion) that #38's IR represents as RequestUserDecision.
	ClassificationUserDecision Classification = "user_decision"
	// ClassificationTaskEffect is a Beads/task-tracker invocation that #43
	// will represent as a typed task effect.
	ClassificationTaskEffect Classification = "task_effect"
	// ClassificationProcessEffect is a process/Git/filesystem invocation
	// that #46 will represent as a typed process/Git/filesystem effect.
	ClassificationProcessEffect Classification = "process_effect"
	// ClassificationPortableVerbatim is native-looking text that is
	// intentionally preserved exactly (e.g. a documentation example
	// showing legacy syntax on purpose) and maps to #38's Verbatim part.
	ClassificationPortableVerbatim Classification = "portable_verbatim"
	// ClassificationTargetLiteral is a reviewed, harness-bound raw escape
	// that maps to #38's exhaustive TargetLiteral part.
	ClassificationTargetLiteral Classification = "target_literal"
	// ClassificationNeutralFalsePositive is a pattern match that, on
	// review, carries no operational meaning at all (e.g. the construct
	// name mentioned in prose without being invoked).
	ClassificationNeutralFalsePositive Classification = "neutral_false_positive"
)

func Classifications

func Classifications() []Classification

Classifications returns a fresh defensive copy of the closed, exhaustive classification set.

func (Classification) IsValid

func (c Classification) IsValid() bool

IsValid reports whether c is one of the closed Classification values.

type ClassificationEntry

type ClassificationEntry struct {
	Owner          string
	Pattern        PatternID
	ContentWindow  string
	Section        string
	Ordinal        int
	Classification Classification
	Notes          string
}

ClassificationEntry is one checked-in classification decision for exactly one candidate occurrence.

The key is (Owner, Pattern, ContentWindow, Section, Ordinal) — content, not a bare regex-matched prefix and not raw whole-file encounter order:

  • ContentWindow is the enclosing source line around the match (see Candidate.ContentWindow), not the short matched prefix (e.g. "Skill(/") Candidate.Snippet still reports for display/diagnostics. Two occurrences of the same pattern almost always sit on different lines, so ContentWindow alone usually makes the key unique without any ordinal at all: keying by the bare prefix made the key indistinguishable between two differently-classified occurrences, so reordering them in the source silently swapped which physical location received which classification, with zero manifest diff and zero scan error.
  • Section additionally scopes Ordinal to the nearest preceding heading (the same Section every Candidate/Location already reports), so Ordinal only disambiguates a genuinely byte-identical ContentWindow within one section, not across the whole file — a section-level reorder (moving a whole subsection, or inserting a new one before an existing group of identical-content duplicates) cannot silently reassign meaning the way whole-file ordinal could.
  • Ordinal is a last resort: it disambiguates two truly identical lines (same owner, pattern, ContentWindow, and Section) — a swap between those specific occurrences is classification-harmless by construction (a maintainer reviewing byte-identical content under one heading has no way to tell them apart either).

type ClassificationManifest

type ClassificationManifest struct {
	// contains filtered or unexported fields
}

ClassificationManifest is the immutable, validated, checked-in set of every explicit candidate classification decision. A candidate with no matching entry is not defaulted to any Classification — see Inventory/Classify — it is reported unclassified.

func DecodeClassificationManifest

func DecodeClassificationManifest(data []byte) (ClassificationManifest, error)

DecodeClassificationManifest strictly decodes and validates a checked-in classification manifest document. It first runs the whole document through ir.StrictJSONWithPresence (duplicate-member rejection, unknown-field rejection, top-level "entries" presence, trailing-content rejection), then independently re-checks per-entry field presence: Ordinal's JSON zero value (0) is a legitimate first-occurrence value, so an omitted "ordinal" must be rejected rather than silently decoded as 0 — the same reasoning ir.StrictJSONWithPresence's own doc comment gives for its top-level requiredFields, applied one level deeper (ir.StrictJSONWithPresence itself only checks top-level document fields).

func NewClassificationManifest

func NewClassificationManifest(entries []ClassificationEntry) (ClassificationManifest, error)

NewClassificationManifest validates and constructs a ClassificationManifest.

func (ClassificationManifest) Len

func (m ClassificationManifest) Len() int

Len returns the number of manifested classification entries.

type ClassifiedCandidate

type ClassifiedCandidate struct {
	Candidate      Candidate
	Classification Classification
	Classified     bool
	Notes          string
	Ordinal        int
}

ClassifiedCandidate pairs one scanned Candidate with its manifest classification decision. Classified is false exactly when no ClassificationManifest entry matched — pasture#47's "no fallback assigns meaning": Classification is the zero value in that case and must not be read as ClassificationOrchestration or any other member. Ordinal is the zero-based occurrence index Classify assigned this candidate within its (owner, pattern, ContentWindow, section) scope — the same value looked up against (and, once classified, matched to) a ClassificationEntry.

type Diagnostic

type Diagnostic struct {
	What   string
	Why    string
	Where  string
	Phase  string
	Impact string
	Fix    string
	Cause  error
}

Diagnostic is this package's actionable error contract. Every field is required so a caller can act on a failure without reading scanner source: what went wrong, why, where it failed, during which phase, what it means for the caller, and how to fix it. This mirrors internal/codegen/ir's Diagnostic shape (see ir/diagnostic.go) without importing it: scan is an independent consumer-facing package and must not fork or re-export #38's exported Diagnostic type, but the same actionable-error contract applies to every error this package returns.

func (*Diagnostic) Error

func (d *Diagnostic) Error() string

func (*Diagnostic) Unwrap

func (d *Diagnostic) Unwrap() error

type Inventory

type Inventory struct {
	// contains filtered or unexported fields
}

Inventory is the classified candidate set #46, #43, and #40 consume before freezing their own closed sets, and #42 consumes through RequireZeroUnclassified before enabling strict rejection.

func Classify

func Classify(candidates []Candidate, manifest ClassificationManifest) Inventory

Classify assigns a classification-manifest decision to every candidate. candidates must already be in deterministic scan order (see scanFileCandidates/ScanCandidates): Classify computes each candidate's classification-manifest ordinal from its position among prior candidates sharing the same owner/pattern/content-window/section tuple, so ordinal assignment is itself deterministic and reproducible across runs and worktrees, and preserves that same order in the returned Inventory.

The returned Inventory also retains manifest and per-key match/occurrence bookkeeping so OrphanedClassifications/RequireNoOrphanedClassifications can later report every checked-in entry that matched no real candidate, without recomputing (and risking drifting from) this same ordinal assignment a second time.

func ScanCanonical

func ScanCanonical(baseDir string) (Inventory, error)

ScanCanonical runs ScanWithManifests against this repository's real, checked-in owner and classification manifests (internal/codegen/scan/ manifest/*.json) over CanonicalRoots. This is the production entrypoint #46, #43, #40, and #42 (and any future CLI) call; baseDir is the pasture module root (see ModuleRoot).

func ScanWithManifests

func ScanWithManifests(baseDir string, roots []string, owners OwnerManifest, classifications ClassificationManifest) (Inventory, error)

ScanWithManifests runs the complete pasture#47 pipeline against baseDir using caller-supplied manifests: independent root discovery, owner reconciliation, before/after byte-for-byte read-only tree hashing, candidate scanning of every active owner, classification, and classification-manifest reconciliation (RequireNoOrphanedClassifications): a checked-in classification entry that matches no real candidate fails the scan here, exactly as an owner-manifest drift already fails it in ReconcileOwners above. Tests use this directly with small synthetic manifests and roots to isolate one behavior; ScanCanonical is the production entrypoint using this repository's real, checked-in manifests.

func (Inventory) Candidates

func (inv Inventory) Candidates() []ClassifiedCandidate

Candidates returns a defensive copy of every classified candidate, in deterministic scan order.

func (Inventory) CountByClassification

func (inv Inventory) CountByClassification(c Classification) int

CountByClassification returns the number of candidates explicitly classified as c.

func (Inventory) Len

func (inv Inventory) Len() int

Len returns the total candidate count, classified and unclassified.

func (Inventory) OrphanedClassifications

func (inv Inventory) OrphanedClassifications() []ClassificationEntry

OrphanedClassifications returns every checked-in ClassificationManifest entry that matched no real candidate during Classify — content that was reviewed and classified but no longer corresponds to anything the current scan actually found (the source occurrence was edited or removed, the owner/pattern was renamed, or the content-window/section/ordinal was mistyped). This is the classification-manifest counterpart to ReconcileOwners' stale-entry detection.

func (Inventory) Unclassified

func (inv Inventory) Unclassified() []ClassifiedCandidate

Unclassified returns every candidate with no matching classification- manifest entry, in deterministic scan order.

func (Inventory) UnclassifiedCount

func (inv Inventory) UnclassifiedCount() int

UnclassifiedCount returns the number of candidates with no matching classification-manifest entry.

type OwnerDisposition

type OwnerDisposition string

OwnerDisposition is the closed, exhaustive whole-file disposition recorded in the checked-in OwnerManifest.

const (
	// OwnerActive means the owner is parsed and scanned for candidates.
	OwnerActive OwnerDisposition = "active"
	// OwnerDead means the owner is discovered and reconciled (it still
	// must be byte-for-byte hashed and present in the manifest) but is
	// explicitly disposed as historically inactive, so it is not parsed
	// for candidates. A dead disposition always requires a non-empty
	// Reason (see OwnerEntry) — pasture#47 requires "explicit dead-owner
	// dispositions", not a silent skip.
	OwnerDead OwnerDisposition = "dead"
)

func OwnerDispositions

func OwnerDispositions() []OwnerDisposition

OwnerDispositions returns a fresh defensive copy of the closed, exhaustive owner-disposition set.

func (OwnerDisposition) IsValid

func (d OwnerDisposition) IsValid() bool

IsValid reports whether d is one of the closed OwnerDisposition values.

type OwnerEntry

type OwnerEntry struct {
	// Path is the owner's path relative to the scan base directory,
	// slash-separated (e.g. "skills/worker/SKILL.md").
	Path string
	// Disposition is the owner's closed active/dead disposition.
	Disposition OwnerDisposition
	// Reason is required (non-empty) when Disposition is OwnerDead and
	// otherwise ignored — pasture#47 requires "explicit dead-owner
	// dispositions", not a silent skip.
	Reason string
}

OwnerEntry is one checked-in disposition for a canonical owner path.

type OwnerManifest

type OwnerManifest struct {
	// contains filtered or unexported fields
}

OwnerManifest is the immutable, validated, checked-in set of every canonical owner's disposition. It is reconciliation input only — see ReconcileOwners — never the discovery source: Discover walks the filesystem independently, and OwnerManifest is compared against what Discover actually found.

func DecodeOwnerManifest

func DecodeOwnerManifest(data []byte) (OwnerManifest, error)

DecodeOwnerManifest strictly decodes and validates a checked-in owner manifest document (see ir.StrictJSONWithPresence and NewOwnerManifest).

func NewOwnerManifest

func NewOwnerManifest(entries []OwnerEntry) (OwnerManifest, error)

NewOwnerManifest validates and constructs an OwnerManifest from entries.

func (OwnerManifest) Len

func (m OwnerManifest) Len() int

Len returns the number of manifested owners.

func (OwnerManifest) Lookup

func (m OwnerManifest) Lookup(path string) (OwnerEntry, bool)

Lookup returns the manifest entry for path, if present.

func (OwnerManifest) Paths

func (m OwnerManifest) Paths() []string

Paths returns every manifested owner path, sorted.

type PatternID

type PatternID string

PatternID identifies one closed, code-owned lexical pattern the scanner looks for inside every active owner's Goldmark AST. The registry is intentionally small and precise (exact call-like syntax, not every prose mention of a construct's name) — see patternRegistry in pattern.go for the exact regular expressions and internal/codegen/scan/manifest for the real classification of every match this registry currently produces against this repository's canonical roots.

const (
	// PatternTeamCreate matches a literal TeamCreate( call prefix — native
	// parallel-team spawning syntax (orchestration).
	PatternTeamCreate PatternID = "team_create"
	// PatternSendMessage matches a literal SendMessage( call prefix —
	// native assignment-messaging syntax (orchestration).
	PatternSendMessage PatternID = "send_message"
	// PatternSkillInvocation matches a literal Skill(/ call prefix —
	// native skill-invocation syntax (orchestration).
	PatternSkillInvocation PatternID = "skill_invocation"
	// PatternAskUserQuestion matches a literal AskUserQuestion( call
	// prefix — native user-decision syntax (user decision).
	PatternAskUserQuestion PatternID = "ask_user_question"
)

func PatternIDs

func PatternIDs() []PatternID

PatternIDs returns a fresh defensive copy of the closed, exhaustive pattern-registry identity set.

func (PatternID) IsValid

func (id PatternID) IsValid() bool

IsValid reports whether id is one of the closed PatternID values.

type ReconcileError

type ReconcileError struct{ Problems []string }

ReconcileError aggregates every owner-manifest drift problem found by ReconcileOwners into one actionable error, so a caller sees every problem in one report instead of fixing them one failed run at a time.

func (*ReconcileError) Error

func (e *ReconcileError) Error() string

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL