Documentation
¶
Overview ¶
Package application orchestrates the matlatl pipeline. It depends on the domain and platform layers and defines the small set of interfaces (ports) that mark genuine test seams. It is wired by cmd/matlatl and must not import cobra. See ADR 0004.
Index ¶
Constants ¶
const ( // DetailTarget is the raw, human-facing link target (path plus #fragment). DetailTarget = "target" // DetailLinkType is the syntactic link type (e.g. "relative-link", "wikilink"). DetailLinkType = "linkType" // DetailExpectedSlug is the anchor slug a broken-anchor reference expected. DetailExpectedSlug = "expectedSlug" // DetailTargetDocument is the resolved document a broken anchor lives in. DetailTargetDocument = "targetDocument" // DetailCandidates is the newline-joined candidate DocumentIDs of an ambiguous // reference (the alternatives an agent can pick a unique path from). DetailCandidates = "candidates" // DetailComponentA / DetailComponentB are the two cluster IDs of a gap, and // DetailRepresentativeA / DetailRepresentativeB a concrete bridge endpoint each. DetailComponentA = "componentA" DetailComponentB = "componentB" DetailRepresentativeA = "representativeA" DetailRepresentativeB = "representativeB" // DetailStatusCode is the final HTTP status of a failed external link // (DeadLink). DetailBlocked is "true" when the SSRF guard refused the URL. // Present only on --check-external runs (kept out of the default output). DetailStatusCode = "statusCode" DetailBlocked = "blocked" // DetailInboundCount is the actual inbound-link count of an under-linked // document (the data behind the discoverability-threshold comparison). DetailInboundCount = "inboundCount" // DetailBowtieBucket is a node's bow-tie bucket // (core/in/out/tendril/disconnected); surfaced in graph.json node data. DetailBowtieBucket = "bowtieBucket" // Suggested-link detail keys (ADR 0013). DetailTargetDocument carries DocA (the // finding's anchor document); these carry the rest of the pair and its scores. DetailSuggestedTarget = "suggestedTarget" DetailCoupling = "coupling" DetailCoCitation = "coCitation" DetailAdamicAdar = "adamicAdar" // Critical-structure detail keys (ADR 0015). DetailBetweenness is the // load-bearing connector's betweenness score (the data behind the // articulation-point finding). DetailBridgeEndpoint is the OTHER endpoint of a // bridge edge (the finding is anchored at the canonical-min endpoint). DetailBetweenness = "betweenness" DetailBridgeEndpoint = "bridgeEndpoint" // Low-scent-anchor detail keys (ADR 0016). DetailAnchorText is the link label // as written; DetailScentScore the Jaccard similarity to the target title // (fixed precision); DetailSuggestedAnchor the recommended replacement (the // target's title); DetailSourceDocument / DetailTargetDocument the endpoints. DetailAnchorText = "anchorText" DetailScentScore = "scentScore" DetailSuggestedAnchor = "suggestedAnchor" DetailSourceDocument = "sourceDocument" // DetailHopsFromRoot is the shortest hop distance from the nearest root of a // far-from-root document (ADR 0021): the data behind the distance-threshold // comparison, so an agent can act without re-deriving it. DetailHopsFromRoot = "hopsFromRoot" // OKF conformance detail keys (ADR 0023). DetailFrontmatterState is "absent" or // "unparseable" on an okf-missing-frontmatter finding; DetailReservedFile names // the reserved file kind ("index.md" / "log.md") on an // okf-reserved-file-structure finding; DetailReason carries the human-readable // cause of an okf-missing-type / okf-reserved-file-structure violation; and // DetailOKFVersion carries the bundle's declared okf_version when known. DetailFrontmatterState = "frontmatterState" DetailReservedFile = "reservedFile" DetailReason = "reason" DetailOKFVersion = "okfVersion" )
Stable structured-detail keys attached to a Finding.Details map. They are the machine-actionable context an agent needs to act on a finding without parsing the prose Message, and are surfaced verbatim in findings.json (schema v2). A key here is part of the findings.json contract: renaming one is a breaking change that bumps the findings schema version.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Artifact ¶
type Artifact struct {
// Name is the artifact filename relative to the output directory.
Name string
// Content is the rendered artifact bytes.
Content []byte
}
Artifact is a single named output to be written to the output directory.
type ArtifactWriter ¶
ArtifactWriter persists rendered artifacts, sanitizing every path to stay under the output directory (reverse zip-slip, ADR 0003). Production implementation: internal/infrastructure/emit.
type BrokenEdge ¶
type BrokenEdge struct {
Origin identity.DocumentID
Target string
}
BrokenEdge is an origin document and the raw target text of a reference that did not resolve to an in-corpus document (a broken link). It carries the presentation data the diagram emitters need without exposing the resolver.
type Config ¶
type Config struct {
// RootPath is the scan root (the repository to analyze).
RootPath string
// Roots is the explicit reachability root set (documents BFS starts from);
// empty means autodetect (README.md/index.md/type:index).
Roots []string
// Ignore holds additional ignore patterns layered on .matlatlignore.
Ignore []string
// ResolutionPolicy selects how raw targets map to documents (ADR 0001).
ResolutionPolicy reference.ResolutionPolicy
// OutputDir is the artifact output directory; empty means no artifacts.
OutputDir string
// Formats selects the emitter formats to render.
Formats []string
// Strict promotes orphan/ambiguous warnings to build failures (ADR 0005).
Strict bool
// CheckExternal enables opt-in external link liveness checks (ADR 0003).
CheckExternal bool
// Quiet suppresses non-essential output.
Quiet bool
// Verbose enables detailed logging.
Verbose bool
// ParseWorkers bounds the parse-stage worker pool. 0 means autodetect
// (GOMAXPROCS, capped). 1 forces the single-threaded path. The merge into
// the Corpus is always single-threaded and deterministic regardless of this
// value (P6 fan-out parsing).
ParseWorkers int
// ExternalChecker, when non-nil and CheckExternal is set, validates
// HealthExternal http(s) links. It is an application port (interface) so the
// domain stays free of net/http; the CLI injects the infrastructure
// implementation. nil disables external checking even under CheckExternal.
ExternalChecker ExternalLinkChecker
// InboundThreshold is the under-linked discoverability floor (ADR 0012): a
// non-exempt document with outbound links but fewer inbound links is reported
// as under-linked. <=0 is normalized to graphmodel.DefaultInboundThreshold (3)
// in the domain.
InboundThreshold int
// StructureFindingsSeverity selects the severity of the graduated structure
// findings (under-linked, dead-end). Default "info" (never fails check);
// "warning" promotes them so they fail `check --strict`.
StructureFindingsSeverity StructureFindingsSeverity
// pair must have to be reported as a suggested-link (ADR 0013). Config-only
// knob (no CLI flag). <=0 is normalized to the domain default (2) in
// PredictLinks.
LinkSuggestionMinShared int
// FarFromRootThreshold is the hop-distance floor for the far-from-root finding
// (ADR 0021): a document reachable from the root set but at or beyond this many
// hops from the nearest root is reported as far-from-root. Config-only knob (no
// CLI flag). <=0 is normalized to graphmodel.DefaultFarFromRootThreshold (6) in
// the domain.
FarFromRootThreshold int
// EmitExclude holds gitignore-syntax patterns for documents that stay in the
// corpus (scanned, link-checked, ranked — the pipeline NEVER reads this field)
// but are not rendered on the consumption surfaces (llms.txt family, index.md,
// trails.json). Sourced from `.matlatl.yml emitExclude` and consumed only at
// the emit boundary by the CLI layer (ADR 0019). Empty = no filtering.
EmitExclude []string
// OKF enables OKF v0.1 conformance mode (ADR 0023): the pipeline runs the
// okf.Check conformance rules over the corpus, appends the mode-scoped
// okf-* Error findings, and reports a CONFORMANT / NOT CONFORMANT verdict.
// Off by default; the effective value is `--okf` flag OR `.matlatl.yml okf`.
// When on, an okf-* finding gates `check` (exit 1) regardless of --strict, but
// the health gate (broken links etc.) is unchanged — the verdict is reported
// separately and never relaxes it (superset gate).
OKF bool
// RespectGitignore unions the repo's effective git-ignore set (tracked and
// nested .gitignore rules, .git/info/exclude, global excludes) with
// .matlatlignore so git-ignored working files stay out of the corpus
// (ADR 0024). Off by default; the effective value is the
// `--respect-gitignore` flag OR `.matlatl.yml respectGitignore`. A no-op
// (with a notice) when the scan root is not a git work tree.
RespectGitignore bool
}
Config holds the resolved run configuration for the pipeline. It is built by the CLI layer from flags and arguments and treated as read-only by the pipeline.
func DefaultConfig ¶
func DefaultConfig() Config
DefaultConfig returns a Config with sane defaults: scan the current directory, longest-suffix resolution, no artifacts, no external checks.
type DocumentParser ¶
type DocumentParser interface {
Parse(ctx context.Context, file ScannedFile) (*corpus.Document, error)
}
DocumentParser turns a scanned file into a pure-domain Document (front matter, section tree, raw references). Production implementation: internal/infrastructure/mdparser (the only package allowed to import goldmark, ADR 0002).
type DocumentParserFactory ¶
type DocumentParserFactory interface {
// New returns a freshly configured DocumentParser.
New() DocumentParser
// Clone returns an independent DocumentParser safe to use on its own
// goroutine. It is equivalent to New but names the per-worker intent.
Clone() DocumentParser
}
DocumentParserFactory mints DocumentParsers. A single DocumentParser is not guaranteed goroutine-safe (its underlying goldmark parser carries per-call mutable state), so the parse stage obtains parsers through this factory: the single-worker fast path uses one parser, and the fan-out path calls Clone per worker for an independent parser. Production implementation: internal/infrastructure/mdparser.
type ExternalLinkChecker ¶
type ExternalLinkChecker interface {
Check(ctx context.Context, urls []string) map[string]ExternalResult
}
ExternalLinkChecker validates a batch of external (http/https) URLs for liveness, applying the SSRF guard, bounded concurrency, per-host rate limiting, redirect caps and result de-duplication (ADR 0003). It is the opt-in --check-external seam: OFF by default so the deterministic pipeline output is unchanged. The domain never imports net/http; this interface keeps the checker in infrastructure (internal/infrastructure/linkcheck). The result map is keyed by the exact input URL string.
type ExternalResult ¶
type ExternalResult struct {
// URL is the checked URL (the map key, repeated for convenience).
URL string
// OK is true when the URL responded with a non-error status and passed the
// SSRF guard.
OK bool
// StatusCode is the final HTTP status (0 when no request was made, e.g. a
// guard refusal or a transport error).
StatusCode int
// Blocked is true when the SSRF guard refused the URL (internal/metadata
// host, disallowed scheme, redirect to an internal host). No network request
// was made for a blocked URL.
Blocked bool
// Err is a human-readable failure reason (empty when OK).
Err string
}
ExternalResult is the outcome of checking one external URL.
type FileScanner ¶
type FileScanner interface {
Scan(ctx context.Context, root string) (ScanResult, error)
}
FileScanner walks a root and returns the markdown files to parse, enforcing the security boundary and ignore rules (ADR 0003). Production implementation: internal/infrastructure/fsscanner.
type Notice ¶
type Notice struct {
Kind NoticeKind
// Path is the offending filesystem path (best-effort; may be relative).
Path string
// Detail is a human-readable explanation.
Detail string
}
Notice is a non-fatal observation from the scan stage (a skipped symlink, an oversized file, a root-escaping path, or a truncated discovery). Notices are reported to the user but do not by themselves fail the run.
type NoticeKind ¶
type NoticeKind int
NoticeKind classifies a non-fatal scanner observation surfaced to the user.
const ( // NoticeSkippedSymlink reports a symlink that was not followed (ADR 0003). NoticeSkippedSymlink NoticeKind = iota // NoticeEscapesRoot reports a path that, after canonicalization, resolves // outside the scan root (the genuine root-escape boundary, ADR 0003). NoticeEscapesRoot // NoticeOversized reports a file skipped for exceeding the size cap. NoticeOversized // NoticeTruncated reports that discovery stopped at the file-count cap. NoticeTruncated // NoticeWalkError reports a directory-walk error on an entry (the entry was // skipped; the walk continued). NoticeWalkError // NoticeIOError reports a stat/info or identity-derivation failure on an // otherwise-eligible file (the file was skipped). NoticeIOError // NoticeConfig reports a tolerated observation from loading the per-repo // `.matlatl.yml` (a missing version field assumed as 1, an unknown/typo key // ignored, an oversized config skipped). It never by itself fails the run; // hard config errors are surfaced as a real error mapped to ExitUsage (ADR // 0011). NoticeConfig // NoticeSkippedNestedRepo reports a directory pruned because it is a nested // git repository — a submodule, linked worktree, or nested clone (detected by // the presence of a `.git` entry inside it; ADR 0017). The scan root's own // `.git` is exempt, so this fires only for nested working trees below the root. NoticeSkippedNestedRepo // NoticeGitignore reports a tolerated observation from collecting the repo's // git-ignore set under --respect-gitignore (ADR 0024): git missing, the root // not being a git work tree, or a git failure. The feature fail-opens (the // scan proceeds with .matlatlignore only), so this never fails the run. NoticeGitignore )
func (NoticeKind) String ¶
func (k NoticeKind) String() string
String returns a short identifier for the notice kind.
type Pipeline ¶
type Pipeline struct {
// contains filtered or unexported fields
}
Pipeline orchestrates the six-stage matlatl flow (Scan → Parse → Resolve → Build → Analyze → Emit; see architecture.md). It holds the configuration and the port implementations it drives.
Stages 1–5 (Scan, Parse, Resolve, Build, Analyze) are all wired here: the pipeline scans the root, parses each discovered file into a Corpus, resolves every reference, builds the reference graph (ADR 0007), and runs the full reachability/orphan/component/HITS/gap analysis into a frozen report + GraphMetrics. Emit (stage 6) is performed by the command layer after Run (e.g. check writes findings.json/JUnit), so the pipeline stays emitter-agnostic.
Concurrency: parsing fans out across a bounded worker pool (each worker owns an independent parser via DocumentParserFactory.Clone, since goldmark parsers are not safe to share), and the parsed documents are merged into the Corpus on this single goroutine in DocumentID-sorted order. That sequential merge is what keeps the corpus, heading inventory and every downstream artifact byte-identical to the single-threaded path at any worker count.
func NewPipeline ¶
func NewPipeline(cfg Config, scanner FileScanner, parserFac DocumentParserFactory, log io.Writer) *Pipeline
NewPipeline constructs a Pipeline from a config, its ports, and a log sink. Parsers are obtained from a DocumentParserFactory: the single-worker fast path uses one parser, and the fan-out path mints one per worker via Clone. A nil log sink discards output.
Emit (stage 6) is the command layer's job, so the pipeline takes no artifact writer: it returns a frozen Result the caller renders/writes (e.g. `check` writes findings.json/JUnit). The ArtifactWriter port stays on the command layer for that reason.
type Result ¶
type Result struct {
// DocumentCount is the number of documents successfully parsed.
DocumentCount int
// HeadingCount is the total number of heading slugs indexed.
HeadingCount int
// ReferenceCount is the total number of references resolved.
ReferenceCount int
// BrokenLinkCount / BrokenAnchorCount / AmbiguousCount / OrphanCount /
// UnreachableCount / KnowledgeGapCount are convenience tallies for the human
// summary and exit-code decision.
BrokenLinkCount int
BrokenAnchorCount int
AmbiguousCount int
OrphanCount int
UnreachableCount int
KnowledgeGapCount int
// SuggestedLinkCount is the number of topology-based suggested-link findings
// (ADR 0013). Info; never affects the exit code. SuggestedLinksTruncated
// reports the suggestion list was capped or a hub neighbour was skipped.
SuggestedLinkCount int
SuggestedLinksTruncated bool
// UnderLinkedCount / DeadEndCount are the graduated structure-tier tallies
// (ADR 0012). They affect the exit code only when StructureFindingsSeverity is
// Warning (consulted by CheckExitCode).
UnderLinkedCount int
DeadEndCount int
// StructureFindingsSeverity is the resolved severity of under-linked/dead-end
// findings for this run, carried so CheckExitCode can decide whether they gate
// --strict.
StructureFindingsSeverity StructureFindingsSeverity
// DeadLinkCount is the number of failed external links (only non-zero when
// --check-external is enabled). It does not affect the default run.
DeadLinkCount int
// ArticulationPointCount / BridgeCount are the critical-path structure tallies
// (ADR 0015): cut vertices and cut edges of the undirected closure. Both are
// Info findings that NEVER affect the exit code (consulted by no exit-code
// path); they are carried for the human summary and machine artifacts.
ArticulationPointCount int
BridgeCount int
// LowScentAnchorCount is the number of low-scent-anchor findings (ADR 0016):
// links whose anchor text barely previews the target. Info; NEVER affects the
// exit code; carried for the human summary and machine artifacts.
LowScentAnchorCount int
// FarFromRootCount is the number of far-from-root findings (ADR 0021):
// documents reachable but at or beyond the hop-distance threshold from every
// root. Info; NEVER affects the exit code; carried for the human summary and
// machine artifacts.
FarFromRootCount int
// OKF conformance (ADR 0023). OKFMode reports whether OKF conformance mode was
// on for this run; the rest are meaningful only when it is. OKFConformant is
// the §9 verdict (all three rule counts zero); OKFVersion is the bundle's
// declared okf_version ("" when none). The three counts are the Error-severity
// okf-* finding tallies. When OKFMode && !OKFConformant, CheckExitCode gates
// exit 1 regardless of --strict.
OKFMode bool
OKFConformant bool
OKFVersion string
OKFMissingFrontmatterCount int
OKFMissingTypeCount int
OKFReservedFileStructureCount int
// Report is the frozen analysis report (all finding kinds).
Report *analysis.AnalysisReport
// Metrics is the frozen P3 graph-analysis carrier (graph, components, HITS,
// degrees, root set, reachability, gaps) for later emitters (P4/P5).
Metrics *graphmodel.GraphMetrics
// Corpus is the frozen corpus the run was computed over. Human emitters read
// it for per-document presentation data (title, description, mod-date) that
// the metrics/report do not carry. It is read-only; emitters must not mutate
// it (ADR 0004). nil for an empty/failed run.
Corpus *corpus.Corpus
// BrokenEdges are the unresolved navigational references (origin → raw
// target) extracted at resolution time. The frozen graph keeps only Valid
// edges, so the P4 diagram emitters read this to render red placeholder
// target nodes (ADR 0003 styling) without re-parsing finding messages.
// Sorted (Origin, Target) for determinism.
BrokenEdges []BrokenEdge
// Notices are non-fatal observations from the scan stage.
Notices []Notice
}
Result summarizes a pipeline run for the caller to present.
func (Result) CheckExitCode ¶
CheckExitCode maps a run Result to the ADR 0005 exit code for `matlatl check`. Broken links and broken anchors always fail (exit 1). Ambiguous links, orphans and unreachable documents are warnings that fail only under --strict. KnowledgeGap and SuggestedLink (both Info) never affect the exit code. A clean repo or an empty corpus returns ExitOK (0).
ArticulationPoint and Bridge (ADR 0015) are likewise Info and DELIBERATELY never gate the exit code, even under --strict: they are structural-resilience hints (single points of failure in the link graph), not defects, so a corpus with a cut vertex is not a failed build. They are reported (findings.json / the human report / graph.json data) but never read here, mirroring SuggestedLink and KnowledgeGap.
FarFromRoot (ADR 0021) is likewise Info and DELIBERATELY never gates the exit code, even under --strict: a document reachable but many hops from any entry point is a discoverability hint, not a defect, so r.FarFromRootCount is intentionally not read here (mirroring the other advisory kinds).
DeadLinkCount is DELIBERATELY excluded from the exit contract, even under --strict (ADR 0005): external link checking is opt-in (--check-external) and non-deterministic (network state, transient timeouts, rate limits), so gating CI on it would make a green build flaky. DeadLink findings are reported (in findings.json/JUnit and the human report) but never change the exit code; CI that wants to fail on dead external links should consume findings.json explicitly. This is why r.DeadLinkCount is intentionally not read here.
type ScanResult ¶
type ScanResult struct {
Files []ScannedFile
Notices []Notice
}
ScanResult is the outcome of a scan: the deterministically sorted files to parse plus any notices.
type ScannedFile ¶
type ScannedFile struct {
// Path is the absolute filesystem path of the file.
Path string
// ID is the canonical document identity derived from the scan root.
ID identity.DocumentID
// ModTime is the file's last-modified time.
ModTime time.Time
// Size is the file size in bytes.
Size int64
}
ScannedFile is a candidate file discovered by a FileScanner, carrying the information the parser needs without performing any parsing itself.
type StructureFindingsSeverity ¶
type StructureFindingsSeverity string
StructureFindingsSeverity selects the severity assigned to the graduated structure findings (under-linked, dead-end). It is a small typed enum so the CLI/config can carry "info" | "warning" and the pipeline can plumb a single value through to the finding builders (ADR 0012).
const ( // StructureFindingsInfo (the default) makes under-linked/dead-end Info: they // are reported but NEVER fail `check`, even under --strict. StructureFindingsInfo StructureFindingsSeverity = "info" // StructureFindingsWarning promotes under-linked/dead-end to Warning: they // then fail `check --strict` like orphans/unreachable. StructureFindingsWarning StructureFindingsSeverity = "warning" )
func (StructureFindingsSeverity) Valid ¶
func (s StructureFindingsSeverity) Valid() bool
Valid reports whether s is a defined severity choice.