Documentation
¶
Overview ¶
Package corpuscheck parses the upstream tree-sitter corpus test format. It compares gotreesitter output with the expected syntax expressions.
A survey of approximately 190 upstream grammars found this format:
==================== test name :attribute1 :attribute2 ==================== <source text, verbatim, may be multiple lines> --- <expected S-expression, may be multiple lines, may be empty>
Repeated for every test in the file. Variations this parser handles:
- Header and divider lengths can vary in one file. Headers use three or more '=' characters. Result dividers use three or more '-' characters.
- A marker can have a non-whitespace suffix. The parser reads the suffix from the opening header. The closing header and result divider must use the same suffix. This rule distinguishes markers from similar source lines.
- Attribute lines can occur inside the header. A ':' at the start of source or expected output is ordinary content.
- The parser accepts carriage return and line feed endings. It removes the carriage return before marker matching.
- An ":error" test can omit the expected output.
ParseFile documents the unsupported format variants.
Index ¶
Constants ¶
const ( CategoryShape = "shape" // nil-ness / child count differs CategoryType = "type" // node type name differs CategoryField = "field" // field name differs CategoryMissing = "missing" // MISSING-ness differs CategoryMissingValue = "missing_value" // both MISSING, different token/type CategoryUnexpectedUnsupported = "unexpected_node_unsupported" // expected side uses UNEXPECTED; gotreesitter has no equivalent node kind )
Mismatch categories. Keep these values stable because callers aggregate them.
Variables ¶
This section is empty.
Functions ¶
func DiscoverCorpusDirs ¶
DiscoverCorpusDirs recursively finds every directory literally named "corpus" (or, to also catch variants such as tree-sitter-foam's "corpus_windows" CRLF suite, named with a "corpus" prefix) under root. Multi-grammar repos (typescript+tsx, markdown+markdown-inline, apex's three sub-grammars, ...) surface as multiple directories; callers that only want the single dominant grammar for a language name should use the first (shortest-path) entry.
Types ¶
type CaseResult ¶
type CaseResult struct {
File string
Name string
Line int
Reason string // skip reason, or mismatch category on failure
Detail string
Path string // divergence path, if applicable
}
CaseResult records one test case's outcome.
type CompareOptions ¶
type CompareOptions struct {
// IgnoreMissingExpectedFields accepts a missing expected field when the
// actual tree has one. It does not accept the reverse case.
// It also rejects two different field names.
// Some upstream repositories added fields without updating their fixtures.
// Repository histories confirm this state for JSON and Go.
// Use this option only as a labeled secondary measure.
IgnoreMissingExpectedFields bool
}
CompareOptions controls how much latitude Compare gives the comparison. The zero value is the fully strict comparison Compare uses.
type Divergence ¶
type Divergence struct {
// Path identifies the mismatching node.
// Example: "/source_file/function_declaration[0]/block[2]".
Path string
// Category classifies the mismatch for aggregate reporting.
Category string
Expected string
Actual string
}
Divergence describes the first place two SNode trees stop matching.
func Compare ¶
func Compare(expected, actual *SNode) *Divergence
Compare walks both trees in lockstep. It returns the first difference. It returns nil when every type, field, missing state, and child matches.
func CompareWithOptions ¶
func CompareWithOptions(expected, actual *SNode, opts CompareOptions) *Divergence
CompareWithOptions is Compare with configurable leniency; see CompareOptions.
func (*Divergence) String ¶
func (d *Divergence) String() string
type FileResult ¶
type FileResult struct {
Path string
Cases int
Pass int
PassFieldLenient int
Fail []CaseResult
Skipped []CaseResult
// FormatErrors holds test-framing errors ParseFile recovered from
// (see TestCase.ParseError); each one means at least one test in the
// file could not be extracted at all.
FormatErrors []error
}
FileResult is the outcome for one corpus file.
type LanguageReport ¶
type LanguageReport struct {
Language string
CorpusRoot string
Files int
Cases int
Pass int
// PassFieldLenient counts cases that pass under the strict
// comparison OR whose only divergences are "fixture has no field
// name, gotreesitter's actual tree does" (see
// CompareOptions.IgnoreMissingExpectedFields). This is a secondary,
// clearly-separate lens: several upstream grammar repos' own corpus
// fixtures are stale with respect to fields their grammar.js added
// years ago (verified via each repo's git history for json and go;
// see the corpuscheck report). It is never folded into Pass.
PassFieldLenient int
Skipped []CaseResult
Failed []CaseResult
FormatErrors []error
// SkipReason, if non-empty, means the whole language was not run at
// all (e.g. unsupported parse backend, corpus directory not found).
SkipReason string
}
LanguageReport aggregates every file's results for one language.
func RunLanguage ¶
func RunLanguage(language, corpusDir string) *LanguageReport
RunLanguage parses every corpus file under corpusDir with language's registered grammar loader and compares each test case against its upstream expected S-expression.
func RunLanguageLimited ¶
func RunLanguageLimited(language, corpusDir string, maxCases int) *LanguageReport
RunLanguageLimited behaves like RunLanguage but stops issuing new parses once report.Cases reaches maxCases (0 means unlimited).
func RunLanguageWithOptions ¶
func RunLanguageWithOptions(language, corpusDir string, opts RunOptions) *LanguageReport
RunLanguageWithOptions is RunLanguage with full control over RunOptions. See RunOptions for the -- language() tag filtering and per-language case cap -- knobs it exposes.
func (*LanguageReport) FailCategoryCounts ¶
func (r *LanguageReport) FailCategoryCounts() map[string]int
FailCategoryCounts tallies Failed by Reason.
func (*LanguageReport) SkipReasonCounts ¶
func (r *LanguageReport) SkipReasonCounts() map[string]int
SkipReasonCounts tallies Skipped by Reason.
type RunOptions ¶
type RunOptions struct {
// MaxCases stops issuing new parses once the report's Cases count
// reaches this many (0 means unlimited).
MaxCases int
// RequireLanguageTag is for languages whose fixtures live inside
// another language's shared corpus directory, distinguished only by
// a `:language(name)` attribute (e.g. tsx and dtd both live inside
// typescript's and xml's corpus directories respectively, alongside
// untagged typescript/xml tests). When true, a test case with no
// `:language()` attribute at all is skipped rather than treated as
// belonging to this language.
RequireLanguageTag bool
}
RunOptions controls RunLanguageWithOptions.
type SNode ¶
type SNode struct {
// Field is the field name this node is attached under in its parent,
// or "" if none.
Field string
// Type is the node's type name: a grammar rule name for ordinary
// nodes, "ERROR" for error nodes, or (for a Missing anonymous token)
// the token's literal text.
Type string
// Missing marks a MISSING node (always a leaf).
Missing bool
// MissingIsAnon marks that Type is a literal anonymous token's text
// (was double-quoted in the fixture) rather than a named node type.
MissingIsAnon bool
// Unexpected marks an UNEXPECTED node -- the C tree-sitter runtime's
// marker for a single lexically-unrecognized byte inside an ERROR
// node. gotreesitter has no equivalent concept (see compare.go); this
// flag exists purely so the comparator can name that gap explicitly
// instead of reporting a generic shape mismatch.
Unexpected bool
// Children holds this node's named children, in order.
Children []*SNode
}
SNode is a canonical, whitespace-insensitive representation of one node in an S-expression tree -- either the one a grammar author wrote by hand in a corpus fixture, or the one gotreesitter's own tree renders to. Both sides are converted to this shape so that comparison never has to worry about indentation, line wrapping, or trailing newlines.
func ParseExpected ¶
ParseExpected parses the expected-output text from a corpus test case (everything after the "---" divider) into a canonical SNode tree.
Supported: node types, field names ("field: (...)"), ERROR nodes, and MISSING nodes (both quoted-anonymous-token and bare-named-type forms, e.g. `(MISSING ".")` and `(MISSING identifier)`).
NOT supported: UNEXPECTED nodes are recognized syntactically (so a fixture that contains one doesn't fail to parse) but are flagged via SNode.Unexpected rather than modeled with real content, since gotreesitter has no equivalent node kind to compare them against.
func RenderTree ¶
func RenderTree(root *gotreesitter.Node, lang *gotreesitter.Language) *SNode
RenderTree converts a live gotreesitter parse tree into the same canonical SNode shape ParseExpected produces from a corpus fixture's text, so the two can be compared structurally without caring about whitespace or indentation.
A node is included if it is named, or if it is a MISSING node (including an anonymous MISSING token, which tree-sitter's own s-expression dump shows even though the corresponding present token would be invisible). Every other unnamed node (ordinary anonymous punctuation/keyword tokens) is skipped entirely, matching upstream tree-sitter's own S-expression dump.
type TestCase ¶
type TestCase struct {
// Name is the test title. An anonymous test can have an empty name.
Name string
// Attrs contains the header attributes without the leading ':'.
Attrs []string
// Input is the verbatim source text for this test.
Input []byte
// Expected contains the raw text after the result divider.
// An ":error" test can omit this text.
Expected string
// Line is the one-based line number of the opening header marker.
Line int
// ParseError contains a test framing error.
// Input and Expected are not reliable when ParseError is not nil.
ParseError error
}
TestCase is one corpus test extracted from a corpus file.
func ParseFile ¶
ParseFile parses one corpus file's contents into its test cases.
Known format variants this parser does NOT support (found during the survey but rare enough, or upstream-tool-specific enough, that implementing them was not worth the complexity for a comparison harness):
- Nothing observed forced an outright parse failure across the surveyed languages; the recovery strategy below (re-synchronize on the next header marker) is believed sufficient for any remaining oddities. If a file produces zero test cases where it plainly has content, that is reported as a parse error by the caller, not silently swallowed.
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
corpuscheck
command
Command corpuscheck runs gotreesitter against upstream tree-sitter corpora.
|
Command corpuscheck runs gotreesitter against upstream tree-sitter corpora. |