gotreesitter

package module
v0.55.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 24, 2026 License: MIT Imports: 31 Imported by: 26

README ¶

gotreesitter

Pure-Go tree-sitter runtime. No CGo, no C toolchain. It cross-compiles to any GOOS/GOARCH target Go supports, including wasip1.

Every Go tree-sitter binding in the ecosystem depends on CGo, which needs a C cross-toolchain per target, breaks go install for downstream users without a C compiler, and hides bugs from go test -race. gotreesitter removes the C dependency entirely: the parser, lexer, query engine, incremental reparsing, arena allocator, external scanners, and tree cursor are all implemented in Go. The grammar blob is the only input.

Install

go get github.com/odvcencio/gotreesitter

gotreesitter loads the same parse-table format that tree-sitter's C runtime uses. ts2go extracts grammar tables from upstream parser.c files, compresses them into binary blobs, and deserializes them on first use. 206 grammars ship in the registry.

The current release is v0.55.0. See docs/roadmap.md for release scope and history.

Quick start

import (
    "fmt"

    "github.com/odvcencio/gotreesitter"
    "github.com/odvcencio/gotreesitter/grammars"
)

func main() {
    lang := grammars.GoLanguage()
    parser := gotreesitter.NewParser(lang)

    tree, _ := parser.Parse([]byte("package main\n\nfunc main() {}\n"))
    fmt.Println(tree.RootNode().SExpr(lang))
}

grammars.DetectLanguage("main.go") resolves a filename to the matching LangEntry.

Features

  • 206 grammars, all producing error-free trees on smoke samples; 119 ship a hand-written Go external scanner.
  • Incremental reparsing that reuses unchanged tree content by reference, plus a no-edit fast path with zero allocations.
  • Queries with the full S-expression pattern language and all standard tree-sitter predicates and directives.
  • Typed query codegen (cmd/tsquery) that generates Go structs and match helpers from .scm files.
  • Injection parsing for multi-language documents (HTML+JS+CSS, Markdown+code fences, Vue/Svelte).
  • Highlighting, tagging, and file outlines built on the same tags-query captures.
  • UTF-16 input and editor coordinates, so editor integrations do not hand-convert offsets.
  • Source rewriting that produces InputEdit records ready for incremental reparse.
  • WebAssembly/browser runtime with both a blob-loading target and an in-browser grammargen target. See the WebAssembly guide.
  • Build-tag-selected grammar embedding (external blobs, a curated core set, or a hand-picked subset) for smaller binaries.

Agent skill

Agents working with gotreesitter should use the using-gotreesitter skill.

Documentation

Topic Where
Parsing, queries, injections, incremental reparse, UTF-16, cursor, highlighting, tagging, outlines docs/api-guide.md
Runtime architecture (parser, lexer, scanners, arena, query engine, grammar loading) docs/architecture.md
Build tags and environment variables docs/build-tags.md
Supported languages, query feature matrix, adding a language docs/languages.md
Running tests and correctness/parity gates docs/testing-guide.md
Benchmarks and methodology BENCH.md, docs/benchmark-notes.md
Current release scope and roadmap docs/roadmap.md
Root package file map and ownership docs/repository-map.md
Root-package file group map by subsystem docs/package-layout.md
Result-compatibility tier (parser_result_*.go) docs/compat-tier.md
Adding a grammar outside this repo docs/authoring-languages.md
External scanner certification and fallback docs/external-scanners.md
Release process docs/releasing.md
Full changelog CHANGELOG.md, docs/changelog/

License

MIT

Documentation ¶

Overview ¶

Package gotreesitter implements a pure Go tree-sitter runtime.

This file defines the core data structures that mirror tree-sitter's TSLanguage C struct and related types. They form the foundation on which the lexer, parser, query engine, and syntax tree are built.

Index ¶

Constants ¶

View Source
const (
	// RuntimeLanguageVersion is the maximum tree-sitter language version this
	// runtime is known to support.
	RuntimeLanguageVersion uint32 = 15
	// MinCompatibleLanguageVersion is the minimum accepted language version.
	MinCompatibleLanguageVersion uint32 = 13
)
View Source
const (
	ConflictPolicyAnyState     StateID = ^StateID(0)
	ConflictPolicyAnyLookahead Symbol  = ^Symbol(0)
)

ConflictPolicyAnyState and ConflictPolicyAnyLookahead are sentinel State/ Lookahead values matching every state or every lookahead symbol instead of one exact table row. They support policies scoped by state and/or reduce symbol identity when enumerating every reachable row would add lookup cost without strengthening the action-shape check. Built-in wildcard policies are hand-certified and blob-SHA-pinned; callers supplying ConflictPolicies directly are responsible for scoping their own wildcard policies safely.

View Source
const (
	// OutlineDeclineNilOutliner reports a call on a nil *Outliner.
	OutlineDeclineNilOutliner = "nil_outliner"
	// OutlineDeclineQueryEmpty reports that the language has no tags query.
	OutlineDeclineQueryEmpty = "query_empty"
	// OutlineDeclineNilTree reports a nil *Tree argument.
	OutlineDeclineNilTree = "nil_tree"
	// OutlineDeclineNilRootNode reports a tree with no root node.
	OutlineDeclineNilRootNode = "nil_root_node"
	// OutlineDeclineLanguageMismatch reports that the tree was parsed with a
	// different language than the outliner was built for. This is caller
	// misuse, not a property of the source.
	OutlineDeclineLanguageMismatch = "language_mismatch"
)

Outline decline reasons. An empty DeclineReason means the outliner ran the query; it does not mean the file holds symbols.

View Source
const BlobRuntimeVersion uint32 = 1

BlobRuntimeVersion is this runtime's current blob-format compatibility version. It increments whenever a change to the Language struct's gob encoding could produce silently incorrect (not just absent) behavior if decoded by an older runtime -- for example, adding a new *Certified capability flag whose zero value is unsafe to assume, the way an older runtime decoding a newer blob would. EncodeLanguageBlob stamps every newly written blob's MinRuntimeVersion with this constant.

This is a distinct axis from LanguageVersion (the tree-sitter grammar ABI version, 13-15): LanguageVersion says whether the grammar rules a blob was compiled from are compatible with this parser; BlobRuntimeVersion says whether this runtime's Go struct layout is new enough to decode the blob's gob stream correctly.

View Source
const DefaultBlobGeneratorVersion = "gotreesitter"

DefaultBlobGeneratorVersion is the generator identity EncodeLanguageBlob stamps on every blob it writes. Callers that want their own tool identity recorded (e.g. "ts2go" or "grammargen", each with its own version string) should use EncodeLanguageBlobWithGenerator instead.

Variables ¶

View Source
var (
	// ErrInvalidUTF16ByteLength is returned when a UTF-16 byte source has a
	// dangling trailing byte.
	ErrInvalidUTF16ByteLength = errors.New("utf16: byte source length must be even")

	// ErrInvalidUTF16ByteOrder is returned for an unknown UTF-16ByteOrder.
	ErrInvalidUTF16ByteOrder = errors.New("utf16: invalid byte order")

	// ErrInvalidUTF16Range is returned when a UTF-16 range does not align to
	// valid code-point boundaries or has an inverted span.
	ErrInvalidUTF16Range = errors.New("utf16: invalid range")
)
View Source
var DebugDFA atomic.Bool

DebugDFA enables trace logging for DFA token production.

Use `DebugDFA.Store(true/false)` to toggle at runtime.

View Source
var ErrDecompressedBlobTooLarge = errors.New("gzip stream exceeds the decompressed-size limit")

ErrDecompressedBlobTooLarge is returned (wrapped, with the offending size and the active limit) by ReadAllGzipWithSizeHint, and so by LoadLanguage, when a gzip stream's decompressed content would exceed MaxDecompressedBlobSize. Test for this condition with errors.Is.

View Source
var ErrNoLanguage = errors.New("parser has no language configured")

ErrNoLanguage is returned when a Parser has no language configured.

View Source
var ErrNoTokenSource = errors.New("parser has no token source")

ErrNoTokenSource is returned when a token-source parse is called without a token source.

View Source
var ErrNoTokenSourceFactory = errors.New("parser has no token source factory")

ErrNoTokenSourceFactory is returned when a factory-based parse is called without a token source factory.

View Source
var ErrParseStoppedEarly = errors.New("parse stopped before accepting input")

ErrParseStoppedEarly is matched by ParseStoppedEarlyError when a strict parse returns a partial tree.

View Source
var MaxDecompressedBlobSize int64 = 64 * 1024 * 1024 // 64 MiB

MaxDecompressedBlobSize is the hard ceiling ReadAllGzipWithSizeHint (and therefore LoadLanguage) enforces on the decompressed size of a gzip grammar blob, regardless of what the ISIZE trailer claims or how much data the gzip stream actually contains. It guards against a decompression bomb: a small compressed blob that decodes to an unbounded stream, whether because ISIZE lies (ISIZE is only the true size mod 2^32) or because the stream really is that large.

The default is set to roughly 4x the largest legitimate shipped grammar blob's decompressed size (Swift, ~16.1 MiB as of v0.43.1, after grammargen's lex-state minimization brought its LexStates table down from 63,150 to 2,067 entries — see grammars/language_memory_ceiling_test.go for the retained-heap figure and grammars/dfa_minimize.go for the fix). That leaves headroom for grammar growth while still catching a bogus or adversarial stream long before it could exhaust available memory.

This is a package-level variable, not a constant, so an embedder that legitimately ships a larger grammar can raise it before calling LoadLanguage. Lowering it below the largest blob actually loaded turns that blob's LoadLanguage call into an error.

Functions ¶

func AdmissionCandidateCounters ¶ added in v0.46.0

func AdmissionCandidateCounters() (routed, fallbacks uint64)

AdmissionCandidateCounters returns the number of full parses the compact candidate route served, and the number of eligible full parses that fell back to production.

func AdmissionCandidateLastFallbackReason ¶ added in v0.46.0

func AdmissionCandidateLastFallbackReason() string

AdmissionCandidateLastFallbackReason returns the most recent candidate-route decline detail, or the empty string if none has been recorded.

func AdmissionCandidateRouteDefault ¶ added in v0.46.0

func AdmissionCandidateRouteDefault() bool

AdmissionCandidateRouteDefault reports the current process-wide default.

func CRecoverEOFBareRootReceipted ¶ added in v0.54.0

func CRecoverEOFBareRootReceipted(name string) bool

func CRecoverEOFBareRootReceiptedNames ¶ added in v0.54.0

func CRecoverEOFBareRootReceiptedNames() []string

CRecoverEOFBareRootReceiptedNames returns every grammar name CRecoverEOFBareRootReceipted answers true for. A caller that must enumerate the full receipted set — for example checking every entry is still a real shipped grammar, not merely checking a fixed candidate list against the table one name at a time — should use this instead of probing individual names, since probing a fixed list can never notice an extra, unexpected entry the table grants.

func DecodeLargeStateGotosTrailer ¶ added in v0.23.1

func DecodeLargeStateGotosTrailer(r *bytes.Reader) (map[uint64]StateID, error)

DecodeLargeStateGotosTrailer reads a trailer written by EncodeLargeStateGotosTrailer from r, which must be positioned immediately after a Language's gob message within the same decompressed blob stream (callers must gob-decode from a *bytes.Reader, not directly from a gzip.Reader: gob's Decoder can read ahead past its own message boundary on a streaming reader, silently discarding trailer bytes it never needed -- bytes.Reader has no such read-ahead and leaves r positioned exactly at the end of the decoded message).

It returns (nil, nil) when r has no remaining bytes -- the common case for every blob whose Language never populated LargeStateGotos, including every blob encoded before this trailer mechanism existed.

func DecodeUTF16Bytes ¶ added in v0.16.0

func DecodeUTF16Bytes(source []byte, order UTF16ByteOrder) ([]uint16, error)

DecodeUTF16Bytes decodes an endian-specific UTF-16 byte source into Go UTF-16 code units.

func DerivationSetCensusBuilt ¶ added in v0.49.0

func DerivationSetCensusBuilt() bool

DerivationSetCensusBuilt reports whether this binary carries the derivation-set census. The shipped build never does.

func DiagnosticParserCoreContextualCloseAngleDeferralDetailForTest ¶ added in v0.52.0

func DiagnosticParserCoreContextualCloseAngleDeferralDetailForTest() string

DiagnosticParserCoreContextualCloseAngleDeferralDetailForTest exposes diagnosticParserCoreContextualCloseAngleDeferralDetail so the external test package can assert on the issue #983 deferral's decline detail without duplicating the literal string.

func DiagnosticParserCoreNoTableActionDetailForTest ¶ added in v0.52.0

func DiagnosticParserCoreNoTableActionDetailForTest() string

DiagnosticParserCoreNoTableActionDetailForTest exposes diagnosticParserCoreNoTableActionDetail so the external test package can assert on the genuinely-empty-row decline's exact detail. This proves that DisablePerHeaderSpanUnlockedRelex restores the legacy route.

func DiagnosticParserCoreOwnedDispatchPendingDetailForTest ¶ added in v0.52.0

func DiagnosticParserCoreOwnedDispatchPendingDetailForTest() string

DiagnosticParserCoreOwnedDispatchPendingDetailForTest exposes the stable activation prefix without exposing scheduler internals.

func DiagnosticParserCoreRaggedRelexDeclineDetailForTest ¶ added in v0.52.0

func DiagnosticParserCoreRaggedRelexDeclineDetailForTest() string

DiagnosticParserCoreRaggedRelexDeclineDetailForTest exposes diagnosticParserCoreRaggedRelexDeclineDetail so the external test package can assert on the ragged-end decline's detail prefix without duplicating the literal string.

func DiagnosticParserCoreRaggedRelexDeclineDetailFormatForTest ¶ added in v0.52.0

func DiagnosticParserCoreRaggedRelexDeclineDetailFormatForTest(relexedWitness, shared Token) string

DiagnosticParserCoreRaggedRelexDeclineDetailFormatForTest exposes diagnosticParserCoreRaggedRelexDeclineDetailFor so the external test package can assert on the exact ragged-end decline detail a given (relexed, shared) token pair produces.

func DiagnosticParserCoreShadowCensusResetForTest ¶ added in v0.49.0

func DiagnosticParserCoreShadowCensusResetForTest()

DiagnosticParserCoreShadowCensusResetForTest clears every accumulated census total (stage 1 and three-proof) and every captured falsifier or class-1 candidate.

func DrainArenaPools ¶ added in v0.14.0

func DrainArenaPools()

DrainArenaPools releases all cached arenas from both incremental and full-parse pools. Arenas held in the pool are strong Go references and are not collected by the GC until explicitly drained or the process exits.

Call this after a large batch scan (e.g. after WalkAndParse returns) to allow the GC to reclaim the arena memory. The next parse will allocate a fresh arena.

func EOFAcceptHistoryCensusBuilt ¶ added in v0.52.0

func EOFAcceptHistoryCensusBuilt() bool

EOFAcceptHistoryCensusBuilt reports whether this binary includes the G2 diagnostic census.

func EnableArenaBreakdown ¶ added in v0.18.0

func EnableArenaBreakdown(enabled bool)

EnableArenaBreakdown toggles detailed arena accounting for subsequently acquired arenas. It is intended for diagnostics and benchmark attribution; normal parser paths leave it disabled to avoid perturbing hot allocation paths.

func EnableArenaProfile ¶ added in v0.6.0

func EnableArenaProfile(enabled bool)

EnableArenaProfile toggles arena pool counters. This debug hook is not concurrency-safe and is intended for single-threaded benchmark/profiling runs.

func EnableGLREquivAudit ¶ added in v0.19.0

func EnableGLREquivAudit(enabled bool)

EnableGLREquivAudit toggles lightweight GLR equivalence attribution. This is intended for parser gap diagnostics and avoids the heavier survivor maps used by EnableRuntimeAudit.

func EnableRecoveryRuntimeTelemetry ¶ added in v0.50.0

func EnableRecoveryRuntimeTelemetry(enabled bool)

EnableRecoveryRuntimeTelemetry enables diagnostic recovery counters.

Use this function for single-threaded profiling and witness collection. Disable it after the diagnostic run to restore the default path.

func EnableRuntimeAudit ¶ added in v0.7.0

func EnableRuntimeAudit(enabled bool)

EnableRuntimeAudit toggles per-parse survivor instrumentation. This debug hook is intended for single-threaded benchmark/profiling runs.

func EncodeLanguageBlob ¶ added in v0.23.1

func EncodeLanguageBlob(lang *Language) ([]byte, error)

EncodeLanguageBlob serializes lang in the runtime's stable grammar-blob format, stamped with DefaultBlobGeneratorVersion. See EncodeLanguageBlobWithGenerator for the full format description and for attaching a caller-specific generator identity.

func EncodeLanguageBlobWithGenerator ¶ added in v0.54.0

func EncodeLanguageBlobWithGenerator(lang *Language, generatorVersion string) ([]byte, error)

EncodeLanguageBlobWithGenerator serializes lang in the runtime's stable grammar-blob format. Languages without LargeStateGotos retain the legacy gzip+gob wire representation byte-for-byte, aside from the version header below. When LargeStateGotos is populated, the map is removed from a shallow exported-field copy, encoded as a sorted trailer, and wrapped in the versioned fail-closed GTSBLOB envelope understood by LoadLanguage.

The result is then wrapped in a version header (see language_blob_version_header.go) recording BlobRuntimeVersion as the blob's MinRuntimeVersion and generatorVersion as its generator identity. LoadLanguage rejects a blob whose MinRuntimeVersion is newer than this runtime's BlobRuntimeVersion, and exposes the header's contents (or its absence, for every blob written before this header existed) via Language.BlobInfo. This function always writes the header; existing shipped blobs are not retroactively rewritten by this change -- they keep loading as headerless ("legacy") blobs until something re-encodes them.

Keeping this encoder in the root package makes the format invariant shared by every producer, including grammargen and ts2go. The input Language is never mutated and can remain in concurrent use while it is encoded.

func EncodeLargeStateGotosTrailer ¶ added in v0.23.1

func EncodeLargeStateGotosTrailer(m map[uint64]StateID) ([]byte, error)

EncodeLargeStateGotosTrailer serializes m as a self-contained gob stream of (key, target) pairs sorted by Key ascending, suitable for deterministic appending after a Language's gob-encoded blob payload (see DecodeLargeStateGotosTrailer). It returns (nil, nil) for an empty map so callers can skip writing a trailer entirely, leaving the blob byte-for-byte identical to one for a Language that never populated LargeStateGotos.

func InferGeneratedRepeatAuxMetadata ¶ added in v0.21.0

func InferGeneratedRepeatAuxMetadata(lang *Language)

InferGeneratedRepeatAuxMetadata fills GeneratedRepeatAux for older language blobs that predate the explicit metadata bit.

func LanguageWantsForest ¶ added in v0.49.0

func LanguageWantsForest(lang *Language) bool

LanguageWantsForest reports whether lang is in the forest-default set (see parserWantsForest). It does not account for the glrForestEnabled global switch (GOT_GLR_FOREST), which can still disable dispatch even for a language this reports true for. Exported so regression gates outside this package (e.g. the regen-guard sweep that reparses N repeated top-level items per forest-default language) can enumerate the same set parserWantsForest uses, without duplicating or drifting from builtinForestDefaults.

func MergeEventCensusBuilt ¶ added in v0.49.0

func MergeEventCensusBuilt() bool

MergeEventCensusBuilt reports whether this binary carries the merge-event census. The shipped build never does.

func ReadAllGzipWithSizeHint ¶ added in v0.43.1

func ReadAllGzipWithSizeHint(r io.Reader, compressed []byte) ([]byte, error)

ReadAllGzipWithSizeHint reads all of r — an open gzip.Reader positioned at the start of the member whose raw (still-compressed) bytes are compressed — into memory, pre-sizing the destination buffer from the gzip ISIZE trailer (the last 4 bytes of compressed) instead of letting io.ReadAll grow the buffer by repeated doubling. ISIZE is the uncompressed size mod 2^32, which is exact for every blob under 4 GB; grammar blobs are always far smaller. Falls back to plain buffered reads when the hint is missing, zero, or exceeds gzipSizeHintCap.

Every code path here is bounded by MaxDecompressedBlobSize: r is always wrapped in an io.LimitReader before any read happens, so neither the ISIZE-preallocated fast path nor the fallback path can be tricked into an unbounded read by a corrupt or adversarial ISIZE trailer or an oversized gzip stream. Reading more than MaxDecompressedBlobSize returns ErrDecompressedBlobTooLarge.

This matters at grammar-load time: io.ReadAll's doubling growth roughly doubles peak transient allocation versus the final size, and for the largest shipped grammar blobs that transient churn is measured in tens of MB (observed via alloc-space pprof on Language() calls), which is what actually trips container memory limits even though the final retained Language is smaller.

func RegisterHighlighterInjection ¶ added in v0.7.0

func RegisterHighlighterInjection(parentLanguage string, spec HighlighterInjectionSpec)

RegisterHighlighterInjection registers nested-highlighting configuration for a parent language name (for example "markdown").

func RepairNoLookaheadLexModes ¶ added in v0.9.0

func RepairNoLookaheadLexModes(lang *Language)

RepairNoLookaheadLexModes marks parser states as no-lookahead when they only need EOF-triggered reductions plus external/trivia handling. Tree-sitter's C runtime uses these states to reduce before lexing the next real token.

func ResetAdmissionCandidateCounters ¶ added in v0.54.0

func ResetAdmissionCandidateCounters()

ResetAdmissionCandidateCounters clears the process-global admission switch counters and the last fallback reason. It is a diagnostics helper: call it at the start of a test that asserts on AdmissionCandidateCounters or AdmissionCandidateLastFallbackReason, so an earlier test's fallback does not leak into the assertion.

func ResetArenaProfile ¶ added in v0.6.0

func ResetArenaProfile()

ResetArenaProfile resets arena pool counters. This debug hook is not concurrency-safe and is intended for single-threaded benchmark/profiling runs.

func ResetParseEnvConfigCacheForTests ¶ added in v0.7.0

func ResetParseEnvConfigCacheForTests()

ResetParseEnvConfigCacheForTests clears memoized parser env config.

Tests in this repo mutate env vars between cases; this helper ensures subsequent parses observe the new values in the same process.

glrFaithfulCapOneMerge is intentionally excluded: unlike the values below, it is a process-start mode rather than a memoized config value. Tests that override it must restore the variable directly. Mixing that mode into this cache reset made cleanup order observable when t.Setenv restored the environment after ResetParseEnvConfigCacheForTests ran.

func ResetPerfCounters ¶ added in v0.6.0

func ResetPerfCounters()

func RunExternalScanner ¶

func RunExternalScanner(lang *Language, payload any, lexer *ExternalLexer, validSymbols []bool) bool

RunExternalScanner invokes the language's external scanner if present. Returns true if the scanner produced a token, false otherwise.

func SetAdmissionCandidateRouteDefault ¶ added in v0.46.0

func SetAdmissionCandidateRouteDefault(enabled bool)

SetAdmissionCandidateRouteDefault sets the process-wide default the Phase-3 admission switch applies to Parsers with no explicit override. A per-Parser override still wins.

Tranche B9 removed the source-length eligibility decline: every input, regardless of size, is eligible to attempt the candidate route. An explicit timeout, cancellation flag, or the scheduler's own memory-budget poll (tranche B8) is honored on the candidate route itself, with a compatible stop receipt, falling back to production when it trips; included ranges and observability hooks still keep a parse on production.

func SetDiagnosticParserCoreShadowCensusEnabledForTest ¶ added in v0.49.0

func SetDiagnosticParserCoreShadowCensusEnabledForTest(on bool) func()

SetDiagnosticParserCoreShadowCensusEnabledForTest overrides the GTS_B4B_SHADOW_CENSUS gate for one test process, mirroring setParserCoreReplayParseStatesForTest's override pattern (parsestate_replay_compact.go). Restore the previous value (the returned func) when done.

func SetGLRForestEnabled ¶ added in v0.20.0

func SetGLRForestEnabled(on bool)

SetGLRForestEnabled toggles the GSS-forest path at runtime (tests/benchmarks).

func SetGLRForestRecover ¶ added in v0.21.0

func SetGLRForestRecover(on bool)

SetGLRForestRecover toggles experimental forest error recovery (tests).

func SetInternLeavesObserveEnabled ¶ added in v0.20.0

func SetInternLeavesObserveEnabled(on bool)

SetInternLeavesObserveEnabled toggles leaf-interning observation at runtime. Tests and benches that want to A/B observation without re-running the test binary set this directly. Not safe to flip while a parse is in flight on another goroutine. Phase 2 scaffolding; the API may change before becoming public.

func SetInternLeavesSubstituteEnabled ¶ added in v0.20.0

func SetInternLeavesSubstituteEnabled(on bool)

SetInternLeavesSubstituteEnabled toggles canonical substitution at runtime. See internLeavesSubstituteEnabled.

func SetParserCoreCorridorEnabledForTest ¶ added in v0.49.0

func SetParserCoreCorridorEnabledForTest(on bool) func()

SetParserCoreCorridorEnabledForTest overrides the corridor gate for one test process. Restore the previous value (the returned func) when done. It mirrors the override pattern the compact core already uses for its own gates.

func SetParserCoreEagerMaterializationEnabledForTest ¶ added in v0.53.0

func SetParserCoreEagerMaterializationEnabledForTest(on bool) func()

SetParserCoreEagerMaterializationEnabledForTest overrides the eager gate for one test process. Restore the previous value (the returned func) when done.

func SetRawShapeElisionDisabledForDiagnostics ¶ added in v0.27.0

func SetRawShapeElisionDisabledForDiagnostics(disabled bool)

SetRawShapeElisionDisabledForDiagnostics toggles rawShapeElisionDisabledForDiagnostics. See its doc comment: this is a diagnostics-only hook, not part of the parser's production behavior contract.

func UnwrapLanguageBlobEnvelope ¶ added in v0.23.1

func UnwrapLanguageBlobEnvelope(data []byte) (compressed []byte, expectsTrailer bool, err error)

UnwrapLanguageBlobEnvelope returns the gzip payload and whether its envelope requires a non-empty LargeStateGotos trailer. Legacy gzip blobs are returned unchanged with expectsTrailer=false, preserving the original wire format.

func Walk ¶

func Walk(node *Node, fn func(node *Node, depth int) WalkAction)

Walk performs a depth-first traversal of the syntax tree rooted at node. The callback receives each node and its depth (0 for the starting node). Return WalkSkipChildren to skip a node's children, or WalkStop to end early.

func WrapLanguageBlobEnvelope ¶ added in v0.23.1

func WrapLanguageBlobEnvelope(compressed []byte) ([]byte, error)

WrapLanguageBlobEnvelope wraps a gzip-compressed Language blob whose decompressed stream contains a LargeStateGotos trailer. The outer magic is deliberately not a gzip header: runtimes predating trailer support therefore reject the blob at gzip.NewReader instead of successfully decoding the gob payload with LargeStateGotos missing.

The envelope is versioned and length-delimited so current runtimes also fail closed on unknown versions, flags, truncation, and trailing bytes. Callers must only wrap blobs that actually contain a non-empty trailer.

func WrapLanguageBlobVersionHeader ¶ added in v0.54.0

func WrapLanguageBlobVersionHeader(payload []byte, minRuntimeVersion uint32, generatorVersion string) ([]byte, error)

WrapLanguageBlobVersionHeader prepends a version header to payload (which may be a legacy gzip blob or an already-GTSBLOB-enveloped blob -- this wrapper is independent of, and layered outside, that inner envelope). The header records the blob schema version, the minimum runtime version required to load it, and an optional generator identity string.

Types ¶

type ASCIIEquivalenceExternalScanner ¶ added in v0.53.0

type ASCIIEquivalenceExternalScanner interface {
	ExternalScanner
	ExternalScannerASCIIEquivalenceClass(byte) uint8
}

ASCIIEquivalenceExternalScanner classifies interchangeable ASCII bytes. Zero means unknown. Equal nonzero classes certify substitutions between bytes. For every payload and valid-symbol set, each Scan must preserve its outcome, cursor, marks, result symbol, and final scanner state, including state omitted from serialization. The maximum examined position must also remain equal. This includes failed scans and substitutions outside the returned token. The guarantee applies at every scan origin and in every surrounding source. It does not certify statelessness or subtree reuse. The classification must be pure and immutable for each scanner binding. Bytes outside ASCII must return zero.

type AmbiguityKey ¶ added in v0.17.0

type AmbiguityKey struct {
	State                          StateID
	Lookahead                      Symbol
	ActionCount                    uint8
	ShiftCount                     uint8
	ReduceCount                    uint8
	ReduceSymbol                   Symbol
	ChildCount                     uint8
	ProductionID                   uint16
	ReduceChainTerminalState       StateID
	ReduceChainTerminalActionClass uint8
}

AmbiguityKey identifies one parse-table ambiguity bucket.

type AmbiguityProfile ¶ added in v0.17.0

type AmbiguityProfile struct {
	// contains filtered or unexported fields
}

AmbiguityProfile aggregates parser states/lookaheads that contribute to GLR fanout. It is intended for diagnostics and benchmark runs, not normal API use.

func NewAmbiguityProfile ¶ added in v0.17.0

func NewAmbiguityProfile() *AmbiguityProfile

NewAmbiguityProfile creates an empty GLR ambiguity profile.

func (*AmbiguityProfile) Reset ¶ added in v0.17.0

func (p *AmbiguityProfile) Reset()

Reset clears all accumulated ambiguity counters.

func (*AmbiguityProfile) SnapshotReduceChainTotals ¶ added in v0.19.0

func (p *AmbiguityProfile) SnapshotReduceChainTotals() AmbiguityStat

SnapshotReduceChainTotals returns aggregate deterministic reduce-chain run counters across all profiled start states/lookaheads.

func (*AmbiguityProfile) SnapshotTop ¶ added in v0.17.0

func (p *AmbiguityProfile) SnapshotTop(limit int) []AmbiguityStat

SnapshotTop returns the highest-impact ambiguity buckets ordered by stack pressure, then hit count.

func (*AmbiguityProfile) SnapshotTopMergeStates ¶ added in v0.19.0

func (p *AmbiguityProfile) SnapshotTopMergeStates(limit int) []AmbiguityStat

SnapshotTopMergeStates returns parser states that most often participate in multi-stack merge passes. These rows are keyed by state only, because merge happens before the next lookahead dispatch.

func (*AmbiguityProfile) SnapshotTopReduceChainRuns ¶ added in v0.19.0

func (p *AmbiguityProfile) SnapshotTopReduceChainRuns(limit int) []AmbiguityStat

SnapshotTopReduceChainRuns returns the starting states/lookaheads that begin the most expensive deterministic reduce chains.

func (*AmbiguityProfile) SnapshotTopReduceChains ¶ added in v0.19.0

func (p *AmbiguityProfile) SnapshotTopReduceChains(limit int) []AmbiguityStat

SnapshotTopReduceChains returns the parser states/lookaheads that spent the most time in deterministic reduce-chain fusion.

type AmbiguityStat ¶ added in v0.17.0

type AmbiguityStat struct {
	State                          StateID
	Lookahead                      Symbol
	ActionCount                    uint8
	ShiftCount                     uint8
	ReduceCount                    uint8
	ReduceSymbol                   Symbol
	ChildCount                     uint8
	ProductionID                   uint16
	Actions                        []ParseAction
	Hits                           uint64
	Forks                          uint64
	MultiStackHits                 uint64
	StackInTotal                   uint64
	StackInMax                     int
	ReduceChainHits                uint64
	ReduceChainSteps               uint64
	ReduceChainMaxLen              int
	ReduceChainNanos               int64
	ReduceChainRuns                uint64
	ReduceChainClassHits           uint64
	ReduceChainStopNoAction        uint64
	ReduceChainStopMulti           uint64
	ReduceChainStopShift           uint64
	ReduceChainStopAccept          uint64
	ReduceChainStopDead            uint64
	ReduceChainStopCycle           uint64
	ReduceChainStopLimit           uint64
	ReduceChainTerminalState       StateID
	ReduceChainTerminalActionClass uint8
	ActionNanos                    int64
	ExtraShiftNanos                int64
	NoActionNanos                  int64
	ConflictChoiceNanos            int64
	ConflictForkNanos              int64
	SingleShiftNanos               int64
	SingleReduceNanos              int64
	SingleAcceptNanos              int64
	SingleRecoverNanos             int64
	SingleOtherNanos               int64
	MergeCalls                     uint64
	MergeStacksIn                  uint64
	MergeStacksOut                 uint64
	MergeStacksInMax               int
	MergeStacksOutMax              int
}

AmbiguityStat is a snapshot row from AmbiguityProfile.

type ArenaBreakdown ¶ added in v0.18.0

type ArenaBreakdown struct {
	CompactReuseDependencyBytesAllocated int64

	NodeStructBytesAllocated            int64
	NodeFieldMetadataBytesAllocated     int64
	NoTreeNodeBytesAllocated            int64
	CompactFullLeafBytesAllocated       int64
	PendingParentBytesAllocated         int64
	PendingChildEntryBytesAllocated     int64
	RawShapeBytesAllocated              int64
	RawShapeChildBytesAllocated         int64
	RawShapeHashCacheBytesAllocated     int64
	FinalChildSidecarBytesAllocated     int64
	MissingNodeDependencyBytesAllocated int64
	CompactCheckpointLeafBytesAllocated int64
	MissingNodeDependencyCount          uint64
	PendingChildEntriesAllocated        uint64
	PendingChildEntryCapacity           uint64
	PendingChildEntryWaste              uint64
	ChildSliceBytesAllocated            int64
	FieldIDBytesAllocated               int64
	FieldSourceBytesAllocated           int64
	MergeScratchBytesAllocated          int64

	ArenaNodesConstructed uint64
	// NodeLiveCount is arena allocation-slot usage, not root-reachable tree
	// liveness. It includes parser alternatives and recovery nodes allocated
	// during the parse.
	NodeLiveCount                     uint64
	NodeCapacityCount                 uint64
	NodeCapacityWaste                 uint64
	PrimaryNodeCapacity               uint64
	PrimaryNodeUsed                   uint64
	OverflowNodeCapacity              uint64
	OverflowNodeUsed                  uint64
	OverflowNodeSlabs                 uint64
	LargestNodeSlabUsedFraction       float64
	LeafNodesConstructed              uint64
	ParentNodesConstructed            uint64
	FieldedParentNodesConstructed     uint64
	UnfieldedParentNodesConstructed   uint64
	ParentConstructedChildLen0        uint64
	ParentConstructedChildLen1        uint64
	ParentConstructedChildLen2        uint64
	ParentConstructedChildLen3        uint64
	ParentConstructedChildLen4Plus    uint64
	ParentConstructedNoLinks          uint64
	ParentConstructedWithLinks        uint64
	ParentConstructedTrackErrors      uint64
	ParentConstructedFieldSources     uint64
	ParentReductionVisible            uint64
	ParentReductionInvisible          uint64
	ParentReductionVisibleFielded     uint64
	ParentReductionVisibleUnfielded   uint64
	ParentReductionInvisibleFielded   uint64
	ParentReductionInvisibleUnfielded uint64
	ParentReductionVisibleChildPtrs   uint64
	ParentReductionInvisibleChildPtrs uint64
	ParentReductionVisibleLen0        uint64
	ParentReductionVisibleLen1        uint64
	ParentReductionVisibleLen2        uint64
	ParentReductionVisibleLen3        uint64
	ParentReductionVisibleLen4Plus    uint64
	ParentReductionInvisibleLen0      uint64
	ParentReductionInvisibleLen1      uint64
	ParentReductionInvisibleLen2      uint64
	ParentReductionInvisibleLen3      uint64
	ParentReductionInvisibleLen4Plus  uint64
	ReduceChildSlicesFastGSS          uint64
	ReduceChildPointersFastGSS        uint64
	ReduceChildSlicesAllVisible       uint64
	ReduceChildPointersAllVisible     uint64
	ReduceChildSlicesScratchGeneral   uint64
	ReduceChildPointersScratchGeneral uint64
	ReduceChildSlicesScratchNoAlias   uint64
	ReduceChildPointersScratchNoAlias uint64
	CollapseRawUnaryAttempts          uint64
	CollapseRawUnarySuccesses         uint64
	CollapseRawUnaryMissShape         uint64
	CollapseRawUnaryMissGrammar       uint64
	CollapseRawUnaryMissChild         uint64
	CollapseRawUnaryMissRule          uint64
	CollapseUnaryAttempts             uint64
	CollapseUnarySuccesses            uint64
	CollapseUnaryMissShape            uint64
	CollapseUnaryMissGrammar          uint64
	CollapseUnaryMissFielded          uint64
	CollapseUnaryMissChild            uint64
	CollapseUnaryMissRule             uint64
	CollapseRuleSameSymbol            uint64
	CollapseRuleInvisibleWrapper      uint64
	CollapseRuleNamedLeafAlias        uint64
	NoTreeReduceNodesConstructed      uint64
	NoTreeLeafNodesConstructed        uint64
	NoTreePlaceholderNodesConstructed uint64
	OtherNodesConstructed             uint64
	ExtraNodesConstructed             uint64
	ErrorSymbolNodesConstructed       uint64
	HasErrorNodesConstructed          uint64
	ChildSlicesConstructed            uint64
	ChildPointersConstructed          uint64
	ChildSlicesLen1                   uint64
	ChildSlicesLen2                   uint64
	ChildSlicesLen3                   uint64
	ChildSlicesLen4Plus               uint64
	ParentChildPointersConstructed    uint64
	ParentChildrenLen0                uint64
	ParentChildrenLen1                uint64
	ParentChildrenLen2                uint64
	ParentChildrenLen3                uint64
	ParentChildrenLen4Plus            uint64
	FieldIDElementsConstructed        uint64
	FieldSourceElementsConstructed    uint64
}

ArenaBreakdown captures optional arena/materialization attribution. It is populated only when EnableArenaBreakdown(true) is set before parsing.

type ArenaProfile ¶ added in v0.6.0

type ArenaProfile struct {
	IncrementalAcquire uint64
	IncrementalNew     uint64
	FullAcquire        uint64
	FullNew            uint64
}

ArenaProfile captures node arena allocation statistics. Enable with EnableArenaProfile(true) and retrieve with ArenaProfileSnapshot(). The counters are plain package-level state, not atomic: read and write them from a single goroutine only, with no parse running concurrently.

func ArenaProfileSnapshot ¶ added in v0.6.0

func ArenaProfileSnapshot() ArenaProfile

ArenaProfileSnapshot returns current arena pool counters. This debug hook is not concurrency-safe and is intended for single-threaded benchmark/profiling runs.

type BoundTree ¶

type BoundTree struct {
	// contains filtered or unexported fields
}

BoundTree pairs a Tree with its Language and source, eliminating the need to pass *Language and []byte to every node method call.

func Bind ¶

func Bind(tree *Tree) *BoundTree

Bind creates a BoundTree from a Tree. The Tree must have been created with a Language (via NewTree or a Parser). Returns a BoundTree that delegates to the underlying Tree's Language and Source.

func (*BoundTree) ChildByField ¶

func (bt *BoundTree) ChildByField(n *Node, fieldName string) *Node

ChildByField returns the first child assigned to the given field name.

func (*BoundTree) Language ¶

func (bt *BoundTree) Language() *Language

Language returns the tree's language.

func (*BoundTree) NodeText ¶

func (bt *BoundTree) NodeText(n *Node) string

NodeText returns the source text covered by the node.

func (*BoundTree) NodeType ¶

func (bt *BoundTree) NodeType(n *Node) string

NodeType returns the node's type name, resolved via the bound language.

func (*BoundTree) ParseRuntime ¶ added in v0.52.0

func (bt *BoundTree) ParseRuntime() ParseRuntime

ParseRuntime returns diagnostics from the parse that built the tree.

func (*BoundTree) ParseStopReason ¶ added in v0.52.0

func (bt *BoundTree) ParseStopReason() ParseStopReason

ParseStopReason reports why parsing ended.

func (*BoundTree) ParseStoppedEarly ¶ added in v0.52.0

func (bt *BoundTree) ParseStoppedEarly() bool

ParseStoppedEarly reports whether parsing hit an early-stop condition.

func (*BoundTree) Release ¶

func (bt *BoundTree) Release()

Release releases the underlying tree's arena memory.

func (*BoundTree) RootNode ¶

func (bt *BoundTree) RootNode() *Node

RootNode returns the tree's root node.

func (*BoundTree) Source ¶

func (bt *BoundTree) Source() []byte

Source returns the tree's source bytes.

func (*BoundTree) TreeCursor ¶ added in v0.6.0

func (bt *BoundTree) TreeCursor() *TreeCursor

TreeCursor returns a new TreeCursor starting at the tree's root node.

type ByteSkippableTokenSource ¶

type ByteSkippableTokenSource interface {
	TokenSource
	SkipToByte(offset uint32) Token
}

ByteSkippableTokenSource can jump to a byte offset and return the first token at or after that position.

type CRecoveryGateDiagnostics ¶ added in v0.21.0

type CRecoveryGateDiagnostics struct {
	Supported bool
	Reason    string

	StateCount       int
	SymbolCount      int
	TokenCount       int
	LexModeCount     int
	LexStateCount    int
	ParseActionCount int

	HasExternalScanner     bool
	ExternalSymbolCount    int
	ExternalTokenCount     int
	ExternalLexStateRows   int
	ExternalLexStateMinLen int
}

CRecoveryGateDiagnostics describes the runtime validation result for the C recovery-cost competition gate. Reason is empty when Supported is true; otherwise it names the first failed validation check.

func CertifyCRecoveryCostCompetition ¶ added in v0.21.0

func CertifyCRecoveryCostCompetition(lang *Language) CRecoveryGateDiagnostics

CertifyCRecoveryCostCompetition validates a language's runtime recovery surface and updates the C recovery metadata.

Capability means the tables satisfy the runtime gate. Default enablement is narrower: grammars with external recovery metadata are enabled only after an actual external scanner is attached and precise ExternalLexStates are available.

func CertifyGeneratedCRecoveryCostCompetition ¶ added in v0.21.0

func CertifyGeneratedCRecoveryCostCompetition(lang *Language) CRecoveryGateDiagnostics

CertifyGeneratedCRecoveryCostCompetition is kept for existing generated grammar call sites. New callers should use CertifyCRecoveryCostCompetition.

func DiagnoseCRecoveryGate ¶ added in v0.21.0

func DiagnoseCRecoveryGate(lang *Language) CRecoveryGateDiagnostics

DiagnoseCRecoveryGate validates the runtime table surface required by the faithful C recovery-cost competition path and returns the first failure.

The full validation scans every parse-table row, so the result is memoized per Language behind an input fingerprint (cRecoveryGateCacheKey): callers on parse-adjacent paths (errorCostCompetitionLanguage via gap-token replay and token-source construction) would otherwise redo an O(tables) scan per call, which measurably dominated error-heavy parses.

type CallRef ¶ added in v0.20.6

type CallRef struct {
	Lang          string
	Kind          string
	Name          string
	Receiver      string
	NodeType      string
	StartByte     uint32
	EndByte       uint32
	NameStartByte uint32
	NameEndByte   uint32
}

CallRef is a compact language-neutral call-site reference extracted from a syntax tree.

func ExtractCalls ¶ added in v0.20.6

func ExtractCalls(tree *Tree) []CallRef

ExtractCalls returns language-neutral call-site references for common code-understanding workflows.

type CheckpointedExternalScanner ¶ added in v0.47.0

type CheckpointedExternalScanner interface {
	ExternalScanner
	UsesExternalScannerCheckpoints() bool
}

CheckpointedExternalScanner is implemented by stateful external scanners whose non-empty serialized payload is a complete checkpoint of every value that can affect a later Scan call. The runtime records that checkpoint at token boundaries and restores it when fast-forwarding across a reused subtree. A scanner may return a zero-length serialization for a reachable state that cannot be represented exactly; that boundary is recorded as checkpoint-absent and fails closed for incremental reuse.

Implementations must encode the empty payload as a non-empty byte sequence: a zero-length serialization is reserved by the checkpoint store to mean "checkpoint absent". A non-empty serialization must never truncate, alias, or otherwise collide with a different scanner state. Returning true is a soundness claim; an incomplete non-empty checkpoint can cause silent incremental corruption.

type CheckpointlessExternalScannerReuse ¶ added in v0.47.0

type CheckpointlessExternalScannerReuse interface {
	ExternalScanner
	AllowsIncrementalReuseWithoutCheckpoint() bool
}

CheckpointlessExternalScannerReuse is implemented by checkpointed scanners that can additionally prove subtree reuse safe when a candidate node has no scanner checkpoint. Most stateful scanners must not implement this: absence of a checkpoint then fails closed.

type CompactRecoverEOFArtifactReceipt ¶ added in v0.52.0

type CompactRecoverEOFArtifactReceipt struct {
	BlobSHA256        [32]byte
	TerminalSymbol    Symbol
	EOFState          StateID
	EOFByteOffset     uint32
	Passes            uint64
	Elections         uint64
	ActionLookups     uint64
	Dispatches        uint64
	OrdinaryShifts    uint64
	OrdinaryCohorts   uint64
	ExtraShifts       uint64
	ExtraCohorts      uint64
	Reductions        uint64
	Conflicts         uint64
	ConflictActions   uint64
	Forks             uint64
	RepetitionFolds   uint64
	RecoveryWork      uint64
	NoActionDrops     uint64
	ReductionPauses   uint64
	Accepts           uint64
	Canonicalizations uint64
	PeakHeaders       uint64
}

CompactRecoverEOFArtifactReceipt is the locked-C certification data for one compact recover_eof boundary. A zero BlobSHA256 disables this route. The receipt binds the exact grammar artifact to the terminal, parser boundary, and compact scheduler facts for the locked-C-certified witness.

The work fields are intentionally explicit. They prevent a Boolean grant from admitting a parse after an unmodeled shift, reduction, conflict, fork, or recovery operation changes the pre-EOF lineage.

type CompactRecoveryTerminalAliasRule ¶ added in v0.52.0

type CompactRecoveryTerminalAliasRule struct {
	ResumeState  StateID
	ResumeSymbol Symbol
	AliasSymbol  Symbol
}

CompactRecoveryTerminalAliasRule certifies one terminal alias that can survive compact strategy-2 recovery. ResumeState and ResumeSymbol identify the exact recovery resume. AliasSymbol identifies the published leaf.

Exact built-in profiles bind these values to one grammar blob. Languages without a rule keep the accepted-root leaf audit fail-closed.

type ConflictPolicy ¶ added in v0.21.0

type ConflictPolicy struct {
	State     StateID
	Lookahead Symbol
	Kind      ConflictPolicyKind
	// CompactOnly prevents the production GLR parser from applying this
	// policy. The compact scheduler can still use it after its safety gates.
	CompactOnly bool
	// CompactMinFrontierHeaders limits this policy to a compact scheduler
	// frontier with at least this many live headers. Production ignores this
	// gate when CompactOnly is false.
	CompactMinFrontierHeaders uint16

	// ReduceSymbols, when non-empty, requires every reduce in the conflict to
	// reduce one of these symbols. The generic action-shape validator still
	// requires at least one reduce.
	ReduceSymbols []Symbol
}

ConflictPolicy describes one table row/lookahead conflict that can be collapsed deterministically after validating the action shape.

type ConflictPolicyKind ¶ added in v0.21.0

type ConflictPolicyKind uint8

ConflictPolicyKind identifies a deterministic conflict policy class.

const (
	ConflictPolicyNone ConflictPolicyKind = iota
	ConflictPolicyRepetitionShift
	ConflictPolicyShift
	// ConflictPolicyRecoveredRepetitionReduce is an exact-row certification
	// for a recovered lineage. It chooses the single reduce from a
	// {1 reduce, 1 repetition shift} row only after the lineage has previously
	// entered recovery and only while no recovery action is currently active.
	// Like other conflict policies, it is disabled during incremental reuse.
	//
	// Keep new kinds append-only: Language blobs encode these numeric values.
	ConflictPolicyRecoveredRepetitionReduce
	// ConflictPolicyRepetitionReduce is an exact-row certification for the
	// ordinary C repetition fold: one reduce wins over one repetition shift.
	ConflictPolicyRepetitionReduce
	// ConflictPolicyDeclaredReduceReduceHighestSymbol resolves a row whose
	// actions are all plain REDUCE and all reduce a symbol declared together
	// in the grammar's own conflicts list. C's ts_stack_merge folds such rows
	// deterministically: when the competing reductions later wrap into
	// shape-equivalent subtrees (same symbol, span, and child count one level
	// up), ts_stack_merge's shallow equivalence test keeps whichever subtree
	// was established first without a deep content comparison, and arrival
	// order is set by table-action order (the last action in a multi-action
	// row runs immediately; earlier actions are deferred and lose). Because
	// the generator lists same-row actions by ascending symbol id, "last
	// processed" is "highest symbol id" for this shape. See
	// declaredReduceReduceHighestSymbolConflictChoice.
	ConflictPolicyDeclaredReduceReduceHighestSymbol
)

type DefinitionSpan ¶ added in v0.20.6

type DefinitionSpan struct {
	Lang          string
	Kind          string
	Name          string
	NodeType      string
	StartByte     uint32
	EndByte       uint32
	NameStartByte uint32
	NameEndByte   uint32
}

DefinitionSpan is a compact language-neutral declaration span extracted from a syntax tree.

func EnclosingDefinition ¶ added in v0.20.6

func EnclosingDefinition(tree *Tree, byteOffset uint32) (DefinitionSpan, bool)

EnclosingDefinition returns the nearest definition node that contains byteOffset. It uses NodeAtByte and parent links instead of scanning every definition span.

func ExtractDefinitionSpans ¶ added in v0.20.6

func ExtractDefinitionSpans(tree *Tree) []DefinitionSpan

ExtractDefinitionSpans returns language-neutral declaration spans for common code-understanding workflows. The extractor is intentionally conservative: unsupported languages or declaration shapes are skipped rather than guessed.

type DiagnosticParserCoreBoundaryKind ¶ added in v0.46.0

type DiagnosticParserCoreBoundaryKind string
const (
	DiagnosticParserCoreExtra      DiagnosticParserCoreBoundaryKind = "extra"
	DiagnosticParserCoreExtraChain DiagnosticParserCoreBoundaryKind = "extra_chain"
	DiagnosticParserCoreNoAction   DiagnosticParserCoreBoundaryKind = "no_action"
	// DiagnosticParserCoreRecovery marks every dispatch shape where only
	// locked-C production's recovery semantics can continue: an explicit
	// ActionRecover cell, an unexpected recover action inside a generic
	// conflict, and (B3 stage S1) the pure no-table-action frontier that
	// mirrors C's cPaused trigger (glr.go: "the stack hit a no-action
	// point"). Dispatch classification only -- every one of these shapes
	// still declines and falls back to production unchanged.
	DiagnosticParserCoreRecovery      DiagnosticParserCoreBoundaryKind = "recovery"
	DiagnosticParserCoreAccept        DiagnosticParserCoreBoundaryKind = "accept_without_materialization"
	DiagnosticParserCoreCap           DiagnosticParserCoreBoundaryKind = "cap"
	DiagnosticParserCoreIdentity      DiagnosticParserCoreBoundaryKind = "identity"
	DiagnosticParserCoreRoute         DiagnosticParserCoreBoundaryKind = "unsupported_route"
	DiagnosticParserCoreGenericClosed DiagnosticParserCoreBoundaryKind = "generic_scheduler_closed"
)

type DiagnosticParserCoreClass1Candidate ¶ added in v0.49.0

type DiagnosticParserCoreClass1Candidate struct {
	Detail string
}

DiagnosticParserCoreClass1Candidate captures one v2-admits/scalar-declines election (spec.b4b-alternative-set.v2 section 7 class 1): the census records the drop and best-candidate-survivor context so a differential harness (or manual investigation) can re-run the owning file with v2 as the opt-in decider and compare the resulting tree against production and, per the campaign's C-oracle adjudication amendment, the locked C oracle.

func DiagnosticParserCoreClass1CandidatesForTest ¶ added in v0.49.0

func DiagnosticParserCoreClass1CandidatesForTest() []DiagnosticParserCoreClass1Candidate

DiagnosticParserCoreClass1CandidatesForTest returns every captured v2-admits/scalar-declines election context observed since the last reset (spec.b4b-alternative-set.v2 section 7 class 1). The stage 2b gate: zero of these may carry a differing tree from the C-oracle-adjudicated production route across the corpora in scope.

type DiagnosticParserCoreDispatchRound ¶ added in v0.46.0

type DiagnosticParserCoreDispatchRound struct {
	Index   int
	Before  []DiagnosticParserCoreHeaderReceipt
	Actions []DiagnosticParserCoreRoundAction
	After   []DiagnosticParserCoreHeaderReceipt
}

type DiagnosticParserCoreElection ¶ added in v0.46.0

type DiagnosticParserCoreElection struct {
	States                 []StateID
	Token                  Token
	ScannerBefore          DiagnosticParserCoreScannerCheckpoint
	ScannerAfter           DiagnosticParserCoreScannerCheckpoint
	CurrentCheckpointValid bool
	CurrentCheckpointStart DiagnosticParserCoreScannerCheckpoint
	CurrentCheckpointEnd   DiagnosticParserCoreScannerCheckpoint
	CurrentCheckpointBytes [2]uint32
}

type DiagnosticParserCoreGenericAcceptance ¶ added in v0.46.0

type DiagnosticParserCoreGenericAcceptance struct {
	ElectionIndex  int
	Token          Token
	Header         DiagnosticParserCoreHeaderPathReceipt
	Payloads       []uint32
	Score          int64
	BranchOrder    uint64
	HasBranchOrder bool
	// MaterialityCertified records the bounded public-tree comparison that
	// makes a multi-derivation selection safe without a certified primary.
	MaterialityCertified bool
	// RecoveredElectionCertified records an authenticated C recovered-root fold.
	RecoveredElectionCertified bool
	// StructuralElectionCertified records an exact artifact's C-order proof.
	// It makes a clean multi-derivation selection safe without a primary proof.
	StructuralElectionCertified bool
	CoreWork                    core.Work
	Accepts                     uint64
	SelectedNodes               uint64
	SelectedParents             uint64
	SelectedLeaves              uint64
	Stats                       core.Stats
	Work                        DiagnosticParserCoreGenericWork
}

DiagnosticParserCoreGenericAcceptance records an authenticated EOF accept after the compact frontier has selected one derivation. A materiality- certified selection may represent more than one live derivation when every candidate publishes the same public tree. Payloads are the selected bottom-to-top compact stack; materialization does not mutate that graph.

type DiagnosticParserCoreGenericCompletion ¶ added in v0.46.0

type DiagnosticParserCoreGenericCompletion struct {
	TargetByte    uint32
	ElectionIndex int
	LastToken     Token
	State         StateID
	Headers       []DiagnosticParserCoreHeaderPathReceipt
	Stats         core.Stats
	Work          DiagnosticParserCoreGenericWork
}

DiagnosticParserCoreGenericCompletion is a caller-selected, successfully closed scheduler frontier. LastToken is consumed; no pending lookahead has been read.

type DiagnosticParserCoreGenericConflict ¶ added in v0.46.0

type DiagnosticParserCoreGenericConflict struct {
	ElectionIndex            int
	Token                    Token
	HeaderIndex              int
	BranchOrderBefore        uint64
	BranchOrderAfter         uint64
	NextCreationSeqBefore    uint64
	NextCreationSeqAfter     uint64
	Round                    DiagnosticParserCoreDispatchRound
	Prefix                   []DiagnosticParserCoreHeaderReceipt
	PrimaryOutput            DiagnosticParserCoreHeaderReceipt
	PrimaryPaused            bool
	PrimaryAdopted           bool
	OriginalSuffix           []DiagnosticParserCoreHeaderReceipt
	SecondaryArms            []DiagnosticParserCoreGenericConflictArm
	AdditionalPrimaryOutputs []DiagnosticParserCoreHeaderReceipt
	After                    []DiagnosticParserCoreHeaderReceipt
}

type DiagnosticParserCoreGenericConflictArm ¶ added in v0.46.0

type DiagnosticParserCoreGenericConflictArm struct {
	Ordinal     int
	BranchOrder uint64
	Outputs     []DiagnosticParserCoreHeaderReceipt
	Paused      bool
	Adopted     bool
}

DiagnosticParserCoreGenericConflict records one table-driven conflict cell. Actions preserve execution order: secondary ordinals first, then primary.

type DiagnosticParserCoreGenericExternalShift ¶ added in v0.46.0

type DiagnosticParserCoreGenericExternalShift struct {
	ElectionIndex int
	Token         Token
	ScannerBefore DiagnosticParserCoreScannerCheckpoint
	ScannerAfter  DiagnosticParserCoreScannerCheckpoint
	RoundIndex    int
	Payloads      []DiagnosticParserCoreTerminalPayloadView
}

DiagnosticParserCoreGenericExternalShift ties every compact external terminal payload created by one generic scheduler round to its scanner-authenticated election without embedding scanner state in the compact graph. The round may be an ordinary or extra shift cohort, or a conflict with one or more shift arms.

type DiagnosticParserCoreGenericNoActionDrop ¶ added in v0.46.0

type DiagnosticParserCoreGenericNoActionDrop struct {
	ElectionIndex int
	Token         Token
	Header        DiagnosticParserCoreHeaderPathReceipt
}

DiagnosticParserCoreGenericNoActionDrop records a paused scheduler head removed only after a sibling made real progress in the same token epoch.

type DiagnosticParserCoreGenericScheduler ¶ added in v0.46.0

type DiagnosticParserCoreGenericScheduler struct {
	ReceiptMode          DiagnosticParserCoreReceiptMode
	StartCheckpoint      DiagnosticParserCoreScannerCheckpoint
	StartHeaders         []DiagnosticParserCoreHeaderPathReceipt
	Rounds               []DiagnosticParserCoreDispatchRound
	Conflicts            []DiagnosticParserCoreGenericConflict
	ExternalShifts       []DiagnosticParserCoreGenericExternalShift
	Elections            []DiagnosticParserCoreElection
	VersionLexerRequests []DiagnosticParserCoreVersionLexerRequest
	NoActionDrops        []DiagnosticParserCoreGenericNoActionDrop
	Completion           *DiagnosticParserCoreGenericCompletion
	Acceptance           *DiagnosticParserCoreGenericAcceptance

	Stop                             DiagnosticParserCoreGenericStop
	Tokens                           uint64
	Dispatches                       uint64
	GlobalBranchOrder                uint64
	NextCreationSeq                  uint64
	PerVersionLexRequests            uint64
	PerVersionLexRestores            uint64
	PerVersionLexPublications        uint64
	PerVersionLexAcceptedRaggedSpans uint64
	PerVersionLexViabilityDrops      uint64
	PeakLiveVersions                 uint64
	PotentialReductionActions        uint64
	PotentialReductionOutputs        uint64
	ReductionPromotions              uint64
	MissingTokenTrials               uint64
	MissingTokenCommits              uint64
	RecoveryDiscontinuityMerges      uint64
	RecoveryCeilingDeclines          uint64
	// contains filtered or unexported fields
}

DiagnosticParserCoreGenericScheduler records one committed compact scheduler run from the sole authenticated seed lifecycle before its first election.

type DiagnosticParserCoreGenericStop ¶ added in v0.46.0

type DiagnosticParserCoreGenericStop struct {
	Boundary      DiagnosticParserCoreBoundaryKind
	Detail        string
	ElectionIndex int
	HeaderIndex   int
	State         StateID
	ByteOffset    uint32
	Token         Token
	Headers       []DiagnosticParserCoreHeaderPathReceipt
	Stats         core.Stats
	Work          DiagnosticParserCoreGenericWork
}

DiagnosticParserCoreGenericStop is the first semantic the table-driven clean scheduler deliberately does not implement.

type DiagnosticParserCoreGenericWork ¶ added in v0.46.0

type DiagnosticParserCoreGenericWork struct {
	Passes uint64
	// PotentialReductionActions counts C-style any-terminal reduction actions
	// examined by the staged S5 frontier.
	PotentialReductionActions uint64
	// PotentialReductionOutputs counts physical outputs returned by staged S5
	// reduction actions.
	PotentialReductionOutputs uint64
	// ReductionPromotions counts outputs promoted into an earlier local source
	// slot after a positive-child reduction.
	ReductionPromotions uint64
	// MissingTokenTrials counts viable missing-terminal trial candidates.
	MissingTokenTrials uint64
	// MissingTokenCommits counts committed missing-terminal versions.
	MissingTokenCommits uint64
	// RecoveryDiscontinuityMerges counts committed recovery-head merges.
	RecoveryDiscontinuityMerges uint64
	// RecoveryCeilingDeclines counts staged recovery searches that hit a
	// configured ceiling and then declined.
	RecoveryCeilingDeclines uint64
	// StackSummaryRecoveryForks counts S4 forks that retain both the
	// ancestor-recovered lineage and the error-absorb lineage.
	StackSummaryRecoveryForks uint64
	// RecoveryLineageSelections counts accepting frontiers resolved by pricing
	// competing recovery lineages instead of declining. Every other
	// head-removal site in this scheduler accounts for itself; without this
	// counter a parse that arbitrated three competitors is indistinguishable
	// in the receipt from one that never competed, and no differential harness
	// could confirm the port picks what C picks.
	// RecoveryRootSelections counts completed recovered-path elections.
	RecoveryRootSelections    uint64
	RecoveryLineageSelections uint64
	// RecoveryLineageRetirements counts C-condense-tail transitions that remove
	// one trailing no-action recovery version after an earlier version shifts.
	RecoveryLineageRetirements uint64
	// RecoveryAmbiguityForks counts ordinary grammar forks whose source already
	// belongs to a recovery competition. The outputs retain recovery cost.
	RecoveryAmbiguityForks uint64
	// RecoveryCondensePasses counts completed C-style recovery condensations.
	RecoveryCondensePasses uint64
	// RecoveryVersionCapDrops counts histories removed because their distinct
	// C merge key ranked after the six-version recovery limit.
	RecoveryVersionCapDrops uint64
	// SingleHeaderPasses counts dispatch passes executed against a
	// single-header frontier (spec.c4-bytecode-isa.v1 section 5, obligation
	// R6). It is count-only and published on the gts_workcount board so
	// corridor coverage is a committed board row rather than a
	// profile-derived figure. It is a strict subset of Passes.
	SingleHeaderPasses uint64
	// CorridorPasses counts the subset of SingleHeaderPasses the C4 bytecode
	// corridor executed. It is zero on every build and every parse with the
	// corridor lane off, so it never perturbs the pinned board.
	CorridorPasses             uint64
	ActionLookups              uint64
	Dispatches                 uint64
	Conflicts                  uint64
	ConflictActions            uint64
	Forks                      uint64
	ConflictActionArmsAdmitted uint64
	CausalConflictForks        uint64
	ConflictHeads              uint64
	// ConvergedReductionSplitDrops counts no-action drops descended from a
	// reduction that split multiple compact predecessor paths into live heads.
	ConvergedReductionSplitDrops uint64
	// ConvergedCoverageDrops counts converged split drops whose dropped
	// header's recorded alternative set was contained in one surviving,
	// non-blended header's recorded set (spec.b4b-alternative-set.v2 section
	// 5, the revised theorem). Renamed from SelectedLineageDrops at stage
	// 2b, when the v2 containment predicate replaced the scalar (rank,
	// lineage) proof as the deciding proof.
	ConvergedCoverageDrops uint64
	RepetitionFolds        uint64
	Reductions             uint64
	OrdinaryShifts         uint64
	OrdinaryCohorts        uint64
	ExtraShifts            uint64
	ExtraCohorts           uint64
	Accepts                uint64
	// RecoverEOFAccepts counts one certified live publication of C's
	// recover_eof ERROR root. It is distinct from ordinary ActionAccept work.
	RecoverEOFAccepts uint64
	ReductionPauses   uint64
	NoActionDrops     uint64
	Elections         uint64
	// PerVersionLexRequests counts lexer calls issued for an owned parser
	// version. The shared lexer election does not contribute to this counter.
	PerVersionLexRequests uint64
	// PerVersionLexRestores counts exact DFA and scanner snapshot restores
	// before an owned request or a state-dependent re-request.
	PerVersionLexRestores uint64
	// PerVersionLexPublications counts immutable snapshots published to a
	// parser header or its scheduler sidecar.
	PerVersionLexPublications uint64
	// PerVersionLexAcceptedRaggedSpans counts different-width token views that
	// the owned scheduler accepted instead of declining at a shared cursor.
	PerVersionLexAcceptedRaggedSpans uint64
	// PerVersionLexViabilityDrops counts owned versions removed only after
	// their own tokens exhausted reductions without an action and a sibling's
	// token shifted from the same byte. These are not grammar-ambiguity drops.
	PerVersionLexViabilityDrops uint64
	// PeakLiveVersions records the largest live owned-header frontier.
	PeakLiveVersions  uint64
	Canonicalizations uint64
	PeakHeaders       uint64
	// ZeroWidthCatchUpMissedMerge counts an ownedZeroWidthCatchUp call whose
	// own header (identified by creationSeq) could not be found: the
	// committed zero-width shift's own canonicalizeOwned call (inside
	// applyGenericShifts/applyGenericExtraShifts, run before
	// ownedZeroWidthCatchUp) already folded that header into a different
	// surviving one before the reopen mark could be set. Not a correctness
	// failure -- the merge target still closes its own barrier normally --
	// but the ragged-drop rescue this election's shift was meant to unlock
	// never activates for it, so a differential harness needs this counter,
	// not a silent no-op, to notice the mechanism did not fire.
	ZeroWidthCatchUpMissedMerge uint64
	Overflow                    bool
}

DiagnosticParserCoreGenericWork records semantic scheduler work separately from the compact core's physical arena storage.

type DiagnosticParserCoreHeaderPathReceipt ¶ added in v0.46.0

type DiagnosticParserCoreHeaderPathReceipt struct {
	Header               DiagnosticParserCoreHeaderReceipt
	Derivations          []DiagnosticParserCorePackedDerivation
	DerivationsTruncated bool
}

type DiagnosticParserCoreHeaderReceipt ¶ added in v0.46.0

type DiagnosticParserCoreHeaderReceipt struct {
	CreationSeq uint64
	State       StateID
	ByteOffset  uint32
	Shifted     bool
	Accepted    bool
	Paused      bool
	ExactPaths  uint64
	Checkpoint  [32]byte
}

type DiagnosticParserCorePackedDerivation ¶ added in v0.46.0

type DiagnosticParserCorePackedDerivation struct {
	Score          int64
	BranchOrder    uint64
	HasBranchOrder bool
}

type DiagnosticParserCorePrefixOptions ¶ added in v0.46.0

type DiagnosticParserCorePrefixOptions struct {
	Recovery       bool
	Retry          bool
	Incremental    bool
	IncludedRanges bool
	// GenericStopAtClosedByte publishes a successful closed-frontier receipt
	// when every authenticated scheduler head closes at this byte. Nil is
	// unbounded. The boundary is checked before another scanner election.
	GenericStopAtClosedByte *uint32
	ReceiptMode             DiagnosticParserCoreReceiptMode

	// DisablePerHeaderSpanUnlockedRelex restores relexTokenForState's
	// pre-D2-1 span-locked probe (only an exact-span relex is eligible; a
	// relex whose EndByte differs from the shared election is declined the
	// same way a scan failure is, never routed through owned lexer
	// activation). The zero value keeps the span-unlocked probe on. This is a
	// test-and-diagnostic lever only: no production caller sets it, and it
	// has no operator-facing surface (config flag, CLI flag, or similar).
	DisablePerHeaderSpanUnlockedRelex bool

	MaxDispatches uint64
	MaxTokens     uint64
	Limits        core.Limits
	// contains filtered or unexported fields
}

type DiagnosticParserCorePrefixResult ¶ added in v0.46.0

type DiagnosticParserCorePrefixResult struct {
	Boundary          DiagnosticParserCoreBoundaryKind
	Detail            string
	Dispatches        uint64
	Tokens            uint64
	State             StateID
	Lookahead         Token
	LastBranchOrder   uint64
	GenericScheduler  *DiagnosticParserCoreGenericScheduler
	Completed         bool
	Elections         []DiagnosticParserCoreElection
	SourceSHA256      [32]byte
	GrammarBlobSHA256 [32]byte
	Grammar           string
	ExactRootDFA      bool
	Materialized      bool
	// MaterializedTree is a structural diagnostic owned by the caller and must
	// be released. It is set only after authenticated EOF acceptance and
	// one-shot compact-tree materialization succeed. The diagnostic runner does
	// not force parser-state replay, so its default tree retains the hard
	// incremental-reuse bar. The production admission runner separately forces
	// replay and may clear that bar only when materialization proves the required
	// states and scanner quiescence per tree.
	MaterializedTree *Tree
}

func DiagnosticParseParserCorePrefix ¶ added in v0.46.0

func DiagnosticParseParserCorePrefix(scanner ExternalScanner, source []byte, options DiagnosticParserCorePrefixOptions) (DiagnosticParserCorePrefixResult, error)

DiagnosticParseParserCorePrefix independently schedules one compact seed against the complete production DFA/scanner election stream. Unsupported boundaries remain fail-closed. It never calls the production parser.

type DiagnosticParserCoreReceiptMode ¶ added in v0.46.0

type DiagnosticParserCoreReceiptMode uint8

DiagnosticParserCoreReceiptMode controls diagnostic observation only. It never changes parser-core scheduling or selection semantics. The zero value preserves the complete historical receipt; summary mode retains only the authenticated result and aggregate work needed for larger-fixture study.

const (
	DiagnosticParserCoreReceiptFull DiagnosticParserCoreReceiptMode = iota
	DiagnosticParserCoreReceiptSummary
)

type DiagnosticParserCoreRoundAction ¶ added in v0.46.0

type DiagnosticParserCoreRoundAction struct {
	HeaderIndex int
	State       StateID
	ByteOffset  uint32
	Ordinal     int
	Action      ParseAction
	BranchOrder uint64
}

type DiagnosticParserCoreScannerCheckpoint ¶ added in v0.46.0

type DiagnosticParserCoreScannerCheckpoint struct {
	Length int
	SHA256 [32]byte
}

type DiagnosticParserCoreShadowCensusFalsifier ¶ added in v0.49.0

type DiagnosticParserCoreShadowCensusFalsifier struct {
	Detail string
}

DiagnosticParserCoreShadowCensusFalsifier captures one old-proved/new- unproved drop context for root-causing a stop-rule hit.

func DiagnosticParserCoreShadowCensusFalsifiersForTest ¶ added in v0.49.0

func DiagnosticParserCoreShadowCensusFalsifiersForTest() []DiagnosticParserCoreShadowCensusFalsifier

DiagnosticParserCoreShadowCensusFalsifiersForTest returns every captured scalar-proved/v2-unproved drop context observed since the last reset. Under the theorem this direction is expected (v2 is strictly more conservative), so a non-empty result documents class 2, not a stop rule by itself; DiagnosticParserCoreClass1CandidatesForTest is the stage 2b stop rule's own class.

type DiagnosticParserCoreShadowCensusTotals ¶ added in v0.49.0

type DiagnosticParserCoreShadowCensusTotals struct {
	// Agree: both proofs reached the same verdict (both proved or both
	// declined).
	Agree uint64
	// OldProvedNewUnproved is a design falsifier candidate (spec.b4b-
	// alternative-set.v2 section 7): the scalar proof proved a drop v2 could
	// not. Under the theorem this is expected and safe (v2 is strictly more
	// conservative, never less), so it is not itself a stop rule; class 2
	// below is its typed name.
	OldProvedNewUnproved uint64
	// NewProvedOldUnproved is the expected, desired direction: the v2 proof
	// is strictly more capable than the scalar proof on this drop.
	NewProvedOldUnproved uint64
	// NeitherProved: both proofs declined.
	NeitherProved uint64
}

DiagnosticParserCoreShadowCensusTotals tallies the comparison between the retired scalar rank/lineage proof (diagnosticParserCoreSelectedLineageDrops) and the v2 alternative-set containment proof (diagnosticParserCoreConvergedCoverageDropsV2, the live decider since stage 2b) across every converged-split no-action drop election observed while the census is enabled. Retained under its stage 1 name for callers already reading it; see DiagnosticParserCoreThreeProofCensusTotals for the full v1/v2/scalar breakdown stage 2a adds.

func DiagnosticParserCoreShadowCensusSnapshotForTest ¶ added in v0.49.0

func DiagnosticParserCoreShadowCensusSnapshotForTest() DiagnosticParserCoreShadowCensusTotals

DiagnosticParserCoreShadowCensusSnapshotForTest returns the accumulated stage 1 (scalar vs. v2) shadow-census totals since the last reset. A harness resets around one language's corpus run to attribute totals to that language.

type DiagnosticParserCoreTerminalPayloadView ¶ added in v0.46.0

type DiagnosticParserCoreTerminalPayloadView struct {
	ID                uint32
	Symbol            Symbol
	ProductionID      uint16
	DynamicPrecedence int16
	StartByte         uint32
	EndByte           uint32
	Children          []uint32
	Fields            []FieldMapEntry
	Aliases           []Symbol
	Extra             bool
	External          bool
	Terminal          bool
}

type DiagnosticParserCoreThreeProofCensusTotals ¶ added in v0.49.0

type DiagnosticParserCoreThreeProofCensusTotals struct {
	// Elections is the number of converged-split no-action drop elections
	// observed (one call to dropGenericNoActionHeads with a non-empty
	// indices batch).
	Elections uint64
	// ScalarProved / V1Proved / V2Proved count how many elections each
	// proof, evaluated independently, proved.
	ScalarProved uint64
	V1Proved     uint64
	V2Proved     uint64
	// Class1V2AdmitsScalarDeclines: v2 proves where the retired scalar proof
	// declines -- the class the stage 1 census structurally could not see,
	// and the Kotlin class detector (spec section 7 class 1). The stage 2b
	// gate required this class to carry zero differing-tree cases
	// (adjudicated against the C oracle, not raw production equality;
	// campaign amendment 2026-08-02) across the corpora in scope; it stays a
	// live regression watch now that v2 is the decider.
	Class1V2AdmitsScalarDeclines uint64
	// Class2ScalarProvesV2Declines: the retired scalar proof proves where v2
	// (the live decider) declines -- expected non-zero (rank-preference
	// drops, the coincidental-premise class v1's own design indicted; spec
	// section 7 class 2). No gate; these are the newly-declined drops v2
	// falls back on.
	Class2ScalarProvesV2Declines uint64
	// Class3V1ProvesV2Declines: v1 (event-only) proves where v2 declines --
	// the branch-discrimination accepted capability cost (spec section 7
	// class 3). No gate; the Kotlin witness is the canonical member.
	Class3V1ProvesV2Declines uint64
	// BlendedVetoFirings: elections where v2 found an exact-member-
	// containing candidate witness but rejected it solely because that
	// witness was blended (spec section 7 class 4 rate; open question 2).
	BlendedVetoFirings uint64
	// OverflowDeclines: elections where v1 or v2 declined because the
	// dropped header's own recorded set had overflowed the hard cap (spec
	// section 7 class 4 rate).
	OverflowDeclines uint64
	// SpillElections / SpillObserved: elections examined, and how many of
	// them touched at least one spilled (beyond-inline) recorded set --
	// re-checks the 1-percent spill-rate gate at uint32 member width (v1
	// open question 5; spec section 7 class 4 rate).
	SpillElections uint64
	SpillObserved  uint64
	// MaxBranchOrdinalObserved: the largest branch ordinal recorded on any
	// member examined by the census (spec section 7 class 4 rate).
	MaxBranchOrdinalObserved uint16
}

DiagnosticParserCoreThreeProofCensusTotals tallies scalar, v1, and v2 verdicts together across every converged-split no-action drop election observed while the census is enabled (spec.b4b-alternative-set.v2 section 7). Every election increments exactly one of ScalarProved/false and one of V1Proved/false and one of V2Proved/false, plus the derived classes below.

func DiagnosticParserCoreThreeProofCensusSnapshotForTest ¶ added in v0.49.0

func DiagnosticParserCoreThreeProofCensusSnapshotForTest() DiagnosticParserCoreThreeProofCensusTotals

DiagnosticParserCoreThreeProofCensusSnapshotForTest returns the accumulated three-proof (scalar/v1/v2) census totals since the last reset.

type DiagnosticParserCoreVersionLexerRequest ¶ added in v0.52.0

type DiagnosticParserCoreVersionLexerRequest struct {
	ElectionIndex     int
	HeaderCreationSeq uint64
	State             StateID
	Token             Token
	InternalDFAToken  bool
	ScannerBefore     DiagnosticParserCoreScannerCheckpoint
	ScannerAfter      DiagnosticParserCoreScannerCheckpoint
}

DiagnosticParserCoreVersionLexerRequest records one full owned lexer request. The pair is the scanner state before and after the token, not a cursor-only approximation. Full receipts retain these records so a caller can audit straddled token spans and their authenticated checkpoints.

type ErrorTreeIncrementalReuseExternalScanner ¶ added in v0.47.0

type ErrorTreeIncrementalReuseExternalScanner interface {
	ExternalScanner
	SupportsIncrementalReuseFromErrorTree() bool
}

ErrorTreeIncrementalReuseExternalScanner is an optional refinement for a scanner whose checkpoints are certified on clean old trees but whose parser recovery ownership has not yet been certified. Returning false makes a changed edit over an error-bearing old tree take the fresh-parse fallback. Independently reauthenticated token-invariant leaf edits may still return before this gate.

type ExternalLexer ¶

type ExternalLexer struct {
	// contains filtered or unexported fields
}

ExternalLexer is the scanner-facing lexer API used by external scanners. It mirrors the essential tree-sitter scanner API: lookahead, advance, mark_end, and result_symbol.

func (*ExternalLexer) Advance ¶

func (l *ExternalLexer) Advance(skip bool)

Advance consumes one rune. When skip is true, consumed bytes are excluded from the token span (scanner whitespace skipping behavior).

func (*ExternalLexer) AdvanceSpaces ¶ added in v0.20.6

func (l *ExternalLexer) AdvanceSpaces(skip bool) int

AdvanceSpaces consumes consecutive ASCII spaces and returns the number of bytes consumed. It is equivalent to repeated Advance(skip) while Lookahead is a space, but avoids per-byte rune decoding in external scanners that skip indentation runs.

func (*ExternalLexer) AdvanceUntilNewline ¶ added in v0.20.6

func (l *ExternalLexer) AdvanceUntilNewline(skip bool) int

AdvanceUntilNewline consumes bytes up to, but not including, '\n' or EOF and returns the number of bytes consumed. For non-newline bytes, Advance updates Column by the UTF-8 width, which is equal to the byte count for the whole consumed span.

func (*ExternalLexer) Column ¶ added in v0.6.0

func (l *ExternalLexer) Column() uint32

Column returns the number of code points since the start of the current line at the scanner cursor (0-based), matching C tree-sitter's ts_lexer__get_column. This is a code-point count, not a byte offset: each multi-byte UTF-8 rune before the cursor on this line counts once. A leading byte order mark at the very start of the source does not count. Token StartPoint/EndPoint columns remain byte offsets; use those for byte positions and Column only for code-point-based scanner logic (for example, fixed-column layouts).

func (*ExternalLexer) GetColumn deprecated

func (l *ExternalLexer) GetColumn() uint32

GetColumn returns the current column (0-based) at the scanner cursor.

Deprecated: use Column.

func (*ExternalLexer) HasPreviousBytes ¶ added in v0.21.0

func (l *ExternalLexer) HasPreviousBytes(text string) bool

HasPreviousBytes reports whether the bytes immediately before the scanner cursor match text. External scanners use this to guard context-sensitive content tokens when merged parser states expose them too broadly.

func (*ExternalLexer) Lookahead ¶

func (l *ExternalLexer) Lookahead() rune

Lookahead returns the current rune or 0 at EOF.

func (*ExternalLexer) MarkEnd ¶

func (l *ExternalLexer) MarkEnd()

MarkEnd marks the current scanner position as the token end.

func (*ExternalLexer) Previous ¶ added in v0.48.0

func (l *ExternalLexer) Previous() rune

Previous returns the rune immediately before the current lexer position. It returns 0 only at the start of the source (pos <= 0) or when pos is out of range. For invalid UTF-8 immediately before pos, it returns utf8.RuneError (U+FFFD), matching utf8.DecodeLastRune -- never 0.

Warning: a scanner that reads Previous() looks behind the current scan position, which is a byte, not a grammar token. A caller cannot assume the preceding byte belongs to the token that logically precedes this position: extras (comments, whitespace) the core lexer matches between two scanner invocations are invisible to the scanner, so the byte immediately before pos can be the tail of a comment rather than the true previous token. A scanner that relies on Previous() to decide something as history-sensitive as this must track token-boundary state itself across calls (see grammars/swift_scanner.go) rather than trust a single raw byte read. This lookbehind also breaks external-scanner quiescence obligation 2 (see external_scanner_quiescence.go): Scan must depend only on bytes at or after the current position plus the valid-symbol set, not on bytes before it. A scanner that calls Previous() must not claim StatelessExternalScanner.

func (*ExternalLexer) SetResultSymbol ¶

func (l *ExternalLexer) SetResultSymbol(sym Symbol)

SetResultSymbol sets the token symbol to emit when Scan returns true.

type ExternalScanner ¶

type ExternalScanner interface {
	Create() any
	Destroy(payload any)
	Serialize(payload any, buf []byte) int
	Deserialize(payload any, buf []byte)
	Scan(payload any, lexer *ExternalLexer, validSymbols []bool) bool
}

ExternalScanner is the interface for language-specific external scanners. Languages like Python and JavaScript need these for indent tracking, template literals, regex vs division, etc.

The value returned by Create must be accepted by Destroy/Serialize/ Deserialize/Scan for that scanner implementation. Most scanners use a concrete payload pointer type and will panic on mismatched payload types.

func AdaptExternalScannerByExternalOrder ¶ added in v0.9.0

func AdaptExternalScannerByExternalOrder(sourceLang, targetLang *Language) (ExternalScanner, bool)

AdaptExternalScannerByExternalOrder builds an ExternalScanner adapter that reuses sourceLang's scanner for targetLang by remapping external symbols.

Mapping strategy:

  1. If either side has duplicate external names, use index mapping (capped to the shorter list length).
  2. Otherwise, prefer exact external-symbol-name matches.
  3. Fill remaining slots by index order (within the shorter dimension).

When source and target have different external symbol counts, name-based matching pairs tokens that exist in both grammars. Target externals with no source match get -1 (the scanner will never produce them). Source externals with no target match are silently ignored.

Returns (nil, false) when adaptation is not possible.

type ExternalScannerCheckpointIdentity ¶ added in v0.52.0

type ExternalScannerCheckpointIdentity struct {
	Scanner []byte
	Grammar []byte
}

ExternalScannerCheckpointIdentity identifies the scanner and grammar that produced an external-scanner checkpoint. Both identifiers must be stable, non-empty, and at most 256 bytes for the scanner capability to be accepted.

type ExternalScannerCheckpointIdentityProvider ¶ added in v0.52.0

type ExternalScannerCheckpointIdentityProvider interface {
	CheckpointedExternalScanner
	CheckpointIdentity() (ExternalScannerCheckpointIdentity, bool)
}

ExternalScannerCheckpointIdentityProvider is an opt-in extension for a checkpointed scanner. CheckpointIdentity must return stable identifiers for the scanner implementation and the exact grammar blob.

The production parser consumes this capability at checkpoint-aware incremental reuse and per-version lexer ownership boundaries. It does not alter GLR scheduling or recovery election.

type ExternalScannerFullParseRetryPolicy ¶ added in v0.24.1

type ExternalScannerFullParseRetryPolicy uint8

ExternalScannerFullParseRetryPolicy controls whether a full parse with an external scanner may schedule the generic second retry ladder after the normal retry ladder has already selected its best tree.

Keep new values append-only: Language blobs encode these numeric values.

const (
	// ExternalScannerFullParseRetryDefault preserves the generic behavior: an
	// accepted error-bearing tree may schedule one more full retry ladder.
	ExternalScannerFullParseRetryDefault ExternalScannerFullParseRetryPolicy = iota

	// ExternalScannerFullParseRetrySkipRepeat certifies that the tree selected
	// by the first retry ladder is authoritative. The parser retains that exact
	// selected tree and does not schedule the extra external-scanner retry.
	ExternalScannerFullParseRetrySkipRepeat
)

type ExternalScannerState ¶

type ExternalScannerState struct {
	Data []byte
}

ExternalScannerState holds serialized state for an external scanner between incremental parse runs.

type ExternalSymbolResolver ¶ added in v0.9.0

type ExternalSymbolResolver struct {
	// contains filtered or unexported fields
}

ExternalSymbolResolver maps external token names to their concrete Symbol IDs in a specific Language. This allows external scanners to resolve symbol IDs at runtime rather than using hardcoded constants, making them compatible with any Language that defines the same external tokens (whether from ts2go extraction or grammargen).

func NewExternalSymbolResolver ¶ added in v0.9.0

func NewExternalSymbolResolver(lang *Language) *ExternalSymbolResolver

NewExternalSymbolResolver builds a resolver from a Language's external symbol definitions. Returns nil if the Language has no external symbols.

func (*ExternalSymbolResolver) ByIndex ¶ added in v0.9.0

func (r *ExternalSymbolResolver) ByIndex(idx int) (Symbol, bool)

ByIndex returns the Symbol ID for the given external token index (position in the grammar's externals array). Returns 0, false if the index is out of range.

func (*ExternalSymbolResolver) ByName ¶ added in v0.9.0

func (r *ExternalSymbolResolver) ByName(name string) (Symbol, bool)

ByName returns the Symbol ID for the given external token name. Returns 0, false if the name is not found.

func (*ExternalSymbolResolver) Count ¶ added in v0.9.0

func (r *ExternalSymbolResolver) Count() int

Count returns the number of external tokens.

type ExternalVMInstr ¶

type ExternalVMInstr struct {
	Op  ExternalVMOp
	A   int32
	B   int32
	Alt int32
}

ExternalVMInstr is one instruction in an external scanner VM program.

Operands:

  • A: primary operand (opcode-specific)
  • B: secondary operand (used by range checks)
  • Alt: alternate program counter when a condition fails

func VMAdvance ¶

func VMAdvance(skip bool) ExternalVMInstr

VMAdvance constructs an advance instruction. When skip is true, the advanced rune is skipped from the token text.

func VMEmit ¶

func VMEmit(sym Symbol) ExternalVMInstr

VMEmit constructs an emit instruction for the given symbol.

func VMFail ¶

func VMFail() ExternalVMInstr

VMFail constructs a fail instruction that terminates scan with no token.

func VMIfRuneClass ¶

func VMIfRuneClass(class ExternalVMRuneClass, alt int) ExternalVMInstr

VMIfRuneClass constructs a rune-class branch with alternate target on miss.

func VMIfRuneEq ¶

func VMIfRuneEq(r rune, alt int) ExternalVMInstr

VMIfRuneEq constructs a rune-equality branch with alternate target on miss.

func VMIfRuneInRange ¶

func VMIfRuneInRange(start, end rune, alt int) ExternalVMInstr

VMIfRuneInRange constructs a rune-range branch with alternate target on miss.

func VMJump ¶

func VMJump(target int) ExternalVMInstr

VMJump constructs an unconditional branch to the target instruction index.

func VMMarkEnd ¶

func VMMarkEnd() ExternalVMInstr

VMMarkEnd constructs a mark-end instruction for the current token extent.

func VMRequireStateEq ¶

func VMRequireStateEq(state uint32, alt int) ExternalVMInstr

VMRequireStateEq constructs a payload-state guard with alternate branch on miss.

func VMRequireValid ¶

func VMRequireValid(validSymbolIndex, alt int) ExternalVMInstr

VMRequireValid constructs a valid-symbol guard with alternate branch on miss.

func VMSetState ¶

func VMSetState(state uint32) ExternalVMInstr

VMSetState constructs a payload-state assignment instruction.

type ExternalVMOp ¶

type ExternalVMOp uint8

ExternalVMOp is an opcode for the native-Go external scanner VM.

const (
	ExternalVMOpFail ExternalVMOp = iota
	ExternalVMOpJump
	ExternalVMOpRequireValid
	ExternalVMOpRequireStateEq
	ExternalVMOpSetState
	ExternalVMOpIfRuneEq
	ExternalVMOpIfRuneInRange
	ExternalVMOpIfRuneClass
	ExternalVMOpAdvance
	ExternalVMOpMarkEnd
	ExternalVMOpEmit
)

type ExternalVMProgram ¶

type ExternalVMProgram struct {
	Code     []ExternalVMInstr
	MaxSteps int // <=0 uses a safe default based on program size
}

ExternalVMProgram is a small bytecode program interpreted by ExternalVMScanner.

type ExternalVMRuneClass ¶

type ExternalVMRuneClass uint8

ExternalVMRuneClass is a character class used by ExternalVMOpIfRuneClass.

const (
	ExternalVMRuneClassWhitespace ExternalVMRuneClass = iota
	ExternalVMRuneClassDigit
	ExternalVMRuneClassLetter
	ExternalVMRuneClassWord
	ExternalVMRuneClassNewline
)

type ExternalVMScanner ¶

type ExternalVMScanner struct {
	// contains filtered or unexported fields
}

ExternalVMScanner executes an ExternalVMProgram and implements ExternalScanner.

func MustNewExternalVMScanner ¶

func MustNewExternalVMScanner(program ExternalVMProgram) *ExternalVMScanner

MustNewExternalVMScanner is like NewExternalVMScanner but panics on error. It is intended for package-level initialization where invalid programs are programmer errors.

func NewExternalVMScanner ¶

func NewExternalVMScanner(program ExternalVMProgram) (*ExternalVMScanner, error)

NewExternalVMScanner validates and constructs an ExternalVMScanner.

func (*ExternalVMScanner) Create ¶

func (s *ExternalVMScanner) Create() any

Create allocates scanner payload (currently a single uint32 state slot).

func (*ExternalVMScanner) Deserialize ¶

func (s *ExternalVMScanner) Deserialize(payload any, buf []byte)

Deserialize restores payload state from buf.

func (*ExternalVMScanner) Destroy ¶

func (s *ExternalVMScanner) Destroy(payload any)

Destroy releases scanner payload resources.

func (*ExternalVMScanner) Scan ¶

func (s *ExternalVMScanner) Scan(payload any, lexer *ExternalLexer, validSymbols []bool) bool

Scan executes the scanner program against the current lexer position.

func (*ExternalVMScanner) Serialize ¶

func (s *ExternalVMScanner) Serialize(payload any, buf []byte) int

Serialize writes payload state into buf.

type FactKind ¶ added in v0.49.0

type FactKind uint8

FactKind selects the outputs that a FactProgram emits.

const (
	// FactDefinitions selects declaration spans.
	FactDefinitions FactKind = 1 << iota
	// FactCalls selects call-site references.
	FactCalls
	// FactHeritage selects inheritance and base-class references.
	FactHeritage
	// FactImports selects package and dependency declarations.
	FactImports

	// FactAll selects every supported fact kind.
	FactAll = FactDefinitions | FactCalls | FactHeritage | FactImports
)

type FactProgram ¶ added in v0.49.0

type FactProgram struct {
	// contains filtered or unexported fields
}

FactProgram is a compiled, reusable syntax-fact extractor.

Each grammar symbol indexes one packed instruction. Extraction executes the selected operations during one tree traversal. A program only accepts trees built with the Language value supplied to NewFactProgram.

func NewFactProgram ¶ added in v0.49.0

func NewFactProgram(lang *Language, kinds FactKind) (*FactProgram, error)

NewFactProgram compiles a reusable extractor for lang and the selected kinds. Compilation resolves grammar symbols and field names once.

func (*FactProgram) Extract ¶ added in v0.49.0

func (p *FactProgram) Extract(tree *Tree) FactSet

Extract emits the selected facts during one tree traversal. It returns an empty set for a nil tree or a tree built with a different Language value.

func (*FactProgram) ExtractBound ¶ added in v0.52.0

func (p *FactProgram) ExtractBound(tree *BoundTree) FactSet

ExtractBound emits selected facts from a BoundTree. It uses the same guards and traversal as Extract.

func (*FactProgram) ExtractInto ¶ added in v0.54.0

func (p *FactProgram) ExtractInto(tree *Tree, dst *FactSet)

ExtractInto replaces dst with the selected facts and reuses its slice storage. It clears previous entries, including facts from kinds that this program excludes. A nil program, invalid tree, or language mismatch leaves dst empty with its capacity retained. A nil dst has no effect.

Results share storage with dst. Clone the result slices before the next extraction to retain them. Use a separate destination for each concurrent extraction. Assign FactSet{} to *dst when its retained storage is no longer needed.

func (*FactProgram) Kinds ¶ added in v0.49.0

func (p *FactProgram) Kinds() FactKind

Kinds returns the outputs selected when the program was compiled.

type FactSet ¶ added in v0.49.0

type FactSet struct {
	Definitions []DefinitionSpan
	Calls       []CallRef
	Heritage    []HeritageRef
	Imports     []ImportRef
}

FactSet contains the language-neutral facts emitted by a FactProgram.

type FailurePreservingExternalScanner ¶ added in v0.20.6

type FailurePreservingExternalScanner interface {
	ExternalScanner
	PreservesStateOnScanFailure() bool
}

FailurePreservingExternalScanner is implemented by external scanners whose Scan method does not mutate serialized scanner payload state before returning false. The token source can defer snapshotting until retry is actually needed.

type FailureStateRetainingExternalScanner ¶ added in v0.52.0

type FailureStateRetainingExternalScanner interface {
	ExternalScanner
	RetainsStateOnScanFailure() bool
}

FailureStateRetainingExternalScanner is implemented by scanners that can change serialized state before Scan returns false. The changed state is part of the scanner contract and must remain live for the next scan.

Scanners without this capability keep the default transactional contract. The token source restores their start state after a failed scan. Retention takes precedence if a scanner reports both failure capabilities.

type FieldID ¶

type FieldID uint16

FieldID is a named field index.

type FieldMapEntry ¶

type FieldMapEntry struct {
	FieldID    FieldID
	ChildIndex uint8
	Inherited  bool
}

FieldMapEntry maps a child index to a field name.

type ForestCapTieReceipt ¶ added in v0.49.0

type ForestCapTieReceipt struct {
	Symbol             Symbol
	CandidateStart     uint32
	CandidateEnd       uint32
	CandidatePrevState StateID
	CandidatePrevByte  uint32
	IncumbentStart     uint32
	IncumbentEnd       uint32
	IncumbentPrevState StateID
	IncumbentPrevByte  uint32
	SameSpan           bool
	CandidateKept      bool
}

ForestCapTieReceipt is one hidden-symbol cap-tie decision: (symbol, span, prev.state, prev.byteOffset) for the arriving candidate and the incumbent it was compared against, matching the spec Section 5 instrumentation shape.

type ForestCapTieStats ¶ added in v0.49.0

type ForestCapTieStats struct {
	HiddenTieDecisions int
	CandidatesPinned   int
	SameSpanTies       int
	Receipts           []ForestCapTieReceipt
}

ForestCapTieStats is Stage 0's cap-event instrument: how many hidden- symbol ties were decided at the forest link cap, how many span-maximal pinning kept over the arrival-order default, how many were themselves an exact-span tie (the case the span-maximal proxy cannot break -- see the spec's Open Questions), and, only when GOT_FOREST_CAP_TIE_DUMP=1, a bounded per-decision receipt list. Valid after any Parse or ParseForestExperimental call.

type FullParseAcceptedErrorRetryProfile ¶ added in v0.25.0

type FullParseAcceptedErrorRetryProfile struct {
	MinSourceBytes                 uint32
	InitialStackCeiling            uint16
	SkipCompleteAcceptedErrorRetry bool
	FreshErrorNoStacksMaxPasses    uint8
	// SkipCompleteMaxEntryScratchPeak limits complete-tree and fresh-result skips
	// to a certified peak number of live GLR scratch entries. Zero is unbounded.
	SkipCompleteMaxEntryScratchPeak uint32
	// FreshErrorNoStacksRetryMaxStacks replaces the generic widened-stack
	// target for a fresh error-bearing no-stacks parse. Zero keeps the generic
	// target. Incremental fallbacks and explicit environment overrides ignore it.
	FreshErrorNoStacksRetryMaxStacks uint16
	// SkipInitialCompleteAcceptedErrorMergeRetry skips only the first
	// same-stack merge retry for a fresh, complete accepted-error parse. Later
	// widened-stack and merge retries remain available. Incremental fallbacks
	// and explicit stack/merge environment overrides ignore it.
	SkipInitialCompleteAcceptedErrorMergeRetry bool
	// SkipCompleteMinSourceBytes limits complete-tree and fresh-result skips to
	// sources at least this large. Zero preserves the unbounded behavior.
	SkipCompleteMinSourceBytes uint32
	// ReuseCleanWideForWideRetry certifies that the complete accepted-error tree
	// from the non-recovery widened-stack pass is identical to the following
	// recovery-enabled widened-stack pass. The parser may retain that tree and
	// substitute it at the recovery-wide slot while preserving retry accounting.
	ReuseCleanWideForWideRetry bool
	// ReuseCleanWideMinSourceBytes limits clean-wide reuse to the certified
	// large-source class. Zero disables the policy even when the boolean is set.
	ReuseCleanWideMinSourceBytes uint32
	// GSSConvergenceAcceptedErrorMergePerKey sets the exact merge width for an
	// accepted-error retry after a certified cap-one full parse. Zero keeps the
	// other retry policies. Explicit environment settings disable this policy.
	GSSConvergenceAcceptedErrorMergePerKey uint16
	// SkipFreshCompleteAcceptedErrorRetry keeps the initial fresh full-parse
	// result when it is a complete accepted-error tree. It does not suppress a
	// later merge retry after a no-stacks or node-limit result.
	SkipFreshCompleteAcceptedErrorRetry bool
}

FullParseAcceptedErrorRetryProfile certifies narrow full-parse retry policies. When a fresh full parse accepts an error-bearing tree that covers EOF, the parser may keep the initial GLR stack ceiling after the ordinary same-stack merge retry or skip that ladder entirely. A grammar may also cap retries or select a narrower widening target for a fresh, error-bearing no-stacks result after proving the generic ladder does not improve the selected tree. The zero value preserves the conservative generic ladder.

Keep fields append-only: Language blobs encode this structure.

type HeritageRef ¶ added in v0.20.6

type HeritageRef struct {
	Lang            string
	Kind            string
	Name            string
	Parent          string
	NodeType        string
	StartByte       uint32
	EndByte         uint32
	ParentStartByte uint32
	ParentEndByte   uint32
}

HeritageRef is a compact language-neutral inheritance/base-class reference.

func ExtractHeritage ¶ added in v0.20.6

func ExtractHeritage(tree *Tree) []HeritageRef

ExtractHeritage returns language-neutral inheritance/base-class references for common code-understanding workflows.

type HighlightRange ¶

type HighlightRange struct {
	StartByte    uint32
	EndByte      uint32
	Capture      string // "keyword", "string", "function", etc.
	PatternIndex int    // query pattern index; later patterns override earlier for identical ranges
}

HighlightRange represents a styled range of source code, mapping a byte span to a capture name from a highlight query. The editor maps capture names (e.g., "keyword", "string", "function") to FSS style classes.

type Highlighter ¶

type Highlighter struct {
	// contains filtered or unexported fields
}

Highlighter is a high-level API that takes source code and returns styled ranges. It combines a Parser, a compiled Query, and a Language to provide a single Highlight() call for the editor.

func NewHighlighter ¶

func NewHighlighter(lang *Language, highlightQuery string, opts ...HighlighterOption) (*Highlighter, error)

NewHighlighter creates a Highlighter for the given language and highlight query (in tree-sitter .scm format). Returns an error if the query fails to compile.

func (*Highlighter) Highlight ¶

func (h *Highlighter) Highlight(source []byte) []HighlightRange

Highlight parses the source code and executes the highlight query, returning a slice of HighlightRange sorted by StartByte. When ranges overlap, inner (more specific) captures take priority over outer ones.

func (*Highlighter) HighlightIncremental ¶

func (h *Highlighter) HighlightIncremental(source []byte, oldTree *Tree) ([]HighlightRange, *Tree)

HighlightIncremental re-highlights source after edits were applied to oldTree. Returns the new highlight ranges and the new parse tree (for use in subsequent incremental calls). Call oldTree.Edit() before calling this: see ParseIncremental's doc comment for what happens to a length-changing source when that call is skipped.

func (*Highlighter) HighlightIncrementalStrict ¶ added in v0.37.0

func (h *Highlighter) HighlightIncrementalStrict(source []byte, oldTree *Tree) ([]HighlightRange, *Tree, error)

HighlightIncrementalStrict is like HighlightIncremental, but reports ErrParseStoppedEarly and skips query execution when parsing is stopped by a timeout, cancellation, token-source EOF, or parser safety limit. The partial tree is returned so callers can release it or use it for diagnostics.

func (*Highlighter) HighlightIncrementalUTF16 ¶ added in v0.16.0

func (h *Highlighter) HighlightIncrementalUTF16(source []uint16, oldTree *Tree) ([]UTF16HighlightRange, *Tree)

HighlightIncrementalUTF16 re-highlights UTF-16 source after edits were applied to oldTree with Tree.EditUTF16.

func (*Highlighter) HighlightIncrementalUTF16Bytes ¶ added in v0.16.0

func (h *Highlighter) HighlightIncrementalUTF16Bytes(source []byte, oldTree *Tree, order UTF16ByteOrder) ([]UTF16HighlightRange, *Tree, error)

HighlightIncrementalUTF16Bytes is like HighlightIncrementalUTF16 for endian-specific UTF-16 bytes.

func (*Highlighter) HighlightTreeUTF16 ¶ added in v0.35.0

func (h *Highlighter) HighlightTreeUTF16(tree *Tree) []UTF16HighlightRange

HighlightTreeUTF16 executes the highlighter query against an already parsed UTF-16 tree. It lets editor runtimes share one persistent parse tree across syntax highlighting, tags, and ad-hoc queries.

func (*Highlighter) HighlightUTF16 ¶ added in v0.16.0

func (h *Highlighter) HighlightUTF16(source []uint16) []UTF16HighlightRange

HighlightUTF16 parses UTF-16 source and returns highlight ranges in UTF-16 code-unit coordinates.

func (*Highlighter) HighlightUTF16Bytes ¶ added in v0.16.0

func (h *Highlighter) HighlightUTF16Bytes(source []byte, order UTF16ByteOrder) ([]UTF16HighlightRange, error)

HighlightUTF16Bytes is like HighlightUTF16 for endian-specific UTF-16 bytes.

type HighlighterInjectionResolver ¶ added in v0.7.0

type HighlighterInjectionResolver func(languageHint string) (lang *Language, highlightQuery string, tokenSourceFactory func(source []byte) TokenSource, ok bool)

HighlighterInjectionResolver maps a language hint (for example "go" from a markdown code fence) to a child language and highlight query.

type HighlighterInjectionSpec ¶ added in v0.7.0

type HighlighterInjectionSpec struct {
	Query           string
	ResolveLanguage HighlighterInjectionResolver
}

HighlighterInjectionSpec configures nested highlighting for a parent language. Query must emit @injection.content and either @injection.language or #set! injection.language metadata.

type HighlighterOption ¶

type HighlighterOption func(*Highlighter)

HighlighterOption configures a Highlighter.

func WithHighlighterAdmissionCandidateRoute ¶ added in v0.55.0

func WithHighlighterAdmissionCandidateRoute(enabled bool) HighlighterOption

WithHighlighterAdmissionCandidateRoute pins the route for the document parser and injected-language parsers. The compact route still obeys eligibility checks.

func WithHighlighterTimeoutMicros ¶ added in v0.37.0

func WithHighlighterTimeoutMicros(timeoutMicros uint64) HighlighterOption

WithHighlighterTimeoutMicros bounds every full and incremental parse performed by the highlighter. A value of zero disables timeout checks.

func WithTokenSourceFactory ¶

func WithTokenSourceFactory(factory func(source []byte) TokenSource) HighlighterOption

WithTokenSourceFactory sets a factory function that creates a TokenSource for each Highlight call. This is needed for languages that use a custom lexer bridge (like Go, which uses go/scanner instead of a DFA lexer).

When set, Highlight() calls ParseWithTokenSource instead of Parse.

type ImportExtractResult ¶ added in v0.18.0

type ImportExtractResult struct {
	Imports             []ImportRef
	Status              ImportExtractStatus
	Reason              string
	FallbackRecommended bool
}

ImportExtractResult is returned by source-only dependency extraction. When FallbackRecommended is true, callers that need exact tree-sitter behavior should parse the file and use ExtractImports.

func ExtractImportsFromSourceWithReport ¶ added in v0.18.0

func ExtractImportsFromSourceWithReport(lang *Language, source []byte) ImportExtractResult

ExtractImportsFromSourceWithReport returns source-only dependency declarations and a confidence report for fallback policy.

type ImportExtractStatus ¶ added in v0.18.0

type ImportExtractStatus string

ImportExtractStatus describes the confidence of source-only import extraction.

const (
	ImportExtractOK                   ImportExtractStatus = "ok"
	ImportExtractUnsupportedConstruct ImportExtractStatus = "unsupported_construct"
	ImportExtractScannerError         ImportExtractStatus = "scanner_error"
	ImportExtractAmbiguous            ImportExtractStatus = "ambiguous"
	ImportExtractFallbackToTree       ImportExtractStatus = "fallback_to_tree"
)

type ImportRef ¶ added in v0.18.0

type ImportRef struct {
	Lang      string
	Kind      string
	Path      string
	From      string
	Name      string
	Alias     string
	Static    bool
	Wildcard  bool
	Relative  int
	StartByte uint32
	EndByte   uint32
}

ImportRef is a compact language-neutral dependency declaration extracted from a syntax tree.

func ExtractImports ¶ added in v0.18.0

func ExtractImports(tree *Tree) []ImportRef

ExtractImports returns package/import declarations for the languages used by Gazelle-style dependency extraction. It is intentionally independent from the generic query engine so it can later be backed by compact parser refs.

func ExtractImportsFromSource ¶ added in v0.18.0

func ExtractImportsFromSource(lang *Language, source []byte) []ImportRef

ExtractImportsFromSource returns language-neutral dependency declarations directly from source text. It is intended for cold dependency-extraction workflows that do not need a public syntax tree.

type IncrementalParseProfile ¶ added in v0.6.0

type IncrementalParseProfile struct {
	ReuseCursorNanos int64
	ReparseNanos     int64
	ReusedSubtrees   uint64
	ReusedBytes      uint64
	// TokenInvariantDependencyChecks counts bounded lexical comparisons.
	// A comparison can reject reuse when an earlier token changes.
	TokenInvariantDependencyChecks     uint64
	NewNodesAllocated                  uint64
	ReuseUnsupported                   bool
	ReuseUnsupportedReason             string
	AcceptedErrorRetryAttempts         uint8
	AcceptedErrorRetryAdopted          bool
	AcceptedErrorRetryMergePerKey      int
	AcceptedErrorRetryCause            IncrementalRetryCause
	OldTreeReuseRoute                  bool
	ReuseRejectDirty                   uint64
	ReuseRejectAncestorDirtyBeforeEdit uint64
	ReuseRejectHasError                uint64
	ReuseRejectInvalidSpan             uint64
	ReuseRejectOutOfBounds             uint64
	ReuseRejectRootNonLeafChanged      uint64
	// ReuseObservedPreGotoStateMismatch counts top-level block-splice candidates
	// observed at a live parser state different from the node's recorded
	// PreGotoState. This is diagnostic only: the established admission contract
	// remains goto-target compatibility plus fragility until #432 replaces it
	// with a complete ownership proof.
	ReuseObservedPreGotoStateMismatch uint64
	ReuseRejectLargeNonLeaf           uint64
	ReuseRejectStaleNonLeafBoundary   uint64
	// ReuseRejectFragileNonLeaf counts interior (non-leaf) reuse candidates
	// rejected because Node.isFragile() reported the candidate was built
	// under an ambiguous parse decision (LR-table conflict, GSS multi-pop, or
	// concurrent GLR stack versions) or is itself an ERROR/MISSING node -- see
	// markReduceFragility (parser_reduce.go) and reuseNonLeafTargetStateOnStack
	// (incremental.go). A nonzero count on a conflict-heavy grammar (e.g. js)
	// is expected and correct: it is exactly the unsound reuse this gate is
	// designed to prevent.
	ReuseRejectFragileNonLeaf uint64
	// BlockSpliceSteps is the number of top-level sibling reuses taken inside
	// the W1 block-splice composition loop (spec.campaign.oedit): one per
	// sibling spliced without a full main-loop round trip. It is O(edit) and
	// deterministic for a fixed (source, edit, language).
	BlockSpliceSteps uint64
	// ReuseRejectScannerUnquiescent counts reuse candidates the external
	// scanner checkpoint/quiescence gate rejected -- a checkpoint state
	// mismatch on a checkpoint language, or a refuted quiescence proof on an
	// opt-out scanner (campaign O(edit) workstream W4, spec.campaign.oedit).
	// It is 0 for stateless-scanner languages such as Go, whose ASI scanner
	// carries no cross-token state and is proven quiescent at every boundary
	// (external_scanner_quiescence.go). A nonzero count marks boundaries where
	// scanner state, not fragility or byte drift, is the binding constraint.
	ReuseRejectScannerUnquiescent uint64
	// ReuseRejectFrontierProofUnavailable counts compact-materialized non-leaf
	// candidates rejected because exact scanner bytes do not prove the parser
	// frontier that owned the original reduction.
	ReuseRejectFrontierProofUnavailable uint64
	RecoverSearches                     uint64
	RecoverStateChecks                  uint64
	RecoverStateSkips                   uint64
	RecoverSymbolSkips                  uint64
	RecoverLookups                      uint64
	RecoverHits                         uint64
	MaxStacksSeen                       int
	EntryScratchPeak                    uint64
	StopReason                          ParseStopReason
	TokensConsumed                      uint64
	LastTokenEndByte                    uint32
	ExpectedEOFByte                     uint32
	ArenaBytesAllocated                 int64
	// ArenaBaselineBytes sums retained arena capacity before each attempt.
	// Subtract it from ArenaBytesAllocated to measure total operation growth.
	ArenaBaselineBytes    int64
	ScratchBytesAllocated int64
	// ScratchBaselineBytes sums retained scratch capacity before each attempt.
	// Subtract it from ScratchBytesAllocated to measure total operation growth.
	ScratchBaselineBytes                int64
	EntryScratchBytesAllocated          int64
	GSSBytesAllocated                   int64
	SingleStackIterations               int
	MultiStackIterations                int
	SingleStackTokens                   uint64
	MultiStackTokens                    uint64
	SingleStackGSSNodes                 uint64
	MultiStackGSSNodes                  uint64
	GSSNodesAllocated                   uint64
	GSSNodesRetained                    uint64
	GSSNodesDroppedSameToken            uint64
	ParentNodesAllocated                uint64
	ParentNodesRetained                 uint64
	ParentNodesDroppedSameToken         uint64
	LeafNodesAllocated                  uint64
	LeafNodesRetained                   uint64
	LeafNodesDroppedSameToken           uint64
	MergeStacksIn                       uint64
	MergeStacksOut                      uint64
	MergeSlotsUsed                      uint64
	GlobalCullStacksIn                  uint64
	GlobalCullStacksOut                 uint64
	ParserLoopNanos                     int64
	TokenNextNanos                      int64
	ActionDispatchNanos                 int64
	ActionLookupNanos                   int64
	GLRMergeNanos                       int64
	GLRCullNanos                        int64
	ResultSelectionNanos                int64
	TransientParentMaterializationNanos int64
	ResultTreeBuildNanos                int64
	TransientChildMaterializationNanos  int64
	ResultPythonKeywordRepairNanos      int64
	ResultPythonRootRepairNanos         int64
	ResultFinalizeRootNanos             int64
	ResultExtendTrailingNanos           int64
	ResultNormalizeRootStartNanos       int64
	ResultCompatibilityNanos            int64
	ResultParentLinkNanos               int64
	ReduceRangeNanos                    int64
	ReducePendingParentNanos            int64
	ReduceChildBuildNanos               int64
	ReduceParentBuildNanos              int64
	ReduceSpanNanos                     int64
	ReduceStackPushNanos                int64
	ReduceNoTreeBuildNanos              int64
	ActionExtraShiftNanos               int64
	ActionNoActionNanos                 int64
	ActionNoActionRelexNanos            int64
	ActionNoActionMissingNanos          int64
	ActionNoActionRecoverNanos          int64
	ActionNoActionErrorNanos            int64
	ActionConflictChoiceNanos           int64
	ActionConflictForkNanos             int64
	ActionSingleShiftNanos              int64
	ActionSingleReduceNanos             int64
	ActionSingleAcceptNanos             int64
	ActionSingleRecoverNanos            int64
	ActionSingleOtherNanos              int64
	NormalizationNanos                  int64
}

IncrementalParseProfile attributes incremental parse time into coarse buckets.

ReuseCursorNanos includes reuse-cursor setup and subtree-candidate checks. ReparseNanos includes the remainder of incremental parsing/rebuild work. ReusedSubtrees, ReusedBytes, and the result fields describe the selected attempt. The result fields include reuse support, the reuse route, and the parse boundary. Retry fields describe the complete operation. Work counters and timing fields aggregate recorded parse attempts. MaxStacksSeen and EntryScratchPeak are the maximum values across all attempts.

type IncrementalPrefixFrontierExternalScanner ¶ added in v0.52.0

type IncrementalPrefixFrontierExternalScanner interface {
	ExternalScanner
	RequiresIncrementalPrefixFrontierProof() bool
}

IncrementalPrefixFrontierExternalScanner is an optional refinement for a checkpointed scanner whose state can depend on reductions before the next top-level sibling. A changed-length or changed-point edit before that sibling must take the fresh-parse fallback unless the parser can prove the old reduction frontier. Python uses this gate for indentation ownership.

type IncrementalRetryCause ¶ added in v0.39.0

type IncrementalRetryCause uint8

IncrementalRetryCause identifies why an incremental parse ran an additional bounded attempt.

const (
	IncrementalRetryCauseNone IncrementalRetryCause = iota
	IncrementalRetryCauseAcceptedErrorBaseMerge
)

type IncrementalReuseExternalScanner ¶ added in v0.7.0

type IncrementalReuseExternalScanner interface {
	ExternalScanner
	SupportsIncrementalReuse() bool
}

IncrementalReuseExternalScanner is implemented by external scanners that can safely participate in DFA subtree reuse during incremental parses. Scanners with serialized mutable state, such as Python's indentation stack, should leave this unimplemented so edited incremental parses fall back to the conservative full-reparse path.

type IncrementalReuseTokenSource ¶ added in v0.7.0

type IncrementalReuseTokenSource interface {
	TokenSource
	SupportsIncrementalReuse() bool
}

IncrementalReuseTokenSource is an opt-in marker for custom token sources that are safe for incremental subtree reuse. Implementations must provide stable token boundaries across edits and support deterministic SkipToByte* behavior so reused-tree fast-forwarding remains correct.

type Injection ¶ added in v0.6.0

type Injection struct {
	// Language is the detected language name (e.g., "javascript").
	Language string
	// Tree is the parse tree for this region, or nil if the language
	// was not registered.
	Tree *Tree
	// Ranges are the source ranges this tree covers.
	Ranges []Range
	// Node is the parent tree node that triggered the injection.
	Node *Node
}

Injection is a single embedded language region.

type InjectionParser ¶ added in v0.6.0

type InjectionParser struct {
	// contains filtered or unexported fields
}

InjectionParser parses documents with embedded languages.

InjectionParser is not safe for concurrent use. It caches child parsers and mutates shared maps during parse operations.

func NewInjectionParser ¶ added in v0.6.0

func NewInjectionParser() *InjectionParser

NewInjectionParser creates an InjectionParser.

func (*InjectionParser) Parse ¶ added in v0.6.0

func (ip *InjectionParser) Parse(source []byte, parentLang string) (*InjectionResult, error)

Parse parses source as parentLang, then recursively parses injected regions.

func (*InjectionParser) ParseIncremental ¶ added in v0.6.0

func (ip *InjectionParser) ParseIncremental(source []byte, parentLang string,
	oldResult *InjectionResult) (*InjectionResult, error)

ParseIncremental re-parses after edits, reusing unchanged child trees.

It detects and parses injections the same way Parse does — recursing into each injected language's own injection query, and routing single-range injections through the slice-and-rebase path — so a ParseIncremental call produces the same nested injection tree as a fresh Parse of the same (edited) source. The only difference is that a detected injection whose byte range does not overlap any changed range reuses its old child tree instead of reparsing it.

func (*InjectionParser) ParseIncrementalUTF16 ¶ added in v0.16.0

func (ip *InjectionParser) ParseIncrementalUTF16(source []uint16, parentLang string,
	oldResult *UTF16InjectionResult) (*UTF16InjectionResult, error)

ParseIncrementalUTF16 re-parses UTF-16 source after edits, reusing unchanged child trees. Call oldResult.Tree.EditUTF16 before calling this.

func (*InjectionParser) ParseIncrementalUTF16Bytes ¶ added in v0.16.0

func (ip *InjectionParser) ParseIncrementalUTF16Bytes(source []byte, parentLang string,
	oldResult *UTF16InjectionResult, order UTF16ByteOrder) (*UTF16InjectionResult, error)

ParseIncrementalUTF16Bytes is like ParseIncrementalUTF16 for endian-specific UTF-16 bytes.

func (*InjectionParser) ParseUTF16 ¶ added in v0.16.0

func (ip *InjectionParser) ParseUTF16(source []uint16, parentLang string) (*UTF16InjectionResult, error)

ParseUTF16 parses UTF-16 source as parentLang, then recursively parses injected regions. The returned injection ranges are in UTF-16 code units.

func (*InjectionParser) ParseUTF16Bytes ¶ added in v0.16.0

func (ip *InjectionParser) ParseUTF16Bytes(source []byte, parentLang string, order UTF16ByteOrder) (*UTF16InjectionResult, error)

ParseUTF16Bytes is like ParseUTF16 for endian-specific UTF-16 bytes.

func (*InjectionParser) RegisterInjectionQuery ¶ added in v0.6.0

func (ip *InjectionParser) RegisterInjectionQuery(parentLang string, query string) error

RegisterInjectionQuery sets the injection query for a parent language. The query should use @injection.content and #set! injection.language conventions. It is compiled against the registered parent language.

func (*InjectionParser) RegisterLanguage ¶ added in v0.6.0

func (ip *InjectionParser) RegisterLanguage(name string, lang *Language)

RegisterLanguage adds a language that can be used as parent or child.

func (*InjectionParser) SetCancellationFlag ¶ added in v0.54.0

func (ip *InjectionParser) SetCancellationFlag(flag *uint32)

SetCancellationFlag configures a caller-owned cancellation flag that stops every parse this InjectionParser performs — the root parse and every nested injected parse, at every recursion depth — when the pointed value becomes non-zero. It applies immediately to already-cached child parsers and to any parser created afterward.

func (*InjectionParser) SetMaxDepth ¶ added in v0.6.0

func (ip *InjectionParser) SetMaxDepth(depth int)

SetMaxDepth overrides the nested injection recursion limit. Depth values <= 0 restore the default limit.

func (*InjectionParser) SetTimeoutMicros ¶ added in v0.54.0

func (ip *InjectionParser) SetTimeoutMicros(timeoutMicros uint64)

SetTimeoutMicros configures a per-parse timeout in microseconds that bounds every parse this InjectionParser performs: the root parse and every nested injected parse, at every recursion depth. A value of zero disables timeout checks. It applies immediately to already-cached child parsers and to any parser created afterward.

type InjectionResult ¶ added in v0.6.0

type InjectionResult struct {
	// Tree is the parent language's parse tree.
	Tree *Tree
	// Injections contains child language parse results, ordered by position.
	Injections []Injection
}

InjectionResult holds parse results for a multi-language document.

type InputEdit ¶

type InputEdit struct {
	StartByte   uint32
	OldEndByte  uint32
	NewEndByte  uint32
	StartPoint  Point
	OldEndPoint Point
	NewEndPoint Point
}

InputEdit describes a single edit to the source text. It tells the parser what byte range was replaced and what the new range looks like, so the incremental parser can skip unchanged subtrees.

type InputEncoding ¶ added in v0.16.0

type InputEncoding uint8

InputEncoding identifies the source encoding used to build a Tree.

const (
	InputEncodingUTF8 InputEncoding = iota
	InputEncodingUTF16
)

func (InputEncoding) String ¶ added in v0.16.0

func (e InputEncoding) String() string

type InternObservationStats ¶ added in v0.20.0

type InternObservationStats struct {
	// Phase 2 counters (parseState-blind observation across ALL leaves).
	LeafLookups uint64
	LeafHits    uint64
	LeafMisses  uint64
	LeafStores  uint64
	LeafGrowths uint64
	// Phase 3 attribution. Shift-path leaves get parseState set per-fork
	// so they can't be canonically substituted via the parseState-blind
	// measurement; non-shift leaves can. "Safe to substitute" via blind
	// measurement = (LeafMisses+LeafHits) - ShiftLeafObserved.
	ShiftLeafObserved uint64
	// Phase 3 parseState-aware measurement. Same hook as LeafLookups
	// but with parseState/preGotoState included in the key. A hit here
	// means a truly dedup-safe duplicate; the difference between this
	// and the blind hit rate quantifies how much of the blind
	// observation was an artifact of ignoring state.
	FullLookups uint64
	FullHits    uint64
	FullMisses  uint64
}

InternObservationStats is the externally-visible snapshot of leaf-interning observation counters for a single parse. Returned from InternStatsFor.

func InternStatsFor ¶ added in v0.20.0

func InternStatsFor(root *Node) InternObservationStats

InternStatsFor returns a snapshot of the leaf-interning observation counters for the arena that owns the given root node. Returns the zero value if observation is disabled or the root is not arena-backed. Exposed so external benches can read hit rates without grepping internal logs.

type Language ¶

type Language struct {
	Name string
	// GeneratedByGrammargen is true for languages assembled by grammargen at
	// runtime rather than decoded from a checked-in ts2go blob.
	GeneratedByGrammargen bool

	// CRecoveryCostCompetitionCapable records parser.c/table evidence that the
	// grammar exposes the C recovery surface: RECOVER actions plus an
	// ERROR_STATE lex mode. It is capability metadata only, not a default-on
	// parity certification.
	CRecoveryCostCompetitionCapable bool

	// CRecoveryCostCompetitionEnabledByDefault explicitly certifies that the
	// faithful C recovery-cost competition gate is parity-safe as default
	// behavior for this language. Runtime gating also requires capability
	// metadata and conservative table validation.
	CRecoveryCostCompetitionEnabledByDefault bool

	// WantsForest opts this language into the GSS-forest GLR fast path.
	// Consumers generating a Language via grammargen set this (directly or via
	// grammargen.Grammar.WantsForest) to enable forest for their own grammar. This
	// bypasses the byte-range parity certification built-ins undergo — the
	// decline->production fallback still prevents hard failures on declined inputs,
	// but a clean-but-different tree is the consumer's responsibility.
	WantsForest bool

	// LanguageVersion is the tree-sitter language ABI version.
	// A value of 0 means "unknown/unspecified" and is treated as compatible.
	LanguageVersion uint32

	// Counts
	SymbolCount        uint32
	TokenCount         uint32
	ExternalTokenCount uint32
	StateCount         uint32
	LargeStateCount    uint32
	FieldCount         uint32
	ProductionIDCount  uint32

	// Symbol metadata
	SymbolNames    []string
	SymbolMetadata []SymbolMetadata
	FieldNames     []string // index 0 is ""

	// Parse tables
	ParseTable         [][]uint16 // dense: [state][symbol] -> action index
	SmallParseTable    []uint16   // compressed sparse table
	SmallParseTableMap []uint32   // state -> offset into SmallParseTable
	ParseActions       []ParseActionEntry
	// LargeStateGotos stores nonterminal GOTO targets that do not fit in the
	// uint16 parse-table cells used by tree-sitter C tables. Keys are
	// uint64(state)<<32 | uint64(symbol). Terminal actions must never live here.
	//
	// This is the only exported map field on Language, which matters for blob
	// serialization: gob's map codec iterates via reflect's randomized
	// MapRange, so blob encoders never gob-encode this field directly when
	// it's non-empty (today, only c_sharp populates it). See
	// large_state_gotos_trailer.go for the deterministic encode/decode path.
	LargeStateGotos map[uint64]StateID

	// ReduceChainHints are optional generated hot-path hints for deterministic
	// reduce runs. They are only consumed when reduce-chain hints are enabled.
	ReduceChainHints []ReduceChainHint

	// ConflictPolicies are optional deterministic conflict policies derived
	// from grammar tables.
	ConflictPolicies []ConflictPolicy

	// Lex tables
	LexModes            []LexMode
	LexStates           []LexState // main lexer DFA
	KeywordLexStates    []LexState // keyword lexer DFA (optional)
	KeywordCaptureToken Symbol
	// LayoutFallbackLexState is an optional broad DFA start state used only in
	// layout-entry parser states. It lets the runtime avoid skipping over
	// zero-width external layout markers before the layout scanner fires.
	LayoutFallbackLexState    uint16
	HasLayoutFallbackLexState bool

	// Field mapping
	FieldMapSlices  [][2]uint16 // [production_id] -> (index, length)
	FieldMapEntries []FieldMapEntry

	// Alias sequences
	AliasSequences [][]Symbol // [production_id][child_index] -> alias symbol

	// ProductionSignatures records LHS/RHS shape for grammargen productions.
	// It is separate from ProductionID because ProductionID is deliberately
	// compacted by field/alias pattern and is not a unique RHS identity.
	ProductionSignatures []ProductionSignature

	// Primary state IDs (for table dedup)
	PrimaryStateIDs []StateID

	// ABI 15: Reserved words — flat array indexed by
	// (reserved_word_set_id * MaxReservedWordSetSize + i), terminated by 0.
	ReservedWords          []Symbol
	MaxReservedWordSetSize uint16

	// ABI 15: Supertype hierarchy
	SupertypeSymbols    []Symbol
	SupertypeMapSlices  [][2]uint16 // [supertype_symbol] -> (index, length)
	SupertypeMapEntries []Symbol

	// HiddenChoicePassthroughSymbols marks generated hidden nonterminals whose
	// productions are only neutral single-symbol pass-throughs. These wrappers
	// are structural parser routes, not tree nodes; checked-in blobs leave this
	// nil to preserve legacy behavior.
	HiddenChoicePassthroughSymbols []bool

	// ABI 15: Grammar semantic version
	Metadata LanguageMetadata

	// External scanner (nil if not needed)
	ExternalScanner ExternalScanner
	ExternalSymbols []Symbol // external token index -> symbol

	// ImmediateTokens is a bitmask of symbol IDs that are token.immediate() tokens.
	// When the lexer matches one of these after consuming whitespace, the match
	// should be rejected — immediate tokens must match at the original position.
	// nil means no immediate tokens (common for ts2go grammars).
	ImmediateTokens []bool
	// ZeroWidthTokens is a bitmask of symbol IDs whose DFA terminal pattern can
	// intentionally match empty input. nil means this information is unavailable,
	// which preserves historical lexer behavior for ts2go blobs.
	ZeroWidthTokens []bool

	// ExternalLexStates maps external lex state IDs (from LexMode.ExternalLexState)
	// to a boolean slice indicating which external tokens are valid. Row 0 is
	// always all-false (no external tokens valid). When non-nil, this table is
	// used instead of parse-action-table probing to compute validSymbols for the
	// external scanner, matching C tree-sitter's ts_external_scanner_states.
	ExternalLexStates [][]bool

	// InitialState is the parser's start state. In tree-sitter grammars
	// this is always 1 (state 0 is reserved for error recovery). For
	// hand-built grammars it defaults to 0.
	InitialState StateID

	// NonTerminalAliasMap mirrors tree-sitter C's ts_non_terminal_alias_map.
	// Rows are indexed by nonterminal symbol and contain aliases that require
	// preserving the wrapper during alias-bearing reductions. This is cold
	// metadata and intentionally lives in the struct's cold tail so parser hot
	// field offsets stay stable.
	NonTerminalAliasMap [][]Symbol

	// ExternalScannerFullParseRetryPolicy is a certified language-level policy
	// for scheduling the extra external-scanner full-parse retry. Zero preserves
	// the generic behavior for legacy blobs and caller-constructed languages.
	ExternalScannerFullParseRetryPolicy ExternalScannerFullParseRetryPolicy

	// FullParseAcceptedErrorRetryProfile is certified against an exact language
	// blob. Zero preserves widened-stack and no-stacks retries for legacy blobs,
	// caller-constructed languages, and language overrides.
	FullParseAcceptedErrorRetryProfile FullParseAcceptedErrorRetryProfile

	// AutomaticForestMemoryAllowanceBytes bounds only the speculative forest
	// phase used by automatic dispatch. Zero preserves the full parse budget for
	// legacy blobs, caller-constructed languages, language overrides, and
	// explicit ParseForestExperimental calls. Built-in values are certified and
	// attached only after exact blob-identity verification.
	AutomaticForestMemoryAllowanceBytes int64

	// AutomaticForestEnabledByDefault certifies the speculative forest route for
	// this exact language artifact. Checked-in built-ins receive this bit only
	// from an exact-blob runtime profile; legacy blobs, same-name custom grammars,
	// adapted grammars, and overrides retain the conservative false default.
	AutomaticForestEnabledByDefault bool

	// FullParseArenaDensityCapEnabled explicitly opts this language into the ASCII
	// structural-density arena cap. Checked-in built-ins are enabled automatically
	// only through an exact-blob runtime profile. Callers may explicitly opt in
	// custom or adapted languages; false preserves the baseline arena policy.
	FullParseArenaDensityCapEnabled bool

	// FullParseGSSConvergenceEnabled certifies faithful convergence for this
	// language artifact. The parser keeps clean alternatives in the graph when
	// one stack survives each merge group. Checked-in languages receive this
	// setting only through an exact blob profile. Callers can enable it for
	// custom languages after equivalent tree tests. An explicit merge limit of
	// one also enables this behavior for a fresh full parse.
	FullParseGSSConvergenceEnabled bool

	// NativeResultCompatibility identifies result-tree shapes produced natively
	// by this exact language artifact. Zero keeps conservative post-parse
	// compatibility fallbacks for legacy blobs, generated grammars, caller-built
	// languages, and overrides whose native behavior has not been certified.
	NativeResultCompatibility ResultCompatibilityCapability

	// NativeUnaryWrapperFlattening identifies exact same-span unary wrappers
	// that C omits below a public parent in one parser state. Exact runtime
	// profiles populate the symbol and state identities. Custom and stale
	// artifacts keep the wrappers.
	NativeUnaryWrapperFlattening []UnaryWrapperFlatteningRule

	// CompactConvergedReductionSplitDropsCertified permits the compact
	// fresh-full route to accept after it drops a no-action head descended from
	// a converged-path reduction split. Exact built-in artifact profiles set
	// this only after C-oracle parity proves that production selects the same
	// surviving path. Custom and adapted languages fail closed by default.
	CompactConvergedReductionSplitDropsCertified bool

	// CompactEOFAcceptNoActionSiblingsCertified permits one authenticated EOF
	// accept head to discard siblings that have no action for that same EOF.
	// Exact built-in profiles set this only after C-oracle parity proves the
	// accepted head matches production. Custom and adapted languages fail closed.
	CompactEOFAcceptNoActionSiblingsCertified bool

	// CompactPrimaryAcceptanceDerivationCertified permits the compact fresh-full
	// route to select one primary derivation over secondary conflict derivations.
	// The primary score must be at least every secondary score. Exact built-in
	// profiles set this only after C-oracle parity proves production selects it.
	CompactPrimaryAcceptanceDerivationCertified bool

	// CompactAcceptanceStructuralElectionCertified permits the compact route
	// to apply C's raw subtree ordering to a clean, tied acceptance frontier.
	// Exact built-in profiles set this only after the locked C oracle proves
	// the compact derivation order and result for that grammar artifact.
	// Custom, adapted, and stale artifacts retain the false default.
	CompactAcceptanceStructuralElectionCertified bool

	// CompactMixedGSSMergeCertified permits one boundary merge to join flat
	// and graph-structured stack forms with C's physical receiver ownership.
	// Exact built-in profiles set this only after locked C parity proves the
	// mixed representation path for that grammar artifact. Custom, adapted,
	// and stale artifacts retain the false default.
	CompactMixedGSSMergeCertified bool

	// CompactLexerSkippedPrefixTilingCertified permits an internal compact
	// reduction gap when the next accepted terminal carries exact DFA evidence
	// for the complete skipped prefix. Exact built-in profiles set this only
	// after locked-C parity proves the result for that grammar artifact.
	// Custom, adapted, and stale artifacts retain the false default.
	CompactLexerSkippedPrefixTilingCertified bool

	// ExactStackNodeEquivalenceCertified preserves deep stack-node alternatives
	// until generic result selection. Exact built-in profiles set this only when
	// bounded equivalence can merge parity-relevant shapes. Custom, adapted, and
	// stale artifacts retain bounded equivalence unless callers opt in.
	ExactStackNodeEquivalenceCertified bool

	// CompactPackedGSSVersionOrderCertified permits the compact fresh-full route
	// to use C's physical stack-version transaction order for packed graph-
	// structured stack reductions. The transaction includes action-slot ownership,
	// same-round scheduling, bounded wave-order pop traversal, C-equivalent link
	// packing, boundary packing, and packed-child election. Exact built-in
	// profiles set this only after locked-C receipts and memory-budget tests
	// certify the complete bundle. Custom, adapted, and stale artifacts retain
	// the false default.
	CompactPackedGSSVersionOrderCertified bool

	// CompactStrategy2ErrorRegionCertified permits the compact fresh-full route
	// to attempt native strategy-2 recovery (error-region absorb and
	// condense-resume, campaign v7 tranche B3 stage S3) for a true no-table-
	// action point: close in-progress productions on a single deterministic
	// path, open an ERROR region, absorb tokens the table cannot place, and
	// resume once the pre-error state accepts the current token. Exact
	// built-in profiles set this only after C-oracle parity proves the
	// resulting tree matches the pinned C oracle exactly for the certified
	// witness class. Custom and adapted languages fail closed: an
	// uncertified grammar keeps declining to production at the same
	// no-action point exactly as before this stage landed.
	CompactStrategy2ErrorRegionCertified bool

	// CompactS3MixedShiftReduceClosureStates permits standalone strategy-2
	// recovery to stop its single-path closure at one certified state that has
	// both shift and reduce actions across terminals. Other mixed states decline.
	CompactS3MixedShiftReduceClosureStates []StateID

	// CompactRecoverEOFCertified permits one exact EOF no-action lineage to
	// publish tree-sitter's non-extra ERROR root from recover_eof. It is a
	// compatibility marker. The compact route also requires the explicit,
	// artifact-bound CompactRecoverEOFArtifactReceipt below.
	CompactRecoverEOFCertified bool

	// CompactRecoverEOFArtifactReceipt carries locked-C boundary and scheduler
	// facts for the one recover_eof route certified on this language artifact.
	// Its zero value disables the route, including when the compatibility marker
	// above is set by an older caller.
	CompactRecoverEOFArtifactReceipt CompactRecoverEOFArtifactReceipt

	// CompactStackSummaryRecoveryCertified permits the compact fresh-full
	// route to scan C's bounded stack summary at a no-action point. A
	// successful scan forks the head into an ancestor-recovered lineage and
	// an error-absorb lineage.
	//
	// The fork also requires CompactStrategy2ErrorRegionCertified. Both
	// lineages continue to acceptance, where C-compatible error pricing selects
	// the result. Exact built-in profiles must certify the complete competition.
	// Custom and adapted languages retain the false default.
	CompactStackSummaryRecoveryCertified bool

	// CompactMissingTokenInsertionCertified permits the compact fresh-full
	// route to scan for C's missing-token recovery candidate. A successful
	// scan forks the head into missing and error-absorb lineages.
	//
	// The fork also requires CompactStrategy2ErrorRegionCertified. Both
	// lineages continue to acceptance, where C-compatible error pricing selects
	// the result. Exact built-in profiles must certify the complete competition.
	// Custom and adapted languages retain the false default.
	CompactMissingTokenInsertionCertified bool

	// CompactS5EOFMissingInsertionCertified permits the compact fresh-full
	// route to run S5 reductions and missing-token insertion when the elected
	// token is EOF. This is distinct from CompactRecoverEOFCertified: S5
	// publishes a grammar root with a missing leaf, not a recover_eof ERROR
	// root. Exact built-in profiles set this only after locked-C parity proves
	// the complete EOF competition. Custom and adapted languages retain the
	// false default.
	CompactS5EOFMissingInsertionCertified bool

	// CompactFaithfulS5RecoveryCertified permits the complete S5 scan
	// to merge equivalent physical recovery heads. The legacy bounded S5 path
	// remains active without this exact artifact capability.
	CompactFaithfulS5RecoveryCertified bool

	// CompactOwnedEOFRecoveryCertified permits the bounded owned EOF route.
	// The admission runner binds its required mechanisms as one bundle.
	// Publication requires executed version-owned EOF recovery, without prior
	// shared recovery or sibling drops. Other recovery grants remain separate.
	// Custom, adapted, and stale artifacts retain the false default.
	CompactOwnedEOFRecoveryCertified bool

	// CompactRecoveryTrailingLineageRetirementCertified permits the compact
	// scheduler to retire one trailing no-action missing lineage after the
	// earlier error-absorb lineage consumed the same elected token. This is the
	// exact two-version shape that C removes in its recovery condense tail.
	// Exact built-in profiles must certify the complete transition. Custom,
	// adapted, and stale artifacts retain the false default.
	CompactRecoveryTrailingLineageRetirementCertified bool

	// CompactRecoveryErrorModeKeywordCaptureCertified permits the compact
	// scheduler to apply the grammar's keyword lexer after an error-mode lex
	// returns the keyword-capture token. C performs this second lex while the
	// recovery stack is in ERROR_STATE. Exact built-in profiles must certify
	// the complete recovery election. Custom, adapted, and stale artifacts
	// retain the false default.
	CompactRecoveryErrorModeKeywordCaptureCertified bool

	// CompactRecoveryTerminalAliasRules permits the accepted-root leaf audit to
	// authenticate a materialized terminal alias after one certified recovery
	// resume. The materializer must also prove the exact raw terminal and alias
	// node relationship. Exact built-in profiles bind each rule to one grammar
	// blob. Custom, adapted, and stale artifacts retain an empty rule set.
	CompactRecoveryTerminalAliasRules []CompactRecoveryTerminalAliasRule

	// CompactRecoveryPlainFirstCertified preserves the ordinary compact lexer
	// for the first attempt. After a fail-closed decline, the route retries with
	// C error-mode lexing and the certified S3/S5 recovery mechanisms.
	//
	// Exact built-in profiles set this only when C-oracle differentials prove
	// that the retry adds recovery routes without removing clean routes. Custom,
	// adapted, and stale artifacts keep the direct recovery attempt.
	CompactRecoveryPlainFirstCertified bool

	// LineContinuationEscapeByte declares the single byte this language's
	// scanner treats as a line-continuation escape when immediately followed
	// by a newline (LF, or CR+LF) — for example PowerShell's backtick. C
	// tree-sitter's scanner consumes an escape+newline pair as ordinary
	// skipped trivia, the same treatment bytesAreParserPadding (mid-parse gap
	// classification) and parserTailAllowsCleanAcceptance (accepted-stack and
	// accepted-tree tail classification) already give backslash+newline
	// unconditionally. Backslash needs no per-language gate because no
	// grammar this parser loads leaves a bare backslash+newline as an
	// uncovered gap that must not be crossed: languages whose grammar assigns
	// backslash+newline its own meaning (for example Python's line_continuation
	// node) tokenize it as a real, accounted-for node rather than leaving a
	// gap for these padding checks to ever see. An arbitrary escape byte
	// cannot get that same unconditional treatment because it can collide
	// with unrelated grammar meaning elsewhere (for example backtick opens a
	// Markdown fence and a shell command substitution), so acceptance
	// requires this explicit per-language declaration. Zero (the default)
	// declares no continuation escape and leaves padding classification
	// exactly as it was before this field existed. Exact built-in profiles
	// set this only after C-oracle parity confirms the escape+newline pair is
	// scanner-owned padding for the certified blob (see
	// grammars/runtime_profiles.go). Custom, adapted, and generated languages
	// default to zero and are unaffected.
	LineContinuationEscapeByte byte
	// contains filtered or unexported fields
}

Language holds all data needed to parse a specific language. It mirrors tree-sitter's TSLanguage C struct, translated into idiomatic Go types with slice-based tables instead of raw pointers.

func LoadLanguage ¶ added in v0.9.0

func LoadLanguage(data []byte) (*Language, error)

LoadLanguage deserializes a compressed grammar blob into a Language. Blobs are produced by EncodeLanguageBlob, grammargen.Generate, or the grammar build toolchain. This is the only function needed at runtime to load pre-compiled grammars — no grammargen import required. It accepts legacy gzip blobs and version-enveloped trailer-bearing blobs.

func (*Language) BlobInfo ¶ added in v0.54.0

func (l *Language) BlobInfo() LanguageBlobInfo

BlobInfo returns the version metadata LoadLanguage recorded when it loaded this Language. The zero value (HasHeader == false) means either the blob predates the version header, came from a producer that does not write one, or this Language was never loaded through LoadLanguage at all (e.g. a static ts2go source-generated Language, or an in-memory grammargen Language that was never round-tripped through a blob). LoadLanguage still loads a headerless blob for compatibility; callers that want a stronger guarantee can check HasHeader and warn or reject.

func (*Language) CompatibleWithRuntime ¶

func (l *Language) CompatibleWithRuntime() bool

CompatibleWithRuntime reports whether this language can be parsed by the current runtime version. Unspecified versions (0) are treated as compatible.

func (*Language) FieldByName ¶

func (l *Language) FieldByName(name string) (FieldID, bool)

FieldByName returns the field ID for a given name, or (0, false) if not found. Returns (0, false) for a nil Language. Builds an internal map on first call for O(1) subsequent lookups.

func (*Language) GrammarBlobSHA256 ¶ added in v0.52.0

func (l *Language) GrammarBlobSHA256() ([32]byte, bool)

GrammarBlobSHA256 returns the exact compressed grammar blob identity: the SHA-256 of the bytes LoadLanguage was called with. It is the mechanism for pinning a grammar: a consumer that needs a specific blob's exact behavior (rather than "whichever blob this build happens to embed") should record this hash alongside the release it certified against and compare it after every embed/vendor update, so a silent grammar swap — same file name, different bytes — fails a check instead of changing behavior unnoticed. See also BlobInfo for the blob's declared generator and version metadata, which is a weaker, informational signal: two different blobs can carry the same declared version, but never the same SHA-256.

func (*Language) IsSupertype ¶ added in v0.6.0

func (l *Language) IsSupertype(sym Symbol) bool

IsSupertype reports whether sym is a supertype symbol.

func (*Language) KeywordLexAsciiTable ¶ added in v0.10.2

func (l *Language) KeywordLexAsciiTable() [][128]int32

KeywordLexAsciiTable returns the ASCII fast-path table for the keyword lexer DFA.

func (*Language) LexAsciiTable ¶ added in v0.10.2

func (l *Language) LexAsciiTable() [][128]int32

LexAsciiTable returns the pre-built ASCII fast-path transition table for the main lexer DFA. The table is built once per Language. Entry format:

bit 31 set  → skip transition (consume and reset token start)
bits 0-30   → next state ID (lexAsciiNoMatch if no transition)

func (*Language) LexModeStarts ¶ added in v0.19.0

func (l *Language) LexModeStarts() []lexModeStart

func (*Language) PublicSymbol ¶ added in v0.7.0

func (l *Language) PublicSymbol(sym Symbol) Symbol

PublicSymbol maps an internal symbol to its canonical public form. Multiple internal symbols may share the same visible name (e.g. HTML's _start_tag_name and _end_tag_name both display as "tag_name"). PublicSymbol returns the first symbol with that name, matching what SymbolByName returns. This ensures query patterns compiled with SymbolByName match nodes regardless of which alias produced them.

func (*Language) PublicSymbolForNamedness ¶ added in v0.19.0

func (l *Language) PublicSymbolForNamedness(sym Symbol, named bool) Symbol

PublicSymbolForNamedness maps an internal symbol to the canonical public symbol with the same display name and requested namedness. This lets query matching distinguish named nodes from anonymous tokens that share text.

func (*Language) QuerySymbolByName ¶ added in v0.53.0

func (l *Language) QuerySymbolByName(name string) (Symbol, bool)

QuerySymbolByName resolves a node type the way a query pattern does. It follows ts_language_symbol_for_name for a named lookup: the first symbol, in symbol order, that is visible or a supertype, is named, and has that name. Hidden non-supertype rules and anonymous tokens are not query node types. The result is the canonical public symbol for that name.

func (*Language) Size ¶ added in v0.20.6

func (l *Language) Size() int64

Size returns an approximate number of bytes retained by the decoded language tables and lazily-built lookup caches. It is intended for diagnostics and cache policy decisions, not as an exact Go heap accounting API.

func (*Language) SupertypeChildren ¶ added in v0.6.0

func (l *Language) SupertypeChildren(sym Symbol) []Symbol

SupertypeChildren returns the subtype symbols for a given supertype. Returns nil if sym is not a supertype or has no entries.

func (*Language) SymbolByName ¶

func (l *Language) SymbolByName(name string) (Symbol, bool)

SymbolByName returns the symbol ID for a given name, or (0, false) if not found. The "_" wildcard returns (0, true) as a special case. Builds an internal map on first call for O(1) subsequent lookups.

func (*Language) TokenSymbolsByName ¶

func (l *Language) TokenSymbolsByName(name string) []Symbol

TokenSymbolsByName returns all terminal token symbols whose display name matches name. The returned symbols are in grammar order.

func (*Language) Version ¶

func (l *Language) Version() uint32

Version returns the tree-sitter language ABI version.

type LanguageBlobInfo ¶ added in v0.54.0

type LanguageBlobInfo struct {
	// HasHeader is true when LoadLanguage found and validated a version
	// header on this blob.
	HasHeader bool
	// SchemaVersion is the blob header's own format version. Zero when
	// HasHeader is false.
	SchemaVersion uint16
	// GeneratorVersion identifies the tool and version that produced the
	// blob, e.g. "ts2go" or "grammargen". Empty when HasHeader is false or
	// the header did not set it.
	GeneratorVersion string
	// MinRuntimeVersion is the minimum BlobRuntimeVersion this runtime must
	// implement to load the blob correctly. Zero when HasHeader is false.
	MinRuntimeVersion uint32
}

LanguageBlobInfo describes the version metadata recorded in a grammar blob's header, or the fact that the blob (or the in-memory Language) has none. See Language.BlobInfo.

func UnwrapLanguageBlobVersionHeader ¶ added in v0.54.0

func UnwrapLanguageBlobVersionHeader(data []byte) (payload []byte, info LanguageBlobInfo, err error)

UnwrapLanguageBlobVersionHeader returns the inner payload and the header's declared version metadata as a LanguageBlobInfo. A blob with no recognized header magic is returned unchanged with info.HasHeader == false, preserving every wire format that predates this header. LoadLanguage calls this; a caller writing its own decoder around EncodeLanguageBlob's output (instead of calling LoadLanguage directly) needs it too, to stay in sync with the wire format and the runtime-version rejection.

type LanguageMetadata ¶ added in v0.6.0

type LanguageMetadata struct {
	MajorVersion uint8
	MinorVersion uint8
	PatchVersion uint8
}

LanguageMetadata holds the grammar's semantic version (ABI 15+).

type LexMode ¶

type LexMode struct {
	LexState                  uint16
	ExternalLexState          uint16
	ReservedWordSetID         uint16
	AfterWhitespaceLexState   uint16 // DFA start state to use after whitespace (0 = same as LexState)
	LexStateID                uint32 // widened DFA start state for grammargen tables with >64K lexer states
	AfterWhitespaceLexStateID uint32
}

LexMode maps a parser state to its lexer configuration.

func (LexMode) AfterWhitespaceLexStateIndex ¶ added in v0.16.0

func (m LexMode) AfterWhitespaceLexStateIndex() uint32

AfterWhitespaceLexStateIndex returns the alternate DFA start state used after whitespace, or zero when the primary lex state should be used.

func (LexMode) LexStateIndex ¶ added in v0.16.0

func (m LexMode) LexStateIndex() uint32

LexStateIndex returns the DFA start state for this lex mode. Older grammar blobs only populate the uint16 LexState field; grammargen-generated tables can populate LexStateID when the DFA table exceeds 64K states.

func (*LexMode) SetAfterWhitespaceLexStateIndex ¶ added in v0.16.0

func (m *LexMode) SetAfterWhitespaceLexStateIndex(idx uint32)

func (*LexMode) SetLexStateIndex ¶ added in v0.16.0

func (m *LexMode) SetLexStateIndex(idx uint32)

type LexState ¶

type LexState struct {
	AcceptToken    Symbol // 0 unless this state accepts a non-end token
	AcceptPriority int16  // lower = higher priority (0 for ts2go blobs = longest-match)
	Skip           bool   // true if accepted chars are whitespace
	AcceptEOF      bool   // true if this state accepts the end token at end of input
	Default        int    // default next state (-1 if none)
	EOF            int    // state on EOF (-1 if none)
	Transitions    []LexTransition
}

LexState is one state in the table-driven lexer DFA.

type LexTransition ¶

type LexTransition struct {
	Lo, Hi    rune // inclusive character range
	NextState int
	// Skip mirrors tree-sitter's SKIP(state): consume the matched rune
	// and continue lexing while resetting token start.
	Skip bool
}

LexTransition maps a character range to a next state.

type Lexer ¶

type Lexer struct {
	// contains filtered or unexported fields
}

Lexer tokenizes source text using a table-driven DFA.

func NewLexer ¶

func NewLexer(states []LexState, source []byte) *Lexer

NewLexer creates a new Lexer that will tokenize source using the given DFA state table.

func (*Lexer) Next ¶

func (l *Lexer) Next(startState uint32) Token

Next lexes the next token starting from the given lex state index. It automatically skips tokens from states where Skip=true (whitespace). Returns a zero-Symbol token with StartByte==EndByte at EOF.

func (*Lexer) NextWithErrorRuns ¶ added in v0.21.0

func (l *Lexer) NextWithErrorRuns(startState uint32) Token

NextWithErrorRuns behaves like Next, except that bytes for which no accepting DFA state exists are not silently dropped: the whole unlexable run is consumed and returned as an errorSymbol token. This mirrors C ts_parser__lex, which surfaces skipped characters as an error subtree — the run starts after any whitespace the DFA legitimately skipped and ends at the first position where a token can be lexed (or EOF).

type LookaheadIterator ¶ added in v0.6.0

type LookaheadIterator struct {
	// contains filtered or unexported fields
}

LookaheadIterator iterates over valid symbols for a given parse state. It precomputes the full set of symbols that have valid parse actions in the specified state, enabling autocomplete and error diagnostic use cases.

func NewLookaheadIterator ¶ added in v0.6.0

func NewLookaheadIterator(lang *Language, state StateID) (*LookaheadIterator, error)

NewLookaheadIterator creates an iterator over all symbols that have valid parse actions in the given state. Returns an error if the state is out of range for the language's parse tables.

func (*LookaheadIterator) CurrentSymbol ¶ added in v0.6.0

func (it *LookaheadIterator) CurrentSymbol() Symbol

CurrentSymbol returns the symbol at the current iterator position. Must be called after a successful Next().

func (*LookaheadIterator) CurrentSymbolName ¶ added in v0.6.0

func (it *LookaheadIterator) CurrentSymbolName() string

CurrentSymbolName returns the name of the symbol at the current iterator position. Returns "" if the position is invalid or the symbol has no name.

func (*LookaheadIterator) Language ¶ added in v0.6.0

func (it *LookaheadIterator) Language() *Language

Language returns the language associated with this iterator.

func (*LookaheadIterator) Next ¶ added in v0.6.0

func (it *LookaheadIterator) Next() bool

Next advances the iterator to the next valid symbol. Returns false when there are no more symbols.

func (*LookaheadIterator) ResetState ¶ added in v0.6.0

func (it *LookaheadIterator) ResetState(state StateID) error

ResetState resets the iterator to enumerate valid symbols for a different parse state within the same language. Returns an error if the state is out of range.

type Node ¶

type Node struct {
	// contains filtered or unexported fields
}

Node is a syntax tree node.

func NewLeafNode ¶

func NewLeafNode(sym Symbol, named bool, startByte, endByte uint32, startPoint, endPoint Point) *Node

NewLeafNode creates a terminal/leaf node.

func NewParentNode ¶

func NewParentNode(sym Symbol, named bool, children []*Node, fieldIDs []FieldID, productionID uint16) *Node

NewParentNode creates a non-terminal node with children. It sets parent pointers on all children and computes byte/point spans from the first and last children. If any child has an error, the parent is marked as having an error too. A nil entry in children is dropped, along with the matching entry in fieldIDs at the same index, before the parent node is built.

func (*Node) Child ¶

func (n *Node) Child(i int) *Node

Child returns the i-th child, or nil if i is out of range.

func (*Node) ChildByFieldName ¶

func (n *Node) ChildByFieldName(name string, lang *Language) *Node

ChildByFieldName returns the first child assigned to the given field name, or nil if no child has that field. The Language is needed to resolve field names to IDs. Uses Language.FieldByName for O(1) lookup.

func (*Node) ChildCount ¶

func (n *Node) ChildCount() int

ChildCount returns the number of children (both named and anonymous).

func (*Node) Children ¶

func (n *Node) Children() []*Node

Children returns a slice of all children.

func (*Node) DescendantForByteRange ¶ added in v0.6.0

func (n *Node) DescendantForByteRange(startByte, endByte uint32) *Node

DescendantForByteRange returns the smallest descendant selected by upstream tree-sitter's byte-range walk. For a valid unsatisfiable or out-of-bounds range it returns the receiver; a nil receiver or reversed range returns nil.

func (*Node) DescendantForPointRange ¶ added in v0.6.0

func (n *Node) DescendantForPointRange(startPoint, endPoint Point) *Node

DescendantForPointRange returns the smallest descendant selected by upstream tree-sitter's point-range walk. For a valid unsatisfiable or out-of-bounds range it returns the receiver; a nil receiver or reversed range returns nil.

func (*Node) Edit ¶ added in v0.7.0

func (n *Node) Edit(edit InputEdit)

Edit adjusts this node's byte/point span for a source edit.

If the node belongs to a larger tree, the edit is applied from the containing root so sibling and ancestor spans remain consistent. Unlike Tree.Edit, this method does not record edit history on a Tree.

Do not call Node.Edit for an edit that Tree.Edit already applied. Tree.Edit updates every node of the tree in place, so a second call moves the spans twice. C code that calls ts_node_edit after ts_tree_edit needs no Node.Edit call here.

func (*Node) EndByte ¶

func (n *Node) EndByte() uint32

EndByte returns the byte offset where this node ends (exclusive). Returns 0 for a nil node.

func (*Node) EndPoint ¶

func (n *Node) EndPoint() Point

EndPoint returns the row/column position where this node ends. Returns the zero Point for a nil node.

func (*Node) FieldNameForChild ¶ added in v0.6.0

func (n *Node) FieldNameForChild(i int, lang *Language) string

FieldNameForChild returns the field name assigned to the i-th child, or an empty string when no field is assigned.

func (*Node) HasChanges ¶ added in v0.6.0

func (n *Node) HasChanges() bool

HasChanges reports whether this node was marked dirty by Tree.Edit.

func (*Node) HasError ¶

func (n *Node) HasError() bool

HasError reports whether this node or any descendant contains a parse error. Returns false for a nil node.

func (*Node) HasErrorOrMissing ¶ added in v0.49.0

func (n *Node) HasErrorOrMissing() bool

HasErrorOrMissing reports whether this node or a descendant contains an ERROR or MISSING node. Use it for strict parse-health checks.

func (*Node) IsError ¶ added in v0.6.0

func (n *Node) IsError() bool

IsError reports whether this node is an explicit error node. Returns false for a nil node.

func (*Node) IsExtra ¶ added in v0.6.0

func (n *Node) IsExtra() bool

IsExtra reports whether this node was marked as extra syntax (e.g. whitespace/comments outside the core parse structure). Returns false for a nil node.

func (*Node) IsMissing ¶

func (n *Node) IsMissing() bool

IsMissing reports whether this node was inserted by error recovery. Returns false for a nil node.

func (*Node) IsNamed ¶

func (n *Node) IsNamed() bool

IsNamed reports whether this is a named node (as opposed to anonymous syntax like punctuation). Returns false for a nil node.

func (*Node) NamedChild ¶

func (n *Node) NamedChild(i int) *Node

NamedChild returns the i-th named child (skipping anonymous children), or nil if i is out of range.

func (*Node) NamedChildCount ¶

func (n *Node) NamedChildCount() int

NamedChildCount returns the number of named children.

func (*Node) NamedDescendantForByteRange ¶ added in v0.6.0

func (n *Node) NamedDescendantForByteRange(startByte, endByte uint32) *Node

NamedDescendantForByteRange returns the smallest named descendant selected by upstream tree-sitter's byte-range walk. For a valid unsatisfiable or out-of-bounds range it returns the receiver; a nil receiver or reversed range returns nil.

func (*Node) NamedDescendantForPointRange ¶ added in v0.6.0

func (n *Node) NamedDescendantForPointRange(startPoint, endPoint Point) *Node

NamedDescendantForPointRange returns the smallest named descendant selected by upstream tree-sitter's point-range walk. For a valid unsatisfiable or out-of-bounds range it returns the receiver; a nil receiver or reversed range returns nil.

func (*Node) NamedNodeAtByte ¶ added in v0.20.6

func (n *Node) NamedNodeAtByte(byteOffset uint32) *Node

NamedNodeAtByte returns the smallest named descendant that contains byteOffset. It follows the same boundary behavior as NodeAtByte.

func (*Node) NextSibling ¶

func (n *Node) NextSibling() *Node

NextSibling returns the next sibling node, or nil when this is the last child or has no parent.

func (*Node) NodeAtByte ¶ added in v0.20.6

func (n *Node) NodeAtByte(byteOffset uint32) *Node

NodeAtByte returns the smallest descendant that contains byteOffset. If the offset is exactly at this node's end byte, it performs a zero-width lookup at that boundary. Returns nil when the offset is outside this node.

func (*Node) Parent ¶

func (n *Node) Parent() *Node

Parent returns this node's parent, or nil if it is the root.

func (*Node) ParseState ¶

func (n *Node) ParseState() StateID

ParseState returns the parser state associated with this node. Returns 0 for a nil node.

func (*Node) PreGotoState ¶ added in v0.6.0

func (n *Node) PreGotoState() StateID

PreGotoState returns the parser state that was on top of the stack before this node was pushed (i.e., the state exposed after popping children during reduce). For non-leaf nodes: lookupGoto(PreGotoState, Symbol) == ParseState. Returns 0 for a nil node.

func (*Node) PrevSibling ¶

func (n *Node) PrevSibling() *Node

PrevSibling returns the previous sibling node, or nil when this is the first child or has no parent.

func (*Node) Range ¶

func (n *Node) Range() Range

Range returns the full span of this node as a Range. Returns the zero Range for a nil node.

func (*Node) SExpr ¶ added in v0.6.0

func (n *Node) SExpr(lang *Language) string

SExpr returns a tree-sitter-style S-expression for this node. It includes only named nodes for stable debug snapshots.

func (*Node) StartByte ¶

func (n *Node) StartByte() uint32

StartByte returns the byte offset where this node begins. Returns 0 for a nil node.

func (*Node) StartPoint ¶

func (n *Node) StartPoint() Point

StartPoint returns the row/column position where this node begins. Returns the zero Point for a nil node.

func (*Node) Symbol ¶

func (n *Node) Symbol() Symbol

Symbol returns the node's grammar symbol. Returns 0 for a nil node, matching a null TSNode handle in the C API.

func (*Node) Text ¶

func (n *Node) Text(source []byte) string

Text returns the source text covered by this node. Returns an empty string for nil nodes or invalid byte ranges.

func (*Node) Type ¶

func (n *Node) Type(lang *Language) string

Type returns the node's type name from the language. Returns "" for a nil node or a nil lang.

type NormalizationPassRuntime ¶ added in v0.20.0

type NormalizationPassRuntime struct {
	Name           string
	Checked        uint64
	Run            uint64
	NodesVisited   uint64
	NodesRewritten uint64
	Nanos          int64
}

type OutlineOwnerRule ¶ added in v0.49.0

type OutlineOwnerRule struct {
	// NodeType is the definition node type the rule applies to, for example
	// "method_declaration".
	NodeType string
	// OwnerField is the field name read through Node.ChildByFieldName, for
	// example "receiver".
	OwnerField string
	// Unwrap lists node types the rule descends through, for example
	// "parameter_list", "parameter_declaration", "pointer_type".
	Unwrap []string
	// NameTypes lists the accepted terminal node types, for example
	// "type_identifier".
	NameTypes []string
}

OutlineOwnerRule is a declarative, per-language rule that resolves the non-lexical owner of a definition, for example the receiver type of a Go method. It reads a named field and unwraps through a fixed list of node types until it reaches exactly one node of an accepted terminal type. If unwrapping does not end at exactly one such node, the rule fails closed and Owner stays empty.

Rules are data. The core holds no rule rows; the grammars package owns the per-language table (grammars.OutlineOwnerRules) and passes it through WithOutlineOwnerRules.

type OutlineReport ¶ added in v0.49.0

type OutlineReport struct {
	// Symbols counts every emitted symbol, nested symbols included.
	Symbols int
	// OmittedNoName counts definition captures whose name capture is absent
	// or whose name text trims to empty.
	OmittedNoName int
	// OmittedDuplicate counts candidates dropped because an earlier
	// candidate holds the same Range, the same Kind, the same Name, and the
	// same NameRange. These are true repeats: dropping one changes nothing.
	OmittedDuplicate int
	// OmittedNameConflict counts candidates dropped because two or more
	// candidates hold the same Range and the same Kind but disagree about
	// the name. Every member of such a group is dropped.
	//
	// This is the common shape when shared tags patterns bind "@name" twice
	// on one definition node. In C-family method syntax the two bindings are
	// the return type and the method name, and no language-neutral rule can
	// tell them apart. Choosing by capture order would publish the return
	// type as the method name, so the outline drops the group instead and
	// counts it here. The remedy is a data edit: constrain the pattern with
	// a "name:" field, the way the Go tags override already does.
	OmittedNameConflict int
	// OmittedConflict counts candidates dropped because two or more
	// candidates hold the same Range with different Kinds. Every member of
	// such a group is dropped; the outline does not guess which one is
	// right.
	OmittedConflict int
	// OmittedOverlap counts candidates dropped because they partially
	// overlap an accepted symbol.
	//
	// This rule is defensive. Two nodes of one tree are always nested or
	// disjoint, so a partial overlap cannot arise from a single tree today.
	// The rule keeps the forest well-defined if a future capture source
	// relaxes that, and the unit cases exercise it directly.
	OmittedOverlap int
	// OmittedInvalidNameRange counts candidates whose NameRange is not byte
	// contained in Range, or whose Range is itself inverted. Like
	// OmittedOverlap this is defensive: the captures of a match all sit
	// inside the matched subtree, so a single tree does not produce this
	// today.
	OmittedInvalidNameRange int
	// OmittedMultipleDefinitions counts matches dropped because they carry
	// more than one "@definition.X" capture. Such a match names two
	// definitions at once and the outline does not choose between them.
	// No inferred pattern does this today; the counter exists so that a data
	// edit that introduces the shape is visible instead of silent.
	OmittedMultipleDefinitions int
	// OwnerRuleMisses counts symbols where an owner rule matched the node
	// type but the field or the normalization did not resolve a single
	// name. It is zero whenever no attached rule names a symbol's NodeType,
	// including whenever WithOutlineOwnerRules was never called.
	OwnerRuleMisses int
	// DeclineReason names why the outliner produced nothing, or is empty
	// when it ran the query. See the OutlineDecline constants. An empty
	// reason with zero Symbols means the query ran and matched nothing,
	// which is a different fact from every declined case.
	DeclineReason string
	// Truncated reports that query execution hit the match limit or the
	// match work budget, so the symbol list is partial.
	Truncated bool
	// TreeHasError reports that the parsed tree holds an ERROR or MISSING
	// node, so the parser did not fully recover the source.
	//
	// Symbols are the query's projection of the RECOVERED tree. When
	// TreeHasError is true, definitions inside or after unrecovered regions
	// may be absent, over-long, or missing entirely, and no omission counter
	// fires for them: the query never produced a candidate to omit. Do not
	// read "every counter is zero" as "the outline is complete" on such a
	// tree. Do not assume the symbols before the first error are
	// trustworthy either; a truncated definition is itself a symbol, and its
	// Range can swallow everything that follows it.
	TreeHasError bool
}

OutlineReport is the receipt for one OutlineTree call. Every candidate the tags query produced is either emitted as a symbol or counted in exactly one omission counter, so a dropped candidate cannot look like an absent one.

The accounting identity the gates assert is:

Symbols + OmittedNoName + OmittedDuplicate + OmittedNameConflict +
    OmittedConflict + OmittedOverlap + OmittedInvalidNameRange +
    OmittedMultipleDefinitions == candidates

OwnerRuleMisses is not an omission counter; a miss keeps the symbol and leaves Owner empty.

READ THIS BEFORE TREATING Omitted() == 0 AS "COMPLETE". Candidates() counts what the tags query produced, not what the file contains. A definition the query has no pattern for produces no candidate and no omission, so it is invisible to every counter here. Call Outliner.DefinitionKinds to see the kinds the compiled query can emit at all; a kind absent from that list can never appear in the outline, however many such definitions the file holds.

func (OutlineReport) Candidates ¶ added in v0.49.0

func (r OutlineReport) Candidates() int

Candidates returns the number of definition candidates the tags query produced: the emitted symbols plus every omission.

This is a count of QUERY OUTPUT, not of source content. A definition shape the query has no pattern for never becomes a candidate.

func (OutlineReport) Declined ¶ added in v0.49.0

func (r OutlineReport) Declined() bool

Declined reports whether the outliner produced nothing because it refused to run, rather than because the query matched nothing.

func (OutlineReport) Omitted ¶ added in v0.49.0

func (r OutlineReport) Omitted() int

Omitted returns the total number of candidates the outliner dropped.

type OutlineSymbol ¶ added in v0.49.0

type OutlineSymbol struct {
	// Kind is the normalized definition kind. It comes from the
	// "@definition.X" capture suffix through one fixed, language-neutral
	// table (outlineKindTable) plus one fixed node-type refinement table
	// (outlineKindRefinement). It never comes from the language name.
	Kind string
	// Name is the text of the "@name" capture, with leading and trailing
	// white space removed. A "#strip!" directive on the capture applies,
	// because the text comes from QueryCapture.Text.
	//
	// Invariant: NameRange is the capture's effective span from
	// QueryCapture.Range, so Name and the source bytes at NameRange agree
	// only when no directive rewrote the text or the range, and the node
	// carries no surrounding white space. Trimming, a "#strip!" directive, or
	// a "#offset!" directive can make Name and NameRange diverge from the
	// capture node's own text and range.
	// TestOutlineNameAgreesWithNameRange pins that agreement on every
	// committed fixture, so a language where the two diverge shows up as a
	// test failure rather than as a silently wrong span.
	Name string
	// NodeType is the grammar node type of the captured definition node.
	NodeType string
	// Range is the effective span of the "@definition.X" capture, from
	// QueryCapture.Range. A "#offset!" directive on that capture adjusts it.
	Range Range
	// NameRange is the effective span of the "@name" capture, from
	// QueryCapture.Range. It is always contained in Range; a candidate that
	// breaks containment is omitted and counted.
	NameRange Range
	// Owner is the non-lexical owner name, for example the receiver type of
	// a Go method. It is set only when a declarative OutlineOwnerRule
	// attached through WithOutlineOwnerRules matches this symbol's NodeType
	// and resolves a single identifier; otherwise it is "". Lexical
	// containment never sets Owner -- that information lives in Children.
	Owner string
	// Children holds the definitions lexically nested in this one, in source
	// order. Nesting comes from byte containment of Range, never from the
	// language name.
	Children []OutlineSymbol
}

OutlineSymbol is one entry of a language-neutral file outline: a definition the tags query captured, its normalized kind, its name, its spans, and the definitions lexically nested inside it.

The projection is read-only. It never changes the tree and never copies a definition body; Name and Owner are the only text it extracts.

type Outliner ¶ added in v0.49.0

type Outliner struct {
	// contains filtered or unexported fields
}

Outliner projects a language-neutral file outline from a parsed tree. It compiles one tags query and reuses it across trees.

The outliner is a read-only projection. It runs no parse, it changes no tree, and it holds no parser state. Build one per language and call OutlineTree with trees that language produced.

An Outliner is safe for concurrent use by several goroutines. OutlineTree writes no Outliner field, and every call takes its own query cursor and its own working slices. TestOutlineTreeIsSafeForConcurrentUse runs the shared path under the race detector and compares every result.

func NewOutliner ¶ added in v0.49.0

func NewOutliner(lang *Language, tagsQuery string, opts ...OutlinerOption) (*Outliner, error)

NewOutliner creates an Outliner for a language and its tags query.

An empty or blank tags query is not an error. The outliner declines: it returns no symbols and sets OutlineReport.QueryEmpty, so a language without tags data is observably uncovered instead of silently empty.

func (*Outliner) DefinitionKinds ¶ added in v0.49.0

func (o *Outliner) DefinitionKinds() []string

DefinitionKinds returns the normalized kinds the compiled query can emit, in sorted order. It is derived from the query's capture names, so it states what the tags DATA can express for this language.

Use it to read an outline honestly. A Go outline, for example, reports only "function" and "method", because the Go tags override carries no pattern for a type, a constant, or a variable. Those definitions never become candidates and never reach an omission counter, so an all-zero receipt does not mean the file held nothing else.

The list is an upper bound on Kind values, not a promise that each appears. A node-type refinement can also map a capture to a kind outside this list; the refinement rows are documented on outlineKindRefinement.

func (*Outliner) Language ¶ added in v0.49.0

func (o *Outliner) Language() *Language

Language returns the language this outliner was built for.

func (*Outliner) OutlineBound ¶ added in v0.52.0

func (o *Outliner) OutlineBound(tree *BoundTree) ([]OutlineSymbol, OutlineReport)

OutlineBound projects an outline from a BoundTree. It uses the same validation, query, and receipt path as OutlineTree.

func (*Outliner) OutlineTree ¶ added in v0.49.0

func (o *Outliner) OutlineTree(tree *Tree) ([]OutlineSymbol, OutlineReport)

OutlineTree projects the outline of an already-parsed tree.

The call is read-only: it runs the tags query over the tree, normalizes the kinds, drops ambiguous candidates, and assembles the containment forest. It parses nothing and mutates nothing.

Trees that hold ERROR or MISSING nodes are not special-cased. The outline is the projection of whatever the tags query matched on the tree as the parser produced it. The receipt says so through OutlineReport.TreeHasError; read that field's documentation before trusting an outline over a damaged tree.

When the outliner refuses to run, the returned report carries a DeclineReason and no symbols. An empty DeclineReason with no symbols is a different fact: the query ran and matched nothing.

func (*Outliner) QueryEmpty ¶ added in v0.49.0

func (o *Outliner) QueryEmpty() bool

QueryEmpty reports whether the outliner declined for want of tags data.

type OutlinerOption ¶ added in v0.49.0

type OutlinerOption func(*Outliner)

OutlinerOption configures an Outliner.

func WithOutlineMatchLimit ¶ added in v0.49.0

func WithOutlineMatchLimit(limit uint32) OutlinerOption

WithOutlineMatchLimit bounds the number of query matches the outliner accepts. When the limit is reached, OutlineReport.Truncated reports true and the symbol list is partial.

A limit of zero keeps the query engine default, exactly as a budget of zero does in WithOutlineMatchWorkBudget. Zero always means "leave the default alone" in both options.

func WithOutlineMatchWorkBudget ¶ added in v0.49.0

func WithOutlineMatchWorkBudget(budget int) OutlinerOption

WithOutlineMatchWorkBudget bounds the enumeration steps the matcher may take for each pattern and node. Exhausting the budget sets OutlineReport.Truncated. Raise it for very large files whose outline comes back truncated.

A budget of zero keeps the query engine default. This option cannot disable the guard: the underlying engine reads zero as "unlimited", and an outline caller writing zero means "default", so the option refuses to pass zero through. Removing the guard is not an outline concern.

func WithOutlineOwnerRules ¶ added in v0.49.0

func WithOutlineOwnerRules(rules []OutlineOwnerRule) OutlinerOption

WithOutlineOwnerRules attaches declarative owner rules, indexed by node type. The grammars package owns the per-language rows; the core holds none. A later call REPLACES the rules an earlier call set; the option does not accumulate.

OutlineTree applies the attached rules to every symbol: a rule whose NodeType matches resolves OutlineSymbol.Owner when its OwnerField is present on the node and its Unwrap/NameTypes walk reaches exactly one accepted terminal node (resolveOutlineOwner, outline_owner.go). Any other outcome leaves Owner empty and, for a NodeType a rule did match, increments OutlineReport.OwnerRuleMisses. A NodeType no attached rule names never touches Owner or OwnerRuleMisses at all.

What construction checks today: every rule must name a NodeType and an OwnerField, because a rule missing either can never resolve an owner.

What construction does NOT check today: it accepts an empty NameTypes list, a blank entry inside Unwrap or NameTypes, and a NodeType or OwnerField the language does not define. Each of those fails closed at resolution time and inflates OwnerRuleMisses rather than producing a wrong Owner. A caller that wants those rows filtered out ahead of time, so a stale rule costs nothing at resolution either, gates its own table on symbol and field presence the way grammars.OutlineOwnerRules does.

type ParseAction ¶

type ParseAction struct {
	Type              ParseActionType
	State             StateID // target state (shift/recover)
	Symbol            Symbol  // reduced symbol (reduce)
	ChildCount        uint8   // children consumed (reduce)
	DynamicPrecedence int16   // precedence (reduce)
	ProductionID      uint16  // which production (reduce)
	Extra             bool    // is this an extra token (shift)
	ExtraChain        bool    // does this shift enter a nonterminal extra chain
	Repetition        bool    // is this a repetition (shift)
}

ParseAction is a single parser action from the parse table.

type ParseActionEntry ¶

type ParseActionEntry struct {
	Reusable bool
	Actions  []ParseAction
}

ParseActionEntry is a group of actions for a (state, symbol) pair.

type ParseActionTiming ¶ added in v0.19.0

type ParseActionTiming struct {
	ExtraShiftNanos      int64
	NoActionNanos        int64
	NoActionRelexNanos   int64
	NoActionMissingNanos int64
	NoActionRecoverNanos int64
	NoActionErrorNanos   int64
	ConflictChoiceNanos  int64
	ConflictForkNanos    int64
	SingleShiftNanos     int64
	SingleReduceNanos    int64
	SingleAcceptNanos    int64
	SingleRecoverNanos   int64
	SingleOtherNanos     int64
}

type ParseActionType ¶

type ParseActionType uint8

ParseActionType identifies the kind of parse action.

const (
	ParseActionShift ParseActionType = iota
	ParseActionReduce
	ParseActionAccept
	ParseActionRecover
)

type ParseEquivStateRuntime ¶ added in v0.19.0

type ParseEquivStateRuntime struct {
	State                                 StateID
	StackEquivCalls                       uint64
	StackEquivTrue                        uint64
	StackEquivDepthMismatch               uint64
	StackEquivHashMismatch                uint64
	StackEquivStateMismatch               uint64
	StackEquivPayloadMismatch             uint64
	StackEquivEntryCompares               uint64
	StackEquivStateMismatchDepthSum       uint64
	StackEquivStateMismatchMaxDepth       uint32
	StackEquivStateMismatchDepthBuckets   [stackEquivMismatchDepthBucketCount]uint64
	StackEquivPayloadMismatchDepthSum     uint64
	StackEquivPayloadMismatchMaxDepth     uint32
	StackEquivPayloadMismatchDepthBuckets [stackEquivMismatchDepthBucketCount]uint64
	StackEquivPayloadHeaderSigDiff        uint64
	StackEquivPayloadHeaderSigSame        uint64
	StackEquivPayloadShallowSigDiff       uint64
	StackEquivPayloadShallowSigSame       uint64
	StackEquivPairKeyed                   uint64
	StackEquivPairUnkeyed                 uint64
	StackEquivPairRepeats                 uint64
	StackEquivPairRepeatTrue              uint64
	StackEquivPairRepeatFalse             uint64
	StackEquivPairRepeatMismatch          uint64
	StackEquivPairStores                  uint64
	MergeHeaderEqTotal                    uint64
	MergeDeepTrue                         uint64
	MergeDeepFalse                        uint64
	MergeHeaderDeepDivergent              uint64
	EquivCacheLookups                     uint64
	EquivCacheHits                        uint64
	EquivCacheStores                      uint64
	EquivCacheMisses                      uint64
	EquivCacheTrueHits                    uint64
	EquivCacheFalseHits                   uint64
	EquivCacheEpochMisses                 uint64
	EquivCacheKeyMisses                   uint64
	EquivCacheVersionMisses               uint64
	EquivSkipError                        uint64
	EquivSkipLeaf                         uint64
	EquivSkipFieldMismatch                uint64
	EquivExactCalls                       uint64
	EquivExactTrue                        uint64
	EquivExactPointerTrue                 uint64
	EquivExactNilMismatch                 uint64
	EquivExactHeaderMismatch              uint64
	EquivExactChildMismatch               uint64
	EquivExactTerminalCalls               uint64
	EquivExactTerminalTrue                uint64
	EquivExactTerminalFalse               uint64
	EquivFrontierCalls                    uint64
	EquivFrontierTrue                     uint64
	EquivExactChildCompares               uint64
	EquivFrontierChildScans               uint64
	EquivFrontierCandidateCompares        uint64
}

type ParseOption ¶ added in v0.6.0

type ParseOption func(*parseConfig)

ParseOption configures ParseWith behavior.

func WithOldTree ¶ added in v0.6.0

func WithOldTree(oldTree *Tree) ParseOption

WithOldTree enables incremental parsing against an edited prior tree.

func WithProfiling ¶ added in v0.6.0

func WithProfiling() ParseOption

WithProfiling enables incremental parse attribution in ParseResult.Profile.

func WithTokenSource ¶ added in v0.6.0

func WithTokenSource(ts TokenSource) ParseOption

WithTokenSource provides a custom token source for parsing.

type ParseReduceTiming ¶ added in v0.19.0

type ParseReduceTiming struct {
	RangeNanos         int64
	PendingParentNanos int64
	ChildBuildNanos    int64
	ParentBuildNanos   int64
	SpanNanos          int64
	StackPushNanos     int64
	NoTreeBuildNanos   int64
}

type ParseResult ¶ added in v0.6.0

type ParseResult struct {
	Tree *Tree
	// Profile is populated only when ParseWith uses WithProfiling for
	// incremental parsing.
	Profile IncrementalParseProfile
	// ProfileAvailable reports whether Profile contains attribution data.
	ProfileAvailable bool
}

ParseResult is returned by ParseWith.

type ParseRuntime ¶ added in v0.6.0

type ParseRuntime struct {
	StopReason     ParseStopReason
	ForestFastPath bool
	// IncrementalAcceptedErrorRetryAttempts records the bounded second
	// incremental pass used when an accepted, full-span ERROR tree was produced
	// under the wider incremental merge policy. The retry keeps the edited old
	// tree and reruns with the ordinary full-parse merge cap; it is not a fresh
	// parse fallback.
	IncrementalAcceptedErrorRetryAttempts uint8
	// IncrementalAcceptedErrorRetryAdopted is true only when that retry produced
	// a strictly better tree and replaced the first incremental result.
	IncrementalAcceptedErrorRetryAdopted bool
	// IncrementalAcceptedErrorRetryMergePerKey is the exact merge-per-key cap
	// used by the bounded retry.
	IncrementalAcceptedErrorRetryMergePerKey int
	// IncrementalAcceptedErrorRetryCause is a stable diagnostic reason for the
	// retry so corpus and benchmark matrices can distinguish this path.
	IncrementalAcceptedErrorRetryCause IncrementalRetryCause
	// IncrementalOldTreeReuseRoute reports whether this result was produced by
	// an actual old-tree reuse parse rather than an internal fresh fallback.
	IncrementalOldTreeReuseRoute bool
	// CompactIncrementalReuseRoute records execution with borrowed compact subtrees.
	CompactIncrementalReuseRoute bool
	// CompactIncrementalFullRecoveryRoute records a fresh compact recovery fallback.
	// This route does not borrow old-tree nodes.
	CompactIncrementalFullRecoveryRoute bool
	CompactIncrementalReusedSubtrees    uint64
	CompactIncrementalReusedBytes       uint64
	// CompactIncrementalFallbackReason records an attempted compact reparse decline.
	CompactIncrementalFallbackReason string
	// CompactReductions counts reductions executed by the compact scheduler.
	CompactReductions uint64
	// CompactPeakHeaders and CompactPeakDerivations are populated only when
	// SetCompactCertificationTelemetry is enabled on a compact parse.
	CompactPeakHeaders     uint64
	CompactPeakDerivations uint64
	// CRecoveryEnteredErrorState is true when the faithful C error-recovery
	// port (parser_recover_c.go) actually ran ts_parser__handle_error at
	// least once while producing this specific tree — i.e. some no-action
	// point was hit for the current lookahead. This is NOT proof the input is
	// malformed: LALR table limitations routinely drive well-formed,
	// compiling input into a momentary no-action point that C-recovery
	// resolves losslessly (ordinary GLR disambiguation). It is only a cheap
	// pre-filter for Parse()'s post-parse swallowed-error safety net
	// (resolveCRecoverySwallowedError) — see CRecoveryDroppedErrorForClean
	// for the actual, precise suspicion signal. It is captured per finalized
	// tree (not read from a raw Parser field) so a discarded retry attempt
	// can never leave a stale value on the tree that is actually returned.
	CRecoveryEnteredErrorState bool
	// CRecoveryDroppedErrorForClean is true when the stack SELECTED as this
	// tree's parse result (buildResultFromGLR, parser_result.go) carried an
	// unvalidated C-recovery marker (see glrStack.cRecoveryUnvalidatedMarker)
	// with no unflagged sibling reaching the same final position — i.e. the
	// selected lineage itself created a real ERROR node via cRecoverToState
	// (for a single-stack dead end with a small recovered span) and was
	// never re-validated by another cost competition. This is the precise
	// signature of the swallowed-error defect class (see
	// resolveCRecoverySwallowedError): unlike an ordinary clean recovery (no
	// unvalidated marker, or a corroborating clean sibling, at the same
	// position), this means the specific result being returned lost real
	// ERROR content somewhere along its own lineage. Deliberately scoped to
	// the selected result only — NOT set for drops/forks on discarded
	// lineages elsewhere in the parse, nor for large/multi-stack recoveries;
	// both were tried and found to fire on ordinary GLR disambiguation for a
	// measurable fraction of valid, compiling source in a real repo-file
	// walk (the discarded-lineage version: thousands of times per large Go
	// file) and were the cause of a prior, over-broad version of this
	// signal's false-fire rate.
	CRecoveryDroppedErrorForClean bool
	// CRecoverySwallowedErrorFallbackAttempted is true when
	// resolveCRecoverySwallowedError actually re-parsed the source with the
	// C-recovery gate disabled to double-check a suspicious clean result (see
	// CRecoveryDroppedErrorForClean). Diagnostic only — lets callers measure
	// the fallback's false-fire rate (extra latency from a re-parse that is
	// usually discarded) against their own corpora, e.g. by walking a tree of
	// known-valid source files and counting how often this is true.
	CRecoverySwallowedErrorFallbackAttempted bool
	// CompactExternalScannerCheckpointTransferProven reports whether compact
	// materialization transferred every required terminal scanner checkpoint
	// into node sidecars. A false value disables later subtree reuse.
	CompactExternalScannerCheckpointTransferProven bool
	// CRecoverReductionCandidateCeilingHits and CRecoverMissingTokenCeilingHits
	// count how many times this parse's cDoAllPotentialReductions /
	// cHandleError missing-token search hit the
	// cRecoverMaxReductionCandidateAttempts / cRecoverMaxMissingTokenTrials
	// Go-side backstop ceilings (parser_recover_c.go), a diagnostic signal for
	// the spore.2026-08-02.walnut-e.memory-exhaustion fix. Both stay zero on
	// every currently-passing parse; neither ceiling halts the parse itself
	// (both fail gracefully into an already-supported "search found nothing"
	// path), so this counter is the only way to observe that either engaged.
	CRecoverReductionCandidateCeilingHits uint64
	CRecoverMissingTokenCeilingHits       uint64
	// CRecoverReductionCandidateAttemptsPeak and
	// CRecoverMissingTokenTrialAttemptsPeak record the single largest
	// candidateAttempts / missingTokenTrialAttempts value any ONE
	// cDoAllPotentialReductions call / cHandleError missing-token search
	// reached during this parse (not cumulative across calls). Diagnostic
	// only: lets a corpus walk report how close real input gets to
	// cRecoverMaxReductionCandidateAttempts / cRecoverMaxMissingTokenTrials.
	CRecoverReductionCandidateAttemptsPeak        uint64
	CRecoverMissingTokenTrialAttemptsPeak         uint64
	SourceLen                                     uint32
	ExpectedEOFByte                               uint32
	RootEndByte                                   uint32
	Truncated                                     bool
	TokenSourceEOFEarly                           bool
	TokensConsumed                                uint64
	LastTokenEndByte                              uint32
	LastTokenSymbol                               Symbol
	LastTokenWasEOF                               bool
	StopDiagnosticCaptured                        bool
	StopDiagnosticCRecoveryEnabled                bool
	StopDiagnosticCRecoveryGateReason             string
	StopDiagnosticRecoverActionAvailable          bool
	StopDiagnosticLastStackState                  StateID
	StopDiagnosticLastStackByte                   uint32
	StopDiagnosticLastStackDepth                  int
	StopDiagnosticTokenSymbol                     Symbol
	StopDiagnosticTokenStartByte                  uint32
	StopDiagnosticTokenEndByte                    uint32
	StopDiagnosticTokenNoLookahead                bool
	StopDiagnosticRootType                        string
	StopDiagnosticRootStartByte                   uint32
	StopDiagnosticRootEndByte                     uint32
	StopDiagnosticRootHasError                    bool
	StopDiagnosticFirstErrorFound                 bool
	StopDiagnosticFirstErrorStartByte             uint32
	StopDiagnosticFirstErrorEndByte               uint32
	StopDiagnosticFrontierStacks                  string
	StopDiagnosticFrontierActions                 string
	StopDiagnosticSameHeaderGroups                string
	StopDiagnosticCondenseGating                  string
	StopDiagnosticActionCaptured                  bool
	StopDiagnosticActionPhase                     string
	StopDiagnosticActionStackState                StateID
	StopDiagnosticActionStackByte                 uint32
	StopDiagnosticActionStackDepth                int
	StopDiagnosticActionTokenSymbol               Symbol
	StopDiagnosticActionTokenStartByte            uint32
	StopDiagnosticActionTokenEndByte              uint32
	StopDiagnosticActionTokenNoLookahead          bool
	StopDiagnosticActionType                      ParseActionType
	StopDiagnosticActionState                     StateID
	StopDiagnosticActionSymbol                    Symbol
	StopDiagnosticActionChildCount                uint8
	StopDiagnosticActionProductionID              uint16
	StopDiagnosticActionDynamicPrecedence         int16
	StopDiagnosticActionCount                     int
	StopDiagnosticActionResultState               StateID
	StopDiagnosticActionInReduceChain             bool
	StopDiagnosticActionReduceChainStep           int
	StopDiagnosticActionRepeatedSignatureCount    int
	StopDiagnosticActionReduceChainCycle          bool
	StopDiagnosticActionForceAdvanceAfterReduce   bool
	StopDiagnosticActionPostDispatchDefaultReduce bool
	StopDiagnosticActionAnyReduced                bool
	StopDiagnosticActionConsumedToken             bool
	StopDiagnosticActionDispatchShiftActions      int
	StopDiagnosticActionDispatchReduceActions     int
	StopDiagnosticActionDispatchAcceptActions     int
	StopDiagnosticActionDispatchRecoverActions    int
	StopDiagnosticActionDispatchOtherActions      int
	StopDiagnosticLastReduceCaptured              bool
	StopDiagnosticLastReducePhase                 string
	StopDiagnosticLastReduceStackState            StateID
	StopDiagnosticLastReduceStackByte             uint32
	StopDiagnosticLastReduceStackDepth            int
	StopDiagnosticLastReduceSymbol                Symbol
	StopDiagnosticLastReduceChildCount            uint8
	StopDiagnosticLastReduceProductionID          uint16
	StopDiagnosticLastReduceDynamicPrecedence     int16
	StopDiagnosticLastReduceResultState           StateID
	StopDiagnosticLastReduceInChain               bool
	StopDiagnosticLastReduceChainStep             int
	StopDiagnosticLastReduceRepeatedSigCount      int
	StopDiagnosticLastReduceChainCycle            bool
	IterationLimit                                int
	StackDepthLimit                               int
	NodeLimit                                     int
	MemoryBudgetBytes                             int64
	MemoryBudgetStopSource                        string // First budget guard: arena, scratch, runtime_heap, runtime_sys, or hard_ceiling.
	RuntimeHeapGrowthBytes                        uint64
	RuntimeSysGrowthBytes                         uint64
	Iterations                                    int
	NodesAllocated                                int
	ArenaBytesAllocated                           int64
	ArenaBaselineBytes                            int64
	ScratchBytesAllocated                         int64
	ScratchBaselineBytes                          int64
	EntryScratchBytesAllocated                    int64
	EntryScratchPeak                              uint64
	GSSBytesAllocated                             int64
	GSSBaselineBytes                              int64
	GSSSlabCount                                  int
	GSSNodesUsed                                  int
	GSSNodesCapacity                              int
	GSSDemotions                                  uint64
	GSSNodesDemoted                               uint64
	TransientScratchCheckpoints                   uint64
	PeakStackDepth                                int
	MaxStacksSeen                                 int
	SingleStackIterations                         int
	MultiStackIterations                          int
	SingleStackTokens                             uint64
	MultiStackTokens                              uint64
	SingleStackGSSNodes                           uint64
	MultiStackGSSNodes                            uint64
	GSSNodesAllocated                             uint64
	GSSNodesRetained                              uint64
	GSSNodesDroppedSameToken                      uint64
	ParentNodesAllocated                          uint64
	ParentNodesRetained                           uint64
	ParentNodesDroppedSameToken                   uint64
	LeafNodesAllocated                            uint64
	LeafNodesRetained                             uint64
	LeafNodesDroppedSameToken                     uint64
	ChildSlicesAllocated                          uint64
	ChildSlicesRetained                           uint64
	ChildSlicesDroppedSameToken                   uint64
	ChildPointersAllocated                        uint64
	ChildPointersRetained                         uint64
	ChildPointersDroppedSameToken                 uint64
	ReduceChildFastGSS                            ReduceChildPathRuntime
	ReduceChildAllVisible                         ReduceChildPathRuntime
	ReduceChildScratchGeneral                     ReduceChildPathRuntime
	ReduceChildScratchNoAlias                     ReduceChildPathRuntime
	TransientChildSlicesAllocated                 uint64
	TransientChildPointersAllocated               uint64
	TransientChildSlicesMaterialized              uint64
	TransientChildPointersMaterialized            uint64
	TransientParentNodesAllocated                 uint64
	TransientParentNodesMaterialized              uint64
	FinalNodes                                    uint64
	FinalParentNodes                              uint64
	FinalLeafNodes                                uint64
	FinalFieldedParentNodes                       uint64
	FinalUnfieldedParentNodes                     uint64
	FinalVisibleParentNodes                       uint64
	FinalHiddenParentNodes                        uint64
	FinalCheckpointLeafNodes                      uint64
	FinalChildSlices                              uint64
	FinalChildPointers                            uint64
	FinalFieldIDElements                          uint64
	FinalFieldSourceElements                      uint64
	FinalChildRefParents                          uint64
	FinalChildRefs                                uint64
	FinalChildRefMaterializedParents              uint64
	FinalChildRefMaterializedChildren             uint64
	FinalChildRefSingleChildAccesses              uint64
	FinalChildRefSingleChildMaterializedChildren  uint64
	MergeStacksIn                                 uint64
	MergeStacksOut                                uint64
	MergeSlotsUsed                                uint64
	GlobalCullStacksIn                            uint64
	GlobalCullStacksOut                           uint64
	StackEquivCalls                               uint64
	StackEquivTrue                                uint64
	StackEquivDepthMismatch                       uint64
	StackEquivHashMismatch                        uint64
	StackEquivStateMismatch                       uint64
	StackEquivPayloadMismatch                     uint64
	StackEquivEntryCompares                       uint64
	StackEquivStateMismatchDepthSum               uint64
	StackEquivStateMismatchMaxDepth               uint32
	StackEquivStateMismatchDepthBuckets           [stackEquivMismatchDepthBucketCount]uint64
	StackEquivPayloadMismatchDepthSum             uint64
	StackEquivPayloadMismatchMaxDepth             uint32
	StackEquivPayloadMismatchDepthBuckets         [stackEquivMismatchDepthBucketCount]uint64
	StackEquivPayloadHeaderSigDiff                uint64
	StackEquivPayloadHeaderSigSame                uint64
	StackEquivPayloadShallowSigDiff               uint64
	StackEquivPayloadShallowSigSame               uint64
	StackEquivPairKeyed                           uint64
	StackEquivPairUnkeyed                         uint64
	StackEquivPairRepeats                         uint64
	StackEquivPairRepeatTrue                      uint64
	StackEquivPairRepeatFalse                     uint64
	StackEquivPairRepeatMismatch                  uint64
	StackEquivPairStores                          uint64
	MergeHeaderEqTotal                            uint64
	MergeDeepTrue                                 uint64
	MergeDeepFalse                                uint64
	MergeHeaderDeepDivergent                      uint64
	EquivCacheLookups                             uint64
	EquivCacheHits                                uint64
	EquivCacheStores                              uint64
	EquivCacheMisses                              uint64
	EquivCacheTrueHits                            uint64
	EquivCacheFalseHits                           uint64
	EquivCacheEpochMisses                         uint64
	EquivCacheKeyMisses                           uint64
	EquivCacheVersionMisses                       uint64
	EquivSkipError                                uint64
	EquivSkipLeaf                                 uint64
	EquivSkipFieldMismatch                        uint64
	EquivExactCalls                               uint64
	EquivExactTrue                                uint64
	EquivExactPointerTrue                         uint64
	EquivExactNilMismatch                         uint64
	EquivExactHeaderMismatch                      uint64
	EquivExactChildMismatch                       uint64
	EquivExactTerminalCalls                       uint64
	EquivExactTerminalTrue                        uint64
	EquivExactTerminalFalse                       uint64
	EquivFrontierCalls                            uint64
	EquivFrontierTrue                             uint64
	EquivExactChildCompares                       uint64
	EquivFrontierChildScans                       uint64
	EquivFrontierCandidateCompares                uint64
	EquivStateStats                               []ParseEquivStateRuntime
	ParseWallNanos                                int64
	ParserLoopNanos                               int64
	TokenNextNanos                                int64
	ActionDispatchNanos                           int64
	ActionLookupNanos                             int64
	GLRMergeNanos                                 int64
	GLRCullNanos                                  int64
	ReduceTiming                                  *ParseReduceTiming
	ActionTiming                                  *ParseActionTiming

	ExternalScannerCheckpointRecords                 uint64
	ExternalScannerCheckpointSlotsAllocated          uint64
	ExternalScannerCheckpointBytesAllocated          int64
	ExternalScannerSnapshotBytesAllocated            uint64
	ExternalScannerCheckpointLeafNodes               uint64
	CompactFullLeafCreated                           uint64
	CompactFullLeafMaterialized                      uint64
	CompactFullLeafMaterializedForParentReduce       uint64
	CompactFullLeafMaterializedForParentReject       PendingParentRejectStats
	CompactFullLeafMaterializedForFinalTree          uint64
	CompactFullLeafMaterializedForNormalization      uint64
	CompactFullLeafMaterializedForRecovery           uint64
	CompactFullLeafMaterializedForQuery              uint64
	CompactFullLeafMaterializedForCursor             uint64
	CompactFullLeafMaterializedForParentAPI          uint64
	CompactFullLeafMaterializedForEdit               uint64
	CompactFullLeafMaterializedForCheckpointRebuild  uint64
	CompactFullLeafDropped                           uint64
	CompactFullLeafMaterializedForFieldRejectPayload PendingParentFieldRejectPayloadStats
	PendingParentCreated                             uint64
	PendingParentMaterialized                        uint64
	PendingParentMaterializedForParentReduce         uint64
	PendingParentMaterializedForParentReject         PendingParentRejectStats
	PendingParentMaterializedForFieldReject          PendingParentFieldRejectStats
	PendingParentMaterializedForFieldRejectPayload   PendingParentFieldRejectPayloadStats
	PendingParentMaterializedForFinalTree            uint64
	PendingParentMaterializedForNormalization        uint64
	PendingParentMaterializedForRecovery             uint64
	PendingParentMaterializedForQuery                uint64
	PendingParentMaterializedForCursor               uint64
	PendingParentMaterializedForParentAPI            uint64
	PendingParentMaterializedForEdit                 uint64
	PendingParentMaterializedForCheckpointRebuild    uint64
	PendingParentDropped                             uint64
	PendingParentsFlattened                          uint64
	PendingChildRefsFlattened                        uint64
	PendingChildEntriesAllocated                     uint64
	PendingChildEntryCapacity                        uint64
	PendingChildEntryWaste                           uint64
	PendingParentCandidates                          uint64
	PendingParentRejectedEmpty                       uint64
	PendingParentRejectedChildLimit                  uint64
	PendingParentRejectedAlias                       uint64
	PendingParentRejectedRawSpan                     uint64
	PendingParentRejectedFields                      uint64
	PendingParentRejectedFieldsParentHidden          uint64
	PendingParentRejectedFieldsNoIDs                 uint64
	PendingParentRejectedFieldsInherited             uint64
	PendingParentRejectedFieldsHiddenChild           uint64
	PendingParentRejectedFieldsHiddenChildPlain      uint64
	PendingParentRejectedFieldsHiddenChildPlainEmpty uint64
	PendingParentRejectedFieldsHiddenChildPlainOne   uint64
	PendingParentRejectedFieldsHiddenChildPlainMany  uint64
	PendingParentRejectedFieldsHiddenChildWithFields uint64
	PendingParentRejectedFieldsChild                 uint64
	PendingParentRejectedFieldsAllVisibleDirect      uint64
	PendingParentRejectedChild                       uint64
	PendingParentRejectedSpan                        uint64
	PendingParentRejectedFill                        uint64
	PreMaterializationFieldRejectCandidates          uint64
	PreMaterializationFieldRejectSameKeyCandidates   uint64
	PreMaterializationFieldRejectOverflowCandidates  uint64

	CheckpointLeafFullNodesAvoided      uint64
	LeafNodesConstructed                uint64
	ParentNodesConstructed              uint64
	NoTreeReduceNodesConstructed        uint64
	NoTreeLeafNodesConstructed          uint64
	ResultSelectionNanos                int64
	TransientParentMaterializationNanos int64
	ResultTreeBuildNanos                int64
	TransientChildMaterializationNanos  int64
	ResultPythonKeywordRepairNanos      int64
	ResultPythonRootRepairNanos         int64
	ResultFinalizeRootNanos             int64
	ResultExtendTrailingNanos           int64
	ResultNormalizeRootStartNanos       int64
	ResultCompatibilityNanos            int64
	ResultParentLinkNanos               int64
	NormalizationPassesChecked          uint64
	NormalizationPassesRun              uint64
	NormalizationNodesVisited           uint64
	NormalizationNodesRewritten         uint64
	NormalizationNanos                  int64
	NormalizationPasses                 *[]NormalizationPassRuntime
	// RecoveryProbeInitialAttempts counts initial-only nested recovery probes.
	// Only scoped clean full-source recovery normalizers use these probes.
	RecoveryProbeInitialAttempts uint64
	// RecoveryProbeInitialAccepted counts probes that met the caller's exact
	// clean and full-span acceptance rule.
	RecoveryProbeInitialAccepted uint64
	// RecoveryProbeLegacyFallbacks counts probes that ran the legacy retry path.
	RecoveryProbeLegacyFallbacks uint64
	// RecoveryProbeInitialRetryPasses is zero when the initial-only contract
	// holds. It records violations for diagnostics.
	RecoveryProbeInitialRetryPasses uint64
	// RecoveryProbeLegacyRetryPasses counts retry-ladder passes after a probe
	// declined its initial result.
	RecoveryProbeLegacyRetryPasses uint64
	// SwiftLegacyRecoverySubparseAttempts counts legacy Swift recovery parses.
	// It excludes the initial-only probe and its measured fallback route.
	SwiftLegacyRecoverySubparseAttempts uint64
	// SwiftLegacyRecoveryRetryPasses counts retry-ladder passes in those Swift
	// legacy recovery parses.
	SwiftLegacyRecoveryRetryPasses uint64
	// The parser sets NativeRecoveredStructureAuthoritative when an exact
	// grammar profile certifies the recovered tree before compatibility.
	NativeRecoveredStructureAuthoritative bool
	// TransientScratchBytesAllocated is the capacity of the transient parent
	// and child slabs this parse held, including slabs inherited from the
	// pool. The scratch lifetime isolation bound applies to this value.
	TransientScratchBytesAllocated int64
	// CRecoverEOFFallbacks counts recovery calls that use the prior exact
	// EOF trial when Go paths cannot be counted as C physical versions.
	CRecoverEOFFallbacks uint64
}

ParseRuntime captures parser-loop diagnostics for a completed tree.

func (ParseRuntime) Summary ¶ added in v0.6.0

func (rt ParseRuntime) Summary() string

Summary returns a stable one-line diagnostic string for parse-runtime stats.

type ParseStopReason ¶ added in v0.6.0

type ParseStopReason string

ParseStopReason reports why parseInternal terminated.

const (
	ParseStopNone            ParseStopReason = "none"
	ParseStopAccepted        ParseStopReason = "accepted"
	ParseStopNoStacksAlive   ParseStopReason = "no_stacks_alive"
	ParseStopTokenSourceEOF  ParseStopReason = "token_source_eof"
	ParseStopTimeout         ParseStopReason = "timeout"
	ParseStopCancelled       ParseStopReason = "cancelled"
	ParseStopIterationLimit  ParseStopReason = "iteration_limit"
	ParseStopStackDepthLimit ParseStopReason = "stack_depth_limit"
	ParseStopNodeLimit       ParseStopReason = "node_limit"
	ParseStopMemoryBudget    ParseStopReason = "memory_budget"
	// ParseStopReuseBudget stops an old-tree reuse parse that built many
	// times the old tree's nodes while reusing almost nothing. The caller
	// runs one plain full parse instead.
	ParseStopReuseBudget        ParseStopReason = "reuse_budget"
	ParseStopInvariantViolation ParseStopReason = "invariant_violation"
)

type ParseStoppedEarlyError ¶ added in v0.20.6

type ParseStoppedEarlyError struct {
	Reason  ParseStopReason
	Runtime ParseRuntime
}

ParseStoppedEarlyError reports a parse that returned a tree but stopped before accepting the input. The returned tree is still available to callers that want diagnostics or partial output.

func (*ParseStoppedEarlyError) Error ¶ added in v0.20.6

func (e *ParseStoppedEarlyError) Error() string

func (*ParseStoppedEarlyError) Is ¶ added in v0.20.6

func (e *ParseStoppedEarlyError) Is(target error) bool

type ParseWorkLimits ¶ added in v0.53.0

type ParseWorkLimits struct {
	// IterationLimit bounds the number of production parser iterations.
	IterationLimit int
	// StackDepthLimit bounds the primary stack depth at parser checkpoints.
	StackDepthLimit int
	// NodeLimit bounds nodes counted by the production parser at checkpoints.
	NodeLimit int
}

ParseWorkLimits overrides deterministic production parser-loop limits for one Parser. Stable parse methods use the production parser when any field is positive, so speculative compact and forest work cannot precede these caps. A zero or negative field keeps the source-derived default for that field. Positive values replace the thresholds reported in ParseRuntime. Each production parser loop uses these thresholds, including recovery parses. They do not count total operation work or allocated bytes. Node and depth checks occur between steps, so a step can exceed a threshold.

These limits count parser work and do not depend on wall-clock time. A timeout, cancellation request, memory budget, or invariant failure can still stop a parse before a configured work limit.

type Parser ¶

type Parser struct {
	// contains filtered or unexported fields
}

Parser reads parse tables from a Language and produces a syntax tree. It supports GLR parsing: when a (state, symbol) pair maps to multiple actions, the parser forks the stack and explores all alternatives in parallel while preserving distinct parse paths. Duplicate stack versions are collapsed and ambiguities are resolved at selection time.

Parser is not safe for concurrent use. Use one parser per goroutine, a ParserPool, or guard shared parser instances with external synchronization.

func NewParser ¶

func NewParser(lang *Language) *Parser

NewParser creates a new Parser for the given language.

func (*Parser) CancellationFlag ¶ added in v0.7.0

func (p *Parser) CancellationFlag() *uint32

CancellationFlag returns the parser's current cancellation flag pointer.

func (*Parser) ClearAdmissionCandidateRoute ¶ added in v0.46.0

func (p *Parser) ClearAdmissionCandidateRoute()

ClearAdmissionCandidateRoute drops the per-Parser override so the Parser follows the process-wide default again.

func (*Parser) DebugCNodeMemoCacheStats ¶ added in v0.44.0

func (p *Parser) DebugCNodeMemoCacheStats() (cacheLen int, thrash uint32)

DebugCNodeMemoCacheStats reports the active cache size and the current tier's collision count. Tree.RecoveryNodeMemoRuntime reports the peak tier and the total collision count for a returned tree. This method is safe on a nil receiver.

func (*Parser) DebugCNodeMemoOperationStats ¶ added in v0.49.0

func (p *Parser) DebugCNodeMemoOperationStats() (peakEntries int, peakBytes uint64, collisions uint64)

DebugCNodeMemoOperationStats reports the peak cache entries, peak bytes, and collisions from the most recent recovery-capable outer parse operation. A compact-route return does not start such an operation.

func (*Parser) DebugRecoveryRuntimeAttempts ¶ added in v0.52.0

func (p *Parser) DebugRecoveryRuntimeAttempts() RecoveryRuntimeAttempts

DebugRecoveryRuntimeAttempts returns no attempt receipt in production builds.

func (*Parser) DebugRecoveryRuntimeStats ¶ added in v0.50.0

func (p *Parser) DebugRecoveryRuntimeStats() RecoveryRuntimeStats

DebugRecoveryRuntimeStats returns the latest recovery facts for parser p. The method returns zero values when telemetry is disabled or no parse ran.

func (*Parser) DiagnosticEnableDropCohortCertificateAdmissionForTest ¶ added in v0.52.0

func (p *Parser) DiagnosticEnableDropCohortCertificateAdmissionForTest() func()

DiagnosticEnableDropCohortCertificateAdmissionForTest enables one cached candidate parse for focused certificate tests. Production code never calls this method, and the returned closure restores the runner before reuse.

func (*Parser) ForestCapTieStats ¶ added in v0.49.0

func (p *Parser) ForestCapTieStats() ForestCapTieStats

ForestCapTieStats returns Stage 0's cap-event counts recorded during the most recent forest parse on this Parser. The returned Receipts slice is a copy: the next parse on this same *Parser reuses forestCapTieStats. Receipts' backing array (resetForestCapTieStats truncates it with [:0] rather than reallocating), so a caller that held the previous slice without copying would see its own already-returned receipts silently rewritten by the next parse's recordForestCapTie appends.

func (*Parser) ForestDeclineInfo ¶ added in v0.20.6

func (p *Parser) ForestDeclineInfo() (offset uint32, sym Symbol, reason string, states []StateID)

ForestDeclineInfo returns where/why the forest fast path last declined: the byte offset and lookahead symbol at the decline, a short reason code, and (for reason "dead_end") the surviving GLR states. The normal Parse path may then fall back to production; ParseForestExperimental does not. This drives language-burndown triage without re-instrumenting. Valid after a ParseForestExperimental that returned ok=false.

func (*Parser) IncludedRanges ¶ added in v0.6.0

func (p *Parser) IncludedRanges() []Range

IncludedRanges returns a copy of the configured include ranges.

func (*Parser) InferredRootSymbol ¶ added in v0.9.0

func (p *Parser) InferredRootSymbol() (Symbol, bool)

InferredRootSymbol returns the root symbol inferred during parser construction, and whether inference succeeded.

func (*Parser) Language ¶ added in v0.7.0

func (p *Parser) Language() *Language

Language returns the parser's configured language.

func (*Parser) Logger ¶ added in v0.7.0

func (p *Parser) Logger() ParserLogger

Logger returns the currently configured parser debug logger.

func (*Parser) MemoryBudgetBytes ¶ added in v0.53.0

func (p *Parser) MemoryBudgetBytes() int64

MemoryBudgetBytes returns the value set by SetMemoryBudgetBytes. Zero means the default budget. A negative value means the per-parse budget is off.

func (*Parser) Parse ¶

func (p *Parser) Parse(source []byte) (*Tree, error)

Parse tokenizes and parses source using the built-in DFA lexer, returning a syntax tree. This works for hand-built grammars that provide LexStates. For real grammars that need a custom lexer, use ParseWithTokenSource. If the input is empty, the returned tree's root depends on the grammar: some grammars return a nil root, others return a non-nil, zero-width root (for example, JSON returns a zero-width document node for empty input). Check Tree.RootNode() for nil before use; do not assume either shape.

func (*Parser) ParseForestExperimental ¶ added in v0.20.0

func (p *Parser) ParseForestExperimental(source []byte) (*Tree, bool)

ParseForestExperimental parses source with the experimental GSS-forest GLR path and returns a releasable forest-produced tree. It returns nil,false when the forest declines for any reason; unlike Parse, this diagnostic entry point never hides a decline by running the production parser. Exported so out-of-tree benchmarks and validation in packages that attach external scanners (e.g. grammars) can drive it; not part of the stable API.

func (*Parser) ParseIncremental ¶

func (p *Parser) ParseIncremental(source []byte, oldTree *Tree) (*Tree, error)

ParseIncremental re-parses source after edits were applied to oldTree. It reuses unchanged subtrees from the old tree for better performance. Call oldTree.Edit() for each edit before calling this method.

A caller that skips Tree.Edit is only safe when source is unchanged or stays the same length as oldTree's own source. Passing a different-length source with no recorded edit falls back to an ordinary fresh parse instead of an error, matching every other case where this method decides oldTree cannot be trusted for reuse (a language or included-ranges mismatch, for example); it does not attempt to guess which edit was skipped.

Release the returned tree and oldTree once each. When source and oldTree are unchanged, the method returns oldTree itself and adds a handle to it, so releasing oldTree does not invalidate the result.

The new tree can share nodes with oldTree. The call updates the parent links of shared nodes, so do not read oldTree from another goroutine during the call. After the call, read the returned tree instead of oldTree.

func (*Parser) ParseIncrementalProfiled ¶ added in v0.6.0

func (p *Parser) ParseIncrementalProfiled(source []byte, oldTree *Tree) (*Tree, IncrementalParseProfile, error)

ParseIncrementalProfiled is like ParseIncremental and also returns runtime attribution for incremental reuse work vs parse/rebuild work.

func (*Parser) ParseIncrementalStrict ¶ added in v0.20.6

func (p *Parser) ParseIncrementalStrict(source []byte, oldTree *Tree) (*Tree, error)

ParseIncrementalStrict is like ParseIncremental, but returns ErrParseStoppedEarly when parsing returns a partial tree.

func (*Parser) ParseIncrementalUTF16 ¶ added in v0.16.0

func (p *Parser) ParseIncrementalUTF16(source []uint16, oldTree *Tree) (*Tree, error)

ParseIncrementalUTF16 re-parses UTF-16 source after edits were applied to oldTree. oldTree should have been produced by ParseUTF16, and UTF-16 edits can be recorded with Tree.EditUTF16.

func (*Parser) ParseIncrementalUTF16Bytes ¶ added in v0.16.0

func (p *Parser) ParseIncrementalUTF16Bytes(source []byte, oldTree *Tree, order UTF16ByteOrder) (*Tree, error)

ParseIncrementalUTF16Bytes re-parses UTF-16 bytes after edits were applied to oldTree.

func (*Parser) ParseIncrementalUTF16BytesWithTokenSourceFactory ¶ added in v0.16.0

func (p *Parser) ParseIncrementalUTF16BytesWithTokenSourceFactory(source []byte, oldTree *Tree, order UTF16ByteOrder, factory TokenSourceFactory) (*Tree, error)

ParseIncrementalUTF16BytesWithTokenSourceFactory re-parses UTF-16 bytes using a token source built from the parser's canonical UTF-8 source view.

func (*Parser) ParseIncrementalUTF16WithTokenSourceFactory ¶ added in v0.16.0

func (p *Parser) ParseIncrementalUTF16WithTokenSourceFactory(source []uint16, oldTree *Tree, factory TokenSourceFactory) (*Tree, error)

ParseIncrementalUTF16WithTokenSourceFactory re-parses UTF-16 source using a token source built from the parser's canonical UTF-8 source view.

func (*Parser) ParseIncrementalWithTokenSource ¶

func (p *Parser) ParseIncrementalWithTokenSource(source []byte, oldTree *Tree, ts TokenSource) (*Tree, error)

ParseIncrementalWithTokenSource is like ParseIncremental but uses a custom token source.

func (*Parser) ParseIncrementalWithTokenSourceFactory ¶ added in v0.16.0

func (p *Parser) ParseIncrementalWithTokenSourceFactory(source []byte, oldTree *Tree, factory TokenSourceFactory) (*Tree, error)

ParseIncrementalWithTokenSourceFactory is like ParseWithTokenSourceFactory for an edited old tree.

func (*Parser) ParseIncrementalWithTokenSourceFactoryStrict ¶ added in v0.20.6

func (p *Parser) ParseIncrementalWithTokenSourceFactoryStrict(source []byte, oldTree *Tree, factory TokenSourceFactory) (*Tree, error)

ParseIncrementalWithTokenSourceFactoryStrict is like ParseIncrementalWithTokenSourceFactory, but returns ErrParseStoppedEarly when parsing returns a partial tree.

func (*Parser) ParseIncrementalWithTokenSourceProfiled ¶ added in v0.6.0

func (p *Parser) ParseIncrementalWithTokenSourceProfiled(source []byte, oldTree *Tree, ts TokenSource) (*Tree, IncrementalParseProfile, error)

ParseIncrementalWithTokenSourceProfiled is like ParseIncrementalWithTokenSource and also returns runtime attribution for incremental reuse work vs parse/rebuild work.

func (*Parser) ParseIncrementalWithTokenSourceStrict ¶ added in v0.20.6

func (p *Parser) ParseIncrementalWithTokenSourceStrict(source []byte, oldTree *Tree, ts TokenSource) (*Tree, error)

ParseIncrementalWithTokenSourceStrict is like ParseIncrementalWithTokenSource, but returns ErrParseStoppedEarly when parsing returns a partial tree.

func (*Parser) ParseNoResultCompatibilityBenchmarkOnly ¶ added in v0.18.0

func (p *Parser) ParseNoResultCompatibilityBenchmarkOnly(source []byte) (*Tree, error)

ParseNoResultCompatibilityBenchmarkOnly parses source while suppressing language-specific result compatibility rewrites while preserving result-tree materialization. Other diagnostic materialization strategies may still key off this mode, so it is not a pure compatibility-only A/B. It is intended only for performance attribution; the returned tree is not API-compatible.

func (*Parser) ParseNoTreeBenchmarkOnly ¶ added in v0.17.0

func (p *Parser) ParseNoTreeBenchmarkOnly(source []byte) (*Tree, error)

ParseNoTreeBenchmarkOnly parses source while suppressing parent/child tree materialization in reduce actions. It is intended only for parser-loop performance experiments; the returned tree is not API-compatible.

func (*Parser) ParseNoTreeWithExternalCheckpointsBenchmarkOnly ¶ added in v0.18.0

func (p *Parser) ParseNoTreeWithExternalCheckpointsBenchmarkOnly(source []byte) (*Tree, error)

ParseNoTreeWithExternalCheckpointsBenchmarkOnly parses source while suppressing parent/child tree materialization in reduce actions but keeping external-scanner checkpoint capture enabled. It is intended only for parser performance attribution; the returned tree is not API-compatible.

func (*Parser) ParseStrict ¶ added in v0.20.6

func (p *Parser) ParseStrict(source []byte) (*Tree, error)

ParseStrict is like Parse, but returns ErrParseStoppedEarly when parsing returns a partial tree due to timeout, cancellation, token-source EOF, or a parser safety limit. The partial tree is returned alongside the error.

func (*Parser) ParseUTF16 ¶ added in v0.16.0

func (p *Parser) ParseUTF16(source []uint16) (*Tree, error)

ParseUTF16 parses UTF-16 source represented as Go UTF-16 code units.

The parser core uses a canonical UTF-8 view internally so existing byte-based APIs remain unchanged. The returned tree retains the original UTF-16 source and can convert node ranges back to UTF-16 code-unit coordinates.

func (*Parser) ParseUTF16Bytes ¶ added in v0.16.0

func (p *Parser) ParseUTF16Bytes(source []byte, order UTF16ByteOrder) (*Tree, error)

ParseUTF16Bytes parses UTF-16 source encoded as bytes with an explicit byte order.

func (*Parser) ParseUTF16BytesWithTokenSourceFactory ¶ added in v0.16.0

func (p *Parser) ParseUTF16BytesWithTokenSourceFactory(source []byte, order UTF16ByteOrder, factory TokenSourceFactory) (*Tree, error)

ParseUTF16BytesWithTokenSourceFactory parses UTF-16 bytes using a token source built from the parser's canonical UTF-8 source view.

func (*Parser) ParseUTF16WithTokenSourceFactory ¶ added in v0.16.0

func (p *Parser) ParseUTF16WithTokenSourceFactory(source []uint16, factory TokenSourceFactory) (*Tree, error)

ParseUTF16WithTokenSourceFactory parses UTF-16 source using a token source built from the parser's canonical UTF-8 source view.

func (*Parser) ParseWith ¶ added in v0.6.0

func (p *Parser) ParseWith(source []byte, opts ...ParseOption) (ParseResult, error)

ParseWith parses source using option-based configuration.

func (*Parser) ParseWithStrict ¶ added in v0.20.6

func (p *Parser) ParseWithStrict(source []byte, opts ...ParseOption) (ParseResult, error)

ParseWithStrict is like ParseWith, but returns ErrParseStoppedEarly when parsing returns a partial tree. The ParseResult still carries that tree.

func (*Parser) ParseWithTokenSource ¶

func (p *Parser) ParseWithTokenSource(source []byte, ts TokenSource) (*Tree, error)

ParseWithTokenSource parses source using a custom token source. This is used for real grammars where the lexer DFA isn't available as data tables (e.g., Go grammar using go/scanner as a bridge).

func (*Parser) ParseWithTokenSourceFactory ¶ added in v0.16.0

func (p *Parser) ParseWithTokenSourceFactory(source []byte, factory TokenSourceFactory) (*Tree, error)

ParseWithTokenSourceFactory parses source using a freshly built custom token source. The factory is also retained for recovery reparses.

func (*Parser) ParseWithTokenSourceFactoryStrict ¶ added in v0.20.6

func (p *Parser) ParseWithTokenSourceFactoryStrict(source []byte, factory TokenSourceFactory) (*Tree, error)

ParseWithTokenSourceFactoryStrict is like ParseWithTokenSourceFactory, but returns ErrParseStoppedEarly when parsing returns a partial tree.

func (*Parser) ParseWithTokenSourceStrict ¶ added in v0.20.6

func (p *Parser) ParseWithTokenSourceStrict(source []byte, ts TokenSource) (*Tree, error)

ParseWithTokenSourceStrict is like ParseWithTokenSource, but returns ErrParseStoppedEarly when parsing returns a partial tree.

func (*Parser) ParseWorkLimits ¶ added in v0.53.0

func (p *Parser) ParseWorkLimits() ParseWorkLimits

ParseWorkLimits returns the parser's configured deterministic work limits. Zero fields use source-derived defaults.

func (*Parser) SetAdmissionCandidateRoute ¶ added in v0.46.0

func (p *Parser) SetAdmissionCandidateRoute(enabled bool)

SetAdmissionCandidateRoute sets a per-Parser override that takes precedence over the process-wide default. enabled=true forces the candidate route on for eligible full parses; enabled=false forces the production route.

enabled=true still respects every remaining eligibility decline: included ranges and observability hooks keep a parse on production. Source length no longer declines eligibility (tranche B9); a large input attempts the candidate route and either completes there or trips the scheduler's stop-control poll (tranche B8) and falls back to production with a compatible stop receipt honoring ParseStopMemoryBudget.

func (*Parser) SetAmbiguityProfile ¶ added in v0.17.0

func (p *Parser) SetAmbiguityProfile(profile *AmbiguityProfile)

SetAmbiguityProfile installs an optional diagnostic ambiguity profile. The profile receives parser state/lookahead/action counters for GLR-heavy benchmark runs. Pass nil to disable profiling.

func (*Parser) SetCancellationFlag ¶ added in v0.7.0

func (p *Parser) SetCancellationFlag(flag *uint32)

SetCancellationFlag configures a caller-owned cancellation flag. Parsing stops when the pointed value becomes non-zero.

func (*Parser) SetCompactCertificationTelemetry ¶ added in v0.55.0

func (p *Parser) SetCompactCertificationTelemetry(enabled bool)

SetCompactCertificationTelemetry records compact frontier and derivation peaks in ParseRuntime. Enable it only for certification runs.

func (*Parser) SetGLRTrace ¶ added in v0.7.0

func (p *Parser) SetGLRTrace(enabled bool)

SetGLRTrace enables verbose GLR stack tracing to stdout (debug only).

func (*Parser) SetIncludedRanges ¶ added in v0.6.0

func (p *Parser) SetIncludedRanges(ranges []Range)

SetIncludedRanges configures parser include ranges. Tokens outside these ranges are skipped.

func (*Parser) SetIncludedUTF16ByteRanges ¶ added in v0.16.0

func (p *Parser) SetIncludedUTF16ByteRanges(source []byte, order UTF16ByteOrder, ranges []UTF16Range) error

SetIncludedUTF16ByteRanges configures parser include ranges from endian-specific UTF-16 bytes.

func (*Parser) SetIncludedUTF16Ranges ¶ added in v0.16.0

func (p *Parser) SetIncludedUTF16Ranges(source []uint16, ranges []UTF16Range) bool

SetIncludedUTF16Ranges configures parser include ranges from UTF-16 code-unit ranges. Internal parser points are derived from source as UTF-8 columns.

func (*Parser) SetLogger ¶ added in v0.7.0

func (p *Parser) SetLogger(logger ParserLogger)

SetLogger installs a parser debug logger. Pass nil to disable logging.

func (*Parser) SetMemoryBudgetBytes ¶ added in v0.53.0

func (p *Parser) SetMemoryBudgetBytes(bytes int64)

SetMemoryBudgetBytes sets a fixed per-parse memory budget for later parse calls. The budget bounds the node arena and parser scratch memory of one parse.

The default budget grows with the input. It is the larger of 512 MiB and 512 bytes for each input byte, so a valid input does not need a larger budget. Use this method to cap parser memory, for example in a service with a memory limit.

  • A positive value sets a fixed budget.
  • Zero restores the default budget.
  • A negative value turns the per-parse budget off.

The process-heap ceiling still stops a runaway parse. It is the larger of 2 GiB and twice the budget. GOT_PARSE_MEMORY_HARD_CEILING_MB sets a fixed ceiling instead.

A parse stopped by the budget returns a partial tree and nil error. Its ParseStopReason is ParseStopMemoryBudget. Use a strict parse method to receive a ParseStoppedEarlyError instead.

func (*Parser) SetParseWorkLimits ¶ added in v0.53.0

func (p *Parser) SetParseWorkLimits(limits ParseWorkLimits)

SetParseWorkLimits sets deterministic limits for later parse calls. Use a zero-value ParseWorkLimits to restore all source-derived defaults.

A parse stopped by a configured limit returns a partial tree and nil error. Inspect Tree.ParseStopReason, or use a strict parse method to receive a ParseStoppedEarlyError.

func (*Parser) SetTimeoutMicros ¶ added in v0.7.0

func (p *Parser) SetTimeoutMicros(timeoutMicros uint64)

SetTimeoutMicros configures a per-parse timeout in microseconds. A value of 0 disables timeout checks. Parse methods preserve tree-sitter's partial-tree behavior on timeout: they return a tree and nil error, with tree.ParseStopReason() == ParseStopTimeout and tree.ParseStoppedEarly() true. Use ParseStrict or another strict parse method to treat early stops as errors.

func (*Parser) TimeoutMicros ¶ added in v0.7.0

func (p *Parser) TimeoutMicros() uint64

TimeoutMicros returns the parser timeout in microseconds.

type ParserCoreCorridorCensus ¶ added in v0.49.0

type ParserCoreCorridorCensus struct {
	Shift            uint64
	ShiftExtra       uint64
	Reduce           uint64
	ReduceChain      uint64
	ReduceShift      uint64
	Accept           uint64
	Fork             uint64
	ExitGeneric      uint64
	ExitUnsupported  uint64
	PopulatedCells   uint64
	States           uint64
	BlockWords       uint64
	BodyWords        uint64
	InternedBodies   uint64
	DenseBlocks      uint64
	SparseBlocks     uint64
	ExternalStates   uint64
	ReservedWordSets uint64
}

ParserCoreCorridorCensus counts compiled dispositions. Reduce includes the fused REDUCE_CHAIN and REDUCE_SHIFT subtypes; the subtype fields report them separately.

type ParserCoreCorridorDecodedCell ¶ added in v0.49.0

type ParserCoreCorridorDecodedCell struct {
	State         StateID
	Symbol        Symbol
	Opcode        string
	TargetState   StateID
	SelfLoop      bool
	ProductionID  uint16
	LHS           Symbol
	ChildCount    uint8
	GotoMode      string
	RowIndex      uint32
	ChainRowIndex uint32
	ShiftRowIndex uint32
	Reason        string
}

ParserCoreCorridorDecodedCell is the decode-back form of one compiled cell (spec section 5, S2). It names the disposition and carries enough operand detail to reconstruct the constituent table cell.

type ParserCoreCorridorProgram ¶ added in v0.49.0

type ParserCoreCorridorProgram struct {
	// contains filtered or unexported fields
}

ParserCoreCorridorProgram is one grammar's compiled corridor program: a single validated []uint32 instruction stream plus the entry index.

Threading (spec section 3.6): every SHIFT target is a word offset into the stream, so corridor control flow never consults a state-to-offset index. The index exists only for corridor entry and for re-entry after a boundary opcode.

func CompileParserCoreCorridorProgram ¶ added in v0.49.0

func CompileParserCoreCorridorProgram(lang *Language) (*ParserCoreCorridorProgram, error)

CompileParserCoreCorridorProgram is the one compiler (spec section 5, S1). grammargen's assemble step and the runtime load path both feed the same decoded table set into this function; there is no second implementation.

func (*ParserCoreCorridorProgram) Bytes ¶ added in v0.49.0

func (p *ParserCoreCorridorProgram) Bytes() int

Bytes reports the retained stream footprint.

func (*ParserCoreCorridorProgram) Census ¶ added in v0.49.0

Census reports the compiled disposition counts.

func (*ParserCoreCorridorProgram) DecodeCell ¶ added in v0.49.0

DecodeCell decodes the compiled disposition of one (state, terminal) cell. It reports ok=false when the state has no compiled action for the symbol.

func (*ParserCoreCorridorProgram) DecodeCheckpointObligation ¶ added in v0.49.0

func (p *ParserCoreCorridorProgram) DecodeCheckpointObligation(state StateID) string

DecodeCheckpointObligation reports the provenance marker for a state: the CHECKPOINT form when the state's EXPECT carries EXT, and the plain EXPECT form otherwise (spec section 3.5, CHECKPOINT row).

func (*ParserCoreCorridorProgram) KeySet ¶ added in v0.49.0

func (p *ParserCoreCorridorProgram) KeySet(state StateID) []Symbol

KeySet returns the interned populated-terminal key set for a state.

func (*ParserCoreCorridorProgram) StateCount ¶ added in v0.49.0

func (p *ParserCoreCorridorProgram) StateCount() int

StateCount reports the compiled state count.

func (*ParserCoreCorridorProgram) Words ¶ added in v0.49.0

func (p *ParserCoreCorridorProgram) Words() int

Words reports the compiled stream length in 32-bit words.

type ParserCoreCorridorTableShape ¶ added in v0.49.0

type ParserCoreCorridorTableShape struct {
	Grammar string `json:"grammar"`
	// BlobSHA256 pins the exact grammar blob the row was measured against.
	BlobSHA256  string `json:"blob_sha256"`
	States      int    `json:"states"`
	TokenCount  uint32 `json:"token_count"`
	SymbolCount uint32 `json:"symbol_count"`

	// Populated (state, terminal) cell census.
	PopulatedCells   uint64 `json:"populated_cells"`
	SoleActionCells  uint64 `json:"sole_action_cells"`
	MultiActionCells uint64 `json:"multi_action_cells"`
	// SoleActionCellShare is the spec's "sole-action cell share" row.
	SoleActionCellShare float64 `json:"sole_action_cell_share"`

	// Compiled disposition census (spec section 4.2). Every populated cell
	// falls in exactly one bucket; the exhaustiveness test asserts the sum.
	Shift           uint64 `json:"disposition_shift"`
	ShiftExtra      uint64 `json:"disposition_shift_extra"`
	Reduce          uint64 `json:"disposition_reduce"`
	ReduceChain     uint64 `json:"disposition_reduce_chain,omitempty"`
	ReduceShift     uint64 `json:"disposition_reduce_shift,omitempty"`
	Accept          uint64 `json:"disposition_accept"`
	Fork            uint64 `json:"disposition_fork"`
	ExitGeneric     uint64 `json:"disposition_exit_generic"`
	ExitUnsupported uint64 `json:"disposition_exit_unsupported"`
	// CorridorCellShare is the fraction of populated cells the corridor
	// executes without leaving the lane.
	CorridorCellShare float64 `json:"corridor_cell_share"`

	// "EXPECT key counts" row: populated terminal keys per state.
	KeyCountMin  int     `json:"key_count_min"`
	KeyCountP50  int     `json:"key_count_p50"`
	KeyCountP90  int     `json:"key_count_p90"`
	KeyCountP99  int     `json:"key_count_p99"`
	KeyCountMax  int     `json:"key_count_max"`
	KeyCountMean float64 `json:"key_count_mean"`
	// StatesWithNoKeys counts states whose terminal row is entirely empty.
	StatesWithNoKeys int `json:"states_with_no_keys"`

	// Unary-chain rows. UnaryReduceCells counts sole-reduce cells with
	// ChildCount == 1; UnaryReduceShare is that count over all reduce cells.
	ReduceCells      uint64  `json:"reduce_cells"`
	UnaryReduceCells uint64  `json:"unary_reduce_cells"`
	UnaryReduceShare float64 `json:"unary_reduce_share"`
	// UnaryChainEdgesDepth2 is the spec's "unary-chain edges depth>=2" row: a
	// (predecessor edge, lookahead) pair whose sole unary reduce lands, after
	// goto, on another sole unary reduce for the same lookahead.
	UnaryChainEdges       uint64  `json:"unary_chain_edges"`
	UnaryChainEdgesDepth2 uint64  `json:"unary_chain_edges_depth2"`
	UnaryChainDepth2Share float64 `json:"unary_chain_depth2_share"`
	// EdgeStaticUnaryReduceCells counts sole-reduce cells with ChildCount == 1
	// whose goto target is the same for every predecessor edge. Its share is
	// taken over UnaryReduceCells, so it answers "how often is a unary chain
	// step edge-unique".
	EdgeStaticUnaryReduceCells uint64  `json:"edge_static_unary_reduce_cells"`
	EdgeStaticUnaryReduceShare float64 `json:"edge_static_unary_reduce_share"`
	// EdgeStaticReduceCells is the same edge-uniqueness measurement over EVERY
	// sole-reduce cell, not only the unary ones. corridorGotoModeStatic's
	// precondition is edge-unique goto, which has nothing to do with the
	// reduction's child count, so this is the row that decides whether a fused
	// static goto target is admissible. Its share is taken over ReduceCells.
	EdgeStaticReduceCells uint64  `json:"edge_static_reduce_cells"`
	EdgeStaticReduceShare float64 `json:"edge_static_reduce_share"`

	// UniformSoleReduceStates counts states where EVERY populated terminal key
	// decodes to the same sole reduce action. Extra-shift cells break
	// uniformity under this definition.
	//
	// Read it with care: comment and whitespace tokens shift as extras in
	// essentially every state of a real grammar, so this row is near zero by
	// construction and says almost nothing about reduce uniformity. It is
	// retained only for continuity with the first published receipt.
	UniformSoleReduceStates int     `json:"uniform_sole_reduce_states"`
	UniformSoleReduceShare  float64 `json:"uniform_sole_reduce_share"`
	// UniformSoleReduceStatesExtraNeutral is the same row with extra-shift
	// cells excluded from the uniformity test: a state qualifies when its
	// populated terminal keys are all either an extra shift or one single,
	// identical sole reduce.
	//
	// This is the definition consistent with the corridor's own semantics. An
	// extra shift compiles to SHIFT_EXTRA, which spec section 3.2 says stays
	// inside the same election, so an extra-shift key does not interrupt a
	// reduce run the way an ordinary shift does. This is the row a fusion
	// argument must use.
	UniformSoleReduceStatesExtraNeutral int     `json:"uniform_sole_reduce_states_extra_neutral"`
	UniformSoleReduceShareExtraNeutral  float64 `json:"uniform_sole_reduce_share_extra_neutral"`
	// UniformSoleReduceCoveredCells counts the reduce cells that sit inside an
	// extra-neutral uniform state, and its share is taken over ReduceCells. It
	// is the reduce mass a state-uniform fusion would reach.
	UniformSoleReduceCoveredCells uint64  `json:"uniform_sole_reduce_covered_cells"`
	UniformSoleReduceCoveredShare float64 `json:"uniform_sole_reduce_covered_share"`

	// REDUCE_SHIFT candidate row: a sole-reduce cell whose post-goto row for
	// the same lookahead is a sole shift, for every predecessor edge. It spans
	// every sole-reduce cell, and its share is taken over ReduceCells.
	ReduceShiftCandidateCells uint64  `json:"reduce_shift_candidate_cells"`
	ReduceShiftCandidateShare float64 `json:"reduce_shift_candidate_share"`

	// Footprint rows (spec section 3.6 memory estimate, measured).
	ProgramWords      int     `json:"program_words"`
	ProgramBytes      int     `json:"program_bytes"`
	ProgramBodyWords  uint64  `json:"program_body_words"`
	ProgramBlockWords uint64  `json:"program_block_words"`
	InternedBodies    uint64  `json:"interned_bodies"`
	DenseBlocks       uint64  `json:"dense_blocks"`
	SparseBlocks      uint64  `json:"sparse_blocks"`
	BytesPerState     float64 `json:"bytes_per_state"`

	// EXPECT flag rows.
	ExternalStates   uint64 `json:"expect_ext_states"`
	ReservedWordSets uint64 `json:"expect_kw_states"`
}

ParserCoreCorridorTableShape is one grammar's committed table-shape receipt. Field names are the analyzer rows the spec cites by name.

func AnalyzeParserCoreCorridorTables ¶ added in v0.49.0

func AnalyzeParserCoreCorridorTables(lang *Language) (ParserCoreCorridorTableShape, error)

AnalyzeParserCoreCorridorTables measures one grammar's static table shape and compiles its corridor program, returning the committed receipt row.

type ParserLogType ¶ added in v0.7.0

type ParserLogType uint8

ParserLogType categorizes parser log messages.

const (
	// ParserLogParse emits parser-loop lifecycle and control-flow logs.
	ParserLogParse ParserLogType = iota
	// ParserLogLex emits token-source and token-consumption logs.
	ParserLogLex
)

type ParserLogger ¶ added in v0.7.0

type ParserLogger func(kind ParserLogType, message string)

ParserLogger receives parser debug logs when configured via SetLogger.

type ParserPool ¶ added in v0.7.0

type ParserPool struct {
	// contains filtered or unexported fields
}

ParserPool provides concurrency-safe parsing by reusing Parser instances.

ParserPool is safe for concurrent use. Each call checks out one parser from an internal sync.Pool, applies configured defaults, runs the parse, and returns the parser to the pool.

Mutable parser state (logger, timeout, cancellation flag, included ranges, GLR trace) is reset on checkout so request-local state cannot bleed across callers.

func NewParserPool ¶ added in v0.7.0

func NewParserPool(lang *Language, opts ...ParserPoolOption) *ParserPool

NewParserPool creates a concurrency-safe parser pool for lang.

func (*ParserPool) Language ¶ added in v0.7.0

func (pp *ParserPool) Language() *Language

Language returns the pool's configured language.

func (*ParserPool) Parse ¶ added in v0.7.0

func (pp *ParserPool) Parse(source []byte) (*Tree, error)

Parse delegates to a pooled Parser.Parse call.

func (*ParserPool) ParseIncrementalUTF16 ¶ added in v0.16.0

func (pp *ParserPool) ParseIncrementalUTF16(source []uint16, oldTree *Tree) (*Tree, error)

ParseIncrementalUTF16 delegates to a pooled Parser.ParseIncrementalUTF16 call.

func (*ParserPool) ParseIncrementalUTF16Bytes ¶ added in v0.16.0

func (pp *ParserPool) ParseIncrementalUTF16Bytes(source []byte, oldTree *Tree, order UTF16ByteOrder) (*Tree, error)

ParseIncrementalUTF16Bytes delegates to a pooled Parser.ParseIncrementalUTF16Bytes call.

func (*ParserPool) ParseIncrementalUTF16BytesWithTokenSourceFactory ¶ added in v0.16.0

func (pp *ParserPool) ParseIncrementalUTF16BytesWithTokenSourceFactory(source []byte, oldTree *Tree, order UTF16ByteOrder, factory TokenSourceFactory) (*Tree, error)

ParseIncrementalUTF16BytesWithTokenSourceFactory delegates to a pooled Parser.ParseIncrementalUTF16BytesWithTokenSourceFactory call.

func (*ParserPool) ParseIncrementalUTF16WithTokenSourceFactory ¶ added in v0.16.0

func (pp *ParserPool) ParseIncrementalUTF16WithTokenSourceFactory(source []uint16, oldTree *Tree, factory TokenSourceFactory) (*Tree, error)

ParseIncrementalUTF16WithTokenSourceFactory delegates to a pooled Parser.ParseIncrementalUTF16WithTokenSourceFactory call.

func (*ParserPool) ParseNoResultCompatibilityBenchmarkOnly ¶ added in v0.18.0

func (pp *ParserPool) ParseNoResultCompatibilityBenchmarkOnly(source []byte) (*Tree, error)

ParseNoResultCompatibilityBenchmarkOnly delegates to Parser.ParseNoResultCompatibilityBenchmarkOnly. It is intended only for performance attribution. The result tree is materialized, but other diagnostic materialization strategies may still key off this mode, and the returned tree is not API-compatible.

func (*ParserPool) ParseNoTreeBenchmarkOnly ¶ added in v0.17.0

func (pp *ParserPool) ParseNoTreeBenchmarkOnly(source []byte) (*Tree, error)

ParseNoTreeBenchmarkOnly delegates to Parser.ParseNoTreeBenchmarkOnly. It is intended only for parser-loop performance experiments; the returned tree is not API-compatible.

func (*ParserPool) ParseNoTreeWithExternalCheckpointsBenchmarkOnly ¶ added in v0.18.0

func (pp *ParserPool) ParseNoTreeWithExternalCheckpointsBenchmarkOnly(source []byte) (*Tree, error)

ParseNoTreeWithExternalCheckpointsBenchmarkOnly delegates to Parser.ParseNoTreeWithExternalCheckpointsBenchmarkOnly. It is intended only for parser performance attribution; the returned tree is not API-compatible.

func (*ParserPool) ParseStrict ¶ added in v0.20.6

func (pp *ParserPool) ParseStrict(source []byte) (*Tree, error)

ParseStrict delegates to a pooled Parser.ParseStrict call.

func (*ParserPool) ParseUTF16 ¶ added in v0.16.0

func (pp *ParserPool) ParseUTF16(source []uint16) (*Tree, error)

ParseUTF16 delegates to a pooled Parser.ParseUTF16 call.

func (*ParserPool) ParseUTF16Bytes ¶ added in v0.16.0

func (pp *ParserPool) ParseUTF16Bytes(source []byte, order UTF16ByteOrder) (*Tree, error)

ParseUTF16Bytes delegates to a pooled Parser.ParseUTF16Bytes call.

func (*ParserPool) ParseUTF16BytesWithTokenSourceFactory ¶ added in v0.16.0

func (pp *ParserPool) ParseUTF16BytesWithTokenSourceFactory(source []byte, order UTF16ByteOrder, factory TokenSourceFactory) (*Tree, error)

ParseUTF16BytesWithTokenSourceFactory delegates to a pooled Parser.ParseUTF16BytesWithTokenSourceFactory call.

func (*ParserPool) ParseUTF16WithTokenSourceFactory ¶ added in v0.16.0

func (pp *ParserPool) ParseUTF16WithTokenSourceFactory(source []uint16, factory TokenSourceFactory) (*Tree, error)

ParseUTF16WithTokenSourceFactory delegates to a pooled Parser.ParseUTF16WithTokenSourceFactory call.

func (*ParserPool) ParseWith ¶ added in v0.7.0

func (pp *ParserPool) ParseWith(source []byte, opts ...ParseOption) (ParseResult, error)

ParseWith delegates to a pooled Parser.ParseWith call.

func (*ParserPool) ParseWithStrict ¶ added in v0.20.6

func (pp *ParserPool) ParseWithStrict(source []byte, opts ...ParseOption) (ParseResult, error)

ParseWithStrict delegates to a pooled Parser.ParseWithStrict call.

func (*ParserPool) ParseWithTokenSource ¶ added in v0.7.0

func (pp *ParserPool) ParseWithTokenSource(source []byte, ts TokenSource) (*Tree, error)

ParseWithTokenSource delegates to a pooled Parser.ParseWithTokenSource call.

func (*ParserPool) ParseWithTokenSourceFactory ¶ added in v0.16.0

func (pp *ParserPool) ParseWithTokenSourceFactory(source []byte, factory TokenSourceFactory) (*Tree, error)

ParseWithTokenSourceFactory delegates to a pooled Parser.ParseWithTokenSourceFactory call.

func (*ParserPool) ParseWithTokenSourceFactoryStrict ¶ added in v0.20.6

func (pp *ParserPool) ParseWithTokenSourceFactoryStrict(source []byte, factory TokenSourceFactory) (*Tree, error)

ParseWithTokenSourceFactoryStrict delegates to a pooled Parser.ParseWithTokenSourceFactoryStrict call.

func (*ParserPool) ParseWithTokenSourceStrict ¶ added in v0.20.6

func (pp *ParserPool) ParseWithTokenSourceStrict(source []byte, ts TokenSource) (*Tree, error)

ParseWithTokenSourceStrict delegates to a pooled Parser.ParseWithTokenSourceStrict call.

type ParserPoolOption ¶ added in v0.7.0

type ParserPoolOption func(*parserPoolConfig)

ParserPoolOption configures a ParserPool.

func WithParserPoolAmbiguityProfile ¶ added in v0.17.0

func WithParserPoolAmbiguityProfile(profile *AmbiguityProfile) ParserPoolOption

WithParserPoolAmbiguityProfile installs an optional diagnostic ambiguity profile on checked-out parsers.

func WithParserPoolGLRTrace ¶ added in v0.7.0

func WithParserPoolGLRTrace(enabled bool) ParserPoolOption

WithParserPoolGLRTrace toggles GLR trace logs on pooled parser instances.

func WithParserPoolIncludedRanges ¶ added in v0.7.0

func WithParserPoolIncludedRanges(ranges []Range) ParserPoolOption

WithParserPoolIncludedRanges sets default include ranges for pooled parsers.

func WithParserPoolLogger ¶ added in v0.7.0

func WithParserPoolLogger(logger ParserLogger) ParserPoolOption

WithParserPoolLogger sets the logger applied to pooled parser instances.

func WithParserPoolMemoryBudgetBytes ¶ added in v0.53.0

func WithParserPoolMemoryBudgetBytes(bytes int64) ParserPoolOption

WithParserPoolMemoryBudgetBytes sets the per-parse memory budget applied to pooled parser instances. See Parser.SetMemoryBudgetBytes.

func WithParserPoolParseWorkLimits ¶ added in v0.53.0

func WithParserPoolParseWorkLimits(limits ParseWorkLimits) ParserPoolOption

WithParserPoolParseWorkLimits sets deterministic parser-loop limits on every parser checked out from the pool. Zero fields use source-derived defaults.

func WithParserPoolTimeoutMicros ¶ added in v0.7.0

func WithParserPoolTimeoutMicros(timeoutMicros uint64) ParserPoolOption

WithParserPoolTimeoutMicros sets the parse timeout for pooled parsers.

type Pattern ¶

type Pattern struct {
	// contains filtered or unexported fields
}

Pattern is a single top-level S-expression pattern in a query.

type PatternIssue ¶ added in v0.50.0

type PatternIssue struct {
	Kind PatternIssueKind
	// PatternIndex is the offending pattern's index, matching
	// QueryMatch.PatternIndex and Query.StartByteForPattern/EndByteForPattern.
	PatternIndex int
	// ParentType and ChildType are the grammar node type names from the
	// pattern: ChildType can never appear as a named child of ParentType.
	ParentType string
	ChildType  string
	// StartByte and EndByte give the offending pattern's full source span
	// (QueryStep does not carry its own byte range, only Pattern does).
	StartByte uint32
	EndByte   uint32
}

PatternIssue reports one pattern step ValidateQueryPatterns found structurally unreachable.

func ValidateQueryPatterns ¶ added in v0.50.0

func ValidateQueryPatterns(lang *Language, q *Query) []PatternIssue

ValidateQueryPatterns checks q's compiled patterns against lang's compiled grammar tables and reports every pattern step that structurally can never match -- gotreesitter's counterpart to the class of query tree-sitter's C ts_query_new statically rejects at compile time with TSQueryErrorStructure ("Impossible pattern"). q must have been compiled against lang (Query does not retain its own Language reference, matching every other Query method that takes lang explicitly).

gotreesitter's NewQuery does not reject these by default -- see WithStrictPatternValidation for an opt-in that does -- so ValidateQueryPatterns exists for callers who want to audit an already-compiled Query on demand, listing every offending step rather than stopping at the first the way a strict compile would.

What this checks ¶

For a pattern step of the form "(parent (child) ...)" -- one concrete, named node type asserted as a direct, field-less child of another concrete, named parent step -- this walks lang's compiled LR parse tables (the same tables the parser itself executes: ParseTable/SmallParseTable/ ParseActions/LargeStateGotos) to determine whether any reduction that produces `parent` can ever place `child` among its immediate children. A step is flagged only when the tables affirmatively prove no such reduction exists among everything they expose; whenever the analysis cannot reach a confident answer, it reports nothing for that step rather than guessing.

What this does not check, and why ¶

tree-sitter's own "Impossible pattern" analysis (lib/src/query.c, ts_query__analyze_patterns) walks the same generated LR automaton this does, but the C tool that builds it still has the grammar's original production right-hand sides in scope. gotreesitter only carries that shape for languages grammargen assembles at build time (Language.ProductionSignatures); every language decoded from a ts2go blob -- which includes every core-nine language this repository has ever found a dead pattern in, and every language this function's own tests exercise -- leaves ProductionSignatures empty, because tree-sitter's generated parser.c never embeds production right-hand sides either (the code generator throws that shape away once the tables are built). So this reconstructs an approximation of "legal child sets" from the compiled action/goto tables instead of reproducing C's analysis exactly:

  • Unknown node type names and unknown field names are not re-checked here: NewQuery already rejects both unconditionally, for every caller, before a *Query can exist. There is nothing left for a post-compile validator to add for those two classes.
  • Field-constrained children ("field: (child)") are never flagged. Legal field/child associations live in per-production field maps keyed by production ID; this analysis tracks which automaton state an edge lands in, not which production ID or child index it belongs to, so it has no ground truth for field legality. Skipped, not guessed.
  • Alternation branches ("[(a) (b)]") are never flagged, whether the alternation is itself a child step or contains one: QueryStep folds an alternation's branches into step.alternatives rather than ordinary nested steps, and this analysis does not walk that shape at all.
  • Anchors (".") and quantifiers (?, *, +) are ignored: this only answers "can child ever occur under parent at all", never "at this exact position" or "exactly this many times".
  • The check is existence-only, not position-aware: it unions together every child position of every production that ever reduces to `parent`. That makes it an over-approximation of the true per-position legal child set -- see Soundness below.
  • Hidden (invisible) wrapper symbols are resolved transitively (their own legal children substitute for the wrapper, recursively, up to a bounded depth) so ordinary grammar-generated flattening (repeat/choice helpers, hidden alternatives) does not produce false positives. A production's AliasSequences overrides are applied at that exact production only, not propagated through further transitive hidden lookups beyond it.
  • GLR conflict actions, error-recovery ("RECOVER") transitions, and "extra" (e.g. comment) tokens are read exactly as the tables encode them, with no special-casing. This can only make the computed legal-child set larger (more conservative), never smaller.
  • A grammar too large for this analysis to walk cleanly (see maxLanguageChildAnalysisStates) is skipped entirely: every step in every one of its queries reports no issue.

Soundness ¶

This analysis is built to be a (possibly loose) over-approximation of the true legal-child relation, not an exact reproduction of it: every signal it collects -- raw automaton edges into parent-interior states, this language's AliasSequences overrides, and hidden-symbol flattening -- can only add candidate children, never remove one the grammar genuinely allows. A returned PatternIssue is therefore a strong, tables-backed claim, but the absence of one is not proof the pattern is meaningful -- only that this analysis could not disprove it. That asymmetry is deliberate: a false "impossible" verdict would silently break a real, working query for an opted-in caller, which this treats as worse than a missed detection (the status quo before this function existed).

func (PatternIssue) String ¶ added in v0.50.0

func (i PatternIssue) String() string

String renders a human-readable diagnostic, e.g. for logging or lint output.

type PatternIssueKind ¶ added in v0.50.0

type PatternIssueKind uint8

PatternIssueKind classifies a problem ValidateQueryPatterns can report.

const (
	// PatternIssueImpossibleChild marks a pattern step that asserts a named
	// node type as a direct child of a parent node type that this
	// language's compiled parse tables prove can never hold it. See the doc
	// comment on ValidateQueryPatterns for exactly what "prove" means here
	// and this analysis's known blind spots.
	PatternIssueImpossibleChild PatternIssueKind = iota
)

type PendingParentFieldRejectPayloadStats ¶ added in v0.19.0

type PendingParentFieldRejectPayloadStats struct {
	Unknown              uint64
	Visible              uint64
	VisibleFinalLike     uint64
	VisibleNestedPayload uint64
	VisibleCompactLeaf   uint64
	VisibleFieldedDesc   uint64
	HiddenEmpty          uint64
	HiddenOne            uint64
	HiddenMany           uint64
	HiddenWithFields     uint64
}

type PendingParentFieldRejectStats ¶ added in v0.19.0

type PendingParentFieldRejectStats struct {
	Unknown               uint64
	ParentHidden          uint64
	NoIDs                 uint64
	Inherited             uint64
	HiddenChild           uint64
	HiddenChildPlain      uint64
	HiddenChildPlainEmpty uint64
	HiddenChildPlainOne   uint64
	HiddenChildPlainMany  uint64
	HiddenChildWithFields uint64
	Child                 uint64
	AllVisibleDirect      uint64
}

type PendingParentRejectStats ¶ added in v0.19.0

type PendingParentRejectStats struct {
	Unknown    uint64
	Empty      uint64
	ChildLimit uint64
	Alias      uint64
	RawSpan    uint64
	Fields     uint64
	Child      uint64
	Span       uint64
	Fill       uint64
}

type PerfCounters ¶ added in v0.6.0

type PerfCounters struct {
	MergeCalls           uint64
	MergeDeadPruned      uint64
	MergePerKeyOverflow  uint64
	MergeReplacements    uint64
	StackEquivalentCalls uint64
	StackEquivalentTrue  uint64
	StackEqHashMissSkips uint64
	StackCompareCalls    uint64
	ConflictRR           uint64
	ConflictRS           uint64
	ConflictOther        uint64
	ForkCount            uint64
	FirstConflictToken   uint64
	MaxConcurrentStacks  uint64
	// GSSCanReachVisits counts gssNode visits inside gssNodeCanReach's DFS
	// (glr.go), across every call in the parse. A fixed nesting depth that
	// forks and merges on every token should keep this roughly linear in
	// token count once depth-based pruning is in effect; before the prune it
	// grows with the stack depth on every call, i.e. superlinearly.
	GSSCanReachVisits uint64
	// ShapePrefixWalkSteps counts the GSS nodes gssMaterializingShapePrefix
	// (glr.go) had to hash because no cached prefix covered them, across the
	// parse. A fixed nesting depth that forks and merges on every token
	// should keep this roughly linear in token count; when every successful
	// merge invalidated the whole cache it grew with the spine depth on every
	// head hash, i.e. superlinearly (issue #454).
	ShapePrefixWalkSteps uint64
	// ShapePrefixEpochBumps counts full shape-prefix cache invalidations
	// (glrMergeScratch.bumpShapePrefixEpoch) across the parse.
	ShapePrefixEpochBumps uint64
	LexBytes              uint64
	LexTokens             uint64
	// ProbeLexBytes and ProbeLexTokens count the compact EOF scanner
	// quiescence probe's own lexing (proveCompactEOFScannerQuiescence,
	// parsercore_phase0_eof_scanner_quiescence.go), kept apart from
	// LexBytes/LexTokens, which name the parse's own token stream only.
	ProbeLexBytes                        uint64
	ProbeLexTokens                       uint64
	ReuseNodesVisited                    uint64
	ReuseNodesPushed                     uint64
	ReuseNodesPopped                     uint64
	ReuseCandidatesChecked               uint64
	ReuseSuccesses                       uint64
	ReuseLeafSuccesses                   uint64
	ReuseNonLeafChecks                   uint64
	ReuseNonLeafSuccesses                uint64
	ReuseNonLeafBytes                    uint64
	ReuseNonLeafNoGoto                   uint64
	ReuseNonLeafNoGotoTerm               uint64
	ReuseNonLeafNoGotoNt                 uint64
	ReuseNonLeafStateMiss                uint64
	ReuseNonLeafStateZero                uint64
	MergeHashZero                        uint64
	GlobalCapCulls                       uint64
	GlobalCapCullDropped                 uint64
	ReduceChainSteps                     uint64
	ReduceChainMaxLen                    uint64
	ReduceChainBreakMulti                uint64
	ReduceChainBreakShift                uint64
	ReduceChainBreakAccept               uint64
	ReduceChainHintCandidates            uint64
	ReduceChainHintTaken                 uint64
	ReduceChainHintSteps                 uint64
	ReduceChainHintTerminalOK            uint64
	ReduceChainHintTerminalMismatch      uint64
	ReduceChainHintLimit                 uint64
	ReduceChainHintDead                  uint64
	ReduceChainHintUnexpected            uint64
	ParentChildPointers                  uint64
	ReduceChildrenFastGSS                uint64
	ReduceChildrenAllVis                 uint64
	ReduceChildrenScratch                uint64
	ReduceScratchNoAlias                 uint64
	ReduceScratchGeneral                 uint64
	ReduceChildEmptyBuilds               uint64
	ReduceChildAllVisibleBuilds          uint64
	ReduceChildScratchNoAliasBuilds      uint64
	ReduceChildScratchGeneralBuilds      uint64
	ReduceForkCalls                      uint64
	ReduceForkWindows                    uint64
	ReduceForkMaxWindows                 uint64
	PostReduceMergeAttempts              uint64
	PostReduceMergePrimarySuccesses      uint64
	PostReduceMergePendingSuccesses      uint64
	PostReduceMergeDisabledSkips         uint64
	PostReduceMergeFinalizationRiskSkips uint64
	PendingForkStackAppends              uint64
	PendingForkStacksMaxLen              uint64
	ForestReduceCalls                    uint64
	ForestReduceZero                     uint64
	ForestReduceLinearNoExtras           uint64
	ForestReduceDFS                      uint64
	ForestReduceDFSLinks                 uint64
	ForestReduceDFSMultiLinkSteps        uint64
	ForestReduceDFSExtraLinks            uint64
	ForestReduceDFSVisits                uint64
	ForestReduceDFSPathEntries           uint64
	ForestReduceGotoHits                 uint64
	ForestReduceGotoMisses               uint64
	ForestReduceMaxPathLen               uint64
	ForestReduceMaxChildCount            uint64
	ForestCoalesceCalls                  uint64
	ForestCoalesceNewNodes               uint64
	ForestCoalesceLinkAppends            uint64
	ForestCoalesceDedupHits              uint64
	ForestCoalesceDedupReplacements      uint64
	ForestCoalescePreCapDrops            uint64
	ForestCoalesceCapDrops               uint64
	ForestCoalesceCapReplacements        uint64
	ExtraNodes                           uint64
	ErrorNodes                           uint64
	SyntheticReplayGapBridgeCalls        uint64
	SyntheticReplayGapSteps              uint64
	SyntheticReplayGapCursorAttempts     uint64
	SyntheticReplayGapCursorDedupHits    uint64
	SyntheticReplayGapCursorPeak         uint64
	SyntheticReplayGapLexAttempts        uint64
	SyntheticReplayGapLexCacheHits       uint64
	SyntheticReplayGapLexCacheMisses     uint64
	SyntheticReplayGapLexCacheStores     uint64
	SyntheticReplayGapLexCacheCapSkips   uint64
	SyntheticReplayAdvanceAttempts       uint64
	SyntheticReplayAdvanceCacheHits      uint64
	SyntheticReplayAdvanceCacheMisses    uint64
	SyntheticReplayAdvanceCacheStores    uint64
	SyntheticReplayAdvanceCacheCapSkips  uint64
	SyntheticReplayAdvanceZeroOutputs    uint64
	SyntheticReplayAdvanceSingleOutputs  uint64
	SyntheticReplayAdvanceMultiOutputs   uint64
	SyntheticReplayAdvanceOutputPeak     uint64
	SyntheticReplayAdvanceOutputFrames   uint64
	SyntheticReplayAdvanceUniqueOutputs  uint64
	SyntheticReplayAdvanceUniqueFrames   uint64
	SyntheticReplayAdvanceUniqueSingle   uint64
	SyntheticReplayAdvanceUniqueMulti    uint64
	SyntheticReplayAdvanceUniquePeak     uint64
	SyntheticReplayStackPushCalls        uint64
	SyntheticReplayStackInternHits       uint64
	SyntheticReplayCloseMemoHits         uint64
	SyntheticReplayCloseMemoMisses       uint64
	SyntheticReplayCloseMemoStores       uint64
	SyntheticReplayCloseMemoCapSkips     uint64
	SyntheticReplayCloseZeroOutputs      uint64
	SyntheticReplayCloseSingleOutputs    uint64
	SyntheticReplayCloseMultiOutputs     uint64
	SyntheticReplayCloseOutputPeak       uint64
	SyntheticReplayCloseOutputFrames     uint64
	SyntheticReplayCloseUniqueOutputs    uint64
	SyntheticReplayCloseUniqueFrames     uint64
	SyntheticReplayCloseUniqueSingle     uint64
	SyntheticReplayCloseUniqueMulti      uint64
	SyntheticReplayCloseUniquePeak       uint64
	MergeStacksInHist                    [maxGLRStacks + 2]uint64
	MergeAliveHist                       [maxGLRStacks + 2]uint64
	MergeOutHist                         [maxGLRStacks + 2]uint64
	ForkActionsHist                      [8]uint64
	CloneTreeCalls                       uint64
	CloneTreePublicNodes                 uint64
	CloneTreeFinalRefs                   uint64
	CloneTreeCompactCopies               uint64
	CloneTreeChildRefs                   uint64
	CloneOffsetCalls                     uint64
	CloneOffsetPublicNodes               uint64
	CloneOffsetCopies                    uint64
	CloneOffsetShifted                   uint64
	NodeEditCalls                        uint64
	NodeEditNoopCalls                    uint64
	NodeEditCompactRefs                  uint64
	NodeEditShifted                      uint64
	NodeEditMarked                       uint64
	DenseMutationCalls                   uint64
	DenseMutationDrains                  uint64
	MutationChildRefCOW                  uint64
}

func PerfCountersSnapshot ¶ added in v0.6.0

func PerfCountersSnapshot() PerfCounters

type Point ¶

type Point struct {
	Row    uint32
	Column uint32
}

Point is a row/column position in source text.

type PointSkippableTokenSource ¶

type PointSkippableTokenSource interface {
	ByteSkippableTokenSource
	SkipToByteWithPoint(offset uint32, pt Point) Token
}

PointSkippableTokenSource extends ByteSkippableTokenSource with a hint-based skip that avoids recomputing row/column from byte offset. During incremental parsing the reused node already carries its endpoint, so passing it directly eliminates the O(n) offset-to-point scan.

type ProductionSignature ¶ added in v0.21.0

type ProductionSignature struct {
	LHS          Symbol
	ProductionID uint16
	RHS          []Symbol
}

ProductionSignature records the raw RHS symbols for a grammar production. It is generated by grammargen, where the normalized grammar still has RHS structure. Legacy ts2go blobs leave this empty.

type Query ¶

type Query struct {
	// contains filtered or unexported fields
}

Query holds compiled patterns parsed from a tree-sitter .scm query file. It can be executed against a syntax tree to find matching nodes and return captured names.

Query is safe for concurrent calls to Execute, ExecuteInto, ExecuteNode, and Exec after construction. Each goroutine must use its own QueryCursor and any ExecuteInto destination slice must remain caller-owned. The mutating methods DisableCapture and DisablePattern are NOT safe to call concurrently with execution (or with each other); call them before sharing the Query.

func NewQuery ¶

func NewQuery(source string, lang *Language) (*Query, error)

NewQuery compiles query source (tree-sitter .scm format) against a language. It returns an error if the query syntax is invalid or references unknown node types or field names.

NewQuery is a thin call to NewQueryWithOptions with no options; its signature and behavior are unchanged from before NewQueryWithOptions existed. See NewQueryWithOptions to opt into WithStrictPatternValidation.

func NewQueryWithOptions ¶ added in v0.50.0

func NewQueryWithOptions(source string, lang *Language, opts ...QueryOption) (*Query, error)

NewQueryWithOptions compiles query source the same way NewQuery does, plus any QueryOptions. opts is empty in every existing call site through NewQuery and defaults to NewQuery's exact behavior; see WithStrictPatternValidation for the one option currently defined.

func (*Query) CaptureCount ¶ added in v0.7.0

func (q *Query) CaptureCount() uint32

CaptureCount returns the number of unique capture names in this query.

func (*Query) CaptureNameForID ¶ added in v0.7.0

func (q *Query) CaptureNameForID(id uint32) (string, bool)

CaptureNameForID returns the capture name for the given capture id.

func (*Query) CaptureNames ¶

func (q *Query) CaptureNames() []string

CaptureNames returns the list of unique capture names used in the query.

func (*Query) DisableCapture ¶ added in v0.7.0

func (q *Query) DisableCapture(name string)

DisableCapture removes captures with the given name from future query results. Matching behavior is unchanged; only returned captures are filtered.

func (*Query) DisablePattern ¶ added in v0.7.0

func (q *Query) DisablePattern(patternIndex uint32)

DisablePattern disables a pattern by index.

func (*Query) EndByteForPattern ¶ added in v0.7.0

func (q *Query) EndByteForPattern(patternIndex uint32) (uint32, bool)

EndByteForPattern returns the query-source end byte for patternIndex.

func (*Query) Exec ¶

func (q *Query) Exec(node *Node, lang *Language, source []byte) *QueryCursor

Exec creates a streaming cursor over matches rooted at node.

func (*Query) Execute ¶

func (q *Query) Execute(tree *Tree) []QueryMatch

Execute runs the query against a syntax tree and returns all matches.

func (*Query) ExecuteInto ¶ added in v0.10.2

func (q *Query) ExecuteInto(tree *Tree, dst []QueryMatch) []QueryMatch

ExecuteInto runs the query against a syntax tree, appending matches into dst and returning the updated slice. Callers can pre-allocate or reuse dst across calls to eliminate the per-call slice allocation from Execute.

Example:

var buf []QueryMatch
for _, tree := range trees {
    buf = q.ExecuteInto(tree, buf[:0])
    process(buf)
}

func (*Query) ExecuteNode ¶

func (q *Query) ExecuteNode(node *Node, lang *Language, source []byte) []QueryMatch

ExecuteNode runs the query starting from a specific node.

source is required for text predicates (like #eq? / #match?); pass the originating source bytes for correct predicate evaluation.

func (*Query) IsPatternGuaranteedAtStep ¶ added in v0.7.0

func (q *Query) IsPatternGuaranteedAtStep(patternIndex uint32, stepIndex uint32) bool

IsPatternGuaranteedAtStep reports whether all steps through stepIndex are definite and non-quantified.

func (*Query) IsPatternNonLocal ¶ added in v0.7.0

func (q *Query) IsPatternNonLocal(patternIndex uint32) bool

IsPatternNonLocal reports whether the pattern can begin at multiple roots.

func (*Query) IsPatternRooted ¶ added in v0.7.0

func (q *Query) IsPatternRooted(patternIndex uint32) bool

IsPatternRooted reports whether the pattern has exactly one root step at depth 0 and that step is not quantified. Quantified roots span siblings. Rooted patterns start matching from a single concrete root.

func (*Query) PatternCount ¶

func (q *Query) PatternCount() int

PatternCount returns the number of patterns in the query.

func (*Query) PredicatesForPattern ¶ added in v0.7.0

func (q *Query) PredicatesForPattern(patternIndex uint32) ([]QueryPredicate, bool)

PredicatesForPattern returns a copy of predicates attached to patternIndex.

func (*Query) PropertyPredicatesForPattern ¶ added in v0.39.0

func (q *Query) PropertyPredicatesForPattern(patternIndex uint32) ([]QueryPropertyPredicate, bool)

PropertyPredicatesForPattern returns the #is? and #is-not? metadata attached to patternIndex in source order. A valid pattern with no property predicates returns an empty slice and ok=true.

func (*Query) StartByteForPattern ¶ added in v0.7.0

func (q *Query) StartByteForPattern(patternIndex uint32) (uint32, bool)

StartByteForPattern returns the query-source start byte for patternIndex.

func (*Query) StepIsDefinite ¶ added in v0.7.0

func (q *Query) StepIsDefinite(patternIndex uint32, stepIndex uint32) bool

StepIsDefinite reports whether a pattern step matches a definite symbol (i.e. not wildcard).

func (*Query) StringCount ¶ added in v0.7.0

func (q *Query) StringCount() uint32

StringCount returns the number of unique string literals in this query.

func (*Query) StringValueForID ¶ added in v0.7.0

func (q *Query) StringValueForID(id uint32) (string, bool)

StringValueForID returns the string literal for the given string id.

type QueryCapture ¶

type QueryCapture struct {
	Name string
	Node *Node
	// TextOverride, when non-empty, replaces the node's source text for
	// downstream consumers. It is set by the #strip! directive.
	TextOverride string
	// contains filtered or unexported fields
}

QueryCapture is a single captured node within a match.

func (QueryCapture) ByteRange ¶ added in v0.54.0

func (c QueryCapture) ByteRange() (start, end uint32)

ByteRange returns the effective start and end byte offsets for this capture. If the #offset! directive adjusted the capture, the adjusted range is returned; otherwise the underlying node's own byte range is returned.

func (QueryCapture) PointRange ¶ added in v0.54.0

func (c QueryCapture) PointRange() (start, end Point)

PointRange returns the effective start and end point for this capture, the same way ByteRange returns byte offsets.

func (QueryCapture) Range ¶ added in v0.54.0

func (c QueryCapture) Range() Range

Range returns this capture's effective range as a Range value. It composes ByteRange and PointRange, so it honors a range adjusted by the #offset! directive the same way both do. Callers that only need the resulting Range value, such as an outline or a tags consumer, use this instead of reading the range off the underlying node.

func (QueryCapture) Text ¶ added in v0.6.0

func (c QueryCapture) Text(source []byte) string

Text returns the effective text for this capture. If TextOverride is set (e.g. by the #strip! directive), it is returned. Otherwise, if the #offset! directive adjusted the capture's range, the source text for that adjusted range is returned. Otherwise the node's own source text is returned.

func (QueryCapture) UTF16Range ¶ added in v0.16.0

func (c QueryCapture) UTF16Range(tree *Tree) (UTF16Range, bool)

UTF16Range returns this capture's effective range in UTF-16 code-unit coordinates for trees produced by UTF-16 parse APIs. It honors a range adjusted by the #offset! directive the same way ByteRange does.

type QueryCursor ¶

type QueryCursor struct {
	// contains filtered or unexported fields
}

QueryCursor incrementally walks a node subtree and yields matches one by one. It is the streaming counterpart to Query.Execute and avoids materializing all matches up front. QueryCursor is not safe for concurrent use.

func (*QueryCursor) DidExceedMatchLimit ¶ added in v0.7.0

func (c *QueryCursor) DidExceedMatchLimit() bool

DidExceedMatchLimit reports whether query execution had additional matches beyond the configured match limit.

func (*QueryCursor) NextCapture ¶

func (c *QueryCursor) NextCapture() (QueryCapture, bool)

NextCapture yields captures in match order by draining NextMatch results. This is a practical first-pass ordering: captures are returned in each match's capture order, then by subsequent matches in DFS match order.

func (*QueryCursor) NextMatch ¶

func (c *QueryCursor) NextMatch() (QueryMatch, bool)

NextMatch yields the next query match from the cursor.

func (*QueryCursor) SetByteRange ¶ added in v0.6.0

func (c *QueryCursor) SetByteRange(startByte, endByte uint32)

SetByteRange restricts matches to nodes that intersect [startByte, endByte). As in tree-sitter C, endByte == 0 is the unbounded-end sentinel.

func (*QueryCursor) SetMatchLimit ¶ added in v0.7.0

func (c *QueryCursor) SetMatchLimit(limit uint32)

SetMatchLimit sets the maximum number of matches this cursor can return. A limit of 0 means unlimited.

func (*QueryCursor) SetMatchWorkBudget ¶ added in v0.40.0

func (c *QueryCursor) SetMatchWorkBudget(limit int)

SetMatchWorkBudget bounds the number of enumeration steps the matcher may take per (pattern,node) attempt, guarding against pathological O(2^n) queries. A limit of 0 means unlimited. The default is defaultQueryMatchWorkBudget. On exhaustion the cursor returns bounded partial results and DidExceedMatchLimit reports true, mirroring C tree-sitter's over-limit behavior.

func (*QueryCursor) SetMaxStartDepth ¶ added in v0.7.0

func (c *QueryCursor) SetMaxStartDepth(depth uint32)

SetMaxStartDepth limits the depth at which new matches can begin. Depth 0 means only the starting node passed to Exec.

func (*QueryCursor) SetPointRange ¶ added in v0.6.0

func (c *QueryCursor) SetPointRange(startPoint, endPoint Point)

SetPointRange restricts matches to nodes that intersect [startPoint, endPoint). As in tree-sitter C, an all-zero end point is the unbounded-end sentinel.

func (*QueryCursor) SetUTF16Range ¶ added in v0.16.0

func (c *QueryCursor) SetUTF16Range(tree *Tree, startCodeUnit, endCodeUnit uint32) bool

SetUTF16Range restricts matches to nodes that intersect the given UTF-16 code-unit range. tree must have been produced by a UTF-16 parse API.

type QueryMatch ¶

type QueryMatch struct {
	PatternIndex int
	Captures     []QueryCapture
}

QueryMatch represents a successful pattern match with its captures.

func (QueryMatch) SetValues ¶ added in v0.6.0

func (m QueryMatch) SetValues(q *Query, key string) []string

SetValues returns the values of a #set! directive with the given key for a match's pattern, or nil if not present. This is used by InjectionParser to read injection.language metadata.

type QueryOption ¶ added in v0.50.0

type QueryOption func(*queryCompileOptions)

QueryOption configures optional NewQuery compile-time behavior. The zero value of every option is a no-op, so existing two-argument NewQuery(source, lang) call sites are unaffected.

func WithStrictPatternValidation ¶ added in v0.50.0

func WithStrictPatternValidation() QueryOption

WithStrictPatternValidation opts NewQuery into rejecting a query whose compiled patterns ValidateQueryPatterns proves structurally impossible, mirroring tree-sitter's C ts_query_new: a query with at least one such pattern fails to compile at all, reporting the first offending pattern (source order) as the error, exactly as C reports only the first "Impossible pattern" it hits and rejects the whole multi-pattern query.

This is opt-in, not the default, for two reasons: gotreesitter's default NewQuery behavior must not change for existing callers, and dozens of this repository's own currently-shipping fleet grammars' inferred queries this analysis flags at least one dead pattern in today (see grammars/tags_query_infer.go's "Impossible-pattern fixes" note for the core-nine languages already audited and fixed) -- enabling this by default would break compilation for languages nobody has audited yet. See ValidateQueryPatterns for exactly what the underlying analysis proves and its known blind spots; this option can reject a query the analysis confidently disproves, but it can also let a genuinely dead pattern through undetected the way NewQuery always has.

type QueryPredicate ¶

type QueryPredicate struct {
	// contains filtered or unexported fields
}

QueryPredicate is a post-match constraint attached to a pattern. Supported forms:

  • (#eq? @a @b)
  • (#eq? @a "literal")
  • (#not-eq? @a @b)
  • (#not-eq? @a "literal")
  • (#match? @a "regex")
  • (#not-match? @a "regex")
  • (#lua-match? @a "lua-pattern")
  • (#any-of? @a "v1" "v2" ...)
  • (#not-any-of? @a "v1" "v2" ...)
  • (#any-eq? @a "literal"), (#any-eq? @a @b)
  • (#any-not-eq? @a "literal"), (#any-not-eq? @a @b)
  • (#any-match? @a "regex")
  • (#any-not-match? @a "regex")
  • (#has-ancestor? @a type ...)
  • (#not-has-ancestor? @a type ...)
  • (#has-parent? @a type ...)
  • (#not-has-parent? @a type ...)
  • (#is? ...), (#is-not? ...)
  • (#set! key value), (#offset! @cap ...)
  • (#count? @a op value) -- op: >, <, >=, <=, ==, !=
  • (#is-exported? @a)

func (QueryPredicate) PropertyPredicate ¶ added in v0.39.0

func (p QueryPredicate) PropertyPredicate() (property QueryPropertyPredicate, ok bool)

PropertyPredicate exposes p when it represents #is? or #is-not? metadata. Other predicate kinds return ok=false.

type QueryPropertyPredicate ¶ added in v0.39.0

type QueryPropertyPredicate struct {
	Property string
	Capture  string
	Positive bool
}

QueryPropertyPredicate is host-consumable metadata for an inert #is? or #is-not? predicate. Property is the queried property name, Capture is the optional capture name without its leading '@', and Positive distinguishes #is? from #is-not?. Property predicates do not filter query matches.

type QueryStep ¶

type QueryStep struct {
	// contains filtered or unexported fields
}

QueryStep is one matching instruction within a pattern.

type Range ¶

type Range struct {
	StartByte  uint32
	EndByte    uint32
	StartPoint Point
	EndPoint   Point
}

Range is a span of source text.

func DiffChangedRanges ¶ added in v0.6.0

func DiffChangedRanges(oldTree, newTree *Tree) []Range

DiffChangedRanges compares two syntax trees and returns the minimal ranges where syntactic structure differs. The old tree should have been edited (via Tree.Edit) to match the new tree's source positions before reparsing.

This is equivalent to C tree-sitter's ts_tree_get_changed_ranges().

func IncludedRangesForUTF16 ¶ added in v0.16.0

func IncludedRangesForUTF16(source []uint16, ranges []UTF16Range) ([]Range, bool)

IncludedRangesForUTF16 converts UTF-16 included ranges into the parser's internal UTF-8 byte ranges. The returned Range points use UTF-8 columns.

func IncludedRangesForUTF16Bytes ¶ added in v0.16.0

func IncludedRangesForUTF16Bytes(source []byte, order UTF16ByteOrder, ranges []UTF16Range) ([]Range, error)

IncludedRangesForUTF16Bytes converts endian-specific UTF-16 byte ranges into the parser's internal UTF-8 byte ranges. The returned Range points use UTF-8 columns.

type RecoveryNodeMemoRuntime ¶ added in v0.49.0

type RecoveryNodeMemoRuntime struct {
	PeakTier   RecoveryNodeMemoTier
	Collisions uint32
}

RecoveryNodeMemoRuntime reports bounded recovery-memo use for one tree.

type RecoveryNodeMemoTier ¶ added in v0.49.0

type RecoveryNodeMemoTier uint8

RecoveryNodeMemoTier identifies the largest bounded recovery memo used by a parse. Entries and Bytes expand the compact telemetry value.

const (
	RecoveryNodeMemoTierNone RecoveryNodeMemoTier = iota
	RecoveryNodeMemoTierInitial
	RecoveryNodeMemoTierStandard
	RecoveryNodeMemoTierTemporary
)

func (RecoveryNodeMemoTier) Bytes ¶ added in v0.49.0

func (tier RecoveryNodeMemoTier) Bytes() uint32

Bytes reports the allocated byte size of this memo tier.

func (RecoveryNodeMemoTier) Entries ¶ added in v0.49.0

func (tier RecoveryNodeMemoTier) Entries() uint32

Entries reports the number of entries in this memo tier.

type RecoveryRuntimeAttemptStats ¶ added in v0.52.0

type RecoveryRuntimeAttemptStats struct {
	Ordinal uint32
	Rung    string
	Cause   string

	StopReason          ParseStopReason
	Truncated           bool
	TokenSourceEOFEarly bool
	AttemptHasError     bool
	AttemptFullSpan     bool

	WallNanos            uint64
	HeapAllocDeltaBytes  int64
	TotalAllocDeltaBytes uint64
	MallocsDelta         uint64

	RecoveryEntryCount           uint64
	Strategy1ElectionCount       uint64
	RecoveryCostCompetitionCount uint64
	RecoveryCostWalkCount        uint64
	RecoveryCostWalkNanos        uint64

	MaterializationNanos                uint64
	ResultSelectionNanos                uint64
	TransientParentMaterializationNanos uint64
	ResultTreeBuildNanos                uint64
	TransientChildMaterializationNanos  uint64
	CondenseNanos                       uint64

	ArenaBytesPeak        uint64
	ScratchBytesPeak      uint64
	EntryScratchBytesPeak uint64
	GSSBytesPeak          uint64
	GSSNodesPeak          uint64
	NodesAllocated        uint64
	MaxStacksSeen         uint64
	PeakStackDepth        uint64
	LiveVersions          uint64
	PeakLiveVersions      uint64

	CandidateSelected          bool
	CandidateReplacedIncumbent bool
}

RecoveryRuntimeAttemptStats reports diagnostics for one parser attempt.

These facts stay separate from RecoveryRuntimeStats. The latter reports the selected tree. A losing attempt can still consume time and memory.

type RecoveryRuntimeAttempts ¶ added in v0.52.0

type RecoveryRuntimeAttempts []RecoveryRuntimeAttemptStats

RecoveryRuntimeAttempts contains attempt-local facts for one parse operation. Use Parser.DebugRecoveryRuntimeAttempts to read this receipt.

type RecoveryRuntimeStats ¶ added in v0.50.0

type RecoveryRuntimeStats struct {
	Enabled   bool
	Completed bool

	RecoveryEntryCount           uint64
	Strategy1ElectionCount       uint64
	RecoveryCostCompetitionCount uint64
	RecoveryCostWalkCount        uint64
	RecoveryCostWalkNanos        uint64
	ErrorNodeCount               uint64
	ErrorSpanBytes               uint32
	RetryPassCount               uint64
	RetryReason                  string
	RetryAttemptCount            uint64
	RetrySelectedAttempt         string
	RetrySelectedAttemptHasError bool
	RetrySelectedAttemptFullSpan bool
	ErrorModeTokenCount          uint64
	ScannerResyncCount           uint64
	LiveVersionCount             uint64
	PeakLiveVersionCount         uint64
}

RecoveryRuntimeStats reports opt-in recovery facts for the most recent completed parse attempt on a parser.

The default value reports no facts. Enable the telemetry before parsing. Existing tree runtime fields and RecoveryNodeMemoRuntime provide the other B16 facts, including materialization, allocation, stop, and memo metrics.

type ReduceChainHint ¶ added in v0.19.0

type ReduceChainHint struct {
	StartState     StateID
	Lookahead      Symbol
	TerminalStates []StateID
	TerminalAction ReduceChainTerminalAction
	MaxSteps       uint16
}

ReduceChainHint describes a terminal-verified parser hot path for a deterministic reduce chain. The runtime still applies normal reduce semantics and stops before the terminal action; this metadata only lets it avoid repeated generic action dispatch for approved state/lookahead pairs.

type ReduceChainTerminalAction ¶ added in v0.19.0

type ReduceChainTerminalAction uint8

ReduceChainTerminalAction describes the action class expected after a generated reduce-chain hint finishes applying deterministic reductions.

const (
	ReduceChainTerminalNoAction ReduceChainTerminalAction = iota
	ReduceChainTerminalSingleReduce
	ReduceChainTerminalSingleShift
	ReduceChainTerminalSingleAccept
	ReduceChainTerminalSingleOther
	ReduceChainTerminalMulti
)

type ReduceChildPathRuntime ¶ added in v0.18.0

type ReduceChildPathRuntime struct {
	SlicesAllocated   uint64
	SlicesRetained    uint64
	SlicesDropped     uint64
	PointersAllocated uint64
	PointersRetained  uint64
	PointersDropped   uint64
}

type ResultCompatibilityCapability ¶ added in v0.33.0

type ResultCompatibilityCapability uint64

ResultCompatibilityCapability records result-tree shapes that a language produces natively and therefore does not need the runtime to repair after parsing. Keep capability values append-only: Language blobs encode these numeric bits, and a zero value deliberately preserves legacy behavior.

const (
	ResultCompatibilityCSharpNativeNotNull                ResultCompatibilityCapability = 1 << 0
	ResultCompatibilityCSharpNativeUnicodeIdentifiers     ResultCompatibilityCapability = 1 << 1
	ResultCompatibilityCSharpNativeScopedLambdaStatements ResultCompatibilityCapability = 1 << 2
	ResultCompatibilityCSharpNativeScopedLambdaBlocks     ResultCompatibilityCapability = 1 << 3
	ResultCompatibilityCSharpNativeQueryExpressions       ResultCompatibilityCapability = 1 << 4
	// ResultCompatibilityNativeCollapsedChildren records the v0.46 exact-profile
	// certification receipt for collapsed-child rows. It admits exact built-ins
	// and true adapted clones; native retention then keys off exact registered
	// parent/raw-child metadata identities. A matching display name or pair-level
	// metadata alone is insufficient. Keep the bit append-only because Language
	// blobs encode capability values.
	ResultCompatibilityNativeCollapsedChildren ResultCompatibilityCapability = 1 << 5
	// ResultCompatibilityNativeRecoveredStructure permits receipt checks for
	// error-bearing native roots. Consumers must also verify the complete source
	// span, raw top-level spans, and the isolated-error shape.
	ResultCompatibilityNativeRecoveredStructure ResultCompatibilityCapability = 1 << 6
)

type Rewriter ¶ added in v0.6.0

type Rewriter struct {
	// contains filtered or unexported fields
}

Rewriter collects source-text edits and applies them atomically. Edits target byte ranges (usually from Node.StartByte/EndByte). Apply returns new source bytes and InputEdit records for incremental reparsing. Rewriter is not safe for concurrent use.

func NewRewriter ¶ added in v0.6.0

func NewRewriter(source []byte) *Rewriter

NewRewriter creates a Rewriter for the given source text.

func (*Rewriter) Apply ¶ added in v0.6.0

func (r *Rewriter) Apply() (newSource []byte, edits []InputEdit, err error)

Apply sorts edits, validates no overlaps, applies them, and returns the new source bytes plus InputEdit records for incremental reparsing. Returns an error if edits overlap, use reversed ranges, or extend beyond the source.

func (*Rewriter) ApplyToTree ¶ added in v0.6.0

func (r *Rewriter) ApplyToTree(tree *Tree) ([]byte, error)

ApplyToTree is a convenience that calls Apply(), then tree.Edit() for each edit, returning the new source ready for ParseIncremental.

func (*Rewriter) Delete ¶ added in v0.6.0

func (r *Rewriter) Delete(node *Node)

Delete removes the source text covered by node.

func (*Rewriter) InsertAfter ¶ added in v0.6.0

func (r *Rewriter) InsertAfter(node *Node, text []byte)

InsertAfter inserts text immediately after node.

func (*Rewriter) InsertBefore ¶ added in v0.6.0

func (r *Rewriter) InsertBefore(node *Node, text []byte)

InsertBefore inserts text immediately before node.

func (*Rewriter) Replace ¶ added in v0.6.0

func (r *Rewriter) Replace(node *Node, newText []byte)

Replace replaces the source text covered by node with newText.

func (*Rewriter) ReplaceRange ¶ added in v0.6.0

func (r *Rewriter) ReplaceRange(startByte, endByte uint32, newText []byte)

ReplaceRange replaces bytes in [startByte, endByte) with newText.

type StateID ¶

type StateID uint32

StateID is a parser state index. uint32 supports grammars with >65K states (e.g. COBOL with 67K states from 1071 rules).

type StatelessExternalScanner ¶ added in v0.45.0

type StatelessExternalScanner interface {
	ExternalScanner
	ExternalScannerIsStateless() bool
}

StatelessExternalScanner is implemented by external scanners that carry no serialized state across tokens. Their Scan decision is a pure function of the byte stream at the lexer position and the valid-symbol set, and the valid-symbol set is itself a pure function of the LR parser state. For such a scanner the state at any boundary equals the state a fresh parse holds there, so the scanner-quiescence proof obligation (campaign O(edit) workstream W4) is discharged at every boundary. Go's automatic-semicolon scanner is the reference example; see external_scanner_quiescence.go for the proof.

A scanner implements this only when it can meet every quiescence obligation. The classifier reads the marker as a proof, so an incorrect true is silent incremental corruption.

type Symbol ¶

type Symbol uint16

Symbol is a grammar symbol ID (terminal or nonterminal).

type SymbolMetadata ¶

type SymbolMetadata struct {
	Name               string
	Visible            bool
	Named              bool
	Supertype          bool
	GeneratedRepeatAux bool
}

SymbolMetadata holds display information about a symbol.

type Tag ¶

type Tag struct {
	Kind      string // e.g. "definition.function", "reference.call"
	Name      string // the captured symbol text
	Range     Range  // full span of the tagged node
	NameRange Range  // span of the @name capture
}

Tag represents a tagged symbol in source code, extracted by a Tagger. Kind follows tree-sitter convention: "definition.function", "reference.call", etc. Name is the captured symbol text (e.g., the function name).

type Tagger ¶

type Tagger struct {
	// contains filtered or unexported fields
}

Tagger extracts symbol definitions and references from source code using tree-sitter tags queries. It is the tagging counterpart to Highlighter.

Tags queries use a convention where captures follow the pattern:

  • @name captures the symbol name (e.g., function identifier)
  • @definition.X or @reference.X captures the kind

Example query:

(function_declaration name: (identifier) @name) @definition.function
(call_expression function: (identifier) @name) @reference.call

func NewTagger ¶

func NewTagger(lang *Language, tagsQuery string, opts ...TaggerOption) (*Tagger, error)

NewTagger creates a Tagger for the given language and tags query.

func (*Tagger) Tag ¶

func (tg *Tagger) Tag(source []byte) []Tag

Tag parses source and returns all tags.

func (*Tagger) TagIncremental ¶

func (tg *Tagger) TagIncremental(source []byte, oldTree *Tree) ([]Tag, *Tree)

TagIncremental re-tags source after edits to oldTree. Returns the tags and the new tree for subsequent incremental calls.

func (*Tagger) TagIncrementalStrict ¶ added in v0.37.0

func (tg *Tagger) TagIncrementalStrict(source []byte, oldTree *Tree) ([]Tag, *Tree, error)

TagIncrementalStrict is like TagIncremental, but reports ErrParseStoppedEarly and skips query execution when parsing is stopped by a timeout, cancellation, token-source EOF, or parser safety limit. The partial tree is returned so callers can release it or use it for diagnostics.

func (*Tagger) TagIncrementalUTF16 ¶ added in v0.16.0

func (tg *Tagger) TagIncrementalUTF16(source []uint16, oldTree *Tree) ([]UTF16Tag, *Tree)

TagIncrementalUTF16 re-tags UTF-16 source after edits to oldTree. Call oldTree.EditUTF16 before calling this.

func (*Tagger) TagIncrementalUTF16Bytes ¶ added in v0.16.0

func (tg *Tagger) TagIncrementalUTF16Bytes(source []byte, oldTree *Tree, order UTF16ByteOrder) ([]UTF16Tag, *Tree, error)

TagIncrementalUTF16Bytes is like TagIncrementalUTF16 for endian-specific UTF-16 bytes.

func (*Tagger) TagStrict ¶ added in v0.51.0

func (tg *Tagger) TagStrict(source []byte) ([]Tag, error)

TagStrict is like Tag, but rejects a partial tree when parsing stops before it accepts all input. It releases the partial tree and returns an error that wraps ErrParseStoppedEarly.

func (*Tagger) TagTree ¶

func (tg *Tagger) TagTree(tree *Tree) []Tag

TagTree extracts tags from an already-parsed tree.

func (*Tagger) TagTreeUTF16 ¶ added in v0.16.0

func (tg *Tagger) TagTreeUTF16(tree *Tree) []UTF16Tag

TagTreeUTF16 extracts tags from an already-parsed UTF-16 tree.

func (*Tagger) TagUTF16 ¶ added in v0.16.0

func (tg *Tagger) TagUTF16(source []uint16) []UTF16Tag

TagUTF16 parses UTF-16 source and returns all tags with UTF-16 ranges.

func (*Tagger) TagUTF16Bytes ¶ added in v0.16.0

func (tg *Tagger) TagUTF16Bytes(source []byte, order UTF16ByteOrder) ([]UTF16Tag, error)

TagUTF16Bytes is like TagUTF16 for endian-specific UTF-16 bytes.

type TaggerOption ¶

type TaggerOption func(*Tagger)

TaggerOption configures a Tagger.

func WithTaggerTimeoutMicros ¶ added in v0.37.0

func WithTaggerTimeoutMicros(timeoutMicros uint64) TaggerOption

WithTaggerTimeoutMicros bounds every full and incremental parse performed by the tagger. A value of zero disables timeout checks.

func WithTaggerTokenSourceFactory ¶

func WithTaggerTokenSourceFactory(factory func(source []byte) TokenSource) TaggerOption

WithTaggerTokenSourceFactory sets a factory function that creates a TokenSource for each Tag call.

type Token ¶

type Token struct {
	// Field order packs the three exported flags, the lex flag byte, and the
	// 16-bit symbol into one word so Token stays at 64 bytes. Tokens are
	// copied by value on every election and dispatch, so the size shows up
	// directly as copy cost.
	Text       string
	StartByte  uint32
	EndByte    uint32
	StartPoint Point
	EndPoint   Point

	// ExternalScannerStartByte is the byte offset where that scanner call
	// began, before scanner-side skip advances moved StartByte forward.
	ExternalScannerStartByte uint32

	Symbol  Symbol
	Missing bool
	// NoLookahead marks a synthetic EOF used to force EOF-table reductions
	// without consuming input, matching tree-sitter's lex_state = -1.
	NoLookahead bool
	// ExternalScannerToken marks tokens produced by an external scanner.
	ExternalScannerToken bool
	// contains filtered or unexported fields
}

Token is a lexed token with position info.

type TokenSource ¶

type TokenSource interface {
	// Next returns the next token. It should skip whitespace and comments
	// as appropriate for the language. Returns a zero-Symbol token at EOF.
	Next() Token
}

TokenSource provides tokens to the parser. This interface abstracts over different lexer implementations: the built-in DFA lexer (for hand-built grammars) or custom bridges like GoTokenSource (for real grammars where we can't extract the C lexer DFA).

type TokenSourceFactory ¶ added in v0.16.0

type TokenSourceFactory func(source []byte) (TokenSource, error)

TokenSourceFactory builds a token source for parser source bytes.

type TokenSourceRebuilder ¶ added in v0.7.0

type TokenSourceRebuilder interface {
	RebuildTokenSource(source []byte, lang *Language) (TokenSource, error)
}

TokenSourceRebuilder is an optional extension for token sources that can build a fresh equivalent token source for another source buffer. Result normalization uses this to reparse isolated fragments with the same lexer backend as the original parse.

type Tree ¶

type Tree struct {
	// contains filtered or unexported fields
}

Tree holds a complete syntax tree along with its source text and language. Tree is safe for concurrent reads after construction. Edit and Release are not safe for concurrent use. Parser.ParseIncremental writes to its old tree, so do not read the old tree from another goroutine during that call.

func NewTree ¶

func NewTree(root *Node, source []byte, lang *Language) *Tree

NewTree creates a new Tree.

func (*Tree) ArenaBreakdown ¶ added in v0.18.0

func (t *Tree) ArenaBreakdown() (ArenaBreakdown, bool)

ArenaBreakdown returns optional arena/materialization attribution captured when EnableArenaBreakdown(true) was set before parsing.

func (*Tree) ChangedRanges ¶ added in v0.6.0

func (t *Tree) ChangedRanges() []Range

ChangedRanges converts this tree's recorded edits into changed source ranges, in the coordinates of the final source (the state after every recorded edit has been applied). Overlapping ranges are coalesced.

func (*Tree) Copy ¶ added in v0.7.0

func (t *Tree) Copy() *Tree

Copy returns an independent copy of this tree.

The copied tree has distinct node objects, so subsequent Tree.Edit calls on either tree do not mutate the other's spans/dirty bits. Source bytes and language pointer are shared (read-only).

Copy is not read-only on the source. It runs the column-dependency fold first, which writes the folded bits into the source arena's side table and releases that arena's span list, exactly as a first Tree.Edit would. The source tree's public shape does not change.

func (*Tree) DOT ¶ added in v0.7.0

func (t *Tree) DOT(lang *Language) string

DOT returns a DOT graph representation of this tree.

func (*Tree) DescendantForUTF16Range ¶ added in v0.16.0

func (t *Tree) DescendantForUTF16Range(startCodeUnit, endCodeUnit uint32) *Node

DescendantForUTF16Range returns the smallest descendant that fully contains the given UTF-16 code-unit range, or nil when no such descendant exists.

func (*Tree) Edit ¶

func (t *Tree) Edit(edit InputEdit)

Edit records an edit on this tree. Call this before ParseIncremental to inform the parser which regions changed. The edit adjusts byte offsets and marks overlapping nodes as dirty so the incremental parser knows what to re-parse. Does nothing for a nil tree.

func (*Tree) EditUTF16 ¶ added in v0.16.0

func (t *Tree) EditUTF16(edit UTF16Edit, newSource []uint16) bool

EditUTF16 records a UTF-16 code-unit edit on a UTF-16 tree.

newSource is the full source after the edit; it is used to derive the internal UTF-8 endpoint for NewEndCodeUnit.

func (*Tree) Edits ¶

func (t *Tree) Edits() []InputEdit

Edits returns the pending edits recorded on this tree. Returns nil for a nil tree.

func (*Tree) EnclosingDefinition ¶ added in v0.20.6

func (t *Tree) EnclosingDefinition(byteOffset uint32) (DefinitionSpan, bool)

EnclosingDefinition returns the nearest definition node that contains byteOffset.

func (*Tree) InputEditForUTF16 ¶ added in v0.16.0

func (t *Tree) InputEditForUTF16(edit UTF16Edit, newSource []uint16) (InputEdit, bool)

InputEditForUTF16 converts a UTF-16 code-unit edit into the parser's internal UTF-8 byte-coordinate edit. The tree must have been produced by ParseUTF16.

func (*Tree) Language ¶

func (t *Tree) Language() *Language

Language returns the language used to parse this tree. Returns nil for a nil tree.

func (*Tree) NamedDescendantForUTF16Range ¶ added in v0.16.0

func (t *Tree) NamedDescendantForUTF16Range(startCodeUnit, endCodeUnit uint32) *Node

NamedDescendantForUTF16Range returns the smallest named descendant that fully contains the given UTF-16 code-unit range, or nil when no such descendant exists.

func (*Tree) NamedNodeAtByte ¶ added in v0.20.6

func (t *Tree) NamedNodeAtByte(byteOffset uint32) *Node

NamedNodeAtByte returns the smallest named root descendant that contains byteOffset.

func (*Tree) NodeAtByte ¶ added in v0.20.6

func (t *Tree) NodeAtByte(byteOffset uint32) *Node

NodeAtByte returns the smallest root descendant that contains byteOffset.

func (*Tree) ParseRuntime ¶ added in v0.6.0

func (t *Tree) ParseRuntime() ParseRuntime

ParseRuntime returns parser-loop diagnostics captured when this tree was built.

func (*Tree) ParseStopReason ¶ added in v0.6.0

func (t *Tree) ParseStopReason() ParseStopReason

ParseStopReason reports why parsing terminated.

func (*Tree) ParseStoppedEarly ¶ added in v0.6.0

func (t *Tree) ParseStoppedEarly() bool

ParseStoppedEarly reports whether parsing hit an early-stop condition.

func (*Tree) RecoveryNodeMemoRuntime ¶ added in v0.49.0

func (t *Tree) RecoveryNodeMemoRuntime() RecoveryNodeMemoRuntime

RecoveryNodeMemoRuntime returns bounded memo telemetry for this tree. The collision count saturates at the uint32 maximum.

func (*Tree) Release ¶

func (t *Tree) Release()

Release decrements arena references held by this tree. After Release, the tree should be treated as invalid and not reused.

Release once for each tree that a parse returns. An unchanged incremental parse can return its old tree, and that return adds a handle. The tree stays valid until its last handle is released. A call on an already released tree does nothing.

func (*Tree) RootNode ¶

func (t *Tree) RootNode() *Node

RootNode returns the tree's root node. Returns nil for a nil tree.

func (*Tree) RootNodeWithOffset ¶ added in v0.7.0

func (t *Tree) RootNodeWithOffset(offsetBytes uint32, offsetExtent Point) *Node

RootNodeWithOffset returns a copy of the root node with all spans shifted by the provided byte and point offsets.

This mirrors tree-sitter C's root-node-with-offset behavior for callers that need to embed a parsed tree at a larger document offset.

func (*Tree) Source ¶

func (t *Tree) Source() []byte

Source returns the original source text. Returns nil for a nil tree.

func (*Tree) SourceEncoding ¶ added in v0.16.0

func (t *Tree) SourceEncoding() InputEncoding

SourceEncoding returns the encoding used by the caller that produced this tree.

For UTF-16 parses, Source still returns the parser's canonical UTF-8 copy. Use SourceUTF16 and UTF16RangeForNode when caller-facing UTF-16 coordinates are needed.

func (*Tree) SourceUTF16 ¶ added in v0.16.0

func (t *Tree) SourceUTF16() []uint16

SourceUTF16 returns the original UTF-16 source for trees produced by ParseUTF16. It returns nil for ordinary UTF-8 parses.

func (*Tree) UTF8ByteForUTF16Offset ¶ added in v0.16.0

func (t *Tree) UTF8ByteForUTF16Offset(offset uint32) (uint32, bool)

UTF8ByteForUTF16Offset converts a UTF-16 code-unit offset to the parser's canonical UTF-8 byte offset for trees produced by ParseUTF16.

func (*Tree) UTF16OffsetForByte ¶ added in v0.16.0

func (t *Tree) UTF16OffsetForByte(offset uint32) (uint32, bool)

UTF16OffsetForByte converts a parser UTF-8 byte offset to a UTF-16 code-unit offset for trees produced by ParseUTF16.

func (*Tree) UTF16PointForByte ¶ added in v0.16.0

func (t *Tree) UTF16PointForByte(offset uint32) (Point, bool)

UTF16PointForByte converts a parser UTF-8 byte offset to a UTF-16 point.

func (*Tree) UTF16RangeForByteRange ¶ added in v0.16.0

func (t *Tree) UTF16RangeForByteRange(startByte, endByte uint32) (UTF16Range, bool)

UTF16RangeForByteRange converts a canonical UTF-8 byte range into UTF-16 code-unit coordinates.

func (*Tree) UTF16RangeForNode ¶ added in v0.16.0

func (t *Tree) UTF16RangeForNode(n *Node) (UTF16Range, bool)

UTF16RangeForNode returns a node range in UTF-16 code-unit coordinates.

func (*Tree) UTF16RangeForRange ¶ added in v0.16.0

func (t *Tree) UTF16RangeForRange(r Range) (UTF16Range, bool)

UTF16RangeForRange converts a canonical UTF-8 Range into UTF-16 code-unit coordinates.

func (*Tree) UTF16SourceForNode ¶ added in v0.16.0

func (t *Tree) UTF16SourceForNode(n *Node) ([]uint16, bool)

UTF16SourceForNode returns the original UTF-16 code units covered by n.

func (*Tree) UsedForestFastPath ¶ added in v0.49.0

func (t *Tree) UsedForestFastPath() bool

UsedForestFastPath reports whether the node data behind this tree was produced by the GSS-forest GLR fast path (Parser.Parse trying tryForestFastPath before the production loop, or a direct ParseForestExperimental call) rather than the production parser. Unlike LanguageWantsForest (which only reports whether a language is eligible for the fast path), this reports what actually produced the data for a FRESH parse. It is provenance of the data, not necessarily of this exact call: reuseTreeWithNewSource (incremental_leaf_fastpath.go) copies the field from the old tree onto a new *Tree sharing the old root when reusing a tree incrementally, so an incrementally-reused tree reports whichever route produced the shared data, not whether this particular reuse call itself dispatched to forest (it did not run parser dispatch at all). Regression gates that always call Parser.Parse fresh (never ParseIncremental) are unaffected by this distinction; callers mixing in incremental reuse should not read this as "this call routed through forest."

func (*Tree) WriteDOT ¶ added in v0.7.0

func (t *Tree) WriteDOT(w io.Writer, lang *Language) error

WriteDOT writes a DOT graph representation of this tree to w.

type TreeCursor ¶ added in v0.6.0

type TreeCursor struct {
	// contains filtered or unexported fields
}

TreeCursor provides stateful, O(1) tree navigation. It maintains a stack of (node, childIndex) frames enabling efficient parent, child, and sibling movement without scanning.

The cursor holds pointers to Nodes. If the underlying Tree is released, edited, or replaced via incremental reparse, the cursor should be recreated.

func NewTreeCursor ¶ added in v0.6.0

func NewTreeCursor(node *Node, tree *Tree) *TreeCursor

NewTreeCursor creates a cursor starting at the given node. The optional tree reference enables field name resolution and text extraction.

func NewTreeCursorFromTree ¶ added in v0.6.0

func NewTreeCursorFromTree(tree *Tree) *TreeCursor

NewTreeCursorFromTree creates a cursor starting at the tree's root node.

func (*TreeCursor) Copy ¶ added in v0.6.0

func (c *TreeCursor) Copy() *TreeCursor

Copy returns an independent copy of the cursor. The copy shares the same tree reference but has its own navigation stack. Returns nil for a nil cursor.

func (*TreeCursor) CurrentFieldID ¶ added in v0.6.0

func (c *TreeCursor) CurrentFieldID() FieldID

CurrentFieldID returns the field ID of the current node within its parent. Returns 0 if the cursor is nil, is at the root, or the node has no field assignment.

func (*TreeCursor) CurrentFieldName ¶ added in v0.6.0

func (c *TreeCursor) CurrentFieldName() string

CurrentFieldName returns the field name of the current node within its parent. Returns "" if the cursor is nil, no tree is associated, the cursor is at the root, or the node has no field assignment.

func (*TreeCursor) CurrentNode ¶ added in v0.6.0

func (c *TreeCursor) CurrentNode() *Node

CurrentNode returns the node the cursor is currently pointing to. Returns nil for a nil cursor.

func (*TreeCursor) CurrentNodeIsNamed ¶ added in v0.6.0

func (c *TreeCursor) CurrentNodeIsNamed() bool

CurrentNodeIsNamed returns whether the current node is a named node.

func (*TreeCursor) CurrentNodeText ¶ added in v0.6.0

func (c *TreeCursor) CurrentNodeText() string

CurrentNodeText returns the source text of the current node. Requires a tree with source to be associated. Returns "" for a nil cursor.

func (*TreeCursor) CurrentNodeType ¶ added in v0.6.0

func (c *TreeCursor) CurrentNodeType() string

CurrentNodeType returns the type name of the current node. Requires a tree with a language to be associated. Returns "" for a nil cursor.

func (*TreeCursor) Depth ¶ added in v0.6.0

func (c *TreeCursor) Depth() int

Depth returns the cursor's current depth (0 at the root). Returns 0 for a nil cursor.

func (*TreeCursor) GotoChildByFieldID ¶ added in v0.6.0

func (c *TreeCursor) GotoChildByFieldID(fid FieldID) bool

GotoChildByFieldID moves the cursor to the first child with the given field ID. Returns false if no child has that field.

func (*TreeCursor) GotoChildByFieldName ¶ added in v0.6.0

func (c *TreeCursor) GotoChildByFieldName(name string) bool

GotoChildByFieldName moves the cursor to the first child with the given field name. Returns false if the cursor is nil, the tree has no language, the field name is unknown, or no child has that field.

func (*TreeCursor) GotoFirstChild ¶ added in v0.6.0

func (c *TreeCursor) GotoFirstChild() bool

GotoFirstChild moves the cursor to the first child of the current node. Returns false if the current node has no children.

func (*TreeCursor) GotoFirstChildForByte ¶ added in v0.6.0

func (c *TreeCursor) GotoFirstChildForByte(targetByte uint32) int64

GotoFirstChildForByte moves the cursor to the first child whose byte range contains targetByte (i.e., first child where endByte > targetByte). Returns the child index, or -1 when no child contains the byte.

func (*TreeCursor) GotoFirstChildForPoint ¶ added in v0.6.0

func (c *TreeCursor) GotoFirstChildForPoint(targetPoint Point) int64

GotoFirstChildForPoint moves the cursor to the first child whose point range contains targetPoint (i.e., first child where endPoint > targetPoint). Returns the child index, or -1 when no child contains the point.

func (*TreeCursor) GotoFirstNamedChild ¶ added in v0.6.0

func (c *TreeCursor) GotoFirstNamedChild() bool

GotoFirstNamedChild moves the cursor to the first named child of the current node, skipping anonymous nodes. Returns false if no named child exists.

func (*TreeCursor) GotoLastChild ¶ added in v0.6.0

func (c *TreeCursor) GotoLastChild() bool

GotoLastChild moves the cursor to the last child of the current node. Returns false if the current node has no children.

func (*TreeCursor) GotoLastNamedChild ¶ added in v0.6.0

func (c *TreeCursor) GotoLastNamedChild() bool

GotoLastNamedChild moves the cursor to the last named child of the current node, skipping anonymous nodes. Returns false if no named child exists.

func (*TreeCursor) GotoNextNamedSibling ¶ added in v0.6.0

func (c *TreeCursor) GotoNextNamedSibling() bool

GotoNextNamedSibling moves the cursor to the next named sibling, skipping anonymous nodes. Returns false if no named sibling follows, or the cursor is nil.

func (*TreeCursor) GotoNextSibling ¶ added in v0.6.0

func (c *TreeCursor) GotoNextSibling() bool

GotoNextSibling moves the cursor to the next sibling. Returns false if the cursor is at the root or the last sibling, or the cursor is nil.

func (*TreeCursor) GotoParent ¶ added in v0.6.0

func (c *TreeCursor) GotoParent() bool

GotoParent moves the cursor to the parent of the current node. Returns false if the cursor is at the root, or the cursor is nil.

func (*TreeCursor) GotoPrevNamedSibling ¶ added in v0.6.0

func (c *TreeCursor) GotoPrevNamedSibling() bool

GotoPrevNamedSibling moves the cursor to the previous named sibling, skipping anonymous nodes. Returns false if no named sibling precedes, or the cursor is nil.

func (*TreeCursor) GotoPrevSibling ¶ added in v0.6.0

func (c *TreeCursor) GotoPrevSibling() bool

GotoPrevSibling moves the cursor to the previous sibling. Returns false if the cursor is at the root or the first sibling, or the cursor is nil.

func (*TreeCursor) Reset ¶ added in v0.6.0

func (c *TreeCursor) Reset(node *Node)

Reset resets the cursor to a new root node, clearing the navigation stack. Does nothing for a nil cursor.

func (*TreeCursor) ResetTree ¶ added in v0.6.0

func (c *TreeCursor) ResetTree(tree *Tree)

ResetTree resets the cursor to the root of a new tree. Does nothing for a nil cursor.

type UTF16ByteOrder ¶ added in v0.16.0

type UTF16ByteOrder uint8

UTF16ByteOrder identifies the byte order used by a UTF-16 byte source.

const (
	UTF16LittleEndian UTF16ByteOrder = iota
	UTF16BigEndian
)

func (UTF16ByteOrder) String ¶ added in v0.16.0

func (o UTF16ByteOrder) String() string

type UTF16Edit ¶ added in v0.16.0

type UTF16Edit struct {
	StartCodeUnit  uint32
	OldEndCodeUnit uint32
	NewEndCodeUnit uint32
}

UTF16Edit describes a source edit in UTF-16 code-unit offsets.

type UTF16HighlightRange ¶ added in v0.16.0

type UTF16HighlightRange struct {
	StartCodeUnit uint32
	EndCodeUnit   uint32
	StartPoint    Point
	EndPoint      Point
	Capture       string
	PatternIndex  int
}

UTF16HighlightRange is a styled source range in UTF-16 code-unit coordinates.

type UTF16Injection ¶ added in v0.16.0

type UTF16Injection struct {
	// Language is the detected language name (e.g., "javascript").
	Language string
	// Tree is the parse tree for this region, or nil if the language
	// was not registered.
	Tree *Tree
	// Ranges are the source ranges this tree covers in UTF-16 code units.
	Ranges []UTF16Range
	// Node is the parent tree node that triggered the injection.
	Node *Node
}

UTF16Injection is a single embedded language region with ranges in UTF-16 code-unit coordinates.

type UTF16InjectionResult ¶ added in v0.16.0

type UTF16InjectionResult struct {
	// Tree is the parent language's parse tree.
	Tree *Tree
	// Injections contains child language parse results, ordered by position.
	Injections []UTF16Injection
	// contains filtered or unexported fields
}

UTF16InjectionResult holds parse results for a UTF-16 multi-language document. Injection ranges are expressed in UTF-16 code units.

type UTF16Range ¶ added in v0.16.0

type UTF16Range struct {
	StartCodeUnit uint32
	EndCodeUnit   uint32
	StartPoint    Point
	EndPoint      Point
}

UTF16Range is a source range in UTF-16 code units.

StartPoint and EndPoint use UTF-16 code-unit columns, matching the coordinate system used by many editors and LSP clients.

type UTF16Tag ¶ added in v0.16.0

type UTF16Tag struct {
	Kind      string
	Name      string
	Range     UTF16Range
	NameRange UTF16Range
}

UTF16Tag represents a tagged symbol with ranges in UTF-16 code-unit coordinates.

type UnaryWrapperFlatteningRule ¶ added in v0.48.0

type UnaryWrapperFlatteningRule struct {
	PublicParent        Symbol
	Wrapper             Symbol
	Leaf                Symbol
	WrapperPreGotoState StateID
}

UnaryWrapperFlatteningRule identifies one exact public-parent, wrapper, leaf, and parser-state chain that native materialization must flatten. Runtime profiles attach these rules only to certified grammar artifacts.

type WalkAction ¶

type WalkAction int

WalkAction controls the tree walk behavior.

const (
	// WalkContinue continues the walk to children and siblings.
	WalkContinue WalkAction = iota
	// WalkSkipChildren skips the current node's children but continues to siblings.
	WalkSkipChildren
	// WalkStop terminates the walk entirely.
	WalkStop
)

Source Files ¶

Directories ¶

Path Synopsis
cgo_harness module
cmd
benchgate command
benchmatrix command
c4tablestats command
Command c4tablestats is the C4 stage-2 table-shape analyzer.
Command c4tablestats is the C4 stage-2 table-shape analyzer.
citestplan command
Command citestplan checks host race-build test packages against the CI workflow.
Command citestplan checks host race-build test packages against the CI workflow.
crecoverygatefleet command
Command crecoverygatefleet is the C-recovery cost-competition gate's fleet receipt generator (task #71, item 2).
Command crecoverygatefleet is the C-recovery cost-competition gate's fleet receipt generator (task #71, item 2).
gen_grammar_packages command
Command gen_grammar_packages generates standalone grammar packages and compatibility declarations.
Command gen_grammar_packages generates standalone grammar packages and compatibility declarations.
gen_license_notices command
Command gen_license_notices regenerates THIRD_PARTY_NOTICES from licenses/grammars.json, the confirmed per-grammar license audit for every upstream tree-sitter grammar gotreesitter vendors.
Command gen_license_notices regenerates THIRD_PARTY_NOTICES from licenses/grammars.json, the confirmed per-grammar license audit for every upstream tree-sitter grammar gotreesitter vendors.
gen_linguist command
Command gen_linguist generates grammars/linguist_gen.go by matching gotreesitter grammar names to GitHub Linguist's languages.yml.
Command gen_linguist generates grammars/linguist_gen.go by matching gotreesitter grammar names to GitHub Linguist's languages.yml.
gen_subset_blob_embeds command
Command gen_subset_blob_embeds generates the per-language z_subset_blob_embed_<lang>.go files that power embedded grammar_subset builds (issue #88: per-language compile-time grammar selection).
Command gen_subset_blob_embeds generates the per-language z_subset_blob_embed_<lang>.go files that power embedded grammar_subset builds (issue #88: per-language compile-time grammar selection).
grammar_update_guard command
Command grammar_update_guard checks lock-update reports for scanner-facing changes that require hand-written scanner review before grammar blobs move.
Command grammar_update_guard checks lock-update reports for scanner-facing changes that require hand-written scanner review before grammar blobs move.
grammar_updater command
Command grammar_updater refreshes pinned grammar commits in grammars/languages.lock and emits a machine-readable update report.
Command grammar_updater refreshes pinned grammar commits in grammars/languages.lock and emits a machine-readable update report.
grammarblobprobe command
Command grammarblobprobe is a minimal binary that blank-imports the grammars package so that whatever grammar blobs are embedded by the active build tags are linked into the binary.
Command grammarblobprobe is a minimal binary that blank-imports the grammars package so that whatever grammar blobs are embedded by the active build tags are linked into the binary.
grammargen command
Command grammargen generates tree-sitter parser artifacts from grammar definitions.
Command grammargen generates tree-sitter parser artifacts from grammar definitions.
harnessgate command
issue454bench command
Command issue454bench reproduces the downstream editor measurements from issue #454 on synthetic single-language fixtures.
Command issue454bench reproduces the downstream editor measurements from issue #454 on synthetic single-language fixtures.
parity_report command
pgo_repdriver command
Command pgo_repdriver is a representative multi-grammar parsing workload used to (a) collect a CPU profile for Go build-time PGO and (b) produce a stable digest of parsed trees for before/after correctness comparisons.
Command pgo_repdriver is a representative multi-grammar parsing workload used to (a) collect a CPU profile for Go build-time PGO and (b) produce a stable digest of parsed trees for before/after correctness comparisons.
releasegate command
ts2go command
Command ts2go reads a tree-sitter generated parser.c file and outputs a Go source file containing a function that returns a populated *gotreesitter.Language with all extracted parse tables.
Command ts2go reads a tree-sitter generated parser.c file and outputs a Go source file containing a function that returns a populated *gotreesitter.Language with all extracted parse tables.
tsquery command
Command tsquery generates type-safe Go code from tree-sitter .scm query files.
Command tsquery generates type-safe Go code from tree-sitter .scm query files.
wasmassets command
Command wasmassets builds the persistent gotreesitter WASM runtime and emits a self-contained browser asset bundle for one registered language.
Command wasmassets builds the persistent gotreesitter WASM runtime and emits a self-contained browser asset bundle for one registered language.
Package corpuscheck parses the upstream tree-sitter corpus test format.
Package corpuscheck parses the upstream tree-sitter corpus test format.
cmd/corpuscheck command
Command corpuscheck runs gotreesitter against upstream tree-sitter corpora.
Command corpuscheck runs gotreesitter against upstream tree-sitter corpora.
Package grammargen implements a pure-Go grammar generator for gotreesitter.
Package grammargen implements a pure-Go grammar generator for gotreesitter.
grammarjs
Package grammarjs wires grammargen's grammar.js importer to gotreesitter's embedded JavaScript grammar.
Package grammarjs wires grammargen's grammar.js importer to gotreesitter's embedded JavaScript grammar.
Package grammars provides built-in and extension tree-sitter grammars with lazy loading.
Package grammars provides built-in and extension tree-sitter grammars with lazy loading.
ada
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
agda
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
angular
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
apex
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
arduino
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
asm
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
astro
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
authzed
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
awk
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
bash
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
bass
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
beancount
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
bibtex
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
bicep
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
bitbake
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
blade
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
brightscript
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
c
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
c_sharp
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
caddy
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
cairo
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
capnp
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
chatito
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
circom
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
clojure
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
cmake
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
cobol
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
comment
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
commonlisp
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
cooklang
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
corn
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
cpon
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
cpp
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
crystal
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
css
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
csv
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
cuda
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
cue
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
cylc
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
d
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
dart
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
desktop
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
devicetree
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
dhall
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
diff
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
disassembly
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
djot
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
dockerfile
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
dot
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
doxygen
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
dtd
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
earthfile
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
ebnf
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
editorconfig
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
eds
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
eex
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
elisp
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
elixir
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
elm
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
elsa
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
embedded_template
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
enforce
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
erlang
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
facility
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
faust
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
fennel
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
fidl
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
firrtl
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
fish
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
foam
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
forth
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
fortran
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
fsharp
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
gdscript
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
git_config
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
git_rebase
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
gitattributes
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
gitcommit
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
gitignore
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
gleam
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
glsl
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
gn
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
go
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
godot_resource
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
gomod
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
grammar_blobs
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
graphql
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
groovy
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
hack
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
hare
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
haskell
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
haxe
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
hcl
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
heex
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
hlsl
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
html
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
http
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
hurl
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
hyprlang
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
ini
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
internal/standaloneregistry
Package standaloneregistry records opt-in grammar metadata without importing the aggregate grammar catalog.
Package standaloneregistry records opt-in grammar metadata without importing the aggregate grammar catalog.
janet
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
java
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
javascript
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
jinja2
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
jq
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
jsdoc
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
json
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
json5
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
jsonnet
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
julia
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
just
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
kconfig
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
kdl
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
kotlin
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
lean
Package lean provides the native gotreesitter grammar for Lean 4.
Package lean provides the native gotreesitter grammar for Lean 4.
ledger
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
less
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
linkerscript
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
liquid
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
llvm
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
lua
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
luau
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
make
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
markdown
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
markdown_inline
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
matlab
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
mermaid
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
meson
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
mojo
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
move
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
nginx
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
nickel
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
nim
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
ninja
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
nix
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
norg
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
nushell
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
objc
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
ocaml
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
odin
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
org
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
pascal
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
pem
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
perl
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
php
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
pkl
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
powershell
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
prisma
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
prolog
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
promql
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
properties
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
proto
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
pug
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
puppet
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
purescript
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
python
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
ql
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
r
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
racket
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
regex
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
rego
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
requirements
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
rescript
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
robot
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
ron
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
rst
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
ruby
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
runtime
Package grammarruntime supplies shared support for packaged grammars.
Package grammarruntime supplies shared support for packaged grammars.
rust
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
scala
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
scheme
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
scss
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
smithy
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
solidity
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
sparql
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
sql
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
squirrel
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
ssh_config
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
starlark
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
svelte
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
swift
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
tablegen
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
tcl
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
teal
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
templ
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
textproto
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
thrift
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
tlaplus
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
tmux
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
todotxt
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
toml
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
tsx
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
turtle
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
twig
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
typescript
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
typst
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
uxntal
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
v
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
verilog
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
vhdl
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
vimdoc
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
vue
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
wat
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
wgsl
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
wolfram
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
xml
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
yaml
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
yuck
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
zig
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Code generated by cmd/gen_grammar_packages; DO NOT EDIT.
Package grep provides structural code search, match, and rewrite using tree-sitter parse trees.
Package grep provides structural code search, match, and rewrite using tree-sitter parse trees.
internal
benchfixtures
Package benchfixtures provides immutable, content-addressed benchmark inputs.
Package benchfixtures provides immutable, content-addressed benchmark inputs.
grammarpatch
Package grammarpatch declares pinned upstream grammar overlays.
Package grammarpatch declares pinned upstream grammar overlays.
luapattern
Package luapattern translates the Lua pattern subset used by tree-sitter queries.
Package luapattern translates the Lua pattern subset used by tree-sitter queries.
parsercorephase0
Package parsercorephase0 contains the admitted compact parser core.
Package parsercorephase0 contains the admitted compact parser core.
Package taproot is the common front-end harness shared by M31 DSLs that use the gotreesitter runtime.
Package taproot is the common front-end harness shared by M31 DSLs that use the gotreesitter runtime.
diag
Package diag provides a generic structured diagnostic type and a source-quoting renderer.
Package diag provides a generic structured diagnostic type and a source-quoting renderer.
walk
Package walk is the grammar-free core of taproot: load a tree-sitter Language from a pre-generated blob and navigate the CST with a Walker.
Package walk is the grammar-free core of taproot: load a tree-sitter Language from a pre-generated blob and navigate the CST with a Walker.
wasm
grammargen command
runtime command

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL