Help us shape the future of Go! Take the 2026 Go Developer Survey

parser

package
v0.9.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 12, 2026 License: MIT Imports: 2 Imported by: 0

Documentation

Overview

Package parser defines a narrow, backend-swappable boundary for turning source bytes into a syntax tree, independent of storage concerns.

Index

Constants

View Source
const MaxSourceBytes = 4 * 1024 * 1024 // 4 MiB

MaxSourceBytes is the default file-size ceiling enforced before any bytes reach a backend's underlying parser. This is the mitigation for the pathological-input denial-of-service threat (Security Domain V5): a single multi-megabyte minified line handed to a tree-sitter grammar's external C scanner can consume unbounded time/memory. Implementations MUST reject input over this ceiling (or a caller-configured ceiling) with ErrSourceTooLarge BEFORE invoking the underlying C/WASM parser.

Variables

View Source
var ErrSourceTooLarge = errors.New("parser: source exceeds size ceiling")

ErrSourceTooLarge is returned by Parse when len(source) exceeds the configured ceiling. Implementations MUST return this sentinel (checked via errors.Is) without calling into the underlying C/WASM parser.

Functions

This section is empty.

Types

type Parser

type Parser interface {
	// Parse produces a Tree from source bytes. If oldTree is non-nil,
	// implementations SHOULD perform an incremental reparse using it as a
	// starting point. Implementations MUST enforce the size ceiling
	// (returning ErrSourceTooLarge) before doing any backend-specific
	// parsing work.
	Parse(source []byte, oldTree *Tree) (*Tree, error)

	// Close releases any resources (C memory, WASM instances) held by the
	// parser. Safe to call once; implementations should make repeat calls
	// a no-op rather than double-free.
	Close() error
}

Parser is the narrow, backend-swappable seam between arbitrary source bytes and a syntax tree (D-05b). Both the CGo tree-sitter backend (internal/parser/cgo) and any future wazero-WASM backend implement this exact shape so callers (storage, extractors) never depend on a specific backend's types.

Security contract (Security Domain V5 / threat T-01-03): implementations MUST reject source input larger than their configured ceiling (see MaxSourceBytes) by returning ErrSourceTooLarge, and MUST perform that check BEFORE handing the bytes to the underlying C or WASM parser. This bounds the pathological-input denial-of-service surface (e.g. a single multi-megabyte minified line) at the seam, before any backend-specific parsing cost is paid.

Crash-isolation contract (threat T-01-01, accepted for Phase 1): a C-level segfault inside a tree-sitter grammar's external scanner (for example Python's INDENT/DEDENT scanner) runs in-process with no memory isolation from the Go host. This is NOT a Go panic and is NOT recoverable via Go's recover() — callers and implementations MUST NOT design any crash-isolation logic around recover() for this path. The CGo backend's crash-isolation behavior under adversarial input is measured (not eliminated) by the Plan 01-07 spike; a subprocess isolation boundary remains a Phase 2+ option if that risk proves unacceptable in practice.

Resource contract: implementations allocate C or WASM-backed memory for parsers/trees. Callers MUST call Close() exactly once when done with a Parser to free those resources; Close() must be safe to call even if no Parse call was made. Callers MUST ALSO call Close() on every *Tree returned by Parse exactly once when done with it — including a superseded oldTree once an incremental reparse produces its replacement — to free the tree's own native memory, which is distinct from and not released by Parser.Close().

type Tree

type Tree struct {
	// contains filtered or unexported fields
}

Tree is a backend-neutral syntax tree handle. Call sites never touch a backend-specific tree type directly (e.g. *tree_sitter.Tree) — every backend wraps its native tree behind this opaque type so storage and future extractors can be written against a single shape regardless of which Parser implementation produced the tree.

func NewTree

func NewTree(inner any, closeFn func()) *Tree

NewTree wraps a backend-specific tree value in the shared, opaque Tree type. Backend packages call this after a successful parse. closeFn, if non-nil, is invoked exactly once by Close() to release the backend's native tree resources (e.g. C-allocated tree-sitter memory).

func (*Tree) Close

func (t *Tree) Close() error

Close releases any backend-native resources (e.g. C-allocated tree-sitter memory) held by this Tree. Safe to call on a nil Tree or a Tree with no associated closeFn, and safe to call more than once (sequentially or concurrently): closeFn is invoked at most once. Callers MUST call Close() exactly once when a Tree is discarded, including when it is superseded by a newer tree produced via incremental reparse — this guard exists as a backstop against that discipline slipping, not as license to skip it.

func (*Tree) Inner

func (t *Tree) Inner() any

Inner returns the backend-specific tree value that was wrapped by NewTree. Callers that need to unwrap it must type-assert against the concrete type their own backend produced.

Directories

Path Synopsis
Package cgo implements the production CGo tree-sitter backend (Option A) for the parser.Parser interface, covering the two spike-partner languages (D-05): Go (LANG-01) and Python (carries a real external C scanner).
Package cgo implements the production CGo tree-sitter backend (Option A) for the parser.Parser interface, covering the two spike-partner languages (D-05): Go (LANG-01) and Python (carries a real external C scanner).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL