parser

package
v0.5.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 8, 2026 License: MIT Imports: 2 Imported by: 0

Documentation

Overview

Package parser defines a narrow, backend-swappable boundary for turning source bytes into a syntax tree, independent of storage concerns.

Index

Constants

View Source
const MaxSourceBytes = 4 * 1024 * 1024 // 4 MiB

MaxSourceBytes is the default file-size ceiling enforced before any bytes reach a backend's underlying parser. This is the mitigation for the pathological-input denial-of-service threat (Security Domain V5): a single multi-megabyte minified line handed to a tree-sitter grammar's external C scanner can consume unbounded time/memory. Implementations MUST reject input over this ceiling (or a caller-configured ceiling) with ErrSourceTooLarge BEFORE invoking the underlying C/WASM parser.

Variables

View Source
var ErrSourceTooLarge = errors.New("parser: source exceeds size ceiling")

ErrSourceTooLarge is returned by Parse when len(source) exceeds the configured ceiling. Implementations MUST return this sentinel (checked via errors.Is) without calling into the underlying C/WASM parser.

Functions

This section is empty.

Types

type Parser

type Parser interface {
	// Parse produces a Tree from source bytes. If oldTree is non-nil,
	// implementations SHOULD perform an incremental reparse using it as a
	// starting point. Implementations MUST enforce the size ceiling
	// (returning ErrSourceTooLarge) before doing any backend-specific
	// parsing work.
	Parse(source []byte, oldTree *Tree) (*Tree, error)

	// Close releases any resources (C memory, WASM instances) held by the
	// parser. Safe to call once; implementations should make repeat calls
	// a no-op rather than double-free.
	Close() error
}

Parser is the narrow, backend-swappable seam between arbitrary source bytes and a syntax tree (D-05b). Both the CGo tree-sitter backend (internal/parser/cgo) and any future wazero-WASM backend implement this exact shape so callers (storage, extractors) never depend on a specific backend's types.

Security contract (Security Domain V5 / threat T-01-03): implementations MUST reject source input larger than their configured ceiling (see MaxSourceBytes) by returning ErrSourceTooLarge, and MUST perform that check BEFORE handing the bytes to the underlying C or WASM parser. This bounds the pathological-input denial-of-service surface (e.g. a single multi-megabyte minified line) at the seam, before any backend-specific parsing cost is paid.

Crash-isolation contract (threat T-01-01, accepted for Phase 1): a C-level segfault inside a tree-sitter grammar's external scanner (for example Python's INDENT/DEDENT scanner) runs in-process with no memory isolation from the Go host. This is NOT a Go panic and is NOT recoverable via Go's recover() — callers and implementations MUST NOT design any crash-isolation logic around recover() for this path. The CGo backend's crash-isolation behavior under adversarial input is measured (not eliminated) by the Plan 01-07 spike; a subprocess isolation boundary remains a Phase 2+ option if that risk proves unacceptable in practice.

Resource contract: implementations allocate C or WASM-backed memory for parsers/trees. Callers MUST call Close() exactly once when done with a Parser to free those resources; Close() must be safe to call even if no Parse call was made. Callers MUST ALSO call Close() on every *Tree returned by Parse exactly once when done with it — including a superseded oldTree once an incremental reparse produces its replacement — to free the tree's own native memory, which is distinct from and not released by Parser.Close().

type Tree

type Tree struct {
	// contains filtered or unexported fields
}

Tree is a backend-neutral syntax tree handle. Call sites never touch a backend-specific tree type directly (e.g. *tree_sitter.Tree) — every backend wraps its native tree behind this opaque type so storage and future extractors can be written against a single shape regardless of which Parser implementation produced the tree.

func NewTree

func NewTree(inner any, closeFn func()) *Tree

NewTree wraps a backend-specific tree value in the shared, opaque Tree type. Backend packages call this after a successful parse. closeFn, if non-nil, is invoked exactly once by Close() to release the backend's native tree resources (e.g. C-allocated tree-sitter memory).

func (*Tree) Close

func (t *Tree) Close() error

Close releases any backend-native resources (e.g. C-allocated tree-sitter memory) held by this Tree. Safe to call on a nil Tree or a Tree with no associated closeFn, and safe to call more than once (sequentially or concurrently): closeFn is invoked at most once. Callers MUST call Close() exactly once when a Tree is discarded, including when it is superseded by a newer tree produced via incremental reparse — this guard exists as a backstop against that discipline slipping, not as license to skip it.

func (*Tree) Inner

func (t *Tree) Inner() any

Inner returns the backend-specific tree value that was wrapped by NewTree. Callers that need to unwrap it must type-assert against the concrete type their own backend produced.

Directories

Path Synopsis
Package cgo implements the production CGo tree-sitter backend (Option A) for the parser.Parser interface, covering the two spike-partner languages (D-05): Go (LANG-01) and Python (carries a real external C scanner).
Package cgo implements the production CGo tree-sitter backend (Option A) for the parser.Parser interface, covering the two spike-partner languages (D-05): Go (LANG-01) and Python (carries a real external C scanner).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL