parser

package
v1.39.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 17, 2026 License: Apache-2.0 Imports: 27 Imported by: 0

Documentation

Overview

Package parser provides multi-language AST parsing via tree-sitter.

Each supported language has a corresponding grammar library and a set of tree-sitter query files (.scm) in the queries/ subdirectory that extract symbols (functions, types, imports, etc.) from the parsed syntax tree.

CGO_ENABLED=1 is required because tree-sitter grammars are C libraries.

Package parser provides multi-language AST parsing via tree-sitter.

Each supported language has a corresponding grammar library and a set of tree-sitter query files (.scm) in the queries/ subdirectory that extract symbols (functions, types, imports, etc.) from the parsed syntax tree.

CGO_ENABLED=1 is required because tree-sitter grammars are C libraries.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CognitiveComplexity

func CognitiveComplexity(body, language string) int

CognitiveComplexity estimates cognitive complexity from a function body. Uses SonarQube-style rules: decision keywords score +1 + nesting depth, continuations (else if, elif) score flat +1, logical operators score +1 each. For Python: uses indent-based nesting. Language is used to select the strategy.

func Complexity

func Complexity(body, language string) int

Complexity estimates cyclomatic complexity from a function body. Heuristic keyword counting (not full AST), fast and language-agnostic. Comments are stripped before counting. Returns 1 for empty or comment-only bodies (base complexity of any function is 1).

func DetectLanguageFromPath

func DetectLanguageFromPath(path string) string

DetectLanguageFromPath returns the language for a file based on its extension. Looks up the handler registry — single source of truth. extLanguageOverride wins when the handler's Language() doesn't match the canonical name for that extension (e.g. TS handler handles .js files). Falls back to fallbackExtToLanguage for languages with no tree-sitter handler. Returns "" if no handler is registered for the extension.

func NestingDepth

func NestingDepth(body, language string) int

NestingDepth returns the maximum brace nesting depth in a function body. The function's own opening brace is excluded (consistent with CognitiveComplexity). For Python: uses indent-based depth.

func ParseFileWithCalls added in v1.31.6

func ParseFileWithCalls(path string, source []byte, opts ParseOpts) (*ParseResult, []CallSite, error)

ParseFileWithCalls parses a file ONCE and returns BOTH its symbol table (identical to ParseFile) and its call sites (identical to ExtractCalls), sharing a single tree-sitter parse for the mainstream path (issue #400). Every symbols+calls consumer (codegraph, callgraph, explore, analyze, ingest focus) should call this instead of ParseFile followed by ExtractCalls to avoid parsing the same bytes with the same grammar twice.

Handlers whose calls cannot ride the symbol-parse tree (vue/svelte/astro extract a virtual source; html has no grammar) fall back transparently to the independent ParseFile + ExtractCalls pairing, so their output is byte-identical to before. The behavior is proven equivalent across every registered language by TestParseFileWithCalls_EquivalentToSeparate.

func RegisteredExtensions added in v1.31.6

func RegisteredExtensions() []string

RegisteredExtensions returns every file extension with a registered handler, sorted. Used by the parse-equivalence test to assert that every registered language has a fixture proving ParseFileWithCalls matches ParseFile+ExtractCalls, so a newly registered handler cannot ship without that coverage.

func RemapSymbolLines

func RemapSymbolLines(r *ParseResult, vs *preproc.VirtualSource)

RemapSymbolLines rewrites StartLine/EndLine on every Symbol in r from virtual coordinates (positions within vs.Code) to original-file coordinates (positions within the source the virtual code was extracted from).

Side effects (all in place):

  • r.Language is set to vs.Lang.
  • Each retained Symbol has its Language field overwritten with vs.Lang (callers must not rely on the pre-call Language of individual symbols).
  • Symbols whose StartLine maps to 0 in vs.LineMap are padding and are dropped from r.Symbols.
  • Symbols whose EndLine maps to 0 keep their StartLine as EndLine so the range stays non-empty.

Pointer identity of RETAINED symbols is preserved — caller-held *Symbol pointers remain valid but their fields have been mutated.

func SupportedLanguages

func SupportedLanguages() []string

SupportedLanguages returns the deduplicated list of tree-sitter-supported language names, sorted alphabetically. Derives from the handler registry plus extLanguageOverride values — so "javascript" is included even though no handler's Language() returns it (it's served by typescriptHandler). Fallback-only languages (scala, lua, perl, dart, elixir) are NOT included — they have no tree-sitter grammar.

Types

type CallSite

type CallSite struct {
	Name     string // called function or method name
	Receiver string // qualifier for method calls (e.g. "fmt" in "fmt.Println"), empty for plain calls
	Line     uint32 // 1-based line number
	File     string // absolute file path
	// IsArgRef is true when this site comes from a heuristic argument-position
	// or struct-literal-value capture (e.g. `Register("x", handler)` —
	// `handler`). Most of these are plain values, not function references, so
	// the call graph drops unresolved IsArgRef sites unless the caller opts in
	// via the MCP field_access=true flag (filtered in the callgraph layer).
	IsArgRef bool
}

CallSite represents a single function/method call extracted from source code.

func ExtractCalls

func ExtractCalls(path string, source []byte, opts ParseOpts) ([]CallSite, error)

ExtractCalls parses a source file and returns all function/method call sites. Returns empty slice (not error) for unsupported languages.

Preprocessor-language handlers (Astro, Svelte) split call extraction into two single-producer regions — script/frontmatter via ScriptCalls, template body via MarkupCalls — instead of a raw CallsQuery over the whole (non-TS) file, which would double-emit garbled template calls. Every other language uses the raw CallsQuery path.

type Capabilities

type Capabilities struct {
	// SitterLanguage is the tree-sitter grammar. Nil for fallback/regex-only handlers.
	SitterLanguage *sitter.Language

	// TagsQuery is the compiled query for symbol extraction (functions, types, etc.).
	TagsQuery *sitter.Query

	// CallsQuery is the compiled query for call-site extraction. Nil if unsupported.
	CallsQuery *sitter.Query

	// RelationshipsQuery is the compiled query for type relationship extraction. Nil if unsupported.
	RelationshipsQuery *sitter.Query

	// MapCapture converts a single tree-sitter capture into a Symbol.
	// Bound to the handler method so it can access handler-specific logic.
	// Returns nil if the capture should be skipped.
	//
	// IMPORTANT: wire this from a POINTER receiver (e.g. goLang.MapCapture where
	// goLang is *goHandler). A value-receiver bound before init() finishes would
	// capture a zero-value copy and miss any subsequent field writes on the
	// singleton.
	MapCapture func(captureName string, node *sitter.Node, src []byte) *Symbol
}

Capabilities holds the compiled tree-sitter queries and language reference for a LanguageHandler. Fields are nil for handlers that do not support a given capability (e.g. no call extraction, no relationship extraction).

type LanguageHandler

type LanguageHandler interface {
	// Language returns the canonical language name (e.g. "go", "python").
	Language() string

	// Extensions returns the file extensions handled (e.g. [".go"]).
	Extensions() []string

	// Parse parses the given source file and returns its symbol table.
	// path is used for language detection and populating Symbol.File fields.
	Parse(path string, src []byte, opts ParseOpts) (*ParseResult, error)

	// Capabilities returns the compiled tree-sitter queries and language
	// grammar reference for this handler. Callers use this to determine
	// whether call extraction or relationship extraction is supported.
	Capabilities() Capabilities

	// MapCapture converts a single tree-sitter capture into a Symbol.
	// Returns nil if the capture should be skipped.
	MapCapture(captureName string, node *sitter.Node, source []byte) *Symbol
}

LanguageHandler abstracts a tree-sitter language grammar and its query logic. Each handler knows how to parse its language and map tree-sitter captures to the common Symbol type.

Implementations should embed parserBase for the default tree-sitter behaviour. Preprocessor handlers (svelte, astro) override Parse to preprocess before delegating.

func HandlerForExt

func HandlerForExt(ext string) LanguageHandler

HandlerForExt returns the LanguageHandler for a given file extension. Returns nil if the extension is not supported.

type NodeKind

type NodeKind string

NodeKind represents the kind of a code symbol extracted from the AST.

const (
	KindFunction  NodeKind = "function"
	KindMethod    NodeKind = "method"
	KindType      NodeKind = "type"
	KindStruct    NodeKind = "struct"
	KindInterface NodeKind = "interface"
	KindConst     NodeKind = "const"
	KindVar       NodeKind = "var"
	KindImport    NodeKind = "import"
	KindClass     NodeKind = "class"
	KindModule    NodeKind = "module"
	// KindRune is a Svelte 5 rune call expression ($state, $derived, $effect, etc.).
	// Svelte-only: the rune detector in runes_svelte.go synthesizes these symbols
	// after tree-sitter parsing and sets RuneKind to the canonical rune category.
	KindRune NodeKind = "rune"
)

type ParseOpts

type ParseOpts struct {
	// Language overrides auto-detection.
	Language string

	// Parser is an optional tree-sitter parser to reuse. When non-nil, Parse
	// will use it instead of creating a new parser. The caller retains
	// ownership and must not use it concurrently.
	Parser *sitter.Parser

	// IncludeBody includes the full source text of each symbol.
	IncludeBody bool

	// IncludeImports includes import declarations in the result.
	IncludeImports bool

	// IncludeTypeRels extracts type relationships (embeds/extends/implements)
	// during parse. When false, TypeRels is left empty.
	IncludeTypeRels bool
}

ParseOpts controls how a file is parsed.

type ParseResult

type ParseResult struct {
	// File is the absolute path to the parsed file.
	File string

	// Language is the detected programming language.
	Language string

	// Symbols is the ordered list of symbols found in the file.
	Symbols []*Symbol

	// Imports is the list of import paths/modules declared in the file.
	Imports []string

	// TemplateRefs holds capitalised JSX-style component tag usages found in
	// the template body of Astro (and future Svelte/Vue) files. Empty for all
	// other languages. Each entry is one occurrence; callers may deduplicate.
	// Resolution of names to file paths requires joining against Imports.
	TemplateRefs []preproc.TemplateRef `json:"template_refs,omitempty"`

	// TypeRels is the list of type relationships (embeds/extends/implements)
	// extracted during parse. Populated only when ParseOpts.IncludeTypeRels is true.
	TypeRels []TypeRelationship `json:"type_rels,omitempty"`

	// Lines is the number of lines in the source file. Optional; populated by
	// callers that already count lines while reading source.
	Lines int `json:"lines,omitempty"`

	// Error is set if parsing failed or produced an error node in the tree.
	Error error
}

ParseResult contains the symbols extracted from a single source file.

func ParseFile

func ParseFile(path string, source []byte, opts ParseOpts) (*ParseResult, error)

ParseFile parses a single source file and returns its symbol table. source contains the raw file bytes. path is used for language detection and to populate Symbol.File fields.

type RelKind

type RelKind string

RelKind represents the kind of type relationship.

const (
	// RelEmbeds represents Go struct embedding or interface composition.
	RelEmbeds RelKind = "embeds"

	// RelExtends represents class/interface inheritance (extends).
	RelExtends RelKind = "extends"

	// RelImplements represents interface implementation.
	RelImplements RelKind = "implements"
)

type Symbol

type Symbol struct {
	// Name is the symbol's identifier (e.g. "ServeHTTP", "Config", "maxRetries").
	Name string

	// Kind is the symbol type (function, struct, etc.).
	Kind NodeKind

	// Language is the source language (go, python, etc.).
	Language string

	// File is the absolute path to the source file.
	File string

	// StartLine is the 1-based line number where the symbol definition begins.
	StartLine uint32

	// EndLine is the 1-based line number where the symbol definition ends.
	EndLine uint32

	// Signature is the function/type signature extracted from the AST.
	// For functions: full signature with parameters and return types.
	// For types: the type definition header.
	Signature string

	// Body is the full source text of the symbol (only populated when requested).
	Body string

	// DocComment is the documentation comment immediately preceding the symbol.
	DocComment string

	// Complexity is the estimated cyclomatic complexity of the function/method body.
	// Only populated for functions and methods (0 for other symbol kinds).
	Complexity int

	// CognitiveComplexity is the cognitive complexity of the function body (nesting-aware).
	// Only populated for functions and methods (0 for other kinds).
	CognitiveComplexity int

	// BodyHash is a content hash of the normalized symbol body.
	// Used for fast equality checks in code comparison (0 means not computed).
	BodyHash uint64

	// Receiver is the type name for methods (e.g. "Config" for impl Config,
	// "Display for Config" for impl Display for Config). Empty for free functions.
	Receiver string

	// IsPublic indicates the symbol has public visibility (pub in Rust, uppercase in Go).
	IsPublic bool

	// Attributes are annotations/decorators (e.g. "#[test]", "#[derive(Clone)]").
	Attributes []string

	// RuneKind is the canonical Svelte 5 rune category for KindRune symbols.
	// Values: "state", "derived", "effect", "props", "bindable", "inspect", "host".
	// Empty for all non-rune symbols.
	RuneKind string
}

Symbol is a named code entity extracted from a parsed file.

type TypeRelationship

type TypeRelationship struct {
	Subject string  // the type declaring the relationship
	Target  string  // the referenced type
	Kind    RelKind // relationship kind
	Line    uint32  // 1-based line number
	File    string  // absolute file path
}

TypeRelationship represents a relationship between two types (e.g. struct embedding, class inheritance, interface implementation).

func ExtractRelationships

func ExtractRelationships(path string, source []byte, opts ParseOpts) ([]TypeRelationship, error)

ExtractRelationships parses a source file and returns all type relationships. Returns empty slice (not error) for unsupported languages.

Directories

Path Synopsis
Package preproc extracts TypeScript-ish code blocks from preprocessor-language files (Svelte, Astro) into a "virtual source" buffer that can be fed to a tree-sitter TypeScript/TSX parser.
Package preproc extracts TypeScript-ish code blocks from preprocessor-language files (Svelte, Astro) into a "virtual source" buffer that can be fed to a tree-sitter TypeScript/TSX parser.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL