ast

package
v0.17.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 15, 2026 License: MIT Imports: 10 Imported by: 0

Documentation

Overview

Package ast (continued) — body extraction.

This file provides an extensible body extraction system via a registry pattern. Each language registers its own BodyExtractor, and new grammars can be supported by calling RegisterBodyExtractor.

Package ast (continued) — BrowserCache stub for non-WASM builds.

BrowserCache persists grammar blob metadata to localStorage in WASM builds, enabling faster subsequent loads by detecting which grammars were already compiled in a previous browser session.

On non-WASM builds this file provides no-op equivalents so that callers do not need build tags to reference the WASM-specific APIs.

Package ast (continued) — grammar blob caching layer.

This package provides a pluggable caching abstraction for compiled gotreesitter.Language grammars. The default implementation is an in-memory map, but the GrammarCache interface can be replaced for WASM builds (e.g. IndexedDB-backed) or other storage backends.

Package ast provides a unified AST parser using gotreesitter (pure Go tree-sitter) for Go, TypeScript, JavaScript, and Python source files.

The parser pre-loads grammar blobs at init time so that the first call to ParseFile does not pay the grammar-loading cost. It is safe for concurrent use: each call to ParseFile creates its own parser instance.

Usage:

result, err := ast.ParseFile("main.go", content)
if err != nil { ... }
for _, sym := range result.Symbols {
    fmt.Printf("%s %s at line %d\n", sym.Kind, sym.Name, sym.StartLine)
}

Package ast (continued) — scoped symbol extraction.

This package provides ExtractSymbols, which walks the AST from root and extracts symbols with scope information up to a configurable nesting depth. Unlike the inline extractSymbols in parser.go (which only walks direct children of root), this implementation tracks parent scope names so that methods inside classes get Scope: "MyClass" and Depth: 1.

The existing Symbol and helper functions (makeSymbol, childText) in parser.go are reused; no modifications to parser.go are required.

Index

Constants

View Source
const DefaultMaxDepth = 2

DefaultMaxDepth is the default maximum nesting depth for ExtractSymbols. Depth 0 = top-level symbols, Depth 1 = children (methods, fields, etc.).

Variables

View Source
var SupportedLanguages = map[string]bool{
	"go":         true,
	"typescript": true,
	"tsx":        true,
	"javascript": true,
	"python":     true,
}

SupportedLanguages is the set of languages this package handles via tree-sitter. Callers can check membership to decide whether to use the AST parser or fall back to regex / language-native tools.

Functions

func CachedGrammarNames

func CachedGrammarNames() []string

CachedGrammarNames returns nil on non-WASM builds. On WASM it returns language names with persisted metadata in localStorage.

func DetectLanguage

func DetectLanguage(filePath string) string

DetectLanguage returns the language name for a file path, or empty string if unsupported.

func EstimateLanguageSize

func EstimateLanguageSize(lang *gotreesitter.Language) int

EstimateLanguageSize estimates the approximate memory footprint (in bytes) of a *gotreesitter.Language struct.

The estimate is based on rough per-element sizes for the major slice fields in the Language struct. It does not include the lazily-built internal maps (symbolNameMap, etc.) since those are built on-demand and vary with usage.

This function does not use reflection; it directly inspects the exported slice fields.

func FileExtension

func FileExtension(filePath string) string

FileExtension returns the normalised file extension including the dot, or empty string if the path has no extension.

func InitBrowserCache

func InitBrowserCache()

InitBrowserCache is a no-op on non-WASM builds. The actual implementation lives in browser_cache.go with a //go:build js&&wasm tag and uses syscall/js to persist grammar metadata to localStorage.

func IsSupported

func IsSupported(filePath string) bool

IsSupported returns true if the file extension maps to a language with a pre-compiled grammar in this package.

func PreloadCache

func PreloadCache() int

PreloadCache loads all SupportedLanguages grammars into the default cache. Unavailable grammars are silently skipped. Returns the number of grammars successfully loaded.

func RegisterBodyExtractor

func RegisterBodyExtractor(lang string, ext BodyExtractor)

RegisterBodyExtractor registers a body extractor for a language. This enables extensibility: new grammar support can register their own extractor.

func SetDefaultCache

func SetDefaultCache(c GrammarCache)

SetDefaultCache replaces the default GrammarCache. Panics if c is nil.

This is intended for WASM builds that want to swap in a persistent storage-backed cache, or for tests that need isolation.

func Walk

func Walk(node *gotreesitter.Node, bt *gotreesitter.BoundTree, fn WalkFn)

Walk performs a depth-first walk of the AST rooted at node, calling fn for each named node. The nodeType is resolved using the BoundTree.

This is a convenience wrapper for callers that need the node type string without managing a BoundTree themselves.

Types

type ASTResult

type ASTResult struct {
	// Language is the detected language name (e.g. "go", "python").
	Language string

	// FilePath is the path that was passed to ParseFile.
	FilePath string

	// Root is the root node of the parse tree.  Use this for direct
	// tree traversal when the caller needs full control.
	Root *gotreesitter.Node

	// Source is a reference to the parsed source bytes.  It is retained
	// only for the lifetime of the BoundTree; callers that need the
	// source longer should keep their own copy.
	Source []byte

	// Tree is the underlying parse tree.  Callers MUST call result.Release()
	// when finished to free arena memory.
	Tree *gotreesitter.Tree

	// Bound is a convenience wrapper that keeps the source buffer alive
	// so that Node.Text / NodeType queries work without the caller
	// tracking the source slice.
	Bound *gotreesitter.BoundTree

	// Symbols is a list of top-level symbols extracted from the AST.
	Symbols []Symbol

	// Calls is a list of call edges extracted from the AST.
	// Each edge represents a function/method call found within a function body.
	Calls []CallEdge
}

ASTResult holds the output of ParseFile: the concrete syntax tree, a bound tree for node-text queries, and extracted top-level symbols.

func ParseContent

func ParseContent(language string, content []byte) (*ASTResult, error)

ParseContent parses source content with an explicit language name. Use this when the file path is unavailable or misleading (e.g. stdin content).

func ParseFile

func ParseFile(filePath string, content []byte) (*ASTResult, error)

ParseFile parses source content using tree-sitter and returns an ASTResult with the concrete syntax tree and extracted top-level symbols.

filePath is used only for language detection (via extension). content is the raw source bytes to parse.

The caller MUST call result.Release() when done to free the parse tree.

func (*ASTResult) Release

func (r *ASTResult) Release()

Release frees the parse tree and bound tree. It is safe to call multiple times. After Release, the Root, Source, Tree, and Bound fields are nilled to prevent use-after-release.

type BodyExtractor

type BodyExtractor interface {
	// ExtractBody returns the source text of the body for the given node,
	// or empty string if the node is not a function-like declaration.
	ExtractBody(node *gotreesitter.Node, bt *gotreesitter.BoundTree) string
}

BodyExtractor extracts the body text from a symbol node. Implementations should return the body text for function-like nodes and empty string for non-function nodes (classes, types, etc.), except where the language's semantics make the body meaningful (e.g. Python classes where the block IS the body).

type CacheStats

type CacheStats struct {
	Hits      int64
	Misses    int64
	Evictions int64
	Size      int // number of entries in the cache
}

CacheStats holds hit/miss/eviction statistics for a GrammarCache.

type CallEdge added in v0.16.19

type CallEdge struct {
	CallerName string // name of the calling function
	CalleeName string // name of the called function
	Line       int    // line number of the call
	CallerLine int    // line number of the caller function
}

CallEdge represents a call from one function to another.

type GrammarBlob

type GrammarBlob struct {
	Language *gotreesitter.Language
	Name     string
	LoadedAt time.Time
	Size     int
}

GrammarBlob wraps a compiled Language with cache metadata.

A GrammarBlob is immutable after creation (except for internal lazily-built maps inside the Language itself). Create one manually after calling EstimateLanguageSize, or use PreloadCache to populate the default cache.

type GrammarCache

type GrammarCache interface {
	// Get returns the cached GrammarBlob for name, or (nil, false) if
	// not found.  Increments the hit counter on found, miss counter
	// on not-found.
	Get(name string) (*GrammarBlob, bool)

	// Put stores a GrammarBlob.  The key is derived from blob.Name.
	// If a blob with the same name already exists, it is replaced and
	// the eviction counter is incremented.
	Put(blob *GrammarBlob)

	// Invalidate removes the cache entry for name.  If no entry exists,
	// this is a no-op.
	Invalidate(name string)

	// InvalidateAll removes all entries from the cache.
	InvalidateAll()

	// Stats returns current cache statistics.
	Stats() CacheStats

	// Names returns a sorted copy of all cached language names.
	Names() []string
}

GrammarCache is a pluggable abstraction for caching compiled grammar blobs.

All methods are safe for concurrent use by multiple goroutines.

The default implementation is MemoryCache. For WASM builds, callers can replace it via SetDefaultCache with an IndexedDB-backed implementation.

func DefaultCache

func DefaultCache() GrammarCache

DefaultCache returns the current default GrammarCache.

type MemoryCache

type MemoryCache struct {
	// contains filtered or unexported fields
}

MemoryCache is the default in-memory implementation of GrammarCache.

func NewMemoryCache

func NewMemoryCache() *MemoryCache

NewMemoryCache creates an empty MemoryCache.

func (*MemoryCache) Get

func (c *MemoryCache) Get(name string) (*GrammarBlob, bool)

Get implements GrammarCache.Get for MemoryCache.

func (*MemoryCache) Invalidate

func (c *MemoryCache) Invalidate(name string)

Invalidate implements GrammarCache.Invalidate for MemoryCache.

func (*MemoryCache) InvalidateAll

func (c *MemoryCache) InvalidateAll()

InvalidateAll implements GrammarCache.InvalidateAll for MemoryCache.

func (*MemoryCache) Names

func (c *MemoryCache) Names() []string

Names implements GrammarCache.Names for MemoryCache.

func (*MemoryCache) Put

func (c *MemoryCache) Put(blob *GrammarBlob)

Put implements GrammarCache.Put for MemoryCache.

func (*MemoryCache) Stats

func (c *MemoryCache) Stats() CacheStats

Stats implements GrammarCache.Stats for MemoryCache.

type ScopedSymbol

type ScopedSymbol struct {
	Symbol // embed the existing Symbol for Name, Kind, StartLine, etc.

	// Scope is the parent scope path, e.g. "MyClass" for a method inside
	// a class, or "MyClass.NestedStruct" for deeper nesting.
	// Empty string for top-level symbols (Depth == 0).
	Scope string

	// Depth is the nesting level: 0 = top-level, 1 = child of top-level, etc.
	Depth int
}

ScopedSymbol extends Symbol with scope/parent information and nesting depth.

func ExtractSymbols

func ExtractSymbols(root *gotreesitter.Node, bt *gotreesitter.BoundTree, lang string) []ScopedSymbol

ExtractSymbols walks the AST from root and extracts symbols with scope information. It goes deeper than just top-level: it finds methods inside classes (depth 1), nested functions, class methods in Python, etc.

The walk is limited to DefaultMaxDepth levels of nesting.

func ExtractSymbolsWithMaxDepth

func ExtractSymbolsWithMaxDepth(root *gotreesitter.Node, bt *gotreesitter.BoundTree, lang string, maxDepth int) []ScopedSymbol

ExtractSymbolsWithMaxDepth is like ExtractSymbols but allows specifying the maximum nesting depth. A value of 1 extracts only top-level symbols; a value of 2 extracts top-level plus one level of nesting. Values above 2 are currently not used for additional nesting levels.

type Symbol

type Symbol struct {
	// Name is the declared identifier (e.g. "MyFunc", "MyStruct").
	Name string

	// Kind is a normalised symbol kind: "function", "method", "class",
	// "interface", "type", "variable", "constant", "import", "decorator",
	// "property", "enum", or "module".
	Kind string

	// StartLine is the 1-based line number where the symbol starts.
	StartLine int

	// EndLine is the 1-based line number where the symbol ends (inclusive).
	EndLine int

	// StartByte is the 0-based byte offset where the symbol starts.
	StartByte int

	// EndByte is the 0-based byte offset where the symbol ends.
	EndByte int

	// Body is the source text of the function/method body (between braces
	// or after colon). Empty for non-function symbols (classes, types,
	// variables, etc.), except for Python classes where the block IS the
	// body.
	Body string
}

Symbol represents a top-level code symbol extracted from the AST.

type WalkFn

type WalkFn func(node *gotreesitter.Node, nodeType string, depth int) bool

WalkFn is the callback type for Walk. Return false to stop walking.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL