Documentation
¶
Overview ¶
Package astkit is the shared code-intelligence layer used by every product in the Provasign toolchain.
It owns:
- Language identification (LanguageKey + file-extension detection).
- Tree-sitter grammar dispatch (one place for every supported language).
- A rich, persistence-agnostic Symbol type that callers project down to their own storage models.
- Per-language extraction strategies that emit Symbol + ImportStatement.
Consumers (Grove, Fuse, Prism, Relay) import astkit and adapt to their own domain types — they do not depend on each other for parsing.
Index ¶
- Constants
- func IsAST(lang LanguageKey) bool
- func IsConfigData(lang LanguageKey) bool
- type CallSite
- type Engine
- type Export
- type ImportStatement
- type LanguageKey
- type LineRange
- type Registry
- func (r *Registry) Extract(lang LanguageKey, tree *sitter.Tree, src []byte) ([]Symbol, error)
- func (r *Registry) ExtractImports(lang LanguageKey, tree *sitter.Tree, src []byte) ([]ImportStatement, error)
- func (r *Registry) Get(lang LanguageKey) Strategy
- func (r *Registry) Register(s Strategy)
- func (r *Registry) RegisterAs(key LanguageKey, s Strategy)
- func (r *Registry) TextCapable(lang LanguageKey) bool
- type Strategy
- type Symbol
- type SymbolKind
- type TextStrategy
Constants ¶
const DefaultParseTimeout = 2 * time.Second
DefaultParseTimeout caps tree-sitter parse calls. Callers can override per Engine via WithTimeout.
const KindVar = KindVariable
KindVar is a historical alias for KindVariable. New code should prefer KindVariable.
const MaxASTDepth = 4000
MaxASTDepth bounds tree nesting before extraction. Every per-language extractor walks the tree by direct Go recursion (one call frame per level); a crafted file nested a few hundred thousand levels deep ("((((((…))))))") parses fine but overflows the goroutine stack during extraction — a FATAL, non-recoverable runtime error that takes down the whole host process and every concurrent request Grove/Fuse is serving. A recover() cannot catch a stack overflow, so the depth must be rejected BEFORE the recursive walk. 4000 is far past any real source file (deeply-generated code rarely exceeds a few hundred) while leaving ample headroom below the stack limit.
Variables ¶
This section is empty.
Functions ¶
func IsAST ¶
func IsAST(lang LanguageKey) bool
IsAST reports whether the language has a tree-sitter grammar registered in astkit (i.e. Parse will succeed).
func IsConfigData ¶
func IsConfigData(lang LanguageKey) bool
IsConfigData reports whether the language is a structured data format.
Types ¶
type CallSite ¶
type CallSite struct {
Callee string `json:"callee"`
Line int `json:"line"`
// Write marks an attribute site used as an assignment target. It is false
// for calls and ordinary reads, preserving older serialized indexes.
Write bool `json:"write,omitempty"`
// ReferenceOnly marks a callable reference that does not execute at this
// location (Java Foo::bar, Go T.Method). Consumers may use it for rename
// and reference analysis but must not turn it into a calls edge.
ReferenceOnly bool `json:"reference_only,omitempty"`
// Argc is the number of argument expressions at the call site (0 may
// mean "no arguments" or "unknown" for older indexes; consumers treat
// it as advisory). Lets graph consumers disambiguate overloads.
Argc int `json:"argc,omitempty"`
// Args holds each argument's identifier when the argument is a bare
// identifier, "" otherwise. With local type inference downstream, this
// disambiguates same-arity overloads (isEmpty(char[]) vs isEmpty(long[])).
Args []string `json:"args,omitempty"`
// Generic reports whether the call carries explicit type arguments
// (C# DeserializeObject<T>(...), the turbofish/`<...>` form). Lets graph
// consumers split a generic overload from its same-arity non-generic
// sibling, which bare-name + argc + value-arg types cannot distinguish.
Generic bool `json:"generic,omitempty"`
}
CallSite records one call expression observed inside a symbol's body. Receiver-qualified calls use "Receiver.callee"; macros end in "!".
type Engine ¶
type Engine struct {
// contains filtered or unexported fields
}
Engine wraps tree-sitter parsing for every language astkit knows about. Engines are safe for concurrent use.
func (*Engine) Parse ¶
Parse parses src under lang and returns a tree-sitter tree. Caller is responsible for tree.Close().
For non-AST languages (config formats, unknown) Parse returns (nil, nil) rather than an error — letting callers branch on `tree == nil` without inspecting error types.
type Export ¶
type Export struct {
Name string `json:"name"`
Kind SymbolKind `json:"kind"`
}
Export is a declared public surface entry. Some consumers compute this from Symbol.Exported; others need it explicit (e.g. `export {x}` re-exports in TS/JS).
type ImportStatement ¶
type ImportStatement struct {
// Raw is the original source text of the clause.
Raw string `json:"raw"`
// Path is the imported module/package path.
Path string `json:"path"`
// Alias is the local binding name, if the import was aliased.
Alias string `json:"alias,omitempty"`
// Names lists the members a from-import binds ("from . import cli, app"
// → ["cli", "app"]). A member may itself be a submodule; consumers that
// resolve module paths decide which. Empty for whole-module imports.
Names []string `json:"names,omitempty"`
// Group classifies the import as stdlib | external | relative when known.
Group string `json:"group,omitempty"`
// Line is the 1-indexed source line of the clause.
Line int `json:"line"`
}
ImportStatement is one parsed import / use / require clause.
type LanguageKey ¶
type LanguageKey string
LanguageKey identifies a programming language or data format known to astkit. The empty string is the sentinel for "unknown".
const ( LangUnknown LanguageKey = "" LangGo LanguageKey = "go" LangTypeScript LanguageKey = "typescript" LangTSX LanguageKey = "tsx" LangJavaScript LanguageKey = "javascript" LangPython LanguageKey = "python" LangJava LanguageKey = "java" LangRust LanguageKey = "rust" LangC LanguageKey = "c" LangCPP LanguageKey = "cpp" LangCSharp LanguageKey = "csharp" LangPHP LanguageKey = "php" // Non-AST data formats — recognized for detection only; astkit does not // extract symbols from them. Callers (e.g. Fuse) handle these via // structured-merge. LangJSON LanguageKey = "json" LangYAML LanguageKey = "yaml" LangTOML LanguageKey = "toml" )
const ( LangCOBOL LanguageKey = "cobol" LangJCL LanguageKey = "jcl" )
Mainframe artifact languages: extracted by TextStrategy implementations (no tree-sitter grammar). LangCOBOL covers programs and copybooks — the strategy distinguishes them by content.
func DetectLanguage ¶
func DetectLanguage(path, content string) LanguageKey
DetectLanguage returns the language for a given file path. Ambiguous .h headers are parsed as C++ when their contents contain a declaration form that is not valid C; otherwise they retain the conservative C default.
type Registry ¶
type Registry struct {
// contains filtered or unexported fields
}
Registry is a thread-safe map of LanguageKey → Strategy.
func (*Registry) ExtractImports ¶
func (r *Registry) ExtractImports(lang LanguageKey, tree *sitter.Tree, src []byte) ([]ImportStatement, error)
ExtractImports is a convenience analogous to Extract.
func (*Registry) Get ¶
func (r *Registry) Get(lang LanguageKey) Strategy
Get returns the strategy for lang, or nil if none registered.
func (*Registry) RegisterAs ¶
func (r *Registry) RegisterAs(key LanguageKey, s Strategy)
RegisterAs adds s under an additional key (used for JS/JSX, TS/TSX aliasing).
func (*Registry) TextCapable ¶ added in v0.5.0
func (r *Registry) TextCapable(lang LanguageKey) bool
Extract is a convenience that looks up the strategy and runs Extract. Returns (nil, nil) when no strategy is registered. TextCapable reports whether the strategy registered under lang extracts from source text alone (Extract accepts a nil tree). Callers use it to decide whether a language with no tree-sitter grammar still yields symbols; it is false for every current language.
type Strategy ¶
type Strategy interface {
Language() LanguageKey
Extensions() []string
Extract(tree *sitter.Tree, src []byte) ([]Symbol, error)
ExtractImports(tree *sitter.Tree, src []byte) ([]ImportStatement, error)
}
Strategy is implemented by every supported language. A Strategy converts a parsed tree-sitter tree into the canonical astkit types.
type Symbol ¶
type Symbol struct {
// Kind classifies the declaration (function, class, etc.).
Kind SymbolKind `json:"kind"`
// Name is the bare identifier (e.g. "Bar" for method Foo.Bar).
Name string `json:"name"`
// QualifiedName disambiguates within a file (e.g. "Foo.Bar" for methods
// of class Foo). For top-level symbols it equals Name.
QualifiedName string `json:"qualifiedName"`
// ParentName is the immediate enclosing type/class name, if any.
ParentName string `json:"parentName,omitempty"`
// Signature is the declaration text without a body (best-effort).
Signature string `json:"signature"`
// Docstring is the language-idiomatic documentation comment immediately
// preceding the declaration, if any. Producers MAY leave this empty and
// let downstream consumers attach it from raw source.
Docstring string `json:"docstring,omitempty"`
// Body is the full source text covering the declaration including its
// body. Required by consumers that need to reconstruct merged source
// (Fuse). Grove may drop this before persisting.
Body string `json:"body,omitempty"`
// Span is the 1-indexed inclusive line range covered by Body.
Span LineRange `json:"span"`
// Exported reports whether the declaration is visible outside its
// defining unit (file/module/package), per language semantics.
Exported bool `json:"exported"`
// Modifiers lists language-idiomatic modifier keywords (public, static,
// async, pub, etc.).
Modifiers []string `json:"modifiers,omitempty"`
// TypeParameters lists generic type parameters declared on the symbol
// (e.g. ["T", "U"] for `func F[T any, U Comparable]`).
TypeParameters []string `json:"typeParameters,omitempty"`
// Annotations lists raw annotation/decorator/attribute text attached to
// the symbol (Java @Override, Rust #[derive(...)], Python @cache, …).
Annotations []string `json:"annotations,omitempty"`
// CallSites records every call expression observed inside Body, when the
// strategy supports call-site extraction.
CallSites []CallSite `json:"callSites,omitempty"`
// AttrSites records attribute accesses that are NOT call expressions
// ("x.blueprints" with no parens). Python-only today: these are how
// @property code executes, so graph consumers need them to model
// property calls. Same shape as CallSites ("qualifier.name").
AttrSites []CallSite `json:"attrSites,omitempty"`
}
Symbol is the canonical, persistence-agnostic representation of a single declaration extracted from source. It is intentionally a superset of what any single consumer needs:
- Grove projects Symbol → SymbolRecord (adding ID/BlobSHA/FilePath/Language and the file's import list; usually retaining Body as RawText).
- Fuse projects Symbol → SymbolData (deriving a per-file Key from QualifiedName, preserving Body for merge reconstruction).
Producers must at minimum set Kind and Name.
type SymbolKind ¶
type SymbolKind string
SymbolKind enumerates the kinds of declarations astkit recognizes. New kinds may be added; consumers should treat unknown kinds as "other".
const ( KindFunction SymbolKind = "function" KindMethod SymbolKind = "method" KindConstructor SymbolKind = "constructor" KindClass SymbolKind = "class" KindInterface SymbolKind = "interface" KindStruct SymbolKind = "struct" KindEnum SymbolKind = "enum" KindTrait SymbolKind = "trait" KindType SymbolKind = "type" KindConst SymbolKind = "const" KindVariable SymbolKind = "variable" KindField SymbolKind = "field" KindModule SymbolKind = "module" KindNamespace SymbolKind = "namespace" KindDecorator SymbolKind = "decorator" KindAnnotation SymbolKind = "annotation" KindFile SymbolKind = "file" KindOther SymbolKind = "other" )
type TextStrategy ¶ added in v0.5.0
type TextStrategy interface {
Strategy
// ExtractsFromText reports that Extract tolerates a nil tree.
ExtractsFromText() bool
}
TextStrategy is an optional capability a Strategy may implement when it extracts from source text alone — no tree-sitter grammar exists for its language, and its Extract/ExtractImports accept a nil tree. Intended for line-structured artifact types (job control, data-division extracts, sort control cards); no current language implements it.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package internalast contains helpers shared by every per-language strategy implementation.
|
Package internalast contains helpers shared by every per-language strategy implementation. |
|
Package strategies provides per-language astkit.Strategy implementations built on the shared tree-sitter extractors defined in extractors.go.
|
Package strategies provides per-language astkit.Strategy implementations built on the shared tree-sitter extractors defined in extractors.go. |