grammars

package
v0.20.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 6, 2026 License: MIT Imports: 26 Imported by: 18

Documentation

Overview

Package grammars provides built-in and extension tree-sitter grammars with lazy loading. Most built-in grammars are currently shipped as compressed ts2go blobs, while extension grammars can come from grammargen-generated loaders. Use AllLanguages to enumerate available grammars, DetectLanguage to match by file extension or shebang, or call individual language functions (e.g. GoLanguage()) for direct access.

Index

Constants

This section is empty.

Variables

View Source 
var ParseSmokeSamples = map[string]string{}/* 206 elements not displayed */

ParseSmokeSamples maps language names to minimal source snippets used for smoke testing. Each sample should parse without errors under the language's supported backend.

Languages not listed here fall back to "x\n" as a generic probe.

Functions

func AdaLanguage 

func AdaLanguage() *gotreesitter.Language

AdaLanguage returns the ada language definition.

func AdaptScannerForLanguage added in v0.9.0 

func AdaptScannerForLanguage(name string, targetLang *gotreesitter.Language) bool

AdaptScannerForLanguage adapts the registered hand-written scanner for the named language to work with a different Language (e.g., one produced by grammargen). It loads the ts2go reference Language to get the scanner's native Symbol IDs, then builds an adapter that remaps them to the target Language's Symbol IDs.

func AgdaLanguage 

func AgdaLanguage() *gotreesitter.Language

AgdaLanguage returns the agda language definition.

func AngularLanguage 

func AngularLanguage() *gotreesitter.Language

AngularLanguage returns the angular language definition.

func ApexLanguage 

func ApexLanguage() *gotreesitter.Language

ApexLanguage returns the apex language definition.

func ArduinoLanguage 

func ArduinoLanguage() *gotreesitter.Language

ArduinoLanguage returns the arduino language definition.

func AsmLanguage 

func AsmLanguage() *gotreesitter.Language

AsmLanguage returns the asm language definition.

func AstroLanguage 

func AstroLanguage() *gotreesitter.Language

AstroLanguage returns the astro language definition.

func AuthzedLanguage 

func AuthzedLanguage() *gotreesitter.Language

AuthzedLanguage returns the authzed language definition.

func AwkLanguage 

func AwkLanguage() *gotreesitter.Language

AwkLanguage returns the awk language definition.

func BashLanguage 

func BashLanguage() *gotreesitter.Language

BashLanguage returns the bash language definition.

func BassLanguage 

func BassLanguage() *gotreesitter.Language

BassLanguage returns the bass language definition.

func BeancountLanguage 

func BeancountLanguage() *gotreesitter.Language

BeancountLanguage returns the beancount language definition.

func BibtexLanguage 

func BibtexLanguage() *gotreesitter.Language

BibtexLanguage returns the bibtex language definition.

func BicepLanguage 

func BicepLanguage() *gotreesitter.Language

BicepLanguage returns the bicep language definition.

func BitbakeLanguage 

func BitbakeLanguage() *gotreesitter.Language

BitbakeLanguage returns the bitbake language definition.

func BladeLanguage 

func BladeLanguage() *gotreesitter.Language

BladeLanguage returns the blade language definition.

func BlobByName added in v0.10.2 

func BlobByName(name string) []byte

BlobByName returns the raw compressed grammar blob for the named language (e.g. "go", "python"). Returns nil if the language blob is not found. The returned bytes are the gzip+gob encoded grammar data suitable for serving to browser-side WASM modules that decode grammars on demand.

func BrightscriptLanguage 

func BrightscriptLanguage() *gotreesitter.Language

BrightscriptLanguage returns the brightscript language definition.

func CLanguage 

func CLanguage() *gotreesitter.Language

CLanguage returns the c language definition.

func CSharpLanguage 

func CSharpLanguage() *gotreesitter.Language

CSharpLanguage returns the c_sharp language definition.

func CaddyLanguage 

func CaddyLanguage() *gotreesitter.Language

CaddyLanguage returns the caddy language definition.

func CairoLanguage 

func CairoLanguage() *gotreesitter.Language

CairoLanguage returns the cairo language definition.

func CapnpLanguage 

func CapnpLanguage() *gotreesitter.Language

CapnpLanguage returns the capnp language definition.

func ChatitoLanguage 

func ChatitoLanguage() *gotreesitter.Language

ChatitoLanguage returns the chatito language definition.

func CircomLanguage 

func CircomLanguage() *gotreesitter.Language

CircomLanguage returns the circom language definition.

func ClojureLanguage 

func ClojureLanguage() *gotreesitter.Language

ClojureLanguage returns the clojure language definition.

func CmakeLanguage 

func CmakeLanguage() *gotreesitter.Language

CmakeLanguage returns the cmake language definition.

func CobolLanguage 

func CobolLanguage() *gotreesitter.Language

CobolLanguage returns the cobol language definition.

func CommentLanguage 

func CommentLanguage() *gotreesitter.Language

CommentLanguage returns the comment language definition.

func CommonlispLanguage 

func CommonlispLanguage() *gotreesitter.Language

CommonlispLanguage returns the commonlisp language definition.

func CooklangLanguage 

func CooklangLanguage() *gotreesitter.Language

CooklangLanguage returns the cooklang language definition.

func Core100LanguageNames added in v0.7.0 

func Core100LanguageNames() []string

Core100LanguageNames returns a copy of the compile-time Core100 grammar set.

func CornLanguage 

func CornLanguage() *gotreesitter.Language

CornLanguage returns the corn language definition.

func CponLanguage 

func CponLanguage() *gotreesitter.Language

CponLanguage returns the cpon language definition.

func CppLanguage 

func CppLanguage() *gotreesitter.Language

CppLanguage returns the cpp language definition.

func CrystalLanguage 

func CrystalLanguage() *gotreesitter.Language

CrystalLanguage returns the crystal language definition.

func CssLanguage 

func CssLanguage() *gotreesitter.Language

CssLanguage returns the css language definition.

func CsvLanguage 

func CsvLanguage() *gotreesitter.Language

CsvLanguage returns the csv language definition.

func CudaLanguage 

func CudaLanguage() *gotreesitter.Language

CudaLanguage returns the cuda language definition.

func CueLanguage 

func CueLanguage() *gotreesitter.Language

CueLanguage returns the cue language definition.

func CylcLanguage 

func CylcLanguage() *gotreesitter.Language

CylcLanguage returns the cylc language definition.

func DLanguage 

func DLanguage() *gotreesitter.Language

DLanguage returns the d language definition.

func DartLanguage 

func DartLanguage() *gotreesitter.Language

DartLanguage returns the dart language definition.

func DesktopLanguage 

func DesktopLanguage() *gotreesitter.Language

DesktopLanguage returns the desktop language definition.

func DevicetreeLanguage 

func DevicetreeLanguage() *gotreesitter.Language

DevicetreeLanguage returns the devicetree language definition.

func DhallLanguage 

func DhallLanguage() *gotreesitter.Language

DhallLanguage returns the dhall language definition.

func DiffLanguage 

func DiffLanguage() *gotreesitter.Language

DiffLanguage returns the diff language definition.

func DisassemblyLanguage 

func DisassemblyLanguage() *gotreesitter.Language

DisassemblyLanguage returns the disassembly language definition.

func DisplayName added in v0.7.0 

func DisplayName(entry *LangEntry) string

DisplayName returns the linguist canonical display name for a language (e.g., "C++" for cpp, "JavaScript" for javascript). Falls back to title-casing the grammar name if no linguist match exists.

func DjotLanguage 

func DjotLanguage() *gotreesitter.Language

DjotLanguage returns the djot language definition.

func DockerfileLanguage 

func DockerfileLanguage() *gotreesitter.Language

DockerfileLanguage returns the dockerfile language definition.

func DotLanguage 

func DotLanguage() *gotreesitter.Language

DotLanguage returns the dot language definition.

func DoxygenLanguage 

func DoxygenLanguage() *gotreesitter.Language

DoxygenLanguage returns the doxygen language definition.

func DtdLanguage 

func DtdLanguage() *gotreesitter.Language

DtdLanguage returns the dtd language definition.

func EarthfileLanguage 

func EarthfileLanguage() *gotreesitter.Language

EarthfileLanguage returns the earthfile language definition.

func EbnfLanguage 

func EbnfLanguage() *gotreesitter.Language

EbnfLanguage returns the ebnf language definition.

func EditorconfigLanguage 

func EditorconfigLanguage() *gotreesitter.Language

EditorconfigLanguage returns the editorconfig language definition.

func EdsLanguage 

func EdsLanguage() *gotreesitter.Language

EdsLanguage returns the eds language definition.

func EexLanguage 

func EexLanguage() *gotreesitter.Language

EexLanguage returns the eex language definition.

func ElispLanguage 

func ElispLanguage() *gotreesitter.Language

ElispLanguage returns the elisp language definition.

func ElixirLanguage 

func ElixirLanguage() *gotreesitter.Language

ElixirLanguage returns the elixir language definition.

func ElmLanguage 

func ElmLanguage() *gotreesitter.Language

ElmLanguage returns the elm language definition.

func ElsaLanguage 

func ElsaLanguage() *gotreesitter.Language

ElsaLanguage returns the elsa language definition.

func EmbeddedLanguageCacheStats 

func EmbeddedLanguageCacheStats() (loaded int, limit int)

EmbeddedLanguageCacheStats returns the current decoded-grammar cache size and configured cache limit.

func EmbeddedLanguageIdleConfig 

func EmbeddedLanguageIdleConfig() (ttl time.Duration, sweepInterval time.Duration)

EmbeddedLanguageIdleConfig returns the current idle eviction settings.

func EmbeddedTemplateLanguage 

func EmbeddedTemplateLanguage() *gotreesitter.Language

EmbeddedTemplateLanguage returns the embedded_template language definition.

func EnforceLanguage 

func EnforceLanguage() *gotreesitter.Language

EnforceLanguage returns the enforce language definition.

func ErlangLanguage 

func ErlangLanguage() *gotreesitter.Language

ErlangLanguage returns the erlang language definition.

func FacilityLanguage 

func FacilityLanguage() *gotreesitter.Language

FacilityLanguage returns the facility language definition.

func FaustLanguage 

func FaustLanguage() *gotreesitter.Language

FaustLanguage returns the faust language definition.

func FennelLanguage 

func FennelLanguage() *gotreesitter.Language

FennelLanguage returns the fennel language definition.

func FidlLanguage 

func FidlLanguage() *gotreesitter.Language

FidlLanguage returns the fidl language definition.

func FirrtlLanguage 

func FirrtlLanguage() *gotreesitter.Language

FirrtlLanguage returns the firrtl language definition.

func FishLanguage 

func FishLanguage() *gotreesitter.Language

FishLanguage returns the fish language definition.

func FoamLanguage 

func FoamLanguage() *gotreesitter.Language

FoamLanguage returns the foam language definition.

func ForthLanguage 

func ForthLanguage() *gotreesitter.Language

ForthLanguage returns the forth language definition.

func FortranLanguage 

func FortranLanguage() *gotreesitter.Language

FortranLanguage returns the fortran language definition.

func FsharpLanguage 

func FsharpLanguage() *gotreesitter.Language

FsharpLanguage returns the fsharp language definition.

func GdscriptLanguage 

func GdscriptLanguage() *gotreesitter.Language

GdscriptLanguage returns the gdscript language definition.

func GitConfigLanguage 

func GitConfigLanguage() *gotreesitter.Language

GitConfigLanguage returns the git_config language definition.

func GitRebaseLanguage 

func GitRebaseLanguage() *gotreesitter.Language

GitRebaseLanguage returns the git_rebase language definition.

func GitattributesLanguage 

func GitattributesLanguage() *gotreesitter.Language

GitattributesLanguage returns the gitattributes language definition.

func GitcommitLanguage 

func GitcommitLanguage() *gotreesitter.Language

GitcommitLanguage returns the gitcommit language definition.

func GitignoreLanguage 

func GitignoreLanguage() *gotreesitter.Language

GitignoreLanguage returns the gitignore language definition.

func GleamLanguage 

func GleamLanguage() *gotreesitter.Language

GleamLanguage returns the gleam language definition.

func GlslLanguage 

func GlslLanguage() *gotreesitter.Language

GlslLanguage returns the glsl language definition.

func GnLanguage 

func GnLanguage() *gotreesitter.Language

GnLanguage returns the gn language definition.

func GoLanguage 

func GoLanguage() *gotreesitter.Language

GoLanguage returns the go language definition.

func GodotResourceLanguage 

func GodotResourceLanguage() *gotreesitter.Language

GodotResourceLanguage returns the godot_resource language definition.

func GomodLanguage 

func GomodLanguage() *gotreesitter.Language

GomodLanguage returns the gomod language definition.

func GraphqlLanguage 

func GraphqlLanguage() *gotreesitter.Language

GraphqlLanguage returns the graphql language definition.

func GroovyLanguage 

func GroovyLanguage() *gotreesitter.Language

GroovyLanguage returns the groovy language definition.

func HackLanguage 

func HackLanguage() *gotreesitter.Language

HackLanguage returns the hack language definition.

func HareLanguage 

func HareLanguage() *gotreesitter.Language

HareLanguage returns the hare language definition.

func HaskellLanguage 

func HaskellLanguage() *gotreesitter.Language

HaskellLanguage returns the haskell language definition.

func HaxeLanguage 

func HaxeLanguage() *gotreesitter.Language

HaxeLanguage returns the haxe language definition.

func HclLanguage 

func HclLanguage() *gotreesitter.Language

HclLanguage returns the hcl language definition.

func HeexLanguage 

func HeexLanguage() *gotreesitter.Language

HeexLanguage returns the heex language definition.

func HlslLanguage 

func HlslLanguage() *gotreesitter.Language

HlslLanguage returns the hlsl language definition.

func HtmlLanguage 

func HtmlLanguage() *gotreesitter.Language

HtmlLanguage returns the html language definition.

func HttpLanguage 

func HttpLanguage() *gotreesitter.Language

HttpLanguage returns the http language definition.

func HurlLanguage 

func HurlLanguage() *gotreesitter.Language

HurlLanguage returns the hurl language definition.

func HyprlangLanguage 

func HyprlangLanguage() *gotreesitter.Language

HyprlangLanguage returns the hyprlang language definition.

func IniLanguage 

func IniLanguage() *gotreesitter.Language

IniLanguage returns the ini language definition.

func JanetLanguage 

func JanetLanguage() *gotreesitter.Language

JanetLanguage returns the janet language definition.

func JavaLanguage 

func JavaLanguage() *gotreesitter.Language

JavaLanguage returns the java language definition.

func JavascriptLanguage 

func JavascriptLanguage() *gotreesitter.Language

JavascriptLanguage returns the javascript language definition.

func Jinja2Language 

func Jinja2Language() *gotreesitter.Language

Jinja2Language returns the jinja2 language definition.

func JqLanguage 

func JqLanguage() *gotreesitter.Language

JqLanguage returns the jq language definition.

func JsdocLanguage 

func JsdocLanguage() *gotreesitter.Language

JsdocLanguage returns the jsdoc language definition.

func Json5Language 

func Json5Language() *gotreesitter.Language

Json5Language returns the json5 language definition.

func JsonLanguage 

func JsonLanguage() *gotreesitter.Language

JsonLanguage returns the json language definition.

func JsonnetLanguage 

func JsonnetLanguage() *gotreesitter.Language

JsonnetLanguage returns the jsonnet language definition.

func JuliaLanguage 

func JuliaLanguage() *gotreesitter.Language

JuliaLanguage returns the julia language definition.

func JustLanguage 

func JustLanguage() *gotreesitter.Language

JustLanguage returns the just language definition.

func KconfigLanguage 

func KconfigLanguage() *gotreesitter.Language

KconfigLanguage returns the kconfig language definition.

func KdlLanguage 

func KdlLanguage() *gotreesitter.Language

KdlLanguage returns the kdl language definition.

func KotlinLanguage 

func KotlinLanguage() *gotreesitter.Language

KotlinLanguage returns the kotlin language definition.

func LedgerLanguage 

func LedgerLanguage() *gotreesitter.Language

LedgerLanguage returns the ledger language definition.

func LessLanguage 

func LessLanguage() *gotreesitter.Language

LessLanguage returns the less language definition.

func LinkerscriptLanguage 

func LinkerscriptLanguage() *gotreesitter.Language

LinkerscriptLanguage returns the linkerscript language definition.

func LiquidLanguage 

func LiquidLanguage() *gotreesitter.Language

LiquidLanguage returns the liquid language definition.

func LlvmLanguage 

func LlvmLanguage() *gotreesitter.Language

LlvmLanguage returns the llvm language definition.

func LookupExternalLexStates added in v0.9.0 

func LookupExternalLexStates(name string) [][]bool

LookupExternalLexStates returns the registered external lex states table for the given language name, or nil if none is registered.

func LookupExternalScanner added in v0.9.0 

func LookupExternalScanner(name string) gotreesitter.ExternalScanner

LookupExternalScanner returns the registered hand-written external scanner for the given language name (e.g. "python"), or nil if none is registered.

func LuaLanguage 

func LuaLanguage() *gotreesitter.Language

LuaLanguage returns the lua language definition.

func LuauLanguage 

func LuauLanguage() *gotreesitter.Language

LuauLanguage returns the luau language definition.

func MakeLanguage 

func MakeLanguage() *gotreesitter.Language

MakeLanguage returns the make language definition.

func MarkdownInlineLanguage 

func MarkdownInlineLanguage() *gotreesitter.Language

MarkdownInlineLanguage returns the markdown_inline language definition.

func MarkdownLanguage 

func MarkdownLanguage() *gotreesitter.Language

MarkdownLanguage returns the markdown language definition.

func MatlabLanguage 

func MatlabLanguage() *gotreesitter.Language

MatlabLanguage returns the matlab language definition.

func MermaidLanguage 

func MermaidLanguage() *gotreesitter.Language

MermaidLanguage returns the mermaid language definition.

func MesonLanguage 

func MesonLanguage() *gotreesitter.Language

MesonLanguage returns the meson language definition.

func MojoLanguage 

func MojoLanguage() *gotreesitter.Language

MojoLanguage returns the mojo language definition.

func MoveLanguage 

func MoveLanguage() *gotreesitter.Language

MoveLanguage returns the move language definition.

func NewAuthzedTokenSourceOrEOF 

func NewAuthzedTokenSourceOrEOF(src []byte, lang *gotreesitter.Language) gotreesitter.TokenSource

NewAuthzedTokenSourceOrEOF returns an authzed token source, or EOF-only fallback if symbol setup fails.

func NewCTokenSourceOrEOF 

func NewCTokenSourceOrEOF(src []byte, lang *gotreesitter.Language) gotreesitter.TokenSource

NewCTokenSourceOrEOF returns a C token source, or EOF-only fallback if symbol setup fails.

func NewGenericTokenSourceOrEOF 

func NewGenericTokenSourceOrEOF(src []byte, lang *gotreesitter.Language) gotreesitter.TokenSource

NewGenericTokenSourceOrEOF returns a generic token source, or EOF-only fallback if setup fails.

func NewGoTokenSourceOrEOF 

func NewGoTokenSourceOrEOF(src []byte, lang *gotreesitter.Language) gotreesitter.TokenSource

NewGoTokenSourceOrEOF returns a token source for callers that cannot surface constructor errors through their own API.

func NewHTMLTokenSourceOrEOF 

func NewHTMLTokenSourceOrEOF(src []byte, lang *gotreesitter.Language) gotreesitter.TokenSource

NewHTMLTokenSourceOrEOF returns an HTML token source, or EOF-only fallback if setup fails.

func NewJSONTokenSourceOrEOF 

func NewJSONTokenSourceOrEOF(src []byte, lang *gotreesitter.Language) gotreesitter.TokenSource

NewJSONTokenSourceOrEOF returns a token source for callers that cannot surface constructor errors through their API.

func NewJavaTokenSourceOrEOF 

func NewJavaTokenSourceOrEOF(src []byte, lang *gotreesitter.Language) gotreesitter.TokenSource

NewJavaTokenSourceOrEOF returns a Java token source, or EOF-only fallback if symbol setup fails.

func NewLuaTokenSourceOrEOF 

func NewLuaTokenSourceOrEOF(src []byte, lang *gotreesitter.Language) gotreesitter.TokenSource

NewLuaTokenSourceOrEOF returns a Lua token source, or EOF-only fallback if symbol setup fails.

func NewTomlTokenSourceOrEOF 

func NewTomlTokenSourceOrEOF(src []byte, lang *gotreesitter.Language) gotreesitter.TokenSource

NewTomlTokenSourceOrEOF returns a TOML token source, or EOF-only fallback if setup fails.

func NginxLanguage 

func NginxLanguage() *gotreesitter.Language

NginxLanguage returns the nginx language definition.

func NickelLanguage 

func NickelLanguage() *gotreesitter.Language

NickelLanguage returns the nickel language definition.

func NimLanguage 

func NimLanguage() *gotreesitter.Language

NimLanguage returns the nim language definition.

func NinjaLanguage 

func NinjaLanguage() *gotreesitter.Language

NinjaLanguage returns the ninja language definition.

func NixLanguage 

func NixLanguage() *gotreesitter.Language

NixLanguage returns the nix language definition.

func NorgLanguage 

func NorgLanguage() *gotreesitter.Language

NorgLanguage returns the norg language definition.

func NushellLanguage 

func NushellLanguage() *gotreesitter.Language

NushellLanguage returns the nushell language definition.

func ObjcLanguage 

func ObjcLanguage() *gotreesitter.Language

ObjcLanguage returns the objc language definition.

func OcamlLanguage 

func OcamlLanguage() *gotreesitter.Language

OcamlLanguage returns the ocaml language definition.

func OdinLanguage 

func OdinLanguage() *gotreesitter.Language

OdinLanguage returns the odin language definition.

func OrgLanguage 

func OrgLanguage() *gotreesitter.Language

OrgLanguage returns the org language definition.

func ParseFile 

func ParseFile(filename string, source []byte) (*gotreesitter.BoundTree, error)

ParseFile detects the language from filename, parses source, and returns a BoundTree. The caller must call Release() on the returned BoundTree.

func ParseFilePooled added in v0.7.1 

func ParseFilePooled(filename string, source []byte) (*gotreesitter.BoundTree, error)

ParseFilePooled is like ParseFile but reuses a per-language ParserPool to avoid allocating a new parser on every call. It is safe for concurrent use. The caller must call Release() on the returned BoundTree.

func ParseSmokeSample added in v0.6.0 

func ParseSmokeSample(name string) string

ParseSmokeSample returns the smoke test sample for the given language, falling back to "x\n" for languages without a dedicated sample.

func PascalLanguage 

func PascalLanguage() *gotreesitter.Language

PascalLanguage returns the pascal language definition.

func PemLanguage 

func PemLanguage() *gotreesitter.Language

PemLanguage returns the pem language definition.

func PerlLanguage 

func PerlLanguage() *gotreesitter.Language

PerlLanguage returns the perl language definition.

func PhpLanguage 

func PhpLanguage() *gotreesitter.Language

PhpLanguage returns the php language definition.

func PklLanguage 

func PklLanguage() *gotreesitter.Language

PklLanguage returns the pkl language definition.

func PowershellLanguage 

func PowershellLanguage() *gotreesitter.Language

PowershellLanguage returns the powershell language definition.

func PrismaLanguage 

func PrismaLanguage() *gotreesitter.Language

PrismaLanguage returns the prisma language definition.

func PrologLanguage 

func PrologLanguage() *gotreesitter.Language

PrologLanguage returns the prolog language definition.

func PromqlLanguage 

func PromqlLanguage() *gotreesitter.Language

PromqlLanguage returns the promql language definition.

func PropertiesLanguage 

func PropertiesLanguage() *gotreesitter.Language

PropertiesLanguage returns the properties language definition.

func ProtoLanguage 

func ProtoLanguage() *gotreesitter.Language

ProtoLanguage returns the proto language definition.

func PugLanguage 

func PugLanguage() *gotreesitter.Language

PugLanguage returns the pug language definition.

func PuppetLanguage 

func PuppetLanguage() *gotreesitter.Language

PuppetLanguage returns the puppet language definition.

func PurescriptLanguage 

func PurescriptLanguage() *gotreesitter.Language

PurescriptLanguage returns the purescript language definition.

func PurgeEmbeddedLanguageCache 

func PurgeEmbeddedLanguageCache() int

PurgeEmbeddedLanguageCache removes all decoded grammar blobs from cache and returns the number of removed entries.

func PythonLanguage 

func PythonLanguage() *gotreesitter.Language

PythonLanguage returns the python language definition.

func QlLanguage 

func QlLanguage() *gotreesitter.Language

QlLanguage returns the ql language definition.

func RLanguage 

func RLanguage() *gotreesitter.Language

RLanguage returns the r language definition.

func RacketLanguage 

func RacketLanguage() *gotreesitter.Language

RacketLanguage returns the racket language definition.

func RegexLanguage 

func RegexLanguage() *gotreesitter.Language

RegexLanguage returns the regex language definition.

func Register 

func Register(entry LangEntry)

Register adds a language to the registry. If an entry with the same name already exists, it is replaced so that grammar updates take effect.

func RegisterExtension added in v0.9.0 

func RegisterExtension(ext ExtensionEntry)

RegisterExtension registers a grammar extension for file detection and markdown code fence highlighting.

func RegisterExternalLexStates added in v0.7.0 

func RegisterExternalLexStates(name string, states [][]bool)

RegisterExternalLexStates registers the external lex states table for a language, matching C tree-sitter's ts_external_scanner_states.

func RegisterExternalScanner added in v0.6.0 

func RegisterExternalScanner(name string, s gotreesitter.ExternalScanner)

RegisterExternalScanner registers an external scanner for a language name. This is called during init() by zzz_scanner_attachments.go.

func RegisterExternalScannerSpec added in v0.16.0 

func RegisterExternalScannerSpec(spec ExternalScannerSpec)

RegisterExternalScannerSpec records scanner-source metadata for a language.

func RegoLanguage 

func RegoLanguage() *gotreesitter.Language

RegoLanguage returns the rego language definition.

func RequirementsLanguage 

func RequirementsLanguage() *gotreesitter.Language

RequirementsLanguage returns the requirements language definition.

func RescriptLanguage 

func RescriptLanguage() *gotreesitter.Language

RescriptLanguage returns the rescript language definition.

func ResolveTagsQuery added in v0.8.0 

func ResolveTagsQuery(entry LangEntry) string

ResolveTagsQuery returns the tags query for a LangEntry, computing an inferred query from grammar symbols on first call if the entry lacks an explicit TagsQuery. The inferred result is cached by language name. This may trigger grammar loading for languages without an explicit TagsQuery.

func RobotLanguage 

func RobotLanguage() *gotreesitter.Language

RobotLanguage returns the robot language definition.

func RonLanguage 

func RonLanguage() *gotreesitter.Language

RonLanguage returns the ron language definition.

func RstLanguage 

func RstLanguage() *gotreesitter.Language

RstLanguage returns the rst language definition.

func RubyLanguage 

func RubyLanguage() *gotreesitter.Language

RubyLanguage returns the ruby language definition.

func RustLanguage 

func RustLanguage() *gotreesitter.Language

RustLanguage returns the rust language definition.

func ScalaLanguage 

func ScalaLanguage() *gotreesitter.Language

ScalaLanguage returns the scala language definition.

func SchemeLanguage 

func SchemeLanguage() *gotreesitter.Language

SchemeLanguage returns the scheme language definition.

func ScssLanguage 

func ScssLanguage() *gotreesitter.Language

ScssLanguage returns the scss language definition.

func SetEmbeddedLanguageCacheLimit 

func SetEmbeddedLanguageCacheLimit(limit int)

SetEmbeddedLanguageCacheLimit sets the maximum number of decoded grammar blobs retained in the in-process cache.

- limit < 0: unlimited cache size (default) - limit == 0: disable cache retention (decode on each call) - limit > 0: retain at most limit most recently used grammars

func SetEmbeddedLanguageIdleSweepInterval 

func SetEmbeddedLanguageIdleSweepInterval(interval time.Duration)

SetEmbeddedLanguageIdleSweepInterval controls how often idle cache entries are checked when idle eviction is enabled.

func SetEmbeddedLanguageIdleTTL 

func SetEmbeddedLanguageIdleTTL(ttl time.Duration)

SetEmbeddedLanguageIdleTTL controls idle-time eviction for decoded grammars. A value <= 0 disables idle eviction.

func SmithyLanguage 

func SmithyLanguage() *gotreesitter.Language

SmithyLanguage returns the smithy language definition.

func SolidityLanguage 

func SolidityLanguage() *gotreesitter.Language

SolidityLanguage returns the solidity language definition.

func SparqlLanguage 

func SparqlLanguage() *gotreesitter.Language

SparqlLanguage returns the sparql language definition.

func SqlLanguage 

func SqlLanguage() *gotreesitter.Language

SqlLanguage returns the sql language definition.

func SquirrelLanguage 

func SquirrelLanguage() *gotreesitter.Language

SquirrelLanguage returns the squirrel language definition.

func SshConfigLanguage 

func SshConfigLanguage() *gotreesitter.Language

SshConfigLanguage returns the ssh_config language definition.

func StarlarkLanguage 

func StarlarkLanguage() *gotreesitter.Language

StarlarkLanguage returns the starlark language definition.

func SvelteLanguage 

func SvelteLanguage() *gotreesitter.Language

SvelteLanguage returns the svelte language definition.

func SwiftLanguage 

func SwiftLanguage() *gotreesitter.Language

SwiftLanguage returns the swift language definition.

func TablegenLanguage 

func TablegenLanguage() *gotreesitter.Language

TablegenLanguage returns the tablegen language definition.

func TclLanguage 

func TclLanguage() *gotreesitter.Language

TclLanguage returns the tcl language definition.

func TealLanguage 

func TealLanguage() *gotreesitter.Language

TealLanguage returns the teal language definition.

func TemplLanguage 

func TemplLanguage() *gotreesitter.Language

TemplLanguage returns the templ language definition.

func TextprotoLanguage 

func TextprotoLanguage() *gotreesitter.Language

TextprotoLanguage returns the textproto language definition.

func ThriftLanguage 

func ThriftLanguage() *gotreesitter.Language

ThriftLanguage returns the thrift language definition.

func TlaplusLanguage 

func TlaplusLanguage() *gotreesitter.Language

TlaplusLanguage returns the tlaplus language definition.

func TmuxLanguage 

func TmuxLanguage() *gotreesitter.Language

TmuxLanguage returns the tmux language definition.

func TodotxtLanguage 

func TodotxtLanguage() *gotreesitter.Language

TodotxtLanguage returns the todotxt language definition.

func TomlLanguage 

func TomlLanguage() *gotreesitter.Language

TomlLanguage returns the toml language definition.

func TsxLanguage 

func TsxLanguage() *gotreesitter.Language

TsxLanguage returns the tsx language definition.

func TurtleLanguage 

func TurtleLanguage() *gotreesitter.Language

TurtleLanguage returns the turtle language definition.

func TwigLanguage 

func TwigLanguage() *gotreesitter.Language

TwigLanguage returns the twig language definition.

func TypescriptLanguage 

func TypescriptLanguage() *gotreesitter.Language

TypescriptLanguage returns the typescript language definition.

func TypstLanguage 

func TypstLanguage() *gotreesitter.Language

TypstLanguage returns the typst language definition.

func UnloadEmbeddedLanguage 

func UnloadEmbeddedLanguage(blobName string) bool

UnloadEmbeddedLanguage removes one grammar blob from the decoded cache. Existing parser instances that already reference the language remain valid.

func UxntalLanguage 

func UxntalLanguage() *gotreesitter.Language

UxntalLanguage returns the uxntal language definition.

func VLanguage 

func VLanguage() *gotreesitter.Language

VLanguage returns the v language definition.

func VerilogLanguage 

func VerilogLanguage() *gotreesitter.Language

VerilogLanguage returns the verilog language definition.

func VhdlLanguage 

func VhdlLanguage() *gotreesitter.Language

VhdlLanguage returns the vhdl language definition.

func VimdocLanguage 

func VimdocLanguage() *gotreesitter.Language

VimdocLanguage returns the vimdoc language definition.

func VueLanguage 

func VueLanguage() *gotreesitter.Language

VueLanguage returns the vue language definition.

func WalkAndParse added in v0.8.0 

func WalkAndParse(ctx context.Context, root string, policy ParsePolicy) (<-chan ParsedFile, func() WalkStats)

WalkAndParse walks root, discovers source files, and streams parsed results on the returned channel. The caller must drain the channel to completion. The returned function provides aggregate statistics and blocks until the walk is fully complete.

Pipeline:

  1. Walk with filepath.WalkDir, skipping SkipDirs, ShouldSkipDir, and SkipExtensions.
  2. Detect language via DetectLanguage; skip unknown files.
  3. Skip binary files (NUL byte in first 8 KB).
  4. Call ShouldParse hook if set; skip if it returns false.
  5. Normal files (< LargeFileThreshold): acquire 1 semaphore slot, read+parse in a goroutine, send result, then release.
  6. Large files (>= LargeFileThreshold): acquire ALL slots, parse inline, send result, release ALL slots.

Backpressure: workers release the semaphore AFTER sending on the channel, not before. ChannelBuffer must be at least MaxConcurrent+1 to prevent deadlock. A slow consumer naturally throttles the producer because workers block on the channel send while holding their semaphore slot.

Cancellation: pass a cancellable context to stop the walk early. In-flight parses may complete, but no new files will be dispatched. The channel will close promptly after cancellation.

Usage: 

policy := grammars.DefaultPolicy()
ch, statsFn := grammars.WalkAndParse(ctx, "/path/to/repo", policy)

for pf := range ch {
    if pf.Err != nil {
        log.Printf("error: %s: %v", pf.Path, pf.Err)
        pf.Close()
        continue
    }
    root := pf.Tree.RootNode()
    // ... inspect the AST ...
    pf.Close() // MUST call to release tree memory
}

stats := statsFn() // blocks until fully done; safe to call after draining
fmt.Printf("parsed %d files (%d bytes)\n", stats.FilesParsed, stats.BytesParsed)

func WatLanguage added in v0.5.0 

func WatLanguage() *gotreesitter.Language

WatLanguage returns the wat language definition.

func WgslLanguage 

func WgslLanguage() *gotreesitter.Language

WgslLanguage returns the wgsl language definition.

func WolframLanguage 

func WolframLanguage() *gotreesitter.Language

WolframLanguage returns the wolfram language definition.

func XmlLanguage 

func XmlLanguage() *gotreesitter.Language

XmlLanguage returns the xml language definition.

func YamlLanguage 

func YamlLanguage() *gotreesitter.Language

YamlLanguage returns the yaml language definition.

func YuckLanguage 

func YuckLanguage() *gotreesitter.Language

YuckLanguage returns the yuck language definition.

func ZigLanguage 

func ZigLanguage() *gotreesitter.Language

ZigLanguage returns the zig language definition.

Types

type AgdaExternalScanner 

type AgdaExternalScanner struct{}

AgdaExternalScanner handles newline/indent/dedent for Agda.

func (AgdaExternalScanner) Create 

func (AgdaExternalScanner) Create() any

func (AgdaExternalScanner) Deserialize 

func (AgdaExternalScanner) Deserialize(payload any, buf []byte)

func (AgdaExternalScanner) Destroy 

func (AgdaExternalScanner) Destroy(payload any)

func (AgdaExternalScanner) Scan 

func (AgdaExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (AgdaExternalScanner) Serialize 

func (AgdaExternalScanner) Serialize(payload any, buf []byte) int

type AngularExternalScanner 

type AngularExternalScanner struct{}

AngularExternalScanner handles HTML tag tracking plus Angular-specific interpolation for Angular templates.

func (AngularExternalScanner) Create 

func (AngularExternalScanner) Create() any

func (AngularExternalScanner) Deserialize 

func (AngularExternalScanner) Deserialize(payload any, buf []byte)

func (AngularExternalScanner) Destroy 

func (AngularExternalScanner) Destroy(payload any)

func (AngularExternalScanner) Scan 

func (AngularExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (AngularExternalScanner) Serialize 

func (AngularExternalScanner) Serialize(payload any, buf []byte) int

type ArduinoExternalScanner 

type ArduinoExternalScanner struct{}

ArduinoExternalScanner handles C++ R"delim(...)delim" raw string literals for Arduino.

func (ArduinoExternalScanner) Create 

func (ArduinoExternalScanner) Create() any

func (ArduinoExternalScanner) Deserialize 

func (ArduinoExternalScanner) Deserialize(payload any, buf []byte)

func (ArduinoExternalScanner) Destroy 

func (ArduinoExternalScanner) Destroy(payload any)

func (ArduinoExternalScanner) Scan 

func (ArduinoExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (ArduinoExternalScanner) Serialize 

func (ArduinoExternalScanner) Serialize(payload any, buf []byte) int

type AstroExternalScanner 

type AstroExternalScanner struct{}

AstroExternalScanner handles Astro-specific external scanning: HTML tag tracking, fragment tags, frontmatter, JS expressions, backtick strings, interpolation, and permissible text.

func (AstroExternalScanner) Create 

func (AstroExternalScanner) Create() any

func (AstroExternalScanner) Deserialize 

func (AstroExternalScanner) Deserialize(payload any, buf []byte)

func (AstroExternalScanner) Destroy 

func (AstroExternalScanner) Destroy(any)

func (AstroExternalScanner) Scan 

func (AstroExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (AstroExternalScanner) Serialize 

func (AstroExternalScanner) Serialize(payload any, buf []byte) int

type AuthzedTokenSource 

type AuthzedTokenSource struct {
	// contains filtered or unexported fields
}

AuthzedTokenSource is a custom lexer for the authzed (SpiceDB/Zanzibar) permission language. The grammar is derived from tree-sitter-go and has keyword-like named tokens (definition_literal, relation_literal, etc.) that the generic lexer cannot handle, plus significant newlines.

func NewAuthzedTokenSource 

func NewAuthzedTokenSource(src []byte, lang *gotreesitter.Language) (*AuthzedTokenSource, error)

NewAuthzedTokenSource creates a token source for authzed source text.

func (*AuthzedTokenSource) Next 

func (ts *AuthzedTokenSource) Next() gotreesitter.Token

Next returns the next token from the source.

func (*AuthzedTokenSource) Reset added in v0.9.0 

func (ts *AuthzedTokenSource) Reset(src []byte)

Reset reinitializes this token source for a new source buffer.

func (*AuthzedTokenSource) SkipToByte 

func (ts *AuthzedTokenSource) SkipToByte(offset uint32) gotreesitter.Token

SkipToByte advances to the given byte offset and returns the next token.

func (*AuthzedTokenSource) SupportsIncrementalReuse added in v0.7.0 

func (ts *AuthzedTokenSource) SupportsIncrementalReuse() bool

SupportsIncrementalReuse reports that AuthzedTokenSource preserves stable token boundaries across edits and supports deterministic SkipToByte behavior.

type AwkExternalScanner 

type AwkExternalScanner struct{}

AwkExternalScanner handles spacing-sensitive tokens for AWK.

func (AwkExternalScanner) Create 

func (AwkExternalScanner) Create() any

func (AwkExternalScanner) Deserialize 

func (AwkExternalScanner) Deserialize(payload any, buf []byte)

func (AwkExternalScanner) Destroy 

func (AwkExternalScanner) Destroy(payload any)

func (AwkExternalScanner) Scan 

func (AwkExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (AwkExternalScanner) Serialize 

func (AwkExternalScanner) Serialize(payload any, buf []byte) int

type BashExternalScanner 

type BashExternalScanner struct{}

BashExternalScanner implements gotreesitter.ExternalScanner for the Bash grammar.

func (BashExternalScanner) Create 

func (BashExternalScanner) Create() any

func (BashExternalScanner) Deserialize 

func (BashExternalScanner) Deserialize(payload any, buf []byte)

func (BashExternalScanner) Destroy 

func (BashExternalScanner) Destroy(payload any)

func (BashExternalScanner) Scan 

func (BashExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (BashExternalScanner) Serialize 

func (BashExternalScanner) Serialize(payload any, buf []byte) int

type BeancountExternalScanner 

type BeancountExternalScanner struct{}

BeancountExternalScanner handles org-mode style section headers and EOF for Beancount.

func (BeancountExternalScanner) Create 

func (BeancountExternalScanner) Create() any

func (BeancountExternalScanner) Deserialize 

func (BeancountExternalScanner) Deserialize(payload any, buf []byte)

func (BeancountExternalScanner) Destroy 

func (BeancountExternalScanner) Destroy(payload any)

func (BeancountExternalScanner) Scan 

func (BeancountExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (BeancountExternalScanner) Serialize 

func (BeancountExternalScanner) Serialize(payload any, buf []byte) int

type BicepExternalScanner 

type BicepExternalScanner struct{}

BicepExternalScanner handles wildcard resource type matching and triple-single-quote multiline strings for Bicep.

func (BicepExternalScanner) Create 

func (BicepExternalScanner) Create() any

func (BicepExternalScanner) Deserialize 

func (BicepExternalScanner) Deserialize(payload any, buf []byte)

func (BicepExternalScanner) Destroy 

func (BicepExternalScanner) Destroy(payload any)

func (BicepExternalScanner) Scan 

func (BicepExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (BicepExternalScanner) Serialize 

func (BicepExternalScanner) Serialize(payload any, buf []byte) int

type BitbakeExternalScanner 

type BitbakeExternalScanner struct{}

BitbakeExternalScanner handles Python-like indent/dedent, strings, concat, and shell content.

func (BitbakeExternalScanner) Create 

func (BitbakeExternalScanner) Create() any

func (BitbakeExternalScanner) Deserialize 

func (BitbakeExternalScanner) Deserialize(payload any, buf []byte)

func (BitbakeExternalScanner) Destroy 

func (BitbakeExternalScanner) Destroy(payload any)

func (BitbakeExternalScanner) Scan 

func (BitbakeExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (BitbakeExternalScanner) Serialize 

func (BitbakeExternalScanner) Serialize(payload any, buf []byte) int

type BladeExternalScanner 

type BladeExternalScanner struct{}

BladeExternalScanner handles HTML tag tracking for Blade templates.

func (BladeExternalScanner) Create 

func (BladeExternalScanner) Create() any

func (BladeExternalScanner) Deserialize 

func (BladeExternalScanner) Deserialize(payload any, buf []byte)

func (BladeExternalScanner) Destroy 

func (BladeExternalScanner) Destroy(payload any)

func (BladeExternalScanner) Scan 

func (BladeExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (BladeExternalScanner) Serialize 

func (BladeExternalScanner) Serialize(payload any, buf []byte) int

type CSharpExternalScanner 

type CSharpExternalScanner struct{}

CSharpExternalScanner handles auto-semicolons, interpolated strings, and raw strings for C#.

func (CSharpExternalScanner) Create 

func (CSharpExternalScanner) Create() any

func (CSharpExternalScanner) Deserialize 

func (CSharpExternalScanner) Deserialize(payload any, buf []byte)

func (CSharpExternalScanner) Destroy 

func (CSharpExternalScanner) Destroy(payload any)

func (CSharpExternalScanner) Scan 

func (CSharpExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (CSharpExternalScanner) Serialize 

func (CSharpExternalScanner) Serialize(payload any, buf []byte) int

type CTokenSource 

type CTokenSource struct {
	// contains filtered or unexported fields
}

CTokenSource is a lightweight lexer bridge for tree-sitter-c.

func NewCTokenSource 

func NewCTokenSource(src []byte, lang *gotreesitter.Language) (*CTokenSource, error)

NewCTokenSource creates a token source for C source text.

func (*CTokenSource) Close added in v0.14.0 

func (ts *CTokenSource) Close()

Close clears parser-owned state and returns this token source to the pool.

func (*CTokenSource) Next 

func (ts *CTokenSource) Next() gotreesitter.Token

func (*CTokenSource) RebuildTokenSource added in v0.19.0 

func (ts *CTokenSource) RebuildTokenSource(src []byte, lang *gotreesitter.Language) (gotreesitter.TokenSource, error)

RebuildTokenSource constructs a fresh C token source for another source buffer while preserving the grammar table identity.

func (*CTokenSource) Reset added in v0.7.0 

func (ts *CTokenSource) Reset(src []byte)

Reset reinitializes this token source for a new source buffer.

func (*CTokenSource) SetGLRStates added in v0.7.0 

func (ts *CTokenSource) SetGLRStates(states []gotreesitter.StateID)

func (*CTokenSource) SetParserState added in v0.7.0 

func (ts *CTokenSource) SetParserState(state gotreesitter.StateID)

func (*CTokenSource) SkipToByte 

func (ts *CTokenSource) SkipToByte(offset uint32) gotreesitter.Token

func (*CTokenSource) SupportsIncrementalReuse added in v0.7.0 

func (ts *CTokenSource) SupportsIncrementalReuse() bool

SupportsIncrementalReuse reports that CTokenSource preserves stable token boundaries across edits and supports deterministic SkipToByte behavior.

type CaddyExternalScanner added in v0.4.0 

type CaddyExternalScanner struct{}

CaddyExternalScanner implements gotreesitter.ExternalScanner for tree-sitter-caddy. Handles _newline, _indent, and _dedent tokens for indentation tracking.

func (CaddyExternalScanner) Create added in v0.4.0 

func (CaddyExternalScanner) Create() any

func (CaddyExternalScanner) Deserialize added in v0.4.0 

func (CaddyExternalScanner) Deserialize(payload any, buf []byte)

func (CaddyExternalScanner) Destroy added in v0.4.0 

func (CaddyExternalScanner) Destroy(payload any)

func (CaddyExternalScanner) Scan added in v0.4.0 

func (CaddyExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (CaddyExternalScanner) Serialize added in v0.4.0 

func (CaddyExternalScanner) Serialize(payload any, buf []byte) int

type CairoExternalScanner 

type CairoExternalScanner struct{}

CairoExternalScanner handles %{ %} hint blocks with embedded Python in Cairo.

func (CairoExternalScanner) Create 

func (CairoExternalScanner) Create() any

func (CairoExternalScanner) Deserialize 

func (CairoExternalScanner) Deserialize(payload any, buf []byte)

func (CairoExternalScanner) Destroy 

func (CairoExternalScanner) Destroy(payload any)

func (CairoExternalScanner) Scan 

func (CairoExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (CairoExternalScanner) Serialize 

func (CairoExternalScanner) Serialize(payload any, buf []byte) int

type CmakeExternalScanner 

type CmakeExternalScanner struct{}

CmakeExternalScanner handles CMake bracket arguments, bracket comments, and line comments.

func (CmakeExternalScanner) Create 

func (CmakeExternalScanner) Create() any

func (CmakeExternalScanner) Deserialize 

func (CmakeExternalScanner) Deserialize(payload any, buf []byte)

func (CmakeExternalScanner) Destroy 

func (CmakeExternalScanner) Destroy(payload any)

func (CmakeExternalScanner) Scan 

func (CmakeExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (CmakeExternalScanner) Serialize 

func (CmakeExternalScanner) Serialize(payload any, buf []byte) int

func (CmakeExternalScanner) SupportsIncrementalReuse added in v0.20.0 

func (CmakeExternalScanner) SupportsIncrementalReuse() bool

type CobolExternalScanner 

type CobolExternalScanner struct{}

CobolExternalScanner handles COBOL's column-based formatting.

func (CobolExternalScanner) Create 

func (CobolExternalScanner) Create() any

func (CobolExternalScanner) Deserialize 

func (CobolExternalScanner) Deserialize(payload any, buf []byte)

func (CobolExternalScanner) Destroy 

func (CobolExternalScanner) Destroy(payload any)

func (CobolExternalScanner) Scan 

func (CobolExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (CobolExternalScanner) Serialize 

func (CobolExternalScanner) Serialize(payload any, buf []byte) int

type CommentExternalScanner 

type CommentExternalScanner struct{}

CommentExternalScanner implements gotreesitter.ExternalScanner for tree-sitter-comment.

The comment grammar (tree-sitter-comment) parses structured comment text such as "TODO: fix this" or "FIXME(user): description". The external scanner is responsible for producing the "name" token which represents a tag keyword (TODO, FIXME, NOTE, HACK, etc.).

The scanner must be careful not to match arbitrary text as a "name", since the DFA handles regular text via _text_token1. A name is only returned when the scanned word is immediately followed by ':' or '(', indicating it forms part of a tag construct.

func (CommentExternalScanner) Create 

func (CommentExternalScanner) Create() any

func (CommentExternalScanner) Deserialize 

func (CommentExternalScanner) Deserialize(payload any, buf []byte)

func (CommentExternalScanner) Destroy 

func (CommentExternalScanner) Destroy(payload any)

func (CommentExternalScanner) Scan 

func (CommentExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (CommentExternalScanner) Serialize 

func (CommentExternalScanner) Serialize(payload any, buf []byte) int

type CooklangExternalScanner 

type CooklangExternalScanner struct{}

CooklangExternalScanner handles newline detection for Cooklang recipe files.

func (CooklangExternalScanner) Create 

func (CooklangExternalScanner) Create() any

func (CooklangExternalScanner) Deserialize 

func (CooklangExternalScanner) Deserialize(payload any, buf []byte)

func (CooklangExternalScanner) Destroy 

func (CooklangExternalScanner) Destroy(payload any)

func (CooklangExternalScanner) Scan 

func (CooklangExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (CooklangExternalScanner) Serialize 

func (CooklangExternalScanner) Serialize(payload any, buf []byte) int

type CppExternalScanner added in v0.4.0 

type CppExternalScanner struct{}

CppExternalScanner handles C++ R"delim(...)delim" raw string literals.

func (CppExternalScanner) Create added in v0.4.0 

func (CppExternalScanner) Create() any

func (CppExternalScanner) Deserialize added in v0.4.0 

func (CppExternalScanner) Deserialize(payload any, buf []byte)

func (CppExternalScanner) Destroy added in v0.4.0 

func (CppExternalScanner) Destroy(payload any)

func (CppExternalScanner) Scan added in v0.4.0 

func (CppExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (CppExternalScanner) Serialize added in v0.4.0 

func (CppExternalScanner) Serialize(payload any, buf []byte) int

type CrystalExternalScanner 

type CrystalExternalScanner struct{}

func (CrystalExternalScanner) Create 

func (CrystalExternalScanner) Create() any

func (CrystalExternalScanner) Deserialize 

func (CrystalExternalScanner) Deserialize(payload any, buf []byte)

func (CrystalExternalScanner) Destroy 

func (CrystalExternalScanner) Destroy(payload any)

func (CrystalExternalScanner) Scan 

func (CrystalExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (CrystalExternalScanner) Serialize 

func (CrystalExternalScanner) Serialize(payload any, buf []byte) int

type CssExternalScanner 

type CssExternalScanner struct{}

CssExternalScanner implements gotreesitter.ExternalScanner for tree-sitter-css.

This is a Go port of the C external scanner from tree-sitter-css. The scanner handles three tokens:

  • _descendant_operator: whitespace between two selectors (descendant combinator)
  • pseudo_class_selector_colon: a ":" that starts a pseudo-class (vs property-value separator)
  • __error_recovery: sentinel that causes immediate bail-out

The key challenge is contextual disambiguation: whitespace might be a descendant combinator or just formatting, and ":" might start a pseudo-class or separate a property from its value.

func (CssExternalScanner) Create 

func (CssExternalScanner) Create() any

func (CssExternalScanner) Deserialize 

func (CssExternalScanner) Deserialize(payload any, buf []byte)

func (CssExternalScanner) Destroy 

func (CssExternalScanner) Destroy(payload any)

func (CssExternalScanner) Scan 

func (CssExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (CssExternalScanner) Serialize 

func (CssExternalScanner) Serialize(payload any, buf []byte) int

func (CssExternalScanner) SupportsIncrementalReuse added in v0.7.0 

func (CssExternalScanner) SupportsIncrementalReuse() bool

type CudaExternalScanner 

type CudaExternalScanner struct{}

CudaExternalScanner handles C++ R"delim(...)delim" raw string literals for CUDA.

func (CudaExternalScanner) Create 

func (CudaExternalScanner) Create() any

func (CudaExternalScanner) Deserialize 

func (CudaExternalScanner) Deserialize(payload any, buf []byte)

func (CudaExternalScanner) Destroy 

func (CudaExternalScanner) Destroy(payload any)

func (CudaExternalScanner) Scan 

func (CudaExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (CudaExternalScanner) Serialize 

func (CudaExternalScanner) Serialize(payload any, buf []byte) int

type CueExternalScanner 

type CueExternalScanner struct{}

CueExternalScanner handles string content scanning for CUE's various string types: multi-line, raw, and multi-line raw strings/bytes.

func (CueExternalScanner) Create 

func (CueExternalScanner) Create() any

func (CueExternalScanner) Deserialize 

func (CueExternalScanner) Deserialize(payload any, buf []byte)

func (CueExternalScanner) Destroy 

func (CueExternalScanner) Destroy(payload any)

func (CueExternalScanner) Scan 

func (CueExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (CueExternalScanner) Serialize 

func (CueExternalScanner) Serialize(payload any, buf []byte) int

type DExternalScanner 

type DExternalScanner struct{}

DExternalScanner handles external tokens for the D grammar. Ported from tree-sitter-d/src/scanner.c.

func (DExternalScanner) Create 

func (DExternalScanner) Create() any

func (DExternalScanner) Deserialize 

func (DExternalScanner) Deserialize(payload any, buf []byte)

func (DExternalScanner) Destroy 

func (DExternalScanner) Destroy(payload any)

func (DExternalScanner) Scan 

func (DExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (DExternalScanner) Serialize 

func (DExternalScanner) Serialize(payload any, buf []byte) int

type DartExternalScanner 

type DartExternalScanner struct{}

DartExternalScanner implements gotreesitter.ExternalScanner for tree-sitter-dart.

This is a Go port of the C external scanner from UserNobody14/tree-sitter-dart. The scanner is stateless and handles:

  • Template/string content tokens for 4 string variants (single/double x single/multi-line)
  • Raw string backslash passthrough
  • Nestable block comments (/* */ and /** */)

func (DartExternalScanner) Create 

func (DartExternalScanner) Create() any

func (DartExternalScanner) Deserialize 

func (DartExternalScanner) Deserialize(payload any, buf []byte)

func (DartExternalScanner) Destroy 

func (DartExternalScanner) Destroy(payload any)

func (DartExternalScanner) Scan 

func (DartExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (DartExternalScanner) Serialize 

func (DartExternalScanner) Serialize(payload any, buf []byte) int

type DhallExternalScanner 

type DhallExternalScanner struct{}

DhallExternalScanner handles nestable {- -} block comments for Dhall.

func (DhallExternalScanner) Create 

func (DhallExternalScanner) Create() any

func (DhallExternalScanner) Deserialize 

func (DhallExternalScanner) Deserialize(payload any, buf []byte)

func (DhallExternalScanner) Destroy 

func (DhallExternalScanner) Destroy(payload any)

func (DhallExternalScanner) Scan 

func (DhallExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (DhallExternalScanner) Serialize 

func (DhallExternalScanner) Serialize(payload any, buf []byte) int

type DisassemblyExternalScanner 

type DisassemblyExternalScanner struct{}

DisassemblyExternalScanner handles assembly instruction vs memory dump disambiguation.

func (DisassemblyExternalScanner) Create 

func (DisassemblyExternalScanner) Create() any

func (DisassemblyExternalScanner) Deserialize 

func (DisassemblyExternalScanner) Deserialize(payload any, buf []byte)

func (DisassemblyExternalScanner) Destroy 

func (DisassemblyExternalScanner) Destroy(payload any)

func (DisassemblyExternalScanner) Scan 

func (DisassemblyExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (DisassemblyExternalScanner) Serialize 

func (DisassemblyExternalScanner) Serialize(payload any, buf []byte) int

type DjotExternalScanner 

type DjotExternalScanner struct{}

DjotExternalScanner implements gotreesitter.ExternalScanner for tree-sitter-djot.

func (DjotExternalScanner) Create 

func (DjotExternalScanner) Create() any

func (DjotExternalScanner) Deserialize 

func (DjotExternalScanner) Deserialize(payload any, buf []byte)

func (DjotExternalScanner) Destroy 

func (DjotExternalScanner) Destroy(payload any)

func (DjotExternalScanner) Scan 

func (DjotExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (DjotExternalScanner) Serialize 

func (DjotExternalScanner) Serialize(payload any, buf []byte) int

type DockerfileExternalScanner 

type DockerfileExternalScanner struct{}

DockerfileExternalScanner implements gotreesitter.ExternalScanner for tree-sitter-dockerfile.

This is a Go port of the C external scanner from camdencheek/tree-sitter-dockerfile. The scanner manages Dockerfile heredoc syntax (<<MARKER / <<-MARKER) with a stack of up to 10 delimiter strings and handles:

  • heredoc_marker: the <<[-]DELIM opening
  • heredoc_line: content lines within a heredoc
  • heredoc_end: the closing delimiter line
  • _heredoc_nl: newlines within heredoc context
  • error_sentinel: error recovery bail-out

func (DockerfileExternalScanner) Create 

func (DockerfileExternalScanner) Create() any

func (DockerfileExternalScanner) Deserialize 

func (DockerfileExternalScanner) Deserialize(payload any, buf []byte)

func (DockerfileExternalScanner) Destroy 

func (DockerfileExternalScanner) Destroy(payload any)

func (DockerfileExternalScanner) Scan 

func (DockerfileExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (DockerfileExternalScanner) Serialize 

func (DockerfileExternalScanner) Serialize(payload any, buf []byte) int

type DoxygenExternalScanner 

type DoxygenExternalScanner struct{}

DoxygenExternalScanner implements gotreesitter.ExternalScanner for tree-sitter-doxygen. The doxygen grammar parses documentation comment body text (without the // or /* comment markers).

Five external tokens are handled:

  • brief_text: captures text after @brief/@short/\brief/\short until EOL
  • code_block_start: matches @code or \code
  • code_block_language: matches {.lang} immediately after code_block_start
  • code_block_content: scans all text until @endcode or \endcode
  • code_block_end: matches @endcode or \endcode

func (DoxygenExternalScanner) Create 

func (DoxygenExternalScanner) Create() any

func (DoxygenExternalScanner) Deserialize 

func (DoxygenExternalScanner) Deserialize(payload any, buf []byte)

func (DoxygenExternalScanner) Destroy 

func (DoxygenExternalScanner) Destroy(payload any)

func (DoxygenExternalScanner) Scan 

func (DoxygenExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (DoxygenExternalScanner) Serialize 

func (DoxygenExternalScanner) Serialize(payload any, buf []byte) int

type DtdExternalScanner 

type DtdExternalScanner struct{}

DtdExternalScanner handles processing instructions and <!-- --> comments for DTD.

func (DtdExternalScanner) Create 

func (DtdExternalScanner) Create() any

func (DtdExternalScanner) Deserialize 

func (DtdExternalScanner) Deserialize(payload any, buf []byte)

func (DtdExternalScanner) Destroy 

func (DtdExternalScanner) Destroy(payload any)

func (DtdExternalScanner) Scan 

func (DtdExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (DtdExternalScanner) Serialize 

func (DtdExternalScanner) Serialize(payload any, buf []byte) int

type EarthfileExternalScanner 

type EarthfileExternalScanner struct{}

EarthfileExternalScanner handles indent/dedent for Earthfile.

func (EarthfileExternalScanner) Create 

func (EarthfileExternalScanner) Create() any

func (EarthfileExternalScanner) Deserialize 

func (EarthfileExternalScanner) Deserialize(payload any, buf []byte)

func (EarthfileExternalScanner) Destroy 

func (EarthfileExternalScanner) Destroy(payload any)

func (EarthfileExternalScanner) Scan 

func (EarthfileExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (EarthfileExternalScanner) Serialize 

func (EarthfileExternalScanner) Serialize(payload any, buf []byte) int

type EditorconfigExternalScanner 

type EditorconfigExternalScanner struct{}

EditorconfigExternalScanner handles EOF and integer-range detection for .editorconfig files.

func (EditorconfigExternalScanner) Create 

func (EditorconfigExternalScanner) Create() any

func (EditorconfigExternalScanner) Deserialize 

func (EditorconfigExternalScanner) Deserialize(payload any, buf []byte)

func (EditorconfigExternalScanner) Destroy 

func (EditorconfigExternalScanner) Destroy(payload any)

func (EditorconfigExternalScanner) Scan 

func (EditorconfigExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (EditorconfigExternalScanner) Serialize 

func (EditorconfigExternalScanner) Serialize(payload any, buf []byte) int

type ElixirExternalScanner 

type ElixirExternalScanner struct{}

func (ElixirExternalScanner) Create 

func (ElixirExternalScanner) Create() any

func (ElixirExternalScanner) Deserialize 

func (ElixirExternalScanner) Deserialize(payload any, buf []byte)

func (ElixirExternalScanner) Destroy 

func (ElixirExternalScanner) Destroy(payload any)

func (ElixirExternalScanner) Scan 

func (ElixirExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (ElixirExternalScanner) Serialize 

func (ElixirExternalScanner) Serialize(payload any, buf []byte) int

type ElmExternalScanner 

type ElmExternalScanner struct{}

ElmExternalScanner handles indentation-based layout for Elm.

func (ElmExternalScanner) Create 

func (ElmExternalScanner) Create() any

func (ElmExternalScanner) Deserialize 

func (ElmExternalScanner) Deserialize(payload any, buf []byte)

func (ElmExternalScanner) Destroy 

func (ElmExternalScanner) Destroy(payload any)

func (ElmExternalScanner) Scan 

func (ElmExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (ElmExternalScanner) Serialize 

func (ElmExternalScanner) Serialize(payload any, buf []byte) int

type ErlangExternalScanner 

type ErlangExternalScanner struct{}

ErlangExternalScanner implements gotreesitter.ExternalScanner for tree-sitter-erlang.

This is a Go port of the C external scanner from tree-sitter-erlang (WhatsApp/tree-sitter-erlang). The scanner handles Erlang's triple-quoted strings (EEP-0064): strings delimited by 3+ quote characters, where the closing delimiter must appear at the start of a line (after optional whitespace) and match the same number of quotes. Sigil strings optionally have a ~[sSbB]? prefix.

func (ErlangExternalScanner) Create 

func (ErlangExternalScanner) Create() any

func (ErlangExternalScanner) Deserialize 

func (ErlangExternalScanner) Deserialize(payload any, buf []byte)

func (ErlangExternalScanner) Destroy 

func (ErlangExternalScanner) Destroy(payload any)

func (ErlangExternalScanner) Scan 

func (ErlangExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (ErlangExternalScanner) Serialize 

func (ErlangExternalScanner) Serialize(payload any, buf []byte) int

type ExtensionEntry added in v0.9.0 

type ExtensionEntry struct {
	Name              string
	Extensions        []string // file extensions: [".dmj", ".dingo", ".fw"]
	Aliases           []string // markdown fence aliases: ["dmj", "danmuji"]
	GenerateLanguage  func() (*gotreesitter.Language, error)
	HighlightQuery    string
	InheritHighlights string // parent language for highlight query composition (e.g. "go")
}

RegisterExtension registers a grammargen-based grammar extension with the language registry. This enables detection by file extension, markdown code fence highlighting, and LSP support. The language is generated lazily on first access.

Usage from an extension package: 

func init() {
    grammars.RegisterExtension(grammars.ExtensionEntry{
        Name:           "danmuji",
        Extensions:     []string{".dmj"},
        Aliases:        []string{"dmj"},
        GenerateLanguage: func() (*gotreesitter.Language, error) {
            return grammargen.GenerateLanguage(danmuji.Grammar())
        },
        HighlightQuery: danmuji.HighlightQueries(),
    })
}

type ExternalScannerSourceFile added in v0.16.0 

type ExternalScannerSourceFile struct {
	Path   string
	SHA256 string
}

ExternalScannerSourceFile identifies an upstream scanner-facing source file. The hash is SHA-256 over the exact file bytes at ExternalScannerSpec.UpstreamCommit.

type ExternalScannerSpec added in v0.16.0 

type ExternalScannerSpec struct {
	Language       string
	UpstreamRepo   string
	UpstreamCommit string
	SourceFiles    []ExternalScannerSourceFile
	Externals      []string
}

ExternalScannerSpec records the source contract for a hand-written external scanner port. It lets updater tooling distinguish grammar-only changes from external token or scanner-source changes that require scanner review.

func ExternalScannerSpecs added in v0.16.0 

func ExternalScannerSpecs() []ExternalScannerSpec

ExternalScannerSpecs returns all registered scanner-source specs, sorted by language.

func LookupExternalScannerSpec added in v0.16.0 

func LookupExternalScannerSpec(name string) (ExternalScannerSpec, bool)

LookupExternalScannerSpec returns scanner-source metadata for a language.

type FennelExternalScanner 

type FennelExternalScanner struct{}

FennelExternalScanner handles reader macros and shebang for Fennel.

func (FennelExternalScanner) Create 

func (FennelExternalScanner) Create() any

func (FennelExternalScanner) Deserialize 

func (FennelExternalScanner) Deserialize(payload any, buf []byte)

func (FennelExternalScanner) Destroy 

func (FennelExternalScanner) Destroy(payload any)

func (FennelExternalScanner) Scan 

func (FennelExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (FennelExternalScanner) Serialize 

func (FennelExternalScanner) Serialize(payload any, buf []byte) int

type FirrtlExternalScanner 

type FirrtlExternalScanner struct{}

FirrtlExternalScanner handles newline/indent/dedent for FIRRTL.

func (FirrtlExternalScanner) Create 

func (FirrtlExternalScanner) Create() any

func (FirrtlExternalScanner) Deserialize 

func (FirrtlExternalScanner) Deserialize(payload any, buf []byte)

func (FirrtlExternalScanner) Destroy 

func (FirrtlExternalScanner) Destroy(payload any)

func (FirrtlExternalScanner) Scan 

func (FirrtlExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (FirrtlExternalScanner) Serialize 

func (FirrtlExternalScanner) Serialize(payload any, buf []byte) int

type FishExternalScanner 

type FishExternalScanner struct{}

FishExternalScanner handles concatenation detection and begin-brace disambiguation for the Fish shell.

func (FishExternalScanner) Create 

func (FishExternalScanner) Create() any

func (FishExternalScanner) Deserialize 

func (FishExternalScanner) Deserialize(payload any, buf []byte)

func (FishExternalScanner) Destroy 

func (FishExternalScanner) Destroy(payload any)

func (FishExternalScanner) Scan 

func (FishExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (FishExternalScanner) Serialize 

func (FishExternalScanner) Serialize(payload any, buf []byte) int

type FoamExternalScanner 

type FoamExternalScanner struct{}

FoamExternalScanner implements gotreesitter.ExternalScanner for tree-sitter-foam.

This is a Go port of the C external scanner from tree-sitter-foam (https://github.com/FoamScience/tree-sitter-foam). The scanner handles:

  • identifier: OpenFOAM identifiers (keyword names, paths, etc.)
  • boolean: "on", "off", "true", "false"
  • _eof: end-of-file marker

func (FoamExternalScanner) Create 

func (FoamExternalScanner) Create() any

func (FoamExternalScanner) Deserialize 

func (FoamExternalScanner) Deserialize(payload any, buf []byte)

func (FoamExternalScanner) Destroy 

func (FoamExternalScanner) Destroy(payload any)

func (FoamExternalScanner) Scan 

func (FoamExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (FoamExternalScanner) Serialize 

func (FoamExternalScanner) Serialize(payload any, buf []byte) int

type FortranExternalScanner 

type FortranExternalScanner struct{}

FortranExternalScanner implements gotreesitter.ExternalScanner for tree-sitter-fortran.

func (FortranExternalScanner) Create 

func (FortranExternalScanner) Create() any

func (FortranExternalScanner) Deserialize 

func (FortranExternalScanner) Deserialize(payload any, buf []byte)

func (FortranExternalScanner) Destroy 

func (FortranExternalScanner) Destroy(payload any)

func (FortranExternalScanner) Scan 

func (FortranExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (FortranExternalScanner) Serialize 

func (FortranExternalScanner) Serialize(payload any, buf []byte) int

type FsharpExternalScanner 

type FsharpExternalScanner struct{}

FsharpExternalScanner handles indent/dedent, keywords, preprocessor directives, and comments for F#.

func (FsharpExternalScanner) Create 

func (FsharpExternalScanner) Create() any

func (FsharpExternalScanner) Deserialize 

func (FsharpExternalScanner) Deserialize(payload any, buf []byte)

func (FsharpExternalScanner) Destroy 

func (FsharpExternalScanner) Destroy(payload any)

func (FsharpExternalScanner) Scan 

func (FsharpExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (FsharpExternalScanner) Serialize 

func (FsharpExternalScanner) Serialize(payload any, buf []byte) int

type GdscriptExternalScanner 

type GdscriptExternalScanner struct{}

GdscriptExternalScanner handles indent/dedent, strings, and body_end for GDScript.

func (GdscriptExternalScanner) Create 

func (GdscriptExternalScanner) Create() any

func (GdscriptExternalScanner) Deserialize 

func (GdscriptExternalScanner) Deserialize(payload any, buf []byte)

func (GdscriptExternalScanner) Destroy 

func (GdscriptExternalScanner) Destroy(payload any)

func (GdscriptExternalScanner) Scan 

func (GdscriptExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (GdscriptExternalScanner) Serialize 

func (GdscriptExternalScanner) Serialize(payload any, buf []byte) int

type GenericTokenSource 

type GenericTokenSource struct {
	// contains filtered or unexported fields
}

GenericTokenSource is a best-effort scanner that maps source text to tree-sitter token symbols using token-name heuristics.

It is intended as a rollout bridge for grammars without DFA tables.

func NewGenericTokenSource 

func NewGenericTokenSource(src []byte, lang *gotreesitter.Language) (*GenericTokenSource, error)

NewGenericTokenSource creates a best-effort generic token source.

func (*GenericTokenSource) Next 

func (ts *GenericTokenSource) Next() gotreesitter.Token

func (*GenericTokenSource) Reset added in v0.9.0 

func (ts *GenericTokenSource) Reset(src []byte)

Reset reinitializes this token source for a new source buffer.

func (*GenericTokenSource) SkipToByte 

func (ts *GenericTokenSource) SkipToByte(offset uint32) gotreesitter.Token

type GitcommitExternalScanner 

type GitcommitExternalScanner struct{}

GitcommitExternalScanner handles conventional commit prefix detection.

func (GitcommitExternalScanner) Create 

func (GitcommitExternalScanner) Create() any

func (GitcommitExternalScanner) Deserialize 

func (GitcommitExternalScanner) Deserialize(payload any, buf []byte)

func (GitcommitExternalScanner) Destroy 

func (GitcommitExternalScanner) Destroy(payload any)

func (GitcommitExternalScanner) Scan 

func (GitcommitExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (GitcommitExternalScanner) Serialize 

func (GitcommitExternalScanner) Serialize(payload any, buf []byte) int

type GleamExternalScanner 

type GleamExternalScanner struct{}

GleamExternalScanner implements gotreesitter.ExternalScanner for tree-sitter-gleam.

The gleam grammar uses an external scanner to produce two tokens:

  • quoted_content: the interior of a string literal, consuming characters until a closing " or escape \ is encountered.
  • doc_comment_content: a single line of a doc comment, consuming characters until end-of-line or EOF.

func (GleamExternalScanner) Create 

func (GleamExternalScanner) Create() any

func (GleamExternalScanner) Deserialize 

func (GleamExternalScanner) Deserialize(payload any, buf []byte)

func (GleamExternalScanner) Destroy 

func (GleamExternalScanner) Destroy(payload any)

func (GleamExternalScanner) Scan 

func (GleamExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (GleamExternalScanner) Serialize 

func (GleamExternalScanner) Serialize(payload any, buf []byte) int

type GnExternalScanner 

type GnExternalScanner struct{}

GnExternalScanner handles string content for GN (Generate Ninja) build files. Scans content inside "..." strings, stopping at closing quote, escape sequences, and ${...} interpolations.

func (GnExternalScanner) Create 

func (GnExternalScanner) Create() any

func (GnExternalScanner) Deserialize 

func (GnExternalScanner) Deserialize(payload any, buf []byte)

func (GnExternalScanner) Destroy 

func (GnExternalScanner) Destroy(payload any)

func (GnExternalScanner) Scan 

func (GnExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (GnExternalScanner) Serialize 

func (GnExternalScanner) Serialize(payload any, buf []byte) int

type GoTokenSource 

type GoTokenSource struct {
	// contains filtered or unexported fields
}

GoTokenSource bridges Go's standard library scanner to tree-sitter's token format. It implements gotreesitter.TokenSource.

The tree-sitter Go grammar expects tokens at a finer granularity than go/scanner provides. In particular:

  • String literals are split into open-quote, content, close-quote
  • Raw string literals similarly split with backtick delimiters
  • Comments are emitted as explicit tokens
  • Newline-based automatic semicolons are mapped to ";"

func NewGoTokenSource 

func NewGoTokenSource(src []byte, lang *gotreesitter.Language) (*GoTokenSource, error)

NewGoTokenSource creates a token source that lexes Go source code and produces tree-sitter tokens compatible with the Go grammar.

func (*GoTokenSource) Next 

func (ts *GoTokenSource) Next() gotreesitter.Token

Next returns the next token. Returns a zero-Symbol token at EOF.

func (*GoTokenSource) RebuildTokenSource added in v0.7.0 

func (ts *GoTokenSource) RebuildTokenSource(src []byte, lang *gotreesitter.Language) (gotreesitter.TokenSource, error)

RebuildTokenSource constructs a fresh Go token source for the given source.

func (*GoTokenSource) Reset 

func (ts *GoTokenSource) Reset(src []byte)

Reset reinitializes this token source for a new source buffer.

func (*GoTokenSource) SkipToByte 

func (ts *GoTokenSource) SkipToByte(offset uint32) gotreesitter.Token

SkipToByte advances until it reaches the first token at or after offset.

func (*GoTokenSource) SkipToByteWithPoint 

func (ts *GoTokenSource) SkipToByteWithPoint(offset uint32, pt gotreesitter.Point) gotreesitter.Token

SkipToByteWithPoint jumps to offset using the provided Point, avoiding the O(n) offset-to-point scan that SkipToByte performs.

func (*GoTokenSource) SupportsIncrementalReuse added in v0.7.0 

func (ts *GoTokenSource) SupportsIncrementalReuse() bool

SupportsIncrementalReuse reports that GoTokenSource preserves stable token boundaries across edits and supports deterministic SkipToByte* behavior.

type GodotResourceExternalScanner 

type GodotResourceExternalScanner struct{}

GodotResourceExternalScanner handles multiline string literals in Godot .tres/.tscn resource files. Strings are "..." with \" escapes.

func (GodotResourceExternalScanner) Create 

func (GodotResourceExternalScanner) Create() any

func (GodotResourceExternalScanner) Deserialize 

func (GodotResourceExternalScanner) Deserialize(payload any, buf []byte)

func (GodotResourceExternalScanner) Destroy 

func (GodotResourceExternalScanner) Destroy(payload any)

func (GodotResourceExternalScanner) Scan 

func (GodotResourceExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (GodotResourceExternalScanner) Serialize 

func (GodotResourceExternalScanner) Serialize(payload any, buf []byte) int

type GrammarSource added in v0.10.2 

type GrammarSource string

GrammarSource describes where a LangEntry's language loader comes from. 

const (
	GrammarSourceUnknown    GrammarSource = "unknown"
	GrammarSourceTS2GoBlob  GrammarSource = "ts2go_blob"
	GrammarSourceGrammargen GrammarSource = "grammargen"
)

type HTMLExternalScanner added in v0.6.0 

type HTMLExternalScanner struct{}

HTMLExternalScanner implements gotreesitter.ExternalScanner for tree-sitter-html.

This is a Go port of the C external scanner from tree-sitter-html (https://github.com/tree-sitter/tree-sitter-html). It reuses the shared HTML scanning infrastructure (html_tags.go, blade_scanner.go) that is also used by the blade, svelte, vue, angular, and astro scanners.

func (HTMLExternalScanner) Create added in v0.6.0 

func (HTMLExternalScanner) Create() any

func (HTMLExternalScanner) Deserialize added in v0.6.0 

func (HTMLExternalScanner) Deserialize(payload any, buf []byte)

func (HTMLExternalScanner) Destroy added in v0.6.0 

func (HTMLExternalScanner) Destroy(payload any)

func (HTMLExternalScanner) Scan added in v0.6.0 

func (HTMLExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (HTMLExternalScanner) Serialize added in v0.6.0 

func (HTMLExternalScanner) Serialize(payload any, buf []byte) int

type HTMLTokenSource 

type HTMLTokenSource struct {
	// contains filtered or unexported fields
}

HTMLTokenSource is a lightweight lexer bridge for tree-sitter-html. It targets standard tag/text/comment flows and is intended for parser coverage and editor use-cases.

func NewHTMLTokenSource 

func NewHTMLTokenSource(src []byte, lang *gotreesitter.Language) (*HTMLTokenSource, error)

NewHTMLTokenSource creates a token source for HTML source text.

func (*HTMLTokenSource) Next 

func (ts *HTMLTokenSource) Next() gotreesitter.Token

func (*HTMLTokenSource) Reset added in v0.9.0 

func (ts *HTMLTokenSource) Reset(src []byte)

Reset reinitializes this token source for a new source buffer.

func (*HTMLTokenSource) SkipToByte 

func (ts *HTMLTokenSource) SkipToByte(offset uint32) gotreesitter.Token

func (*HTMLTokenSource) SupportsIncrementalReuse added in v0.7.0 

func (ts *HTMLTokenSource) SupportsIncrementalReuse() bool

SupportsIncrementalReuse reports that HTMLTokenSource preserves stable token boundaries across edits and supports deterministic SkipToByte behavior.

type HackExternalScanner 

type HackExternalScanner struct{}

HackExternalScanner handles heredoc/nowdoc strings for Hack.

func (HackExternalScanner) Create 

func (HackExternalScanner) Create() any

func (HackExternalScanner) Deserialize 

func (HackExternalScanner) Deserialize(payload any, buf []byte)

func (HackExternalScanner) Destroy 

func (HackExternalScanner) Destroy(payload any)

func (HackExternalScanner) Scan 

func (HackExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (HackExternalScanner) Serialize 

func (HackExternalScanner) Serialize(payload any, buf []byte) int

type HaskellExternalScanner added in v0.4.0 

type HaskellExternalScanner struct{}

HaskellExternalScanner implements the gotreesitter.ExternalScanner interface.

func (HaskellExternalScanner) Create added in v0.4.0 

func (HaskellExternalScanner) Create() any

func (HaskellExternalScanner) Deserialize added in v0.4.0 

func (HaskellExternalScanner) Deserialize(payload any, buf []byte)

func (HaskellExternalScanner) Destroy added in v0.4.0 

func (HaskellExternalScanner) Destroy(_ any)

func (HaskellExternalScanner) Scan added in v0.4.0 

func (HaskellExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (HaskellExternalScanner) Serialize added in v0.4.0 

func (HaskellExternalScanner) Serialize(payload any, buf []byte) int

type HaxeExternalScanner 

type HaxeExternalScanner struct{}

HaxeExternalScanner handles lookback semicolons and closing brace detection for Haxe automatic semicolon insertion.

func (HaxeExternalScanner) Create 

func (HaxeExternalScanner) Create() any

func (HaxeExternalScanner) Deserialize 

func (HaxeExternalScanner) Deserialize(payload any, buf []byte)

func (HaxeExternalScanner) Destroy 

func (HaxeExternalScanner) Destroy(payload any)

func (HaxeExternalScanner) Scan 

func (HaxeExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (HaxeExternalScanner) Serialize 

func (HaxeExternalScanner) Serialize(payload any, buf []byte) int

type HclExternalScanner added in v0.4.0 

type HclExternalScanner struct{}

HclExternalScanner implements gotreesitter.ExternalScanner for the HCL grammar.

func (HclExternalScanner) Create added in v0.4.0 

func (HclExternalScanner) Create() any

func (HclExternalScanner) Deserialize added in v0.4.0 

func (HclExternalScanner) Deserialize(payload any, buf []byte)

func (HclExternalScanner) Destroy added in v0.4.0 

func (HclExternalScanner) Destroy(payload any)

func (HclExternalScanner) Scan added in v0.4.0 

func (HclExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (HclExternalScanner) Serialize added in v0.4.0 

func (HclExternalScanner) Serialize(payload any, buf []byte) int

type HlslExternalScanner 

type HlslExternalScanner struct{}

HlslExternalScanner handles C++ R"delim(...)delim" raw string literals for HLSL.

func (HlslExternalScanner) Create 

func (HlslExternalScanner) Create() any

func (HlslExternalScanner) Deserialize 

func (HlslExternalScanner) Deserialize(payload any, buf []byte)

func (HlslExternalScanner) Destroy 

func (HlslExternalScanner) Destroy(payload any)

func (HlslExternalScanner) Scan 

func (HlslExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (HlslExternalScanner) Serialize 

func (HlslExternalScanner) Serialize(payload any, buf []byte) int

type JSONTokenSource 

type JSONTokenSource struct {
	// contains filtered or unexported fields
}

JSONTokenSource bridges raw JSON source to tree-sitter token symbols. It emits punctuation/literal tokens directly and splits strings into: open quote, string_content / escape_sequence chunks, close quote.

func NewJSONTokenSource 

func NewJSONTokenSource(src []byte, lang *gotreesitter.Language) (*JSONTokenSource, error)

NewJSONTokenSource creates a token source for JSON.

func (*JSONTokenSource) Next 

func (ts *JSONTokenSource) Next() gotreesitter.Token

func (*JSONTokenSource) Reset added in v0.9.0 

func (ts *JSONTokenSource) Reset(src []byte)

Reset reinitializes this token source for a new source buffer.

func (*JSONTokenSource) SkipToByte 

func (ts *JSONTokenSource) SkipToByte(offset uint32) gotreesitter.Token

func (*JSONTokenSource) SupportsIncrementalReuse added in v0.7.0 

func (ts *JSONTokenSource) SupportsIncrementalReuse() bool

SupportsIncrementalReuse reports that JSONTokenSource preserves stable token boundaries across edits and supports deterministic SkipToByte behavior.

type JanetExternalScanner 

type JanetExternalScanner struct{}

JanetExternalScanner handles @`...` long buffers and `...` long strings for Janet.

func (JanetExternalScanner) Create 

func (JanetExternalScanner) Create() any

func (JanetExternalScanner) Deserialize 

func (JanetExternalScanner) Deserialize(payload any, buf []byte)

func (JanetExternalScanner) Destroy 

func (JanetExternalScanner) Destroy(payload any)

func (JanetExternalScanner) Scan 

func (JanetExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (JanetExternalScanner) Serialize 

func (JanetExternalScanner) Serialize(payload any, buf []byte) int

type JavaScriptExternalScanner added in v0.4.0 

type JavaScriptExternalScanner struct{}

JavaScriptExternalScanner handles automatic semicolons, template strings, JSX text, ternary question marks, and HTML comments for JavaScript.

func (JavaScriptExternalScanner) Create added in v0.4.0 

func (JavaScriptExternalScanner) Create() any

func (JavaScriptExternalScanner) Deserialize added in v0.4.0 

func (JavaScriptExternalScanner) Deserialize(payload any, buf []byte)

func (JavaScriptExternalScanner) Destroy added in v0.4.0 

func (JavaScriptExternalScanner) Destroy(payload any)

func (JavaScriptExternalScanner) Scan added in v0.4.0 

func (JavaScriptExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (JavaScriptExternalScanner) Serialize added in v0.4.0 

func (JavaScriptExternalScanner) Serialize(payload any, buf []byte) int

func (JavaScriptExternalScanner) SupportsIncrementalReuse added in v0.7.0 

func (JavaScriptExternalScanner) SupportsIncrementalReuse() bool

type JavaTokenSource 

type JavaTokenSource struct {
	// contains filtered or unexported fields
}

JavaTokenSource is a lexer bridge for tree-sitter-java.

func NewJavaTokenSource 

func NewJavaTokenSource(src []byte, lang *gotreesitter.Language) (*JavaTokenSource, error)

NewJavaTokenSource creates a token source for Java source text.

func (*JavaTokenSource) Next 

func (ts *JavaTokenSource) Next() gotreesitter.Token

func (*JavaTokenSource) Reset added in v0.9.0 

func (ts *JavaTokenSource) Reset(src []byte)

Reset reinitializes this token source for a new source buffer.

func (*JavaTokenSource) SetGLRStates added in v0.17.0 

func (ts *JavaTokenSource) SetGLRStates(states []gotreesitter.StateID)

SetGLRStates lets the token source consider all active GLR branches when deciding whether a contextual keyword is valid.

func (*JavaTokenSource) SetParserState added in v0.17.0 

func (ts *JavaTokenSource) SetParserState(state gotreesitter.StateID)

SetParserState lets the parser drive contextual keyword tokenization.

func (*JavaTokenSource) SkipToByte 

func (ts *JavaTokenSource) SkipToByte(offset uint32) gotreesitter.Token

func (*JavaTokenSource) SupportsIncrementalReuse added in v0.7.0 

func (ts *JavaTokenSource) SupportsIncrementalReuse() bool

SupportsIncrementalReuse reports that JavaTokenSource preserves stable token boundaries across edits and supports deterministic SkipToByte behavior.

type JsdocExternalScanner 

type JsdocExternalScanner struct{}

JsdocExternalScanner implements gotreesitter.ExternalScanner for tree-sitter-jsdoc.

The jsdoc grammar uses an external scanner to match the "type" token, which represents text inside a JSDoc type annotation between { and }. The scanner tracks brace nesting depth and consumes all content until the unmatched closing } is found.

func (JsdocExternalScanner) Create 

func (JsdocExternalScanner) Create() any

func (JsdocExternalScanner) Deserialize 

func (JsdocExternalScanner) Deserialize(payload any, buf []byte)

func (JsdocExternalScanner) Destroy 

func (JsdocExternalScanner) Destroy(payload any)

func (JsdocExternalScanner) Scan 

func (JsdocExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (JsdocExternalScanner) Serialize 

func (JsdocExternalScanner) Serialize(payload any, buf []byte) int

type JsonnetExternalScanner 

type JsonnetExternalScanner struct{}

JsonnetExternalScanner handles Jsonnet string literals including single/double quoted strings and ||| block strings.

func (JsonnetExternalScanner) Create 

func (JsonnetExternalScanner) Create() any

func (JsonnetExternalScanner) Deserialize 

func (JsonnetExternalScanner) Deserialize(payload any, buf []byte)

func (JsonnetExternalScanner) Destroy 

func (JsonnetExternalScanner) Destroy(payload any)

func (JsonnetExternalScanner) Scan 

func (JsonnetExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (JsonnetExternalScanner) Serialize 

func (JsonnetExternalScanner) Serialize(payload any, buf []byte) int

type JuliaExternalScanner 

type JuliaExternalScanner struct{}

JuliaExternalScanner handles block comments, immediate tokens, and string/command content for Julia.

func (JuliaExternalScanner) Create 

func (JuliaExternalScanner) Create() any

func (JuliaExternalScanner) Deserialize 

func (JuliaExternalScanner) Deserialize(payload any, buf []byte)

func (JuliaExternalScanner) Destroy 

func (JuliaExternalScanner) Destroy(payload any)

func (JuliaExternalScanner) Scan 

func (JuliaExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (JuliaExternalScanner) Serialize 

func (JuliaExternalScanner) Serialize(payload any, buf []byte) int

type JustExternalScanner 

type JustExternalScanner struct{}

JustExternalScanner handles indent/dedent/newline/text for justfiles.

func (JustExternalScanner) Create 

func (JustExternalScanner) Create() any

func (JustExternalScanner) Deserialize 

func (JustExternalScanner) Deserialize(payload any, buf []byte)

func (JustExternalScanner) Destroy 

func (JustExternalScanner) Destroy(payload any)

func (JustExternalScanner) Scan 

func (JustExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (JustExternalScanner) Serialize 

func (JustExternalScanner) Serialize(payload any, buf []byte) int

type KconfigExternalScanner 

type KconfigExternalScanner struct{}

KconfigExternalScanner handles indented help text blocks in Linux Kconfig files. Help text continues as long as lines have consistent indentation.

func (KconfigExternalScanner) Create 

func (KconfigExternalScanner) Create() any

func (KconfigExternalScanner) Deserialize 

func (KconfigExternalScanner) Deserialize(payload any, buf []byte)

func (KconfigExternalScanner) Destroy 

func (KconfigExternalScanner) Destroy(payload any)

func (KconfigExternalScanner) Scan 

func (KconfigExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (KconfigExternalScanner) Serialize 

func (KconfigExternalScanner) Serialize(payload any, buf []byte) int

type KdlExternalScanner 

type KdlExternalScanner struct{}

KdlExternalScanner handles EOF detection, nestable /* */ comments, and r#"..."# raw strings for KDL.

func (KdlExternalScanner) Create 

func (KdlExternalScanner) Create() any

func (KdlExternalScanner) Deserialize 

func (KdlExternalScanner) Deserialize(payload any, buf []byte)

func (KdlExternalScanner) Destroy 

func (KdlExternalScanner) Destroy(payload any)

func (KdlExternalScanner) Scan 

func (KdlExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (KdlExternalScanner) Serialize 

func (KdlExternalScanner) Serialize(payload any, buf []byte) int

type KotlinExternalScanner 

type KotlinExternalScanner struct {
	// contains filtered or unexported fields
}

KotlinExternalScanner implements gotreesitter.ExternalScanner for tree-sitter-kotlin.

This is a Go port of the C external scanner from fwcd/tree-sitter-kotlin. The scanner handles 9 external tokens including automatic semicolon insertion (ASI), safe navigation (?.), nested multiline comments, string start/end/content with interpolation support, primary constructor keyword detection, and import path handling.

func (KotlinExternalScanner) Create 

func (KotlinExternalScanner) Create() any

func (KotlinExternalScanner) Deserialize 

func (KotlinExternalScanner) Deserialize(payload any, buf []byte)

func (KotlinExternalScanner) Destroy 

func (KotlinExternalScanner) Destroy(payload any)

func (KotlinExternalScanner) ExternalScannerForLanguage added in v0.16.0 

func (KotlinExternalScanner) ExternalScannerForLanguage(lang *gotreesitter.Language) gotreesitter.ExternalScanner

func (KotlinExternalScanner) Scan 

func (k KotlinExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (KotlinExternalScanner) Serialize 

func (KotlinExternalScanner) Serialize(payload any, buf []byte) int

type LangEntry 

type LangEntry struct {
	Name               string
	Extensions         []string                      // e.g. [".go", ".mod"]
	Shebangs           []string                      // e.g. ["#!/usr/bin/env python"]
	Language           func() *gotreesitter.Language // lazy loader
	GrammarSource      GrammarSource                 // e.g. ts2go blob or grammargen
	HighlightQuery     string
	InheritHighlights  string                                                                 // language name to inherit highlight queries from (e.g. "javascript" for TypeScript)
	TagsQuery          string                                                                 // tree-sitter tags.scm query for symbol extraction
	TokenSourceFactory func(src []byte, lang *gotreesitter.Language) gotreesitter.TokenSource // nil = use DFA
	Quality            ParseQuality                                                           // populated by AuditParseSupport
}

LangEntry holds a registered language with its grammar, extensions, and highlight query.

func AllLanguages 

func AllLanguages() []LangEntry

AllLanguages returns all registered languages. This is a metadata-only operation — it does NOT load grammar parse tables or decompress blobs. Languages that lack an explicit TagsQuery will have an empty TagsQuery field; call ResolveTagsQuery when you actually need the inferred query.

func DetectLanguage 

func DetectLanguage(filename string) *LangEntry

DetectLanguage returns the LangEntry for a filename, or nil if unknown. Checks in order: exact filename match (linguist), registry extensions, then linguist extended extensions. Exact filenames take priority over suffix matching so that e.g. ".tmux.conf" resolves to bash rather than matching the generic ".conf" extension.

func DetectLanguageByName added in v0.7.0 

func DetectLanguageByName(name string) *LangEntry

DetectLanguageByName returns the LangEntry for any linguist canonical name, alias, or gotreesitter grammar name. Returns nil if unknown.

Accepts: "C++", "cpp", "Go", "golang", "Shell", "bash", "F#", "fsharp", etc. Direct grammar names always take priority over linguist aliases to prevent shadowing (e.g., "eex" resolves to the eex grammar, not heex via alias).

func DetectLanguageByShebang 

func DetectLanguageByShebang(firstLine string) *LangEntry

DetectLanguageByShebang checks the first line of content for shebang matches. Handles both "#!/usr/bin/env python3" and "#!/usr/bin/python3" forms.

type LessExternalScanner 

type LessExternalScanner struct{}

LessExternalScanner handles the CSS descendant combinator for Less. Nearly identical to the SCSS descendant operator scanner.

func (LessExternalScanner) Create 

func (LessExternalScanner) Create() any

func (LessExternalScanner) Deserialize 

func (LessExternalScanner) Deserialize(payload any, buf []byte)

func (LessExternalScanner) Destroy 

func (LessExternalScanner) Destroy(payload any)

func (LessExternalScanner) Scan 

func (LessExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (LessExternalScanner) Serialize 

func (LessExternalScanner) Serialize(payload any, buf []byte) int

type LiquidExternalScanner 

type LiquidExternalScanner struct{}

LiquidExternalScanner handles comment content, raw blocks, and front matter for Liquid templates.

func (LiquidExternalScanner) Create 

func (LiquidExternalScanner) Create() any

func (LiquidExternalScanner) Deserialize 

func (LiquidExternalScanner) Deserialize(payload any, buf []byte)

func (LiquidExternalScanner) Destroy 

func (LiquidExternalScanner) Destroy(payload any)

func (LiquidExternalScanner) Scan 

func (LiquidExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (LiquidExternalScanner) Serialize 

func (LiquidExternalScanner) Serialize(payload any, buf []byte) int

type LuaTokenSource 

type LuaTokenSource struct {
	// contains filtered or unexported fields
}

LuaTokenSource is a lexer bridge for tree-sitter-lua.

func NewLuaTokenSource 

func NewLuaTokenSource(src []byte, lang *gotreesitter.Language) (*LuaTokenSource, error)

NewLuaTokenSource creates a token source for Lua source text.

func (*LuaTokenSource) Next 

func (ts *LuaTokenSource) Next() gotreesitter.Token

func (*LuaTokenSource) Reset added in v0.7.0 

func (ts *LuaTokenSource) Reset(src []byte)

Reset reinitializes this token source for a new source buffer.

func (*LuaTokenSource) SkipToByte 

func (ts *LuaTokenSource) SkipToByte(offset uint32) gotreesitter.Token

func (*LuaTokenSource) SupportsIncrementalReuse added in v0.7.0 

func (ts *LuaTokenSource) SupportsIncrementalReuse() bool

SupportsIncrementalReuse reports that LuaTokenSource preserves stable token boundaries across edits and supports deterministic SkipToByte behavior.

type LuauExternalScanner 

type LuauExternalScanner struct{}

LuauExternalScanner handles Lua-style block comments --[=[ ... ]=] and block/quoted strings for Luau.

func (LuauExternalScanner) Create 

func (LuauExternalScanner) Create() any

func (LuauExternalScanner) Deserialize 

func (LuauExternalScanner) Deserialize(payload any, buf []byte)

func (LuauExternalScanner) Destroy 

func (LuauExternalScanner) Destroy(payload any)

func (LuauExternalScanner) Scan 

func (LuauExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (LuauExternalScanner) Serialize 

func (LuauExternalScanner) Serialize(payload any, buf []byte) int

type MarkdownExternalScanner 

type MarkdownExternalScanner struct{}

MarkdownExternalScanner handles block-level markdown parsing.

func (MarkdownExternalScanner) Create 

func (MarkdownExternalScanner) Create() any

func (MarkdownExternalScanner) Deserialize 

func (MarkdownExternalScanner) Deserialize(payload any, buf []byte)

func (MarkdownExternalScanner) Destroy 

func (MarkdownExternalScanner) Destroy(payload any)

func (MarkdownExternalScanner) Scan 

func (MarkdownExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (MarkdownExternalScanner) Serialize 

func (MarkdownExternalScanner) Serialize(payload any, buf []byte) int

type MarkdownInlineExternalScanner 

type MarkdownInlineExternalScanner struct{}

MarkdownInlineExternalScanner handles external scanning for the markdown_inline grammar.

func (MarkdownInlineExternalScanner) Create 

func (MarkdownInlineExternalScanner) Create() any

func (MarkdownInlineExternalScanner) Deserialize 

func (MarkdownInlineExternalScanner) Deserialize(payload any, buf []byte)

func (MarkdownInlineExternalScanner) Destroy 

func (MarkdownInlineExternalScanner) Destroy(payload any)

func (MarkdownInlineExternalScanner) Scan 

func (MarkdownInlineExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (MarkdownInlineExternalScanner) Serialize 

func (MarkdownInlineExternalScanner) Serialize(payload any, buf []byte) int

type MatlabExternalScanner 

type MatlabExternalScanner struct{}

MatlabExternalScanner handles the external scanning for the MATLAB grammar.

func (MatlabExternalScanner) Create 

func (MatlabExternalScanner) Create() any

func (MatlabExternalScanner) Deserialize 

func (MatlabExternalScanner) Deserialize(payload any, buf []byte)

func (MatlabExternalScanner) Destroy 

func (MatlabExternalScanner) Destroy(payload any)

func (MatlabExternalScanner) Scan 

func (MatlabExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (MatlabExternalScanner) Serialize 

func (MatlabExternalScanner) Serialize(payload any, buf []byte) int

type MojoExternalScanner 

type MojoExternalScanner struct{}

MojoExternalScanner handles indent/dedent and string literals for Mojo. Mojo is Python-like; this reuses the pythonScannerState type.

func (MojoExternalScanner) Create 

func (MojoExternalScanner) Create() any

func (MojoExternalScanner) Deserialize 

func (MojoExternalScanner) Deserialize(payload any, buf []byte)

func (MojoExternalScanner) Destroy 

func (MojoExternalScanner) Destroy(payload any)

func (MojoExternalScanner) Scan 

func (MojoExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (MojoExternalScanner) Serialize 

func (MojoExternalScanner) Serialize(payload any, buf []byte) int

func (MojoExternalScanner) SupportsIncrementalReuse added in v0.7.0 

func (MojoExternalScanner) SupportsIncrementalReuse() bool

type NginxExternalScanner 

type NginxExternalScanner struct{}

NginxExternalScanner implements gotreesitter.ExternalScanner for tree-sitter-nginx.

This is a Go port of the C external scanner from tree-sitter-nginx (https://github.com/opa-oz/tree-sitter-nginx). The scanner handles:

  • _newline: newline characters
  • _indent: increase in indentation level
  • _dedent: decrease in indentation level

func (NginxExternalScanner) Create 

func (NginxExternalScanner) Create() any

func (NginxExternalScanner) Deserialize 

func (NginxExternalScanner) Deserialize(payload any, buf []byte)

func (NginxExternalScanner) Destroy 

func (NginxExternalScanner) Destroy(payload any)

func (NginxExternalScanner) Scan 

func (NginxExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (NginxExternalScanner) Serialize 

func (NginxExternalScanner) Serialize(payload any, buf []byte) int

type NickelExternalScanner 

type NickelExternalScanner struct{}

NickelExternalScanner handles Nickel's multi-line strings, interpolation, and comments.

func (NickelExternalScanner) Create 

func (NickelExternalScanner) Create() any

func (NickelExternalScanner) Deserialize 

func (NickelExternalScanner) Deserialize(payload any, buf []byte)

func (NickelExternalScanner) Destroy 

func (NickelExternalScanner) Destroy(payload any)

func (NickelExternalScanner) Scan 

func (NickelExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (NickelExternalScanner) Serialize 

func (NickelExternalScanner) Serialize(payload any, buf []byte) int

type NimExternalScanner 

type NimExternalScanner struct{}

--------------------------------------------------------------------------- NimExternalScanner implements gotreesitter.ExternalScanner ---------------------------------------------------------------------------

func (NimExternalScanner) Create 

func (NimExternalScanner) Create() any

func (NimExternalScanner) Deserialize 

func (NimExternalScanner) Deserialize(payload any, buf []byte)

func (NimExternalScanner) Destroy 

func (NimExternalScanner) Destroy(payload any)

func (NimExternalScanner) Scan 

func (NimExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (NimExternalScanner) Serialize 

func (NimExternalScanner) Serialize(payload any, buf []byte) int

type NixExternalScanner 

type NixExternalScanner struct{}

NixExternalScanner handles string fragments, path literals, and dollar escapes for the Nix expression language.

func (NixExternalScanner) Create 

func (NixExternalScanner) Create() any

func (NixExternalScanner) Deserialize 

func (NixExternalScanner) Deserialize(payload any, buf []byte)

func (NixExternalScanner) Destroy 

func (NixExternalScanner) Destroy(payload any)

func (NixExternalScanner) Scan 

func (NixExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (NixExternalScanner) Serialize 

func (NixExternalScanner) Serialize(payload any, buf []byte) int

type NorgExternalScanner added in v0.6.0 

type NorgExternalScanner struct{}

NorgExternalScanner handles norg markup disambiguation.

func (NorgExternalScanner) Create added in v0.6.0 

func (NorgExternalScanner) Create() any

func (NorgExternalScanner) Deserialize added in v0.6.0 

func (NorgExternalScanner) Deserialize(payload any, buf []byte)

func (NorgExternalScanner) Destroy added in v0.6.0 

func (NorgExternalScanner) Destroy(_ any)

func (NorgExternalScanner) Scan added in v0.6.0 

func (NorgExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (NorgExternalScanner) Serialize added in v0.6.0 

func (NorgExternalScanner) Serialize(payload any, buf []byte) int

type NushellExternalScanner 

type NushellExternalScanner struct{}

NushellExternalScanner implements gotreesitter.ExternalScanner for tree-sitter-nu.

This is a Go port of the C external scanner from tree-sitter-nu (https://github.com/nushell/tree-sitter-nu). The scanner handles:

  • raw_string_begin: r#' (with variable # count)
  • string_content: content between raw string delimiters
  • raw_string_end: '# closing delimiter

func (NushellExternalScanner) Create 

func (NushellExternalScanner) Create() any

func (NushellExternalScanner) Deserialize 

func (NushellExternalScanner) Deserialize(payload any, buf []byte)

func (NushellExternalScanner) Destroy 

func (NushellExternalScanner) Destroy(payload any)

func (NushellExternalScanner) Scan 

func (NushellExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (NushellExternalScanner) Serialize 

func (NushellExternalScanner) Serialize(payload any, buf []byte) int

type OcamlExternalScanner 

type OcamlExternalScanner struct{}

OcamlExternalScanner implements gotreesitter.ExternalScanner for tree-sitter-ocaml.

This is a Go port of the C external scanner from tree-sitter/tree-sitter-ocaml. The scanner handles:

  • Nestable (* *) comments (lexically aware of strings/chars inside)
  • Quoted string delimiters {id|...|id}
  • String open/close with in_string state tracking
  • Line number directives (# <num> "file")
  • Literal null characters (\0 that isn't EOF)

func (OcamlExternalScanner) Create 

func (OcamlExternalScanner) Create() any

func (OcamlExternalScanner) Deserialize 

func (OcamlExternalScanner) Deserialize(payload any, buf []byte)

func (OcamlExternalScanner) Destroy 

func (OcamlExternalScanner) Destroy(payload any)

func (OcamlExternalScanner) Scan 

func (OcamlExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (OcamlExternalScanner) Serialize 

func (OcamlExternalScanner) Serialize(payload any, buf []byte) int

type OdinExternalScanner 

type OdinExternalScanner struct{}

OdinExternalScanner handles newlines, floats, block comments, and other context-sensitive tokens for Odin.

func (OdinExternalScanner) Create 

func (OdinExternalScanner) Create() any

func (OdinExternalScanner) Deserialize 

func (OdinExternalScanner) Deserialize(payload any, buf []byte)

func (OdinExternalScanner) Destroy 

func (OdinExternalScanner) Destroy(payload any)

func (OdinExternalScanner) Scan 

func (OdinExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (OdinExternalScanner) Serialize 

func (OdinExternalScanner) Serialize(payload any, buf []byte) int

type OrgExternalScanner 

type OrgExternalScanner struct{}

OrgExternalScanner handles list/section/headline detection for org mode.

func (OrgExternalScanner) Create 

func (OrgExternalScanner) Create() any

func (OrgExternalScanner) Deserialize 

func (OrgExternalScanner) Deserialize(payload any, buf []byte)

func (OrgExternalScanner) Destroy 

func (OrgExternalScanner) Destroy(payload any)

func (OrgExternalScanner) Scan 

func (OrgExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (OrgExternalScanner) Serialize 

func (OrgExternalScanner) Serialize(payload any, buf []byte) int

type ParseBackend 

type ParseBackend string

ParseBackend describes how a language can be parsed in this runtime. 

const (
	ParseBackendUnsupported ParseBackend = "unsupported"
	ParseBackendDFA         ParseBackend = "dfa"
	ParseBackendDFAPartial  ParseBackend = "dfa-partial"
	ParseBackendTokenSource ParseBackend = "token_source"
)

type ParsePolicy added in v0.8.0 

type ParsePolicy struct {
	// LargeFileThreshold is the byte size above which a file is parsed with
	// exclusive access to all worker slots (serialized). Files at or above
	// this threshold are considered "large."
	LargeFileThreshold int64

	// MaxConcurrent limits the number of files parsed simultaneously.
	MaxConcurrent int

	// ChannelBuffer is the buffer size for the output channel. Must be at
	// least MaxConcurrent+1 to prevent deadlock when workers hold semaphore
	// slots while sending on the channel.
	ChannelBuffer int

	// SkipDirs lists directory base names to skip entirely during the walk.
	SkipDirs []string

	// ShouldSkipDir, if non-nil, is called for each directory before descent.
	// The root directory is never skipped. Return true to skip the directory
	// and all descendants.
	ShouldSkipDir func(path string) bool

	// SkipExtensions lists file suffixes (e.g., ".min.js") to skip.
	SkipExtensions []string

	// ShouldParse, if non-nil, is called for each candidate file after
	// language detection. Return false to skip the file.
	ShouldParse func(path string, size int64, modTime time.Time) bool

	// SkipTreeParse, if non-nil, is called for candidate files that pass
	// ShouldParse. Return true to read the file (populating Source) but
	// skip the tree-sitter parse (Tree will be nil, Err will be nil).
	// This is useful for large generated files where the consumer wants
	// the raw source bytes for lightweight extraction without paying for
	// a full AST parse.
	SkipTreeParse func(path string, size int64) bool

	// OnProgress, if non-nil, receives progress events during the walk.
	OnProgress func(ProgressEvent)
}

ParsePolicy controls how WalkAndParse discovers and parses files. Configure concurrency limits, file size thresholds, directory/extension filters, and optional hooks for progress reporting and file filtering. Use DefaultPolicy to get a policy with sensible defaults.

func DefaultPolicy added in v0.8.0 

func DefaultPolicy() ParsePolicy

DefaultPolicy returns a ParsePolicy with sensible defaults for typical source repositories:

  • LargeFileThreshold: 256 KB (env: GTS_LARGE_FILE_THRESHOLD, in bytes)
  • MaxConcurrent: runtime.GOMAXPROCS(0) (env: GTS_MAX_CONCURRENT)
  • ChannelBuffer: MaxConcurrent + 1
  • SkipDirs: .git, .graft, .hg, .svn, vendor, node_modules
  • SkipExtensions: .min.js, .min.css, .map, .wasm

Environment variables are read on each call and must be positive integers. Invalid or non-positive values are silently ignored (defaults apply).

type ParseQuality 

type ParseQuality string

ParseQuality summarizes how trustworthy a grammar's parse output is. 

const (
	ParseQualityFull    ParseQuality = "full"    // token_source or dfa with scanner
	ParseQualityPartial ParseQuality = "partial" // dfa-partial (missing ext scanner)
	ParseQualityNone    ParseQuality = "none"    // cannot parse
)

type ParseSupport 

type ParseSupport struct {
	Name                    string
	LanguageVersion         uint32
	VersionCompatible       bool
	Backend                 ParseBackend
	Reason                  string
	HasTokenSourceFactory   bool
	HasDFALexer             bool
	RequiresExternalScanner bool
	HasExternalScanner      bool
}

ParseSupport summarizes parser support status for one registered language.

func AuditParseSupport 

func AuditParseSupport() []ParseSupport

AuditParseSupport evaluates parse support for all registered languages.

func EvaluateParseSupport 

func EvaluateParseSupport(entry LangEntry, lang *gotreesitter.Language) ParseSupport

EvaluateParseSupport reports whether a language can parse using either the built-in DFA lexer or a registered custom token source factory.

type ParsedFile added in v0.8.0 

type ParsedFile struct {
	Path   string                  // absolute path to the source file
	Tree   *gotreesitter.BoundTree // parsed AST; nil on error. Consumer MUST call Close()
	Lang   *LangEntry              // detected language entry
	Source []byte                  // raw file contents; nil on I/O error
	Size   int64                   // file size in bytes (from stat, before read)
	Err    error                   // nil on success
	IsRead bool                    // false = I/O error during read, true = file was read (parse may have failed)
}

ParsedFile is a single result from WalkAndParse. Each ParsedFile owns its BoundTree and Source slice. The consumer MUST call ParsedFile.Close when finished inspecting the tree to release arena memory. Failing to close results will leak memory proportional to the parsed file sizes.

When Err is non-nil, check IsRead to distinguish I/O errors (IsRead=false, file could not be read) from parse errors (IsRead=true, file was read but the grammar could not parse it). On I/O errors, Source and Tree are nil.

func (*ParsedFile) Close added in v0.8.0 

func (pf *ParsedFile) Close()

Close releases the BoundTree's arena memory and nils both Tree and Source to allow garbage collection. Close is safe to call multiple times and on a nil receiver.

Ownership rule: the consumer that receives a ParsedFile from the channel owns it. Once Close is called, Tree becomes nil and must not be used. Always defer Close or call it explicitly after processing each result: 

for pf := range ch {
    if pf.Err != nil { handleErr(pf); pf.Close(); continue }
    process(pf.Tree, pf.Source)
    pf.Close()
}

type PerlExternalScanner 

type PerlExternalScanner struct{}

PerlExternalScanner implements gotreesitter.ExternalScanner for Perl.

func (PerlExternalScanner) Create 

func (PerlExternalScanner) Create() any

func (PerlExternalScanner) Deserialize 

func (PerlExternalScanner) Deserialize(payload any, buf []byte)

func (PerlExternalScanner) Destroy 

func (PerlExternalScanner) Destroy(payload any)

func (PerlExternalScanner) Scan 

func (PerlExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (PerlExternalScanner) Serialize 

func (PerlExternalScanner) Serialize(payload any, buf []byte) int

type PhpExternalScanner 

type PhpExternalScanner struct{}

PhpExternalScanner handles heredocs, encapsed strings, auto-semicolons, and EOF for PHP.

func (PhpExternalScanner) Create 

func (PhpExternalScanner) Create() any

func (PhpExternalScanner) Deserialize 

func (PhpExternalScanner) Deserialize(payload any, buf []byte)

func (PhpExternalScanner) Destroy 

func (PhpExternalScanner) Destroy(payload any)

func (PhpExternalScanner) Scan 

func (PhpExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (PhpExternalScanner) Serialize 

func (PhpExternalScanner) Serialize(payload any, buf []byte) int

type PklExternalScanner 

type PklExternalScanner struct{}

PklExternalScanner handles string content and contextual operators for Pkl.

func (PklExternalScanner) Create 

func (PklExternalScanner) Create() any

func (PklExternalScanner) Deserialize 

func (PklExternalScanner) Deserialize(payload any, buf []byte)

func (PklExternalScanner) Destroy 

func (PklExternalScanner) Destroy(payload any)

func (PklExternalScanner) Scan 

func (PklExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (PklExternalScanner) Serialize 

func (PklExternalScanner) Serialize(payload any, buf []byte) int

type PowershellExternalScanner 

type PowershellExternalScanner struct{}

PowershellExternalScanner handles statement termination detection for PowerShell. A statement terminator is a zero-width token that fires when the next significant character is EOF, }, ;, ), or newline.

func (PowershellExternalScanner) Create 

func (PowershellExternalScanner) Create() any

func (PowershellExternalScanner) Deserialize 

func (PowershellExternalScanner) Deserialize(payload any, buf []byte)

func (PowershellExternalScanner) Destroy 

func (PowershellExternalScanner) Destroy(payload any)

func (PowershellExternalScanner) Scan 

func (PowershellExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (PowershellExternalScanner) Serialize 

func (PowershellExternalScanner) Serialize(payload any, buf []byte) int

type ProgressEvent added in v0.8.0 

type ProgressEvent struct {
	Phase   string // one of: "walking", "parsing", "large_file", "walk_complete", "done"
	Path    string // file path (empty for walk_complete and done phases)
	Size    int64  // file size in bytes
	FileNum int    // 1-based ordinal of this file among discovered files
	Total   int    // total files found (set only in walk_complete phase)
	Message string // optional human-readable detail
}

ProgressEvent reports progress during WalkAndParse. The Phase field indicates the stage of processing:

  • "walking" — a file was discovered and queued for parsing
  • "parsing" — a file is being parsed (normal or large)
  • "large_file" — a large file is acquiring exclusive access
  • "walk_complete" — directory walk finished; Total is set
  • "done" — all parsing complete, channel about to close

type PropertiesExternalScanner 

type PropertiesExternalScanner struct{}

PropertiesExternalScanner handles EOF detection for Java .properties files.

func (PropertiesExternalScanner) Create 

func (PropertiesExternalScanner) Create() any

func (PropertiesExternalScanner) Deserialize 

func (PropertiesExternalScanner) Deserialize(payload any, buf []byte)

func (PropertiesExternalScanner) Destroy 

func (PropertiesExternalScanner) Destroy(payload any)

func (PropertiesExternalScanner) Scan 

func (PropertiesExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (PropertiesExternalScanner) Serialize 

func (PropertiesExternalScanner) Serialize(payload any, buf []byte) int

type PugExternalScanner 

type PugExternalScanner struct{}

PugExternalScanner handles newline/indent/dedent for Pug templates.

func (PugExternalScanner) Create 

func (PugExternalScanner) Create() any

func (PugExternalScanner) Deserialize 

func (PugExternalScanner) Deserialize(payload any, buf []byte)

func (PugExternalScanner) Destroy 

func (PugExternalScanner) Destroy(payload any)

func (PugExternalScanner) Scan 

func (PugExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (PugExternalScanner) Serialize 

func (PugExternalScanner) Serialize(payload any, buf []byte) int

type PurescriptExternalScanner 

type PurescriptExternalScanner struct{}

PurescriptExternalScanner implements gotreesitter.ExternalScanner for tree-sitter-purescript.

This is a Go port of the C external scanner from tree-sitter-purescript (https://github.com/purerl/tree-sitter-purescript). The scanner handles layout-sensitive indentation for PureScript's offside rule, tracking an indent stack and emitting layout tokens (semicolons, starts, ends), as well as qualified module dots, comments, and various keyword tokens.

func (PurescriptExternalScanner) Create 

func (PurescriptExternalScanner) Create() any

func (PurescriptExternalScanner) Deserialize 

func (PurescriptExternalScanner) Deserialize(payload any, buf []byte)

func (PurescriptExternalScanner) Destroy 

func (PurescriptExternalScanner) Destroy(payload any)

func (PurescriptExternalScanner) Scan 

func (PurescriptExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (PurescriptExternalScanner) Serialize 

func (PurescriptExternalScanner) Serialize(payload any, buf []byte) int

type PythonExternalScanner 

type PythonExternalScanner struct{}

func (PythonExternalScanner) Create 

func (PythonExternalScanner) Create() any

func (PythonExternalScanner) Deserialize 

func (PythonExternalScanner) Deserialize(payload any, buf []byte)

func (PythonExternalScanner) Destroy 

func (PythonExternalScanner) Destroy(payload any)

func (PythonExternalScanner) Scan 

func (PythonExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (PythonExternalScanner) Serialize 

func (PythonExternalScanner) Serialize(payload any, buf []byte) int

func (PythonExternalScanner) SupportsIncrementalReuse added in v0.7.0 

func (PythonExternalScanner) SupportsIncrementalReuse() bool

type RExternalScanner 

type RExternalScanner struct{}

RExternalScanner implements gotreesitter.ExternalScanner for tree-sitter-r.

This is a Go port of the C external scanner from tree-sitter-r (https://github.com/r-lib/tree-sitter-r). The scanner handles:

  • _start: zero-width token emitted at the beginning of the file
  • _newline: contextual newlines in top-level and brace scopes
  • _semicolon: semicolons
  • _raw_string_literal: R raw string literals (r"(...)", R"[...]", etc.)
  • else: the 'else' keyword with special newline handling in brace scopes
  • bracket/brace/paren: scope tracking for (, ), {, }, [, ], [[, ]]
  • _error_sentinel: error recovery detection

func (RExternalScanner) Create 

func (RExternalScanner) Create() any

func (RExternalScanner) Deserialize 

func (RExternalScanner) Deserialize(payload any, buf []byte)

func (RExternalScanner) Destroy 

func (RExternalScanner) Destroy(payload any)

func (RExternalScanner) Scan 

func (RExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (RExternalScanner) Serialize 

func (RExternalScanner) Serialize(payload any, buf []byte) int

type RacketExternalScanner 

type RacketExternalScanner struct{}

RacketExternalScanner handles #<<DELIM here-strings for Racket. The scanner reads the terminator from the first line, then consumes subsequent lines until it finds one matching the terminator exactly.

func (RacketExternalScanner) Create 

func (RacketExternalScanner) Create() any

func (RacketExternalScanner) Deserialize 

func (RacketExternalScanner) Deserialize(payload any, buf []byte)

func (RacketExternalScanner) Destroy 

func (RacketExternalScanner) Destroy(payload any)

func (RacketExternalScanner) Scan 

func (RacketExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (RacketExternalScanner) Serialize 

func (RacketExternalScanner) Serialize(payload any, buf []byte) int

type RescriptExternalScanner added in v0.4.0 

type RescriptExternalScanner struct{}

RescriptExternalScanner implements gotreesitter.ExternalScanner for tree-sitter-rescript.

This is a Go port of the C external scanner from rescript-lang/tree-sitter-rescript. The scanner handles 12 external tokens: newlines (with statement-termination semantics), line and block comments, template string characters, string delimiters (" and `), parenthesis nesting, list/dict constructors, and decorator identifiers.

func (RescriptExternalScanner) Create added in v0.4.0 

func (RescriptExternalScanner) Create() any

func (RescriptExternalScanner) Deserialize added in v0.4.0 

func (RescriptExternalScanner) Deserialize(payload any, buf []byte)

func (RescriptExternalScanner) Destroy added in v0.4.0 

func (RescriptExternalScanner) Destroy(payload any)

func (RescriptExternalScanner) Scan added in v0.4.0 

func (RescriptExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (RescriptExternalScanner) Serialize added in v0.4.0 

func (RescriptExternalScanner) Serialize(payload any, buf []byte) int

type RonExternalScanner 

type RonExternalScanner struct{}

RonExternalScanner handles string content, raw strings, floats, and nestable block comments for RON (Rusty Object Notation).

func (RonExternalScanner) Create 

func (RonExternalScanner) Create() any

func (RonExternalScanner) Deserialize 

func (RonExternalScanner) Deserialize(payload any, buf []byte)

func (RonExternalScanner) Destroy 

func (RonExternalScanner) Destroy(payload any)

func (RonExternalScanner) Scan 

func (RonExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (RonExternalScanner) Serialize 

func (RonExternalScanner) Serialize(payload any, buf []byte) int

type RstExternalScanner added in v0.4.0 

type RstExternalScanner struct{}

func (RstExternalScanner) Create added in v0.4.0 

func (RstExternalScanner) Create() any

func (RstExternalScanner) Deserialize added in v0.4.0 

func (RstExternalScanner) Deserialize(payload any, buf []byte)

func (RstExternalScanner) Destroy added in v0.4.0 

func (RstExternalScanner) Destroy(payload any)

func (RstExternalScanner) Scan added in v0.4.0 

func (RstExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (RstExternalScanner) Serialize added in v0.4.0 

func (RstExternalScanner) Serialize(payload any, buf []byte) int

type RubyExternalScanner 

type RubyExternalScanner struct{}

RubyExternalScanner handles all external tokens for the Ruby grammar.

func (RubyExternalScanner) Create 

func (RubyExternalScanner) Create() any

func (RubyExternalScanner) Deserialize 

func (RubyExternalScanner) Deserialize(payload any, buf []byte)

func (RubyExternalScanner) Destroy 

func (RubyExternalScanner) Destroy(payload any)

func (RubyExternalScanner) Scan 

func (RubyExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (RubyExternalScanner) Serialize 

func (RubyExternalScanner) Serialize(payload any, buf []byte) int

type RustExternalScanner added in v0.4.0 

type RustExternalScanner struct{}

RustExternalScanner implements gotreesitter.ExternalScanner for tree-sitter-rust.

This is a Go port of the C external scanner from tree-sitter/tree-sitter-rust. The scanner handles 10 external tokens:

  • String content (regular string escape sequences)
  • Raw string literals (r#"..."# with hash counting for start/content/end)
  • Float literals (disambiguation from integer + method call)
  • Block comment content (nested /* */ comments)
  • Doc comments (/// and //!)
  • Outer/inner doc comment markers
  • Error sentinel for error recovery detection

func (RustExternalScanner) Create added in v0.4.0 

func (RustExternalScanner) Create() any

func (RustExternalScanner) Deserialize added in v0.4.0 

func (RustExternalScanner) Deserialize(payload any, buf []byte)

func (RustExternalScanner) Destroy added in v0.4.0 

func (RustExternalScanner) Destroy(payload any)

func (RustExternalScanner) Scan added in v0.4.0 

func (RustExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (RustExternalScanner) Serialize added in v0.4.0 

func (RustExternalScanner) Serialize(payload any, buf []byte) int

type ScalaExternalScanner 

type ScalaExternalScanner struct{}

ScalaExternalScanner handles auto-semicolons, indent/outdent, and string scanning for Scala.

func (ScalaExternalScanner) Create 

func (ScalaExternalScanner) Create() any

func (ScalaExternalScanner) Deserialize 

func (ScalaExternalScanner) Deserialize(payload any, buf []byte)

func (ScalaExternalScanner) Destroy 

func (ScalaExternalScanner) Destroy(payload any)

func (ScalaExternalScanner) Scan 

func (ScalaExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (ScalaExternalScanner) Serialize 

func (ScalaExternalScanner) Serialize(payload any, buf []byte) int

type ScssExternalScanner 

type ScssExternalScanner struct{}

ScssExternalScanner implements gotreesitter.ExternalScanner for tree-sitter-scss.

Ported from tree-sitter-scss/src/scanner.c (pinned commit bca847c). The scanner handles four external tokens:

  • _descendant_operator: whitespace between two selectors (e.g., "div p")
  • _pseudo_class_selector_colon: colon in selector context (e.g., ":hover")
  • __error_recovery: sentinel — always declined
  • _concat: adjacent selector concatenation (e.g., "#foo.bar")

func (ScssExternalScanner) Create 

func (ScssExternalScanner) Create() any

func (ScssExternalScanner) Deserialize 

func (ScssExternalScanner) Deserialize(payload any, buf []byte)

func (ScssExternalScanner) Destroy 

func (ScssExternalScanner) Destroy(payload any)

func (ScssExternalScanner) Scan 

func (ScssExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (ScssExternalScanner) Serialize 

func (ScssExternalScanner) Serialize(payload any, buf []byte) int

func (ScssExternalScanner) SupportsIncrementalReuse added in v0.7.0 

func (ScssExternalScanner) SupportsIncrementalReuse() bool

type SqlExternalScanner 

type SqlExternalScanner struct{}

SqlExternalScanner implements gotreesitter.ExternalScanner for tree-sitter-sql.

This is a Go port of the C external scanner from DerekStride/tree-sitter-sql. The scanner handles PostgreSQL dollar-quoted strings: $tag$..content..$tag$. It uses state to remember the opening tag so the closing tag can be matched.

func (SqlExternalScanner) Create 

func (SqlExternalScanner) Create() any

func (SqlExternalScanner) Deserialize 

func (SqlExternalScanner) Deserialize(payload any, buf []byte)

func (SqlExternalScanner) Destroy 

func (SqlExternalScanner) Destroy(payload any)

func (SqlExternalScanner) Scan 

func (SqlExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (SqlExternalScanner) Serialize 

func (SqlExternalScanner) Serialize(payload any, buf []byte) int

type SquirrelExternalScanner 

type SquirrelExternalScanner struct{}

SquirrelExternalScanner handles @"..." verbatim string literals for Squirrel. Inside a verbatim string, "" is an escaped double-quote.

func (SquirrelExternalScanner) Create 

func (SquirrelExternalScanner) Create() any

func (SquirrelExternalScanner) Deserialize 

func (SquirrelExternalScanner) Deserialize(payload any, buf []byte)

func (SquirrelExternalScanner) Destroy 

func (SquirrelExternalScanner) Destroy(payload any)

func (SquirrelExternalScanner) Scan 

func (SquirrelExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (SquirrelExternalScanner) Serialize 

func (SquirrelExternalScanner) Serialize(payload any, buf []byte) int

type StarlarkExternalScanner 

type StarlarkExternalScanner struct{}

StarlarkExternalScanner handles indent/dedent and string literals for Starlark. Starlark is essentially Python syntax; this reuses the pythonScannerState type.

func (StarlarkExternalScanner) Create 

func (StarlarkExternalScanner) Create() any

func (StarlarkExternalScanner) Deserialize 

func (StarlarkExternalScanner) Deserialize(payload any, buf []byte)

func (StarlarkExternalScanner) Destroy 

func (StarlarkExternalScanner) Destroy(payload any)

func (StarlarkExternalScanner) Scan 

func (StarlarkExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (StarlarkExternalScanner) Serialize 

func (StarlarkExternalScanner) Serialize(payload any, buf []byte) int

func (StarlarkExternalScanner) SupportsIncrementalReuse added in v0.7.0 

func (StarlarkExternalScanner) SupportsIncrementalReuse() bool

type SvelteExternalScanner added in v0.4.0 

type SvelteExternalScanner struct{}

SvelteExternalScanner handles HTML tag tracking plus Svelte-specific raw text scanning (for expression blocks like {#each}, {@html}, etc.) and special sigil characters (@, #, /, :).

func (SvelteExternalScanner) Create added in v0.4.0 

func (SvelteExternalScanner) Create() any

func (SvelteExternalScanner) Deserialize added in v0.4.0 

func (SvelteExternalScanner) Deserialize(payload any, buf []byte)

func (SvelteExternalScanner) Destroy added in v0.4.0 

func (SvelteExternalScanner) Destroy(any)

func (SvelteExternalScanner) Scan added in v0.4.0 

func (SvelteExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (SvelteExternalScanner) Serialize added in v0.4.0 

func (SvelteExternalScanner) Serialize(payload any, buf []byte) int

type SwiftExternalScanner added in v0.7.1 

type SwiftExternalScanner struct {
	// contains filtered or unexported fields
}

SwiftExternalScanner handles all external tokens for the Swift grammar.

func (SwiftExternalScanner) Create added in v0.7.1 

func (SwiftExternalScanner) Create() any

func (SwiftExternalScanner) Deserialize added in v0.7.1 

func (SwiftExternalScanner) Deserialize(payload any, buf []byte)

func (SwiftExternalScanner) Destroy added in v0.7.1 

func (SwiftExternalScanner) Destroy(payload any)

func (SwiftExternalScanner) ExternalScannerForLanguage added in v0.16.0 

func (SwiftExternalScanner) ExternalScannerForLanguage(lang *gotreesitter.Language) gotreesitter.ExternalScanner

func (SwiftExternalScanner) Scan added in v0.7.1 

func (s SwiftExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (SwiftExternalScanner) Serialize added in v0.7.1 

func (SwiftExternalScanner) Serialize(payload any, buf []byte) int

type TablegenExternalScanner 

type TablegenExternalScanner struct{}

TablegenExternalScanner handles nestable /* */ comments for TableGen.

func (TablegenExternalScanner) Create 

func (TablegenExternalScanner) Create() any

func (TablegenExternalScanner) Deserialize 

func (TablegenExternalScanner) Deserialize(payload any, buf []byte)

func (TablegenExternalScanner) Destroy 

func (TablegenExternalScanner) Destroy(payload any)

func (TablegenExternalScanner) Scan 

func (TablegenExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (TablegenExternalScanner) Serialize 

func (TablegenExternalScanner) Serialize(payload any, buf []byte) int

type TclExternalScanner 

type TclExternalScanner struct{}

TclExternalScanner handles concatenation detection for Tcl.

func (TclExternalScanner) Create 

func (TclExternalScanner) Create() any

func (TclExternalScanner) Deserialize 

func (TclExternalScanner) Deserialize(payload any, buf []byte)

func (TclExternalScanner) Destroy 

func (TclExternalScanner) Destroy(payload any)

func (TclExternalScanner) Scan 

func (TclExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (TclExternalScanner) Serialize 

func (TclExternalScanner) Serialize(payload any, buf []byte) int

type TealExternalScanner 

type TealExternalScanner struct{}

TealExternalScanner handles Teal/Lua string and comment scanning.

func (TealExternalScanner) Create 

func (TealExternalScanner) Create() any

func (TealExternalScanner) Deserialize 

func (TealExternalScanner) Deserialize(payload any, buf []byte)

func (TealExternalScanner) Destroy 

func (TealExternalScanner) Destroy(payload any)

func (TealExternalScanner) Scan 

func (TealExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (TealExternalScanner) Serialize 

func (TealExternalScanner) Serialize(payload any, buf []byte) int

type TemplExternalScanner 

type TemplExternalScanner struct{}

TemplExternalScanner handles CSS property values, script blocks, and element text for templ.

func (TemplExternalScanner) Create 

func (TemplExternalScanner) Create() any

func (TemplExternalScanner) Deserialize 

func (TemplExternalScanner) Deserialize(payload any, buf []byte)

func (TemplExternalScanner) Destroy 

func (TemplExternalScanner) Destroy(payload any)

func (TemplExternalScanner) Scan 

func (TemplExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (TemplExternalScanner) Serialize 

func (TemplExternalScanner) Serialize(payload any, buf []byte) int

type TlaplusExternalScanner 

type TlaplusExternalScanner struct{}

TlaplusExternalScanner handles junction lists, proofs, extramodular text, fairness operators, and PlusCal context nesting for the TLA+ grammar.

func (TlaplusExternalScanner) Create 

func (TlaplusExternalScanner) Create() any

func (TlaplusExternalScanner) Deserialize 

func (TlaplusExternalScanner) Deserialize(payload any, buf []byte)

func (TlaplusExternalScanner) Destroy 

func (TlaplusExternalScanner) Destroy(payload any)

func (TlaplusExternalScanner) Scan 

func (TlaplusExternalScanner) Scan(
	payload any,
	lexer *gotreesitter.ExternalLexer,
	validSymbols []bool,
) bool

func (TlaplusExternalScanner) Serialize 

func (TlaplusExternalScanner) Serialize(payload any, buf []byte) int

type TomlTokenSource 

type TomlTokenSource struct {
	// contains filtered or unexported fields
}

TomlTokenSource is a lightweight lexer bridge for tree-sitter-toml. It focuses on practical coverage for common editor workflows and incremental parsing.

func NewTomlTokenSource 

func NewTomlTokenSource(src []byte, lang *gotreesitter.Language) (*TomlTokenSource, error)

NewTomlTokenSource creates a token source for TOML source text.

func (*TomlTokenSource) Next 

func (ts *TomlTokenSource) Next() gotreesitter.Token

func (*TomlTokenSource) Reset added in v0.7.0 

func (ts *TomlTokenSource) Reset(src []byte)

Reset reinitializes this token source for a new source buffer.

func (*TomlTokenSource) SkipToByte 

func (ts *TomlTokenSource) SkipToByte(offset uint32) gotreesitter.Token

func (*TomlTokenSource) SupportsIncrementalReuse added in v0.7.0 

func (ts *TomlTokenSource) SupportsIncrementalReuse() bool

SupportsIncrementalReuse reports that TomlTokenSource preserves stable token boundaries across edits and supports deterministic SkipToByte behavior.

type TsxExternalScanner 

type TsxExternalScanner struct{}

TsxExternalScanner handles automatic semicolons, template strings, JSX text, ternary question marks, and HTML comments for TSX.

func (TsxExternalScanner) Create 

func (TsxExternalScanner) Create() any

func (TsxExternalScanner) Deserialize 

func (TsxExternalScanner) Deserialize(payload any, buf []byte)

func (TsxExternalScanner) Destroy 

func (TsxExternalScanner) Destroy(payload any)

func (TsxExternalScanner) Scan 

func (TsxExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (TsxExternalScanner) Serialize 

func (TsxExternalScanner) Serialize(payload any, buf []byte) int

func (TsxExternalScanner) SupportsIncrementalReuse added in v0.7.0 

func (TsxExternalScanner) SupportsIncrementalReuse() bool

type TypeScriptExternalScanner added in v0.4.0 

type TypeScriptExternalScanner struct{}

TypeScriptExternalScanner handles automatic semicolons, template strings, JSX text, ternary question marks, and HTML comments for TypeScript.

func (TypeScriptExternalScanner) Create added in v0.4.0 

func (TypeScriptExternalScanner) Create() any

func (TypeScriptExternalScanner) Deserialize added in v0.4.0 

func (TypeScriptExternalScanner) Deserialize(payload any, buf []byte)

func (TypeScriptExternalScanner) Destroy added in v0.4.0 

func (TypeScriptExternalScanner) Destroy(payload any)

func (TypeScriptExternalScanner) Scan added in v0.4.0 

func (TypeScriptExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (TypeScriptExternalScanner) Serialize added in v0.4.0 

func (TypeScriptExternalScanner) Serialize(payload any, buf []byte) int

func (TypeScriptExternalScanner) SupportsIncrementalReuse added in v0.7.0 

func (TypeScriptExternalScanner) SupportsIncrementalReuse() bool

type TypstExternalScanner 

type TypstExternalScanner struct{}

TypstExternalScanner implements gotreesitter.ExternalScanner for tree-sitter-typst.

This is a Go port of the C external scanner from tree-sitter-typst (https://github.com/Enter-tainer/tree-sitter-typst). The scanner handles 49 external tokens including indentation tracking, content/strong/emph containers, raw blocks, comments, headings, list items, math mode, and more.

func (TypstExternalScanner) Create 

func (TypstExternalScanner) Create() any

func (TypstExternalScanner) Deserialize 

func (TypstExternalScanner) Deserialize(payload any, buf []byte)

func (TypstExternalScanner) Destroy 

func (TypstExternalScanner) Destroy(payload any)

func (TypstExternalScanner) Scan 

func (TypstExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (TypstExternalScanner) Serialize 

func (TypstExternalScanner) Serialize(payload any, buf []byte) int

type UxntalExternalScanner 

type UxntalExternalScanner struct{}

UxntalExternalScanner handles nestable ( ) Forth-style comments for Uxntal.

func (UxntalExternalScanner) Create 

func (UxntalExternalScanner) Create() any

func (UxntalExternalScanner) Deserialize 

func (UxntalExternalScanner) Deserialize(payload any, buf []byte)

func (UxntalExternalScanner) Destroy 

func (UxntalExternalScanner) Destroy(payload any)

func (UxntalExternalScanner) Scan 

func (UxntalExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (UxntalExternalScanner) Serialize 

func (UxntalExternalScanner) Serialize(payload any, buf []byte) int

type VhdlExternalScanner added in v0.4.0 

type VhdlExternalScanner struct{}

VhdlExternalScanner implements gotreesitter.ExternalScanner for tree-sitter-vhdl (jpt13653903). It handles identifier/keyword matching, number/string literals, comments, directives, and library tokens.

func (VhdlExternalScanner) Create added in v0.4.0 

func (VhdlExternalScanner) Create() any

func (VhdlExternalScanner) Deserialize added in v0.4.0 

func (VhdlExternalScanner) Deserialize(payload any, buf []byte)

func (VhdlExternalScanner) Destroy added in v0.4.0 

func (VhdlExternalScanner) Destroy(payload any)

func (VhdlExternalScanner) Scan added in v0.4.0 

func (VhdlExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (VhdlExternalScanner) Serialize added in v0.4.0 

func (VhdlExternalScanner) Serialize(payload any, buf []byte) int

type VueExternalScanner 

type VueExternalScanner struct{}

VueExternalScanner handles HTML tag tracking plus Vue-specific text fragments and interpolation.

func (VueExternalScanner) Create 

func (VueExternalScanner) Create() any

func (VueExternalScanner) Deserialize 

func (VueExternalScanner) Deserialize(payload any, buf []byte)

func (VueExternalScanner) Destroy 

func (VueExternalScanner) Destroy(payload any)

func (VueExternalScanner) Scan 

func (VueExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (VueExternalScanner) Serialize 

func (VueExternalScanner) Serialize(payload any, buf []byte) int

type WalkStats added in v0.8.0 

type WalkStats struct {
	FilesFound    int   // files with a recognized language extension
	FilesParsed   int   // files successfully parsed into a BoundTree
	FilesFailed   int   // files that encountered read or parse errors
	FilesFiltered int   // files skipped by SkipExtensions or ShouldParse
	LargeFiles    int   // files at or above LargeFileThreshold
	BinarySkipped int   // files detected as binary (NUL in first 8 KB)
	BytesParsed   int64 // total bytes of successfully parsed files
}

WalkStats summarizes the results of a WalkAndParse run. The stats function returned by WalkAndParse blocks until all work is done, then returns this snapshot.

type WgslExternalScanner 

type WgslExternalScanner struct{}

WgslExternalScanner handles nestable /* */ block comments for WGSL.

func (WgslExternalScanner) Create 

func (WgslExternalScanner) Create() any

func (WgslExternalScanner) Deserialize 

func (WgslExternalScanner) Deserialize(payload any, buf []byte)

func (WgslExternalScanner) Destroy 

func (WgslExternalScanner) Destroy(payload any)

func (WgslExternalScanner) Scan 

func (WgslExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (WgslExternalScanner) Serialize 

func (WgslExternalScanner) Serialize(payload any, buf []byte) int

type WolframExternalScanner 

type WolframExternalScanner struct{}

WolframExternalScanner handles nestable (* *) comments for Wolfram/Mathematica.

func (WolframExternalScanner) Create 

func (WolframExternalScanner) Create() any

func (WolframExternalScanner) Deserialize 

func (WolframExternalScanner) Deserialize(payload any, buf []byte)

func (WolframExternalScanner) Destroy 

func (WolframExternalScanner) Destroy(payload any)

func (WolframExternalScanner) Scan 

func (WolframExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (WolframExternalScanner) Serialize 

func (WolframExternalScanner) Serialize(payload any, buf []byte) int

type XMLExternalScanner 

type XMLExternalScanner struct{}

XMLExternalScanner implements gotreesitter.ExternalScanner for tree-sitter-xml.

This is a Go port of the C external scanner from tree-sitter-xml (https://github.com/tree-sitter-grammars/tree-sitter-xml). The scanner manages a tag name stack and handles 11 external tokens: PITarget, PIContent, Comment, CharData, CData, xml-model, xml-stylesheet, StartTagName, EndTagName, ErroneousEndName, and SelfClosingTagDelimiter.

func (XMLExternalScanner) Create 

func (XMLExternalScanner) Create() any

func (XMLExternalScanner) Deserialize 

func (XMLExternalScanner) Deserialize(payload any, buf []byte)

func (XMLExternalScanner) Destroy 

func (XMLExternalScanner) Destroy(payload any)

func (XMLExternalScanner) Scan 

func (XMLExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (XMLExternalScanner) Serialize 

func (XMLExternalScanner) Serialize(payload any, buf []byte) int

type YamlExternalScanner added in v0.6.0 

type YamlExternalScanner struct{}

func (YamlExternalScanner) Create added in v0.6.0 

func (YamlExternalScanner) Create() any

func (YamlExternalScanner) Deserialize added in v0.6.0 

func (YamlExternalScanner) Deserialize(payload any, buf []byte)

func (YamlExternalScanner) Destroy added in v0.6.0 

func (YamlExternalScanner) Destroy(payload any)

func (YamlExternalScanner) Scan added in v0.6.0 

func (YamlExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (YamlExternalScanner) Serialize added in v0.6.0 

func (YamlExternalScanner) Serialize(payload any, buf []byte) int

type YuckExternalScanner 

type YuckExternalScanner struct{}

YuckExternalScanner implements gotreesitter.ExternalScanner for tree-sitter-yuck. Handles unescaped string fragments for single-quote, double-quote, and backtick strings.

func (YuckExternalScanner) Create 

func (YuckExternalScanner) Create() any

func (YuckExternalScanner) Deserialize 

func (YuckExternalScanner) Deserialize(payload any, buf []byte)

func (YuckExternalScanner) Destroy 

func (YuckExternalScanner) Destroy(payload any)

func (YuckExternalScanner) Scan 

func (YuckExternalScanner) Scan(payload any, lexer *gotreesitter.ExternalLexer, validSymbols []bool) bool

func (YuckExternalScanner) Serialize 

func (YuckExternalScanner) Serialize(payload any, buf []byte) int

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.