goparser

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 4, 2026 License: BSD-3-Clause Imports: 18 Imported by: 0

README

Parser

See if we can build an almost single pass compiler for Go doing syntax directed translation, without any complex data structure (no syntax tree), only lists of tokens.

The goal is to have the shortest and simplest path from source to bytecode.

Design

The input of parser is a list of tokens produced by the scanner. Multiple tokens are processed at once. The minimal set to get meaningful results (not an error or nil) is a complete statement.

The output of parser is also a list of tokens, to be consumed by the compiler to produce bytecode. The output tokens set is identical to the bytecode instructions set except that:

  • code locations may be provided as labels instead of numerical values,
  • memory locations for constants and variables may be provided as symbol names instead of numerical values.

Status

Go language support:

  • named functions
  • anonymous functions (closures)
  • methods
  • internal function calls
  • external function calls (calling runtime symbols in interpreter)
  • export to runtime
  • builtin calls (new, make, copy, delete, len, cap, ...)
  • out of order declarations
  • arbirtrary precision constants
  • basic types
  • complete numeric types
  • complex numbers
  • function types
  • variadic functions
  • pointers
  • structures
  • embedded structures
  • recursive structures
  • literal composite objects
  • interfaces
  • arrays, slices
  • maps
  • deterministic maps
  • channel types
  • channel operations
  • [x]� multi-assign expressions
  • var defined by assign :=
  • var assign =
  • var declaration
  • type declaration
  • func declaration
  • const declaration
  • iota expression
  • panic statement
  • defer statement
  • recover statement
  • range clause
  • go statement
  • if statement (including else and else if)
  • for statement
  • switch statement
  • type switch statement
  • break statement
  • continue statement
  • fallthrough statement
  • goto statement
  • label statement
  • select statement
  • binary operators
  • unary operators
  • logical operators && and ||
  • assign operators
  • operator precedence rules
  • parenthesis expressions
  • call expressions
  • index expressions
  • selector expressions
  • slice expressions
  • type convertions
  • type assertions
  • parametric types (generic)
  • type parametric functions (generic)
  • type constraints (generic)
  • type checking
  • comment pragmas
  • package import
  • init functions
  • modules

Other items:

  • REPL
  • multiline statements in REPL
  • completion, history in REPL
  • eval strings
  • eval files (including stdin, ...)
  • debug traces for scanner, parser, compiler, bytecode vm
  • simple interpreter tests to exercise from source to execution
  • compile time error detection and diagnosis
  • stack dump
  • symbol tables, data tables, code binded to source lines
  • interactive debugger: breaks, continue, instrospection, ...
  • machine level debugger
  • source level debugger
  • replay debugger, backward instruction execution
  • vm monitor: live memory / code display during run
  • stdlib wrappers a la yaegi
  • system and environment sandboxing
  • build constraints (arch, sys, etc)
  • test command (running go test / benchmark / example files)
  • skipping / including test files
  • test coverage
  • fuzzy tests for scanner, vm, ...

Documentation

Overview

Package goparser implements a structured parser for Go.

Index

Constants

This section is empty.

Variables

View Source 
var (
	ErrEllipsisArray  = errors.New("[...] array")
	ErrFuncType       = errors.New("invalid function type")
	ErrInvalidType    = errors.New("invalid type")
	ErrMissingType    = errors.New("missing type")
	ErrSize           = errors.New("invalid size")
	ErrSyntax         = errors.New("syntax error")
	ErrNotImplemented = errors.New("not implemented")
)

Type parsing error definitions.

Functions

func IsExported 

func IsExported(name string) bool

IsExported reports whether the given name starts with an upper-case letter.

func MatchFileName 

func MatchFileName(name string, ctx *buildContext) bool

MatchFileName reports whether name (a .go file basename) matches the given build context's GOOS/GOARCH constraints encoded in the file name.

func MatchFileNameFor 

func MatchFileNameFor(name, goos, goarch string) bool

MatchFileNameFor reports whether name matches the given GOOS/GOARCH constraints encoded in the file name. It is like MatchFileName but for an explicit platform.

func PackageName 

func PackageName(importPath string) string

PackageName returns the identifier used to reference the package given its import path: the last segment, or the second-to-last when the last matches "v[0-9]*" (module versioning suffix).

Types

type ErrUndefined 

type ErrUndefined struct{ Name string }

ErrUndefined is returned during parsing when a referenced symbol is not yet defined. It is retryable: the lazy fixpoint loop in interp.Eval defers the declaration and retries after other declarations have been processed.

func (ErrUndefined) Error 

func (e ErrUndefined) Error() string

type Parser 

type Parser struct {
	*scan.Scanner

	Symbols  symbol.SymMap
	Packages map[string]*symbol.Package

	InitFuncs []string // ordered list of init function internal names
	// contains filtered or unexported fields
}

Parser represents the state of a parser.

func NewParser 

func NewParser(spec *lang.Spec, noPkg bool) *Parser

NewParser returns a new parser.

func (*Parser) ImportPackageValues 

func (p *Parser) ImportPackageValues(m map[string]map[string]reflect.Value)

ImportPackageValues populates packages with values.

func (*Parser) ParseAll 

func (p *Parser) ParseAll(name, src string) (out []Tokens, err error)

ParseAll parses code and its dependencies, and returns slices of Tokens or an error.

func (*Parser) ParseDecl 

func (p *Parser) ParseDecl(toks Tokens) (handled bool, err error)

ParseDecl resolves a declaration's symbols (Phase 1) without emitting code. Returns handled=true if fully resolved, false if code generation is needed.

func (*Parser) ParseOneStmt 

func (p *Parser) ParseOneStmt(toks Tokens) (Tokens, error)

ParseOneStmt parses a single pre-scanned statement token slice.

func (*Parser) SetBuildContext 

func (p *Parser) SetBuildContext(goos, goarch string)

SetBuildContext overrides the parser's target GOOS/GOARCH for build constraint filtering.

func (*Parser) SetPkgfs 

func (p *Parser) SetPkgfs(pkgPath string)

SetPkgfs sets the parser virtual filesystem for reading sources.

func (*Parser) SetStdlibFS 

func (p *Parser) SetStdlibFS(fsys fs.FS)

SetStdlibFS installs a fallback filesystem for resolving imported source packages that are not present in the primary pkgfs. This is used to resolve generics-first stdlib packages (cmp, slices, maps, ...) whose sources are embedded in the interpreter binary.

func (*Parser) SymAdd 

func (p *Parser) SymAdd(i int, name string, v vm.Value, k symbol.Kind, t *vm.Type)

SymAdd adds a new named symbol, recording the key for potential rollback.

func (*Parser) SymSet 

func (p *Parser) SymSet(key string, sym *symbol.Symbol)

SymSet inserts sym at key in the symbol table, recording the key for potential rollback.

type SelectCaseDesc 

type SelectCaseDesc struct {
	Dir     reflect.SelectDir
	ValName string // scoped name of recv value var ("" if none)
	OkName  string // scoped name of recv ok var ("" if none)
}

SelectCaseDesc describes one case of a select statement for the compiler.

type Token 

type Token struct {
	scan.Token
	Arg []any
}

Token represents a parser token.

type Tokens 

type Tokens []Token

Tokens represents slice of tokens.

func (Tokens) Index 

func (toks Tokens) Index(tok lang.Token) int

Index returns the index in toks of the first matching tok, or -1.

func (Tokens) LastIndex 

func (toks Tokens) LastIndex(tok lang.Token) int

LastIndex returns the index in toks of the last matching tok, or -1.

func (Tokens) Split 

func (toks Tokens) Split(tok lang.Token) (result []Tokens)

Split returns a slice of token arrays, separated by tok.

func (Tokens) SplitStart 

func (toks Tokens) SplitStart(tok lang.Token) (result []Tokens)

SplitStart is similar to Split, except the first token in toks is skipped.

func (Tokens) String 

func (toks Tokens) String() (s string)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.