lexer

package

v1.62.3 Latest Latest Go to latest Published: Sep 22, 2026 License: BSD-3-Clause Imports: 7 Imported by: 0

Documentation ¶

Index ¶

type LexFn
type Lexer
- func New(s *token.Scanner) *Lexer
- func (lex *Lexer) Err() error
- func (lex *Lexer) ReadToken() []*token.Token

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

This section is empty.

Types ¶

type LexFn ¶

type LexFn func(*Lexer) []*token.Token

type Lexer ¶

type Lexer struct {
	// contains filtered or unexported fields
}

func New ¶

func New(s *token.Scanner) *Lexer

New creates a lexer, ignoring a single UTF-8 BOM only at source offset zero. The BOM is not emitted, so format-preserving readers also drop it on output.

func (*Lexer) Err ¶ added in v1.62.0

func (lex *Lexer) Err() error

Err returns the terminal scanner error, if any. Lexical errors such as malformed literals do not set Err and may allow parsing to continue.

func (*Lexer) ReadToken ¶

func (lex *Lexer) ReadToken() []*token.Token

ReadToken returns the next tokens. The returned slice is valid only until the next call; callers retaining it must copy its elements. The Token and Location pointers themselves remain valid and are never reused.

Source Files ¶

View all Source files

lexer.go

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL