lexer

package

v0.29.0 Latest Latest Go to latest Published: Oct 5, 2026 License: MIT Imports: 3 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/paularlott/scriptling

Links

Open Source Insights

Documentation ¶

Index ¶

Constants
func AppendEscape(b *strings.Builder, s string, i int, bytesMode bool) int
func UnescapeString(s string, bytesMode bool) string
type Lexer
- func New(input string) *Lexer
- func (l *Lexer) NextToken() token.Token
- func (l *Lexer) Reset(input string)

Constants ¶

View Source

const UnterminatedString = "unterminated string literal"

UnterminatedString is the literal of the ILLEGAL token for a string literal with no closing quote.

Variables ¶

This section is empty.

Functions ¶

func AppendEscape ¶ added in v0.28.0

func AppendEscape(b *strings.Builder, s string, i int, bytesMode bool) int

AppendEscape decodes the escape sequence starting at the backslash s[i], writes the result to b and returns the index just past it. Sequences that are not escapes (\d, \N{...}, an incomplete \x) are kept as written, as Python keeps unknown escapes.

func UnescapeString ¶ added in v0.28.0

func UnescapeString(s string, bytesMode bool) string

UnescapeString decodes Python escape sequences in a string literal's content. In bytesMode, \x and octal escapes give raw bytes and \u, \U and \N are not escapes (as in a bytes literal).

Types ¶

type Lexer ¶

type Lexer struct {
	// contains filtered or unexported fields
}

func New ¶

func New(input string) *Lexer

func (*Lexer) NextToken ¶

func (l *Lexer) NextToken() token.Token

func (*Lexer) Reset ¶ added in v0.5.6

func (l *Lexer) Reset(input string)

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL