lexer

package

v1.0.0 Latest Latest Go to latest Published: Jan 2, 2026 License: GPL-3.0 Imports: 4 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/atinylittleshell/gsh

Links

Open Source Insights

Documentation ¶

Index ¶

func IsKeyword(t TokenType) bool
type Lexer
- func New(input string) *Lexer
type Token
type TokenType
- func LookupIdent(ident string) TokenType

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

func IsKeyword ¶

func IsKeyword(t TokenType) bool

IsKeyword returns true if the token type is a keyword

Types ¶

type Lexer ¶

type Lexer struct {
	// contains filtered or unexported fields
}

Lexer tokenizes gsh script source code

func New ¶

func New(input string) *Lexer

New creates a new Lexer instance

func (*Lexer) Error ¶

func (l *Lexer) Error(msg string) string

Error returns a formatted error message with line and column information

func (*Lexer) Errors ¶

func (l *Lexer) Errors() []string

Errors returns the list of lexer errors

func (*Lexer) NextToken ¶

func (l *Lexer) NextToken() Token

NextToken returns the next token from the input

type Token ¶

type Token struct {
	Type    TokenType
	Literal string
	Line    int
	Column  int
}

Token represents a lexical token

type TokenType ¶

type TokenType int

TokenType represents the type of a token

const (
	// Special tokens
	ILLEGAL TokenType = iota
	EOF
	COMMENT

	// Identifiers and literals
	IDENT            // variable names, function names
	NUMBER           // 123, 45.67
	STRING           // "hello", 'world', """multiline"""
	TEMPLATE_LITERAL // `template ${expr}`

	// Keywords
	KW_MCP
	KW_MODEL
	KW_AGENT
	KW_TOOL
	KW_IF
	KW_ELSE
	KW_FOR
	KW_OF
	KW_WHILE
	KW_BREAK
	KW_CONTINUE
	KW_TRY
	KW_CATCH
	KW_FINALLY
	KW_RETURN
	KW_IMPORT
	KW_EXPORT
	KW_FROM
	KW_GO // Reserved for future concurrency support (fire-and-forget)

	// Operators
	OP_ASSIGN   // =
	OP_PLUS     // +
	OP_MINUS    // -
	OP_ASTERISK // *
	OP_SLASH    // /
	OP_PERCENT  // %
	OP_BANG     // !
	OP_EQ       // ==
	OP_NEQ      // !=
	OP_LT       // <
	OP_GT       // >
	OP_LTE      // <=
	OP_GTE      // >=
	OP_AND      // &&
	OP_OR       // ||
	OP_PIPE     // |
	OP_QUESTION // ?
	OP_NULLCOAL // ??

	// Delimiters
	COMMA     // ,
	COLON     // :
	SEMICOLON // ;
	DOT       // .
	LPAREN    // (
	RPAREN    // )
	LBRACE    // {
	RBRACE    // }
	LBRACKET  // [
	RBRACKET  // ]
)

func LookupIdent ¶

func LookupIdent(ident string) TokenType

LookupIdent checks if an identifier is a keyword and returns the appropriate token type

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL