lexer

package
v0.1.0-alpha Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 6, 2026 License: Apache-2.0 Imports: 0 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Lexer

type Lexer struct {
	// contains filtered or unexported fields
}

Lexer stores the current state of the lexer

func New

func New(source string) *Lexer

Creates a new lexer

func (*Lexer) NextToken

func (l *Lexer) NextToken() Token

Creates a token based on the current character and then advances the lexer

func (*Lexer) Parse

func (l *Lexer) Parse() []Token

Parses the entire source code, returning the tokens it generated

type Token

type Token struct {
	Type    TokenType
	Literal string
	Line    int
}

type TokenType

type TokenType string
const (
	TokenIdent TokenType = "IDENTIFIER"

	TokenIntLiteral   TokenType = "INTEGER"
	TokenFloatLiteral TokenType = "FLOAT"
	TokenCharLiteral  TokenType = "CHAR"

	TokenLParen TokenType = "("
	TokenRParen TokenType = ")"
	TokenLBrace TokenType = "{"
	TokenRBrace TokenType = "}"

	TokenSemicolon TokenType = ";"

	TokenEqual TokenType = "="
	TokenDash  TokenType = "-"
	TokenPlus  TokenType = "+"
	TokenStar  TokenType = "*"
	TokenSlash TokenType = "/"

	TokenPlusEqual   TokenType = "+="
	TokenMinusEqual  TokenType = "-="
	TokenTimesEqual  TokenType = "*="
	TokenDivideEqual TokenType = "/="

	TokenIntKeyword   TokenType = "int"
	TokenFloatKeyword TokenType = "float"
	TokenCharKeyword  TokenType = "char"
	TokenReturn       TokenType = "return"
	TokenConst        TokenType = "const"

	TokenEndOfFile TokenType = "EOF"
	TokenIllegal   TokenType = "ILLEGAL"
)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL