lexer

package

v1.0.0 Latest Latest Go to latest Published: Jan 15, 2026 License: MIT Imports: 6 Imported by: 0

Details

Package lexer implements a lexer for ClickHouse SQL.

This section is empty.

This section is empty.

This section is empty.

type Item struct {
	Token  token.Token
	Value  string
	Pos    token.Position
	Quoted bool // true if this identifier was double-quoted
}

Item represents a lexical token with its value and position.

func Tokenize(r io.Reader) []Item

Tokenize returns all tokens from the reader.

type Lexer struct {
	// contains filtered or unexported fields
}

Lexer tokenizes ClickHouse SQL input.

func New(r io.Reader) *Lexer

New creates a new Lexer from an io.Reader.

func (l *Lexer) NextToken() Item

NextToken returns the next token from the input.