lex

package

v1.2.23 Latest Latest Go to latest Published: Jul 22, 2026 License: Apache-2.0 Imports: 2 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/LFDT-Lineth/zkc

Links

Open Source Insights

Documentation ¶

Overview ¶

Copyright Consensys Software Inc.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

SPDX-License-Identifier: Apache-2.0

Copyright Consensys Software Inc.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

SPDX-License-Identifier: Apache-2.0

Index ¶

type LexRule
- func Rule[T any](scanner Scanner[T], tag uint) LexRule[T]
type Lexer
- func NewLexer[T any](input []T, rules ...LexRule[T]) *Lexer[T]
type Scanner
type Token

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

This section is empty.

Types ¶

type LexRule ¶

type LexRule[T any] struct {
	// contains filtered or unexported fields
}

LexRule is simply a rule for associating groups of characters with a given tag.

nolint

func Rule ¶

func Rule[T any](scanner Scanner[T], tag uint) LexRule[T]

Rule constructs a new lexing rule which maps matching characters to a given tag.

type Lexer ¶

type Lexer[T any] struct {
	// contains filtered or unexported fields
}

Lexer provides a top-level construct for tokenising a given input string.

func NewLexer ¶

func NewLexer[T any](input []T, rules ...LexRule[T]) *Lexer[T]

NewLexer constructs a new lexer with a given set of lexing rules.

func (*Lexer[T]) Collect ¶

func (p *Lexer[T]) Collect() []Token

Collect is a convenience function which parses all remaining tokens in one go, producing an array of tokens.

func (*Lexer[T]) HasNext ¶

func (p *Lexer[T]) HasNext() bool

HasNext checks whether or not there are any items remaining to visit.

func (*Lexer[T]) Index ¶

func (p *Lexer[T]) Index() uint

Index returns the current index within the items array.

func (*Lexer[T]) Next ¶

func (p *Lexer[T]) Next() Token

Next returns the next item and advances the lexer.

func (*Lexer[T]) Remaining ¶

func (p *Lexer[T]) Remaining() uint

Remaining determines how many characters from the original sequence were left.

type Scanner ¶

type Scanner[T any] func(item []T) uint

Scanner is a function which accepts a given item or not.

func And ¶

func And[T any](scanners ...Scanner[T]) Scanner[T]

And combines zero or more scanners such that the resulting scanner succeeds if all of the scanners succeed. Observe, however, that there is an implicit left-to-right order of evaluation.

func Eof ¶

func Eof[T any]() Scanner[T]

Eof matches the end of the input stream.

func Many ¶

func Many[T any](acceptor Scanner[T]) Scanner[T]

Many matches zero or more of a given item.

func Not ¶

func Not[T comparable](item T) Scanner[T]

Not matches everything except the given item.

func Or ¶

func Or[T any](scanners ...Scanner[T]) Scanner[T]

Or combines zero or more scanners such that the resulting scanner succeeds if any of the scanners succeeds. Observe, however, that there is an implicit left-to-right order of evaluation.

func Sequence ¶

func Sequence[T comparable](scanners ...Scanner[T]) Scanner[T]

Sequence matches all the scanners in order. Each scanner consumes the input right after the previous one ends.

func SequenceNullableLast ¶

func SequenceNullableLast[T comparable](scanners ...Scanner[T]) Scanner[T]

SequenceNullableLast matches all the scanners in order. Each scanner consumes the input right after the previous one ends. Only the final scanner is allowed a match length of 0.

func String ¶

func String(s string) Scanner[int32]

String expects a given string s. It is equivalent to Unit(s[0], s[1], ...)

func Unit ¶

func Unit[T comparable](chars ...T) Scanner[T]

Unit accepts a given sequence of characters. That is, for this scanner to match, it must match all the given characters (one after the other) in their given order.

func Until ¶

func Until[T comparable](item T) Scanner[T]

Until matches everything until a particular item is matched.

func Within ¶

func Within[T cmp.Ordered](lowest T, highest T) Scanner[T]

Within accepts any character within a given range.

type Token ¶

type Token struct {
	Kind uint
	Span source.Span
}

Token associates a piece of information with a given range of characters in the string being scanned.

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL