token

package

v0.1.24 Latest Latest Go to latest Published: Aug 3, 2026 License: AGPL-3.0 Imports: 2 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/yupsh/repl

Links

Open Source Insights

Documentation ¶

Overview ¶

Package token tokenizes a REPL input line into pipeline segments.

It honours single and double quotes (a backslash inside quotes escapes the quote character), splits on unquoted pipe characters, and records which tokens carried quotes so later expansion can skip them. It holds no knowledge of the REPL or of command semantics; it is the reusable lexer behind the line domain.

Index ¶

type Line
type Segment
- func Parse(line Line) ([]Segment, error)
type Token

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

This section is empty.

Types ¶

type Line ¶

type Line string

Line is one line of input typed at the prompt.

type Segment ¶

type Segment []Token

Segment is the unexpanded token list of one pipeline stage.

func Parse ¶

func Parse(line Line) ([]Segment, error)

Parse tokenizes a command line into pipeline segments. Empty segments are preserved so the caller can report an explicit "empty command" for inputs like "a |".

type Token ¶

type Token struct {
	Text     string
	IsQuoted bool
}

Token is one tokenized word plus whether any of it was quoted. Quoting suppresses tilde and glob expansion, exactly as in a POSIX shell.

Source Files ¶

View all Source files

token.go

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL