lex

package
v0.9.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 13, 2026 License: MIT Imports: 3 Imported by: 0

Documentation

Overview

Package lex implements a clean-room lexer for bash COMP_LINE command lines (docs/completion-wire-protocol.md). It provides the quote-aware word extraction, redirect and pipeline classification, and COMP_WORDBREAKS prefix extraction that the native bash driver's Patch seam needs to reject redirect targets, isolate the current pipeline segment, and trim wordbreak prefixes from candidate values.

The lexer follows the GNU Bash manual (Programmable Completion) and the POSIX Shell Command Language: single quotes preserve every character, double quotes honor backslash escapes for '"', '\', '$', '`', and newline, backslash escapes any character outside quotes, and unclosed quotes at the cursor tolerate the token through end of line (the user is mid-quote).

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func LastWordbreakPrefix

func LastWordbreakPrefix(word string, wordbreaks string) string

LastWordbreakPrefix returns the prefix of word up to and including its last occurrence of any rune in wordbreaks, or "" when none occurs.

func WordsUpTo

func WordsUpTo(line string, point int) []string

WordsUpTo lexes line truncated at point and returns the quotes-resolved words.

Types

type Kind

type Kind int

Kind classifies a lexed token.

const (
	// Word is a command word: a contiguous run of non-whitespace,
	// non-operator characters with quotes resolved. Raw keeps the original
	// source text including quote and wordbreak characters.
	Word Kind = iota
	// Redirect is an I/O redirection operator (<, >, >>, <>, <<<, <&, &>,
	// >&, &>>) optionally preceded by an fd number (e.g. "2>").
	Redirect
	// Pipe is a pipeline delimiter ("|", "|&").
	Pipe
	// Separator is a command list separator (";", "&", "&&", "||").
	Separator
)

Token kinds: Word carries a quotes-resolved value; Redirect, Pipe, and Separator carry the operator text.

type Line

type Line struct {
	Tokens []Token
}

Line is a lexed command line: a sequence of tokens with the trailing empty word preserved for whitespace-terminated lines (the word being completed at the cursor may be empty).

func Parse

func Parse(line string) Line

Parse lexes the cursor-truncated command line into tokens. Unclosed quotes are tolerated (the token runs to end of line). A line that is empty or ends with whitespace separating words yields a trailing empty Word token. Whitespace inside quotes or escaped with a backslash belongs to the word (`foo\ ` is the one word "foo "). An unquoted '#' at the start of a word begins a comment that consumes the rest of the line.

func (Line) CurrentPipeline

func (l Line) CurrentPipeline() Line

CurrentPipeline returns the tokens of the last pipeline segment: the tokens after the final Pipe or Separator (completion after "|", ";", "&&", "||" concerns only the segment at the cursor).

func (Line) RedirectAtEnd

func (l Line) RedirectAtEnd() bool

RedirectAtEnd reports whether the completion position sits at a redirect: either the trailing token is a redirect operator ("cloudctl deploy >"), or the token before the trailing word is a redirect operator ("cloudctl deploy > <cursor>"), in which case no candidates must be offered.

func (Line) WithoutRedirects

func (l Line) WithoutRedirects() Line

WithoutRedirects drops every Redirect token together with the word that follows it as its target, so completing after a redirect (e.g. "corpusctl deploy > /tmp/out.log --env ") resumes at the preceding words.

func (Line) WordbreakPrefix

func (l Line) WordbreakPrefix(wordbreaks string) string

WordbreakPrefix returns the quotes-resolved part of the last word that readline leaves in place when it inserts a completion: the text up to and including the last wordbreak rune outside quotes and not escaped, or up to a quote left open at the end of the word, whichever comes later (`--env=` for `--env=pro`, `--env=` for `--env="pro`, "" for `"My Dir/in` and `My\ Dir/in`). wordbreaks is the rune set, normally the COMP_WORDBREAKS value. The prefix is empty when the line ends with an operator or no such boundary exists.

func (Line) Words

func (l Line) Words() []string

Words returns the quotes-resolved word values in order. Redirect operators and their targets are included; combine with CurrentPipeline and WithoutRedirects for engine-ready words.

type Token

type Token struct {
	// Kind is the token classification.
	Kind Kind
	// Value is the quotes-resolved text for Word tokens and the operator
	// spelling for the other kinds.
	Value string
	// Raw is the original source text of the token. For words it retains
	// quote and wordbreak characters (e.g. `--env="pro`), which
	// WordbreakPrefix needs to reproduce readline's replacement semantics.
	Raw string
	// OpenQuote is the quote character ('\” or '"') left unclosed at the end
	// of a word (the user is mid-quote), or 0 when every quote is closed.
	OpenQuote byte
}

Token is one lexed element of a command line.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL