token

package

v0.6.0 Latest Latest Go to latest Published: Aug 26, 2026 License: MIT Imports: 2 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/rune-task-runner/rune

Links

Open Source Insights

Documentation ¶

Overview ¶

Package token defines the lexical tokens of the Runefile language and the source-position types that every token and AST node carries. Precise positions are the foundation of Principle II ("errors are a feature"): every diagnostic renders file:line:col with a caret-underlined span.

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

This section is empty.

Types ¶

type Kind ¶

type Kind int

Kind enumerates the lexical token kinds produced by the lexer.

const (
	ILLEGAL Kind = iota // an unrecognized rune (carries a diagnostic)
	EOF                 // end of input

	// Layout (significant indentation).
	NEWLINE // end of a logical line
	INDENT  // start of a more-indented block (a task body)
	DEDENT  // end of an indented block

	COMMENT // # ... (declaration context only; bodies are raw text)
	IDENT   // a name: letter { letter | digit | _ | - }
	STRING  // a string literal; Lit holds the decoded value

	// Keywords.
	SET
	IMPORT
	MOD
	IF
	ELSE

	// Punctuation & operators.
	ASSIGN     // :=
	COLON      // :
	COLONCOLON // :: (namespace separator)
	COMMA      // ,
	AT         // @  (body: suppress echo)
	DASH       // -  (body: continue on error)
	PLUS       // +  (concat / variadic-plus)
	STAR       // *  (variadic-star)
	SLASH      // /  (path join)
	AMPAMP     // &&
	PIPEPIPE   // ||
	EQ         // ==
	NEQ        // !=
	MATCH      // =~
	EQUALS     // =  (param default, cache(inputs=...))
	LPAREN     // (
	RPAREN     // )
	LBRACK     // [
	RBRACK     // ]
	LBRACE     // {
	RBRACE     // }
	QUESTION   // ?  (import? optional-import marker)

	BODYTEXT // raw text of a task body line (may contain {{ ... }})
)

func Lookup ¶

func Lookup(ident string) Kind

Lookup maps an identifier to its keyword kind, or IDENT if it is not a reserved word.

func (Kind) String ¶

func (k Kind) String() string

String returns the canonical name of the kind (used in golden token streams).

type Position ¶

type Position struct {
	Offset int // 0-based byte offset into the file
	Line   int // 1-based line number
	Col    int // 1-based column number (bytes)
}

Position is a location in a source file: a 0-based byte offset plus a 1-based line and column (column counted in bytes from the start of the line).

func (Position) String ¶

func (p Position) String() string

String renders the position as line:col (used in diagnostics).

type Span ¶

type Span struct {
	File  string
	Start Position
	End   Position
}

Span is a half-open range [Start, End) within a single file. It is attached to every AST node and every Diagnostic.

func (Span) IsValid ¶

func (s Span) IsValid() bool

IsValid reports whether the span has been populated.

func (Span) String ¶

func (s Span) String() string

String renders the span as file:line:col anchored at its start.

func (Span) To ¶

func (s Span) To(other Span) Span

To returns a span covering from s.Start to other.End (same file as s).

type Token ¶

type Token struct {
	Kind Kind
	Lit  string // decoded literal text where meaningful (IDENT, STRING, COMMENT, BODYTEXT)
	Span Span
}

Token is a single lexical token with its decoded literal and source span.

func (Token) String ¶

func (t Token) String() string

String renders the token for golden token streams, e.g. IDENT("greet") or ASSIGN. Layout tokens render bare.

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL