lexer

Overview ¶

Package lexer is a hand-written, state-function lexer (Pike style) for the Runefile language. It emits a flat token stream with byte/line/col spans on every token, including the layout tokens INDENT, DEDENT and NEWLINE that drive the significant-indentation grammar.

Indentation model: the only indented construct is a task body. The lexer scans structural lines at column 0; the first more-indented line opens a body (INDENT), every following line that stays more indented than the header is captured as raw body text, and the first line that returns to the header's indentation closes the body (DEDENT). Body text is NOT tokenized — it is kept verbatim (including {{ ... }} interpolation) for the evaluator.