Documentation
¶
Overview ¶
Package parser implements R7RS Scheme syntax parsing.
The parser converts a token stream into syntax values with source location:
Features ¶
- All Scheme datums: literals, symbols, lists, vectors, bytevectors
- Full numeric tower: integers, floats, rationals, complex, big numbers
- Quote forms: ', `, ,, ,@ and syntax variants #', #`, #,, #,@
- Datum labels: #n= and #n# for shared/circular structures (R7RS 2.4)
- Case folding: #!fold-case and #!no-fold-case directives
- Symbol creation at parse time (with fold-case applied when active)
Usage ¶
p := parser.NewParserWithFile(env, true, reader, "example.scm")
for {
stx, err := p.ReadSyntax(ctx)
if errors.Is(err, io.EOF) {
break
}
// process stx
}
Error Handling ¶
Parse errors are wrapped in ParserError with source location from the offending token. State that persists across expressions is deliberate and limited to fold-case mode (R7RS §2.1) and the trailing-EOF lookahead; datum labels are scoped to a single datum and cleared on every read (R7RS §2.4).
Index ¶
- Constants
- Variables
- func MakeExactFromLiteral(text string, parsed values.Number) (values.Number, error)
- func MakeExactNumber(n values.Number) (values.Number, error)
- func MakeInexactNumber(n values.Number) values.Number
- func ParseComplexStringNumber(s string) (values.Number, error)
- func ParseImaginaryStringNumber(s string) (values.Number, error)
- func ParseRealFloatString(s string) (values.Number, error)
- func ParseRealFloatStringWithBase(text string, base int) (values.Number, error)
- func ParseSpecialFloat(s string) (*values.Float, bool)
- type Parser
- type ParserError
- func NewParserError(tok tokenizer.Token, mess string) *ParserError
- func NewParserErrorWithWrap(err error, tok tokenizer.Token, mess string) *ParserError
- func NewParserErrorWithWrapf(err error, tok tokenizer.Token, mess string, vs ...any) *ParserError
- func NewParserErrorf(tok tokenizer.Token, mess string, vs ...any) *ParserError
Constants ¶
const ( ConstQuote = "quote" ConstQuasiquote = "quasiquote" ConstUnquote = "unquote" ConstUnquoteSplicing = "unquote-splicing" ConstSyntax = "syntax" ConstQuasisyntax = "quasisyntax" ConstUnsyntax = "unsyntax" ConstUnsyntaxSplicing = "unsyntax-splicing" )
Quote form identifiers.
const DefaultMaxParseDepth int = 10000
DefaultMaxParseDepth bounds structural nesting depth during parsing. Without a bound, adversarial input such as deeply nested parentheses triggers a fatal, unrecoverable Go stack overflow that kills the host process. 0 means unlimited. Mirrors the VM's DefaultMaxCallDepth.
Variables ¶
var ( // ErrUnknownTokenType is returned when the parser encounters an unrecognized token. ErrUnknownTokenType = werr.NewStaticError("unknown token type") ErrAlreadyClosed = werr.NewStaticError("parser already closed") )
Functions ¶
func MakeExactFromLiteral ¶ added in v1.20.0
MakeExactFromLiteral implements the #e prefix (R7RS §7.1.1), which applies to the number **as written** rather than to the value it would otherwise denote.
That distinction is the whole point of this function. A decimal literal is notation for a decimal value, so #e must convert its digits:
#e1e400 => 10^400 exactly #e0.1 => 1/10
Reading the literal first and converting the result instead gives the nearest binary float's exact value, which is a different number:
(exact 1e400) => the 256-bit BigFloat neighbour of 10^400 (exact 0.1) => 3602879701896397/36028797018963968
Both answers are correct for their own question; only the first is #e. The procedure `exact` keeps the second, since by then the decimal is already gone.
text is the literal's source text with '#' digit placeholders already substituted; parsed is what reading it produced. Only Float and BigFloat (i.e. decimal or scientific notation) take the digits path; every other shape — already exact, radix-prefixed, complex — defers to MakeExactNumber, as does any text big.Rat rejects (inf, nan), which MakeExactNumber then refuses.
func MakeExactNumber ¶ added in v1.20.0
MakeExactNumber returns the exact representation of n, implementing the #e prefix (R7RS §7.1.1) and the exact procedure (R7RS §6.2.6).
It is the single source of truth for the conversion, shared by the reader (parser_number.go makeExact) and string->number (extensions/math). The two must not diverge: a type handled by one and not the other silently drops the caller's #e prefix, which is exactly how *BigFloat came to be honored by the reader and ignored by string->number.
On a magnitude with no exact representation (inf, NaN, an inexact BigComplex) it returns a wrapped ErrExactnessConversion. Callers choose the policy: the reader wraps it with a source location, string->number discards it and yields #f per R7RS §6.2.7.
func MakeInexactNumber ¶ added in v1.20.0
MakeInexactNumber returns the inexact representation of n, implementing the #i prefix (R7RS §7.1.1), the '#' digit placeholder (§7.1.1), and the inexact procedure (§6.2.6). Companion to MakeExactNumber and likewise the single source of truth, shared by the reader and string->number.
It cannot fail: every exact number has an inexact image, and the already- inexact types (Float, BigFloat, Complex) are the identity. §6.2.6 sanctions silent precision loss here, so the accuracy/exact bools returned by big.Float and big.Rat are deliberately discarded.
func ParseComplexStringNumber ¶
ParseComplexStringNumber parses a rectangular complex number string ending in 'i'. Handles "3+4i", "1.5-2.5i", "1+inf.0i", "0+3/4i", etc.
On reject it returns a non-nil error carrying a werr sentinel (ErrInvalidFormat for a shape that is not this grammar; the part parser's own sentinel for a malformed real/imaginary part). See ParseImaginaryStringNumber for how the two callers treat the error differently.
func ParseImaginaryStringNumber ¶
ParseImaginaryStringNumber parses a pure imaginary string (ending in 'i') and returns the resulting complex number. Handles "+3i", "-2.5i", "+i", "-i", "+inf.0i", "-nan.0i", etc.
On reject it returns a non-nil error carrying a werr sentinel (ErrInvalidFormat for a shape that is not this grammar; the coefficient parser's own sentinel — ErrInvalidFormat / ErrDivisionByZero — for a malformed coefficient). Callers choose how to treat it: string->number discards it and tries the next grammar (R7RS §6.2.7 returns #f), while the reader wraps it with a source location so the cause stays reachable via errors.Is / errors.Unwrap.
func ParseRealFloatString ¶
ParseRealFloatString parses a real decimal or scientific-notation string as an inexact number. In-range magnitudes yield a float64-backed Float; magnitudes beyond float64 range (strconv reports ErrRange as ±Inf) promote to a BigFloat rather than failing. This mirrors the int64 -> BigInteger promotion in parseIntegerWithBase and, crucially, keeps the reader and string->number symmetric with the writer, which renders an out-of-range bigfloat in scientific notation (e.g. (write 1e1000) -> "1l+1000"). Underflow to zero (float64 rounds a tiny magnitude to 0 without ErrRange) is left as the float64 0.0 it already produces, matching float64-based Schemes. The exponent marker is normalized to 'e' first. Returns a wrapped ErrInvalidFormat sentinel on a malformed string.
func ParseRealFloatStringWithBase ¶ added in v1.20.0
ParseRealFloatStringWithBase parses a real fraction written in an explicit radix, e.g. "1.8" in base 16 (1.5) or "101.101" in base 2 (5.625).
This is an extension, not conformance: R7RS §7.1.1 defines ⟨decimal R⟩ only for R = 10, so a radix-prefixed fraction is outside the standard grammar. Racket, Chez, and MIT all accept it, and this matches them.
Base 10 delegates to ParseRealFloatString so the decimal path — which every bare literal and every string->number call takes — stays exactly one implementation. Only the non-decimal bases are new code.
The 'p'/'P' rejection is not defensive clutter. big.ParseFloat accepts a *binary* exponent in a non-decimal base, so "1.8p3" in base 16 is 12, a grammar Scheme does not have. The reader cannot currently supply such a string (the tokenizer stops the numeral at 'p'; see requireDelimiterAfterRadixNumeral), but this function is exported next to parsers string->number already calls, so the guard belongs on the function rather than on its present caller. Note the asymmetry: 'e' is a hex *digit*, so big.ParseFloat("1e2", 16) is 482 — which is precisely what TestExponentMarkersStayDecimalOnly requires, and comes for free.
Types ¶
type Parser ¶
type Parser struct {
// contains filtered or unexported fields
}
Parser represents a R7RS compliant Scheme syntax parser.
func NewParser ¶
func NewParser(env *environment.EnvironmentFrame, skipComments bool, rdr io.RuneReader) *Parser
NewParser creates a new parser for the given reader and environment.
func NewParserWithFile ¶
func NewParserWithFile(env *environment.EnvironmentFrame, skipComments bool, rdr io.RuneReader, file string) *Parser
NewParserWithFile creates a new parser with a specified source filename.
func (*Parser) Close ¶
Close closes the parser and releases resources. Every later ReadSyntax returns an error wrapping ErrAlreadyClosed, including on a parser that was never read from: releasing the tokenizer alone did not close anything, since ReadSyntax rebuilds one from the underlying reader whenever there is none.
Close's own return is unchanged: ErrAlreadyClosed if the parser is already closed or was never read from (the tokenizer is created lazily on the first ReadSyntax), otherwise the tokenizer's Close error.
func (*Parser) ReadSyntax ¶
ReadSyntax reads and returns the next syntax value from the input. It honors ctx: a cancelled or deadline-exceeded context returns its error before reading the next form, so a long multi-form parse can be interrupted.
func (*Parser) SetFoldCase ¶
SetFoldCase enables or disables R7RS §2.1 fold-case mode at construction time, before any forms are read. This is the programmatic equivalent of a leading #!fold-case directive; it lets a caller (e.g. include-ci) read an entire file case-insensitively without the file itself carrying the directive. An in-file #!fold-case / #!no-fold-case directive still toggles the mode mid-stream.
func (*Parser) SetMaxDepth ¶
SetMaxDepth sets the maximum structural nesting depth allowed during parsing. A value of 0 (or negative, clamped to 0) disables the limit. Mirrors MachineContext.SetMaxCallDepth.
type ParserError ¶
type ParserError struct {
// contains filtered or unexported fields
}
ParserError represents an error that occurred during parsing.
The offending token (tok) carries the source position; file is the source name (empty for unnamed input such as a REPL stream). Location formats the two so the error's provenance survives to the final error chain — see REVIEW.md "Error Chain Losslessness".
func NewParserError ¶
func NewParserError(tok tokenizer.Token, mess string) *ParserError
NewParserError creates a new parser error for the given token.
func NewParserErrorWithWrap ¶
func NewParserErrorWithWrap(err error, tok tokenizer.Token, mess string) *ParserError
NewParserErrorWithWrap creates a new parser error wrapping another error.
func NewParserErrorWithWrapf ¶
NewParserErrorWithWrapf creates a new parser error wrapping another error.
func NewParserErrorf ¶
func NewParserErrorf(tok tokenizer.Token, mess string, vs ...any) *ParserError
NewParserErrorf creates a new parser error for the given token.
func (*ParserError) Error ¶
func (p *ParserError) Error() string
Error renders the message and, when the failure came from a lower layer, the cause's own text after it.
Rendering the cause is not cosmetic. locateReaderErr's fallback message is a fixed generic phrase; the entire diagnostic — which lexical rule was violated, at which index, on which character, in which tokenizer state — lives on the cause. Returning only mess left that reachable through errors.Unwrap but absent from every rendering, which is exactly the text-chain/unwrap-chain divergence REVIEW.md forbids.
A cause the message already contains is not repeated: a wrap that only adds a sentinel identity must not render as "x: x".
func (*ParserError) Is ¶
func (p *ParserError) Is(err error) bool
Is implements errors.Is for ParserError. Two parser errors match when they carry the same message (the token is positional and is not part of error identity — see EqualTo for value-level equality that includes it), so errors.Is can distinguish parser failures; a wrapped cause is still reached through Unwrap.
func (*ParserError) IsVoid ¶
func (p *ParserError) IsVoid() bool
func (*ParserError) Location ¶
func (p *ParserError) Location() string
Location returns the offending token's position as "file:line:col", or "line:col" when the input is unnamed (e.g. a REPL stream, where the parser has no file name). It returns "" when the error has no located token.
Unlike SourceContext.Location, the unnamed-input form omits the leading colon ("1:0" rather than ":1:0"); line is 1-based and column 0-based, as in SourceContext.Location.
func (*ParserError) SchemeString ¶
func (p *ParserError) SchemeString() string
SchemeString renders the error as a Scheme value. It goes through Error() so the cause's diagnostic reaches this surface too — a Scheme program that displays a read error must not see less than a Go caller does.
Only the location is added, and only because it is the one fact Error() does not carry: Error() reports index/line/column but not the file name. The offending token is deliberately NOT repeated here — Error() already names it.
func (*ParserError) Unwrap ¶
func (p *ParserError) Unwrap() error
Source Files