syntax

package
v0.6.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 6, 2026 License: BSD-3-Clause Imports: 8 Imported by: 0

Documentation

Overview

Package syntax is a lexer and parser for R 4.6.2, written from the R Language Definition and checked against the reference R (see cmd/rparity): it produces R's own language objects as an AST with a source map, the terminal tokens that utils::getParseData() reports, and R's parse errors word for word.

The grammar and the context-sensitive handling of newlines follow R's documented behaviour; nothing is translated from R's sources.

Index

Constants

This section is empty.

Variables

View Source
var ErrLimit = errors.New("syntax: limit exceeded")

ErrLimit is wrapped by the error returned when an input exceeds a configured limit (bytes, nesting depth or nodes).

Functions

func Deparse added in v0.2.0

func Deparse(n Node, opt *DeparseOptions) []string

Deparse writes n as deparse() does, one string per line, byte for byte. The text is written from the documented behaviour of deparse() and checked against the oracle (cmd/rparity).

func DeparseClosure added in v0.3.0

func DeparseClosure(params []Param, body Node) []string

DeparseClosure writes a function the way print() shows one without source references: "function (x, y = 2) " and then the body on the next lines.

func Dump

func Dump(exprs []Node) []string

Dump renders the expressions in the canonical line format the oracle's "tree" operation produces from R's own language objects (see internal/oracle/oracle.R): "top N", then one node per line, children indented by one "." per level; names and strings are hex (UTF-8 bytes), doubles are their IEEE bits. Two parses are the same tree exactly when their dumps are equal.

func EncodeString added in v0.3.0

func EncodeString(s string, quote byte) string

EncodeString writes s in the given quote with R's escapes, as print() and deparse() do.

func FormatNumber added in v0.3.0

func FormatNumber(x float64) string

FormatNumber writes x as R does with 15 significant digits (the text of as.character(x) and of a deparsed double constant).

func PrimitiveNames added in v0.2.0

func PrimitiveNames() map[string]bool

PrimitiveNames returns the names bound to primitives in R's base environment (a copy; see prims_gen.go).

func Scientific added in v0.3.0

func Scientific(x float64, digits int) (neg, kpower, nsig int, roundingWidens bool)

Scientific is R's scientific() for the given number of significant digits (7 for print(), 15 for deparse()): the sign, the decimal exponent, the number of significant digits needed (at most digits), and whether rounding to digits makes the number one decimal place wider than it is.

Types

type Arg

type Arg struct {
	// Name is the argument name; HasName tells `f(a = 1)` from `f(1)` (and
	// `f("" = 1)` from both).
	Name    string
	HasName bool
	// Value is nil for an empty argument: f(,), x[1, ].
	Value Node
	Pos   Span
}

Arg is one argument of a call.

type Call

type Call struct {
	Pos  Span
	Fun  Node
	Args []Arg
	// contains filtered or unexported fields
}

Call is a function call, and every operator.

func (*Call) FunName

func (c *Call) FunName() string

FunName returns the name of the symbol a call's function is, or "".

func (*Call) Span

func (n *Call) Span() Span

type Const

type Const struct {
	Pos  Span
	Kind ConstKind
	// Logical: 0 or 1, NA when NA is set. Integer: Int, NA when NA is set.
	// Double: Double (NA_real_ has the bits of rds.NAReal; use NA to tell).
	// Complex: Re and Im. String: Str, NA when NA is set.
	Int      int32
	Double   float64
	Im       float64
	Str      string
	NA       bool
	LogicalV bool
	// contains filtered or unexported fields
}

Const is a constant of length one (or NULL).

func (*Const) Span

func (n *Const) Span() Span

type ConstKind

type ConstKind int

ConstKind is the type of a constant.

const (
	Null ConstKind = iota
	Logical
	Integer
	Double
	Complex
	String
)

The constant types: NULL, logical, integer, double, complex, character.

type DeparseOptions added in v0.2.0

type DeparseOptions struct {
	// Cutoff is the line width, in bytes, after which a line is broken at the
	// next break point (default 60).
	Cutoff int
	// NoBackquote turns off the backquoting of non-syntactic names, as
	// deparse(backtick = FALSE) does (R turns it off for a bare symbol).
	NoBackquote bool
	// NoKeepInteger drops the L suffix and NA_integer_/NA_real_ spellings.
	NoKeepInteger bool
}

DeparseOptions are the options of deparse() that matter for syntax trees: the default width cutoff is 60 bytes, names are backquoted when they are not syntactic, and integers keep their L suffix (R's default "keepInteger", "keepNA").

type Error

type Error struct {
	// Line and Col locate the error as R reports it (Col counts characters,
	// tab-expanded; 0 at the end of input).
	Line, Col int
	// Message is R's own description: "unexpected symbol", "unexpected end
	// of input", "'\q' is an unrecognized escape in character string", ...
	Message string
	// Lexical is set for errors R raises from its lexer or from the pipe
	// rewriting (their text carries "(<input>:line:col)" and no context).
	Lexical bool
	// contains filtered or unexported fields
}

Error is a parse failure. Its text is, byte for byte, the message R's parse() puts in the error condition it signals.

func (*Error) Error

func (e *Error) Error() string

Error returns the complete message, including the source context lines and the caret for a syntax error.

type File

type File struct {
	// Source is the text that was parsed: the input and the newline R adds.
	Source []byte
	// Exprs are the top-level expressions, in order.
	Exprs []Node
	// Tokens are the terminals getParseData() lists, in source order,
	// comments included.
	Tokens []Token
}

File is the result of a parse: the top-level expressions, the terminal tokens, and the source they came from.

func Parse

func Parse(src string, opt Options) (*File, error)

Parse parses src with the given options.

func ParseAll

func ParseAll(src string) (*File, error)

ParseAll parses R source text as parse(text = src) would (a newline is appended to it), with the default limits. On a syntax error it returns the expressions before the error together with the *Error.

func ParseAllLimits

func ParseAllLimits(src string, lim Limits) (*File, error)

ParseAllLimits is ParseAll with explicit limits.

type Function

type Function struct {
	Pos    Span
	Params []Param
	Body   Node
	// Lambda is set for the backslash form; both are the same call to
	// `function` in R.
	Lambda bool
	// contains filtered or unexported fields
}

Function is `function(params) body` or `\(params) body`.

func (*Function) Span

func (n *Function) Span() Span

type Kind

type Kind int

Kind is a token kind. The single-character tokens use their character code; the others are the constants below.

type LimitError

type LimitError struct {
	What  string // "bytes", "depth" or "nodes"
	Limit int
}

LimitError reports a configured limit being exceeded; it wraps ErrLimit.

func (*LimitError) Error

func (e *LimitError) Error() string

func (*LimitError) Unwrap

func (e *LimitError) Unwrap() error

Unwrap lets errors.Is(err, ErrLimit) work.

type Limits

type Limits struct {
	// MaxBytes bounds the source size (default 64 MiB).
	MaxBytes int
	// MaxDepth bounds the nesting of the syntax tree (default 5000).
	MaxDepth int
	// MaxNodes bounds the number of tree nodes (default 8 million).
	MaxNodes int
}

Limits bound what the parser accepts; a zero field selects its default. Exceeding one is a *LimitError (which wraps ErrLimit).

type Names

type Names struct {
	// Assigned are the names the cell binds in the environment it runs in
	// (the top level), in order of first assignment: x <- v, x = v, v -> x,
	// "x" <- v, f(x) <- v and x$a <- v (which assign x), assign("x", v) with
	// a literal name, and the variable of a top-level for loop. Function
	// definitions are assignments of a function value like any other.
	Assigned []string
	// Read are the names the cell uses without having bound them first:
	// free variables of top-level code, and free variables of function bodies
	// that the cell does not itself assign anywhere at top level. Called
	// functions are included (f in f(x)), as are get("x") literals. Names
	// bound inside a function (its parameters and its local assignments) or
	// inside local() are not.
	Read []string
	// Dynamic is set when the set of names cannot be resolved statically: see
	// Reasons for which construct made it so.
	Dynamic bool
	// Reasons lists the constructs that set Dynamic, each once, in order of
	// appearance: "<<-", "eval", "assign with a computed name", ...
	Reasons []string
}

Names is what a notebook cell defines and uses at top level, worked out from the syntax alone under R's scoping rules.

func NamesOf

func NamesOf(exprs []Node) *Names

NamesOf is TopLevelNames for expressions that are already parsed.

func TopLevelNames

func TopLevelNames(cell string) (*Names, error)

TopLevelNames parses a cell and returns the names it assigns and reads. The error is the parse error (*Error or *LimitError), as from ParseAll; the Names are valid only when it is nil.

type Node

type Node interface {
	// Span is the source range of the node.
	Span() Span
	// contains filtered or unexported methods
}

Node is an R language object: the result of parse() is a list of Nodes. The tree has the shape R gives it: every operator, control-flow construct, subscript and brace is a call whose function is a symbol (`if`, `+`, `[`, `{`, `(`, ...), exactly as quote(x + 1)[[1]] is `+`. Only function definitions have a node of their own, because R stores their formals and body outside the argument list.

type Options

type Options struct {
	Limits
	// Filename is the name R shows in syntax errors (default "<text>", what
	// parse(text = ) uses).
	Filename string
	// KeepSource makes lexical errors use Filename too; without it they say
	// "<input>", as parse(keep.source = FALSE) does.
	KeepSource bool
}

Options configure a parse.

type Param

type Param struct {
	Name string
	// Default is nil when there is none.
	Default Node
	Pos     Span
}

Param is one formal argument of a function definition.

type Pos

type Pos struct{ Line, Col int }

Pos is a position as R counts it: the line (from 1) and the column (from 1; 0 for the position of a line end) in characters, with a tab advancing to the next multiple of 8.

type Span

type Span struct {
	Start, End         Pos
	StartByte, EndByte int
}

Span is a source range. Start and End are the positions of the first and the last character (End is inclusive, as getParseData's col2); StartByte and EndByte are byte offsets into the source, End exclusive.

type Sym

type Sym struct {
	Pos  Span
	Name string
}

Sym is a symbol (a name, possibly backquoted in the source).

func (*Sym) Span

func (n *Sym) Span() Span

type Token

type Token struct {
	// Kind is getParseData's token name: SYMBOL, SYMBOL_FUNCTION_CALL,
	// NUM_CONST, '+', LEFT_ASSIGN, COMMENT, ...
	Kind string
	Text string
	Span
}

Token is one terminal of the parse, as utils::getParseData() lists it.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL