Documentation
¶
Overview ¶
Package rds reads and writes R's serialization format (version 2 and 3, XDR byte order): the format of serialize(), saveRDS() and readRDS().
The format is described in "R Internals", section "Serialization Formats". The numeric codes of the pseudo-types it lists but does not number (references, the special environments, ALTREP) were observed on the reference R 4.6.2 (see internal/oracle) and are checked against its output by the golden tests. Nothing here is translated from R's sources.
A decoded document is a tree of [Value]s that keeps everything needed to write the same bytes again: attributes in order, the object bit, the general-purpose bits, the encoding flags of every string, NaN payloads, reference sharing of environments and symbols, and ALTREP objects in their serialized (not expanded) form; Altrep.Materialize expands them.
The decoder is built for untrusted input: it refuses inputs larger than Limits.MaxBytes (after decompression), nesting deeper than Limits.MaxDepth, and any length that the remaining bytes cannot hold, before allocating.
Index ¶
- Constants
- Variables
- func Decode(b []byte, lim Limits) (Value, Header, error)
- func DecodeReader(r io.Reader, lim Limits) (Value, Header, error)
- func Encode(v Value) ([]byte, error)
- func EncodeWith(v Value, h Header, maxDepth int) ([]byte, error)
- type Altrep
- type Attr
- type Builtin
- type Character
- type Closure
- type Complex
- type Env
- type Error
- type ExtPtr
- type Header
- type Integer
- type Item
- type Limits
- type List
- type Logical
- type Meta
- type NamespaceEnv
- type PackageEnv
- type Pairlist
- type Raw
- type Real
- type S4
- type Special
- type Str
- type Sym
- type Type
- type Value
- type WeakRef
- type WithMeta
Constants ¶
const ( // NAInteger is NA_integer_ and NA_logical_: INT_MIN. NAInteger int32 = -1 << 31 // NAReal is the bit pattern of NA_real_: a NaN whose low word is 1954. NAReal uint64 = 0x7FF00000000007A2 )
NA values of the atomic types, as R stores them.
Variables ¶
var ( GlobalEnv = &Special{"R_GlobalEnv", globalEnvSXP} EmptyEnv = &Special{"R_EmptyEnv", emptyEnvSXP} BaseEnv = &Special{"R_BaseEnv", baseEnvSXP} BaseNamespace = &Special{"R_BaseNamespace", baseNamespaceSXP} MissingArg = &Special{"R_MissingArg", missingArgSXP} UnboundValue = &Special{"R_UnboundValue", unboundValueSXP} )
The singletons.
var DefaultHeader = Header{Version: 3, Writer: 0x00040602, MinReader: 0x00030500, Encoding: "UTF-8"}
DefaultHeader is what Encode writes: version 3 as written by R 4.6.2.
var ErrTooLarge = errors.New("rds: input exceeds the size limit")
ErrTooLarge is wrapped by the error returned when the input, after decompression, exceeds Limits.MaxBytes.
var ErrUnsupportedAltrep = errors.New("rds: unsupported ALTREP class")
ErrUnsupportedAltrep is wrapped by the error Altrep.Materialize returns for a class it cannot expand.
Functions ¶
func Decode ¶
Decode reads one serialized object from b, which may be gzip- or bzip2-compressed (detected by magic number, as readRDS does for files). The XDR binary format, versions 2 and 3, is supported; the ASCII and native-binary variants, xz compression, byte-code and promises are refused with an *Error.
func DecodeReader ¶
DecodeReader is Decode over a reader, reading at most Limits.MaxBytes (plus one byte to detect an overflow) from it.
func Encode ¶
Encode writes v as an XDR version-3 stream with DefaultHeader, the bytes R 4.6.2's serialize(v, NULL) produces for the same object.
func EncodeWith ¶
EncodeWith writes v with the given header. maxDepth bounds the nesting (0 selects the decoder's default). Version 2 cannot hold ALTREP objects: Altrep.Materialize them first.
Types ¶
type Altrep ¶
Altrep is an ALTREP object in its serialized form: the class (and the package defining it), the vector type it stands for, and its state.
func (*Altrep) Materialize ¶
Materialize expands an ALTREP object into the ordinary vector it stands for. The classes of R's base package that serialize() writes for ordinary data are handled: compact_intseq and compact_realseq (the result of `a:b` and seq_len), the wrap_* classes (a vector with cached metadata), and deferred_string over an integer vector. maxLen bounds the length of the vector produced (a compact sequence is a few bytes however long it is).
type Character ¶
Character is a STRSXP.
type Closure ¶
type Closure struct {
Meta
Env Value // the enclosing environment
Formals Value // a *Pairlist or Null
Body Value
}
Closure is a CLOSXP.
type Env ¶
Env is an ordinary environment (ENVSXP). Frame and HashTab are the two representations R uses; exactly one is non-NULL for a non-empty environment.
type Error ¶
type Error struct {
Offset int // byte offset in the (decompressed) input, -1 when not applicable
Msg string
}
Error is the type of every failure of the decoder and encoder.
type Header ¶
type Header struct {
Version int32 // 2 or 3
Writer uint32 // R version that wrote it, packed as 65536*major+256*minor+patch
MinReader uint32 // the oldest R version able to read it
Encoding string // native encoding at write time (version 3 only)
}
Header is the preamble of a serialized stream.
type Limits ¶
type Limits struct {
// MaxBytes bounds the input after decompression (default 256 MiB). A
// compressed input that expands beyond it is refused, not truncated.
MaxBytes int
// MaxDepth bounds the nesting of the object tree (default 2000). The
// cells of a pairlist do not count: they are read iteratively.
MaxDepth int
}
Limits bound what the decoder accepts. The zero value of a field selects its default.
type Meta ¶
type Meta struct {
// Attrs are the attributes in storage order.
Attrs []Attr
// Obj is the object bit (set when the node has a class attribute, and
// for S4 instances).
Obj bool
// Gp is the 16-bit general-purpose field ("levels" in the flag word).
Gp uint16
}
Meta is what every non-trivial node carries besides its data.
type NamespaceEnv ¶
type NamespaceEnv struct{ Info []string }
NamespaceEnv is a namespace environment, written by name and version.
type PackageEnv ¶
type PackageEnv struct{ Info []string }
PackageEnv is a package environment ("package:stats"), written by name.
type Pairlist ¶
Pairlist is a LISTSXP, LANGSXP or DOTSXP: a chain of cells. Meta belongs to the first cell. Tail is the final cdr when it is not NULL (an improper list, which the format allows).
type Special ¶
type Special struct {
// contains filtered or unexported fields
}
Special is one of the singleton objects the format writes as a bare pseudo-type: the global, empty and base environments, the base namespace, the missing-argument marker and the unbound-value marker.
type Str ¶
type Str struct {
S string
NA bool // NA_character_; S is empty
// Gp is the CHARSXP levels: 64 ASCII, 8 UTF-8, 4 Latin-1, 2 bytes.
Gp uint16
}
Str is one element of a character vector (a CHARSXP).
type Sym ¶
type Sym struct {
Name string
// Gp is the flag-word levels of the symbol's print name (its encoding
// bits), kept so that the bytes round-trip.
Gp uint16
}
Sym is a symbol (SYMSXP). Within one document, equal names share one *Sym.
type Type ¶
type Type uint8
Type is an R SEXPTYPE (or, above 200, a serialization pseudo-type).
const ( NILSXP Type = 0 SYMSXP Type = 1 LISTSXP Type = 2 CLOSXP Type = 3 ENVSXP Type = 4 PROMSXP Type = 5 LANGSXP Type = 6 SPECIALSXP Type = 7 BUILTINSXP Type = 8 CHARSXP Type = 9 LGLSXP Type = 10 INTSXP Type = 13 REALSXP Type = 14 CPLXSXP Type = 15 STRSXP Type = 16 DOTSXP Type = 17 VECSXP Type = 19 EXPRSXP Type = 20 BCODESXP Type = 21 EXTPTRSXP Type = 22 WEAKREFSXP Type = 23 RAWSXP Type = 24 OBJSXP Type = 25 // S4SXP before R 4.4 )
SEXPTYPE codes (R Internals, "SEXPs"; R Language Definition, typeof()).
type Value ¶
type Value interface {
// Type is the SEXPTYPE the node is written as.
Type() Type
}
Value is a node of a decoded document.
var Null Value = nullValue{}
Null is R's NULL.