rds

package
v0.6.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 6, 2026 License: BSD-3-Clause Imports: 9 Imported by: 0

Documentation

Overview

Package rds reads and writes R's serialization format (version 2 and 3, XDR byte order): the format of serialize(), saveRDS() and readRDS().

The format is described in "R Internals", section "Serialization Formats". The numeric codes of the pseudo-types it lists but does not number (references, the special environments, ALTREP) were observed on the reference R 4.6.2 (see internal/oracle) and are checked against its output by the golden tests. Nothing here is translated from R's sources.

A decoded document is a tree of [Value]s that keeps everything needed to write the same bytes again: attributes in order, the object bit, the general-purpose bits, the encoding flags of every string, NaN payloads, reference sharing of environments and symbols, and ALTREP objects in their serialized (not expanded) form; Altrep.Materialize expands them.

The decoder is built for untrusted input: it refuses inputs larger than Limits.MaxBytes (after decompression), nesting deeper than Limits.MaxDepth, and any length that the remaining bytes cannot hold, before allocating.

Index

Constants

View Source
const (
	// NAInteger is NA_integer_ and NA_logical_: INT_MIN.
	NAInteger int32 = -1 << 31
	// NAReal is the bit pattern of NA_real_: a NaN whose low word is 1954.
	NAReal uint64 = 0x7FF00000000007A2
)

NA values of the atomic types, as R stores them.

Variables

View Source
var (
	GlobalEnv     = &Special{"R_GlobalEnv", globalEnvSXP}
	EmptyEnv      = &Special{"R_EmptyEnv", emptyEnvSXP}
	BaseEnv       = &Special{"R_BaseEnv", baseEnvSXP}
	BaseNamespace = &Special{"R_BaseNamespace", baseNamespaceSXP}
	MissingArg    = &Special{"R_MissingArg", missingArgSXP}
	UnboundValue  = &Special{"R_UnboundValue", unboundValueSXP}
)

The singletons.

View Source
var DefaultHeader = Header{Version: 3, Writer: 0x00040602, MinReader: 0x00030500, Encoding: "UTF-8"}

DefaultHeader is what Encode writes: version 3 as written by R 4.6.2.

View Source
var ErrTooLarge = errors.New("rds: input exceeds the size limit")

ErrTooLarge is wrapped by the error returned when the input, after decompression, exceeds Limits.MaxBytes.

View Source
var ErrUnsupportedAltrep = errors.New("rds: unsupported ALTREP class")

ErrUnsupportedAltrep is wrapped by the error Altrep.Materialize returns for a class it cannot expand.

Functions

func Decode

func Decode(b []byte, lim Limits) (Value, Header, error)

Decode reads one serialized object from b, which may be gzip- or bzip2-compressed (detected by magic number, as readRDS does for files). The XDR binary format, versions 2 and 3, is supported; the ASCII and native-binary variants, xz compression, byte-code and promises are refused with an *Error.

func DecodeReader

func DecodeReader(r io.Reader, lim Limits) (Value, Header, error)

DecodeReader is Decode over a reader, reading at most Limits.MaxBytes (plus one byte to detect an overflow) from it.

func Encode

func Encode(v Value) ([]byte, error)

Encode writes v as an XDR version-3 stream with DefaultHeader, the bytes R 4.6.2's serialize(v, NULL) produces for the same object.

func EncodeWith

func EncodeWith(v Value, h Header, maxDepth int) ([]byte, error)

EncodeWith writes v with the given header. maxDepth bounds the nesting (0 selects the decoder's default). Version 2 cannot hold ALTREP objects: Altrep.Materialize them first.

Types

type Altrep

type Altrep struct {
	Meta
	Class, Pkg *Sym
	Of         Type
	State      Value
}

Altrep is an ALTREP object in its serialized form: the class (and the package defining it), the vector type it stands for, and its state.

func (*Altrep) Materialize

func (a *Altrep) Materialize(maxLen int) (Value, error)

Materialize expands an ALTREP object into the ordinary vector it stands for. The classes of R's base package that serialize() writes for ordinary data are handled: compact_intseq and compact_realseq (the result of `a:b` and seq_len), the wrap_* classes (a vector with cached metadata), and deferred_string over an integer vector. maxLen bounds the length of the vector produced (a compact sequence is a few bytes however long it is).

func (*Altrep) Type

func (a *Altrep) Type() Type

Type implements Value: the type of the vector it stands for.

type Attr

type Attr struct {
	Name  string
	Value Value
}

Attr is one attribute, in the order R stores them.

type Builtin

type Builtin struct {
	Kind Type
	Name string
}

Builtin is a SPECIALSXP or BUILTINSXP, written by name.

func (*Builtin) Type

func (b *Builtin) Type() Type

Type implements Value.

type Character

type Character struct {
	Meta
	V []Str
}

Character is a STRSXP.

func Strings

func Strings(s ...string) *Character

Strings is a convenience constructor for a character vector.

func (*Character) Type

func (*Character) Type() Type

Type implements Value.

type Closure

type Closure struct {
	Meta
	Env     Value // the enclosing environment
	Formals Value // a *Pairlist or Null
	Body    Value
}

Closure is a CLOSXP.

func (*Closure) Type

func (*Closure) Type() Type

Type implements Value.

type Complex

type Complex struct {
	Meta
	V []complex128
}

Complex is a CPLXSXP.

func (*Complex) Type

func (*Complex) Type() Type

Type implements Value.

type Env

type Env struct {
	Meta
	Locked  int32
	Enclos  Value
	Frame   Value
	HashTab Value
}

Env is an ordinary environment (ENVSXP). Frame and HashTab are the two representations R uses; exactly one is non-NULL for a non-empty environment.

func (*Env) Type

func (*Env) Type() Type

Type implements Value.

type Error

type Error struct {
	Offset int // byte offset in the (decompressed) input, -1 when not applicable
	Msg    string
}

Error is the type of every failure of the decoder and encoder.

func (*Error) Error

func (e *Error) Error() string

type ExtPtr

type ExtPtr struct {
	Meta
	Prot, Tag Value
}

ExtPtr is an external pointer: only its protected value and tag survive.

func (*ExtPtr) Type

func (*ExtPtr) Type() Type

Type implements Value.

type Header struct {
	Version   int32  // 2 or 3
	Writer    uint32 // R version that wrote it, packed as 65536*major+256*minor+patch
	MinReader uint32 // the oldest R version able to read it
	Encoding  string // native encoding at write time (version 3 only)
}

Header is the preamble of a serialized stream.

type Integer

type Integer struct {
	Meta
	V []int32
}

Integer is an INTSXP; NA is NAInteger.

func (*Integer) Type

func (*Integer) Type() Type

Type implements Value.

type Item

type Item struct {
	Tag   *Sym // nil when untagged
	Value Value
	Gp    uint16
}

Item is one cell of a pairlist, call or dots list.

type Limits

type Limits struct {
	// MaxBytes bounds the input after decompression (default 256 MiB). A
	// compressed input that expands beyond it is refused, not truncated.
	MaxBytes int
	// MaxDepth bounds the nesting of the object tree (default 2000). The
	// cells of a pairlist do not count: they are read iteratively.
	MaxDepth int
}

Limits bound what the decoder accepts. The zero value of a field selects its default.

type List

type List struct {
	Meta
	Expr bool
	V    []Value
}

List is a VECSXP, or an EXPRSXP when Expr is set.

func (*List) Type

func (l *List) Type() Type

Type implements Value.

type Logical

type Logical struct {
	Meta
	V []int32
}

Logical is a LGLSXP; TRUE is 1, FALSE 0, NA NAInteger.

func (*Logical) Type

func (*Logical) Type() Type

Type implements Value.

type Meta

type Meta struct {
	// Attrs are the attributes in storage order.
	Attrs []Attr
	// Obj is the object bit (set when the node has a class attribute, and
	// for S4 instances).
	Obj bool
	// Gp is the 16-bit general-purpose field ("levels" in the flag word).
	Gp uint16
}

Meta is what every non-trivial node carries besides its data.

func (*Meta) Attr

func (m *Meta) Attr(name string) Value

Attr returns the value of the named attribute, or nil.

type NamespaceEnv

type NamespaceEnv struct{ Info []string }

NamespaceEnv is a namespace environment, written by name and version.

func (*NamespaceEnv) Type

func (*NamespaceEnv) Type() Type

Type implements Value.

type PackageEnv

type PackageEnv struct{ Info []string }

PackageEnv is a package environment ("package:stats"), written by name.

func (*PackageEnv) Type

func (*PackageEnv) Type() Type

Type implements Value.

type Pairlist

type Pairlist struct {
	Meta
	Kind  Type
	Items []Item
	Tail  Value // nil means NULL
}

Pairlist is a LISTSXP, LANGSXP or DOTSXP: a chain of cells. Meta belongs to the first cell. Tail is the final cdr when it is not NULL (an improper list, which the format allows).

func (*Pairlist) Type

func (p *Pairlist) Type() Type

Type implements Value.

type Raw

type Raw struct {
	Meta
	V []byte
}

Raw is a RAWSXP.

func (*Raw) Type

func (*Raw) Type() Type

Type implements Value.

type Real

type Real struct {
	Meta
	V []float64
}

Real is a REALSXP. NA_real_ is the NaN with bits NAReal; payloads are preserved.

func (*Real) Type

func (*Real) Type() Type

Type implements Value.

type S4

type S4 struct{ Meta }

S4 is an S4 instance without a vector payload (OBJSXP).

func (*S4) Type

func (*S4) Type() Type

Type implements Value.

type Special

type Special struct {
	// contains filtered or unexported fields
}

Special is one of the singleton objects the format writes as a bare pseudo-type: the global, empty and base environments, the base namespace, the missing-argument marker and the unbound-value marker.

func (*Special) String

func (s *Special) String() string

func (*Special) Type

func (s *Special) Type() Type

Type implements Value; the environments report ENVSXP, the two markers SYMSXP, as typeof() does.

type Str

type Str struct {
	S  string
	NA bool // NA_character_; S is empty
	// Gp is the CHARSXP levels: 64 ASCII, 8 UTF-8, 4 Latin-1, 2 bytes.
	Gp uint16
}

Str is one element of a character vector (a CHARSXP).

func NewStr

func NewStr(s string) Str

NewStr builds a non-NA string with the flag R would set for it in a UTF-8 session: 64 for pure ASCII, 8 otherwise.

type Sym

type Sym struct {
	Name string
	// Gp is the flag-word levels of the symbol's print name (its encoding
	// bits), kept so that the bytes round-trip.
	Gp uint16
}

Sym is a symbol (SYMSXP). Within one document, equal names share one *Sym.

func (*Sym) Type

func (*Sym) Type() Type

Type implements Value.

type Type

type Type uint8

Type is an R SEXPTYPE (or, above 200, a serialization pseudo-type).

const (
	NILSXP     Type = 0
	SYMSXP     Type = 1
	LISTSXP    Type = 2
	CLOSXP     Type = 3
	ENVSXP     Type = 4
	PROMSXP    Type = 5
	LANGSXP    Type = 6
	SPECIALSXP Type = 7
	BUILTINSXP Type = 8
	CHARSXP    Type = 9
	LGLSXP     Type = 10
	INTSXP     Type = 13
	REALSXP    Type = 14
	CPLXSXP    Type = 15
	STRSXP     Type = 16
	DOTSXP     Type = 17
	VECSXP     Type = 19
	EXPRSXP    Type = 20
	BCODESXP   Type = 21
	EXTPTRSXP  Type = 22
	WEAKREFSXP Type = 23
	RAWSXP     Type = 24
	OBJSXP     Type = 25 // S4SXP before R 4.4
)

SEXPTYPE codes (R Internals, "SEXPs"; R Language Definition, typeof()).

type Value

type Value interface {
	// Type is the SEXPTYPE the node is written as.
	Type() Type
}

Value is a node of a decoded document.

var Null Value = nullValue{}

Null is R's NULL.

type WeakRef

type WeakRef struct{ Meta }

WeakRef is a weak reference (nothing but attributes is written).

func (*WeakRef) Type

func (*WeakRef) Type() Type

Type implements Value.

type WithMeta

type WithMeta interface {
	Value
	// contains filtered or unexported methods
}

WithMeta is implemented by every node that carries a Meta.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL