Documentation
¶
Overview ¶
Package packstream implements the PackStream binary serialisation format used by the Bolt protocol.
PackStream is a strongly typed, binary format built around a set of marker bytes that describe the type and size of the value that follows. The Encoder and Decoder types in this package provide low-level, zero-copy access to individual values; the higher-level WriteValue/ReadValue helpers operate on the Value sum type for convenience.
Concurrency: Encoder and Decoder are NOT safe for concurrent use. Each goroutine must hold its own instance. Use EncodePool/DecodePool to amortise allocation costs.
Index ¶
- Variables
- type DecodePool
- type Decoder
- func (d *Decoder) PeekType() (Type, error)
- func (d *Decoder) ReadBool() (bool, error)
- func (d *Decoder) ReadBytes() ([]byte, error)
- func (d *Decoder) ReadFloat() (float64, error)
- func (d *Decoder) ReadInt() (int64, error)
- func (d *Decoder) ReadListHeader() (int, error)
- func (d *Decoder) ReadMapHeader() (int, error)
- func (d *Decoder) ReadNull() error
- func (d *Decoder) ReadString() (string, error)
- func (d *Decoder) ReadStructHeader() (tag byte, n int, err error)
- func (d *Decoder) ReadValue() (Value, error)
- func (d *Decoder) Reset(r io.Reader)
- type EncodePool
- type Encoder
- func (e *Encoder) Flush() error
- func (e *Encoder) Reset(w io.Writer)
- func (e *Encoder) WriteBool(v bool) error
- func (e *Encoder) WriteBytes(v []byte) error
- func (e *Encoder) WriteFloat(v float64) error
- func (e *Encoder) WriteInt(v int64) error
- func (e *Encoder) WriteListHeader(n int) error
- func (e *Encoder) WriteMapHeader(n int) error
- func (e *Encoder) WriteNull() error
- func (e *Encoder) WriteString(v string) error
- func (e *Encoder) WriteStructHeader(tag byte, n int) error
- func (e *Encoder) WriteValue(v Value) error
- type Struct
- type Type
- type Value
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ErrLengthExceedsInput = errors.New("packstream: declared length exceeds remaining input")
ErrLengthExceedsInput is returned by the decoder when a length or count prefix declares more elements than can possibly remain in the current message. Inspect with errors.Is.
PackStream sizes Bytes/String payloads and List/Map/Struct collections with a length prefix that is up to a uint32 (~4.29e9). The 16 MiB message cap in github.com/FlavioCFOliveira/GoGraph/bolt/proto.ChunkedReader bounds the bytes a client may send, but it does NOT bound the allocation those bytes can *request*: a 5-byte frame such as 0xCE 0xFF 0xFF 0xFF 0xFF claims a ~4.29 GB Bytes payload, and a 5-byte List32 header claims billions of 16-byte interface slots (~64 GB). The eager make() that follows would OOM the process before the inevitable short read failed.
The decoder defends against this by carrying a per-message byte budget (see [Decoder.remaining]) and rejecting any prefix whose minimum on-wire cost exceeds the bytes still available, BEFORE allocating. Every byte/string of length n needs n payload bytes; every collection of count n needs at least n bytes (each element is at least one wire byte). This is reachable pre-authentication during the first HELLO decode, so the bound is a hard security boundary, not a convenience.
var ErrNestingTooDeep = errors.New("packstream: value nesting too deep")
ErrNestingTooDeep is returned by ReadValue when a composite value (List/Map/Structure) nests deeper than maxValueDepth. It guards against stack-overflow denial-of-service from maliciously crafted messages; see maxValueDepth for the rationale.
var ErrPayloadTooLarge = fmt.Errorf("packstream: payload length exceeds 32-bit length prefix (max %d)", uint64(math.MaxUint32))
ErrPayloadTooLarge is returned by WriteBytes/WriteString/WriteListHeader/ WriteMapHeader when the supplied length exceeds the largest size the PackStream wire format can encode in its 32-bit length field. Bolt payloads in practice are bounded well below this limit by the server's frame-size limit; the explicit check guards against silent truncation when callers pass an oversize slice from upstream batching code.
Functions ¶
This section is empty.
Types ¶
type DecodePool ¶
type DecodePool struct {
// contains filtered or unexported fields
}
DecodePool is a pool of Decoders backed by bytes.Reader readers. It is safe for concurrent use.
func (*DecodePool) Get ¶
func (dp *DecodePool) Get(src *bytes.Reader) *Decoder
Get retrieves a Decoder from the pool, resetting it to read from src. The caller must call Put when done.
func (*DecodePool) Put ¶
func (dp *DecodePool) Put(dec *Decoder)
Put returns dec to the pool for reuse.
type Decoder ¶
type Decoder struct {
// contains filtered or unexported fields
}
Decoder reads PackStream-encoded values from an underlying buffered reader. It is NOT safe for concurrent use.
func NewDecoder ¶
NewDecoder returns a new Decoder that reads from r.
When r exposes its length (it is a *bytes.Reader, *bytes.Buffer, or *strings.Reader — as the Bolt server's per-message decode path is, reading a reassembled message from a bytes.Reader), the decoder uses that length as an exact byte budget and rejects any length/count prefix that exceeds the bytes actually remaining. For any other reader the length is unknown and the decoder falls back to capping allocations at the default 16 MiB message ceiling.
func (*Decoder) PeekType ¶
PeekType returns the PackStream type of the next value without consuming it.
func (*Decoder) ReadInt ¶
ReadInt reads and returns an Integer value, regardless of width.
PackStream defines INT_8/INT_16/INT_32/INT_64 as fixed-width signed two's-complement integers. The byte/uint→int reinterpretation casts below preserve the wire bit pattern: they are the canonical decode and not unchecked overflows; gosec G115 is a false positive at each site.
func (*Decoder) ReadListHeader ¶
ReadListHeader reads the list marker and returns the number of elements. The caller is responsible for reading exactly that many values.
func (*Decoder) ReadMapHeader ¶
ReadMapHeader reads the map marker and returns the number of key/value pairs. The caller is responsible for reading exactly 2*n items.
func (*Decoder) ReadNull ¶
ReadNull consumes the NULL marker. Returns an error if the next value is not NULL.
func (*Decoder) ReadString ¶
ReadString reads and returns a String value.
func (*Decoder) ReadStructHeader ¶
ReadStructHeader reads the struct marker and returns the tag byte and the number of fields. PackStream v2 supports only TinyStruct (0..15 fields).
type EncodePool ¶
type EncodePool struct {
// contains filtered or unexported fields
}
EncodePool is a pool of Encoders backed by bytes.Buffer writers. It is safe for concurrent use.
func (*EncodePool) Get ¶
func (ep *EncodePool) Get(dst *bytes.Buffer) *Encoder
Get retrieves an Encoder from the pool, resetting it to write into dst. The caller must call Put when done.
func (*EncodePool) Put ¶
func (ep *EncodePool) Put(enc *Encoder)
Put returns enc to the pool for reuse.
type Encoder ¶
type Encoder struct {
// contains filtered or unexported fields
}
Encoder writes PackStream-encoded values to an underlying buffered writer. It is NOT safe for concurrent use.
func NewEncoder ¶
NewEncoder returns a new Encoder that writes to w. The caller retains ownership of w.
func (*Encoder) Reset ¶
Reset points the encoder at a new underlying writer. Used by pool helpers to reuse Encoder objects.
func (*Encoder) WriteBytes ¶
WriteBytes encodes a PackStream Bytes value. It selects BYTES8, BYTES16, or BYTES32 based on the slice length. Returns ErrPayloadTooLarge when len(v) exceeds math.MaxUint32.
func (*Encoder) WriteFloat ¶
WriteFloat encodes a PackStream Float64 value (8-byte big-endian IEEE-754).
func (*Encoder) WriteInt ¶
WriteInt encodes a PackStream Integer value using the most compact representation: TinyInt for values in [-16, 127], then INT8/16/32/64.
The numeric reinterpretation casts below (byte(v), int8(v), int16(v), int32(v), uint64(v)) are bit-pattern-preserving two's-complement conversions mandated by the PackStream wire format; gosec G115 is a false positive here because the enclosing switch has already bounded v to the destination type's range before each cast.
Example ¶
ExampleEncoder_WriteInt shows the wire form of a TINY_INT. PackStream encodes any integer in the range -16..127 as a single byte equal to its two's complement, so 42 serialises to one byte: 0x2a.
package main
import (
"bytes"
"fmt"
"github.com/FlavioCFOliveira/GoGraph/bolt/packstream"
)
func main() {
var buf bytes.Buffer
enc := packstream.NewEncoder(&buf)
if err := enc.WriteInt(42); err != nil {
fmt.Println("write:", err)
return
}
if err := enc.Flush(); err != nil {
fmt.Println("flush:", err)
return
}
fmt.Printf("bytes: % x\n", buf.Bytes())
}
Output: bytes: 2a
func (*Encoder) WriteListHeader ¶
WriteListHeader writes the PackStream list marker for a list with n elements. The caller is responsible for encoding exactly n elements after this call. n must be non-negative and at most math.MaxUint32; otherwise the function returns ErrPayloadTooLarge.
func (*Encoder) WriteMapHeader ¶
WriteMapHeader writes the PackStream map marker for a map with n key/value pairs. The caller is responsible for encoding exactly 2n items (alternating keys and values). Returns ErrPayloadTooLarge when n exceeds math.MaxUint32.
func (*Encoder) WriteString ¶
WriteString encodes a PackStream String value. It selects TinyString (len 0..15), STRING8, STRING16, or STRING32. Returns ErrPayloadTooLarge when len(v) exceeds math.MaxUint32.
func (*Encoder) WriteStructHeader ¶
WriteStructHeader writes the PackStream struct marker byte (TinyStruct) and the tag byte. PackStream v2 supports only TinyStruct (0..15 fields). n must be in [0, 15].
func (*Encoder) WriteValue ¶
WriteValue encodes v into the stream using the Encoder. It dispatches on the concrete type of v.
Example ¶
ExampleEncoder_WriteValue round-trips a heterogeneous list through the Value-level helpers. WriteValue dispatches on the Go type; ReadValue reconstructs the same values (integers come back as int64, strings as string), so the decoded list equals the input.
package main
import (
"bytes"
"fmt"
"github.com/FlavioCFOliveira/GoGraph/bolt/packstream"
)
func main() {
in := []packstream.Value{int64(42), "go", true}
var buf bytes.Buffer
enc := packstream.NewEncoder(&buf)
if err := enc.WriteValue(in); err != nil {
fmt.Println("write:", err)
return
}
if err := enc.Flush(); err != nil {
fmt.Println("flush:", err)
return
}
dec := packstream.NewDecoder(&buf)
out, err := dec.ReadValue()
if err != nil {
fmt.Println("read:", err)
return
}
list := out.([]packstream.Value)
fmt.Println("len:", len(list))
fmt.Printf("values: %v %q %v\n", list[0], list[1], list[2])
}
Output: len: 3 values: 42 "go" true
type Struct ¶
type Struct struct {
// Tag is the single-byte structure signature.
Tag byte
// Fields contains the structure's fields in declaration order.
Fields []Value
}
Struct represents a PackStream Structure with its tag byte and ordered fields.
type Type ¶
type Type uint8
Type identifies the PackStream type of the next value in the stream.
const ( // TypeNull represents the PackStream NULL type. TypeNull Type = iota // TypeBool represents the PackStream Boolean type. TypeBool // TypeInt represents the PackStream Integer type. TypeInt // TypeFloat represents the PackStream Float type (Float64). TypeFloat // TypeBytes represents the PackStream Bytes type. TypeBytes // TypeString represents the PackStream String type. TypeString // TypeList represents the PackStream List type. TypeList // TypeMap represents the PackStream Map type. TypeMap // TypeStruct represents the PackStream Structure type. TypeStruct )
Source Files