Documentation
¶
Overview ¶
Package frame defines the shared, engine-agnostic non-proto columnar binary frame carried as the SendResponse.body for a vec-native query topic. It is a copy-fork of measure's mature frame core (pkg/query/vectorized/measure/frame) lifted into a reusable base so the stream and trace engines do not each fork the fail-loud magic/version/validity-bitmap/uvarint logic. Per-engine bindings supply a Codec whose Magic, WireVersion and role/type wire mappings select their own numbering; the byte layout below is identical across engines.
Wire layout:
off len field description
--- -------- --------------- ---------------------------------------------
0 4 Magic codec.Magic; first byte 0x00
(data.RawFrameMagicLeadingByte) forces a
flag-off proto.Unmarshal of the body to
return a non-nil error deterministically.
4 1 WireVersion codec.WireVersion.
5 uvarint NumRows number of (active) rows.
? uvarint NumCols number of column blocks.
? ... Columns NumCols column blocks, in schema order.
Each column block (header + body):
off len field description
--- -------- --------------- ---------------------------------------------
0 1 Role codec.RoleToWire(def.Role).
1 1 Type codec.TypeToWire(def.Type).
2 uvarint NameLen length of the UTF-8 column name.
? NameLen Name column name bytes.
? uvarint TagFamilyLen length of the UTF-8 tag family name.
Empty for non-RoleTag columns.
? TFL TagFamily tag family name bytes.
? ⌈N/8⌉ Validity bitmap N = NumRows; bit i set ⇒ row i is NULL
(1 = null). Empty for N=0.
? ... Data type-specific; see appendColumnData.
Per-type body encoding (N = NumRows; null-row slots are present but the validity bitmap is the source of truth for nullness):
- Int64: N × 8 bytes little-endian.
- Float64: N × 8 bytes IEEE-754 little-endian.
- String: For each row: uvarint(len) + len UTF-8 bytes.
- Bytes: Same shape as String, opaque bytes.
- TagValue: For each row: uvarint(len) + proto.Marshal(*TagValue) bytes.
- FieldValue: For each row: uvarint(len) + proto.Marshal(*FieldValue) bytes.
Index ¶
Constants ¶
const MagicLen = 4
MagicLen is the length of a frame magic prefix in bytes.
const MinHeaderLen = MagicLen + 1 + 1 + 1
MinHeaderLen is the smallest possible frame header — 4 magic bytes, 1 version byte, and the minimal 1-byte uvarint encodings of NumRows=0 and NumCols=0.
Variables ¶
var ( // ErrTruncated signals a frame whose length is below the minimum header // length, or whose declared lengths run past the buffer. ErrTruncated = errors.New("vectorized.frame: truncated frame") // ErrBadMagic signals a frame whose leading 4 bytes do not match the // codec's Magic. A flag-off (proto) body received on the raw path fails // here loudly, never silently mis-decoded. ErrBadMagic = errors.New("vectorized.frame: bad magic") // ErrBadVersion signals a frame whose WireVersion byte does not match the // codec's WireVersion. Hard-cutover means there is no recovery — the // receiver must surface this loudly. ErrBadVersion = errors.New("vectorized.frame: bad wire version") // ErrUnsupportedColumnType signals that a column whose // vectorized.ColumnType has no wire mapping crossed the codec. Surfacing // this at encode/decode time prevents silently-wrong wire bytes. ErrUnsupportedColumnType = errors.New("vectorized.frame: unsupported column type") // ErrUnsupportedColumnRole signals that a column whose // vectorized.ColumnRole has no wire mapping crossed the codec. ErrUnsupportedColumnRole = errors.New("vectorized.frame: unsupported column role") )
Sentinel errors. Decode and ValidateHeader wrap these with context so callers can errors.Is against specific failure classes — most importantly, ErrBadMagic at the very first byte is the engineered fail-loud guard the raw-wire hard-cutover model relies on.
Functions ¶
This section is empty.
Types ¶
type Codec ¶
type Codec struct {
// RoleToWire maps a vectorized.ColumnRole to its wire byte, or returns
// ErrUnsupportedColumnRole for a role the engine does not emit.
RoleToWire func(vectorized.ColumnRole) (uint8, error)
// WireToRole is the inverse of RoleToWire.
WireToRole func(uint8) (vectorized.ColumnRole, error)
// TypeToWire maps a vectorized.ColumnType to its wire byte, or returns
// ErrUnsupportedColumnType for a type the engine does not emit.
TypeToWire func(vectorized.ColumnType) (uint8, error)
// WireToType is the inverse of TypeToWire.
WireToType func(uint8) (vectorized.ColumnType, error)
Magic [4]byte
WireVersion uint8
}
Codec is a parameterized encoder/decoder for the shared vec columnar frame. The byte layout is fixed and engine-agnostic; only the frame signature (Magic), the format version (WireVersion), and the numeric role/type wire mappings are supplied per engine. Two codecs configured with the same Magic, WireVersion, and mapping tables produce byte-identical frames — the property the golden-bytes cross-implementation test verifies against measure's shipped frame package.
func (Codec) Decode ¶
func (c Codec) Decode(b []byte) (*vectorized.RecordBatch, error)
Decode is the inverse of Encode: it parses a raw frame body produced by the same codec and returns an in-memory RecordBatch with the same schema, columns and (active) rows. The returned batch's Selection is nil and its Len equals the encoded NumRows.
Decode is the load-bearing fail-loud guard on the consumer side: ValidateHeader rejects bad magic / wire version loudly before any column is touched, and the per-column readers reject truncation, unknown role / type bytes, and varint underflows the same way. There is no lossy fallback.
A nil/empty body is rejected here with ErrTruncated; the codec-layer carve-out for legitimately-empty distributed results lives one level up.
func (Codec) Encode ¶
func (c Codec) Encode(b *vectorized.RecordBatch) ([]byte, error)
Encode serializes a vec columnar RecordBatch into a non-proto raw frame body. The returned bytes begin with the codec's Magic (0x00-leading), so a flag-off node's proto.Unmarshal of the body deterministically fails loud.
Only the batch's active rows are encoded: if b.Selection is nil, every row in [0, b.Len) is active; otherwise the rows listed in b.Selection are active in the order they appear. Empty Selection produces a 0-row frame.
A column whose vectorized.ColumnType has no wire mapping yields ErrUnsupportedColumnType; an unmapped role yields ErrUnsupportedColumnRole — both surface here at encode time, not as silently-wrong wire bytes.
func (Codec) ValidateHeader ¶
ValidateHeader is the fail-loud preflight a decoder calls before parsing any column data. It is the SOLE technical guard for the raw-wire hard-cutover model: a frame whose magic does not match the codec's Magic, or whose wire-version is not the codec's WireVersion, is rejected loudly here — never silently mis-decoded.
It does NOT parse the columnar body; that is Decode's job. Returns the parsed Header, the number of bytes consumed from b (so the caller can slice b[bytesRead:] to reach the first column block), and a non-nil error wrapping one of ErrTruncated, ErrBadMagic, ErrBadVersion on any failure.
Source Files