Documentation
¶
Overview ¶
Package vectorcodec is the on-disk byte codec for HNSW vector columns, wire-compatible with Java's RealVector.fromBytes / VectorType. It is a leaf package (stdlib only) so it can be shared by the record-layer maintainer and the Cascades values package without an import cycle — the latter needs it to decode a stored VECTOR column for a row-by-row distance expression.
Format: byte 0 = VectorType ordinal, bytes 1.. = big-endian IEEE-754 payload.
0 = HALF (16-bit, 2 bytes/component) 1 = SINGLE (32-bit, 4 bytes/component) 2 = DOUBLE (64-bit, 8 bytes/component) 3 = RABITQ (quantized — not decodable here; use the quantizer)
Index ¶
- Constants
- func Deserialize(data []byte) ([]float64, error)
- func HalfToFloat32(h uint16) float32
- func Payload(data []byte) (typeOrdinal byte, payload []byte, stride int, ok bool)
- func Serialize(vec []float64) []byte
- func SerializeAs(typeOrdinal byte, vec []float64) []byte
- func SerializeHalf(vec []float64) []byte
Constants ¶
const ( TypeHalf = typeHalf TypeSingle = typeSingle TypeDouble = typeDouble TypeRaBitQ = typeRaBitQ )
Type ordinals re-exported for callers that read components directly.
Variables ¶
This section is empty.
Functions ¶
func Deserialize ¶
Deserialize decodes a stored vector's bytes into float64 components. The precision is self-describing (byte 0), so no external type info is needed. RaBitQ-quantized vectors are not decodable here (they require the quantizer) and return an error.
func HalfToFloat32 ¶
HalfToFloat32 converts an IEEE-754 half-precision (16-bit) value to float32. Exported for zero-alloc readers that decode HALF payloads component by component (see Payload).
func Payload ¶
Payload exposes a stored vector's raw IEEE-754 payload for zero-allocation, component-at-a-time reads (e.g. computing a distance without materializing a []float64). It returns the type ordinal, the payload slice (sans the leading type byte), the number of bytes per component, and ok=false when the data is empty, RaBitQ-quantized (which must go through the VectorQuantizer instead), or truncated so that Java's RealVector.fromBytes rejects it (javaUnderflows; Deserialize reports that as an error).
func Serialize ¶
Serialize encodes a float64 vector into the on-disk DOUBLE byte format the HNSW vector index reads (Java RealVector.fromBytes, VectorType.DOUBLE).
func SerializeAs ¶
SerializeAs encodes vec at the precision named by a VectorType ordinal, as a Java RealVector of that type stores itself; RaBitQ and unknown ordinals fall back to DOUBLE.
func SerializeHalf ¶
SerializeHalf encodes a float64 vector into the HALF on-disk format (byte 0 = VectorType.HALF, then 2 big-endian bytes per component). Values are quantized as Java HalfRealVector (Half.valueOf): normal mantissas truncate, subnormals round, and magnitudes beyond half range become ±Inf. The SPFresh index (RFC-094) uses this for centroid/sidecar/staging vector fields — a raw fixed-width layout with no tuple escaping.
Types ¶
This section is empty.
Source Files