Documentation
¶
Overview ¶
Package rabitq defines TreeDB's clean-room rabitq_1bit v1 reference codec.
The package is a correctness oracle for durable asset builders and future accelerated scorers. It deliberately does not publish collection assets or integrate search. The v1 storage contract is quantizedasset RolePackedCodes backed by typed-column packed_bit_vector rows: one LSB-first bit per code dimension, zero high padding bits in the final byte, and CodeWidthBits == 1.
For cosine column_graph indexes, callers encode finite non-zero float32 vectors after unit-L2 normalization. The reference rotation is a deterministic seeded signed permutation followed by a normalized Walsh-Hadamard transform over next-power-of-two padded dimensions. Data codes store the sign of each rotated component. Query encoding stores the rotated query sign bits and absolute per-dimension weights used by the pure-Go weighted-popcount scorer.
Index ¶
- Constants
- Variables
- func PrepareQueryByteMismatchWeights(query Query, ws *Workspace) ([]float64, float64, bool)
- func QueryByteMismatchWeightsValid(query Query, weights []float64, queryWeightSum float64) bool
- type Config
- type EncodedVector
- type Plan
- func (p *Plan) BytesPerCode() int
- func (p *Plan) CodeDimensions() int
- func (p *Plan) Config() Config
- func (p *Plan) CountCodeBits(code []byte) uint32
- func (p *Plan) Encode(dst []byte, vector []float32, ws *Workspace) (EncodedVector, error)
- func (p *Plan) EncodeQuery(query []float32, ws *Workspace) (Query, error)
- func (p *Plan) ScoreCosine(query Query, code []byte, codeCount uint32, quantizedDotProductInv float32) (float64, error)
- func (p *Plan) ScoreEncoded(query Query, encoded EncodedVector) (float64, error)
- func (p *Plan) ValidateCode(code []byte, codeCount uint32) error
- func (p *Plan) ValidateQuantizedDotProductInv(value float32) error
- func (p *Plan) ValidateQuery(q Query) error
- func (p *Plan) VectorDimensions() int
- type Query
- type Workspace
Constants ¶
const ( // CodecName is the durable quantized codec name for TreeDB RaBitQ v1. CodecName = "rabitq_1bit" // CodecVersion is the first TreeDB RaBitQ codec contract version. CodecVersion uint32 = 1 // CodeWidthBits is the number of stored bits per code dimension. CodeWidthBits = 1 // StorageRole is the quantizedasset role selected by the v1 contract. StorageRole = "packed_codes" // StorageLogicalType is the typed-column logical/physical code shape. StorageLogicalType = "packed_bit_vector" // StorageEncoding is the raw typed-column encoding for StorageLogicalType. StorageEncoding = "raw_packed_bit_vector" // BitOrder documents the TreeDB packed-code bit order used on disk. BitOrder = "lsb0" // RotationName is part of the canonical codec config identity. RotationName = "signed_permutation_fwht_padded_v1" // DefaultSeed is the deterministic v1 seed used when callers do not supply a // workload-specific seed. It is intentionally non-zero and stable. DefaultSeed uint64 = 0x7261626974710001 )
const ByteMismatchTableEntries = 256
ByteMismatchTableEntries is the number of per-byte XOR masks in a query-local mismatch-weight table.
Variables ¶
var ( // ErrInvalidConfig reports an impossible or unsupported codec shape. ErrInvalidConfig = errors.New("rabitq: invalid config") // ErrDimensionMismatch reports vector/query/code shape mismatch. ErrDimensionMismatch = errors.New("rabitq: dimension mismatch") // ErrDegenerateVector reports zero, non-finite, or otherwise unencodable input. ErrDegenerateVector = errors.New("rabitq: degenerate vector") )
Functions ¶
func PrepareQueryByteMismatchWeights ¶
PrepareQueryByteMismatchWeights builds query-local byte/XOR mask tables for the v1 weighted sign-dot scorer. The returned slice aliases ws and remains valid until the next operation that reuses the same workspace scratch.
Types ¶
type Config ¶
type Config struct {
Seed uint64
}
Config is the v1 codec configuration identity. Vector dimensions are schema shape rather than serialized config; NewPlan binds Config to dimensions.
func DefaultConfig ¶
func DefaultConfig() Config
DefaultConfig returns the stable TreeDB rabitq_1bit v1 config.
func (Config) CanonicalBytes ¶
CanonicalBytes returns the stable byte identity used for manifest config comparison and hashing. The format is line-oriented ASCII by design so docs, tests, and future non-Go implementations can reproduce it exactly.
type EncodedVector ¶
EncodedVector contains the data-code row and side-array values produced by Encode. Code is a packed_bit_vector row using TreeDB LSB-first layout.
type Plan ¶
type Plan struct {
// contains filtered or unexported fields
}
Plan binds a Config to a vector dimension and owns deterministic rotation metadata. Plan is immutable and safe for concurrent use; caller-provided Workspace values are not safe for concurrent sharing.
func NewPlan ¶
NewPlan validates dimensions, derives code shape, and precomputes the v1 deterministic signed permutation. CodeDimensions is next_power_of_two(dims), matching the padded Walsh-Hadamard rotation length.
func (*Plan) BytesPerCode ¶
BytesPerCode returns ceil(CodeDimensions/8), the packed row width.
func (*Plan) CodeDimensions ¶
CodeDimensions returns the number of one-bit dimensions stored per row.
func (*Plan) CountCodeBits ¶
CountCodeBits returns the popcount over logical code dimensions, excluding any zero padding bits in the final byte.
func (*Plan) Encode ¶
Encode normalizes vector, rotates it, packs sign bits into dst, and returns code_count plus quantized_dot_product_inv side-array values. It allocates only when dst or workspace scratch lacks capacity.
func (*Plan) EncodeQuery ¶
EncodeQuery normalizes and rotates query, then prepares the sign bits and absolute weights consumed by ScoreCosine. Returned slices alias ws.
func (*Plan) ScoreCosine ¶
func (p *Plan) ScoreCosine(query Query, code []byte, codeCount uint32, quantizedDotProductInv float32) (float64, error)
ScoreCosine validates side-array inputs and returns the v1 estimated cosine score for one encoded data vector. This is the reference oracle; accelerated implementations may skip repeated validation only after matching this result.
func (*Plan) ScoreEncoded ¶
func (p *Plan) ScoreEncoded(query Query, encoded EncodedVector) (float64, error)
ScoreEncoded is a convenience wrapper around ScoreCosine.
func (*Plan) ValidateCode ¶
ValidateCode verifies row width, zero high padding bits, and code_count.
func (*Plan) ValidateQuantizedDotProductInv ¶
ValidateQuantizedDotProductInv verifies the finite side-array range implied by unit-L2 vectors and an orthonormal rotation: L1(rotated) is in [1, sqrt(CodeDimensions)], so its inverse is in [1/sqrt(CodeDimensions), 1] modulo float32 rounding tolerance.
func (*Plan) ValidateQuery ¶
ValidateQuery verifies query shape, sign-bit padding, finite non-negative weights, and finite positive WeightSum.
func (*Plan) VectorDimensions ¶
VectorDimensions returns the source float32 vector dimensions.
type Query ¶
Query contains weighted-popcount inputs for ScoreCosine. SignBits uses the same LSB-first bit order as data codes. AbsWeights[i] is abs(rotated_query[i]).
type Workspace ¶
type Workspace struct {
// contains filtered or unexported fields
}
Workspace holds reusable scratch for Encode, EncodeQuery, and query byte-table preparation. Query values returned by EncodeQuery alias query-specific workspace buffers and remain valid until the next EncodeQuery call using the same Workspace.
Source Files