Documentation
¶
Index ¶
Constants ¶
const ( PQEncoderTypeTile = "tile" PQEncoderTypeKMeans = "kmeans" PQEncoderDistributionLogNormal = "log-normal" PQEncoderDistributionNormal = "normal" )
Encoder constants for PQ encoder types.
const (
DefaultFastRotationSeed = uint64(0x535ab5105169b1df)
)
Variables ¶
This section is empty.
Functions ¶
func FastWalshHadamardTransform64 ¶
func FastWalshHadamardTransform64(x []float32)
FastWalshHadamardTransform64 performs a fast Walsh-Hadamard transform on a 64-element vector. This explicit instantiation is about 10% faster.
func FastWalshHadamardTransform256 ¶
func FastWalshHadamardTransform256(x []float32)
Types ¶
type BRQData ¶
type BRQData struct {
InputDim uint32
Rotation FastRotation
Rounding []float32
}
BRQData holds the serialization data for Binary Rotational Quantization compression.
type Encoder ¶
type Encoder byte
Encoder represents the type of encoder used for product quantization.
type FastRotation ¶
type FastRotation struct {
OutputDim uint32 // The dimension of the output returned by Rotate().
Rounds uint32 // The number of rounds of random signs, swaps, and blocked transforms that the Rotate() function is going to apply.
Swaps [][]Swap // Random swaps to apply each round prior to transforming.
Signs [][]float32 // Random signs to apply each round prior to transforming. We store these as float32 values for performance reasons, to avoid casts.
}
FastRotation holds the configuration for fast random rotation.
func NewFastRotation ¶
func NewFastRotation(inputDim int, rounds int, seed uint64) *FastRotation
func RestoreFastRotation ¶
func RestoreFastRotation(outputDim int, rounds int, swaps [][]Swap, signs [][]float32) *FastRotation
func (*FastRotation) Rotate ¶
func (r *FastRotation) Rotate(x []float32) []float32
func (*FastRotation) UnRotate ¶
func (r *FastRotation) UnRotate(rx []float32) []float32
func (*FastRotation) UnRotateInPlace ¶
func (r *FastRotation) UnRotateInPlace(x []float32) []float32
type MuveraData ¶
type MuveraData struct {
KSim uint32 // 4 bytes
NumClusters uint32 // 4 bytes
Dimensions uint32 // 4 bytes
DProjections uint32 // 4 bytes
Repetitions uint32 // 4 bytes
Gaussians [][][]float32 // (repetitions, kSim, dimensions)
S [][][]float32 // (repetitions, dProjections, dimensions)
}
MuveraData holds the serialization data for Muvera multi-vector encoding.
type PQData ¶
type PQData struct {
Ks uint16
M uint16
Dimensions uint16
EncoderType Encoder
EncoderDistribution byte
Encoders []PQSegmentEncoder
UseBitsEncoding bool
TrainingLimit int
}
PQData holds the serialization data for Product Quantization compression.
type PQSegmentEncoder ¶
type PQSegmentEncoder interface {
Encode(x []float32) byte
Centroid(b byte) []float32
Add(x []float32)
Fit(data [][]float32) error
ExposeDataForRestore() []byte
}
PQSegmentEncoder is the interface for product quantization segment encoders. Implementations include TileEncoder and KMeansEncoder.
type RQData ¶
type RQData struct {
InputDim uint32
Bits uint32
Rotation FastRotation
Rounding []float32
}
RQData holds the serialization data for Rotational Quantization compression.
type State ¶
type State struct {
// Compressed indicates if the index uses compression.
Compressed bool
// PQData holds Product Quantization compression data.
PQData *PQData
// SQData holds Scalar Quantization compression data.
SQData *SQData
// RQData holds Rotational Quantization compression data.
RQData *RQData
// BRQData holds Binary Rotational Quantization compression data.
BRQData *BRQData
// MuveraEnabled indicates if Muvera multi-vector encoding is enabled.
MuveraEnabled bool
// MuveraData holds Muvera multi-vector encoding data.
MuveraData *MuveraData
}
State holds all compression-related data for an HNSW index. This structure is optional - if nil, the index is uncompressed.
func (*State) HasCompression ¶
HasCompression returns true if any compression is enabled.