index

package
v2.9.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 6, 2026 License: MIT Imports: 10 Imported by: 0

Documentation

Overview

Package index provides vector indexing implementations

Package index provides advanced indexing structures

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CosineDistance

func CosineDistance(a, b []float32) float32

CosineDistance computes cosine distance (1 - cosine similarity)

func DotProductDistance

func DotProductDistance(a, b []float32) float32

DotProductDistance computes negative dot product (for similarity)

func EuclideanDistance

func EuclideanDistance(a, b []float32) float32

EuclideanDistance computes Euclidean distance

Types

type CombineStrategy

type CombineStrategy string

CombineStrategy defines how to combine results from multiple indices

const (
	// Take best results from primary index only
	StrategyPrimaryOnly CombineStrategy = "primary_only"

	// Merge results from all indices
	StrategyMergeAll CombineStrategy = "merge_all"

	// Use primary for candidates, secondary for reranking
	StrategyRerank CombineStrategy = "rerank"

	// Voting-based combination
	StrategyVoting CombineStrategy = "voting"
)

type FlatIndex

type FlatIndex struct {
	// contains filtered or unexported fields
}

FlatIndex implements a brute-force exact search index This guarantees finding the exact nearest neighbors but with O(n) complexity

func NewFlatIndex

func NewFlatIndex(dimension int, distFunc func([]float32, []float32) float32) *FlatIndex

NewFlatIndex creates a new brute-force index

func NewFlatIndexCosine

func NewFlatIndexCosine(dimension int) *FlatIndex

NewFlatIndexCosine creates a flat index optimized for cosine similarity

func (*FlatIndex) BatchInsert

func (f *FlatIndex) BatchInsert(ids []string, vectors [][]float32) error

BatchInsert adds multiple vectors efficiently

func (*FlatIndex) Clear

func (f *FlatIndex) Clear()

Clear removes all vectors from the index

func (*FlatIndex) Delete

func (f *FlatIndex) Delete(id string) bool

Delete removes a vector from the index

func (*FlatIndex) GetVector

func (f *FlatIndex) GetVector(id string) ([]float32, bool)

GetVector returns a copy of the vector for the given ID

func (*FlatIndex) Insert

func (f *FlatIndex) Insert(id string, vector []float32) error

Insert adds a vector to the index

func (*FlatIndex) RangeSearch

func (f *FlatIndex) RangeSearch(query []float32, radius float32) ([]string, []float32)

RangeSearch finds all vectors within a specified distance from the query

func (*FlatIndex) Search

func (f *FlatIndex) Search(query []float32, k int) ([]string, []float32)

Search performs exact brute-force search

func (*FlatIndex) Size

func (f *FlatIndex) Size() int

Size returns the number of vectors in the index

func (*FlatIndex) Stats

func (f *FlatIndex) Stats() map[string]interface{}

Stats returns statistics about the index

type HNSW

type HNSW struct {
	// Parameters
	M              int     // Max number of bi-directional links per node
	MaxM           int     // Max number of links for layer 0
	EfConstruction int     // Size of dynamic candidate list
	ML             float64 // Level assignment probability
	Seed           int64   // Random seed

	// Index data
	Nodes      map[string]*HNSWNode
	EntryPoint string

	// Distance function
	DistFunc func(a, b []float32) float32

	// Quantization
	Quantizer Quantizer
	// contains filtered or unexported fields
}

HNSW implements Hierarchical Navigable Small World index

func NewHNSW

func NewHNSW(M, efConstruction int, distFunc func(a, b []float32) float32) *HNSW

NewHNSW creates a new HNSW index

func (*HNSW) Delete

func (h *HNSW) Delete(id string) error

Delete marks a node as deleted (soft delete)

func (*HNSW) Insert

func (h *HNSW) Insert(id string, vector []float32) error

Insert adds a new vector to the index

func (*HNSW) InsertBatch added in v2.8.0

func (h *HNSW) InsertBatch(vectors []struct {
	ID     string
	Vector []float32
}) error

InsertBatch inserts multiple vectors into the index efficiently This is faster than calling Insert multiple times as it avoids repeated lock/unlock overhead

func (*HNSW) InsertBatchParallel added in v2.8.0

func (h *HNSW) InsertBatchParallel(vectors []struct {
	ID     string
	Vector []float32
}, numWorkers int) error

InsertBatchParallel inserts vectors in parallel using multiple goroutines

func (*HNSW) Load

func (h *HNSW) Load(r io.Reader) error

Load deserializes the index from a reader

func (*HNSW) Save

func (h *HNSW) Save(w io.Writer) error

Save serializes the index to a writer

func (*HNSW) Search

func (h *HNSW) Search(query []float32, k int, ef int) ([]string, []float32)

Search performs k-NN search

func (*HNSW) SearchVectorIndex

func (h *HNSW) SearchVectorIndex(query []float32, k int) ([]string, []float32)

SearchVectorIndex provides VectorIndex-compatible search with default ef

func (*HNSW) SetQuantizer

func (h *HNSW) SetQuantizer(q Quantizer)

SetQuantizer sets the quantizer for the index

func (*HNSW) Size

func (h *HNSW) Size() int

Size returns the number of nodes in the index

func (*HNSW) Stats

func (h *HNSW) Stats() map[string]interface{}

Stats returns index statistics

type HNSWAdapter

type HNSWAdapter struct {
	*HNSW
	// contains filtered or unexported fields
}

HNSWAdapter wraps HNSW to implement VectorIndex interface

func NewHNSWAdapter

func NewHNSWAdapter(dimension, M int, distFunc func([]float32, []float32) float32) *HNSWAdapter

NewHNSWAdapter creates a new HNSW adapter

func (*HNSWAdapter) Search

func (h *HNSWAdapter) Search(query []float32, k int) ([]string, []float32)

Search implements VectorIndex interface

func (*HNSWAdapter) SetEf

func (h *HNSWAdapter) SetEf(ef int)

SetEf sets the default ef parameter

type HNSWNode

type HNSWNode struct {
	ID        string
	Vector    []float32
	Quantized []byte // Quantized vector data (optional)
	Level     int
	Neighbors [][]string // Neighbors at each level
	Deleted   bool
}

HNSWNode represents a node in the HNSW graph

type HybridIndex

type HybridIndex struct {
	// contains filtered or unexported fields
}

HybridIndex combines HNSW for speed with IVF for accuracy

func NewHybridIndex

func NewHybridIndex(dimension int, hnswM int, ivfCentroids int) *HybridIndex

NewHybridIndex creates a new hybrid HNSW+IVF index

func (*HybridIndex) Delete

func (h *HybridIndex) Delete(id string) error

Delete removes from both indices

func (*HybridIndex) Insert

func (h *HybridIndex) Insert(id string, vector []float32) error

Insert adds a vector to both indices

func (*HybridIndex) Search

func (h *HybridIndex) Search(query []float32, k int) ([]string, []float32)

Search performs hybrid search

func (*HybridIndex) SetAlpha

func (h *HybridIndex) SetAlpha(alpha float32)

SetAlpha sets the weight between HNSW (0) and IVF (1)

func (*HybridIndex) Size

func (h *HybridIndex) Size() int

Size returns the size

func (*HybridIndex) Train

func (h *HybridIndex) Train(vectors [][]float32) error

Train trains the IVF component

type IVFAdapter

type IVFAdapter struct {
	*IVFIndex
}

IVFAdapter wraps IVFIndex to implement VectorIndex interface

func NewIVFAdapter

func NewIVFAdapter(dimension, nCentroids int) *IVFAdapter

NewIVFAdapter creates a new IVF adapter

func (*IVFAdapter) Insert

func (ivf *IVFAdapter) Insert(id string, vector []float32) error

Insert implements VectorIndex interface

func (*IVFAdapter) Search

func (ivf *IVFAdapter) Search(query []float32, k int) ([]string, []float32)

Search implements VectorIndex interface

type IVFIndex

type IVFIndex struct {
	NCentroids int         // Number of cluster centroids
	Dimension  int         // Vector dimension
	Centroids  [][]float32 // Cluster centroids
	Invlists   [][]int     // Inverted lists (vector IDs per cluster)
	Vectors    [][]float32 // Original vectors (optional, for reranking)
	IDs        []string    // Vector IDs
	Trained    bool
	NProbe     int // Number of clusters to search
	// contains filtered or unexported fields
}

IVFIndex implements Inverted File Index for partitioned vector search

func NewIVFIndex

func NewIVFIndex(dimension, nCentroids int) *IVFIndex

NewIVFIndex creates a new IVF index

func (*IVFIndex) Add

func (ivf *IVFIndex) Add(id string, vector []float32) error

Add adds a vector to the index

func (*IVFIndex) Clear

func (ivf *IVFIndex) Clear()

Clear removes all vectors from the index

func (*IVFIndex) Delete

func (ivf *IVFIndex) Delete(id string) error

Delete removes a vector from the index

func (*IVFIndex) Load

func (ivf *IVFIndex) Load(r io.Reader) error

Load deserializes the index from a reader

func (*IVFIndex) Save

func (ivf *IVFIndex) Save(w io.Writer) error

Save serializes the index to a writer

func (*IVFIndex) Search

func (ivf *IVFIndex) Search(query []float32, k int) ([]string, []float32, error)

Search performs approximate nearest neighbor search

func (*IVFIndex) SetNProbe

func (ivf *IVFIndex) SetNProbe(nprobe int)

SetNProbe sets the number of clusters to search

func (*IVFIndex) Size

func (ivf *IVFIndex) Size() int

Size returns the number of vectors in the index

func (*IVFIndex) Stats

func (ivf *IVFIndex) Stats() map[string]interface{}

Stats returns index statistics

func (*IVFIndex) Train

func (ivf *IVFIndex) Train(vectors [][]float32) error

Train learns cluster centroids from training data

type IndexType

type IndexType string

IndexType represents the type of index

const (
	IndexTypeHNSW   IndexType = "hnsw"
	IndexTypeIVF    IndexType = "ivf"
	IndexTypeFlat   IndexType = "flat"
	IndexTypeHybrid IndexType = "hybrid"
)

type LSHConfig

type LSHConfig struct {
	NumTables    int   // Number of hash tables (more = better recall, more memory)
	NumHashFuncs int   // Number of hash functions per table (more = more selective)
	Dimension    int   // Vector dimension
	Seed         int64 // Random seed for reproducibility
}

LSHConfig contains configuration for LSH index

type LSHIndex

type LSHIndex struct {
	// contains filtered or unexported fields
}

LSHIndex implements Locality Sensitive Hashing for fast approximate nearest neighbor search

func NewLSHIndex

func NewLSHIndex(config LSHConfig) *LSHIndex

NewLSHIndex creates a new LSH index

func (*LSHIndex) Clear

func (lsh *LSHIndex) Clear()

Clear removes all vectors from the index

func (*LSHIndex) Delete

func (lsh *LSHIndex) Delete(id string) bool

Delete removes a vector from the index

func (*LSHIndex) Insert

func (lsh *LSHIndex) Insert(id string, vector []float32) error

Insert adds a vector to the LSH index

func (*LSHIndex) Search

func (lsh *LSHIndex) Search(query []float32, k int) ([]LSHSearchResult, error)

Search finds approximate nearest neighbors

func (*LSHIndex) SearchWithMultiProbe

func (lsh *LSHIndex) SearchWithMultiProbe(query []float32, k int, numProbes int) ([]LSHSearchResult, error)

SearchWithMultiProbe uses multi-probe LSH for better recall

func (*LSHIndex) SetDistanceFunc

func (lsh *LSHIndex) SetDistanceFunc(distFunc func([]float32, []float32) float32)

SetDistanceFunc sets the distance function

func (*LSHIndex) Size

func (lsh *LSHIndex) Size() int

Size returns the number of vectors in the index

func (*LSHIndex) Stats

func (lsh *LSHIndex) Stats() map[string]interface{}

Stats returns statistics about the LSH index

type LSHSearchResult

type LSHSearchResult struct {
	ID       string
	Distance float32
}

LSHSearchResult represents a search result from LSH index

type MultiIndex

type MultiIndex struct {
	// contains filtered or unexported fields
}

MultiIndex combines multiple index types for optimal performance

func NewMultiIndex

func NewMultiIndex(config MultiIndexConfig) *MultiIndex

NewMultiIndex creates a new multi-index

func (*MultiIndex) AddIndex

func (m *MultiIndex) AddIndex(indexType IndexType, index VectorIndex)

AddIndex adds an index to the multi-index

func (*MultiIndex) Delete

func (m *MultiIndex) Delete(id string) error

Delete removes a vector from all indices

func (*MultiIndex) Insert

func (m *MultiIndex) Insert(id string, vector []float32) error

Insert inserts a vector into all indices

func (*MultiIndex) Search

func (m *MultiIndex) Search(query []float32, k int) ([]string, []float32)

Search performs multi-index search based on strategy

func (*MultiIndex) Size

func (m *MultiIndex) Size() int

Size returns the size of the primary index

func (*MultiIndex) Stats

func (m *MultiIndex) Stats() map[string]interface{}

Stats returns statistics for all indices

type MultiIndexConfig

type MultiIndexConfig struct {
	// Primary index for fast approximate search
	PrimaryIndex IndexType

	// Secondary indices for refinement
	SecondaryIndices []IndexType

	// Strategy for combining results
	CombineStrategy CombineStrategy

	// Reranking configuration
	RerankTopK int

	// Parallel search
	Parallel bool
}

MultiIndexConfig configures the multi-index strategy

type Quantizer

type Quantizer interface {
	Encode(vec []float32) ([]byte, error)
	Decode(encoded []byte) ([]float32, error)
}

Quantizer defines interface for vector quantization

type VectorIndex

type VectorIndex interface {
	Insert(id string, vector []float32) error
	Search(query []float32, k int) ([]string, []float32)
	Delete(id string) error
	Size() int
}

VectorIndex is the common interface for all index types

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL