Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ComputeCentroids ¶
ComputeCentroids performs K-Means clustering on the given items to find k centroids. It uses K-Means++ for initialization to improve convergence and cluster quality. The function iterates up to a maximum number of times (20) or until convergence.
Types ¶
type Architecture ¶
type Architecture struct {
// Centroids stores the centroid vectors.
// Key: CentroidID (int) -> Value: Centroid struct
// Name: "{domain}_centroids"
Centroids btree.BtreeInterface[int, ai.Centroid]
// Vectors stores the item vectors, indexed by centroid and distance.
// Key: VectorKey{CentroidID, Distance, ItemID} -> Value: ItemVector ([]float32)
// Name: "{domain}_vectors"
// Optimization: Keeping this small allows more vectors to fit in CPU cache during scanning.
Vectors btree.BtreeInterface[ai.VectorKey, []float32]
// Content stores the actual item data.
// Key: ContentKey (Metadata) -> Value: Document/JSON (string)
// Name: "{domain}_content"
Content btree.BtreeInterface[ai.ContentKey, string]
// Lookup (Int -> ID): Maps integer IDs to string IDs, enabling efficient random sampling.
// Key: SequenceID (int) -> Value: ItemID (string)
// Name: "{domain}_lookup"
Lookup btree.BtreeInterface[int, string]
// TempVectors stores vectors temporarily during the ingestion phase.
// Key: ItemID (string) -> Value: Vector ([]float32)
// Name: "{domain}_temp_vectors"
TempVectors btree.BtreeInterface[string, []float32]
// Version tracks the current version of the index (Centroids/Vectors).
Version int64
}
Architecture demonstrates the 3-B-Tree layout for optimal performance. It uses three separate B-Trees to optimize for different access patterns: 1. Directory (Centroids): Fast lookup of centroids for narrowing down the search space. 2. Library (Vectors): Fast scanning of vectors within a centroid bucket. Kept compact for cache efficiency. 3. Content (Data): Storage of the actual item data (JSON/Document), retrieved only for the final results.
func OpenDomainStore ¶
func OpenDomainStore(ctx context.Context, trans sop.Transaction, domain string, version int64, config Config) (*Architecture, error)
OpenDomainStore initializes the B-Trees for the vertical. version is applied ONLY to Centroids and Vectors (the Index). Content, TempVectors, and Lookup are shared across versions.
type Config ¶
type Config struct {
// TransactionOptions specifies the transaction options for the store.
TransactionOptions sop.TransactionOptions
ContentSize sop.ValueDataSize
UsageMode ai.UsageMode
// EnableIngestionBuffer enables the initial "TempVectors" stage (Stage 0)
// which buffers vectors for faster ingestion (O(1)) before they are indexed.
// If false (default), vectors are written directly to the main index (Stage 1),
// which is slower for ingestion but allows immediate querying and structure.
EnableIngestionBuffer bool
// Cache is the L2 cache client used for optimization locking and other operations.
Cache sop.L2Cache
// Metadata contains the knowledge base information, such as the required Embedder details.
Metadata Metadata
}
Config holds the configuration for the Vector Store.
type EmbedderInfo ¶
type EmbedderInfo struct {
Provider string `json:"provider,omitempty"`
Model string `json:"model,omitempty"`
Dimensions int `json:"dimensions,omitempty"`
}
EmbedderInfo represents the configured embedder used to build this vector store.
type Metadata ¶
type Metadata struct {
ActiveVersion int64 `json:"active_version"`
Embedder EmbedderInfo `json:"embedder,omitempty"`
Description string `json:"description,omitempty"`
}
Metadata represents the JSON payload stored inside the vector store's configuration.