vector

package
v0.0.0-...-edaca3c Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 22, 2026 License: MIT Imports: 16 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ComputeCentroids

func ComputeCentroids[T any](items []ai.Item[T], k int) (map[int][]float32, error)

ComputeCentroids performs K-Means clustering on the given items to find k centroids. It uses K-Means++ for initialization to improve convergence and cluster quality. The function iterates up to a maximum number of times (20) or until convergence.

func Open

func Open[T any](ctx context.Context, trans sop.Transaction, domain string, cfg Config) (ai.VectorStore[T], error)

Open returns an Index for the specified domain. It verifies that the database configuration matches the persisted state.

Types

type Architecture

type Architecture struct {
	// Centroids stores the centroid vectors.
	// Key: CentroidID (int) -> Value: Centroid struct
	// Name: "{domain}_centroids"
	Centroids btree.BtreeInterface[int, ai.Centroid]

	// Vectors stores the item vectors, indexed by centroid and distance.
	// Key: VectorKey{CentroidID, Distance, ItemID} -> Value: ItemVector ([]float32)
	// Name: "{domain}_vectors"
	// Optimization: Keeping this small allows more vectors to fit in CPU cache during scanning.
	Vectors btree.BtreeInterface[ai.VectorKey, []float32]

	// Content stores the actual item data.
	// Key: ContentKey (Metadata) -> Value: Document/JSON (string)
	// Name: "{domain}_content"
	Content btree.BtreeInterface[ai.ContentKey, string]

	// Lookup (Int -> ID): Maps integer IDs to string IDs, enabling efficient random sampling.
	// Key: SequenceID (int) -> Value: ItemID (string)
	// Name: "{domain}_lookup"
	Lookup btree.BtreeInterface[int, string]

	// TempVectors stores vectors temporarily during the ingestion phase.
	// Key: ItemID (string) -> Value: Vector ([]float32)
	// Name: "{domain}_temp_vectors"
	TempVectors btree.BtreeInterface[string, []float32]

	// Version tracks the current version of the index (Centroids/Vectors).
	Version int64
}

Architecture demonstrates the 3-B-Tree layout for optimal performance. It uses three separate B-Trees to optimize for different access patterns: 1. Directory (Centroids): Fast lookup of centroids for narrowing down the search space. 2. Library (Vectors): Fast scanning of vectors within a centroid bucket. Kept compact for cache efficiency. 3. Content (Data): Storage of the actual item data (JSON/Document), retrieved only for the final results.

func OpenDomainStore

func OpenDomainStore(ctx context.Context, trans sop.Transaction, domain string, version int64, config Config) (*Architecture, error)

OpenDomainStore initializes the B-Trees for the vertical. version is applied ONLY to Centroids and Vectors (the Index). Content, TempVectors, and Lookup are shared across versions.

func (*Architecture) Add

func (a *Architecture) Add(ctx context.Context, id string, vector []float32, data string) error

Add demonstrates how to write to all 3 stores transactionally.

func (*Architecture) Search

func (a *Architecture) Search(ctx context.Context, query []float32, k int) ([]string, error)

Search demonstrates the optimized retrieval pipeline.

type Config

type Config struct {
	// TransactionOptions specifies the transaction options for the store.
	TransactionOptions sop.TransactionOptions
	ContentSize        sop.ValueDataSize
	UsageMode          ai.UsageMode
	// EnableIngestionBuffer enables the initial "TempVectors" stage (Stage 0)
	// which buffers vectors for faster ingestion (O(1)) before they are indexed.
	// If false (default), vectors are written directly to the main index (Stage 1),
	// which is slower for ingestion but allows immediate querying and structure.
	EnableIngestionBuffer bool
	// Cache is the L2 cache client used for optimization locking and other operations.
	Cache sop.L2Cache
	// Metadata contains the knowledge base information, such as the required Embedder details.
	Metadata Metadata
}

Config holds the configuration for the Vector Store.

type EmbedderInfo

type EmbedderInfo struct {
	Provider   string `json:"provider,omitempty"`
	Model      string `json:"model,omitempty"`
	Dimensions int    `json:"dimensions,omitempty"`
}

EmbedderInfo represents the configured embedder used to build this vector store.

type Metadata

type Metadata struct {
	ActiveVersion int64        `json:"active_version"`
	Embedder      EmbedderInfo `json:"embedder,omitempty"`
	Description   string       `json:"description,omitempty"`
}

Metadata represents the JSON payload stored inside the vector store's configuration.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL