Documentation
¶
Overview ¶
Package vectorstore provides interfaces and implementations for vector storage backends.
Overview ¶
The vectorstore package defines the VectorStore interface for storing and searching documents by their vector embeddings. This enables semantic search, RAG (Retrieval-Augmented Generation), and similarity matching workflows.
Core Concepts ¶
- Document: A storable item with content, embedding, and metadata
- VectorStore: Interface for Add/Search/Delete operations
- EmbeddingStore: Helper that auto-generates embeddings from text
- Filter: Metadata-based filtering for search queries
Available Implementations ¶
- memory.Store: In-memory store for testing and development
- qdrant.Store: Qdrant vector database (gRPC-based)
- pgvector.Store: PostgreSQL with pgvector extension
- pinecone.Store: Pinecone managed vector database
- weaviate.Store: Weaviate vector search engine
- s3vectors.Store: Amazon S3 Vectors native storage
Basic Usage ¶
// Create an in-memory store
store := memory.New()
// Add documents with embeddings
docs := []vectorstore.Document{
{ID: "1", Content: "Hello world", Embedding: []float32{0.1, 0.2, ...}},
{ID: "2", Content: "Goodbye world", Embedding: []float32{0.3, 0.4, ...}},
}
store.Add(ctx, docs)
// Search by embedding vector
results, _ := store.Search(ctx, queryEmbedding, vectorstore.SearchOptions{K: 5})
Using EmbeddingStore ¶
For convenience, EmbeddingStore auto-generates embeddings:
embedder := openai.NewEmbedder()
store := memory.New()
es := vectorstore.NewEmbeddingStore(store, embedder)
// Add texts (embeddings generated automatically)
es.AddTexts(ctx, []string{"doc1", "doc2"}, nil)
// Search by text query
results, _ := es.SearchText(ctx, "find similar docs", vectorstore.SearchOptions{K: 10})
Metadata Filtering ¶
Filter search results by metadata:
results, _ := store.Search(ctx, embedding, vectorstore.SearchOptions{
K: 10,
Filter: vectorstore.Eq("category", "science"),
})
Namespaces ¶
Partition data for multi-tenant scenarios:
store.Add(ctx, docs, func(o *vectorstore.AddOptions) {
o.Namespace = "tenant-123"
})
results, _ := store.Search(ctx, embedding, vectorstore.SearchOptions{
Namespace: "tenant-123",
})
Index ¶
- Constants
- type AddOptions
- type Document
- type EmbeddingStore
- func (es *EmbeddingStore) AddDocuments(ctx context.Context, docs []Document, opts ...func(*AddOptions)) error
- func (es *EmbeddingStore) AddTexts(ctx context.Context, texts []string, metadata []Metadata, ...) error
- func (es *EmbeddingStore) Close() error
- func (es *EmbeddingStore) Delete(ctx context.Context, ids []string, namespace string) error
- func (es *EmbeddingStore) Embedder() embedding.Embedder
- func (es *EmbeddingStore) SearchText(ctx context.Context, query string, opts SearchOptions) ([]Document, error)
- func (es *EmbeddingStore) Store() VectorStore
- type Filter
- type FusionAlgorithm
- type HybridSearchOptions
- type Indexer
- type Metadata
- type SearchOptions
- type Stats
- type StatsProvider
- type TextSearcher
- type VectorStore
Constants ¶
const DefaultK = 10
DefaultK is the default number of results if K is not specified.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type AddOptions ¶
type AddOptions struct {
// Namespace partitions the store.
Namespace string
// Upsert controls whether to update existing documents. Default: true
Upsert bool
}
AddOptions configures document addition behavior.
type Document ¶
type Document struct {
// ID is the unique identifier. If empty on Add, one will be generated.
ID string
// Content is the original text content.
Content string
// Embedding is the vector representation.
Embedding embedding.Vector
// Metadata contains arbitrary key-value pairs for filtering.
Metadata Metadata
// Score is the similarity score (populated on search results).
Score float64
// Timestamp is when this document was added/updated.
Timestamp time.Time
}
Document represents a storable/retrievable item with vector and metadata.
type EmbeddingStore ¶
type EmbeddingStore struct {
// contains filtered or unexported fields
}
EmbeddingStore wraps a VectorStore with automatic embedding generation. It simplifies common workflows by handling embedding creation internally.
func NewEmbeddingStore ¶
func NewEmbeddingStore(store VectorStore, embedder embedding.Embedder) *EmbeddingStore
NewEmbeddingStore creates a store that auto-embeds content.
func (*EmbeddingStore) AddDocuments ¶
func (es *EmbeddingStore) AddDocuments(ctx context.Context, docs []Document, opts ...func(*AddOptions)) error
AddDocuments adds pre-structured documents, generating embeddings from their content. Documents with existing embeddings are added as-is.
func (*EmbeddingStore) AddTexts ¶
func (es *EmbeddingStore) AddTexts(ctx context.Context, texts []string, metadata []Metadata, opts ...func(*AddOptions)) error
AddTexts embeds and stores text documents. If metadata slice is shorter than texts, remaining documents get nil metadata.
func (*EmbeddingStore) Close ¶
func (es *EmbeddingStore) Close() error
Close releases resources (delegates to underlying store).
func (*EmbeddingStore) Embedder ¶
func (es *EmbeddingStore) Embedder() embedding.Embedder
Embedder returns the underlying Embedder.
func (*EmbeddingStore) SearchText ¶
func (es *EmbeddingStore) SearchText(ctx context.Context, query string, opts SearchOptions) ([]Document, error)
SearchText embeds query and performs similarity search.
func (*EmbeddingStore) Store ¶
func (es *EmbeddingStore) Store() VectorStore
Store returns the underlying VectorStore.
type Filter ¶
Filter represents a metadata filter for search queries. Simple map-based approach - backends translate as needed.
type FusionAlgorithm ¶
type FusionAlgorithm string
FusionAlgorithm specifies how to combine keyword and vector search results.
const ( // FusionRRF uses Reciprocal Rank Fusion to combine results. // Score = sum(1 / (k + rank)) for each result list. FusionRRF FusionAlgorithm = "rrf" // FusionRelativeScore normalizes and combines scores from both searches. FusionRelativeScore FusionAlgorithm = "relativeScore" )
type HybridSearchOptions ¶
type HybridSearchOptions struct {
SearchOptions
// Alpha controls the balance between keyword and vector search.
// 0.0 = pure keyword (BM25/sparse), 1.0 = pure vector (dense).
// Default: 0.5 (equal weighting)
Alpha float64
// FusionAlgorithm specifies how to combine results.
// Default: RRF (Reciprocal Rank Fusion)
FusionAlgorithm FusionAlgorithm
}
HybridSearchOptions configures hybrid search behavior.
func (*HybridSearchOptions) Normalize ¶
func (o *HybridSearchOptions) Normalize()
Normalize ensures hybrid options have sensible defaults.
type Indexer ¶
type Indexer interface {
VectorStore
// CreateIndex creates a new index/collection.
CreateIndex(ctx context.Context, name string, dims int, metric embedding.Metric) error
// DeleteIndex removes an index and all its data.
DeleteIndex(ctx context.Context, name string) error
// ListIndexes returns all available indexes.
ListIndexes(ctx context.Context) ([]string, error)
}
Indexer extends VectorStore with index lifecycle management.
type SearchOptions ¶
type SearchOptions struct {
// K is the maximum number of results to return. Default: 10
K int
// MinScore filters results below this similarity threshold (0.0-1.0).
MinScore float64
// Filter applies metadata-based filtering.
Filter Filter
// IncludeEmbeddings controls whether embeddings are returned in results.
IncludeEmbeddings bool
// Namespace partitions the store (for multi-tenant scenarios).
Namespace string
}
SearchOptions configures similarity search behavior.
func (*SearchOptions) Normalize ¶
func (o *SearchOptions) Normalize()
Normalize ensures options have sensible defaults.
type StatsProvider ¶
StatsProvider optionally exposes storage statistics.
type TextSearcher ¶
type TextSearcher interface {
VectorStore
// SearchHybrid combines keyword and vector search.
// The alpha parameter in HybridSearchOptions controls the balance:
// 0.0 = pure keyword search, 1.0 = pure vector search.
SearchHybrid(ctx context.Context, query string, embedding embedding.Vector, opts HybridSearchOptions) ([]Document, error)
}
TextSearcher extends VectorStore with hybrid (keyword + vector) search.
type VectorStore ¶
type VectorStore interface {
// Add inserts or updates documents in the store.
Add(ctx context.Context, docs []Document, opts ...func(*AddOptions)) error
// Search finds documents similar to the query embedding.
Search(ctx context.Context, embedding embedding.Vector, opts SearchOptions) ([]Document, error)
// Delete removes documents by ID.
Delete(ctx context.Context, ids []string, namespace string) error
// Close releases resources.
Close() error
}
VectorStore defines the contract for vector storage backends.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package memory provides an in-memory VectorStore implementation.
|
Package memory provides an in-memory VectorStore implementation. |
|
Package pgvector provides a PostgreSQL pgvector-backed VectorStore implementation.
|
Package pgvector provides a PostgreSQL pgvector-backed VectorStore implementation. |
|
Package pinecone provides a Pinecone-backed VectorStore implementation.
|
Package pinecone provides a Pinecone-backed VectorStore implementation. |
|
Package qdrant provides a Qdrant-backed VectorStore implementation.
|
Package qdrant provides a Qdrant-backed VectorStore implementation. |
|
Package s3vectors provides an Amazon S3 Vectors-backed VectorStore implementation.
|
Package s3vectors provides an Amazon S3 Vectors-backed VectorStore implementation. |
|
Package weaviate provides a Weaviate-backed VectorStore implementation.
|
Package weaviate provides a Weaviate-backed VectorStore implementation. |