vectorstore

package
v1.2.35 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 31, 2026 License: Apache-2.0 Imports: 25 Imported by: 2

Documentation

Overview

Package vectorstore provides a generic interface for vector stores.

Index

Constants

View Source 
const (
	// BatchLimit is the default limit used for pagination and batch operations
	BatchLimit = 100
	// RedisMaxSearchResults is the maximum number of results Redis Search returns in a single query.
	// This is the default MAXSEARCHRESULTS configuration in Redis Search.
	RedisMaxSearchResults = 10000
)
View Source 
const (
	// Default class names (Weaviate prefers PascalCase)
	DefaultClassName = "BifrostStore"
)

Default values for Weaviate vector index configuration

Variables

View Source 
var (
	ErrNotFound     = errors.New("vectorstore: not found")
	ErrNotSupported = errors.New("vectorstore: operation not supported on this store")
)

Functions

func IsScanFallbackDisabled added in v1.2.33 

func IsScanFallbackDisabled(ctx context.Context) bool

IsScanFallbackDisabled reports whether scan fallback has been disabled for the current vector store operation.

func WithDisableScanFallback added in v1.2.33 

func WithDisableScanFallback(ctx context.Context) context.Context

WithDisableScanFallback returns a derived context that tells vector stores not to fall back to full scans when indexed search fails.

Types

type Config 

type Config struct {
	Enabled bool            `json:"enabled"`
	Type    VectorStoreType `json:"type"`
	Config  any             `json:"config"`
}

Config represents the configuration for the vector store.

func (*Config) UnmarshalJSON 

func (c *Config) UnmarshalJSON(data []byte) error

UnmarshalJSON unmarshals the config from JSON.

type DeleteResult added in v1.0.3 

type DeleteResult struct {
	ID     string
	Status DeleteStatus
	Error  string
}

DeleteResult represents the result of a delete operation.

type DeleteStatus added in v1.0.3 

type DeleteStatus string
const (
	DeleteStatusSuccess DeleteStatus = "success"
	DeleteStatusError   DeleteStatus = "error"
)

type PineconeConfig added in v1.2.15 

type PineconeConfig struct {
	APIKey    schemas.EnvVar `json:"api_key"`    // Pinecone API key - REQUIRED
	IndexHost schemas.EnvVar `json:"index_host"` // Index host URL from Pinecone console - REQUIRED
}

PineconeConfig represents the configuration for the Pinecone vector store.

type PineconeStore added in v1.2.15 

type PineconeStore struct {
	// contains filtered or unexported fields
}

PineconeStore represents the Pinecone vector store.

func (*PineconeStore) Add added in v1.2.15 

func (s *PineconeStore) Add(ctx context.Context, namespace string, id string, embedding []float32, metadata map[string]interface{}) error

Add stores a new vector in the Pinecone vector store.

func (*PineconeStore) Close added in v1.2.15 

func (s *PineconeStore) Close(ctx context.Context, namespace string) error

Close closes the Pinecone client connection. If namespace is non-empty, only that namespace connection is closed. If namespace is empty, all connections (indexConn and all namespaces) are closed.

func (*PineconeStore) CreateNamespace added in v1.2.15 

func (s *PineconeStore) CreateNamespace(ctx context.Context, namespace string, dimension int, properties map[string]VectorStoreProperties) error

CreateNamespace creates a new namespace in the Pinecone vector store. Note: Pinecone namespaces are created implicitly when upserting vectors. This method is a no-op but ensures the connection is valid.

func (*PineconeStore) Delete added in v1.2.15 

func (s *PineconeStore) Delete(ctx context.Context, namespace string, id string) error

Delete removes a vector from the Pinecone vector store.

func (*PineconeStore) DeleteAll added in v1.2.15 

func (s *PineconeStore) DeleteAll(ctx context.Context, namespace string, queries []Query) ([]DeleteResult, error)

DeleteAll removes multiple vectors matching the filter.

func (*PineconeStore) DeleteNamespace added in v1.2.15 

func (s *PineconeStore) DeleteNamespace(ctx context.Context, namespace string) error

DeleteNamespace deletes a namespace from the Pinecone vector store.

func (*PineconeStore) GetAll added in v1.2.15 

func (s *PineconeStore) GetAll(ctx context.Context, namespace string, queries []Query, selectFields []string, cursor *string, limit int64) ([]SearchResult, *string, error)

GetAll retrieves all vectors with optional filtering and pagination. Note: This implementation uses QueryByVectorValues with a zero vector instead of ListVectors because ListVectors has severe eventual consistency issues on Pinecone Serverless/Starter indexes. The metadata filtering is done server-side by Pinecone, providing much better consistency.

func (*PineconeStore) GetChunk added in v1.2.15 

func (s *PineconeStore) GetChunk(ctx context.Context, namespace string, id string) (SearchResult, error)

GetChunk retrieves a single vector from the Pinecone vector store.

func (*PineconeStore) GetChunks added in v1.2.15 

func (s *PineconeStore) GetChunks(ctx context.Context, namespace string, ids []string) ([]SearchResult, error)

GetChunks retrieves multiple vectors from the Pinecone vector store.

func (*PineconeStore) GetNearest added in v1.2.15 

func (s *PineconeStore) GetNearest(ctx context.Context, namespace string, vector []float32, queries []Query, selectFields []string, threshold float64, limit int64) ([]SearchResult, error)

GetNearest retrieves the nearest vectors to a given vector.

func (*PineconeStore) Ping added in v1.2.15 

func (s *PineconeStore) Ping(ctx context.Context) error

Ping checks if the Pinecone server is reachable.

func (*PineconeStore) RequiresVectors added in v1.2.15 

func (s *PineconeStore) RequiresVectors() bool

RequiresVectors returns true because Pinecone is a dedicated vector database that requires vectors for all entries with a specific dimension.

type QdrantConfig added in v1.1.38 

type QdrantConfig struct {
	Host   schemas.EnvVar `json:"host"`              // Qdrant server host - REQUIRED
	Port   schemas.EnvVar `json:"port"`              // Qdrant server port  (fallback to 6334 for gRPC)
	APIKey schemas.EnvVar `json:"api_key,omitempty"` // API key for authentication - Optional
	UseTLS schemas.EnvVar `json:"use_tls,omitempty"` // Use TLS for connection - Optional
}

QdrantConfig represents the configuration for the Qdrant vector store.

type QdrantStore added in v1.1.38 

type QdrantStore struct {
	// contains filtered or unexported fields
}

QdrantStore represents the Qdrant vector store.

func (*QdrantStore) Add added in v1.1.38 

func (s *QdrantStore) Add(ctx context.Context, namespace string, id string, embedding []float32, metadata map[string]interface{}) error

Add stores a new point in the Qdrant vector store.

func (*QdrantStore) Close added in v1.1.38 

func (s *QdrantStore) Close(ctx context.Context, namespace string) error

Close closes the Qdrant client connection.

func (*QdrantStore) CreateNamespace added in v1.1.38 

func (s *QdrantStore) CreateNamespace(ctx context.Context, namespace string, dimension int, properties map[string]VectorStoreProperties) error

CreateNamespace creates a new collection in the Qdrant vector store.

func (*QdrantStore) Delete added in v1.1.38 

func (s *QdrantStore) Delete(ctx context.Context, namespace string, id string) error

Delete removes a point from the Qdrant vector store.

func (*QdrantStore) DeleteAll added in v1.1.38 

func (s *QdrantStore) DeleteAll(ctx context.Context, namespace string, queries []Query) ([]DeleteResult, error)

DeleteAll removes multiple points matching the filter.

func (*QdrantStore) DeleteNamespace added in v1.1.38 

func (s *QdrantStore) DeleteNamespace(ctx context.Context, namespace string) error

DeleteNamespace deletes a collection from the Qdrant vector store.

func (*QdrantStore) GetAll added in v1.1.38 

func (s *QdrantStore) GetAll(ctx context.Context, namespace string, queries []Query, selectFields []string, cursor *string, limit int64) ([]SearchResult, *string, error)

GetAll retrieves all points with optional filtering and pagination.

func (*QdrantStore) GetChunk added in v1.1.38 

func (s *QdrantStore) GetChunk(ctx context.Context, namespace string, id string) (SearchResult, error)

GetChunk retrieves a single point from the Qdrant vector store.

func (*QdrantStore) GetChunks added in v1.1.38 

func (s *QdrantStore) GetChunks(ctx context.Context, namespace string, ids []string) ([]SearchResult, error)

GetChunks retrieves multiple points from the Qdrant vector store.

func (*QdrantStore) GetNearest added in v1.1.38 

func (s *QdrantStore) GetNearest(ctx context.Context, namespace string, vector []float32, queries []Query, selectFields []string, threshold float64, limit int64) ([]SearchResult, error)

GetNearest retrieves the nearest points to a vector.

func (*QdrantStore) Ping added in v1.1.38 

func (s *QdrantStore) Ping(ctx context.Context) error

Ping checks if the Qdrant server is reachable.

func (*QdrantStore) RequiresVectors added in v1.2.15 

func (s *QdrantStore) RequiresVectors() bool

RequiresVectors returns true because Qdrant is a dedicated vector database that requires vectors for all points/entries.

type Query added in v1.0.3 

type Query struct {
	Field    string
	Operator QueryOperator
	Value    interface{}
}

Query represents a query to the vector store.

type QueryOperator added in v1.0.3 

type QueryOperator string
const (
	QueryOperatorEqual              QueryOperator = "Equal"
	QueryOperatorNotEqual           QueryOperator = "NotEqual"
	QueryOperatorGreaterThan        QueryOperator = "GreaterThan"
	QueryOperatorLessThan           QueryOperator = "LessThan"
	QueryOperatorGreaterThanOrEqual QueryOperator = "GreaterThanOrEqual"
	QueryOperatorLessThanOrEqual    QueryOperator = "LessThanOrEqual"
	QueryOperatorLike               QueryOperator = "Like"
	QueryOperatorContainsAny        QueryOperator = "ContainsAny"
	QueryOperatorContainsAll        QueryOperator = "ContainsAll"
	QueryOperatorIsNull             QueryOperator = "IsNull"
	QueryOperatorIsNotNull          QueryOperator = "IsNotNull"
)

type RedisConfig 

type RedisConfig struct {
	// Connection settings
	Addr     *schemas.EnvVar `json:"addr"`               // Redis server address (host:port) - REQUIRED
	Username *schemas.EnvVar `json:"username,omitempty"` // Username for Redis AUTH (optional)
	Password *schemas.EnvVar `json:"password,omitempty"` // Password for Redis AUTH (optional)
	DB       *schemas.EnvVar `json:"db,omitempty"`       // Redis database number (default: 0)

	// Connection pool and timeout settings (passed directly to Redis client)
	PoolSize        int           `json:"pool_size,omitempty"`          // Maximum number of socket connections (optional)
	MaxActiveConns  int           `json:"max_active_conns,omitempty"`   // Maximum number of active connections (optional)
	MinIdleConns    int           `json:"min_idle_conns,omitempty"`     // Minimum number of idle connections (optional)
	MaxIdleConns    int           `json:"max_idle_conns,omitempty"`     // Maximum number of idle connections (optional)
	ConnMaxLifetime time.Duration `json:"conn_max_lifetime,omitempty"`  // Connection maximum lifetime (optional)
	ConnMaxIdleTime time.Duration `json:"conn_max_idle_time,omitempty"` // Connection maximum idle time (optional)
	DialTimeout     time.Duration `json:"dial_timeout,omitempty"`       // Timeout for socket connection (optional)
	ReadTimeout     time.Duration `json:"read_timeout,omitempty"`       // Timeout for socket reads (optional)
	WriteTimeout    time.Duration `json:"write_timeout,omitempty"`      // Timeout for socket writes (optional)
	ContextTimeout  time.Duration `json:"context_timeout,omitempty"`    // Timeout for Redis operations (optional)
}

type RedisStore 

type RedisStore struct {
	// contains filtered or unexported fields
}

RedisStore represents the Redis vector store.

func (*RedisStore) Add 

func (s *RedisStore) Add(ctx context.Context, namespace string, id string, embedding []float32, metadata map[string]interface{}) error

Add stores a new chunk in the Redis vector store.

func (*RedisStore) Close 

func (s *RedisStore) Close(ctx context.Context, namespace string) error

Close closes the Redis vector store.

func (*RedisStore) CreateNamespace added in v1.0.16 

func (s *RedisStore) CreateNamespace(ctx context.Context, namespace string, dimension int, properties map[string]VectorStoreProperties) error

CreateNamespace creates a new namespace in the Redis vector store.

func (*RedisStore) Delete 

func (s *RedisStore) Delete(ctx context.Context, namespace string, id string) error

Delete deletes a chunk from the Redis vector store.

func (*RedisStore) DeleteAll added in v1.0.16 

func (s *RedisStore) DeleteAll(ctx context.Context, namespace string, queries []Query) ([]DeleteResult, error)

DeleteAll deletes all chunks from the Redis vector store.

func (*RedisStore) DeleteNamespace added in v1.0.16 

func (s *RedisStore) DeleteNamespace(ctx context.Context, namespace string) error

DeleteNamespace deletes a namespace from the Redis vector store.

func (*RedisStore) GetAll 

func (s *RedisStore) GetAll(ctx context.Context, namespace string, queries []Query, selectFields []string, cursor *string, limit int64) ([]SearchResult, *string, error)

GetAll retrieves all chunks from the Redis vector store.

func (*RedisStore) GetChunk 

func (s *RedisStore) GetChunk(ctx context.Context, namespace string, id string) (SearchResult, error)

GetChunk retrieves a chunk from the Redis vector store.

func (*RedisStore) GetChunks 

func (s *RedisStore) GetChunks(ctx context.Context, namespace string, ids []string) ([]SearchResult, error)

GetChunks retrieves multiple chunks from the Redis vector store.

func (*RedisStore) GetNearest added in v1.0.16 

func (s *RedisStore) GetNearest(ctx context.Context, namespace string, vector []float32, queries []Query, selectFields []string, threshold float64, limit int64) ([]SearchResult, error)

GetNearest retrieves the nearest chunks from the Redis vector store.

func (*RedisStore) Ping added in v1.1.8 

func (s *RedisStore) Ping(ctx context.Context) error

Ping checks if the Redis server is reachable.

func (*RedisStore) RequiresVectors added in v1.2.15 

func (s *RedisStore) RequiresVectors() bool

RequiresVectors returns false because Redis can store hash data with or without vectors.

type SearchResult added in v1.0.3 

type SearchResult struct {
	ID         string
	Score      *float64
	Properties map[string]interface{}
}

SearchResult represents a search result with metadata.

type VectorStore 

type VectorStore interface {
	// Health check
	Ping(ctx context.Context) error
	// CreateNamespace creates a new namespace in the vector store.
	CreateNamespace(ctx context.Context, namespace string, dimension int, properties map[string]VectorStoreProperties) error
	// DeleteNamespace deletes a namespace from the vector store.
	DeleteNamespace(ctx context.Context, namespace string) error
	// GetChunk retrieves a single vector from the vector store.
	GetChunk(ctx context.Context, namespace string, id string) (SearchResult, error)
	// GetChunks retrieves multiple vectors from the vector store.
	GetChunks(ctx context.Context, namespace string, ids []string) ([]SearchResult, error)
	// GetAll retrieves all vectors from the vector store.
	GetAll(ctx context.Context, namespace string, queries []Query, selectFields []string, cursor *string, limit int64) ([]SearchResult, *string, error)
	// GetNearest retrieves the nearest vectors from the vector store.
	GetNearest(ctx context.Context, namespace string, vector []float32, queries []Query, selectFields []string, threshold float64, limit int64) ([]SearchResult, error)
	// RequiresVectors returns true if the vector store requires vectors for all entries.
	// Dedicated vector databases like Qdrant and Pinecone require vectors, while
	// more flexible stores like Weaviate and Redis can store metadata-only entries.
	RequiresVectors() bool
	// Add stores a new vector in the vector store.
	Add(ctx context.Context, namespace string, id string, embedding []float32, metadata map[string]interface{}) error
	// Delete removes a vector from the vector store.
	Delete(ctx context.Context, namespace string, id string) error
	// DeleteAll deletes all vectors from the vector store.
	DeleteAll(ctx context.Context, namespace string, queries []Query) ([]DeleteResult, error)
	// Close closes the vector store.
	Close(ctx context.Context, namespace string) error
}

VectorStore represents the interface for the vector store.

func NewVectorStore 

func NewVectorStore(ctx context.Context, config *Config, logger schemas.Logger) (VectorStore, error)

NewVectorStore returns a new vector store based on the configuration.

type VectorStoreProperties added in v1.0.6 

type VectorStoreProperties struct {
	DataType    VectorStorePropertyType `json:"data_type"`
	Description string                  `json:"description"`
}

type VectorStorePropertyType added in v1.0.6 

type VectorStorePropertyType string
const (
	VectorStorePropertyTypeString      VectorStorePropertyType = "string"
	VectorStorePropertyTypeInteger     VectorStorePropertyType = "integer"
	VectorStorePropertyTypeBoolean     VectorStorePropertyType = "boolean"
	VectorStorePropertyTypeStringArray VectorStorePropertyType = "string[]"
)

type VectorStoreType 

type VectorStoreType string
const (
	VectorStoreTypeWeaviate VectorStoreType = "weaviate"
	VectorStoreTypeRedis    VectorStoreType = "redis"
	VectorStoreTypeQdrant   VectorStoreType = "qdrant"
	VectorStoreTypePinecone VectorStoreType = "pinecone"
)

type WeaviateConfig added in v1.0.3 

type WeaviateConfig struct {
	// Connection settings
	Scheme     string              `json:"scheme"`                // "http" or "https" - REQUIRED
	Host       *schemas.EnvVar     `json:"host"`                  // "localhost:8080" - REQUIRED
	GrpcConfig *WeaviateGrpcConfig `json:"grpc_config,omitempty"` // grpc config for weaviate (optional)

	// Authentication settings (optional)
	APIKey  *schemas.EnvVar   `json:"api_key,omitempty"` // API key for authentication
	Headers map[string]string `json:"headers,omitempty"` // Additional headers

	// Connection settings
	Timeout time.Duration `json:"timeout,omitempty"` // Request timeout (optional)
}

WeaviateConfig represents the configuration for the Weaviate vector store.

type WeaviateGrpcConfig added in v1.0.6 

type WeaviateGrpcConfig struct {
	// Host is the host of the weaviate server (host:port).
	// If host is without a port number then the 80 port for insecured and 443 port for secured connections will be used.
	Host *schemas.EnvVar `json:"host"`
	// Secured is a boolean flag indicating if the connection is secured
	Secured bool `json:"secured"`
}

type WeaviateStore added in v1.0.3 

type WeaviateStore struct {
	// contains filtered or unexported fields
}

WeaviateStore represents the Weaviate vector store.

func (*WeaviateStore) Add added in v1.0.3 

func (s *WeaviateStore) Add(ctx context.Context, className string, id string, embedding []float32, metadata map[string]interface{}) error

Add stores a new object (with or without embedding)

func (*WeaviateStore) Close added in v1.0.3 

func (s *WeaviateStore) Close(ctx context.Context, className string) error

func (*WeaviateStore) CreateNamespace added in v1.0.6 

func (s *WeaviateStore) CreateNamespace(ctx context.Context, className string, dimension int, properties map[string]VectorStoreProperties) error

func (*WeaviateStore) Delete added in v1.0.3 

func (s *WeaviateStore) Delete(ctx context.Context, className string, id string) error

Delete removes multiple objects by ID

func (*WeaviateStore) DeleteAll added in v1.0.3 

func (s *WeaviateStore) DeleteAll(ctx context.Context, className string, queries []Query) ([]DeleteResult, error)

func (*WeaviateStore) DeleteNamespace added in v1.0.10 

func (s *WeaviateStore) DeleteNamespace(ctx context.Context, className string) error

func (*WeaviateStore) GetAll added in v1.0.3 

func (s *WeaviateStore) GetAll(ctx context.Context, className string, queries []Query, selectFields []string, cursor *string, limit int64) ([]SearchResult, *string, error)

GetAll with filtering + pagination

func (*WeaviateStore) GetChunk added in v1.0.3 

func (s *WeaviateStore) GetChunk(ctx context.Context, className string, id string) (SearchResult, error)

GetChunk returns the "metadata" for a single key

func (*WeaviateStore) GetChunks added in v1.0.3 

func (s *WeaviateStore) GetChunks(ctx context.Context, className string, ids []string) ([]SearchResult, error)

GetChunks returns multiple objects by ID

func (*WeaviateStore) GetNearest added in v1.0.3 

func (s *WeaviateStore) GetNearest(
	ctx context.Context,
	className string,
	vector []float32,
	queries []Query,
	selectFields []string,
	threshold float64,
	limit int64,
) ([]SearchResult, error)

GetNearest with explicit filters only

func (*WeaviateStore) Ping added in v1.1.8 

func (s *WeaviateStore) Ping(ctx context.Context) error

Ping checks if the Weaviate server is reachable.

func (*WeaviateStore) RequiresVectors added in v1.2.15 

func (s *WeaviateStore) RequiresVectors() bool

RequiresVectors returns true because Weaviate's HNSW index requires vectors for proper object indexing and retrieval.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.