vectorstore

package
v1.11.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 29, 2026 License: Apache-2.0 Imports: 37 Imported by: 2

Documentation

Overview

Package vectorstore provides a generic interface for vector stores.

Index

Constants

View Source 
const (
	// BatchLimit is the default limit used for pagination and batch operations
	BatchLimit = 100
	// RedisMaxSearchResults is the maximum number of results Redis Search returns in a single query.
	// This is the default MAXSEARCHRESULTS configuration in Redis Search.
	RedisMaxSearchResults = 10000
)
View Source 
const (
	// Default class names (Weaviate prefers PascalCase)
	DefaultClassName = "BifrostStore"
)

Default values for Weaviate vector index configuration

Variables

View Source 
var (
	ErrNotFound     = errors.New("vectorstore: not found")
	ErrNotSupported = errors.New("vectorstore: operation not supported on this store")
	ErrQuerySyntax  = errors.New("vectorstore: query syntax error")
	// ErrDimensionMismatch is wrapped by CreateNamespace when the namespace already holds vectors
	// of a different size than the caller asked for. It is the caller's configuration that is
	// wrong, so callers can surface it as a client error rather than a server failure.
	ErrDimensionMismatch = errors.New("vectorstore: namespace dimension mismatch")
)

Functions

func FiltersVectorReadsOnServer added in v1.11.0 

func FiltersVectorReadsOnServer(store VectorStore) bool

FiltersVectorReadsOnServer reports whether store filters a paged read with vectors on the server. A store that does not say is taken not to.

func IncludeVectorsRequested added in v1.11.0 

func IncludeVectorsRequested(ctx context.Context) bool

IncludeVectorsRequested reports whether the current read asked for vectors.

func IsScanFallbackDisabled added in v1.2.33 

func IsScanFallbackDisabled(ctx context.Context) bool

IsScanFallbackDisabled reports whether scan fallback has been disabled for the current vector store operation.

func VectorFromAdditional added in v1.11.0 

func VectorFromAdditional(raw interface{}) []float32

VectorFromAdditional converts the vector a backend returns in its "_additional" block (a []interface{} of float64, as GraphQL decodes numbers) into []float32. Anything that is not a numeric list yields nil.

func WithDisableScanFallback added in v1.2.33 

func WithDisableScanFallback(ctx context.Context) context.Context

WithDisableScanFallback returns a derived context that tells vector stores not to fall back to full scans when indexed search fails.

func WithIncludeVectors added in v1.11.0 

func WithIncludeVectors(ctx context.Context) context.Context

WithIncludeVectors asks paging reads (GetAll) to return each entry's stored vector alongside its properties. Off by default: a vector is thousands of floats, and a listing that only wants ids should not carry them.

Types

type ChromemConfig added in v1.6.1 

type ChromemConfig struct {
	Path     string `json:"path,omitempty"`
	Compress bool   `json:"compress,omitempty"`
}

ChromemConfig represents the configuration for the embedded chromem vector store. Chromem runs in-process (pure Go, no external service). With an empty Path the store is memory-only; with a Path set, documents are persisted to disk and reloaded on startup.

type ChromemStore added in v1.6.1 

type ChromemStore struct {
	// contains filtered or unexported fields
}

ChromemStore is an embedded VectorStore backed by chromem-go.

Chromem has no document-enumeration API and no typed metadata, so this backend bridges both gaps itself:

  • metadata values are JSON-encoded into chromem's map[string]string and decoded back to typed values on read;
  • GetAll/DeleteAll/GetNearest enumerate via an exhaustive QueryEmbedding with a probe vector, which requires the namespace dimension. Like the Pinecone backend, the dimension is recorded in memory by CreateNamespace, so callers must CreateNamespace after every process start (all current callers already do);
  • a paged GetAll enumerates once, on its first page, and later pages read only their own documents (see chromemPageSnapshot).

func (*ChromemStore) Add added in v1.6.1 

func (s *ChromemStore) Add(ctx context.Context, namespace string, id string, embedding []float32, metadata map[string]interface{}) error

Add upserts a document with a precomputed embedding.

func (*ChromemStore) Close added in v1.6.1 

func (s *ChromemStore) Close(ctx context.Context, namespace string) error

Close is a no-op: chromem persists synchronously on each write.

func (*ChromemStore) CreateNamespace added in v1.6.1 

func (s *ChromemStore) CreateNamespace(ctx context.Context, namespace string, dimension int, properties map[string]VectorStoreProperties) error

CreateNamespace creates a collection if it does not exist and records its dimension. Idempotent: callers invoke it on every startup.

func (*ChromemStore) Delete added in v1.6.1 

func (s *ChromemStore) Delete(ctx context.Context, namespace string, id string) error

Delete removes a single document. Deleting a missing document is not an error.

func (*ChromemStore) DeleteAll added in v1.6.1 

func (s *ChromemStore) DeleteAll(ctx context.Context, namespace string, queries []Query) ([]DeleteResult, error)

DeleteAll removes all documents matching the queries (all documents when no queries are given) and reports per-document status.

func (*ChromemStore) DeleteNamespace added in v1.6.1 

func (s *ChromemStore) DeleteNamespace(ctx context.Context, namespace string) error

DeleteNamespace removes a collection. Idempotent: deleting a missing namespace is not an error.

func (*ChromemStore) GetAll added in v1.6.1 

func (s *ChromemStore) GetAll(ctx context.Context, namespace string, queries []Query, selectFields []string, cursor *string, limit int64) ([]SearchResult, *string, error)

GetAll retrieves documents with optional filtering, using an offset cursor for pagination. Results are ordered by ID for deterministic paging.

The cursor is "<snapshot>:<offset>". A snapshot that has expired, or a bare offset, is rebuilt from the namespace as it is now and read from the same offset, which is what every page cost before snapshots. A document deleted, or changed so it no longer matches, after the first page is skipped, and the page is filled from further on so it is never short before the end.

func (*ChromemStore) GetChunk added in v1.6.1 

func (s *ChromemStore) GetChunk(ctx context.Context, namespace string, id string) (SearchResult, error)

GetChunk retrieves a single document by ID.

func (*ChromemStore) GetChunks added in v1.6.1 

func (s *ChromemStore) GetChunks(ctx context.Context, namespace string, ids []string) ([]SearchResult, error)

GetChunks retrieves multiple documents by ID. Missing IDs are skipped, matching the behavior of the other backends.

func (*ChromemStore) GetNearest added in v1.6.1 

func (s *ChromemStore) GetNearest(ctx context.Context, namespace string, vector []float32, queries []Query, selectFields []string, threshold float64, limit int64) ([]SearchResult, error)

GetNearest performs an exhaustive cosine-similarity search.

func (*ChromemStore) ListNamespaces added in v1.6.1 

func (s *ChromemStore) ListNamespaces(ctx context.Context, prefix string) ([]string, error)

ListNamespaces returns the collections chromem currently holds. A persistent chromem database reloads its collections at open, so this reflects what is on disk as well as what this process created.

func (*ChromemStore) Ping added in v1.6.1 

func (s *ChromemStore) Ping(ctx context.Context) error

Ping checks the health of the store. Chromem is in-process, so this only verifies the store was initialized.

func (*ChromemStore) RequiresVectors added in v1.6.1 

func (s *ChromemStore) RequiresVectors() bool

RequiresVectors returns true: chromem entries always carry vectors.

type Config 

type Config struct {
	Enabled bool            `json:"enabled"`
	Type    VectorStoreType `json:"type"`
	Config  any             `json:"config"`
}

Config represents the configuration for the vector store.

func (*Config) UnmarshalJSON 

func (c *Config) UnmarshalJSON(data []byte) error

UnmarshalJSON unmarshals the config from JSON.

type DeleteResult added in v1.0.3 

type DeleteResult struct {
	ID     string
	Status DeleteStatus
	Error  string
}

DeleteResult represents the result of a delete operation.

type DeleteStatus added in v1.0.3 

type DeleteStatus string
const (
	DeleteStatusSuccess DeleteStatus = "success"
	DeleteStatusError   DeleteStatus = "error"
)

type PineconeConfig added in v1.2.15 

type PineconeConfig struct {
	APIKey    schemas.SecretVar `json:"api_key"`    // Pinecone API key - REQUIRED
	IndexHost schemas.SecretVar `json:"index_host"` // Index host URL from Pinecone console - REQUIRED
}

PineconeConfig represents the configuration for the Pinecone vector store.

type PineconeStore added in v1.2.15 

type PineconeStore struct {
	// contains filtered or unexported fields
}

PineconeStore represents the Pinecone vector store.

func (*PineconeStore) Add added in v1.2.15 

func (s *PineconeStore) Add(ctx context.Context, namespace string, id string, embedding []float32, metadata map[string]interface{}) error

Add stores a new vector in the Pinecone vector store.

func (*PineconeStore) Close added in v1.2.15 

func (s *PineconeStore) Close(ctx context.Context, namespace string) error

Close closes the Pinecone client connection. If namespace is non-empty, only that namespace connection is closed. If namespace is empty, all connections (indexConn and all namespaces) are closed.

func (*PineconeStore) CreateNamespace added in v1.2.15 

func (s *PineconeStore) CreateNamespace(ctx context.Context, namespace string, dimension int, properties map[string]VectorStoreProperties) error

CreateNamespace creates a new namespace in the Pinecone vector store. Note: Pinecone namespaces are created implicitly when upserting vectors. This method is a no-op but ensures the connection is valid.

func (*PineconeStore) Delete added in v1.2.15 

func (s *PineconeStore) Delete(ctx context.Context, namespace string, id string) error

Delete removes a vector from the Pinecone vector store.

func (*PineconeStore) DeleteAll added in v1.2.15 

func (s *PineconeStore) DeleteAll(ctx context.Context, namespace string, queries []Query) ([]DeleteResult, error)

DeleteAll removes multiple vectors matching the filter.

func (*PineconeStore) DeleteNamespace added in v1.2.15 

func (s *PineconeStore) DeleteNamespace(ctx context.Context, namespace string) error

DeleteNamespace deletes a namespace from the Pinecone vector store.

func (*PineconeStore) FiltersVectorReadsOnServer added in v1.11.0 

func (s *PineconeStore) FiltersVectorReadsOnServer() bool

FiltersVectorReadsOnServer reports whether a filtered read is served by fetch-by-metadata. Not every index has it, and one without falls back to listing, which works and costs the whole namespace for every read - so the index is asked, once, rather than assumed. Anything but a plain refusal is taken as having it: the read itself will report what is wrong.

func (*PineconeStore) GetAll added in v1.2.15 

func (s *PineconeStore) GetAll(ctx context.Context, namespace string, queries []Query, selectFields []string, cursor *string, limit int64) ([]SearchResult, *string, error)

GetAll retrieves all vectors with optional filtering and pagination. Note: This implementation uses QueryByVectorValues with a zero vector instead of ListVectors because ListVectors has severe eventual consistency issues on Pinecone Serverless/Starter indexes. The metadata filtering is done server-side by Pinecone, providing much better consistency.

func (*PineconeStore) GetChunk added in v1.2.15 

func (s *PineconeStore) GetChunk(ctx context.Context, namespace string, id string) (SearchResult, error)

GetChunk retrieves a single vector from the Pinecone vector store.

func (*PineconeStore) GetChunks added in v1.2.15 

func (s *PineconeStore) GetChunks(ctx context.Context, namespace string, ids []string) ([]SearchResult, error)

GetChunks retrieves multiple vectors from the Pinecone vector store.

func (*PineconeStore) GetNearest added in v1.2.15 

func (s *PineconeStore) GetNearest(ctx context.Context, namespace string, vector []float32, queries []Query, selectFields []string, threshold float64, limit int64) ([]SearchResult, error)

GetNearest retrieves the nearest vectors to a given vector.

func (*PineconeStore) ListNamespaces added in v1.6.1 

func (s *PineconeStore) ListNamespaces(ctx context.Context, prefix string) ([]string, error)

ListNamespaces returns the namespaces in the configured Pinecone index. Pinecone filters by prefix server-side and pages the result, so this follows the pagination token to the end rather than returning a partial view — a caller reconciling what exists against what it expects must see all of it.

func (*PineconeStore) Ping added in v1.2.15 

func (s *PineconeStore) Ping(ctx context.Context) error

Ping checks if the Pinecone server is reachable.

func (*PineconeStore) RequiresVectors added in v1.2.15 

func (s *PineconeStore) RequiresVectors() bool

RequiresVectors returns true because Pinecone is a dedicated vector database that requires vectors for all entries with a specific dimension.

type QdrantConfig added in v1.1.38 

type QdrantConfig struct {
	Host             schemas.SecretVar `json:"host"`                           // Qdrant server host - REQUIRED
	Port             schemas.SecretVar `json:"port"`                           // Qdrant server port  (fallback to 6334 for gRPC)
	APIKey           schemas.SecretVar `json:"api_key,omitempty"`              // API key for authentication - Optional
	UseTLS           schemas.SecretVar `json:"use_tls,omitempty"`              // Use TLS for connection - Optional
	MaxRecvMsgSizeMB schemas.SecretVar `json:"max_recv_msg_size_mb,omitempty"` // gRPC max receive message size in MB (default: 64). Increase when caching large payloads such as image generation responses.
}

QdrantConfig represents the configuration for the Qdrant vector store.

type QdrantStore added in v1.1.38 

type QdrantStore struct {
	// contains filtered or unexported fields
}

QdrantStore represents the Qdrant vector store.

func (*QdrantStore) Add added in v1.1.38 

func (s *QdrantStore) Add(ctx context.Context, namespace string, id string, embedding []float32, metadata map[string]interface{}) error

Add stores a new point in the Qdrant vector store.

func (*QdrantStore) Close added in v1.1.38 

func (s *QdrantStore) Close(ctx context.Context, namespace string) error

Close closes the Qdrant client connection.

func (*QdrantStore) CreateNamespace added in v1.1.38 

func (s *QdrantStore) CreateNamespace(ctx context.Context, namespace string, dimension int, properties map[string]VectorStoreProperties) error

CreateNamespace creates a new collection in the Qdrant vector store.

func (*QdrantStore) Delete added in v1.1.38 

func (s *QdrantStore) Delete(ctx context.Context, namespace string, id string) error

Delete removes a point from the Qdrant vector store.

func (*QdrantStore) DeleteAll added in v1.1.38 

func (s *QdrantStore) DeleteAll(ctx context.Context, namespace string, queries []Query) ([]DeleteResult, error)

DeleteAll removes multiple points matching the filter.

func (*QdrantStore) DeleteNamespace added in v1.1.38 

func (s *QdrantStore) DeleteNamespace(ctx context.Context, namespace string) error

DeleteNamespace deletes a collection from the Qdrant vector store.

func (*QdrantStore) FiltersVectorReadsOnServer added in v1.11.0 

func (s *QdrantStore) FiltersVectorReadsOnServer() bool

FiltersVectorReadsOnServer reports that a scroll applies its filter on the server.

func (*QdrantStore) GetAll added in v1.1.38 

func (s *QdrantStore) GetAll(ctx context.Context, namespace string, queries []Query, selectFields []string, cursor *string, limit int64) ([]SearchResult, *string, error)

GetAll retrieves all points with optional filtering and pagination.

func (*QdrantStore) GetChunk added in v1.1.38 

func (s *QdrantStore) GetChunk(ctx context.Context, namespace string, id string) (SearchResult, error)

GetChunk retrieves a single point from the Qdrant vector store.

func (*QdrantStore) GetChunks added in v1.1.38 

func (s *QdrantStore) GetChunks(ctx context.Context, namespace string, ids []string) ([]SearchResult, error)

GetChunks retrieves multiple points from the Qdrant vector store.

func (*QdrantStore) GetNearest added in v1.1.38 

func (s *QdrantStore) GetNearest(ctx context.Context, namespace string, vector []float32, queries []Query, selectFields []string, threshold float64, limit int64) ([]SearchResult, error)

GetNearest retrieves the nearest points to a vector.

func (*QdrantStore) ListNamespaces added in v1.6.1 

func (s *QdrantStore) ListNamespaces(ctx context.Context, prefix string) ([]string, error)

ListNamespaces returns the Qdrant collections beginning with prefix. Qdrant's list API takes no filter, so the prefix is applied here.

func (*QdrantStore) Ping added in v1.1.38 

func (s *QdrantStore) Ping(ctx context.Context) error

Ping checks if the Qdrant server is reachable.

func (*QdrantStore) RequiresVectors added in v1.2.15 

func (s *QdrantStore) RequiresVectors() bool

RequiresVectors returns true because Qdrant is a dedicated vector database that requires vectors for all points/entries.

type Query added in v1.0.3 

type Query struct {
	Field    string
	Operator QueryOperator
	Value    interface{}
}

Query represents a query to the vector store.

type QueryOperator added in v1.0.3 

type QueryOperator string
const (
	QueryOperatorEqual              QueryOperator = "Equal"
	QueryOperatorNotEqual           QueryOperator = "NotEqual"
	QueryOperatorGreaterThan        QueryOperator = "GreaterThan"
	QueryOperatorLessThan           QueryOperator = "LessThan"
	QueryOperatorGreaterThanOrEqual QueryOperator = "GreaterThanOrEqual"
	QueryOperatorLessThanOrEqual    QueryOperator = "LessThanOrEqual"
	QueryOperatorLike               QueryOperator = "Like"
	QueryOperatorContainsAny        QueryOperator = "ContainsAny"
	QueryOperatorContainsAll        QueryOperator = "ContainsAll"
	QueryOperatorIsNull             QueryOperator = "IsNull"
	QueryOperatorIsNotNull          QueryOperator = "IsNotNull"
)

type RedisConfig 

type RedisConfig struct {
	// Connection settings
	Addr     *schemas.SecretVar `json:"addr"`               // Redis server address (host:port) - REQUIRED
	Username *schemas.SecretVar `json:"username,omitempty"` // Username for Redis AUTH (optional)
	Password *schemas.SecretVar `json:"password,omitempty"` // Password for Redis AUTH (optional)
	DB       *schemas.SecretVar `json:"db,omitempty"`       // Redis database number (default: 0)

	// TLS settings
	UseTLS             *schemas.SecretVar `json:"use_tls,omitempty"`              // Enable TLS for connection (default: false)
	InsecureSkipVerify *schemas.SecretVar `json:"insecure_skip_verify,omitempty"` // Skip TLS cert verification (default: false)
	CACertPEM          *schemas.SecretVar `json:"ca_cert_pem,omitempty"`          // PEM-encoded CA certificate to trust for Redis/Valkey TLS

	// Cluster mode
	ClusterMode *schemas.SecretVar `json:"cluster_mode,omitempty"` // Use Redis Cluster client (default: false)

	// Connection pool and timeout settings (passed directly to Redis client).
	// All duration fields accept either a Go duration string (e.g. "5s", "500ms",
	// "1m30s") or a plain integer nanosecond value for backward compatibility.
	PoolSize        int              `json:"pool_size,omitempty"`          // Maximum number of socket connections (optional)
	MaxActiveConns  int              `json:"max_active_conns,omitempty"`   // Maximum number of active connections (optional)
	MinIdleConns    int              `json:"min_idle_conns,omitempty"`     // Minimum number of idle connections (optional)
	MaxIdleConns    int              `json:"max_idle_conns,omitempty"`     // Maximum number of idle connections (optional)
	ConnMaxLifetime schemas.Duration `json:"conn_max_lifetime,omitempty"`  // Connection maximum lifetime (optional)
	ConnMaxIdleTime schemas.Duration `json:"conn_max_idle_time,omitempty"` // Connection maximum idle time (optional)
	DialTimeout     schemas.Duration `json:"dial_timeout,omitempty"`       // Timeout for socket connection (optional)
	ReadTimeout     schemas.Duration `json:"read_timeout,omitempty"`       // Timeout for socket reads (optional)
	WriteTimeout    schemas.Duration `json:"write_timeout,omitempty"`      // Timeout for socket writes (optional)
	ContextTimeout  schemas.Duration `json:"context_timeout,omitempty"`    // Timeout for Redis operations (optional)
}

type RedisStore 

type RedisStore struct {
	// contains filtered or unexported fields
}

RedisStore represents the Redis vector store.

func (*RedisStore) Add 

func (s *RedisStore) Add(ctx context.Context, namespace string, id string, embedding []float32, metadata map[string]interface{}) error

Add stores a new chunk in the Redis vector store.

func (*RedisStore) Close 

func (s *RedisStore) Close(ctx context.Context, namespace string) error

Close closes the Redis vector store.

func (*RedisStore) CreateNamespace added in v1.0.16 

func (s *RedisStore) CreateNamespace(ctx context.Context, namespace string, dimension int, properties map[string]VectorStoreProperties) error

CreateNamespace creates a new namespace in the Redis vector store.

func (*RedisStore) Delete 

func (s *RedisStore) Delete(ctx context.Context, namespace string, id string) error

Delete deletes a chunk from the Redis vector store.

func (*RedisStore) DeleteAll added in v1.0.16 

func (s *RedisStore) DeleteAll(ctx context.Context, namespace string, queries []Query) ([]DeleteResult, error)

DeleteAll deletes all chunks from the Redis vector store.

func (*RedisStore) DeleteNamespace added in v1.0.16 

func (s *RedisStore) DeleteNamespace(ctx context.Context, namespace string) error

func (*RedisStore) FiltersVectorReadsOnServer added in v1.11.0 

func (s *RedisStore) FiltersVectorReadsOnServer() bool

FiltersVectorReadsOnServer reports that a filtered read is served by the search index.

func (*RedisStore) GetAll 

func (s *RedisStore) GetAll(ctx context.Context, namespace string, queries []Query, selectFields []string, cursor *string, limit int64) ([]SearchResult, *string, error)

GetAll retrieves all chunks from the Redis vector store.

A caller that asked for vectors gets each entry's stored one. The vector is a field of the entry's hash like any other, so it is read by asking for that field too and lifting it out of the properties afterwards.

Unfiltered, that caller is reading the namespace out, so it is served by walking the keys rather than by search: a search stops at MAXSEARCHRESULTS, 10,000 by default, and answers a read past it with nothing - which looks exactly like the end of the namespace. Filtered, it is taking a slice, and search is what can find one without visiting every key; the slice has to fit under that ceiling, which is the caller's to arrange.

func (*RedisStore) GetChunk 

func (s *RedisStore) GetChunk(ctx context.Context, namespace string, id string) (SearchResult, error)

GetChunk retrieves a chunk from the Redis vector store.

func (*RedisStore) GetChunks 

func (s *RedisStore) GetChunks(ctx context.Context, namespace string, ids []string) ([]SearchResult, error)

GetChunks retrieves multiple chunks from the Redis vector store.

func (*RedisStore) GetNearest added in v1.0.16 

func (s *RedisStore) GetNearest(ctx context.Context, namespace string, vector []float32, queries []Query, selectFields []string, threshold float64, limit int64) ([]SearchResult, error)

GetNearest retrieves the nearest chunks from the Redis vector store.

func (*RedisStore) ListNamespaces added in v1.6.1 

func (s *RedisStore) ListNamespaces(ctx context.Context, prefix string) ([]string, error)

DeleteNamespace deletes a namespace from the Redis vector store. ListNamespaces returns the search indices beginning with prefix. A namespace here is an FT index, so FT._LIST is the enumeration; the keys it owns live under the index's own "<namespace>:" prefix and are not separately listed.

func (*RedisStore) Ping added in v1.1.8 

func (s *RedisStore) Ping(ctx context.Context) error

Ping checks if the Redis server is reachable.

func (*RedisStore) RequiresVectors added in v1.2.15 

func (s *RedisStore) RequiresVectors() bool

RequiresVectors returns false because Redis can store hash data with or without vectors.

type SearchResult added in v1.0.3 

type SearchResult struct {
	ID         string
	Score      *float64
	Properties map[string]interface{}
	// Vector is the stored embedding. Populated only when the read was made
	// with WithIncludeVectors; vectors are large and most callers want the
	// properties alone.
	Vector []float32
}

SearchResult represents a search result with metadata.

type ServerFilteredReader added in v1.11.0 

type ServerFilteredReader interface {
	FiltersVectorReadsOnServer() bool
}

ServerFilteredReader is implemented by a store that can say whether it applies a filter on the server while paging a read that carries vectors.

It decides how a caller should take a large namespace out. A store that filters on the server is read a slice at a time, each costing what the slice holds. One that does not pays for the whole namespace on every filtered read - it walks everything and drops what does not match - so it is read once, unfiltered, and the caller does the dropping.

type VectorStore 

type VectorStore interface {
	// Health check
	Ping(ctx context.Context) error
	// CreateNamespace creates a new namespace in the vector store.
	CreateNamespace(ctx context.Context, namespace string, dimension int, properties map[string]VectorStoreProperties) error
	// DeleteNamespace deletes a namespace from the vector store.
	DeleteNamespace(ctx context.Context, namespace string) error
	// ListNamespaces returns the names of existing namespaces beginning with
	// prefix, sorted. An empty prefix lists everything the backend exposes.
	//
	// Callers that create namespaces under a naming scheme of their own need
	// this to find the ones they left behind: a name derived from content
	// cannot be reconstructed once the content has changed, so without
	// enumeration an abandoned namespace becomes unreachable rather than
	// merely unused. Filtering happens in the backend where the API supports
	// it and here where it does not, so the result is the same either way.
	ListNamespaces(ctx context.Context, prefix string) ([]string, error)
	// GetChunk retrieves a single vector from the vector store.
	GetChunk(ctx context.Context, namespace string, id string) (SearchResult, error)
	// GetChunks retrieves multiple vectors from the vector store.
	GetChunks(ctx context.Context, namespace string, ids []string) ([]SearchResult, error)
	// GetAll retrieves all vectors from the vector store.
	GetAll(ctx context.Context, namespace string, queries []Query, selectFields []string, cursor *string, limit int64) ([]SearchResult, *string, error)
	// GetNearest retrieves the nearest vectors from the vector store. threshold and each result's
	// Score are cosine similarity on every store.
	GetNearest(ctx context.Context, namespace string, vector []float32, queries []Query, selectFields []string, threshold float64, limit int64) ([]SearchResult, error)
	// RequiresVectors returns true if the vector store requires vectors for all entries.
	// Dedicated vector databases like Qdrant and Pinecone require vectors, while
	// more flexible stores like Weaviate and Redis can store metadata-only entries.
	RequiresVectors() bool
	// Add stores a new vector in the vector store.
	Add(ctx context.Context, namespace string, id string, embedding []float32, metadata map[string]interface{}) error
	// Delete removes a vector from the vector store.
	Delete(ctx context.Context, namespace string, id string) error
	// DeleteAll deletes all vectors from the vector store.
	DeleteAll(ctx context.Context, namespace string, queries []Query) ([]DeleteResult, error)
	// Close closes the vector store.
	Close(ctx context.Context, namespace string) error
}

VectorStore represents the interface for the vector store.

func NewVectorStore 

func NewVectorStore(ctx context.Context, config *Config, logger schemas.Logger) (VectorStore, error)

NewVectorStore returns a new vector store based on the configuration.

type VectorStoreProperties added in v1.0.6 

type VectorStoreProperties struct {
	DataType    VectorStorePropertyType `json:"data_type"`
	Description string                  `json:"description"`
	// Filterable marks a property used in Query filters; stores may encode it.
	Filterable bool `json:"filterable,omitempty"`
}

type VectorStorePropertyType added in v1.0.6 

type VectorStorePropertyType string
const (
	VectorStorePropertyTypeString      VectorStorePropertyType = "string"
	VectorStorePropertyTypeInteger     VectorStorePropertyType = "integer"
	VectorStorePropertyTypeBoolean     VectorStorePropertyType = "boolean"
	VectorStorePropertyTypeStringArray VectorStorePropertyType = "string[]"
)

type VectorStoreType 

type VectorStoreType string
const (
	VectorStoreTypeWeaviate VectorStoreType = "weaviate"
	VectorStoreTypeRedis    VectorStoreType = "redis"
	VectorStoreTypeQdrant   VectorStoreType = "qdrant"
	VectorStoreTypePinecone VectorStoreType = "pinecone"
	VectorStoreTypeChromem  VectorStoreType = "chromem"
)

type WeaviateConfig added in v1.0.3 

type WeaviateConfig struct {
	// Connection settings
	Scheme     string              `json:"scheme"`                // "http" or "https" - REQUIRED
	Host       *schemas.SecretVar  `json:"host"`                  // "localhost:8080" - REQUIRED
	GrpcConfig *WeaviateGrpcConfig `json:"grpc_config,omitempty"` // grpc config for weaviate (optional)

	// Authentication settings (optional)
	APIKey  *schemas.SecretVar `json:"api_key,omitempty"` // API key for authentication
	Headers map[string]string  `json:"headers,omitempty"` // Additional headers

	// Connection settings
	// Timeout accepts either a Go duration string (e.g. "5s", "30s") or an
	// integer nanosecond value for backward compatibility.
	Timeout schemas.Duration `json:"timeout,omitempty"` // Request timeout (optional)
}

WeaviateConfig represents the configuration for the Weaviate vector store.

type WeaviateGrpcConfig added in v1.0.6 

type WeaviateGrpcConfig struct {
	// Host is the host of the weaviate server (host:port).
	// If host is without a port number then the 80 port for insecured and 443 port for secured connections will be used.
	Host *schemas.SecretVar `json:"host"`
	// Secured is a boolean flag indicating if the connection is secured
	Secured bool `json:"secured"`
}

type WeaviateStore added in v1.0.3 

type WeaviateStore struct {
	// contains filtered or unexported fields
}

WeaviateStore represents the Weaviate vector store.

func (*WeaviateStore) Add added in v1.0.3 

func (s *WeaviateStore) Add(ctx context.Context, className string, id string, embedding []float32, metadata map[string]interface{}) error

Add stores a new object (with or without embedding), replacing any object already stored under the same id.

Weaviate is the only backend whose create path is not itself an upsert: Creator() is a POST, which Weaviate rejects with 422 when the id exists, while Qdrant, Pinecone and Redis all overwrite. Callers write deterministic, content-derived ids (the complexity router derives one per exemplar from its configuration fingerprint), so a repeated Add is an ordinary retry or a second writer racing on identical content, not a caller error. Falling back to a replacing update keeps that uniform across backends.

func (*WeaviateStore) Close added in v1.0.3 

func (s *WeaviateStore) Close(ctx context.Context, className string) error

func (*WeaviateStore) CreateNamespace added in v1.0.6 

func (s *WeaviateStore) CreateNamespace(ctx context.Context, className string, dimension int, properties map[string]VectorStoreProperties) error

func (*WeaviateStore) Delete added in v1.0.3 

func (s *WeaviateStore) Delete(ctx context.Context, className string, id string) error

Delete removes multiple objects by ID Delete removes one object. Deleting one that is not there is not an error, as on the other backends: Weaviate answers 404, and a caller clearing out an id it may never have written has nothing to act on.

func (*WeaviateStore) DeleteAll added in v1.0.3 

func (s *WeaviateStore) DeleteAll(ctx context.Context, className string, queries []Query) ([]DeleteResult, error)

func (*WeaviateStore) DeleteNamespace added in v1.0.10 

func (s *WeaviateStore) DeleteNamespace(ctx context.Context, className string) error

func (*WeaviateStore) FiltersVectorReadsOnServer added in v1.11.0 

func (s *WeaviateStore) FiltersVectorReadsOnServer() bool

FiltersVectorReadsOnServer reports that a filtered read is served by the class's own indexes.

func (*WeaviateStore) GetAll added in v1.0.3 

func (s *WeaviateStore) GetAll(ctx context.Context, className string, queries []Query, selectFields []string, cursor *string, limit int64) ([]SearchResult, *string, error)

GetAll with filtering + pagination

func (*WeaviateStore) GetChunk added in v1.0.3 

func (s *WeaviateStore) GetChunk(ctx context.Context, className string, id string) (SearchResult, error)

GetChunk returns the "metadata" for a single key

func (*WeaviateStore) GetChunks added in v1.0.3 

func (s *WeaviateStore) GetChunks(ctx context.Context, className string, ids []string) ([]SearchResult, error)

GetChunks returns multiple objects by ID

func (*WeaviateStore) GetNearest added in v1.0.3 

func (s *WeaviateStore) GetNearest(
	ctx context.Context,
	className string,
	vector []float32,
	queries []Query,
	selectFields []string,
	threshold float64,
	limit int64,
) ([]SearchResult, error)

GetNearest with explicit filters only

func (*WeaviateStore) ListNamespaces added in v1.6.1 

func (s *WeaviateStore) ListNamespaces(ctx context.Context, prefix string) ([]string, error)

ListNamespaces returns the Weaviate classes beginning with prefix. Weaviate serves the whole schema in one read and has no server-side name filter, so the prefix is applied here.

func (*WeaviateStore) Ping added in v1.1.8 

func (s *WeaviateStore) Ping(ctx context.Context) error

Ping checks if the Weaviate server is reachable.

func (*WeaviateStore) RequiresVectors added in v1.2.15 

func (s *WeaviateStore) RequiresVectors() bool

RequiresVectors returns true because Weaviate's HNSW index requires vectors for proper object indexing and retrieval.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.