Documentation
¶
Overview ¶
Package redis stores prepared vectors in host-owned Redis HASHes and performs native KNN search. NewStore and operations inspect the concrete RESP3 FT.INFO definition: one unfiltered HASH prefix ending with SpaceID.StorageKey()+":", one FLOAT32 embedding VECTOR (COSINE, L2 or IP), optional content TEXT and optional metadata_terms TAG with comma separator. The Host provisions the index and owns the client, topology, credentials and timeouts.
Upsert prepares every row before writing. Each HASH contains content, vector bytes, canonical metadata_json, and a derived metadata_terms projection. Lua replaces one HASH atomically. WriteBatchSize bounds the nontransactional pipeline; vectorstore.WriteError exposes confirmed, uncertain and unattempted IDs. Search index visibility can lag a confirmed write.
Query has no enumeration path. It ranks with FT.SEARCH; its native predicate subset is scalar equality on a top-level string when the TAG index exists. Field names and exact string values are encoded as lowercase hexadecimal TAG tokens. Response warnings, incomplete replies, damaged records and predicates outside the supported subset are errors.
QueryByScan and DeleteMatching explicitly accept a time, record and byte grant. SCAN COUNT is only a hint: each master bounds key-page output in Lua. Each distinct key is charged before its HASH payload is fetched; bytes count the key plus the four field names and values, excluding protocol framing. A second atomic size check rejects concurrent growth; vanished keys are charged for their names. The host binds ScanNodes to the complete authoritative keyspace; ClusterClient.ForEachMaster supports decorated clients without type discovery. Ring.ForEachShard is insufficient because it can skip unavailable shards. A slow scan may hit its deadline and has no snapshot isolation. No partial search result is returned. DeleteMatching finishes selection then compares current canonical metadata before deleting each record; earlier deletions may survive a later failure. DeleteIDs accepts explicit identities and uses single-key DEL.
Integration fixtures require SCOPE_REDIS_ADDR on an isolated Redis Search instance. They create disposable native indexes and remove them afterwards.
Index ¶
- Constants
- Variables
- type RedisClient
- type Store
- func (s *Store) CheckDocuments(ctx context.Context, space vectorstore.SpaceID, docs []*document.Document) error
- func (s *Store) CheckQuery(ctx context.Context, space vectorstore.SpaceID, opts vectorstore.SearchOptions) error
- func (s *Store) DeleteIDs(ctx context.Context, ids []string) error
- func (s *Store) DeleteMatching(ctx context.Context, predicate filter.Predicate, b vectorstore.ScanBudget) error
- func (s *Store) Query(ctx context.Context, q *vectorstore.Query) (*vectorstore.SearchResponse, error)
- func (s *Store) QueryByScan(ctx context.Context, q *vectorstore.Query, b vectorstore.ScanBudget) (*vectorstore.SearchResponse, error)
- func (s *Store) Upsert(ctx context.Context, b *vectorstore.Batch) error
- type StoreConfig
Examples ¶
Constants ¶
const ( Provider = "Redis" DefaultIndexName = "scope-vector-index" DefaultWriteBatchSize = 256 )
Variables ¶
var ErrIncompatibleIndex = errors.New("redis: native index is incompatible")
Functions ¶
This section is empty.
Types ¶
type RedisClient ¶ added in v0.43.0
type RedisClient interface {
Do(context.Context, ...any) *goredis.Cmd
Pipeline() goredis.Pipeliner
// contains filtered or unexported methods
}
RedisClient supplies host-owned native commands. StoreConfig.ScanNodes owns complete keyspace traversal independently of command-client decorators.
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
func NewStore ¶
func NewStore(ctx context.Context, config StoreConfig) (*Store, error)
Example ¶
package main
import (
"context"
"os"
"time"
goredis "github.com/redis/go-redis/v9"
"github.com/Tangerg/scope/core/document"
"github.com/Tangerg/scope/core/vectorstore"
"github.com/Tangerg/scope/vectorstores/redis"
)
func main() {
ctx, cancel := context.WithTimeout(context.Background(), time.Minute)
defer cancel()
client := goredis.NewClient(&goredis.Options{Addr: os.Getenv("SCOPE_REDIS_ADDR"), Protocol: 3, Username: os.Getenv("SCOPE_REDIS_USERNAME"), Password: os.Getenv("SCOPE_REDIS_PASSWORD"), ContextTimeoutEnabled: true, ReadTimeout: 5 * time.Second, WriteTimeout: 5 * time.Second})
defer client.Close()
const space vectorstore.SpaceID = "paired-v1"
// Provision once through the native host API, before constructing Store.
if err := client.FTCreate(ctx, "documents", &goredis.FTCreateOptions{OnHash: true, Prefix: []any{"documents:" + space.StorageKey() + ":"}}, &goredis.FieldSchema{FieldName: "embedding", FieldType: goredis.SearchFieldTypeVector, VectorArgs: &goredis.FTVectorArgs{FlatOptions: &goredis.FTFlatOptions{Type: "FLOAT32", Dim: 2, DistanceMetric: "COSINE"}}}).Err(); err != nil {
panic(err)
}
store, err := redis.NewStore(ctx, redis.StoreConfig{Client: client, ScanNodes: func(ctx context.Context, visit func(context.Context, *goredis.Client) error) error {
return visit(ctx, client)
}, IndexName: "documents", Space: space})
if err != nil {
panic(err)
}
if err = store.Upsert(ctx, &vectorstore.Batch{Space: space, Records: []vectorstore.Record{{Document: &document.Document{ID: "one", Text: "example"}, Values: []float64{1, 0}}}}); err != nil {
panic(err)
}
if _, err = store.Query(ctx, &vectorstore.Query{Space: space, Values: []float64{1, 0}}); err != nil {
panic(err)
}
if err = store.DeleteIDs(ctx, []string{"one"}); err != nil {
panic(err)
}
}
Output:
func (*Store) CheckDocuments ¶ added in v0.59.0
func (*Store) CheckQuery ¶ added in v0.59.0
func (s *Store) CheckQuery(ctx context.Context, space vectorstore.SpaceID, opts vectorstore.SearchOptions) error
func (*Store) DeleteMatching ¶ added in v0.59.0
func (s *Store) DeleteMatching(ctx context.Context, predicate filter.Predicate, b vectorstore.ScanBudget) error
func (*Store) Query ¶ added in v0.59.0
func (s *Store) Query(ctx context.Context, q *vectorstore.Query) (*vectorstore.SearchResponse, error)
func (*Store) QueryByScan ¶ added in v0.60.0
func (s *Store) QueryByScan(ctx context.Context, q *vectorstore.Query, b vectorstore.ScanBudget) (*vectorstore.SearchResponse, error)
type StoreConfig ¶
type StoreConfig struct {
Client RedisClient
IndexName string
Space vectorstore.SpaceID
// ScanNodes visits every authoritative node in Client's keyspace and fails
// if any node is unavailable. ClusterClient.ForEachMaster satisfies this
// contract; Ring.ForEachShard does not, because it can skip failed shards.
// The host binds commands and traversal to the same logical Redis store.
ScanNodes func(context.Context, func(context.Context, *goredis.Client) error) error
// WriteBatchSize bounds commands submitted in one nontransactional pipeline.
WriteBatchSize int
}
func (StoreConfig) Validate ¶
func (s StoreConfig) Validate() error
Source Files