README
¶
Cache Embedder for Eino
This module provides a cache embedder for Eino, which is designed to store and retrieve embeddings efficiently. The cache embedder can be used to speed up the embedding process by caching previously computed embeddings.
Installation
go get github.com/cloudwego/eino-ext/components/embedding/cache
Usage
package main
import (
"context"
"crypto/md5"
"log"
"github.com/cloudwego/eino-ext/components/embedding/cache"
cacheredis "github.com/cloudwego/eino-ext/components/embedding/cache/redis"
"github.com/cloudwego/eino/components/embedding"
"github.com/redis/go-redis/v9"
)
func main() {
rdb := redis.NewClient(&redis.Options{
Addr: "localhost:6379",
})
// the original embedder, you can replace it with any other embedder implementation
// It's only a example, you need to bring a real embedder implementation here.
var originalEmbedder embedding.Embedder
// embedder, err := openai.NewEmbedder(ctx, &openai.EmbeddingConfig{
// APIKey: accessKey,
// }
// ...
embedder, err := cache.NewEmbedder(originalEmbedder,
cache.WithCacher(cacheredis.NewCacher(rdb)), // using Redis as the cache
cache.WithGenerator(cache.NewHashGenerator(md5.New())), // using md5 for generating unique keys
)
if err != nil {
log.Fatal(err)
}
embeddings, err := embedder.EmbedStrings(context.Background(), []string{"hello", "how are you"})
if err != nil {
log.Fatal(err)
}
log.Printf("embeddings: %v", embeddings)
}
Features
- Cache: The cache embedder stores embeddings in a cache to avoid recomputing them for the same input.
- Cacher: The cache embedder supports different caching backends, such as Redis.
- Currently, Redis is supported.
- Generator: The cache embedder uses a generator to create unique keys for caching embeddings.
- Currently, a simple generator and a hash generator base on hash.Hash interface are supported.
Examples
See the examples directory for complete usage examples.
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ( ErrCacherRequired = errors.New("embedding/cache: cacher is required") ErrGeneratorRequired = errors.New("embedding/cache: generator is required") )
Functions ¶
This section is empty.
Types ¶
type Cacher ¶
type Cacher interface {
// Set stores the value in the cache with the given key.
// If the key already exists, it will be overwritten.
Set(ctx context.Context, key string, value []float64, expire time.Duration) error
// Get retrieves the value from the cache with the given key.
// If the key does not exist, the bool return value is false，otherwise it returns true
// If the value is not of type []float64, it returns an error.
Get(ctx context.Context, key string) ([]float64, bool, error)
}
type Embedder ¶
type Embedder struct {
// contains filtered or unexported fields
}
func NewEmbedder ¶
NewEmbedder creates a new Embedder instance with cache support.
type Generator ¶
type Generator interface {
Generate(ctx context.Context, text string, opt GeneratorOption) string
}
Generator is an interface for generating unique keys based on text and optional embedding options. It is used to create cache keys for embedding results.
type GeneratorOption ¶
type GeneratorOption struct {
Model string
}
GeneratorOption holds options for generating unique keys.
type HashGenerator ¶
type HashGenerator struct {
*SimpleGenerator
// contains filtered or unexported fields
}
HashGenerator is a concrete implementation of the Generator interface that uses a hash function to generate a unique key based on the provided text and optional embedding options. It wraps a SimpleGenerator and applies a hash function to the generated key.
Note: Because of the use of the hash.Hash algorithm, there is a probability that data with different text and options will generate the same key. This is a trade-off between uniqueness and performance. If you need guaranteed uniqueness, consider using a different generator or a more complex hashing strategy.
func NewHashGenerator ¶
func NewHashGenerator(hasher hash.Hash) *HashGenerator
NewHashGenerator creates a new HashGenerator with the specified hash function.
func (*HashGenerator) Generate ¶
func (g *HashGenerator) Generate(ctx context.Context, text string, opt GeneratorOption) string
type Option ¶
type Option interface {
// contains filtered or unexported methods
}
func WithCacher ¶
WithCacher returns an Option that sets the Cacher for the Embedder.
func WithExpiration ¶
WithExpiration returns an Option that sets the expiration duration for cached embeddings in the Embedder.
type SimpleGenerator ¶
type SimpleGenerator struct{}
SimpleGenerator is a concrete implementation of the Generator interface that generates a simple key by concatenating the text and model without hashing.
func NewSimpleGenerator ¶
func NewSimpleGenerator() *SimpleGenerator
NewSimpleGenerator creates a new SimpleGenerator instance.
func (*SimpleGenerator) Generate ¶
func (g *SimpleGenerator) Generate(_ context.Context, text string, opt GeneratorOption) string
Source Files
Directories