Documentation
¶
Index ¶
- func CacheKeyFromBody(model string, body []byte, userGroup string) string
- func Init(cfg *Config)
- func Lookup(ctx context.Context, model string, body []byte, userGroup string) ([]byte, bool)
- func Store(ctx context.Context, model string, body []byte, response []byte, ...)
- type CacheStats
- type Config
- type SemanticCacheV2
- func (c *SemanticCacheV2) GetStats() map[string]interface{}
- func (c *SemanticCacheV2) Lookup(ctx context.Context, model string, body []byte, userGroup string) ([]byte, bool)
- func (c *SemanticCacheV2) Stop()
- func (c *SemanticCacheV2) Store(ctx context.Context, model string, body []byte, response []byte, ...) error
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func CacheKeyFromBody ¶
CacheKeyFromBody generates cache key from model, request body and user group
func Init ¶
func Init(cfg *Config)
Init initializes the global semantic cache instance with the given config. If cfg is nil, the default config will be used. Uses its own once so it always takes effect regardless of GetSemanticCacheV2 call order.
Types ¶
type CacheStats ¶
type CacheStats struct {
Hits int64
Misses int64
L1Hits int64
L2Hits int64
// contains filtered or unexported fields
}
CacheStats tracks cache performance
type Config ¶
type Config struct {
Enabled bool
SimilarityThreshold float32
TTL int64
MaxEntries int64
// BypassModels lists models that should skip semantic cache
BypassModels map[string]bool
// CacheOnlyModels if non-empty, ONLY these models use semantic cache
CacheOnlyModels map[string]bool
}
Config holds semantic cache configuration
type SemanticCacheV2 ¶
type SemanticCacheV2 struct {
// contains filtered or unexported fields
}
SemanticCacheV2 provides hybrid semantic caching L1: Redis exact match (fast) L2: Vector similarity search (accurate)
func GetSemanticCacheV2 ¶
func GetSemanticCacheV2() *SemanticCacheV2
GetSemanticCacheV2 returns the default hybrid cache instance
func NewSemanticCacheV2 ¶
func NewSemanticCacheV2(cfg *Config, redisClient *redis.Client) *SemanticCacheV2
NewSemanticCacheV2 creates a new hybrid semantic cache
func (*SemanticCacheV2) GetStats ¶
func (c *SemanticCacheV2) GetStats() map[string]interface{}
GetStats returns cache statistics
func (*SemanticCacheV2) Lookup ¶
func (c *SemanticCacheV2) Lookup(ctx context.Context, model string, body []byte, userGroup string) ([]byte, bool)
Lookup checks both L1 and L2 cache
func (*SemanticCacheV2) Stop ¶
func (c *SemanticCacheV2) Stop()
Stop gracefully stops background goroutines.
Source Files