README
¶
AI Cache Service (ai/cache)
cache 包提供了两层缓存机制，旨在优化 AI Agent 的响应速度并降低 API 调用成本。它包含通用的内存 LRU 缓存和专用的语义缓存 (Semantic Cache)。
架构设计
classDiagram
class CacheService {
<<interface>>
+Get(ctx, key)
+Set(ctx, key, val, ttl)
+Invalidate(ctx, pattern)
}
class LRUCache~K,V~ {
-cache map[K]*entry
-order *list.List
+Get(key)
+Set(key, val, ttl)
}
class SemanticCache {
-exactCache *ByteLRUCache
-semanticCache map
-stats SemanticCacheStats
+Get(text)
+Set(text, embedding)
}
CacheService <|.. LRUCache : implements
SemanticCache --* LRUCache : uses (L1)
1. LRU Cache (LRUCache[K, V])
- 通用性: 使用 Go 泛型实现，支持任意类型的键值对。
- 并发安全: 使用
sync.RWMutex保证并发访问安全，采用“读写分离锁”和“双重检查锁定”策略优化性能。 - TTL 支持: 每个缓存项都包含过期时间，这就支持了自动过期清理。
2. Semantic Cache (SemanticCache)
语义缓存实现了两级缓存策略，用于检索相似的文本 Embedding，减少重复的 Embedding 计算。
- Level 1: 精确匹配 (Exact Match)
- 使用 SHA256 对输入文本进行哈希。
- 基于
ByteLRUCache存储，速度极快。
- Level 2: 语义匹配 (Semantic Match)
- 存储文本的 Vector Embedding。
- 通过计算 Cosine Similarity (余弦相似度) 查找相似的已缓存条目。
算法设计
语义匹配流程
flowchart TD
Start([输入文本]) --> L1{L1: 精确匹配?}
L1 -- Hit (SHA256) --> Return[返回缓存 Embedding]
L1 -- Miss --> Embed[调用 EmbeddingService 生成向量]
Embed --> L2{L2: 语义匹配?}
L2 -- Hit (Similarity > 0.95) --> Return
L2 -- Miss --> Store[存入 L1 & L2 缓存]
Store --> Return
- 输入: 待 Embedding 的文本。
- L1 查找: 计算 SHA256 哈希，查表。命中则直接返回缓存的 Embedding (相似度 1.0)。
- L2 查找:
- 调用
EmbeddingService生成当前文本的 Embedding 向量。 - 遍历语义缓存中的所有有效条目。
- 计算当前向量与缓存条目向量的 余弦相似度 (Cosine Similarity).
- 记录最大相似度
max_sim和对应的条目。
- 调用
- 阈值判定: 如果
max_sim>SimilarityThreshold(默认 0.95)，则视为语义命中，返回缓存结果。 - 更新: 如果都未命中，将新生成的 Embedding 存入 L1 和 L2 缓存。
业务流程
- 对于上层服务 (如 Team B/C): 调用
Get(text)。 - 内部处理:
SemanticCache透明地处理精确匹配和模糊语义匹配，上层无需感知底层的向量计算细节，只需关注是否拿到了可用的 Embedding。
特定配置
- MaxEntries:最大缓存条目数。
- SimilarityThreshold: 语义相似度阈值 (0-1)。
- TTL: 缓存有效期。
Documentation
¶
Overview ¶
Package cache provides the cache service interface for AI agents. This interface is consumed by Team B (Assistant+Schedule) and Team C (Memo Enhancement).
Package cache provides semantic caching for AI agents. Issue #91: 语义缓存层实现 - 基于 Embedding 相似度匹配
Index ¶
- type ByteLRUCache
- type CacheService
- type EmbeddingService
- type LRUCache
- func (c *LRUCache[K, V]) Capacity() int
- func (c *LRUCache[K, V]) CleanupExpired() int
- func (c *LRUCache[K, V]) Clear()
- func (c *LRUCache[K, V]) Contains(key K) bool
- func (c *LRUCache[K, V]) Get(key K) (V, bool)
- func (c *LRUCache[K, V]) Invalidate(pattern string) int
- func (c *LRUCache[K, V]) Remove(key K) bool
- func (c *LRUCache[K, V]) Set(key K, value V, ttl time.Duration)
- func (c *LRUCache[K, V]) SetWithDefaultTTL(key K, value V)
- func (c *LRUCache[K, V]) Size() int
- type MockCacheService
- func (m *MockCacheService) Clear()
- func (m *MockCacheService) Get(ctx context.Context, key string) ([]byte, bool)
- func (m *MockCacheService) Invalidate(ctx context.Context, pattern string) error
- func (m *MockCacheService) Set(ctx context.Context, key string, value []byte, ttl time.Duration) error
- func (m *MockCacheService) Size() int
- type SemanticCache
- type SemanticCacheConfig
- type SemanticCacheStats
- type Service
- func (s *Service) Clear()
- func (s *Service) Close()
- func (s *Service) Get(_ context.Context, key string) ([]byte, bool)
- func (s *Service) Invalidate(_ context.Context, pattern string) error
- func (s *Service) Set(_ context.Context, key string, value []byte, ttl time.Duration) error
- func (s *Service) Size() int
- type ServiceConfig
- type StringLRUCache
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type ByteLRUCache ¶ added in v0.100.0
ByteLRUCache is a type alias for LRUCache with string keys and []byte values. Provided for backward compatibility with existing code that uses []byte values.
func NewByteLRUCache ¶ added in v0.100.0
func NewByteLRUCache(capacity int, defaultTTL time.Duration) *ByteLRUCache
NewByteLRUCache creates a new LRU cache with string keys and []byte values. This is a convenience function for the common case of caching byte slices.
type CacheService ¶
type CacheService interface {
// Get retrieves a value from cache.
// Returns: value, whether it exists
Get(ctx context.Context, key string) ([]byte, bool)
// Set stores a value in cache.
// ttl: expiration time
Set(ctx context.Context, key string, value []byte, ttl time.Duration) error
// Invalidate invalidates cache entries.
// pattern: supports wildcards (user:123:*)
Invalidate(ctx context.Context, pattern string) error
}
Consumers: Team B (Assistant+Schedule), Team C (Memo Enhancement).
type EmbeddingService ¶ added in v0.94.0
type EmbeddingService interface {
// Embed generates a vector embedding for the given text.
Embed(ctx context.Context, text string) ([]float32, error)
// EmbedBatch generates vector embeddings for multiple texts.
EmbedBatch(ctx context.Context, texts []string) ([][]float32, error)
}
EmbeddingService defines the interface for generating vector embeddings. This is a local interface to avoid circular dependencies.
type LRUCache ¶
type LRUCache[K comparable, V any] struct { // contains filtered or unexported fields }
LRUCache implements an LRU cache with TTL support and generics. LRUCache 实现支持 TTL 和泛型的 LRU 缓存。
func NewLRUCache ¶
NewLRUCache creates a new LRU cache.
func (*LRUCache[K, V]) Capacity ¶ added in v0.94.0
Capacity returns the maximum capacity of the cache.
func (*LRUCache[K, V]) CleanupExpired ¶
CleanupExpired removes all expired entries. Returns the number of entries removed.
func (*LRUCache[K, V]) Clear ¶
func (c *LRUCache[K, V]) Clear()
Clear removes all entries from the cache.
func (*LRUCache[K, V]) Contains ¶ added in v0.100.0
Contains checks if a key exists in the cache (without updating access order).
IMPORTANT: Unlike Get, this method does NOT remove expired entries. It only checks if the key exists AND has not expired. This means Contains() may return true while a subsequent Get() returns false (if the entry expired between the two calls).
This "read-only" semantics is intentional for performance. If you need consistent behavior with Get, call Get instead.
Example:
// DON'T: Check then Get (race condition possible)
if cache.Contains(key) {
val, ok := cache.Get(key) // ok may be false!
}
// DO: Just call Get directly
if val, ok := cache.Get(key); ok {
// use val
}
func (*LRUCache[K, V]) Get ¶
Get retrieves a value from the cache. Uses a two-phase locking strategy: RLock for read, upgrade to Lock only if modification needed.
func (*LRUCache[K, V]) Invalidate ¶
Invalidate removes entries matching the pattern. Supports * wildcard at the end (e.g., "user:123:*"). Note: This method only works for string keys. For other key types, use Remove.
func (*LRUCache[K, V]) SetWithDefaultTTL ¶ added in v0.100.0
func (c *LRUCache[K, V]) SetWithDefaultTTL(key K, value V)
SetWithDefaultTTL stores a value using the default TTL.
type MockCacheService ¶
type MockCacheService struct {
// contains filtered or unexported fields
}
MockCacheService is a mock implementation of CacheService for testing.
func NewMockCacheService ¶
func NewMockCacheService() *MockCacheService
NewMockCacheService creates a new MockCacheService.
func (*MockCacheService) Clear ¶
func (m *MockCacheService) Clear()
Clear removes all items from the cache (for testing).
func (*MockCacheService) Invalidate ¶
func (m *MockCacheService) Invalidate(ctx context.Context, pattern string) error
Invalidate invalidates cache entries.
func (*MockCacheService) Set ¶
func (m *MockCacheService) Set(ctx context.Context, key string, value []byte, ttl time.Duration) error
Set stores a value in cache.
func (*MockCacheService) Size ¶
func (m *MockCacheService) Size() int
Size returns the number of items in the cache (for testing).
type SemanticCache ¶ added in v0.94.0
type SemanticCache struct {
// contains filtered or unexported fields
}
SemanticCache provides two-layer caching: exact match (SHA256) and semantic match (cosine similarity).
func NewSemanticCache ¶ added in v0.94.0
func NewSemanticCache(cfg SemanticCacheConfig) *SemanticCache
NewSemanticCache creates a new semantic cache.
func (*SemanticCache) Clear ¶ added in v0.94.0
func (c *SemanticCache) Clear()
Clear clears all cache entries.
func (*SemanticCache) Get ¶ added in v0.94.0
Get retrieves a cached embedding. Returns (embedding, found, similarity, isExactMatch).
func (*SemanticCache) GetStats ¶ added in v0.94.0
func (c *SemanticCache) GetStats() SemanticCacheStats
GetStats returns the cache statistics.
type SemanticCacheConfig ¶ added in v0.94.0
type SemanticCacheConfig struct {
// MaxEntries is the maximum number of entries in the cache.
MaxEntries int
// SimilarityThreshold is the minimum cosine similarity for a match (0-1).
SimilarityThreshold float32
// TTL is the time-to-live for cache entries.
TTL time.Duration
// EmbeddingService is the vector embedding service.
EmbeddingService EmbeddingService
}
SemanticCacheConfig configures the semantic cache.
func DefaultSemanticCacheConfig ¶ added in v0.94.0
func DefaultSemanticCacheConfig() SemanticCacheConfig
DefaultSemanticCacheConfig returns the default configuration.
type SemanticCacheStats ¶ added in v0.94.0
type SemanticCacheStats struct {
ExactHits int64
ExactMisses int64
SemanticHits int64
SemanticMisses int64
SemanticSize int
SimilarityDistribution map[string]int64
}
SemanticCacheStats represents cache statistics.
func (SemanticCacheStats) MarshalJSON ¶ added in v0.94.0
func (s SemanticCacheStats) MarshalJSON() ([]byte, error)
MarshalJSON implements json.Marshaler for stats.
type Service ¶
type Service struct {
// contains filtered or unexported fields
}
Service implements CacheService with LRU eviction.
func NewService ¶
func NewService(cfg ServiceConfig) *Service
NewService creates a new cache service.
func (*Service) Invalidate ¶
Invalidate invalidates cache entries matching the pattern.
type ServiceConfig ¶
type ServiceConfig struct {
Capacity int // Maximum number of entries (default: 1000)
DefaultTTL time.Duration // Default TTL for entries (default: 5 minutes)
CleanupInterval time.Duration // Interval for expired entry cleanup (default: 1 minute)
}
ServiceConfig configures the cache service.
func DefaultServiceConfig ¶
func DefaultServiceConfig() ServiceConfig
DefaultServiceConfig returns default cache service configuration.
type StringLRUCache ¶ added in v0.100.0
StringLRUCache is a type alias for LRUCache with string keys and string values.
func NewStringLRUCache ¶ added in v0.100.0
func NewStringLRUCache(capacity int, defaultTTL time.Duration) *StringLRUCache
NewStringLRUCache creates a new LRU cache with string keys and string values.
Source Files