Documentation
¶
Overview ¶
Package semcache implements a small, durable, on-disk semantic cache for LLM-generated payloads (roadmap P2-27).
The cache is keyed by SHA-256 of the *exact* inputs that produce the generated bytes — task label, provider name (which transitively captures the model), the system prompt, and the serialised message list. A second call with the same inputs returns the prior output without re-billing the LLM. Entries are stored under `~/.promptzero/cache/generations/` (one JSON file per entry).
Eviction is LRU on entry count: when the cache exceeds Capacity, the oldest LastAccessed entries are deleted. Operators who want a clean slate can `rm -rf ~/.promptzero/cache/generations` — there is no in-process state besides a sync.Mutex.
Bypass is by construction: callers who don't want a cache pass nil for the *Cache, and the integration site short-circuits.
Index ¶
Constants ¶
const DefaultCapacity = 256
DefaultCapacity is the LRU bound enforced by [Evict] when callers don't override it. 256 entries × ~32 KiB-per-Result ≈ 8 MiB on disk in the worst case — safe for the small SD-card-sized files the generate package produces.
Variables ¶
This section is empty.
Functions ¶
func DefaultRoot ¶
DefaultRoot returns ~/.promptzero/cache/generations. Mirrors the snapshot package's default so operators recognise the layout.
Types ¶
type Cache ¶
type Cache struct {
// contains filtered or unexported fields
}
Cache stores generated payloads keyed by a content hash. A nil *Cache is a "no cache" sentinel — Get always reports miss, Put is a no-op. Construct with New.
func New ¶
New constructs a Cache rooted at the given directory. The root is created lazily on the first Cache.Put call so passing a non- existent dir is not an error (matches the snapshot package's convention). Capacity ≤ 0 falls back to DefaultCapacity.
func (*Cache) Clear ¶
Clear removes the entire cache directory. Useful for `--clear-cache` CLI options or test cleanup. A nil *Cache returns nil — clearing a disabled cache is a no-op, not an error.
func (*Cache) Get ¶
Get returns the cached entry for key, or false on miss. On a hit, LastAccessed is updated and Hits incremented before the entry is re-serialised — this drives the LRU policy. Errors reading or re-writing the entry are swallowed and reported as a miss; the cache must never block the generation hot path.
Safe to call with a nil *Cache (always returns miss). Concurrency- safe via the cache's internal mutex.
func (*Cache) Put ¶
Put stores entry under key. Creates the cache directory on first use. After a successful write, [Cache.evictLocked] runs to honour the LRU bound.
A nil *Cache makes Put a no-op (so generators can unconditionally call into the cache without a "if c != nil" wrapper). Errors are surfaced — unlike Get, Put failures should be visible at the call site so operators see when a cache directory is read-only.
type Entry ¶
type Entry struct {
Key string `json:"key"`
Task string `json:"task"`
Provider string `json:"provider"`
Content string `json:"content"`
Created time.Time `json:"created"`
LastAccessed time.Time `json:"last_accessed"`
Hits int `json:"hits"`
}
Entry is one cached generation. The fields are persisted verbatim as JSON; Hits / LastAccessed are mutated on Get to drive LRU eviction.
Source Files