Documentation
¶
Overview ¶
Package cache provides a caching decorator around an app.Generator: it serves a previously synthesized answer from an app.AnswerCache when the same question is asked over the same grounding, skipping the LLM round-trip. It is wired in the composition root only when caching is enabled.
Index ¶
- type Generator
- func (g *Generator) Synthesize(ctx context.Context, question string, hits []domain.ChunkHit, ...) (app.Answer, error)
- func (g *Generator) SynthesizeDeterministic(ctx context.Context, question string, hits []domain.ChunkHit, ...) (app.Answer, error)
- func (g *Generator) SynthesizeStream(ctx context.Context, question string, hits []domain.ChunkHit, ...) (app.Answer, error)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Generator ¶
type Generator struct {
// contains filtered or unexported fields
}
Generator wraps an inner app.Generator with a persistent answer cache. The key is a hash of the salt (model + prompt identity, supplied by the composition root), the question, and the ordered grounding chunks INCLUDING their text — because chunk IDs are path-stable (a re-ingested document keeps the same IDs with new text), keying on IDs alone would serve stale answers. Requests with attachments are never cached (ephemeral, and rarely repeated). Cache errors are best-effort: they never fail the answer.
func NewGenerator ¶
func NewGenerator(inner app.Generator, store app.AnswerCache, salt string, ttl time.Duration, now func() time.Time) *Generator
NewGenerator wraps inner so its answers are cached in store. salt scopes the cache to a model/prompt identity; ttl bounds how long an entry stays fresh; now supplies the clock (time.Now in production, a fake in tests).
func (*Generator) Synthesize ¶
func (g *Generator) Synthesize(ctx context.Context, question string, hits []domain.ChunkHit, attachments []domain.Attachment) (app.Answer, error)
Synthesize returns a cached answer when one exists for the same question and grounding within the TTL; otherwise it delegates to the inner generator and caches the result.
func (*Generator) SynthesizeDeterministic ¶ added in v1.0.0
func (g *Generator) SynthesizeDeterministic(ctx context.Context, question string, hits []domain.ChunkHit, attachments []domain.Attachment) (app.Answer, error)
SynthesizeDeterministic forwards to the inner generator's reproducible path and BYPASSES the cache entirely: an audited run must get a freshly pinned (temperature 0 + seed) generation, never a possibly-non-deterministic cached answer, and must not write one back under the normal key. When the inner generator cannot pin generation the determinism guarantee cannot be met, so it returns ErrReproducibleUnsupported. Satisfies app.DeterministicGenerator.
func (*Generator) SynthesizeStream ¶
func (g *Generator) SynthesizeStream(ctx context.Context, question string, hits []domain.ChunkHit, attachments []domain.Attachment, onDelta func(string)) (app.Answer, error)
SynthesizeStream serves a cached answer in one whole-text delta when one exists for the same question and grounding within the TTL (a hit is instant — nothing to stream); otherwise it streams from the inner generator and caches the result. Attachment-bearing requests bypass the cache, like Synthesize.
Source Files