llmcache

package
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 7, 2026 License: MIT Imports: 9 Imported by: 0

Documentation

Overview

Package llmcache is a content-addressed, on-disk cache for LLM review responses. Keys hash provider+model+prompt so a model or prompt change never produces a false hit; entries expire by TTL and the store is safe for concurrent writers (temp file + atomic rename). It depends only on the standard library.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Key

func Key(provider, model, system, user string) string

Key hashes provider, model, system and user prompts into a hex SHA-256 key.

Types

type Cache

type Cache struct {
	// contains filtered or unexported fields
}

Cache stores LLM responses on disk under dir, expiring entries older than ttl.

func New

func New(ttl time.Duration) (*Cache, error)

New creates a Cache under os.UserCacheDir()/gitl/review. It returns an error when the user cache dir cannot be resolved; callers degrade to "no cache" rather than crashing.

func NewInDir

func NewInDir(dir string, ttl time.Duration) *Cache

NewInDir creates a Cache rooted at an explicit directory (for tests).

func (*Cache) Get

func (c *Cache) Get(key string) (llm.Response, bool, error)

Get returns the cached response for key. It reports (zero, false, nil) on a miss (including an expired entry, which it best-effort deletes) and (zero, false, err) when an existing entry exists but cannot be read or parsed.

func (*Cache) Put

func (c *Cache) Put(key string, resp llm.Response) error

Put atomically writes resp to the sharded path for key. It creates the shard directory, writes to a uniquely-named temp file in the same directory, then renames it into place so concurrent writers never observe a partial file.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL