cache

package module

v0.1.0 Latest Latest Go to latest Published: Feb 19, 2025 License: MIT Imports: 5 Imported by: 0

Details

View Source

const Version = "cache/v0.1.0"

This section is empty.

This section is empty.

type Cache interface {
	Getter
	Putter
}

Cache interface

type Client struct {
	chatter.Chatter
	// contains filtered or unexported fields
}

func New(cache Cache, chatter chatter.Chatter) *Client

Creates caching layer for LLM client.

Use github.com/akrylysov/pogreb to cache chatter on local file systems:

llm, err := /* create LLM client */
db, err := pogreb.Open("llm.cache", nil)
text := cache.New(db, llm)

func (c *Client) HashKey(prompt string) []byte

func (c *Client) Prompt(ctx context.Context, prompt []fmt.Stringer, opts ...chatter.Opt) (chatter.Reply, error)

type Getter interface{ Get([]byte) ([]byte, error) }

Getter interface abstract storage

type Putter interface{ Put([]byte, []byte) error }

Setter interface abstract storage