llm

package
v0.4.21 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 28, 2026 License: AGPL-3.0 Imports: 12 Imported by: 0

Documentation

Overview

Package llm holds the opt-in consolidation pipeline: on each write it decides whether a new memory is novel, a refinement, or a contradiction that supersedes an existing one. Without an LLM the service stores raw.

Two backends implement the same surface: an OpenAI-compatible chat client (openai-go) and an Anthropic Messages client (anthropic-sdk-go). The Anthropic backend caches the static system prompt, which makes high-write consolidation cheaper against providers like MiniMax that support prompt caching on the Anthropic-compatible API.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type API

type API string

API selects the chat backend.

const (
	// APIOpenAI talks to an OpenAI-compatible /chat/completions endpoint.
	APIOpenAI API = "openai"
	// APIAnthropic talks to an Anthropic Messages endpoint (with prompt caching).
	APIAnthropic API = "anthropic"
)

type Action

type Action string

Action is the consolidation decision for a new memory.

const (
	// ActionNew stores the new memory as a distinct record.
	ActionNew Action = "new"
	// ActionUpdate merges the new memory into an existing one (Target), which is
	// rewritten with Content; no new record is created.
	ActionUpdate Action = "update"
	// ActionSupersede stores the new memory and tombstones Target as superseded.
	ActionSupersede Action = "supersede"
)

type AnthropicClient

type AnthropicClient struct {
	// contains filtered or unexported fields
}

AnthropicClient is a Client backed by an Anthropic Messages endpoint. It works against the real Anthropic API and against Anthropic-compatible providers (e.g. MiniMax) via a BaseURL override. The static system prompt is marked for prompt caching, so repeated consolidation calls reuse it at the cache-read rate.

func NewAnthropic

func NewAnthropic(cfg Config) (*AnthropicClient, error)

NewAnthropic builds an Anthropic Messages client. Model is required; BaseURL is optional (defaults to the Anthropic API, override it for compatible providers).

func (*AnthropicClient) Complete

func (c *AnthropicClient) Complete(ctx context.Context, system, user string) (string, error)

Complete is a single-turn message returning the concatenated text blocks.

func (*AnthropicClient) Consolidate

func (c *AnthropicClient) Consolidate(ctx context.Context, in Input) (Decision, error)

Consolidate asks the model how the new memory relates to the candidates.

func (*AnthropicClient) Distill

func (c *AnthropicClient) Distill(ctx context.Context, in DistillInput) ([]Fact, error)

Distill compresses episodic memories into durable semantic facts.

type AnthropicConfig

type AnthropicConfig = Config

OpenAIConfig and AnthropicConfig are aliases kept for call-site clarity.

type Candidate

type Candidate struct {
	ID      string `json:"id"`
	Content string `json:"content"`
}

Candidate is an existing memory offered to the consolidator for comparison.

type Client

type Client interface {
	Consolidator
	Completer
	Distiller
}

Client is a chat backend that can consolidate memories, distill facts, and answer single-turn prompts.

func New

func New(api API, cfg Config) (Client, error)

New builds a chat client for the given API ("openai" default, or "anthropic").

type Completer

type Completer interface {
	Complete(ctx context.Context, system, user string) (string, error)
}

Completer is a single-turn chat completion (used by the benchmark harness).

type Config

type Config struct {
	BaseURL string // OpenAI: e.g. https://host/v1 ; Anthropic: e.g. https://api.minimax.io/anthropic
	APIKey  string
	Model   string
	// MaxTokens caps the completion length (defaults to defaultMaxTokens). A
	// budget is required for reasoning models, which otherwise spend the server
	// default on hidden reasoning and return empty content.
	MaxTokens  int
	HTTPClient *http.Client
}

Config configures a chat client. The same fields apply to both the OpenAI-compatible and Anthropic backends.

type Consolidator

type Consolidator interface {
	Consolidate(ctx context.Context, in Input) (Decision, error)
}

Consolidator decides how a new memory relates to existing candidates.

type Decision

type Decision struct {
	Action Action `json:"action"`
	// Target is the candidate ID affected by update/supersede (empty for new).
	Target string `json:"target"`
	// Content is the merged text to persist for an update (else the new content).
	Content string `json:"content"`
	// Summary is an optional one-line summary of the resulting memory.
	Summary string `json:"summary"`
	// Reason explains the decision (for logs/debugging).
	Reason string `json:"reason"`
}

Decision is the consolidator's verdict.

type DistillInput

type DistillInput struct {
	Episodes []Episode `json:"episodes"`
	Now      string    `json:"now,omitempty"`
}

DistillInput is a batch of episodic memories to distill. Now is the current date (YYYY-MM-DD); the model grounds relative dates against each episode's Date, falling back to Now. Both empty disables grounding.

type Distiller

type Distiller interface {
	Distill(ctx context.Context, in DistillInput) ([]Fact, error)
}

Distiller compresses episodic memories into durable semantic facts. Used by the episodic→semantic promotion job.

type Episode added in v0.4.19

type Episode struct {
	Content string `json:"content"`
	// Date is the YYYY-MM-DD the episode was recorded. Empty when unknown.
	Date string `json:"date,omitempty"`
}

Episode is one episodic memory to distill, paired with the date it was recorded so the model can resolve relative dates in the text ("yesterday", "last week") to absolute ones.

type Fact

type Fact struct {
	Content string `json:"content"`
	Summary string `json:"summary,omitempty"`
	// Category routes the fact to a tier: "procedure" (incl. error→recovery) →
	// procedural; "preference" and "fact" → semantic. Empty defaults to semantic.
	Category string `json:"category,omitempty"`
}

Fact is a durable memory distilled from episodic observations.

type Input

type Input struct {
	New        string      `json:"new"`
	Tier       string      `json:"tier"`
	Candidates []Candidate `json:"candidates"`
}

Input is the consolidation request.

type OpenAIClient

type OpenAIClient struct {
	// contains filtered or unexported fields
}

OpenAIClient is a Client backed by an OpenAI-compatible /chat/completions endpoint.

func NewOpenAI

func NewOpenAI(cfg Config) (*OpenAIClient, error)

NewOpenAI builds a chat client. BaseURL and Model are required.

func (*OpenAIClient) Complete

func (c *OpenAIClient) Complete(ctx context.Context, system, user string) (string, error)

Complete is a single-turn chat completion returning the assistant message text.

func (*OpenAIClient) Consolidate

func (c *OpenAIClient) Consolidate(ctx context.Context, in Input) (Decision, error)

Consolidate asks the model how the new memory relates to the candidates.

func (*OpenAIClient) Distill

func (c *OpenAIClient) Distill(ctx context.Context, in DistillInput) ([]Fact, error)

Distill compresses episodic memories into durable semantic facts.

type OpenAIConfig

type OpenAIConfig = Config

OpenAIConfig and AnthropicConfig are aliases kept for call-site clarity.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL