llm

package
v0.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 10, 2026 License: AGPL-3.0 Imports: 11 Imported by: 0

Documentation

Overview

Package llm holds the opt-in consolidation pipeline: on each write it decides whether a new memory is novel, a refinement, or a contradiction that supersedes an existing one. Without an LLM the service stores raw.

Two backends implement the same surface: an OpenAI-compatible chat client (openai-go) and an Anthropic Messages client (anthropic-sdk-go). The Anthropic backend caches the static system prompt, which makes high-write consolidation cheaper against providers like MiniMax that support prompt caching on the Anthropic-compatible API.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type API

type API string

API selects the chat backend.

const (
	// APIOpenAI talks to an OpenAI-compatible /chat/completions endpoint.
	APIOpenAI API = "openai"
	// APIAnthropic talks to an Anthropic Messages endpoint (with prompt caching).
	APIAnthropic API = "anthropic"
)

type Action

type Action string

Action is the consolidation decision for a new memory.

const (
	// ActionNew stores the new memory as a distinct record.
	ActionNew Action = "new"
	// ActionUpdate merges the new memory into an existing one (Target), which is
	// rewritten with Content; no new record is created.
	ActionUpdate Action = "update"
	// ActionSupersede stores the new memory and tombstones Target as superseded.
	ActionSupersede Action = "supersede"
)

type AnthropicClient

type AnthropicClient struct {
	// contains filtered or unexported fields
}

AnthropicClient is a Client backed by an Anthropic Messages endpoint. It works against the real Anthropic API and against Anthropic-compatible providers (e.g. MiniMax) via a BaseURL override. The static system prompt is marked for prompt caching, so repeated consolidation calls reuse it at the cache-read rate.

func NewAnthropic

func NewAnthropic(cfg Config) (*AnthropicClient, error)

NewAnthropic builds an Anthropic Messages client. Model is required; BaseURL is optional (defaults to the Anthropic API, override it for compatible providers).

func (*AnthropicClient) Complete

func (c *AnthropicClient) Complete(ctx context.Context, system, user string) (string, error)

Complete is a single-turn message returning the concatenated text blocks.

func (*AnthropicClient) Consolidate

func (c *AnthropicClient) Consolidate(ctx context.Context, in Input) (Decision, error)

Consolidate asks the model how the new memory relates to the candidates.

func (*AnthropicClient) Distill

func (c *AnthropicClient) Distill(ctx context.Context, in DistillInput) ([]Fact, error)

Distill compresses episodic memories into durable semantic facts.

type AnthropicConfig

type AnthropicConfig = Config

OpenAIConfig and AnthropicConfig are aliases kept for call-site clarity.

type Candidate

type Candidate struct {
	ID      string `json:"id"`
	Content string `json:"content"`
}

Candidate is an existing memory offered to the consolidator for comparison.

type Client

type Client interface {
	Consolidator
	Completer
	Distiller
}

Client is a chat backend that can consolidate memories, distill facts, and answer single-turn prompts.

func New

func New(api API, cfg Config) (Client, error)

New builds a chat client for the given API ("openai" default, or "anthropic").

type Completer

type Completer interface {
	Complete(ctx context.Context, system, user string) (string, error)
}

Completer is a single-turn chat completion (used by the benchmark harness).

type Config

type Config struct {
	BaseURL string // OpenAI: e.g. https://host/v1 ; Anthropic: e.g. https://api.minimax.io/anthropic
	APIKey  string
	Model   string
	// MaxTokens caps the completion length (defaults to defaultMaxTokens). A
	// budget is required for reasoning models, which otherwise spend the server
	// default on hidden reasoning and return empty content.
	MaxTokens  int
	HTTPClient *http.Client
}

Config configures a chat client. The same fields apply to both the OpenAI-compatible and Anthropic backends.

type Consolidator

type Consolidator interface {
	Consolidate(ctx context.Context, in Input) (Decision, error)
}

Consolidator decides how a new memory relates to existing candidates.

type Decision

type Decision struct {
	Action Action `json:"action"`
	// Target is the candidate ID affected by update/supersede (empty for new).
	Target string `json:"target"`
	// Content is the merged text to persist for an update (else the new content).
	Content string `json:"content"`
	// Summary is an optional one-line summary of the resulting memory.
	Summary string `json:"summary"`
	// Reason explains the decision (for logs/debugging).
	Reason string `json:"reason"`
}

Decision is the consolidator's verdict.

type DistillInput

type DistillInput struct {
	Episodes []string `json:"episodes"`
}

DistillInput is a batch of episodic memory contents to distill.

type Distiller

type Distiller interface {
	Distill(ctx context.Context, in DistillInput) ([]Fact, error)
}

Distiller compresses episodic memories into durable semantic facts. Used by the episodic→semantic promotion job.

type Fact

type Fact struct {
	Content string `json:"content"`
	Summary string `json:"summary,omitempty"`
}

Fact is a durable memory distilled from episodic observations.

type Input

type Input struct {
	New        string      `json:"new"`
	Tier       string      `json:"tier"`
	Candidates []Candidate `json:"candidates"`
}

Input is the consolidation request.

type OpenAIClient

type OpenAIClient struct {
	// contains filtered or unexported fields
}

OpenAIClient is a Client backed by an OpenAI-compatible /chat/completions endpoint.

func NewOpenAI

func NewOpenAI(cfg Config) (*OpenAIClient, error)

NewOpenAI builds a chat client. BaseURL and Model are required.

func (*OpenAIClient) Complete

func (c *OpenAIClient) Complete(ctx context.Context, system, user string) (string, error)

Complete is a single-turn chat completion returning the assistant message text.

func (*OpenAIClient) Consolidate

func (c *OpenAIClient) Consolidate(ctx context.Context, in Input) (Decision, error)

Consolidate asks the model how the new memory relates to the candidates.

func (*OpenAIClient) Distill

func (c *OpenAIClient) Distill(ctx context.Context, in DistillInput) ([]Fact, error)

Distill compresses episodic memories into durable semantic facts.

type OpenAIConfig

type OpenAIConfig = Config

OpenAIConfig and AnthropicConfig are aliases kept for call-site clarity.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL