llm

package
v0.6.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 10, 2026 License: AGPL-3.0 Imports: 12 Imported by: 0

Documentation

Overview

Package llm holds the opt-in consolidation pipeline: on each write it decides whether a new memory is novel, a refinement, or a contradiction that supersedes an existing one. Without an LLM the service stores raw.

Two backends implement the same surface: an OpenAI-compatible chat client (openai-go) and an Anthropic Messages client (anthropic-sdk-go). The Anthropic backend caches the static system prompt, which makes high-write consolidation cheaper against providers like MiniMax that support prompt caching on the Anthropic-compatible API.

Index

Constants

View Source
const (
	RoleUser      = "user"
	RoleAssistant = "assistant"
	RoleTool      = "tool"
)

Chat turn roles.

Variables

This section is empty.

Functions

This section is empty.

Types

type API

type API string

API selects the chat backend.

const (
	// APIOpenAI talks to an OpenAI-compatible /chat/completions endpoint.
	APIOpenAI API = "openai"
	// APIAnthropic talks to an Anthropic Messages endpoint (with prompt caching).
	APIAnthropic API = "anthropic"
)

type Action

type Action string

Action is the consolidation decision for a new memory.

const (
	// ActionNew stores the new memory as a distinct record.
	ActionNew Action = "new"
	// ActionUpdate merges the new memory into an existing one (Target), which is
	// rewritten with Content; no new record is created.
	ActionUpdate Action = "update"
	// ActionSupersede stores the new memory and tombstones Target as superseded.
	ActionSupersede Action = "supersede"
)

type AnthropicClient

type AnthropicClient struct {
	// contains filtered or unexported fields
}

AnthropicClient is a Client backed by an Anthropic Messages endpoint. It works against the real Anthropic API and against Anthropic-compatible providers (e.g. MiniMax) via a BaseURL override. The static system prompt is marked for prompt caching, so repeated consolidation calls reuse it at the cache-read rate.

func NewAnthropic

func NewAnthropic(cfg Config) (*AnthropicClient, error)

NewAnthropic builds an Anthropic Messages client. Model is required; BaseURL is optional (defaults to the Anthropic API, override it for compatible providers).

func (*AnthropicClient) ChatTools added in v0.5.9

func (c *AnthropicClient) ChatTools(
	ctx context.Context, system string, turns []ChatTurn, tools []Tool, choice ToolChoice,
) (ChatResult, error)

ChatTools runs one round of a tool-calling conversation, translating the canonical tool/choice vocabulary to the Messages encoding: tool results become tool_result blocks in a user message (consecutive results coalesce into one), and ToolRequired maps to Anthropic's "any".

func (*AnthropicClient) Complete

func (c *AnthropicClient) Complete(ctx context.Context, system, user string) (string, error)

Complete is a single-turn message returning the concatenated text blocks.

func (*AnthropicClient) Consolidate

func (c *AnthropicClient) Consolidate(ctx context.Context, in Input) (Decision, error)

Consolidate asks the model how the new memory relates to the candidates.

func (*AnthropicClient) Distill

func (c *AnthropicClient) Distill(ctx context.Context, in DistillInput) ([]Fact, error)

Distill compresses episodic memories into durable semantic facts.

func (*AnthropicClient) MergeMemories added in v0.5.12

func (c *AnthropicClient) MergeMemories(ctx context.Context, contents []string) (string, error)

MergeMemories merges a cluster of near-duplicate memory texts into one comprehensive memory via LLM, using the mergePrompt.

type AnthropicConfig

type AnthropicConfig = Config

OpenAIConfig and AnthropicConfig are aliases kept for call-site clarity.

type Candidate

type Candidate struct {
	ID      string `json:"id"`
	Content string `json:"content"`
}

Candidate is an existing memory offered to the consolidator for comparison.

type ChatResult added in v0.5.9

type ChatResult struct {
	Text  string
	Calls []ToolCall
}

ChatResult is one round of a tool loop: final text, or tool calls to run.

type ChatTurn added in v0.5.9

type ChatTurn struct {
	// Role is one of RoleUser, RoleAssistant, RoleTool.
	Role string
	// Text is the user/assistant text, or the tool result content.
	Text string
	// Calls are the tool calls an assistant turn requested.
	Calls []ToolCall
	// CallID names the call a tool turn answers.
	CallID string
	// Name is the tool turn's tool name.
	Name string
}

ChatTurn is one entry of a tool-loop transcript.

type Client

type Client interface {
	Consolidator
	Completer
	Distiller
	Merger
}

Client is a chat backend that can consolidate memories, distill facts, answer single-turn prompts, and merge memory clusters.

func New

func New(api API, cfg Config) (Client, error)

New builds a chat client for the given API ("openai" default, or "anthropic").

type Completer

type Completer interface {
	Complete(ctx context.Context, system, user string) (string, error)
}

Completer is a single-turn chat completion (used by the benchmark harness).

type Config

type Config struct {
	BaseURL string // OpenAI: e.g. https://host/v1 ; Anthropic: e.g. https://api.minimax.io/anthropic
	APIKey  string
	Model   string
	// MaxTokens caps the completion length (defaults to defaultMaxTokens). A
	// budget is required for reasoning models, which otherwise spend the server
	// default on hidden reasoning and return empty content.
	MaxTokens  int
	HTTPClient *http.Client
}

Config configures a chat client. The same fields apply to both the OpenAI-compatible and Anthropic backends.

type Consolidator

type Consolidator interface {
	Consolidate(ctx context.Context, in Input) (Decision, error)
}

Consolidator decides how a new memory relates to existing candidates.

type Decision

type Decision struct {
	Action Action `json:"action"`
	// Target is the candidate ID affected by update/supersede (empty for new).
	Target string `json:"target"`
	// Content is the merged text to persist for an update (else the new content).
	Content string `json:"content"`
	// Summary is an optional one-line summary of the resulting memory.
	Summary string `json:"summary"`
	// Reason explains the decision (for logs/debugging).
	Reason string `json:"reason"`
	// LinkedIDs are IDs of existing memories related to this one (same
	// entity/topic) but neither duplicate nor contradiction. Empty for "new"
	// with no related candidates or update/supersede actions.
	LinkedIDs []string `json:"linked_ids,omitempty"`
}

Decision is the consolidator's verdict.

type DistillInput

type DistillInput struct {
	Episodes []Episode `json:"episodes"`
	Now      string    `json:"now,omitempty"`
}

DistillInput is a batch of episodic memories to distill. Now is the current date (YYYY-MM-DD); the model grounds relative dates against each episode's Date, falling back to Now. Both empty disables grounding.

type Distiller

type Distiller interface {
	Distill(ctx context.Context, in DistillInput) ([]Fact, error)
}

Distiller compresses episodic memories into durable semantic facts. Used by the episodic→semantic promotion job.

type Episode added in v0.4.19

type Episode struct {
	Content string `json:"content"`
	// Date is the YYYY-MM-DD the episode was recorded. Empty when unknown.
	Date string `json:"date,omitempty"`
}

Episode is one episodic memory to distill, paired with the date it was recorded so the model can resolve relative dates in the text ("yesterday", "last week") to absolute ones.

type Fact

type Fact struct {
	Content string `json:"content"`
	Summary string `json:"summary,omitempty"`
	// Category routes the fact to a tier: "procedure" (incl. error→recovery) →
	// procedural; "preference" and "fact" → semantic. Empty defaults to semantic.
	Category string `json:"category,omitempty"`
	// Confidence is the LLM's self-assessed reliability of this fact, in [0.1, 0.7].
	// nil means unset; the service layer falls back to ConfidenceSeedFresh.
	Confidence *float64 `json:"confidence,omitempty"`
}

Fact is a durable memory distilled from episodic observations.

type Input

type Input struct {
	New        string      `json:"new"`
	Tier       string      `json:"tier"`
	Candidates []Candidate `json:"candidates"`
}

Input is the consolidation request.

type Merger added in v0.5.12

type Merger interface {
	MergeMemories(ctx context.Context, contents []string) (string, error)
}

Merger merges a cluster of near-duplicate memories into a single, comprehensive memory text. Used by the batch dedup job (maintenance.DedupJob) to produce merged content for cluster representatives, rather than keeping the representative's original (potentially incomplete) text.

type OpenAIClient

type OpenAIClient struct {
	// contains filtered or unexported fields
}

OpenAIClient is a Client backed by an OpenAI-compatible /chat/completions endpoint.

func NewOpenAI

func NewOpenAI(cfg Config) (*OpenAIClient, error)

NewOpenAI builds a chat client. BaseURL and Model are required.

func (*OpenAIClient) ChatTools added in v0.5.9

func (c *OpenAIClient) ChatTools(
	ctx context.Context, system string, turns []ChatTurn, tools []Tool, choice ToolChoice,
) (ChatResult, error)

ChatTools runs one round of a tool-calling conversation, translating the canonical tool/choice vocabulary to the /chat/completions encoding.

func (*OpenAIClient) Complete

func (c *OpenAIClient) Complete(ctx context.Context, system, user string) (string, error)

Complete is a single-turn chat completion returning the assistant message text.

func (*OpenAIClient) Consolidate

func (c *OpenAIClient) Consolidate(ctx context.Context, in Input) (Decision, error)

Consolidate asks the model how the new memory relates to the candidates.

func (*OpenAIClient) Distill

func (c *OpenAIClient) Distill(ctx context.Context, in DistillInput) ([]Fact, error)

Distill compresses episodic memories into durable semantic facts.

func (*OpenAIClient) MergeMemories added in v0.5.12

func (c *OpenAIClient) MergeMemories(ctx context.Context, contents []string) (string, error)

MergeMemories merges a cluster of near-duplicate memory texts into one comprehensive memory via LLM, using the mergePrompt.

type OpenAIConfig

type OpenAIConfig = Config

OpenAIConfig and AnthropicConfig are aliases kept for call-site clarity.

type Tool added in v0.5.9

type Tool struct {
	Name        string
	Description string
	Schema      map[string]any
}

Tool describes one callable tool exposed to the model. Schema is the JSON Schema of the tool's arguments (type object, properties, required) — each backend translates it to its provider's tool encoding.

type ToolCall added in v0.5.9

type ToolCall struct {
	ID   string
	Name string
	Args json.RawMessage
}

ToolCall is one tool invocation requested by the model. Args is the raw JSON arguments string; validate before use (models hallucinate fields).

type ToolChat added in v0.5.9

type ToolChat interface {
	ChatTools(ctx context.Context, system string, turns []ChatTurn, tools []Tool, choice ToolChoice) (ChatResult, error)
}

ToolChat runs one round of a tool-calling conversation. Implemented by both backends; a caller holding a Completer can type-assert for loop support.

type ToolChoice added in v0.5.9

type ToolChoice string

ToolChoice is the canonical cross-provider tool-selection vocabulary, translated per backend (OpenAI none/auto/required; Anthropic none/auto/any).

const (
	// ToolAuto lets the model choose between calling tools and answering.
	ToolAuto ToolChoice = "auto"
	// ToolNone forbids tool calls — the forced final-synthesis turn.
	ToolNone ToolChoice = "none"
	// ToolRequired forces at least one tool call.
	ToolRequired ToolChoice = "required"
)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL