Documentation
¶
Overview ¶
Package llm holds the opt-in consolidation pipeline: on each write it decides whether a new memory is novel, a refinement, or a contradiction that supersedes an existing one. Without an LLM the service stores raw.
Two backends implement the same surface: an OpenAI-compatible chat client (openai-go) and an Anthropic Messages client (anthropic-sdk-go). The Anthropic backend caches the static system prompt, which makes high-write consolidation cheaper against providers like MiniMax that support prompt caching on the Anthropic-compatible API.
Index ¶
- Constants
- type API
- type Action
- type AnthropicClient
- func (c *AnthropicClient) ChatTools(ctx context.Context, system string, turns []ChatTurn, tools []Tool, ...) (ChatResult, error)
- func (c *AnthropicClient) Complete(ctx context.Context, system, user string) (string, error)
- func (c *AnthropicClient) Consolidate(ctx context.Context, in Input) (Decision, error)
- func (c *AnthropicClient) Distill(ctx context.Context, in DistillInput) ([]Fact, error)
- type AnthropicConfig
- type Candidate
- type ChatResult
- type ChatTurn
- type Client
- type Completer
- type Config
- type Consolidator
- type Decision
- type DistillInput
- type Distiller
- type Episode
- type Fact
- type Input
- type OpenAIClient
- func (c *OpenAIClient) ChatTools(ctx context.Context, system string, turns []ChatTurn, tools []Tool, ...) (ChatResult, error)
- func (c *OpenAIClient) Complete(ctx context.Context, system, user string) (string, error)
- func (c *OpenAIClient) Consolidate(ctx context.Context, in Input) (Decision, error)
- func (c *OpenAIClient) Distill(ctx context.Context, in DistillInput) ([]Fact, error)
- type OpenAIConfig
- type Tool
- type ToolCall
- type ToolChat
- type ToolChoice
Constants ¶
const ( RoleUser = "user" RoleAssistant = "assistant" RoleTool = "tool" )
Chat turn roles.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Action ¶
type Action string
Action is the consolidation decision for a new memory.
const ( // ActionNew stores the new memory as a distinct record. ActionNew Action = "new" // ActionUpdate merges the new memory into an existing one (Target), which is // rewritten with Content; no new record is created. ActionUpdate Action = "update" // ActionSupersede stores the new memory and tombstones Target as superseded. ActionSupersede Action = "supersede" )
type AnthropicClient ¶
type AnthropicClient struct {
// contains filtered or unexported fields
}
AnthropicClient is a Client backed by an Anthropic Messages endpoint. It works against the real Anthropic API and against Anthropic-compatible providers (e.g. MiniMax) via a BaseURL override. The static system prompt is marked for prompt caching, so repeated consolidation calls reuse it at the cache-read rate.
func NewAnthropic ¶
func NewAnthropic(cfg Config) (*AnthropicClient, error)
NewAnthropic builds an Anthropic Messages client. Model is required; BaseURL is optional (defaults to the Anthropic API, override it for compatible providers).
func (*AnthropicClient) ChatTools ¶ added in v0.5.9
func (c *AnthropicClient) ChatTools( ctx context.Context, system string, turns []ChatTurn, tools []Tool, choice ToolChoice, ) (ChatResult, error)
ChatTools runs one round of a tool-calling conversation, translating the canonical tool/choice vocabulary to the Messages encoding: tool results become tool_result blocks in a user message (consecutive results coalesce into one), and ToolRequired maps to Anthropic's "any".
func (*AnthropicClient) Complete ¶
Complete is a single-turn message returning the concatenated text blocks.
func (*AnthropicClient) Consolidate ¶
Consolidate asks the model how the new memory relates to the candidates.
func (*AnthropicClient) Distill ¶
func (c *AnthropicClient) Distill(ctx context.Context, in DistillInput) ([]Fact, error)
Distill compresses episodic memories into durable semantic facts.
type AnthropicConfig ¶
type AnthropicConfig = Config
OpenAIConfig and AnthropicConfig are aliases kept for call-site clarity.
type ChatResult ¶ added in v0.5.9
ChatResult is one round of a tool loop: final text, or tool calls to run.
type ChatTurn ¶ added in v0.5.9
type ChatTurn struct {
// Role is one of RoleUser, RoleAssistant, RoleTool.
Role string
// Text is the user/assistant text, or the tool result content.
Text string
// Calls are the tool calls an assistant turn requested.
Calls []ToolCall
// CallID names the call a tool turn answers.
CallID string
// Name is the tool turn's tool name.
Name string
}
ChatTurn is one entry of a tool-loop transcript.
type Client ¶
type Client interface {
Consolidator
Completer
Distiller
}
Client is a chat backend that can consolidate memories, distill facts, and answer single-turn prompts.
type Config ¶
type Config struct {
BaseURL string // OpenAI: e.g. https://host/v1 ; Anthropic: e.g. https://api.minimax.io/anthropic
APIKey string
Model string
// MaxTokens caps the completion length (defaults to defaultMaxTokens). A
// budget is required for reasoning models, which otherwise spend the server
// default on hidden reasoning and return empty content.
MaxTokens int
HTTPClient *http.Client
}
Config configures a chat client. The same fields apply to both the OpenAI-compatible and Anthropic backends.
type Consolidator ¶
Consolidator decides how a new memory relates to existing candidates.
type Decision ¶
type Decision struct {
Action Action `json:"action"`
// Target is the candidate ID affected by update/supersede (empty for new).
Target string `json:"target"`
// Content is the merged text to persist for an update (else the new content).
Content string `json:"content"`
// Summary is an optional one-line summary of the resulting memory.
Summary string `json:"summary"`
// Reason explains the decision (for logs/debugging).
Reason string `json:"reason"`
}
Decision is the consolidator's verdict.
type DistillInput ¶
DistillInput is a batch of episodic memories to distill. Now is the current date (YYYY-MM-DD); the model grounds relative dates against each episode's Date, falling back to Now. Both empty disables grounding.
type Distiller ¶
type Distiller interface {
Distill(ctx context.Context, in DistillInput) ([]Fact, error)
}
Distiller compresses episodic memories into durable semantic facts. Used by the episodic→semantic promotion job.
type Episode ¶ added in v0.4.19
type Episode struct {
Content string `json:"content"`
// Date is the YYYY-MM-DD the episode was recorded. Empty when unknown.
Date string `json:"date,omitempty"`
}
Episode is one episodic memory to distill, paired with the date it was recorded so the model can resolve relative dates in the text ("yesterday", "last week") to absolute ones.
type Fact ¶
type Fact struct {
Content string `json:"content"`
Summary string `json:"summary,omitempty"`
// Category routes the fact to a tier: "procedure" (incl. error→recovery) →
// procedural; "preference" and "fact" → semantic. Empty defaults to semantic.
Category string `json:"category,omitempty"`
}
Fact is a durable memory distilled from episodic observations.
type Input ¶
type Input struct {
New string `json:"new"`
Tier string `json:"tier"`
Candidates []Candidate `json:"candidates"`
}
Input is the consolidation request.
type OpenAIClient ¶
type OpenAIClient struct {
// contains filtered or unexported fields
}
OpenAIClient is a Client backed by an OpenAI-compatible /chat/completions endpoint.
func NewOpenAI ¶
func NewOpenAI(cfg Config) (*OpenAIClient, error)
NewOpenAI builds a chat client. BaseURL and Model are required.
func (*OpenAIClient) ChatTools ¶ added in v0.5.9
func (c *OpenAIClient) ChatTools( ctx context.Context, system string, turns []ChatTurn, tools []Tool, choice ToolChoice, ) (ChatResult, error)
ChatTools runs one round of a tool-calling conversation, translating the canonical tool/choice vocabulary to the /chat/completions encoding.
func (*OpenAIClient) Complete ¶
Complete is a single-turn chat completion returning the assistant message text.
func (*OpenAIClient) Consolidate ¶
Consolidate asks the model how the new memory relates to the candidates.
func (*OpenAIClient) Distill ¶
func (c *OpenAIClient) Distill(ctx context.Context, in DistillInput) ([]Fact, error)
Distill compresses episodic memories into durable semantic facts.
type OpenAIConfig ¶
type OpenAIConfig = Config
OpenAIConfig and AnthropicConfig are aliases kept for call-site clarity.
type Tool ¶ added in v0.5.9
Tool describes one callable tool exposed to the model. Schema is the JSON Schema of the tool's arguments (type object, properties, required) — each backend translates it to its provider's tool encoding.
type ToolCall ¶ added in v0.5.9
type ToolCall struct {
ID string
Name string
Args json.RawMessage
}
ToolCall is one tool invocation requested by the model. Args is the raw JSON arguments string; validate before use (models hallucinate fields).
type ToolChat ¶ added in v0.5.9
type ToolChat interface {
ChatTools(ctx context.Context, system string, turns []ChatTurn, tools []Tool, choice ToolChoice) (ChatResult, error)
}
ToolChat runs one round of a tool-calling conversation. Implemented by both backends; a caller holding a Completer can type-assert for loop support.
type ToolChoice ¶ added in v0.5.9
type ToolChoice string
ToolChoice is the canonical cross-provider tool-selection vocabulary, translated per backend (OpenAI none/auto/required; Anthropic none/auto/any).
const ( // ToolAuto lets the model choose between calling tools and answering. ToolAuto ToolChoice = "auto" // ToolNone forbids tool calls — the forced final-synthesis turn. ToolNone ToolChoice = "none" // ToolRequired forces at least one tool call. ToolRequired ToolChoice = "required" )