llm

type Capabilities ¶

type Capabilities struct {
	Streaming  bool
	Tools      bool
	Vision     bool
	Embeddings bool
}

Capabilities advertises optional features a provider supports, so callers can branch without triggering a failed request.

type Chunk ¶

type Chunk struct {
	// Text is an incremental text delta (empty on non-text chunks).
	Text string

	// ToolCall is a fully-assembled tool call (nil unless this chunk is a call).
	ToolCall *ToolCallPart

	// Reasoning is a fully-assembled reasoning trace for the turn (nil unless
	// this chunk carries one). The caller must keep it on the assistant turn it
	// re-sends, exactly like a ReasoningPart from Generate.
	Reasoning *ReasoningPart

	// FinishReason is why generation stopped. Set only on the Done chunk.
	FinishReason FinishReason

	// Usage is token accounting for the whole generation. Set only on the Done chunk.
	Usage Usage

	// Done is true on the single terminal chunk that carries FinishReason+Usage.
	Done bool
}

Chunk is one increment of a streaming generation (decision 5: the core streaming primitive is iter.Seq2[Chunk, error]). A stream yields a sequence of content chunks followed by exactly one terminal chunk:

• Text holds an incremental text delta. It is the common chunk; concatenating every Text across a stream reconstructs the full message text.
• ToolCall is a tool call surfaced ONCE FULLY ASSEMBLED (id + name + complete JSON args), not as raw argument fragments — a streamed tool call arrives as partial JSON across many wire chunks, and almost every caller wants the whole call, so the adapter accumulates it and emits it complete. nil on a text chunk.
• Reasoning is a provider reasoning trace surfaced ONCE FULLY ASSEMBLED, like ToolCall — a stateful trace (Anthropic's signed thinking blocks) is only replayable complete, and replay is the sole reason it exists (see ReasoningPart). A provider that returns none never yields this shape.
• Done marks the single TERMINAL chunk. It carries no content; instead it reports FinishReason and the final Usage for the whole generation. It is the streaming analogue of a Response's tail fields, so a caller that only needs totals can ignore every prior chunk and read this one.

A chunk is one of those four shapes — a text delta, a completed tool call, an assembled reasoning trace, or the terminal Done — never a mix.

type ContentPart ¶

type ContentPart interface {
	// contains filtered or unexported methods
}

ContentPart is a single piece of a message's content. It is a closed sum type: the only implementations are those defined in this package (TextPart, ImagePart, ToolCallPart, ToolResultPart).

The content model is part-based from day one (decision 3) because both underlying wire formats are natively part-based and because vision and tool use require it. Text-first helpers keep the common case a one-liner.

type ErrorKind ¶

type ErrorKind string

ErrorKind is a normalized error classification (decision 9). Adapters map provider/SDK errors into one of these so callers can react uniformly across providers (e.g. back off a rate-limited model in a comparison run).

const (
	KindRateLimit     ErrorKind = "rate_limit"
	KindAuth          ErrorKind = "auth"
	KindContextLength ErrorKind = "context_length"
	KindContentFilter ErrorKind = "content_filter"
	KindCanceled      ErrorKind = "canceled"
	KindUnsupported   ErrorKind = "unsupported"
	KindUnknown       ErrorKind = "unknown"
)

type FinishReason ¶

type FinishReason string

FinishReason is the normalized reason generation stopped.

const (
	FinishStop          FinishReason = "stop"           // natural end
	FinishLength        FinishReason = "length"         // hit a token cap
	FinishToolUse       FinishReason = "tool_use"       // model wants to call a tool
	FinishContentFilter FinishReason = "content_filter" // blocked by a safety filter
	FinishOther         FinishReason = "other"          // unrecognized / provider-specific
)

type ImagePart ¶

type ImagePart struct {
	// URL is a remote image reference. Mutually exclusive with Data.
	URL string
	// Data is raw image bytes. Mutually exclusive with URL.
	Data []byte
	// MIME is the media type for Data, e.g. "image/png".
	MIME string
}

ImagePart is image content for vision-capable models.

func ImageData(mime string, data []byte) ImagePart

func ImageURL(url string) ImagePart

type Message ¶

type Message struct {
	Role  Role
	Parts []ContentPart
}

Message is one turn in a conversation: a role plus structured content parts.

func Assistant(s string) Message

func System(s string) Message

func ToolResultMsg(id, content string, isErr bool) Message

func User(s string) Message

func UserParts(parts ...ContentPart) Message

func (m Message) MarshalJSON() ([]byte, error)

func (m Message) Text() string

func (m *Message) UnmarshalJSON(data []byte) error

type ModelInfo ¶ added in v0.2.0

type ModelInfo struct {
	ContextWindow   int `json:"context_window"`
	MaxOutputTokens int `json:"max_output_tokens"`
}

ModelInfo describes model limits that are useful to callers before a request is sent. A zero field means that limit is unknown; callers must treat an unknown limit as having no proactive limit.

func Lookup(model string) ModelInfo

type Provider ¶

type Provider interface {
	// Name is the registered identity of this provider (e.g. "xai").
	Name() string

	// Capabilities advertises what this provider/model supports.
	Capabilities() Capabilities

	// Generate runs a single, non-streaming completion.
	Generate(ctx context.Context, req Request) (*Response, error)

	// Stream runs a completion incrementally, yielding Chunks as they arrive
	// (decision 5). The iterator yields content chunks (text deltas, completed
	// tool calls) followed by one terminal Done chunk carrying FinishReason and
	// Usage; a non-nil error ends the stream. A backend that cannot stream returns
	// UnsupportedStream and reports Streaming=false in Capabilities.
	Stream(ctx context.Context, req Request) iter.Seq2[Chunk, error]
}

Provider is the uniform interface every LLM backend implements. It is kept small (decision 2) so swapping providers is seamless for the common path.

Capabilities that only some providers have (embeddings, ...) are exposed as separate optional interfaces promoted via type-assertion, not added here.

type ProviderError ¶

type ProviderError struct {
	Provider   string    // which provider produced it (e.g. "xai")
	Kind       ErrorKind // normalized classification
	StatusCode int       // HTTP status, if known (0 otherwise)
	Retryable  bool      // whether a retry might succeed
	Err        error     // the wrapped original error
}

ProviderError wraps an underlying error with a normalized classification. The original error is preserved (Unwrap), so errors.Is/As reach the SDK's own error type and the Raw details remain accessible.

func (e *ProviderError) Error() string

func (e *ProviderError) Is(target error) bool

func (e *ProviderError) Unwrap() error

type ReasoningPart ¶

type ReasoningPart struct{ Raw json.RawMessage }

ReasoningPart carries a provider's OPAQUE reasoning trace for an assistant turn (OpenRouter's `reasoning_details`), so the loop can replay it verbatim when it re-sends that turn. It exists for ENCRYPTED reasoning that is STATEFUL: Gemini returns its chain of thought as an encrypted "thought signature" (reasoning.encrypted / google-gemini-v1) and its API requires that signature be sent back on the following turn — without it the model is amnesiac across tool calls and spirals (re-issuing the same action until a no-progress detector kills it). Plaintext reasoning (e.g. deepseek's reasoning.text) is carried too, but is re-derivable so dropping it is harmless; the encrypted kind is not. The harness never INTERPRETS Raw — it only round-trips it, so the field stays a provider black box.

type Registry ¶

type Registry struct {
	// contains filtered or unexported fields
}

Registry holds named providers so they can be configured once and used interchangeably or all at once (decision 7). It is safe for concurrent use.

func NewRegistry() *Registry

func (r *Registry) Add(name string, p Provider) *Registry

func (r *Registry) All() []Provider

func (r *Registry) Get(name string) (Provider, bool)

func (r *Registry) MustGet(name string) Provider

func (r *Registry) Names() []string

type Request ¶

type Request struct {
	// Model overrides the provider's default model for this request.
	// Empty means "use the provider's configured default".
	Model string

	// System is a convenience top-level system prompt. Adapters place it
	// where the provider expects it (a system message for OpenAI-compatible
	// providers, a top-level field for Claude).
	System string

	// Messages is the conversation so far.
	Messages []Message

	// MaxTokens caps generated tokens. 0 means unset (provider default).
	MaxTokens int

	// Temperature, TopP, Seed are optional sampling controls.
	Temperature *float64
	TopP        *float64
	Seed        *int64

	// Stop is an optional set of stop sequences.
	Stop []string

	// Tools are the functions the model may call this turn. When non-empty, a
	// tool-capable adapter advertises them and may return ToolCallParts with
	// FinishToolUse; the caller executes them and replies with ToolResultParts.
	// Adapters that don't support tools ignore this (preserving swap-ability).
	Tools []Tool

	// ReasoningEffort asks a reasoning model to think harder or less
	// ("minimal" | "low" | "medium" | "high" | "xhigh" — passed through
	// untyped, so a new tier needs no code change). Empty means unset: the
	// provider's default, which is what every measurement before 2026-07-03
	// ran at. Adapters map it to their wire param (openaicompat →
	// `reasoning_effort`, the OpenAI chat-completions param OpenRouter
	// normalizes into its unified reasoning system); adapters without a
	// reasoning surface ignore it.
	ReasoningEffort string

	// ProviderOptions is a passthrough for provider-specific knobs (e.g.
	// reasoning effort, thinking budget). Adapters merge recognized keys into
	// the underlying request and ignore the rest. Prefer the typed helpers
	// each provider package exposes over setting raw keys here.
	ProviderOptions map[string]any
}

Request is a provider-agnostic generation request.

Broadly-shared sampling params are typed fields (decision 6). Optional numeric knobs are pointers so "unset" is distinguishable from a deliberate zero. Provider-specific knobs ride in ProviderOptions and are interpreted by whichever adapter understands them; adapters ignore options they don't.

type Response ¶

type Response struct {
	// Content is the generated content parts (text for now).
	Content []ContentPart

	// FinishReason is why generation stopped.
	FinishReason FinishReason

	// Usage is token accounting.
	Usage Usage

	// Model is the model that actually served the request, as reported by the
	// provider (may differ from the requested model, e.g. via OpenRouter).
	Model string

	// Raw is the underlying SDK response. Type-assert to the provider's
	// concrete type to reach un-normalized fields (logprobs, reasoning, etc.).
	Raw any
}

Response is a provider-agnostic generation result. Raw holds the underlying SDK response for deep inspection of anything not normalized here (decision 8).

func (r *Response) Text() string

type Role ¶

type Role string

Role identifies the author of a message.

const (
	RoleSystem    Role = "system"
	RoleUser      Role = "user"
	RoleAssistant Role = "assistant"
	RoleTool      Role = "tool"
)

type TextPart ¶

type TextPart struct{ Text string }

TextPart is plain text content.

func Text(s string) TextPart

type Tool ¶

type Tool struct {
	// Name is the function name the model calls and the loop dispatches on.
	Name string
	// Description tells the model when and how to use the tool.
	Description string
	// Schema is a JSON Schema object describing the arguments. An empty schema
	// declares a no-argument tool.
	Schema json.RawMessage
}

Tool is a provider-agnostic description of a function the model may call. It carries NO handler — execution lives in the caller's loop (decision 4), which matches a returned ToolCallPart to its Tool by Name and runs it. Adapters translate Schema into their provider's tool format (OpenAI's `function.parameters`, Claude's `input_schema`), so the same Tool works across providers.

type ToolCallPart ¶

type ToolCallPart struct {
	ID   string
	Name string
	Args json.RawMessage
}

ToolCallPart is the model asking to invoke a tool: a provider-assigned call ID, the tool Name, and the raw JSON Args object. It appears in an assistant message when FinishReason is FinishToolUse. The caller runs the named tool and answers with a ToolResultPart carrying the same ID.

type ToolResultPart ¶

type ToolResultPart struct {
	ToolCallID string
	Content    string
	IsError    bool
}

ToolResultPart is the caller's answer to a ToolCallPart: the observation text, tagged with the ToolCallID it answers and whether the tool failed. IsError is a real outcome the model should see and react to (a failed tool is information, not a transport error) — not a Go error. Carried in a RoleTool message.

type Usage ¶

type Usage struct {
	PromptTokens     int     `json:"prompt_tokens"`
	CompletionTokens int     `json:"completion_tokens"`
	TotalTokens      int     `json:"total_tokens"`
	CachedTokens     int     `json:"cached_tokens"`
	ReasoningTokens  int     `json:"reasoning_tokens"`
	Cost             float64 `json:"cost"` // native provider-reported cost in USD for this call; 0 when the provider does not report one.
}

Usage reports token accounting for a request. CachedTokens is the portion of PromptTokens served from a prompt cache, where the provider reports it. ReasoningTokens is the portion of CompletionTokens the model spent on hidden reasoning (thinking models: Gemini, o-series, DeepSeek). It is a SUBSET of CompletionTokens — already billed inside it — so never add it to a total; it is broken out only to see where completion spend goes.

JSON tags are snake_case so every record that embeds Usage (agent transcripts, eval Trial files, council records, cmd/agent result JSON) uses one consistent shape on disk. Before 2026-07 the struct had no tags and the on-disk corpus (~600+ runs) used PascalCase keys ("PromptTokens"). The custom UnmarshalJSON accepts BOTH shapes so old unversioned records (eval Trial, council) don't silently zero out on read.

func (u *Usage) UnmarshalJSON(data []byte) error