Help us shape the future of Go! Take the 2026 Go Developer Survey

provider

package
v0.2.11 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 19, 2026 License: Apache-2.0 Imports: 15 Imported by: 0

Documentation

Overview

Package provider implements LLM provider interfaces and the Anthropic Messages API client with tool-use support.

Index

Constants

View Source
const (
	TurnCategoryUser     = "user"     // human-typed prompt
	TurnCategoryAI       = "ai"       // assistant text / tool_use turns
	TurnCategoryShell    = "shell"    // shell-derived context injected into the conversation
	TurnCategoryRysh     = "rysh"     // rysh system-command derived turns
	TurnCategoryChat     = "chat"     // chat-mode messages (agents registered to a pane)
	TurnCategoryTool     = "tool"     // tool_result turns (Origin = tool name)
	TurnCategorySubAgent = "subagent" // sub-agent summary turns (Origin = child orchestrator id)
	TurnCategorySystem   = "system"   // synthetic turns (compaction summaries, interrupts)
)

Turn categories classify the origin of a conversation turn in the session JSON. They mirror the pane's per-mode output categorisation (shell / ai / rysh / chat) and extend it with agentic origins (tool / subagent). External renderers (TUI, Slack flow, session_history) key off these instead of re-deriving them from Role.

View Source
const DefaultClaudeModel = "claude-opus-4-8"

DefaultClaudeModel is the model used when the host doesn't configure one. claude-opus-4-8 is the current Opus-tier alias (un-dated; date-suffixed IDs like claude-sonnet-4-20250514 are deprecated — that one already 404s).

Exported because every host used to carry its OWN copy of this string, and copies rot at different rates: rysh-server sat on claude-opus-4-5 long after the engine had moved on, and AgentConfig's doc comment still advertised a model the API no longer serves. Hosts that want "whatever the engine defaults to" must be able to SAY that rather than re-type it.

Variables

This section is empty.

Functions

func CategorizeTurn

func CategorizeTurn(t ConversationTurn) string

CategorizeTurn returns the default category for a turn based on its role, used when the producer didn't set one explicitly.

func ChatStreamSupported

func ChatStreamSupported(cp ChatProvider) bool

ChatStreamSupported reports whether cp streams incrementally. Providers that don't self-report are assumed capable (they implement ChatStream by contract); the compat adapter self-reports by type-asserting the wrapped provider against StreamingProvider, so the answer matches the shim path's capability check exactly.

func DefaultMaxTokensForModel

func DefaultMaxTokensForModel(model string) int

DefaultMaxTokensForModel returns a sensible per-model max_tokens budget. Used when the caller passes maxTokens <= 0 to NewClaudeAgenticProvider.

Rationale: current-generation models (Opus 4.6+, Sonnet 4.6+/5, Fable 5) support up to 128k output tokens, but very large non-streaming responses risk HTTP timeouts, so 16k is the safe ceiling that also stops truncating long edits / refactors (the old 4096 Opus default cut them off mid-file). Haiku models get a smaller budget to keep otherwise-cheap calls cheap. Unknown models fall back to 8192 — conservative but rarely truncating.

func DetectImageMediaType

func DetectImageMediaType(b64 string) (string, bool)

DetectImageMediaType decodes the first few bytes of a base64 image payload and returns the media type indicated by its magic bytes:

PNG:  \x89PNG
JPEG: \xFF\xD8\xFF
WebP: RIFF....WEBP
GIF:  GIF8

A leading dataURL prefix is tolerated (stripped before sniffing). Returns ("", false) when the payload is unrecognized or undecodable.

func IsEffortUnsupportedErr

func IsEffortUnsupportedErr(err error) bool

IsEffortUnsupportedErr reports whether an API error says the model does not support the effort parameter (the shape Anthropic returns for e.g. Haiku 4.5). Used to self-heal seats that paired effort with such a model.

func IsInvalidRequestErr

func IsInvalidRequestErr(err error) bool

IsInvalidRequestErr reports whether an API error is a deterministic HTTP 400 invalid_request_error (e.g. an image block whose declared media_type doesn't match its bytes). These are permanent for a given request payload: re-sending the same conversation yields the same rejection, so callers must fail fast instead of retrying — a retry loop on a 400 silently burns the run's entire budget on identical failures. Matches the error strings produced by completeOnce/streamOnce ("…: invalid_request_error (HTTP 400): …" or "…: HTTP 400: …").

func SniffImageMediaType

func SniffImageMediaType(b64 string) string

SniffImageMediaType is DetectImageMediaType with an "image/png" fallback — the historical default, so behavior is unchanged for payloads the sniffer cannot read.

func StripImageDataURLPrefix

func StripImageDataURLPrefix(s string) string

StripImageDataURLPrefix removes a leading `data:image/...;base64,` (or any `data:*;base64,`) prefix so the remainder is raw base64 suitable for an Anthropic image source block. Input without a dataURL prefix is returned unchanged.

func ValidateToolPairing

func ValidateToolPairing(turns []ConversationTurn) []string

ValidateToolPairing reports every pairing violation in a transcript. An empty result means buildMessages can emit the conversation safely. Used by tests (and available for debug logging).

Types

type AgenticProvider

type AgenticProvider interface {
	Provider
	CompleteWithTools(
		ctx context.Context,
		conversation []ConversationTurn,
		tools []ToolSpec,
		systemPrompt string,
	) (*AgenticResponse, error)
}

AgenticProvider extends Provider with multi-turn tool-use capabilities.

type AgenticResponse

type AgenticResponse struct {
	TextBlocks     []TextBlock       // text content blocks
	ToolCalls      []ToolCallRequest // tool_use blocks
	ThinkingBlocks []ThinkingBlock   // extended-thinking content blocks (Phase 3 F)
	StopReason     StopReason        // why the model stopped
	Usage          Usage             // token accounting for this response
}

AgenticResponse is the structured response from CompleteWithTools.

func AgenticResponseFromChat

func AgenticResponseFromChat(cr *ChatResponse) *AgenticResponse

AgenticResponseFromChat converts a neutral response back into the shim shape, grouping blocks by kind (AgenticResponse cannot express cross-kind ordering — see ChatResponse). tool_result and image blocks cannot appear in a model RESPONSE under the shim contract and are converted to text blocks carrying their textual content (documented, tested — never silently dropped). Nil-safe.

type Block

type Block struct {
	Kind BlockKind `json:"kind"`

	// Text carries BlockKindText content.
	Text string `json:"text,omitempty"`
	// Thinking carries BlockKindThinking content (hidden reasoning + the
	// opaque provenance signature some providers require on replay).
	Thinking *ThinkingBlock `json:"thinking,omitempty"`
	// ToolCall carries BlockKindToolCall content.
	ToolCall *ToolCall `json:"tool_call,omitempty"`
	// ToolResult carries BlockKindToolResult content.
	ToolResult *ToolResult `json:"tool_result,omitempty"`
	// Image carries BlockKindImage content. May be nil for a degenerate
	// image block with no source (preserved as-is for round-tripping).
	Image *ImageSource `json:"image,omitempty"`
}

Block is one ordered content element of a Turn. Kind selects which payload field is meaningful; the others are zero.

type BlockKind

type BlockKind string

BlockKind discriminates the union carried by Block.

const (
	BlockKindText       BlockKind = "text"
	BlockKindThinking   BlockKind = "thinking"
	BlockKindToolCall   BlockKind = "tool_call"
	BlockKindToolResult BlockKind = "tool_result"
	BlockKindImage      BlockKind = "image"
)

type ChatProvider

type ChatProvider interface {
	Name() string
	Chat(ctx context.Context, req ChatRequest) (*ChatResponse, error)
	ChatStream(ctx context.Context, req ChatRequest, cb StreamCallback) (*ChatResponse, error)
}

ChatProvider is the neutral multi-turn tool-use provider interface (design 002 §3.2). Chat is the one-shot path; ChatStream is the incremental path and MUST behave identically apart from delivering StreamEvents to cb along the way (implementations without true streaming may satisfy it by completing one-shot and delivering no events — callers discover real streaming support via ChatStreamSupported).

func AsChatProvider

func AsChatProvider(p AgenticProvider) ChatProvider

AsChatProvider adapts p into a ChatProvider. The two bare native providers (Claude / OpenAI-compatible) are returned as-is — their Chat/ChatStream translate neutral turns straight to the wire, no converters involved. Everything else — wrappers included — is adapted at the edge via the neutral converters, so decorator semantics (sanitization, defaults, overrides) keep applying. A wrapper that EMBEDS a native provider inherits nativeChatSelf by promotion, but the promoted method returns the embedded provider, not the wrapper, so the identity check below keeps it on the adapter path instead of silently bypassing its decoration. Nil in, nil out.

type ChatRequest

type ChatRequest struct {
	// System is the system prompt ("" for none).
	System string
	// Turns is the neutral conversation history, oldest first.
	Turns []Turn
	// Tools are the tool definitions offered to the model (nil for none).
	Tools []ToolSpec
	// Model optionally overrides the provider's configured model for this
	// request. Routed through ModelEffortOverridable; providers without that
	// seam return an error rather than silently ignoring the override.
	Model string
	// MaxTokens optionally caps the response size (max_tokens on the wire)
	// for this request. Routed through MaxTokensOverridable; 0 keeps the
	// provider's construction-time cap, and providers without that seam
	// return an error rather than silently ignoring the cap.
	MaxTokens int
}

ChatRequest is one neutral chat completion request.

type ChatResponse

type ChatResponse struct {
	Blocks     []Block
	StopReason StopReason
	Usage      Usage
}

ChatResponse is one neutral chat completion response. Blocks preserve the model's output order where the source can express it; responses adapted from an AgenticResponse carry the canonical order thinking, text, tool calls (AgenticResponse does not record cross-kind ordering).

func ChatResponseFromAgentic

func ChatResponseFromAgentic(ar *AgenticResponse) *ChatResponse

ChatResponseFromAgentic converts a shim response into neutral blocks in canonical replay order: thinking, text, tool calls. Nil-safe.

type ClaudeAgenticProvider

type ClaudeAgenticProvider struct {
	// contains filtered or unexported fields
}

ClaudeAgenticProvider implements AgenticProvider using the Anthropic Messages API.

func NewClaudeAgenticProvider

func NewClaudeAgenticProvider(apiKey, apiURL, model string, maxTokens int) *ClaudeAgenticProvider

NewClaudeAgenticProvider creates a new agentic provider with explicit parameters.

func (*ClaudeAgenticProvider) Chat

Chat implements ChatProvider natively: neutral turns -> Anthropic Messages request -> neutral response. Per-request Model/MaxTokens overrides route through the same seams as the adapter path (0 keeps the construction-time cap; invalid values error loudly), and the retry policy, effort self-heal, and model fallback apply exactly as on CompleteWithTools.

func (*ClaudeAgenticProvider) ChatStream

ChatStream implements ChatProvider natively; the SSE consumption and StreamEvent sequence are the same code path CompleteWithToolsStream uses.

func (*ClaudeAgenticProvider) ChatStreamSupported

func (c *ClaudeAgenticProvider) ChatStreamSupported() bool

ChatStreamSupported reports genuine SSE streaming support.

func (*ClaudeAgenticProvider) Complete

func (c *ClaudeAgenticProvider) Complete(ctx context.Context, prompt string) (string, error)

func (*ClaudeAgenticProvider) CompleteWithTools

func (c *ClaudeAgenticProvider) CompleteWithTools(
	ctx context.Context,
	conversation []ConversationTurn,
	tools []ToolSpec,
	systemPrompt string,
) (*AgenticResponse, error)

CompleteWithTools sends a multi-turn conversation with tool definitions to the Anthropic Messages API and returns the structured response. Transient API errors (429/529/5xx, network) are retried per the provider's RetryPolicy; when the policy is exhausted and a model fallback is configured, ONE final attempt is made against the fallback model before returning the error.

func (*ClaudeAgenticProvider) CompleteWithToolsStream

func (c *ClaudeAgenticProvider) CompleteWithToolsStream(
	ctx context.Context,
	conversation []ConversationTurn,
	tools []ToolSpec,
	systemPrompt string,
	cb StreamCallback,
) (*AgenticResponse, error)

CompleteWithToolsStream is the streaming equivalent of CompleteWithTools. It sets stream=true on the request body and consumes Anthropic's SSE response, invoking cb on each event and returning the assembled AgenticResponse once the stream completes.

The assembled response is byte-for-byte equivalent in shape to what CompleteWithTools returns (same TextBlocks, ToolCalls, StopReason, Usage), so callers can swap the two with no downstream changes.

func (*ClaudeAgenticProvider) Model

func (c *ClaudeAgenticProvider) Model() string

Model returns the model this provider is configured to call. Used by the usage ledger (design 003) to attribute token spend to a specific model for pricing. Additive accessor — not part of the AgenticProvider interface, so consumers type-assert for it and fall back to an empty model when absent.

func (*ClaudeAgenticProvider) Name

func (c *ClaudeAgenticProvider) Name() string

func (*ClaudeAgenticProvider) SetCacheEnabled

func (c *ClaudeAgenticProvider) SetCacheEnabled(enabled bool)

SetCacheEnabled toggles prompt caching. Caching is enabled by default.

func (*ClaudeAgenticProvider) SetModelFallback

func (c *ClaudeAgenticProvider) SetModelFallback(fb ModelFallback)

SetModelFallback installs a one-shot secondary model used when the primary exhausts its retry budget. Pass ModelFallback{} (empty Secondary) to disable the fallback. Phase 3 G.

func (*ClaudeAgenticProvider) SetRetryPolicy

func (c *ClaudeAgenticProvider) SetRetryPolicy(p RetryPolicy)

SetRetryPolicy installs a custom retry/backoff policy. Pass RetryPolicy{MaxAttempts: 1} to disable retries entirely.

func (*ClaudeAgenticProvider) SetThinking

func (c *ClaudeAgenticProvider) SetThinking(cfg *ThinkingConfig)

SetThinking installs the extended-thinking configuration. Pass nil to disable thinking. Phase 3 F.

func (*ClaudeAgenticProvider) WithMaxTokens

func (c *ClaudeAgenticProvider) WithMaxTokens(maxTokens int) AgenticProvider

WithMaxTokens returns a shallow copy of the provider with the max_tokens cap applied. Safe for concurrent use for the same reason WithModelEffort is: the copy shares the HTTP client but no mutable state is written after construction.

func (*ClaudeAgenticProvider) WithModelEffort

func (c *ClaudeAgenticProvider) WithModelEffort(model, effort string) AgenticProvider

WithModelEffort returns a shallow copy of the provider with the non-empty model/effort overrides applied. Safe for concurrent use: the copy shares the HTTP client but no mutable state is written after construction.

type ContentBlock

type ContentBlock struct {
	Type   string       `json:"type"`             // "text" | "image"
	Text   string       `json:"text,omitempty"`   // when Type == "text"
	Source *ImageSource `json:"source,omitempty"` // when Type == "image"
}

ContentBlock is one structured content element in a user/assistant message. Use ContentBlockText / ContentBlockImage constructors to avoid hand-rolling the discriminator field.

func ContentBlockImageBase64

func ContentBlockImageBase64(mediaType, data string) ContentBlock

ContentBlockImageBase64 is a convenience constructor for inline base64 image blocks. mediaType is e.g. "image/png".

func ContentBlockText

func ContentBlockText(text string) ContentBlock

ContentBlockText is a convenience constructor for text blocks.

func HealImageBlock

func HealImageBlock(cb ContentBlock) ContentBlock

HealImageBlock returns cb with a base64 image source's Data stripped of any dataURL prefix and its declared MediaType corrected to the sniffed type when the magic bytes are recognized. Non-image blocks, non-base64 sources, and empty payloads pass through untouched, as does a declared media type the sniffer cannot verify.

This heals conversations persisted BEFORE media types were sniffed at capture time: a turn restored from session KV may still carry e.g. JPEG bytes labeled "image/png", and re-sending it 400s deterministically until corrected. buildMessages applies this at the single chokepoint every request passes through.

type ConversationTurn

type ConversationTurn struct {
	Role          string            // "user", "assistant", or "tool"
	Content       string            // text content (for user messages or assistant text)
	ContentBlocks []ContentBlock    // structured content (e.g. text + image); when non-empty for a user turn, supersedes Content. Follow-up item 1.
	ToolCalls     []ToolCallRequest // tool_use blocks produced by assistant
	ToolCallID    string            // for role "tool": the tool_use id this result corresponds to
	IsError       bool              // for role "tool": whether this result is an error

	// Category classifies the turn's origin (TurnCategory* constants above):
	// shell / ai / rysh / chat for pane-mode parity, tool for tool results,
	// subagent for sub-agent summaries, system for synthetic turns. Serialized
	// into the session JSON so downstream consumers (Slack flow, history tools)
	// respect the pane categorisation.
	Category string `json:"category,omitempty"`
	// Origin identifies the producer within the category: the tool name for
	// Category "tool", the child orchestrator id for "subagent", the pane /
	// agent / humanoid name for "shell"/"chat" injections.
	Origin string `json:"origin,omitempty"`
	// Summary is a short human-readable digest of the turn (a step title for
	// tool calls, an LLM/deterministic summary for sub-agent results). Used by
	// low-bandwidth renderers (Slack step flow) instead of full Content.
	Summary string `json:"summary,omitempty"`
	// TimestampMs records when the turn was appended (unix milliseconds).
	TimestampMs int64 `json:"timestamp_ms,omitempty"`
	// Thinking carries the assistant turn's extended-thinking blocks so they
	// can be replayed on the next request (the Messages API requires thinking
	// blocks to precede tool_use blocks in replayed assistant turns when
	// extended thinking is enabled).
	Thinking []ThinkingBlock `json:"thinking,omitempty"`

	// ProviderName / Model record WHICH model produced an assistant turn.
	//
	// A conversation outlives the model that started it: `##llm select` swaps
	// the provider on a live orchestrator, so the next model inherits a
	// transcript of assistant turns it did not write. Without attribution it
	// reads them as its own — one model answered "I'm ChatGPT", the next was
	// asked the same question, saw that answer in its own voice, and
	// "corrected" itself. Role alone cannot express this: every assistant turn
	// is role "assistant" no matter who wrote it.
	//
	// Empty on user/tool turns, and on assistant turns recorded before this
	// existed — treat empty as "unknown", never as "mine".
	ProviderName string `json:"provider_name,omitempty"`
	Model        string `json:"model,omitempty"`

	// SubAgentCalls is set only on the tool-result turn of a sub_agent call:
	// the tool calls the child orchestrator made, in order, each nested call
	// carrying its own children. It is an AUDIT record for consumers that
	// account for every tool call of a turn (the meeting audit ledger) — the
	// child's steps otherwise reach them only over the asynchronous step
	// stream. It is never rendered into a provider request: the model sees
	// exactly the Content it saw before this field existed.
	//
	// In-process only (json:"-"), like MsgOrchestratorDone.Conversation: the
	// audit reconciles from the orchestrator's in-process result, so the
	// field is never persisted (session KV), restored, or sent over NATS
	// (##hop fork) — the child calls' titles carry tool targets that none of
	// those flows carried before.
	SubAgentCalls []SubAgentCall `json:"-"`
}

ConversationTurn represents a single turn in a multi-turn conversation.

The un-tagged fields keep their historical Go-default JSON names (Role, Content, …) so conversations persisted to KV before categorisation existed still restore. The tagged fields below are additive metadata.

func ConversationFromTurns

func ConversationFromTurns(turns []Turn) []ConversationTurn

ConversationFromTurns converts neutral turns back into the shim shape.

Turns produced by TurnsFromConversation come back 1:1 (exactly equal up to the documented canonicalizations C1/C2). Arbitrary neutral turns are converted totally: each tool_result block yields its own tool-role ConversationTurn — a turn carrying several results fans out into several shim turns, preserving order — and text/image blocks that FOLLOW a tool_result attach to that result's turn as ContentBlocks (attachments).

func HealConversationHead

func HealConversationHead(turns []ConversationTurn) []ConversationTurn

HealConversationHead returns turns with leading entries removed up to the first genuine user turn (Role == "user"). The Anthropic Messages API requires the first message to be a user message and rejects a tool_result block whose tool_use is not in the previous message ("unexpected tool_use_id found in tool_result blocks"). A conversation window cut between an assistant tool_use and its "tool" result — e.g. by a naive tail-trim — leaves such a dangling head, so it is trimmed here. Conversations that already begin with a user turn (and degenerate ones with no user turn at all) are returned unchanged.

func HealToolPairing

func HealToolPairing(turns []ConversationTurn) []ConversationTurn

HealToolPairing returns a transcript in which every assistant tool_use is answered by a matching tool turn in the run of consecutive tool turns that immediately follows it, and no tool turn exists without such an owner.

  • Missing results are synthesized (IsError=true, healedToolResultNote) in the assistant turn's ToolCalls order, after any real results.
  • Orphaned tool turns (no owning assistant tool_use immediately before) are dropped.
  • Duplicate results for the same id keep the first and drop the rest.

Non-tool turns are passed through untouched. The input slice is not mutated.

func (ConversationTurn) Attribution added in v0.2.3

func (t ConversationTurn) Attribution() string

Attribution renders a turn's producer for display, e.g. "openai (gpt-5.6-luna)". Empty when the turn carries no attribution.

func (ConversationTurn) ProducedBy added in v0.2.3

func (t ConversationTurn) ProducedBy(providerName, model string) bool

ProducedBy reports whether this turn was written by the given provider/model pair. An unattributed turn belongs to nobody: it answers false for every model, which is what keeps a legacy transcript from being claimed.

type ImageSource

type ImageSource struct {
	Type      string `json:"type"`                 // "base64" | "url" | "file_id"
	MediaType string `json:"media_type,omitempty"` // e.g. "image/png"
	Data      string `json:"data,omitempty"`       // base64-encoded when Type=="base64"
	URL       string `json:"url,omitempty"`        // when Type=="url"
	FileID    string `json:"file_id,omitempty"`    // when Type=="file_id"
}

ImageSource carries the payload for an image content block. Anthropic accepts base64-inline (Type "base64"), URL (Type "url"), or File-ID (Type "file_id"). MVP only exercises "base64".

type MaxTokensOverridable

type MaxTokensOverridable interface {
	// WithMaxTokens returns a provider identical to the receiver except for
	// its max_tokens output cap; maxTokens <= 0 keeps the receiver's cap.
	// The receiver is not mutated. Wrappers that forward the seam return nil
	// when the wrapped provider cannot honor the cap, so callers can fail
	// loudly instead of silently sending an uncapped request.
	WithMaxTokens(maxTokens int) AgenticProvider
}

MaxTokensOverridable is implemented by providers that can produce a per-request variant of themselves with a different max_tokens output cap — the seam behind ChatRequest.MaxTokens, mirroring ModelEffortOverridable. Callers type-assert; providers without the seam keep their construction-time cap (and the ChatProvider path errors loudly rather than silently dropping a requested cap).

type ModelEffortOverridable

type ModelEffortOverridable interface {
	// WithModelEffort returns a provider identical to the receiver except for
	// the non-empty overrides. The receiver is not mutated.
	WithModelEffort(model, effort string) AgenticProvider
}

ModelEffortOverridable is implemented by providers that can produce a per-run variant of themselves with a different model and/or effort — the seam behind recipe-level `do.model`/`do.effort` (and the finalizer seat). Callers type-assert; providers that don't implement it simply run with their configured defaults.

type ModelFallback

type ModelFallback struct {
	Secondary string // e.g. "claude-3-5-haiku-20241022"
}

ModelFallback configures an optional one-shot fallback when the primary model's retry budget is exhausted. The secondary model is tried once with the same arguments before returning the final error to the caller.

Set Secondary to "" to disable the fallback (default).

type OpenAIAgenticProvider

type OpenAIAgenticProvider struct {
	// contains filtered or unexported fields
}

OpenAIAgenticProvider implements AgenticProvider against any OpenAI-compatible Chat Completions endpoint — OpenAI itself, and crucially local Ollama (http://127.0.0.1:11434/v1), which powers rysh's air-gapped mode (design 002).

It translates rysh's neutral ConversationTurn / ToolSpec shapes to the OpenAI dialect at the edge, so the agentic loop (orchestrator) stays provider-neutral with no refactor. SSE streaming lives in openai_streaming.go (CompleteWithToolsStream), which satisfies the same StreamingProvider interface the Claude provider implements.

func NewOpenAIAgenticProvider

func NewOpenAIAgenticProvider(name, apiKey, apiURL, model string, maxTokens int) *OpenAIAgenticProvider

NewOpenAIAgenticProvider builds a provider. name distinguishes ollama vs openai for usage attribution; apiURL should include the /v1 suffix.

func (*OpenAIAgenticProvider) Chat

Chat implements ChatProvider natively: neutral turns -> wire request -> neutral response, in whichever dialect this endpoint speaks (OpenAI proper takes Responses, every other OpenAI-compatible endpoint takes Chat Completions — the same split CompleteWithTools applies, so the two paths do not diverge). Per-request Model/MaxTokens overrides route through the same seams (and error semantics) as the adapter path.

func (*OpenAIAgenticProvider) ChatStream

ChatStream implements ChatProvider natively; SSE consumption, the StreamEvent sequence, and the HTTP 400 non-streaming degradation are the same code paths CompleteWithToolsStream uses, per dialect.

func (*OpenAIAgenticProvider) ChatStreamSupported

func (c *OpenAIAgenticProvider) ChatStreamSupported() bool

ChatStreamSupported reports genuine SSE streaming support.

func (*OpenAIAgenticProvider) Complete

func (c *OpenAIAgenticProvider) Complete(ctx context.Context, prompt string) (string, error)

Complete is the single-turn shim (no tools).

func (*OpenAIAgenticProvider) CompleteWithTools

func (c *OpenAIAgenticProvider) CompleteWithTools(ctx context.Context, conversation []ConversationTurn, tools []ToolSpec, systemPrompt string) (*AgenticResponse, error)

CompleteWithTools runs one turn with tool support, in whichever dialect this endpoint speaks: OpenAI proper takes the Responses API (openai_responses.go), every other OpenAI-compatible endpoint takes Chat Completions.

func (*OpenAIAgenticProvider) CompleteWithToolsStream

func (c *OpenAIAgenticProvider) CompleteWithToolsStream(
	ctx context.Context,
	conversation []ConversationTurn,
	tools []ToolSpec,
	systemPrompt string,
	cb StreamCallback,
) (*AgenticResponse, error)

CompleteWithToolsStream is the streaming equivalent of CompleteWithTools, and routes by dialect the same way it does: OpenAI proper streams from the Responses API (openai_responses_streaming.go), every other OpenAI-compatible endpoint streams Chat Completions from here. It sets stream=true (plus stream_options.include_usage) on the request body and consumes the SSE response, invoking cb on each event and returning the assembled AgenticResponse once the stream completes.

Setup failures degrade gracefully, in two steps. An HTTP 400 is the signature of an OpenAI-compatible server rejecting the payload, but the usual culprit is stream_options alone — older Ollama builds and strict proxies 400 on the field while streaming itself works. Abandoning streaming on that first 400 would silently cost air-gapped users (design 002's whole point) their incremental output, so retry once WITHOUT stream_options and only fall through to the non-streaming path if that is refused too. A genuinely bad payload still surfaces the same error CompleteWithTools would return. Other statuses return to the orchestrator's retry loop: re-sending an unstreamed 401 or 429 fails identically.

func (*OpenAIAgenticProvider) Model

func (c *OpenAIAgenticProvider) Model() string

func (*OpenAIAgenticProvider) Name

func (c *OpenAIAgenticProvider) Name() string

func (*OpenAIAgenticProvider) SetRetryPolicy added in v0.2.3

func (c *OpenAIAgenticProvider) SetRetryPolicy(p RetryPolicy)

SetRetryPolicy installs a custom retry/backoff policy. Pass RetryPolicy{MaxAttempts: 1} to disable retries entirely. Mirrors the Claude provider's setter.

func (*OpenAIAgenticProvider) WithMaxTokens

func (c *OpenAIAgenticProvider) WithMaxTokens(maxTokens int) AgenticProvider

WithMaxTokens returns a copy with the max_tokens cap applied (maxTokens <= 0 keeps the configured cap). Satisfies the MaxTokensOverridable seam behind ChatRequest.MaxTokens; the cap reaches the wire via buildRequest on both the one-shot and streaming paths.

func (*OpenAIAgenticProvider) WithModelEffort

func (c *OpenAIAgenticProvider) WithModelEffort(model, _ string) AgenticProvider

WithModelEffort returns a copy pinned to model (effort is a no-op for the OpenAI dialect). Satisfies the ModelEffortOverridable seam used by WithSessionDefaults so non-Claude providers slot into the same wrapper.

type Provider

type Provider interface {
	Name() string
	Complete(ctx context.Context, prompt string) (string, error)
}

Provider is the basic single-turn LLM interface.

type RetryPolicy

type RetryPolicy struct {
	// MaxAttempts is the upper bound on total attempts (including the first).
	// 1 means "try once, no retries". Default 5.
	MaxAttempts int

	// BaseDelay is the starting backoff window. Each subsequent attempt
	// doubles up to MaxDelay. Default 1 second.
	BaseDelay time.Duration

	// MaxDelay caps the backoff between attempts. Default 30 seconds.
	MaxDelay time.Duration

	// Jitter adds randomised fuzz to each backoff so synchronised clients
	// don't herd. Expressed as a fraction of the computed delay; 0.2 means
	// ±20%. Default 0.2.
	Jitter float64

	// HonourRetryAfter, when true, overrides the computed backoff with the
	// server-sent Retry-After header (if shorter than MaxDelay). Default true.
	HonourRetryAfter bool
}

RetryPolicy controls how transient errors are retried. The zero value disables retries entirely (single attempt only); production code should construct one via DefaultRetryPolicy() and customise from there.

func DefaultRetryPolicy

func DefaultRetryPolicy() RetryPolicy

DefaultRetryPolicy returns the tuned defaults used in production. Callers can customise fields without re-declaring the whole struct.

type StaticAgenticProvider

type StaticAgenticProvider struct {
	Response string
}

StaticAgenticProvider returns a fixed response. Used when no API key is set.

func (*StaticAgenticProvider) Complete

func (s *StaticAgenticProvider) Complete(_ context.Context, _ string) (string, error)

func (*StaticAgenticProvider) CompleteWithTools

func (s *StaticAgenticProvider) CompleteWithTools(
	_ context.Context,
	_ []ConversationTurn,
	_ []ToolSpec,
	_ string,
) (*AgenticResponse, error)

func (*StaticAgenticProvider) Name

func (s *StaticAgenticProvider) Name() string

type StopReason

type StopReason string

StopReason indicates why the model stopped generating.

const (
	StopReasonEndTurn   StopReason = "end_turn"
	StopReasonToolUse   StopReason = "tool_use"
	StopReasonMaxTokens StopReason = "max_tokens"
	StopReasonStopSeq   StopReason = "stop_sequence"
)

type StreamCallback

type StreamCallback func(StreamEvent)

StreamCallback is invoked once per parsed event. Implementations should return quickly; a slow callback blocks the SSE reader. Pass nil to stream-and-collect without intermediate updates (useful for tests).

type StreamEvent

type StreamEvent struct {
	Type        StreamEventType
	Index       int        // for *content_block_* events
	Text        string     // text_delta
	ToolUseID   string     // content_block_start (tool_use)
	ToolUseName string     // content_block_start (tool_use)
	PartialJSON string     // tool_use_delta
	Thinking    string     // thinking_delta (Phase 3 F)
	StopReason  StopReason // message_delta
	Usage       Usage      // message_start / message_delta
}

StreamEvent is the single event type delivered to a StreamCallback. Only the fields relevant to a given Type are populated.

type StreamEventType

type StreamEventType string

StreamEventType enumerates the kinds of stream events delivered to a StreamCallback. The set is intentionally small: a faithful low-fidelity mapping of Anthropic's SSE events to a single union type. Callers don't need to know how Anthropic frames its events; they need only what blocks arrived and what text/tool_use deltas accompanied them.

const (
	StreamEventMessageStart      StreamEventType = "message_start"
	StreamEventContentBlockStart StreamEventType = "content_block_start"
	StreamEventTextDelta         StreamEventType = "text_delta"
	StreamEventToolUseDelta      StreamEventType = "tool_use_delta"
	StreamEventThinkingDelta     StreamEventType = "thinking_delta" // Phase 3 F
	StreamEventContentBlockStop  StreamEventType = "content_block_stop"
	StreamEventMessageDelta      StreamEventType = "message_delta"
	StreamEventMessageStop       StreamEventType = "message_stop"
)

type StreamingProvider

type StreamingProvider interface {
	CompleteWithToolsStream(
		ctx context.Context,
		conversation []ConversationTurn,
		tools []ToolSpec,
		systemPrompt string,
		cb StreamCallback,
	) (*AgenticResponse, error)
}

StreamingProvider is the optional capability extension that signals a provider supports incremental token delivery. Orchestrators check via type assertion and fall back to CompleteWithTools when absent.

type SubAgentCall added in v0.2.11

type SubAgentCall struct {
	ID             string         `json:"id"`
	Name           string         `json:"name"`
	Title          string         `json:"title"`
	IsError        bool           `json:"is_error"`
	Ended          bool           `json:"ended"`
	Interrupted    bool           `json:"interrupted,omitempty"`
	OrchestratorID string         `json:"orchestrator_id,omitempty"`
	Calls          []SubAgentCall `json:"calls,omitempty"`
}

SubAgentCall is one tool call a sub-agent made, as recorded in its final conversation. Ended is true when the conversation holds the call's result turn; IsError mirrors that result turn, and Interrupted marks the synthetic "[error kind=cancelled]" result CloseDanglingToolCalls appends for a call cut off by an interrupt (it may never have run). For a nested sub_agent call, OrchestratorID is the grandchild orchestrator (the OrchestratorID on its step events) and Calls holds that orchestrator's calls.

type TextBlock

type TextBlock struct {
	Text string
}

TextBlock is a text segment in the model response.

type ThinkingBlock

type ThinkingBlock struct {
	Text      string `json:"thinking,omitempty"`
	Signature string `json:"signature,omitempty"`
}

ThinkingBlock is a hidden-reasoning content block in the model response. Carries the raw thinking text plus an opaque signature the API uses to verify thinking provenance across multi-turn exchanges. The signature MUST be echoed verbatim if a subsequent turn needs to reference the same thinking block.

type ThinkingConfig

type ThinkingConfig struct {
	// Type is "adaptive" (recommended, 4.6+ models) or "enabled" (legacy).
	Type string `json:"type"`

	// BudgetTokens is the legacy fixed thinking budget. Only set with
	// Type == "enabled" on pre-4.6 models; omitted from the wire form
	// otherwise (adaptive mode has no budget field).
	BudgetTokens int `json:"budget_tokens,omitempty"`
}

ThinkingConfig is sent as `request.thinking` to enable extended thinking. Two modes:

{"type": "adaptive"}                          — Claude 4.6+ models (recommended);
                                                the model decides when/how much to think.
{"type": "enabled", "budget_tokens": 8192}    — legacy pre-4.6 models only. This
                                                form is REJECTED with a 400 on
                                                Opus 4.7+ / Sonnet 5 — do not use
                                                it on current models.

Use AdaptiveThinkingConfig() for current models; LegacyThinkingConfig(n) only when explicitly targeting a pre-4.6 model.

func AdaptiveThinkingConfig

func AdaptiveThinkingConfig() *ThinkingConfig

AdaptiveThinkingConfig returns the adaptive thinking config recommended for Claude 4.6+ models: the model dynamically decides when and how much to think, and interleaved thinking between tool calls is enabled automatically.

func DefaultThinkingConfig

func DefaultThinkingConfig() *ThinkingConfig

DefaultThinkingConfig returns the recommended thinking config for current models. Adaptive since the Claude 4.6 family; the old fixed-budget form is available via LegacyThinkingConfig for pre-4.6 models.

func LegacyThinkingConfig

func LegacyThinkingConfig(budgetTokens int) *ThinkingConfig

LegacyThinkingConfig returns the pre-4.6 fixed-budget thinking config. Rejected with a 400 by Opus 4.7+ / Sonnet 5 — use AdaptiveThinkingConfig unless explicitly targeting an older model.

type ToolCall

type ToolCall struct {
	// ID is the provider-assigned call identifier; the matching ToolResult
	// echoes it as CallID.
	ID string `json:"id,omitempty"`
	// Name is the tool name.
	Name string `json:"name,omitempty"`
	// ParamsJSON is the raw JSON arguments object.
	ParamsJSON json.RawMessage `json:"params_json,omitempty"`
}

ToolCall is a tool invocation requested by the model, in neutral shape.

type ToolCallRequest

type ToolCallRequest struct {
	ID    string          // unique id assigned by the API
	Name  string          // tool name
	Input json.RawMessage // raw JSON input arguments
}

ToolCallRequest is a single tool invocation requested by the model.

type ToolResult

type ToolResult struct {
	// CallID references the ToolCall.ID this result answers.
	CallID string `json:"call_id,omitempty"`
	// Content is the textual result payload.
	Content string `json:"content,omitempty"`
	// IsError marks the result as a failure the model should read as such.
	IsError bool `json:"is_error,omitempty"`
}

ToolResult is the outcome of one tool call, in neutral shape.

type ToolSpec

type ToolSpec struct {
	Name        string          // tool name
	Description string          // human-readable description
	Parameters  json.RawMessage // JSON Schema for the tool input
}

ToolSpec defines a tool that the model may invoke.

type Turn

type Turn struct {
	Role   TurnRole `json:"role"`
	Blocks []Block  `json:"blocks,omitempty"`

	// Category classifies the turn's origin (TurnCategory* constants).
	Category string `json:"category,omitempty"`
	// Origin identifies the producer within the category.
	Origin string `json:"origin,omitempty"`
	// Summary is a short human-readable digest of the turn.
	Summary string `json:"summary,omitempty"`
	// TimestampMs records when the turn was appended (unix milliseconds).
	TimestampMs int64 `json:"timestamp_ms,omitempty"`
}

Turn is one neutral conversation turn: a role plus ordered content blocks. The metadata fields (Category/Origin/Summary/TimestampMs) are rysh's own turn bookkeeping — provider-independent, carried through conversion verbatim. JSON tags are chosen for stability of any FUTURE persistence of neutral turns; nothing persists Turn today (session KV stores ConversationTurn), and the tags deliberately do not mirror any provider's wire format.

func TurnFromConversation

func TurnFromConversation(ct ConversationTurn) Turn

TurnFromConversation converts one shim turn into a neutral Turn.

Block order is canonical replay order: thinking blocks first, then the turn's text, then tool calls (assistant) / the tool result (tool role), then ContentBlocks (attachments). For tool-role turns, Content is the tool RESULT payload and lands inside the ToolResult block, not in a text block.

func TurnsFromConversation

func TurnsFromConversation(conv []ConversationTurn) []Turn

TurnsFromConversation converts a shim conversation into neutral turns, 1:1.

type TurnRole

type TurnRole string

TurnRole identifies the speaker of a neutral Turn. The underlying string values match ConversationTurn.Role so conversion never invents roles; unknown roles pass through verbatim (the shim ignores them on the wire, and losing them here would break round-tripping).

const (
	RoleUser      TurnRole = "user"
	RoleAssistant TurnRole = "assistant"
	RoleTool      TurnRole = "tool"
)

type Usage

type Usage struct {
	InputTokens              int // non-cached input tokens billed at full rate
	OutputTokens             int // generated output tokens
	CacheReadInputTokens     int // input tokens served from the prompt cache
	CacheCreationInputTokens int // input tokens written to the prompt cache
}

Usage reports token accounting for a single API response, including prompt-cache hit/write counts. Callers use the input figures to drive context-window budgeting and compaction.

func (Usage) NewTokens

func (u Usage) NewTokens() int

NewTokens returns the tokens genuinely processed on this call — new input (uncached + newly cached) plus generated output — but EXCLUDING cache reads, which are the re-sent conversation prefix already counted on earlier calls. Summing NewTokens across calls approximates the run's unique token volume ("total reading"), growing ~linearly with real work; summing TotalInputTokens instead re-counts the whole growing conversation every call (quadratic-ish), which made a cumulative token budget fire far too early. Used for the run's token_budget accounting (NOT context-window compaction, which uses the peak TotalInputTokens).

func (Usage) TotalInputTokens

func (u Usage) TotalInputTokens() int

TotalInputTokens returns the full prompt size for this turn (uncached + cache-read + cache-write). This is the figure to compare against the model's context window when deciding whether to compact the conversation.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL