Documentation
¶
Overview ¶
claude.go is the native Anthropic Messages provider. Auth is direct API-key only (ANTHROPIC_API_KEY via x-api-key header). bee identifies itself honestly as bee/0.1; impersonating first-party clients to access subscription pricing is out of scope.
Package llm defines the Provider interface and shared event types.
Concrete provider adapters (openai_compat.go, anthropic.go) live alongside and translate the internal types in github.com/elhenro/bee/internal/types to/from each provider's wire format. The rest of the codebase only depends on this interface.
Index ¶
- func ClearModelCache()
- func ContextWindow(modelID string) int
- func IsQwen3HybridThinking(modelID string) bool
- func OllamaBaseURL(baseURL string) string
- func ProbeContextLength(ctx context.Context, name string, cfg config.ProviderConfig, modelID string) int
- func ProbeOllamaContext(ctx context.Context, client *http.Client, baseURL, modelID string) (int, error)
- func Qwen3ThinkingHint(t Thinking) string
- func RememberContextLength(modelID string, n int)
- func ResetLiveContextLengths()
- func ResetProbed()
- func SetModelsHTTPClient(c *http.Client)
- func SetProbeHTTPClient(c *http.Client)
- func SupportsThinking(modelID string) bool
- func SupportsVision(modelID string) bool
- func ThinkingApplies(modelID string) bool
- func ThinkingBudget(t Thinking) int
- type AnthropicProvider
- type ChatGPTConfig
- type ChatGPTProvider
- type ClaudeConfig
- type ClaudeProvider
- type Event
- type EventType
- type GeminiConfig
- type GeminiProvider
- type Model
- type OpenAICompatConfig
- type OpenAICompatProvider
- type Provider
- type Request
- type TextModeOptions
- type TextModeProvider
- type Thinking
- type ToolSpec
- type Usage
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ClearModelCache ¶
func ClearModelCache()
ClearModelCache drops every cached entry. Tests call this between cases.
func ContextWindow ¶
ContextWindow returns a best-effort context length (in tokens) for the given model id. Live API-learned values win over the hardcoded table. Tries the exact id, then the trailing path segment so OpenRouter ids ("anthropic/claude-sonnet-4-6") resolve to the bare model name. Returns 0 when unknown — TUI treats that as "no fill indicator".
func IsQwen3HybridThinking ¶
IsQwen3HybridThinking reports whether modelID names a Qwen3 model that supports the `/think` `/no_think` system-prompt toggle. Unlike the reasoning_effort wire path (covered by SupportsThinking), Qwen3 hybrid inference servers (omlx/lmstudio/ollama running qwen3-*-a3b, qwen3-235b, etc.) consume the toggle as a literal token in the prompt.
Excludes already-explicit thinking variants (qwq, qwen3-thinking, qwen3-reasoner): those models think unconditionally, no toggle needed.
Also excludes the coder family (qwen3-coder-*): those ship with reasoning disabled, emit no trace, and treat the toggle as a no-op. Injecting it just adds prompt noise and misreports the model as a thinker.
Heuristic: substring "qwen3" or "qwen-3" present AND no explicit thinking suffix AND not a coder variant. Matches sparse-MoE (a3b, a7b) and flagship (qwen3-235b) families.
func OllamaBaseURL ¶
OllamaBaseURL trims the trailing "/v1" segment from an openai-compat base URL so ollama's native paths (/api/show, /api/tags) resolve cleanly. Returns the input unchanged when no /v1 suffix is present.
func ProbeContextLength ¶
func ProbeContextLength(ctx context.Context, name string, cfg config.ProviderConfig, modelID string) int
ProbeContextLength returns the live context window for modelID, learning it from the provider if not already in the cache. Order:
- ContextWindow cache (live + hardcoded) — short-circuits.
- Ollama native /api/show — POST {"name": modelID}, parse model_info.
- OpenAI-compat /v1/models — populates cache for any model the server advertises a context_length for (openrouter, some lm-studio versions).
Returns the learned value (also stored via RememberContextLength) or 0 when nothing could be determined. Safe to call concurrently and repeatedly — dedupes per (provider,model).
func ProbeOllamaContext ¶
func ProbeOllamaContext(ctx context.Context, client *http.Client, baseURL, modelID string) (int, error)
ProbeOllamaContext POSTs to <baseURL>/api/show with {"name": modelID}, then extracts the model's real context length from either the `parameters` free-form string (`num_ctx <N>`) or the `model_info` map (any key ending in `.context_length`). Best-effort: returns (0, nil) when the endpoint exists but yields nothing usable, and (0, err) only on network/HTTP failures so callers can log without aborting.
func Qwen3ThinkingHint ¶
Qwen3ThinkingHint maps a resolved Thinking level into the literal toggle token Qwen3 hybrid models consume. ThinkingOff / ThinkingLow → `/no_think` (skip reasoning trace entirely). ThinkingMedium / High / Max → `/think`. Empty string means "no toggle" — caller skips injection.
func RememberContextLength ¶
RememberContextLength records a live-learned context window for modelID. Both the raw id and its trailing path segment are stored so future lookups resolve regardless of which form ContextWindow gets.
func ResetLiveContextLengths ¶
func ResetLiveContextLengths()
ResetLiveContextLengths drops every learned entry. Tests call this.
func SetModelsHTTPClient ¶
SetModelsHTTPClient replaces the package HTTP client. Tests use this to route through an httptest server with a stricter timeout.
func SetProbeHTTPClient ¶
SetProbeHTTPClient swaps the package probe client. Tests route through httptest servers.
func SupportsThinking ¶
SupportsThinking reports whether modelID belongs to a family that honors a reasoning_effort / thinking-budget request. Best-effort: substring match against thinkingModelSubstrings on both the full id and trailing path segment. Unknown models return false → ThinkingAuto resolves to off so we don't break non-reasoning providers that choke on the field.
func SupportsVision ¶
SupportsVision reports whether modelID belongs to a family that accepts image content blocks. Substring match against visionModelSubstrings on both the full id and trailing path segment. Unknown models return false → the loop's vision fallback kicks in (describe-then-inject) when one is configured.
func ThinkingApplies ¶
ThinkingApplies reports whether modelID uses a reasoning/thinking mechanism in any form — the reasoning_effort wire field (SupportsThinking families) or the Qwen3 hybrid /think system-prompt token. False means an effort level is a pure no-op for the model (e.g. qwen3-coder, plain instruct models): callers use this to avoid showing or sending effort that the model can't act on.
func ThinkingBudget ¶
ThinkingBudget maps a level to a token budget for thinking-enabled providers. Off returns 0 → caller should omit the thinking field.
Types ¶
type AnthropicProvider ¶
type AnthropicProvider struct {
// contains filtered or unexported fields
}
AnthropicProvider is a stub. The full native implementation is in claude.go. Most users should route through OpenAI-compatible paths or use the anthropic-messages wire_api config option.
func NewAnthropic ¶
func NewAnthropic(apiKeyEnv, baseURL string) *AnthropicProvider
NewAnthropic builds a stub provider. apiKeyEnv is the env var for the bearer token; baseURL defaults to https://api.anthropic.com/v1 when empty.
func (*AnthropicProvider) Name ¶
func (p *AnthropicProvider) Name() string
Name returns the static provider identifier.
type ChatGPTConfig ¶
type ChatGPTConfig struct {
// Name shows up in logs and Provider.Name(). Defaults to "chatgpt".
Name string
// BaseURL e.g. "https://chatgpt.com/backend-api/codex". Path "/responses"
// is appended automatically.
BaseURL string
// ClientID is the OAuth client id used to refresh tokens on 401.
ClientID string
// TokenEndpoint is the OAuth token URL for refresh.
TokenEndpoint string
// AccountIDHeader is the HTTP header that carries the per-account id
// (e.g. "chatgpt-account-id"). Empty disables injection.
AccountIDHeader string
// HTTPClient is overridable for tests. Defaults to a long-timeout client.
HTTPClient *http.Client
// ExtraHeaders are merged into every request.
ExtraHeaders map[string]string
}
ChatGPTConfig drives an OpenAI Responses-API provider with subscription auth. Use this for chatgpt.com/backend-api/codex (Plus/Pro/Team plans).
type ChatGPTProvider ¶
type ChatGPTProvider struct {
// contains filtered or unexported fields
}
ChatGPTProvider implements Provider against OpenAI's Responses API on the ChatGPT-subscription backend.
func (*ChatGPTProvider) Name ¶
func (p *ChatGPTProvider) Name() string
Name returns the configured display name.
type ClaudeConfig ¶
type ClaudeConfig struct {
// Name shows up in logs and Provider.Name(). Defaults to "claude".
Name string
// BaseURL e.g. "https://api.anthropic.com/v1". The "/messages" path is
// appended automatically.
BaseURL string
// EnvKey is the API-key env var (e.g. ANTHROPIC_API_KEY).
EnvKey string
// HTTPClient is overridable for tests. Defaults to a long-timeout client.
HTTPClient *http.Client
// ExtraHeaders are merged into every request.
ExtraHeaders map[string]string
}
ClaudeConfig configures the provider. API-key only — the Bearer/OAuth subscription path was removed.
type ClaudeProvider ¶
type ClaudeProvider struct {
// contains filtered or unexported fields
}
ClaudeProvider implements Provider against Anthropic's native Messages API.
func (*ClaudeProvider) Name ¶
func (p *ClaudeProvider) Name() string
Name returns the configured display name.
type Event ¶
type Event struct {
Type EventType
Delta string // for EventTextDelta
ToolUse *types.ToolUse // for EventToolUse
StopReason string // for EventDone
Err error // for EventError
Usage *Usage // optional, on EventDone
}
Event is a streamed token, tool call, or terminal signal from a provider.
type GeminiConfig ¶
GeminiConfig configures the native Gemini provider. BaseURL defaults to Google's generative-language v1beta endpoint; APIKey is appended as a query param (Google's recommended auth path for this API).
type GeminiProvider ¶
type GeminiProvider struct {
// contains filtered or unexported fields
}
GeminiProvider streams from Google's generative-language API via :streamGenerateContent?alt=sse.
func NewGemini ¶
func NewGemini(cfg GeminiConfig) *GeminiProvider
NewGemini builds a provider. APIKey is read from GEMINI_API_KEY env when the config field is blank, mirroring how callers wire other providers.
func (*GeminiProvider) Name ¶
func (p *GeminiProvider) Name() string
Name returns the static provider identifier.
func (*GeminiProvider) Stream ¶
Stream issues streamGenerateContent and emits Events on the returned channel. Caller must read until close; ctx cancellation closes the channel after an EventError. Gemini SSE: each line `data: <json>\n\n`; the model terminates by emitting `finishReason: "STOP"` (no separate `[DONE]` marker, but we handle one defensively).
type Model ¶
Model is a single entry in a provider's catalogue.
func FetchChatGPTModels ¶
FetchChatGPTModels GETs /backend-api/codex/models?client_version=. The server returns a plan-filtered model list once authed; without a high enough client_version it only surfaces a single legacy entry. We send 9.9.9 to opt into the full set the account is entitled to.
func ListModels ¶
ListModels returns the model catalogue for the given provider. For anthropic-wire providers or empty base URLs we return a curated hardcoded list — Anthropic's /models endpoint is gated and not all wire-compat servers expose one. Results are cached per-name with a 10-minute TTL.
type OpenAICompatConfig ¶
type OpenAICompatConfig struct {
// Name shows up in logs and Provider.Name(). Defaults to "openai-compat".
Name string
// BaseURL e.g. "https://openrouter.ai/api/v1". The "/chat/completions"
// suffix is appended automatically.
BaseURL string
// EnvKey is the environment variable holding the bearer token. Optional —
// when blank (local servers like Ollama), no Authorization header is sent.
EnvKey string
// HTTPClient is overridable for tests. Defaults to a long-timeout client.
HTTPClient *http.Client
// ExtraHeaders are merged into every request. Useful for OpenRouter's
// HTTP-Referer + X-Title attribution headers.
ExtraHeaders map[string]string
// StallTimeout overrides the per-chunk SSE inactivity timer. 0 falls back
// to the package default (streamStallTimeout). Negative disables the
// watchdog (test-only).
StallTimeout time.Duration
// ChatTemplateKwargs flows into the MLX/vllm `chat_template_kwargs` body
// field. Lets local servers flip Qwen3 template switches that change tool
// emission shape. Nil = omit. Per-provider (TOML `chat_template_kwargs`).
ChatTemplateKwargs map[string]any
// ReportsCost opts the provider into a request flag asking for actual spend
// in the usage block. Off by default; strict endpoints would reject the
// extra field, so only set it for services that return real per-call cost.
ReportsCost bool
}
OpenAICompatConfig configures a chat-completions-style provider. The same implementation covers OpenRouter, OpenAI, DeepSeek, Groq, Ollama, LM Studio, Together, Fireworks — any service that speaks OpenAI's wire format.
type OpenAICompatProvider ¶
type OpenAICompatProvider struct {
// contains filtered or unexported fields
}
OpenAICompatProvider implements Provider against an OpenAI-compatible API.
func NewOpenAICompat ¶
func NewOpenAICompat(cfg OpenAICompatConfig) *OpenAICompatProvider
NewOpenAICompat builds a provider. Missing fields get sensible defaults.
func (*OpenAICompatProvider) Name ¶
func (p *OpenAICompatProvider) Name() string
Name returns the configured display name.
func (*OpenAICompatProvider) Stream ¶
Stream issues a chat completion and emits Events on the returned channel. Caller must read the channel until closed to avoid leaking the goroutine; ctx cancellation closes the channel after an EventError.
Retries the pre-stream phase (DNS/TCP/TLS failures, 408, 429, 5xx) with exp-backoff. Once a 2xx response lands and streamLoop starts emitting, no further retries — replaying would duplicate tokens.
type Provider ¶
type Provider interface {
Name() string
Stream(ctx context.Context, req Request) (<-chan Event, error)
}
Provider streams a chat completion. The returned channel is closed when the turn ends (stop reason emitted) or the context is canceled.
type Request ¶
type Request struct {
Model string
System string
Messages []types.Message
Tools []ToolSpec
MaxTokens int
Temperature float64
// TopP pins nucleus sampling. 0 = use provider default (omit on wire).
// Set by the active profile (tiny pins 0.8 for 4-bit MoE).
TopP float64
// Stop is the optional list of stop sequences. textmode wraps tool calls
// in `<bash>{…}</bash>`-style envelopes; passing the closing tag halts
// decode immediately after one tool call, saving 50–300 tokens per turn.
Stop []string
Stream bool
// Thinking selects the extended-reasoning budget for providers that
// support it. Off means omit the field entirely.
Thinking Thinking
// ChatTemplateKwargs overrides/merges with the provider's static
// chat_template_kwargs for this request. Used to flip Qwen3's
// enable_thinking switch per-turn from the effort level. Nil = no override.
ChatTemplateKwargs map[string]any
}
Request is the agent-owned shape of a chat call. Adapters translate to the provider's wire format.
type TextModeOptions ¶
type TextModeOptions struct {
// ExtraHint is appended verbatim after the auto-generated tool block.
// Useful for caveman-style brevity nudges.
ExtraHint string
}
TextModeOptions tunes the wrapper. All fields are optional.
type TextModeProvider ¶
type TextModeProvider struct {
// contains filtered or unexported fields
}
TextModeProvider wraps an inner Provider to bypass the native tool_calls channel. Many small local models (llama3.1:8b, gemma3, phi3) silently ignore function-calling deltas but reliably emit inline XML when shown one example. The wrapper:
- strips Request.Tools and injects a text instruction block describing each tool plus the `<tool>{...}</tool>` envelope,
- buffers assistant text deltas,
- scans the buffered text after the stream ends for tool-call tags, synthesizes EventToolUse for each, and emits the cleaned text.
Why a wrapper instead of a separate provider: every existing adapter (openai_compat, chatgpt, claude, gemini) gains XML-mode for free.
func NewTextMode ¶
func NewTextMode(inner Provider, opts TextModeOptions) *TextModeProvider
NewTextMode wraps inner with the text/XML tool-call fallback.
func (*TextModeProvider) Name ¶
func (p *TextModeProvider) Name() string
Name forwards to inner with a "+textmode" suffix so logs/UIs can tell.
func (*TextModeProvider) Stream ¶
Stream injects the text-tool instruction block, nils Tools, then runs the inner stream. Tool-call extraction happens at EventDone.
Side-LLM calls (mode classifier, recap, compact) pass req.Tools == nil. In that case skip injection entirely — pumping a `## Tools (text format)` block with an empty tool list into a classifier/recap prompt pollutes the instruction and pushes small models toward emitting spurious XML.
type Thinking ¶
type Thinking string
Thinking enumerates the supported extended-reasoning levels. Adapters map these to provider-specific fields (Anthropic budget_tokens, OpenAI reasoning_effort).
const ( // ThinkingAuto = "medium when model supports reasoning, off otherwise". // Resolve with ResolveThinking before sending to providers — wire layers // only understand off/low/medium/high. ThinkingAuto Thinking = "auto" ThinkingOff Thinking = "off" ThinkingLow Thinking = "low" ThinkingMedium Thinking = "medium" ThinkingHigh Thinking = "high" // ThinkingMax pushes the reasoning budget to the provider's practical // ceiling. Wire layers without a distinct "max" tier (OpenAI's // reasoning_effort) clamp to "high"; budget-token providers (Anthropic, // Gemini) get a much larger budget than High. ThinkingMax Thinking = "max" )
func ParseThinking ¶
ParseThinking accepts "auto"/"off"/"low"/"medium"/"high"/"max" (case insensitive) and returns the canonical Thinking value. Unknown strings return ThinkingOff.
func ResolveThinking ¶
ResolveThinking turns ThinkingAuto into ThinkingMedium for reasoning-capable models and ThinkingOff for everything else. Non-auto values pass through unchanged. Call before building a Request — provider adapters only see the resolved level, never the sentinel.
type ToolSpec ¶
type ToolSpec struct {
Name string `json:"name"`
Description string `json:"description"`
PromptSnippet string `json:"-"`
Schema map[string]any `json:"schema"`
}
ToolSpec is the agent-side advertisement of a tool to the model.
Description goes to the provider's tool-spec (API) — full sentence is fine. PromptSnippet is the short prompt-manifest line (≤60 chars). When PromptSnippet is empty, the prompt manifest falls back to the first line of Description, truncated by the profile's ToolDescChars budget.
type Usage ¶
type Usage struct {
InputTokens int
OutputTokens int
// CachedTokens is the cached prompt-token count, when the provider reports
// it. Zero otherwise — most providers omit it.
CachedTokens int
// CostUSD is the provider-reported actual spend for this turn, when the
// service returns it (some routed aggregators do). Zero when unreported;
// callers then fall back to a static price-table estimate.
CostUSD float64
}
Usage captures token accounting reported by the provider.
Source Files
¶
- anthropic.go
- chatgpt.go
- chatgpt_auth.go
- chatgpt_models.go
- chatgpt_request.go
- chatgpt_stream.go
- claude.go
- claude_stream.go
- gemini.go
- http.go
- models.go
- models_cache.go
- models_hardcoded.go
- ollama_probe.go
- openai_compat.go
- openai_compat_auth.go
- openai_compat_stream.go
- probe.go
- provider.go
- textmode.go
- textmode_parse.go
- thinking_hybrid.go
- vision_caps.go
Directories
¶
| Path | Synopsis |
|---|---|
|
Package mockprov is a scripted, deterministic llm.Provider for tests.
|
Package mockprov is a scripted, deterministic llm.Provider for tests. |
|
anthropic_messages.go covers Anthropic's native Messages API (POST /v1/messages, SSE streaming).
|
anthropic_messages.go covers Anthropic's native Messages API (POST /v1/messages, SSE streaming). |