Documentation
¶
Overview ¶
Package agent_api: LM Studio model-list discovery.
LM Studio (≥ 0.3.6) exposes a native REST API at /api/v0/models that reports max_context_length and load state per model. Older versions and the OpenAI- compatible layer (/v1/models) don't carry context length, so we hit v0 first and fall back to v1 with a 32k default when v0 is unavailable or reports an unloaded/missing-context entry.
Local-only by design — LM Studio's published endpoint is 127.0.0.1:1234. LMSTUDIO_BASE_URL is honored for users running a remote LM Studio server.
Package agent_api: wire-type structs for Ollama local API communication (split from ollama_local.go)
Package agent_api: Ollama local client synchronous API methods (split from ollama_local.go)
Package agent_api: Ollama local client constructors, environment setup, and request-building (split from ollama_local.go)
Package agent_api: HTTP transport for Ollama local API (split from ollama_local.go)
Package agent_api: Ollama streaming response helpers, converter functions, and streaming API (split from ollama_local.go)
Package api provides API types used across all providers.
Seed canonical types (github.com/sprout-foundry/seed/core) are imported via type aliases. Sprout consumes these types — it does not define them.
Package api — vision capabilities table (AUDIT-GAP-2 / SP-103-D3).
Provides a structured per-provider view of the vision limits that historically lived in a single binary SupportsVision() flag plus ad-hoc constants scattered across resize / batching code paths. Downstream consumers (image resize at SP-103-B2, batch splitting at SP-103-D2) read this table instead of using a one-size 1536px cap.
Index ¶
- Constants
- Variables
- func CalculateOutputBudget(contextLimit int, inputTokens int) (int, bool)
- func CalculateOutputBudgetAnchored(contextLimit, anchoredInput, heuristicInput int) (int, bool)
- func ClassifyEligibleRoles(m ModelInfo) []string
- func CostFromJSON(body []byte) (float64, bool)
- func EstimateInputTokens(messages []Message, tools []Tool) int
- func EstimateMessagesTokens(messages []Message) int
- func EstimateTokens(text string) int
- func FormatHTTPResponseError(statusCode int, headers http.Header, body []byte) error
- func GetCanonicalModelsForProvider(ctx context.Context, clientType ClientType) ([]modelcontract.CanonicalModel, error)
- func GetProviderName(clientType ClientType) string
- func IsProviderAvailable(provider ClientType) bool
- func ModelCachedPricingPerMillion(entry map[string]any) float64
- func ModelPricingPerMillion(entry map[string]any) (inputPerMillion, outputPerMillion float64)
- func RecoverInlineToolCalls(resp *ChatResponse, tools []Tool)
- func ResetPricingResolver()
- func ResolveModelPricing(provider, model string) (inputPerM, outputPerM, cachedPerM float64, ok bool)
- func SeedPricingForTest(provider, model string, inputPerM, outputPerM, cachedPerM float64)
- func UsageCost(u ChatUsage) float64
- type BaseProvider
- func (p *BaseProvider) EstimateCost(promptTokens, completionTokens int, model string) float64
- func (p *BaseProvider) GetEndpoint() string
- func (p *BaseProvider) GetModel() string
- func (p *BaseProvider) GetName() string
- func (p *BaseProvider) GetType() ClientType
- func (p *BaseProvider) IsDebug() bool
- func (p *BaseProvider) MakeAuthRequest(ctx context.Context, method, url string, body io.Reader) (*http.Request, error)
- func (p *BaseProvider) SetDebug(debug bool)
- func (p *BaseProvider) SetModel(model string) error
- func (p *BaseProvider) SupportsConversationalVision() bool
- func (p *BaseProvider) SupportsReasoning() bool
- func (p *BaseProvider) SupportsStreaming() bool
- func (p *BaseProvider) SupportsTools() bool
- func (p *BaseProvider) SupportsVision() bool
- func (p *BaseProvider) VisionCapabilities() VisionCapabilities
- type ChatChoice
- type ChatRequest
- type ChatResponse
- type ChatUsage
- type Choice
- type ClientInterface
- func NewDeepInfraClientWrapper(model string) (ClientInterface, error)
- func NewOpenRouterClientWrapper(model string) (ClientInterface, error)
- func NewUnifiedClient(clientType ClientType) (ClientInterface, error)
- func NewUnifiedClientWithModel(clientType ClientType, model string) (ClientInterface, error)
- type ClientType
- type HTTPClient
- type HarmonyFormatter
- func (h *HarmonyFormatter) AddReturnToken(response string) string
- func (h *HarmonyFormatter) ConvertReturnToEnd(conversation string) string
- func (h *HarmonyFormatter) FormatMessagesForCompletion(messages []Message, tools []Tool, opts *HarmonyOptions) string
- func (h *HarmonyFormatter) StripReturnToken(response string) string
- type HarmonyOptions
- type ImageData
- type Message
- type ModelDetails
- type ModelInfo
- type ModelSelection
- type ModelsListInterface
- type OllamaLocalClient
- func (c *OllamaLocalClient) CheckConnection() error
- func (c *OllamaLocalClient) GetCachedModel(name string) (int, bool)
- func (c *OllamaLocalClient) GetModel() string
- func (c *OllamaLocalClient) GetModelContextLimit() (int, error)
- func (c *OllamaLocalClient) GetProvider() string
- func (c *OllamaLocalClient) GetVisionModel() string
- func (c *OllamaLocalClient) ListModels(ctx context.Context) ([]ModelInfo, error)
- func (c *OllamaLocalClient) SendChatRequest(ctx context.Context, messages []Message, tools []Tool, reasoning string, ...) (*ChatResponse, error)
- func (c *OllamaLocalClient) SendChatRequestStream(ctx context.Context, messages []Message, tools []Tool, reasoning string, ...) (*ChatResponse, error)
- func (c *OllamaLocalClient) SendVisionRequest(ctx context.Context, messages []Message, tools []Tool, reasoning string, ...) (*ChatResponse, error)
- func (c *OllamaLocalClient) SetDebug(debug bool)
- func (c *OllamaLocalClient) SetModel(model string) error
- func (c *OllamaLocalClient) SupportsConversationalVision() bool
- func (c *OllamaLocalClient) SupportsVision() bool
- func (c *OllamaLocalClient) VisionCapabilities() VisionCapabilities
- type Provider
- type ProviderAdapter
- func (a *ProviderAdapter) CheckConnection(ctx context.Context) error
- func (a *ProviderAdapter) GetAvailableModels(ctx context.Context) ([]ModelDetails, error)
- func (a *ProviderAdapter) GetEndpoint() string
- func (a *ProviderAdapter) GetModel() string
- func (a *ProviderAdapter) GetModelContextLimit() (int, error)
- func (a *ProviderAdapter) GetName() string
- func (a *ProviderAdapter) GetType() ClientType
- func (a *ProviderAdapter) IsDebug() bool
- func (a *ProviderAdapter) SendChatRequest(ctx context.Context, req *ProviderChatRequest) (*ChatResponse, error)
- func (a *ProviderAdapter) SetDebug(debug bool)
- func (a *ProviderAdapter) SetModel(model string) error
- func (a *ProviderAdapter) SupportsConversationalVision() bool
- func (a *ProviderAdapter) SupportsReasoning() bool
- func (a *ProviderAdapter) SupportsStreaming() bool
- func (a *ProviderAdapter) SupportsTools() bool
- func (a *ProviderAdapter) SupportsVision() bool
- func (a *ProviderAdapter) VisionCapabilities() VisionCapabilities
- type ProviderChatRequest
- type ProviderInterface
- type RequestOptions
- type SSEReader
- type StreamCallback
- type StreamingChatResponse
- type StreamingChoice
- type StreamingDelta
- type StreamingResponseBuilder
- type StreamingToolCall
- type StreamingToolCallFunction
- type StreamingUsage
- type TPSBase
- type TPSTracker
- func (t *TPSTracker) GetAverageTPS() float64
- func (t *TPSTracker) GetCurrentTPS() float64
- func (t *TPSTracker) GetSmoothTPS() float64
- func (t *TPSTracker) GetStats() map[string]interface{}
- func (t *TPSTracker) RecordRequest(duration time.Duration, completionTokens int) float64
- func (t *TPSTracker) Reset()
- type Tool
- type ToolCall
- type ToolCallFunction
- type ToolFunction
- type ToolParameter
- type ToolParameters
- type UnifiedProviderWrapper
- func (w *UnifiedProviderWrapper) CheckConnection() error
- func (w *UnifiedProviderWrapper) GetLastRequestTPS() float64
- func (w *UnifiedProviderWrapper) GetModel() string
- func (w *UnifiedProviderWrapper) GetModelContextLimit() (int, error)
- func (w *UnifiedProviderWrapper) GetProvider() string
- func (w *UnifiedProviderWrapper) GetTPSStatistics() (float64, float64, int)
- func (w *UnifiedProviderWrapper) GetVisionModel() string
- func (w *UnifiedProviderWrapper) ListModels(ctx context.Context) ([]ModelInfo, error)
- func (w *UnifiedProviderWrapper) ResetTPSStatistics()
- func (w *UnifiedProviderWrapper) SendChatRequest(ctx context.Context, messages []Message, tools []Tool, reasoning string, ...) (*ChatResponse, error)
- func (w *UnifiedProviderWrapper) SendChatRequestStream(ctx context.Context, messages []Message, tools []Tool, reasoning string, ...) (*ChatResponse, error)
- func (w *UnifiedProviderWrapper) SendVisionRequest(ctx context.Context, messages []Message, tools []Tool, reasoning string, ...) (*ChatResponse, error)
- func (w *UnifiedProviderWrapper) SetDebug(debug bool)
- func (w *UnifiedProviderWrapper) SetModel(model string) error
- func (w *UnifiedProviderWrapper) SupportsConversationalVision() bool
- func (w *UnifiedProviderWrapper) SupportsVision() bool
- func (w *UnifiedProviderWrapper) VisionCapabilities() VisionCapabilities
- type VisionCapabilities
Constants ¶
const ( // DefaultBufferTokens is the safety buffer for estimation errors DefaultBufferTokens = 1000 // MinOutputTokens is the minimum output tokens to reserve MinOutputTokens = 512 // ToolTokenEstimate is the approximate token count per tool definition ToolTokenEstimate = 200 // SystemInstructionBuffer accounts for system prompt overhead SystemInstructionBuffer = 500 // MessageOverheadTokens accounts for role/message wrapper overhead MessageOverheadTokens = 4 // ToolCallOverheadTokens accounts for assistant tool_call wrapper overhead ToolCallOverheadTokens = 12 // ToolCallIDOverheadTokens accounts for tool response tool_call_id overhead ToolCallIDOverheadTokens = 8 // ImageMessageOverheadTokens conservatively accounts for multimodal image parts ImageMessageOverheadTokens = 256 // EstimationErrorPercent is how much EstimateTokens can underestimate the // true token count on tool-heavy prompts (observed 25-34% in practice). // CalculateOutputBudget inflates the input estimate by this percent to // get a worst-case figure to budget output against. EstimationErrorPercent = 30 // BaseCushionPercent is a small fixed cushion (percent of context limit) // for output-side rounding/formatting slop, on top of the estimation // error margin above. BaseCushionPercent = 5 // BaseCushionFloor ensures small contexts still get a meaningful cushion. BaseCushionFloor = 2000 )
Token estimation constants
const ReasoningDetailsMetaKey = "reasoning_details"
ReasoningDetailsMetaKey is the Meta key under which structured reasoning blocks (OpenRouter reasoning_details array, JSON-encoded) travel on a Message. api.Message aliases the external seed core.Message whose fields cannot be extended, so the map is the in-memory carrier; history persistence uses the APIMessage.ReasoningDetails field instead.
Variables ¶
var LocalModelsProvider func(ctx context.Context) ([]ModelInfo, error)
LocalModelsProvider, when non-nil, supplies the live model listing for SproutLocalClientType. pkg/localmodel owns the real implementation (the in-process MLX LocalProvider) but already imports this package for shared types, so it can't be imported back here without a cycle — its init() sets this var instead, a standard one-directional registration hook. Left nil (falling through to the generic config-file wrapper, which has no real model list for "sprout-local") only on platforms where pkg/localmodel never gets linked in at all.
Functions ¶
func CalculateOutputBudget ¶
CalculateOutputBudget calculates the safe output token budget given context constraints. It returns the maximum tokens that can be requested for completion. If the input exceeds the context limit, returns 0 and an error message.
func CalculateOutputBudgetAnchored ¶ added in v0.17.14
CalculateOutputBudgetAnchored computes the output budget when part of the input estimate came from a real measurement (Usage.PromptTokens) and only the heuristic portion is subject to estimation error. This prevents double-counting the estimation margin on the anchored portion.
anchoredInput is the portion measured from a real API response (no error). heuristicInput is the portion estimated by the heuristic (subject to EstimationErrorPercent underestimation). The total input is anchoredInput + heuristicInput.
func ClassifyEligibleRoles ¶
ClassifyEligibleRoles returns the agentic roles a model meets the minimum deterministic bar for, based on its context window. Returns nil when the model is below the subagent threshold or its context length is unknown.
func CostFromJSON ¶
CostFromJSON probes raw response JSON for a cost value under any known candidate field name. Used as a fallback when the typed decode (estimated_cost/cost) found nothing, so a provider that names the field differently still surfaces a cost. Returns (0,false) when none match.
func EstimateInputTokens ¶
EstimateInputTokens estimates total input tokens for messages and tools. This includes a buffer for system instructions and message formatting overhead.
func EstimateMessagesTokens ¶ added in v0.17.14
EstimateMessagesTokens estimates tokens for a slice of messages only — no tool catalog or system-instruction buffer. Factored out of EstimateInputTokens so callers that already know the tool/system-prompt contribution from a real measurement (see sproutProvider's token anchor in pkg/agent/seed_provider_token_anchor.go) can estimate just a delta of newly appended messages without double-counting the fixed overhead.
func EstimateTokens ¶
EstimateTokens provides a token estimation based on OpenAI's tiktoken approach. This is the centralized implementation that all providers should use for consistency.
func FormatHTTPResponseError ¶
FormatHTTPResponseError converts an HTTP error response into a concise, user-facing error that avoids dumping full HTML or JSON payloads.
func GetCanonicalModelsForProvider ¶
func GetCanonicalModelsForProvider(ctx context.Context, clientType ClientType) ([]modelcontract.CanonicalModel, error)
GetCanonicalModelsForProvider returns canonical models for a provider — from its adapter where one exists, otherwise by projecting the legacy ModelInfo path up to canonical. Used by the registry publisher to emit the canonical per-provider file.
func GetProviderName ¶
func GetProviderName(clientType ClientType) string
GetProviderName returns the human-readable name for a provider
func IsProviderAvailable ¶
func IsProviderAvailable(provider ClientType) bool
IsProviderAvailable checks if a provider can be used. Uses credentials.HasProviderCredential to avoid hardcoding env var strings.
func ModelCachedPricingPerMillion ¶ added in v0.16.19
ModelCachedPricingPerMillion extracts a model's cached-input price (USD per million tokens) from a raw /models listing entry. Returns 0 when the listing does not expose a distinct cached rate (the provider either does not support prompt caching or folds cached tokens into the standard input price).
func ModelPricingPerMillion ¶
ModelPricingPerMillion extracts a model's input/output price (USD per million tokens) from a raw /models listing entry, probing candidate field names/units so listings that name pricing differently still surface a price. Returns (0,0) when no candidate matches.
func RecoverInlineToolCalls ¶ added in v0.17.17
func RecoverInlineToolCalls(resp *ChatResponse, tools []Tool)
RecoverInlineToolCalls runs text-based tool-call recovery on a ChatResponse for models that emit tool calls inline in message content instead of using the structured tool_calls field. This is called from the streaming path after the response is finalized, where the unified.go recovery hooks don't run (those only cover the non-streaming and unified-provider paths).
Tries Mistral-family `[TOOL_CALLS]` format first, then LFM2's Pythonic `[func(args)]` format. Only runs when no structured tool_calls were parsed and tools were offered.
func ResetPricingResolver ¶ added in v0.16.19
func ResetPricingResolver()
ResetPricingResolver clears the memoized pricing cache. For tests.
func ResolveModelPricing ¶ added in v0.16.19
func ResolveModelPricing(provider, model string) (inputPerM, outputPerM, cachedPerM float64, ok bool)
ResolveModelPricing returns the input/output/cached input/output prices (USD per million tokens) for a (provider, model) pair, resolved from the model registry / canonical adapter path and memoized for the process lifetime. cachedPerM is 0 when the provider/model does not expose a distinct cached rate. The boolean reports whether any pricing was found at all.
Network lookups are timeboxed so the caller (the metrics path) never blocks for long. A lookup failure populates a zero entry so we don't retry every response — the model's pricing won't change mid-session.
func SeedPricingForTest ¶ added in v0.16.19
SeedPricingForTest populates the resolver cache for a specific (provider, model) pair without hitting the registry. For tests that need a known pricing rate to exercise the exact-savings branch in Agent.calculateCachedTokenSavings.
Types ¶
type BaseProvider ¶
type BaseProvider struct {
// contains filtered or unexported fields
}
BaseProvider implements common functionality for all providers
func NewBaseProvider ¶
func NewBaseProvider(name string, clientType ClientType, endpoint string, apiKey string) *BaseProvider
NewBaseProvider creates a base provider with common settings
func (*BaseProvider) EstimateCost ¶
func (p *BaseProvider) EstimateCost(promptTokens, completionTokens int, model string) float64
EstimateCost calculates the estimated cost for a response
func (*BaseProvider) GetEndpoint ¶
func (p *BaseProvider) GetEndpoint() string
GetEndpoint returns the API endpoint
func (*BaseProvider) GetModel ¶
func (p *BaseProvider) GetModel() string
GetModel returns the current model
func (*BaseProvider) GetName ¶
func (p *BaseProvider) GetName() string
GetName returns the provider name
func (*BaseProvider) GetType ¶
func (p *BaseProvider) GetType() ClientType
GetType returns the provider type
func (*BaseProvider) IsDebug ¶
func (p *BaseProvider) IsDebug() bool
IsDebug returns whether debug mode is enabled
func (*BaseProvider) MakeAuthRequest ¶
func (p *BaseProvider) MakeAuthRequest(ctx context.Context, method, url string, body io.Reader) (*http.Request, error)
MakeAuthRequest creates an HTTP request with authentication
func (*BaseProvider) SetDebug ¶
func (p *BaseProvider) SetDebug(debug bool)
SetDebug enables or disables debug mode
func (*BaseProvider) SetModel ¶
func (p *BaseProvider) SetModel(model string) error
SetModel sets the current model
func (*BaseProvider) SupportsConversationalVision ¶ added in v0.16.19
func (p *BaseProvider) SupportsConversationalVision() bool
SupportsConversationalVision returns whether the provider handles inline multimodal chat messages (vs. OCR-only models). Default: supportsVision.
func (*BaseProvider) SupportsReasoning ¶
func (p *BaseProvider) SupportsReasoning() bool
SupportsReasoning returns whether the provider supports reasoning
func (*BaseProvider) SupportsStreaming ¶
func (p *BaseProvider) SupportsStreaming() bool
SupportsStreaming returns whether the provider supports streaming
func (*BaseProvider) SupportsTools ¶
func (p *BaseProvider) SupportsTools() bool
SupportsTools returns whether the provider supports tools
func (*BaseProvider) SupportsVision ¶
func (p *BaseProvider) SupportsVision() bool
SupportsVision returns whether the provider supports vision
func (*BaseProvider) VisionCapabilities ¶ added in v0.16.20
func (p *BaseProvider) VisionCapabilities() VisionCapabilities
VisionCapabilities returns the per-provider vision limits configured on this BaseProvider. The zero value means "unknown — caller should fall back to VisionCapabilitiesOrDefault()". Concrete providers (Anthropic, OpenAI, etc.) populate p.visionCaps at construction; this method just exposes it. SP-103-D3 / AUDIT-GAP-2.
type ChatChoice ¶
type ChatChoice = core.ChatChoice
type ChatRequest ¶
type ChatRequest = core.ChatRequest
type ChatResponse ¶
type ChatResponse = core.ChatResponse
type Choice ¶
type Choice = ChatChoice
type ClientInterface ¶
type ClientInterface interface {
SendChatRequest(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool) (*ChatResponse, error)
SendChatRequestStream(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool, callback StreamCallback) (*ChatResponse, error)
CheckConnection() error
SetDebug(debug bool)
SetModel(model string) error
GetModel() string
GetProvider() string
GetModelContextLimit() (int, error)
ListModels(ctx context.Context) ([]ModelInfo, error)
SupportsVision() bool
// SupportsConversationalVision reports whether the model is suitable as
// the inline multimodal target for chat-format vision messages. Some
// models (OCR-only, e.g. glm-ocr) accept image input but produce
// extraction outputs unsuitable for free-form multimodal conversation.
// Defaults to true when SupportsVision() is true.
SupportsConversationalVision() bool
// VisionCapabilities returns the per-provider vision limits (max bytes
// per image, max images per request, max dimension, supported detail
// tiers). Zero-valued fields mean "unknown — use default". Concrete
// implementations populate this with provider-specific data; callers
// should pass the result through VisionCapabilitiesOrDefault() before
// reading individual fields. SP-103-D3 / AUDIT-GAP-2.
VisionCapabilities() VisionCapabilities
GetVisionModel() string
SendVisionRequest(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool) (*ChatResponse, error)
// TPS (Tokens Per Second) tracking methods
GetLastTPS() float64
GetAverageTPS() float64
GetTPSStats() map[string]float64
ResetTPSStats()
}
ClientInterface defines the common interface for all API clients.
The ctx on Send* methods is forwarded to the underlying HTTP request (via http.NewRequestWithContext) so callers can abort in-flight LLM calls when the user clicks Stop. See SP-034 for the design.
func NewDeepInfraClientWrapper ¶
func NewDeepInfraClientWrapper(model string) (ClientInterface, error)
NewDeepInfraClientWrapper creates a DeepInfra client wrapper
func NewOpenRouterClientWrapper ¶
func NewOpenRouterClientWrapper(model string) (ClientInterface, error)
NewOpenRouterClientWrapper is deprecated - use factory.CreateProviderClient instead
func NewUnifiedClient ¶
func NewUnifiedClient(clientType ClientType) (ClientInterface, error)
NewUnifiedClient creates a client with default model for the provider
func NewUnifiedClientWithModel ¶
func NewUnifiedClientWithModel(clientType ClientType, model string) (ClientInterface, error)
NewUnifiedClientWithModel creates a client with a specific model
type ClientType ¶
type ClientType string
ClientType represents the type of client to use
const ( OllamaClientType ClientType = "ollama" // Alias for ollama-local OllamaLocalClientType ClientType = "ollama-local" OllamaCloudClientType ClientType = "ollama-cloud" TestClientType ClientType = "test" // Mock provider for CI/testing EditorClientType ClientType = "editor" // Editor-only mode, no AI provider )
Special providers that don't have configs (or have special behavior)
const ( OpenAIClientType ClientType = "openai" OpenRouterClientType ClientType = "openrouter" ZAIClientType ClientType = "zai" DeepInfraClientType ClientType = "deepinfra" DeepSeekClientType ClientType = "deepseek" LMStudioClientType ClientType = "lmstudio" MistralClientType ClientType = "mistral" MinimaxClientType ClientType = "minimax" ChutesClientType ClientType = "chutes" CerebrasClientType ClientType = "cerebras" ZAICodingClientType ClientType = "zai-coding" )
Built-in provider names (ClientType constants) are maintained here to provide type-safe constants. These are kept in sync with provider_gen.go (generated by go generate from provider configs). When adding a new provider, update both the provider config and add a ClientType constant here.
Use providers.AllProviderNames(), providers.KnownProviders(), or providers.ProviderDisplayNames() to get the list of providers without hardcoding them in this package.
const SproutLocalClientType ClientType = "sprout-local"
SproutLocalClientType is the on-device LLM provider. It runs a local Go MLX server (cmd/llm_server) on Apple Silicon, managed automatically by the agent — no API key needed, no manual server start.
func BuiltInClientTypes ¶ added in v0.17.7
func BuiltInClientTypes() []ClientType
BuiltInClientTypes returns the canonical list of all built-in provider ClientType constants defined in this package. Useful for callers that need to enumerate providers (e.g. building reverse display-name maps) without having to hardcode the list in multiple places.
Note: this list intentionally excludes aliases like "ollama" — those are resolved by ParseProviderName to a canonical ClientType. Display name → ClientType lookup handles the alias separately (see configuration.MapProviderStringToClientType).
func DetermineProvider ¶
func DetermineProvider(explicitProvider string, lastUsedProvider ClientType) (ClientType, error)
DetermineProvider provides unified provider detection with clear precedence: 1. Command-line flag (if provided) 2. Environment variable (SPROUT_PROVIDER, with SPROUT_PROVIDER backward-compat) 3. Config file (last_used_provider)
Returns an error if no available provider is found; callers should surface this to the user and offer interactive provider selection.
func ParseProviderName ¶
func ParseProviderName(name string) (ClientType, error)
ParseProviderName converts a string provider name to ClientType Handles special aliases and validates provider names.
type HTTPClient ¶
HTTPClient interface for testing
type HarmonyFormatter ¶
type HarmonyFormatter struct {
// contains filtered or unexported fields
}
HarmonyFormatter handles conversion from OpenAI format to harmony format
func NewHarmonyFormatter ¶
func NewHarmonyFormatter() *HarmonyFormatter
NewHarmonyFormatter creates a new harmony formatter
func NewHarmonyFormatterWithReasoning ¶
func NewHarmonyFormatterWithReasoning(reasoning string) *HarmonyFormatter
NewHarmonyFormatterWithReasoning creates a harmony formatter with specific reasoning level
func (*HarmonyFormatter) AddReturnToken ¶
func (h *HarmonyFormatter) AddReturnToken(response string) string
AddReturnToken adds the completion token to a harmony response
func (*HarmonyFormatter) ConvertReturnToEnd ¶
func (h *HarmonyFormatter) ConvertReturnToEnd(conversation string) string
ConvertReturnToEnd converts <|return|> tokens to <|end|> for conversation history
func (*HarmonyFormatter) FormatMessagesForCompletion ¶
func (h *HarmonyFormatter) FormatMessagesForCompletion(messages []Message, tools []Tool, opts *HarmonyOptions) string
FormatMessagesForCompletion converts OpenAI-style messages to harmony format
func (*HarmonyFormatter) StripReturnToken ¶
func (h *HarmonyFormatter) StripReturnToken(response string) string
StripReturnToken removes <|return|> tokens from model responses
type HarmonyOptions ¶
type HarmonyOptions struct {
ReasoningLevel string // "low", "medium", "high" - empty disables explicit reasoning tag
EnableAnalysis bool // Whether to enable analysis channel guidance
}
HarmonyOptions configures the harmony formatting
type ModelDetails ¶
type ModelDetails struct {
ID string
Name string
ContextLength int
InputCostPer1K float64
OutputCostPer1K float64
Features []string // e.g., "vision", "tools", "reasoning"
IsDefault bool
}
ModelDetails represents detailed information about a model from a provider
type ModelInfo ¶
type ModelInfo struct {
ID string `json:"id"`
Name string `json:"name,omitempty"`
Description string `json:"description,omitempty"`
Provider string `json:"provider,omitempty"`
Size string `json:"size,omitempty"`
Cost float64 `json:"cost,omitempty"`
InputCost float64 `json:"input_cost,omitempty"`
OutputCost float64 `json:"output_cost,omitempty"`
CachedInputCost float64 `json:"cached_input_cost,omitempty"`
ContextLength int `json:"context_length,omitempty"`
Tags []string `json:"tags,omitempty"`
// EligibleRoles lists the agentic roles a model meets the minimum
// deterministic bar for ("primary", "subagent"). This is an
// eligibility pre-filter (currently context-window based), NOT a
// quality recommendation — the capability probe provides the
// authoritative agentic-capable signal. Empty means below the bar or
// unknown. Additive/omitempty so older clients ignore it.
EligibleRoles []string `json:"eligible_roles,omitempty"`
// RecommendedRoles ⊆ EligibleRoles, gated on passing the capability probe
// (subagent ← gates passed, primary ← complex stage passed). Empty when
// un-probed or not recommended. Populated from the published registry.
RecommendedRoles []string `json:"recommended_roles,omitempty"`
// Warnings are non-blocking caveats to surface in the picker (e.g. a small
// context window in the 64K–128K band). Populated from the published registry.
Warnings []string `json:"warnings,omitempty"`
// VisionProbe carries the probe-tested vision capability for this model.
// nil = never probed (fall back to config-based detection).
// Non-nil = probe ground truth (true = can see images, false = cannot).
VisionProbe *bool `json:"vision_probe,omitempty"`
}
ModelInfo represents information about an available model
func CanonicalToModelInfo ¶
func CanonicalToModelInfo(m modelcontract.CanonicalModel) ModelInfo
CanonicalToModelInfo projects a canonical model down to the ModelInfo shape existing consumers expect. Known-true capabilities are surfaced as Tags so callers that inspect tags (e.g. the CLI "Supports tools" line) work unchanged. Exported for the registry publisher (cmd/refresh_provider_catalog).
func GetAvailableModels ¶
GetAvailableModels returns available models for the current provider. Returns an error if no provider can be determined; callers should surface this to the user and offer interactive provider selection.
func GetModelsForProvider ¶
func GetModelsForProvider(clientType ClientType) ([]ModelInfo, error)
GetModelsForProvider returns available models for a specific provider
func GetModelsForProviderCtx ¶
func GetModelsForProviderCtx(ctx context.Context, clientType ClientType) ([]ModelInfo, error)
GetModelsForProviderCtx returns available models for a specific provider with context support. It checks the model registry first (if enabled), falling back to direct per-provider API calls.
type ModelSelection ¶
type ModelSelection struct {
// contains filtered or unexported fields
}
ModelSelection represents a model selection system This is a stub implementation for backward compatibility The actual model selection logic has been moved to configuration-based system
func NewModelSelection ¶
func NewModelSelection(config interface{}) *ModelSelection
NewModelSelection creates a new ModelSelection instance This is a stub for backward compatibility - the actual model selection is now handled through the configuration system
type ModelsListInterface ¶
type ModelsListInterface interface {
ListAvailableModels() ([]ModelInfo, error)
GetDefaultModel() string
IsModelAvailable(modelID string) bool
}
ModelsListInterface defines methods for listing available models
type OllamaLocalClient ¶
type OllamaLocalClient struct {
*TPSBase
// contains filtered or unexported fields
}
OllamaLocalClient handles local Ollama API requests
func NewOllamaLocalClient ¶
func NewOllamaLocalClient(model string) (*OllamaLocalClient, error)
NewOllamaLocalClient creates a new local Ollama client
func (*OllamaLocalClient) CheckConnection ¶
func (c *OllamaLocalClient) CheckConnection() error
CheckConnection verifies local Ollama is accessible
func (*OllamaLocalClient) GetCachedModel ¶ added in v0.17.7
func (c *OllamaLocalClient) GetCachedModel(name string) (int, bool)
GetCachedModel returns a cached context length when the model list is fresh. Matches by ID or Name; callers typically pass the Ollama tag (e.g. "llama3:8b") which ListModels stores in both fields for symmetry with lmStudioListModelsWrapper.
func (*OllamaLocalClient) GetModel ¶
func (c *OllamaLocalClient) GetModel() string
GetModel returns the current model
func (*OllamaLocalClient) GetModelContextLimit ¶
func (c *OllamaLocalClient) GetModelContextLimit() (int, error)
GetModelContextLimit returns the context limit for the model.
Resolution layers (most-specific to least):
- Cached list-models entry whose context Ollama reported via /api/show.
- Fresh /api/show lookup for the current model (2s timeout; log on failure so a misconfigured Ollama server doesn't silently degrade to the wrong default context length).
- Static DefaultContextLimit from config (set at construction).
- Hardcoded 32000 fallback.
Replaces the previous substring match on "qwen3-coder"/"gpt-oss", which missed most models and silently returned 32k for everything else.
func (*OllamaLocalClient) GetProvider ¶
func (c *OllamaLocalClient) GetProvider() string
GetProvider returns the provider name
func (*OllamaLocalClient) GetVisionModel ¶
func (c *OllamaLocalClient) GetVisionModel() string
GetVisionModel returns empty string as vision is not supported
func (*OllamaLocalClient) ListModels ¶
func (c *OllamaLocalClient) ListModels(ctx context.Context) ([]ModelInfo, error)
ListModels returns available local models.
func (*OllamaLocalClient) SendChatRequest ¶
func (c *OllamaLocalClient) SendChatRequest(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool) (*ChatResponse, error)
SendChatRequest sends a chat request to local Ollama
func (*OllamaLocalClient) SendChatRequestStream ¶
func (c *OllamaLocalClient) SendChatRequestStream(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool, callback StreamCallback) (*ChatResponse, error)
SendChatRequestStream streams responses from local Ollama as they arrive
func (*OllamaLocalClient) SendVisionRequest ¶
func (c *OllamaLocalClient) SendVisionRequest(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool) (*ChatResponse, error)
SendVisionRequest handles vision/OCR requests for Ollama Delegates to SendChatRequest since the image handling is done in buildChatRequest
func (*OllamaLocalClient) SetDebug ¶
func (c *OllamaLocalClient) SetDebug(debug bool)
SetDebug enables or disables debug mode
func (*OllamaLocalClient) SetModel ¶
func (c *OllamaLocalClient) SetModel(model string) error
SetModel updates the active model after validating it exists locally
func (*OllamaLocalClient) SupportsConversationalVision ¶ added in v0.16.19
func (c *OllamaLocalClient) SupportsConversationalVision() bool
SupportsConversationalVision returns true only for multimodal chat models. OCR-only models (e.g. glm-ocr) accept images but produce extraction output that doesn't help free-form conversational turns — the tool path (analyze_image_content) is the right channel for them. Inline embedding is only useful for chat models like llama3.2-vision.
func (*OllamaLocalClient) SupportsVision ¶
func (c *OllamaLocalClient) SupportsVision() bool
SupportsVision returns true for OCR-capable models
func (*OllamaLocalClient) VisionCapabilities ¶ added in v0.16.20
func (c *OllamaLocalClient) VisionCapabilities() VisionCapabilities
VisionCapabilities returns the local-Ollama vision limits used by llama3.2-vision / glm-ocr family models.
Conservative defaults reflect the documented Ollama API constraints: the older base64 payload cap is ~3.5MB (we use 5MB), llama3.2-vision works best at 1024px on the longest side, and we cap to a handful of images per request since local context windows are tight. Detail tiers are intentionally left nil — Ollama picks automatically. The returned values are static (don't depend on c.model) so the table is safe to share across clients; per-model overrides can be added later. SP-103-D3 / AUDIT-GAP-2.
type Provider ¶
type Provider interface {
// Core functionality
SendChatRequest(ctx context.Context, req *ProviderChatRequest) (*ChatResponse, error)
CheckConnection(ctx context.Context) error
// Model management
GetModel() string
SetModel(model string) error
GetAvailableModels(ctx context.Context) ([]ModelDetails, error)
GetModelContextLimit() (int, error)
// Provider information
GetName() string
GetType() ClientType
GetEndpoint() string
// Feature support
SupportsVision() bool
// SupportsConversationalVision reports whether the provider is suitable as
// the inline multimodal target for chat-format vision messages. OCR-only
// models accept image input but produce extraction outputs unsuitable for
// free-form multimodal conversation. Default: SupportsVision().
SupportsConversationalVision() bool
SupportsTools() bool
SupportsStreaming() bool
SupportsReasoning() bool
// VisionCapabilities returns the per-provider vision limits (max bytes
// per image, max images per request, max dimension, supported detail
// tiers). Zero-valued fields mean "unknown" and should be filled from
// VisionCapabilitiesDefault() at the call site. SP-103-D3 / AUDIT-GAP-2.
VisionCapabilities() VisionCapabilities
// Configuration
SetDebug(debug bool)
IsDebug() bool
}
Provider defines the interface all LLM providers must implement
func CreateProviderFromClient ¶
func CreateProviderFromClient(clientType ClientType, client ClientInterface) Provider
CreateProviderFromClient creates a Provider from an existing ClientInterface
type ProviderAdapter ¶
type ProviderAdapter struct {
// contains filtered or unexported fields
}
ProviderAdapter adapts the existing ClientInterface to the new Provider interface
func NewProviderAdapter ¶
func NewProviderAdapter(clientType ClientType, client ClientInterface) *ProviderAdapter
NewProviderAdapter creates an adapter for existing clients
func (*ProviderAdapter) CheckConnection ¶
func (a *ProviderAdapter) CheckConnection(ctx context.Context) error
CheckConnection verifies connectivity
func (*ProviderAdapter) GetAvailableModels ¶
func (a *ProviderAdapter) GetAvailableModels(ctx context.Context) ([]ModelDetails, error)
GetAvailableModels returns available models for this provider
func (*ProviderAdapter) GetEndpoint ¶
func (a *ProviderAdapter) GetEndpoint() string
GetEndpoint returns the API endpoint from the underlying client. Falls back to an empty string if the client doesn't expose GetEndpoint.
func (*ProviderAdapter) GetModel ¶
func (a *ProviderAdapter) GetModel() string
GetModel returns the current model
func (*ProviderAdapter) GetModelContextLimit ¶
func (a *ProviderAdapter) GetModelContextLimit() (int, error)
GetModelContextLimit returns the context window size
func (*ProviderAdapter) GetName ¶
func (a *ProviderAdapter) GetName() string
GetName returns the provider name
func (*ProviderAdapter) GetType ¶
func (a *ProviderAdapter) GetType() ClientType
GetType returns the provider type
func (*ProviderAdapter) IsDebug ¶
func (a *ProviderAdapter) IsDebug() bool
IsDebug returns whether debug mode is enabled
func (*ProviderAdapter) SendChatRequest ¶
func (a *ProviderAdapter) SendChatRequest(ctx context.Context, req *ProviderChatRequest) (*ChatResponse, error)
SendChatRequest adapts the old interface to the new one.
Note: This uses the same global per-provider rate limiter as APIClient.sendRequest(). Both paths share one bucket per provider to coordinate across all agents, preventing cascading 429s when multiple subagents run concurrently. Do NOT add additional rate limiting at this layer without coordinating with pkg/agent/api_client.go.
func (*ProviderAdapter) SetDebug ¶
func (a *ProviderAdapter) SetDebug(debug bool)
SetDebug enables or disables debug mode
func (*ProviderAdapter) SetModel ¶
func (a *ProviderAdapter) SetModel(model string) error
SetModel sets the current model
func (*ProviderAdapter) SupportsConversationalVision ¶ added in v0.16.19
func (a *ProviderAdapter) SupportsConversationalVision() bool
SupportsConversationalVision returns whether the provider handles inline multimodal chat messages. Delegates to the underlying client; falls back to SupportsVision() if the client doesn't implement the new method.
func (*ProviderAdapter) SupportsReasoning ¶
func (a *ProviderAdapter) SupportsReasoning() bool
SupportsReasoning returns whether the provider supports reasoning
func (*ProviderAdapter) SupportsStreaming ¶
func (a *ProviderAdapter) SupportsStreaming() bool
SupportsStreaming returns whether the provider supports streaming
func (*ProviderAdapter) SupportsTools ¶
func (a *ProviderAdapter) SupportsTools() bool
SupportsTools returns whether the provider supports tools
func (*ProviderAdapter) SupportsVision ¶
func (a *ProviderAdapter) SupportsVision() bool
SupportsVision returns whether the provider supports vision
func (*ProviderAdapter) VisionCapabilities ¶ added in v0.16.20
func (a *ProviderAdapter) VisionCapabilities() VisionCapabilities
VisionCapabilities returns the per-provider vision limits by delegating to the wrapped client. If the client does not implement VisionCapabilities() (e.g. legacy / mock clients), returns the zero value; callers should pass that through VisionCapabilitiesOrDefault() to get a safe usable configuration. SP-103-D3 / AUDIT-GAP-2.
type ProviderChatRequest ¶
type ProviderChatRequest struct {
Messages []Message
Tools []Tool
Options *RequestOptions
}
ProviderChatRequest represents a unified request structure for providers This extends the basic ChatRequest with provider-specific options
type ProviderInterface ¶
type ProviderInterface interface {
SendChatRequest(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool) (*ChatResponse, error)
SendChatRequestStream(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool, callback StreamCallback) (*ChatResponse, error)
CheckConnection() error
SetDebug(debug bool)
SetModel(model string) error
GetModel() string
GetProvider() string
GetModelContextLimit() (int, error)
ListModels(ctx context.Context) ([]ModelInfo, error)
SupportsVision() bool
SendVisionRequest(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool) (*ChatResponse, error)
}
ProviderInterface defines the interface that all providers must implement. Mirrors ClientInterface's ctx-bearing send methods — see SP-034.
type RequestOptions ¶
type RequestOptions struct {
Temperature *float64
MaxTokens *int
TopP *float64
FrequencyPenalty *float64
PresencePenalty *float64
StopSequences []string
Stream bool
ReasoningEffort string // For reasoning models
DisableThinking *bool // Disable thinking/reasoning mode for thinking-capable models
}
RequestOptions contains optional parameters for requests
type SSEReader ¶
type SSEReader struct {
// contains filtered or unexported fields
}
SSEReader reads Server-Sent Events from a reader
func NewSSEReader ¶
NewSSEReader creates a new SSE reader
func (*SSEReader) ReadWithTimeout ¶
ReadWithTimeout processes the SSE stream with a timeout. Each line read is performed in a short-lived goroutine so that the select can respond to the timeout without blocking the main loop. The background goroutine will exit when the underlying HTTP response body is closed (which triggers ReadString to return an error).
type StreamCallback ¶
StreamCallback is called for each content chunk received. contentType is "assistant_text" for regular content or "reasoning" for thinking/reasoning content.
type StreamingChatResponse ¶
type StreamingChatResponse struct {
ID string `json:"id"`
Object string `json:"object"`
Created int64 `json:"created"`
Model string `json:"model"`
Choices []StreamingChoice `json:"choices"`
Usage *StreamingUsage `json:"usage,omitempty"`
}
StreamingChatResponse represents a streaming response chunk
func ParseSSEData ¶
func ParseSSEData(data string) (*StreamingChatResponse, error)
ParseSSEData parses SSE data into a streaming response
type StreamingChoice ¶
type StreamingChoice struct {
Index int `json:"index"`
Delta StreamingDelta `json:"delta"`
FinishReason *string `json:"finish_reason"`
}
StreamingChoice represents a streaming response choice
type StreamingDelta ¶
type StreamingDelta struct {
Role string `json:"role,omitempty"`
Content string `json:"content,omitempty"`
Reasoning string `json:"reasoning,omitempty"` // GLM reasoning field
ReasoningContent string `json:"reasoning_content,omitempty"`
ReasoningDetails json.RawMessage `json:"reasoning_details,omitempty"` // Can be string (Minimax) or array (GLM models)
ToolCalls []StreamingToolCall `json:"tool_calls,omitempty"`
}
StreamingDelta contains incremental updates
type StreamingResponseBuilder ¶
type StreamingResponseBuilder struct {
// contains filtered or unexported fields
}
StreamingResponseBuilder accumulates streaming chunks into a complete response
func NewStreamingResponseBuilder ¶
func NewStreamingResponseBuilder(callback StreamCallback) *StreamingResponseBuilder
NewStreamingResponseBuilder creates a new streaming response builder
func (*StreamingResponseBuilder) GetResponse ¶
func (b *StreamingResponseBuilder) GetResponse() *ChatResponse
GetResponse returns the accumulated response
func (*StreamingResponseBuilder) GetTokenGenerationDuration ¶
func (b *StreamingResponseBuilder) GetTokenGenerationDuration() time.Duration
GetTokenGenerationDuration returns the duration from first token to last token
func (*StreamingResponseBuilder) ProcessChunk ¶
func (b *StreamingResponseBuilder) ProcessChunk(chunk *StreamingChatResponse) error
ProcessChunk processes a streaming chunk and updates the builder state
type StreamingToolCall ¶
type StreamingToolCall struct {
Index int `json:"index"`
ID string `json:"id,omitempty"`
Type string `json:"type,omitempty"`
Function *StreamingToolCallFunction `json:"function,omitempty"`
}
StreamingToolCall represents an incremental tool call update
type StreamingToolCallFunction ¶
type StreamingToolCallFunction struct {
Name string `json:"name,omitempty"`
Arguments string `json:"arguments,omitempty"`
}
StreamingToolCallFunction contains function details
type StreamingUsage ¶
type StreamingUsage struct {
PromptTokens int `json:"prompt_tokens"`
CompletionTokens int `json:"completion_tokens"`
TotalTokens int `json:"total_tokens"`
EstimatedCost float64 `json:"estimated_cost"`
Cost float64 `json:"cost,omitempty"` // OpenRouter returns cost directly
ImageTokens int `json:"image_tokens,omitempty"`
PromptTokensDetails struct {
CachedTokens int `json:"cached_tokens"`
CacheWriteTokens *int `json:"cache_write_tokens"`
} `json:"prompt_tokens_details,omitempty"`
}
StreamingUsage is the usage block on a streaming chunk (providers send it on the final data object). Named (not anonymous) so the flexible cost fallback in ParseSSEData can allocate one when a provider reports cost without the standard token block.
type TPSBase ¶
type TPSBase struct {
// contains filtered or unexported fields
}
TPSBase provides a default implementation of TPS tracking methods Other providers can embed this struct to get TPS functionality
func (*TPSBase) GetAverageTPS ¶
GetAverageTPS returns the average TPS across all requests
func (*TPSBase) GetLastTPS ¶
GetLastTPS returns the most recent TPS measurement
func (*TPSBase) GetTPSStats ¶
GetTPSStats returns comprehensive TPS statistics
func (*TPSBase) GetTracker ¶
func (t *TPSBase) GetTracker() *TPSTracker
GetTracker returns the underlying TPS tracker
func (*TPSBase) ResetTPSStats ¶
func (t *TPSBase) ResetTPSStats()
ResetTPSStats clears all TPS tracking data
type TPSTracker ¶
type TPSTracker struct {
// contains filtered or unexported fields
}
func (*TPSTracker) GetAverageTPS ¶
func (t *TPSTracker) GetAverageTPS() float64
GetAverageTPS returns the average TPS across all recorded requests
func (*TPSTracker) GetCurrentTPS ¶
func (t *TPSTracker) GetCurrentTPS() float64
GetCurrentTPS returns the most recent TPS value
func (*TPSTracker) GetSmoothTPS ¶
func (t *TPSTracker) GetSmoothTPS() float64
GetSmoothTPS returns an exponentially smoothed TPS value
func (*TPSTracker) GetStats ¶
func (t *TPSTracker) GetStats() map[string]interface{}
GetStats returns comprehensive TPS statistics
func (*TPSTracker) RecordRequest ¶
func (t *TPSTracker) RecordRequest(duration time.Duration, completionTokens int) float64
RecordRequest records the timing and token usage of an API request
type ToolCall ¶
func RecoverLFM2ToolCalls ¶ added in v0.17.17
RecoverLFM2ToolCalls extracts Liquid AI LFM2-style tool calls from the assistant message content. LFM2 emits Pythonic syntax between special tokens:
<|tool_call_start|>[read_file(path='/some/file.go')]<|tool_call_end|> <|tool_call_start|>[get_weather(city='Paris')]<|tool_call_end|>
Multiple calls may appear sequentially. Returns the recovered calls, the content with markers+payloads stripped, and ok=true when at least one call was parsed. When ok is false, content is returned unchanged.
func RecoverMistralToolCalls ¶
RecoverMistralToolCalls recovers tool calls from the Mistral-family text format, where the model emits `[TOOL_CALLS]…` inside the message content instead of the structured tool_calls field. It returns the recovered calls and the content with the marker and its payload stripped. ok is false when no marker is present or nothing parseable follows it (content is returned unchanged in that case).
Two payload shapes are handled:
[TOOL_CALLS][{"name":"f","arguments":{…}}, …] (JSON array — Mistral native)
[TOOL_CALLS]f{…}g{…} (name + JSON object, repeated)
type ToolCallFunction ¶
type ToolCallFunction = core.ToolCallFunction
type ToolFunction ¶
type ToolFunction = core.ToolFunction
type ToolParameter ¶
type ToolParameter = core.ToolParameter
type ToolParameters ¶
type ToolParameters = core.ToolParameters
type UnifiedProviderWrapper ¶
type UnifiedProviderWrapper struct {
*TPSBase
// contains filtered or unexported fields
}
UnifiedProviderWrapper wraps any provider that implements ProviderInterface
func NewUnifiedProviderWrapper ¶
func NewUnifiedProviderWrapper(provider ProviderInterface) *UnifiedProviderWrapper
NewUnifiedProviderWrapper creates a wrapper for any provider
func (*UnifiedProviderWrapper) CheckConnection ¶
func (w *UnifiedProviderWrapper) CheckConnection() error
Forward all other methods to the provider
func (*UnifiedProviderWrapper) GetLastRequestTPS ¶
func (w *UnifiedProviderWrapper) GetLastRequestTPS() float64
GetLastRequestTPS returns the TPS for the last API request
func (*UnifiedProviderWrapper) GetModel ¶
func (w *UnifiedProviderWrapper) GetModel() string
func (*UnifiedProviderWrapper) GetModelContextLimit ¶
func (w *UnifiedProviderWrapper) GetModelContextLimit() (int, error)
func (*UnifiedProviderWrapper) GetProvider ¶
func (w *UnifiedProviderWrapper) GetProvider() string
func (*UnifiedProviderWrapper) GetTPSStatistics ¶
func (w *UnifiedProviderWrapper) GetTPSStatistics() (float64, float64, int)
GetTPSStatistics returns tokens per second statistics
func (*UnifiedProviderWrapper) GetVisionModel ¶
func (w *UnifiedProviderWrapper) GetVisionModel() string
func (*UnifiedProviderWrapper) ListModels ¶
func (w *UnifiedProviderWrapper) ListModels(ctx context.Context) ([]ModelInfo, error)
func (*UnifiedProviderWrapper) ResetTPSStatistics ¶
func (w *UnifiedProviderWrapper) ResetTPSStatistics()
ResetTPSStatistics resets the TPS tracking
func (*UnifiedProviderWrapper) SendChatRequest ¶
func (w *UnifiedProviderWrapper) SendChatRequest(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool) (*ChatResponse, error)
SendChatRequest converts types and forwards to provider
func (*UnifiedProviderWrapper) SendChatRequestStream ¶
func (w *UnifiedProviderWrapper) SendChatRequestStream(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool, callback StreamCallback) (*ChatResponse, error)
SendChatRequestStream sends a streaming chat request (not yet implemented for unified providers)
func (*UnifiedProviderWrapper) SendVisionRequest ¶
func (w *UnifiedProviderWrapper) SendVisionRequest(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool) (*ChatResponse, error)
func (*UnifiedProviderWrapper) SetDebug ¶
func (w *UnifiedProviderWrapper) SetDebug(debug bool)
func (*UnifiedProviderWrapper) SetModel ¶
func (w *UnifiedProviderWrapper) SetModel(model string) error
func (*UnifiedProviderWrapper) SupportsConversationalVision ¶ added in v0.16.19
func (w *UnifiedProviderWrapper) SupportsConversationalVision() bool
SupportsConversationalVision reports whether inline multimodal turns should embed the image. Delegates to the underlying provider if it implements the method; otherwise falls back to SupportsVision().
func (*UnifiedProviderWrapper) SupportsVision ¶
func (w *UnifiedProviderWrapper) SupportsVision() bool
func (*UnifiedProviderWrapper) VisionCapabilities ¶ added in v0.16.20
func (w *UnifiedProviderWrapper) VisionCapabilities() VisionCapabilities
VisionCapabilities returns the per-provider vision limits by delegating to the wrapped provider. If the provider does not implement VisionCapabilities() (e.g. legacy third-party providers), returns the zero value; callers should run the result through VisionCapabilitiesOrDefault() to fill in safe defaults. SP-103-D3 / AUDIT-GAP-2.
type VisionCapabilities ¶ added in v0.16.20
type VisionCapabilities struct {
// MaxImageBytes is the largest single image the provider accepts
// inline (oversized images must be resized before embedding).
MaxImageBytes int
// MaxImageCount is the max number of inline images per request.
MaxImageCount int
// MaxImageDimension is the longest-side cap (px). Providers differ
// widely: Anthropic auto-resizes to 1568px, OpenAI keeps native
// at low/auto, Gemini up to 3072 in some configs. Oversized
// images should be resized to MaxImageDimension before embedding.
MaxImageDimension int
// DetailTiers lists supported detail levels (e.g. "low","high" or
// "low","high","auto"). Empty means the provider picks automatically.
DetailTiers []string
}
VisionCapabilities describes per-provider vision limits. Populated from provider/model metadata and read once at construction. Fields with zero values mean "unknown — use default".
Why per provider? Real provider limits diverge sharply:
- Anthropic caps single-image base64 payloads at ~5MB and auto-resizes to 1568px on the longest side, accepting up to 100 images per turn.
- OpenAI's gpt-4o accepts ~20MB images and supports low/high/auto detail tiers, with up to 500 images in some endpoints.
- Local Ollama (llama3.2-vision) caps out around 3.5MB and works best at 1024px on the longest side with only a handful of images.
A single 1536px + 5MB + N-image cap is a poor fit for any of them.
The defaults used when a field is zero are intentionally safe (not permissive): they keep the call working while image-resize / batch-splitting code paths evolve. See VisionCapabilitiesDefault.
func VisionCapabilitiesDefault ¶ added in v0.16.20
func VisionCapabilitiesDefault() VisionCapabilities
VisionCapabilitiesDefault returns the package-wide safe fallback when a provider-specific capability table is empty or missing. The defaults are chosen to be conservative (works on every supported provider) rather than best-of-breed:
- 5_000_000 bytes ≈ Anthropic's official single-image cap.
- 20 images per request — well under every provider limit.
- 1536 px on the longest side — matches the historical hard-coded resize cap so existing behavior is preserved when a provider returns zero fields.
Callers that need provider-tuned behavior should read the actual VisionCapabilities() from the provider, not the defaults.
func VisionCapabilitiesOrDefault ¶ added in v0.16.20
func VisionCapabilitiesOrDefault(caps VisionCapabilities) VisionCapabilities
VisionCapabilitiesOrDefault returns caps with zero-valued fields filled in from VisionCapabilitiesDefault(). Non-zero fields are preserved untouched. Use this at the call site (resize / batch-split) so partially-populated tables still produce safe behavior.
Example:
caps := client.VisionCapabilities() // may be all zeros
caps = api.VisionCapabilitiesOrDefault(caps) // now safe to read
if len(images) > caps.MaxImageCount { ... }
Source Files
¶
- harmony.go
- interface.go
- lfm2_toolcalls.go
- lmstudio_models.go
- mistral_toolcalls.go
- models.go
- models_canonical.go
- models_http_errors.go
- models_providers.go
- models_selection.go
- ollama_local.go
- ollama_local_api.go
- ollama_local_env.go
- ollama_local_http.go
- ollama_local_streaming.go
- pricing_resolver.go
- provider_adapter.go
- provider_interface.go
- status_prefix.go
- streaming.go
- token_utils.go
- toolcall_recovery.go
- tps_base.go
- tps_tracker.go
- types.go
- unified.go
- usage_normalize.go
- vision_capabilities.go