api

package
v0.17.13 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 30, 2026 License: MIT Imports: 28 Imported by: 0

Documentation

Overview

Package agent_api: LM Studio model-list discovery.

LM Studio (≥ 0.3.6) exposes a native REST API at /api/v0/models that reports max_context_length and load state per model. Older versions and the OpenAI- compatible layer (/v1/models) don't carry context length, so we hit v0 first and fall back to v1 with a 32k default when v0 is unavailable or reports an unloaded/missing-context entry.

Local-only by design — LM Studio's published endpoint is 127.0.0.1:1234. LMSTUDIO_BASE_URL is honored for users running a remote LM Studio server.

Package agent_api: wire-type structs for Ollama local API communication (split from ollama_local.go)

Package agent_api: Ollama local client synchronous API methods (split from ollama_local.go)

Package agent_api: Ollama local client constructors, environment setup, and request-building (split from ollama_local.go)

Package agent_api: HTTP transport for Ollama local API (split from ollama_local.go)

Package agent_api: Ollama streaming response helpers, converter functions, and streaming API (split from ollama_local.go)

Package api provides API types used across all providers.

Seed canonical types (github.com/sprout-foundry/seed/core) are imported via type aliases. Sprout consumes these types — it does not define them.

Package api — vision capabilities table (AUDIT-GAP-2 / SP-103-D3).

Provides a structured per-provider view of the vision limits that historically lived in a single binary SupportsVision() flag plus ad-hoc constants scattered across resize / batching code paths. Downstream consumers (image resize at SP-103-B2, batch splitting at SP-103-D2) read this table instead of using a one-size 1536px cap.

Index

Constants

View Source 
const (
	// DefaultBufferTokens is the safety buffer for estimation errors
	DefaultBufferTokens = 1000
	// MinOutputTokens is the minimum output tokens to reserve
	MinOutputTokens = 512
	// ToolTokenEstimate is the approximate token count per tool definition
	ToolTokenEstimate = 200
	// SystemInstructionBuffer accounts for system prompt overhead
	SystemInstructionBuffer = 500
	// MessageOverheadTokens accounts for role/message wrapper overhead
	MessageOverheadTokens = 4
	// ToolCallOverheadTokens accounts for assistant tool_call wrapper overhead
	ToolCallOverheadTokens = 12
	// ToolCallIDOverheadTokens accounts for tool response tool_call_id overhead
	ToolCallIDOverheadTokens = 8
	// ImageMessageOverheadTokens conservatively accounts for multimodal image parts
	ImageMessageOverheadTokens = 256
)

Token estimation constants

Variables

This section is empty.

Functions

func CalculateOutputBudget 

func CalculateOutputBudget(contextLimit int, inputTokens int) (int, bool)

CalculateOutputBudget calculates the safe output token budget given context constraints. It returns the maximum tokens that can be requested for completion. If the input exceeds the context limit, returns 0 and an error message.

func ClassifyEligibleRoles 

func ClassifyEligibleRoles(m ModelInfo) []string

ClassifyEligibleRoles returns the agentic roles a model meets the minimum deterministic bar for, based on its context window. Returns nil when the model is below the subagent threshold or its context length is unknown.

func CostFromJSON 

func CostFromJSON(body []byte) (float64, bool)

CostFromJSON probes raw response JSON for a cost value under any known candidate field name. Used as a fallback when the typed decode (estimated_cost/cost) found nothing, so a provider that names the field differently still surfaces a cost. Returns (0,false) when none match.

func EstimateInputTokens 

func EstimateInputTokens(messages []Message, tools []Tool) int

EstimateInputTokens estimates total input tokens for messages and tools. This includes a buffer for system instructions and message formatting overhead.

func EstimateTokens 

func EstimateTokens(text string) int

EstimateTokens provides a token estimation based on OpenAI's tiktoken approach. This is the centralized implementation that all providers should use for consistency.

func FormatHTTPResponseError 

func FormatHTTPResponseError(statusCode int, headers http.Header, body []byte) error

FormatHTTPResponseError converts an HTTP error response into a concise, user-facing error that avoids dumping full HTML or JSON payloads.

func GetCanonicalModelsForProvider 

func GetCanonicalModelsForProvider(ctx context.Context, clientType ClientType) ([]modelcontract.CanonicalModel, error)

GetCanonicalModelsForProvider returns canonical models for a provider — from its adapter where one exists, otherwise by projecting the legacy ModelInfo path up to canonical. Used by the registry publisher to emit the canonical per-provider file.

func GetProviderName 

func GetProviderName(clientType ClientType) string

GetProviderName returns the human-readable name for a provider

func IsProviderAvailable 

func IsProviderAvailable(provider ClientType) bool

IsProviderAvailable checks if a provider can be used. Uses credentials.HasProviderCredential to avoid hardcoding env var strings.

func ModelCachedPricingPerMillion added in v0.16.19 

func ModelCachedPricingPerMillion(entry map[string]any) float64

ModelCachedPricingPerMillion extracts a model's cached-input price (USD per million tokens) from a raw /models listing entry. Returns 0 when the listing does not expose a distinct cached rate (the provider either does not support prompt caching or folds cached tokens into the standard input price).

func ModelPricingPerMillion 

func ModelPricingPerMillion(entry map[string]any) (inputPerMillion, outputPerMillion float64)

ModelPricingPerMillion extracts a model's input/output price (USD per million tokens) from a raw /models listing entry, probing candidate field names/units so listings that name pricing differently still surface a price. Returns (0,0) when no candidate matches.

func ResetPricingResolver added in v0.16.19 

func ResetPricingResolver()

ResetPricingResolver clears the memoized pricing cache. For tests.

func ResolveModelPricing added in v0.16.19 

func ResolveModelPricing(provider, model string) (inputPerM, outputPerM, cachedPerM float64, ok bool)

ResolveModelPricing returns the input/output/cached input/output prices (USD per million tokens) for a (provider, model) pair, resolved from the model registry / canonical adapter path and memoized for the process lifetime. cachedPerM is 0 when the provider/model does not expose a distinct cached rate. The boolean reports whether any pricing was found at all.

Network lookups are timeboxed so the caller (the metrics path) never blocks for long. A lookup failure populates a zero entry so we don't retry every response — the model's pricing won't change mid-session.

func SeedPricingForTest added in v0.16.19 

func SeedPricingForTest(provider, model string, inputPerM, outputPerM, cachedPerM float64)

SeedPricingForTest populates the resolver cache for a specific (provider, model) pair without hitting the registry. For tests that need a known pricing rate to exercise the exact-savings branch in Agent.calculateCachedTokenSavings.

func UsageCost 

func UsageCost(u ChatUsage) float64

UsageCost returns the canonical cost from a typed ChatUsage, preferring the provider-reported Cost over EstimatedCost. Both are populated by the normal decode (and by the flexible fallback below) so callers don't need to know which field a given provider used.

Types

type BaseProvider 

type BaseProvider struct {
	// contains filtered or unexported fields
}

BaseProvider implements common functionality for all providers

func NewBaseProvider 

func NewBaseProvider(name string, clientType ClientType, endpoint string, apiKey string) *BaseProvider

NewBaseProvider creates a base provider with common settings

func (*BaseProvider) EstimateCost 

func (p *BaseProvider) EstimateCost(promptTokens, completionTokens int, model string) float64

EstimateCost calculates the estimated cost for a response

func (*BaseProvider) GetEndpoint 

func (p *BaseProvider) GetEndpoint() string

GetEndpoint returns the API endpoint

func (*BaseProvider) GetModel 

func (p *BaseProvider) GetModel() string

GetModel returns the current model

func (*BaseProvider) GetName 

func (p *BaseProvider) GetName() string

GetName returns the provider name

func (*BaseProvider) GetType 

func (p *BaseProvider) GetType() ClientType

GetType returns the provider type

func (*BaseProvider) IsDebug 

func (p *BaseProvider) IsDebug() bool

IsDebug returns whether debug mode is enabled

func (*BaseProvider) MakeAuthRequest 

func (p *BaseProvider) MakeAuthRequest(ctx context.Context, method, url string, body io.Reader) (*http.Request, error)

MakeAuthRequest creates an HTTP request with authentication

func (*BaseProvider) SetDebug 

func (p *BaseProvider) SetDebug(debug bool)

SetDebug enables or disables debug mode

func (*BaseProvider) SetModel 

func (p *BaseProvider) SetModel(model string) error

SetModel sets the current model

func (*BaseProvider) SupportsConversationalVision added in v0.16.19 

func (p *BaseProvider) SupportsConversationalVision() bool

SupportsConversationalVision returns whether the provider handles inline multimodal chat messages (vs. OCR-only models). Default: supportsVision.

func (*BaseProvider) SupportsReasoning 

func (p *BaseProvider) SupportsReasoning() bool

SupportsReasoning returns whether the provider supports reasoning

func (*BaseProvider) SupportsStreaming 

func (p *BaseProvider) SupportsStreaming() bool

SupportsStreaming returns whether the provider supports streaming

func (*BaseProvider) SupportsTools 

func (p *BaseProvider) SupportsTools() bool

SupportsTools returns whether the provider supports tools

func (*BaseProvider) SupportsVision 

func (p *BaseProvider) SupportsVision() bool

SupportsVision returns whether the provider supports vision

func (*BaseProvider) VisionCapabilities added in v0.16.20 

func (p *BaseProvider) VisionCapabilities() VisionCapabilities

VisionCapabilities returns the per-provider vision limits configured on this BaseProvider. The zero value means "unknown — caller should fall back to VisionCapabilitiesOrDefault()". Concrete providers (Anthropic, OpenAI, etc.) populate p.visionCaps at construction; this method just exposes it. SP-103-D3 / AUDIT-GAP-2.

type ChatChoice 

type ChatChoice = core.ChatChoice

type ChatRequest 

type ChatRequest = core.ChatRequest

type ChatResponse 

type ChatResponse = core.ChatResponse

type ChatUsage 

type ChatUsage = core.ChatUsage

type Choice 

type Choice = ChatChoice

type ClientInterface 

type ClientInterface interface {
	SendChatRequest(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool) (*ChatResponse, error)
	SendChatRequestStream(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool, callback StreamCallback) (*ChatResponse, error)
	CheckConnection() error
	SetDebug(debug bool)
	SetModel(model string) error
	GetModel() string
	GetProvider() string
	GetModelContextLimit() (int, error)
	ListModels(ctx context.Context) ([]ModelInfo, error)
	SupportsVision() bool
	// SupportsConversationalVision reports whether the model is suitable as
	// the inline multimodal target for chat-format vision messages. Some
	// models (OCR-only, e.g. glm-ocr) accept image input but produce
	// extraction outputs unsuitable for free-form multimodal conversation.
	// Defaults to true when SupportsVision() is true.
	SupportsConversationalVision() bool
	// VisionCapabilities returns the per-provider vision limits (max bytes
	// per image, max images per request, max dimension, supported detail
	// tiers). Zero-valued fields mean "unknown — use default". Concrete
	// implementations populate this with provider-specific data; callers
	// should pass the result through VisionCapabilitiesOrDefault() before
	// reading individual fields. SP-103-D3 / AUDIT-GAP-2.
	VisionCapabilities() VisionCapabilities
	GetVisionModel() string
	SendVisionRequest(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool) (*ChatResponse, error)
	// TPS (Tokens Per Second) tracking methods
	GetLastTPS() float64
	GetAverageTPS() float64
	GetTPSStats() map[string]float64
	ResetTPSStats()
}

ClientInterface defines the common interface for all API clients.

The ctx on Send* methods is forwarded to the underlying HTTP request (via http.NewRequestWithContext) so callers can abort in-flight LLM calls when the user clicks Stop. See SP-034 for the design.

func NewDeepInfraClientWrapper 

func NewDeepInfraClientWrapper(model string) (ClientInterface, error)

NewDeepInfraClientWrapper creates a DeepInfra client wrapper

func NewOpenRouterClientWrapper 

func NewOpenRouterClientWrapper(model string) (ClientInterface, error)

NewOpenRouterClientWrapper is deprecated - use factory.CreateProviderClient instead

func NewUnifiedClient 

func NewUnifiedClient(clientType ClientType) (ClientInterface, error)

NewUnifiedClient creates a client with default model for the provider

func NewUnifiedClientWithModel 

func NewUnifiedClientWithModel(clientType ClientType, model string) (ClientInterface, error)

NewUnifiedClientWithModel creates a client with a specific model

type ClientType 

type ClientType string

ClientType represents the type of client to use 

const (
	OllamaClientType      ClientType = "ollama" // Alias for ollama-local
	OllamaLocalClientType ClientType = "ollama-local"
	OllamaCloudClientType ClientType = "ollama-cloud"
	TestClientType        ClientType = "test"   // Mock provider for CI/testing
	EditorClientType      ClientType = "editor" // Editor-only mode, no AI provider
)

Special providers that don't have configs (or have special behavior) 

const (
	OpenAIClientType     ClientType = "openai"
	OpenRouterClientType ClientType = "openrouter"
	ZAIClientType        ClientType = "zai"
	DeepInfraClientType  ClientType = "deepinfra"
	DeepSeekClientType   ClientType = "deepseek"
	LMStudioClientType   ClientType = "lmstudio"
	MistralClientType    ClientType = "mistral"
	MinimaxClientType    ClientType = "minimax"
	ChutesClientType     ClientType = "chutes"
	CerebrasClientType   ClientType = "cerebras"
	ZAICodingClientType  ClientType = "zai-coding"
)

Built-in provider names (ClientType constants) are maintained here to provide type-safe constants. These are kept in sync with provider_gen.go (generated by go generate from provider configs). When adding a new provider, update both the provider config and add a ClientType constant here.

Use providers.AllProviderNames(), providers.KnownProviders(), or providers.ProviderDisplayNames() to get the list of providers without hardcoding them in this package.

func BuiltInClientTypes added in v0.17.7 

func BuiltInClientTypes() []ClientType

BuiltInClientTypes returns the canonical list of all built-in provider ClientType constants defined in this package. Useful for callers that need to enumerate providers (e.g. building reverse display-name maps) without having to hardcode the list in multiple places.

Note: this list intentionally excludes aliases like "ollama" — those are resolved by ParseProviderName to a canonical ClientType. Display name → ClientType lookup handles the alias separately (see configuration.MapProviderStringToClientType).

func DetermineProvider 

func DetermineProvider(explicitProvider string, lastUsedProvider ClientType) (ClientType, error)

DetermineProvider provides unified provider detection with clear precedence: 1. Command-line flag (if provided) 2. Environment variable (SPROUT_PROVIDER, with SPROUT_PROVIDER backward-compat) 3. Config file (last_used_provider)

Returns an error if no available provider is found; callers should surface this to the user and offer interactive provider selection.

func ParseProviderName 

func ParseProviderName(name string) (ClientType, error)

ParseProviderName converts a string provider name to ClientType Handles special aliases and validates provider names.

type HTTPClient 

type HTTPClient interface {
	Do(req *http.Request) (*http.Response, error)
}

HTTPClient interface for testing

type HarmonyFormatter 

type HarmonyFormatter struct {
	// contains filtered or unexported fields
}

HarmonyFormatter handles conversion from OpenAI format to harmony format

func NewHarmonyFormatter 

func NewHarmonyFormatter() *HarmonyFormatter

NewHarmonyFormatter creates a new harmony formatter

func NewHarmonyFormatterWithReasoning 

func NewHarmonyFormatterWithReasoning(reasoning string) *HarmonyFormatter

NewHarmonyFormatterWithReasoning creates a harmony formatter with specific reasoning level

func (*HarmonyFormatter) AddReturnToken 

func (h *HarmonyFormatter) AddReturnToken(response string) string

AddReturnToken adds the completion token to a harmony response

func (*HarmonyFormatter) ConvertReturnToEnd 

func (h *HarmonyFormatter) ConvertReturnToEnd(conversation string) string

ConvertReturnToEnd converts <|return|> tokens to <|end|> for conversation history

func (*HarmonyFormatter) FormatMessagesForCompletion 

func (h *HarmonyFormatter) FormatMessagesForCompletion(messages []Message, tools []Tool, opts *HarmonyOptions) string

FormatMessagesForCompletion converts OpenAI-style messages to harmony format

func (*HarmonyFormatter) StripReturnToken 

func (h *HarmonyFormatter) StripReturnToken(response string) string

StripReturnToken removes <|return|> tokens from model responses

type HarmonyOptions 

type HarmonyOptions struct {
	ReasoningLevel string // "low", "medium", "high" - empty disables explicit reasoning tag
	EnableAnalysis bool   // Whether to enable analysis channel guidance
}

HarmonyOptions configures the harmony formatting

type ImageData 

type ImageData = core.ImageData

type Message 

type Message = core.Message

type ModelDetails 

type ModelDetails struct {
	ID              string
	Name            string
	ContextLength   int
	InputCostPer1K  float64
	OutputCostPer1K float64
	Features        []string // e.g., "vision", "tools", "reasoning"
	IsDefault       bool
}

ModelDetails represents detailed information about a model from a provider

type ModelInfo 

type ModelInfo struct {
	ID              string   `json:"id"`
	Name            string   `json:"name,omitempty"`
	Description     string   `json:"description,omitempty"`
	Provider        string   `json:"provider,omitempty"`
	Size            string   `json:"size,omitempty"`
	Cost            float64  `json:"cost,omitempty"`
	InputCost       float64  `json:"input_cost,omitempty"`
	OutputCost      float64  `json:"output_cost,omitempty"`
	CachedInputCost float64  `json:"cached_input_cost,omitempty"`
	ContextLength   int      `json:"context_length,omitempty"`
	Tags            []string `json:"tags,omitempty"`
	// EligibleRoles lists the agentic roles a model meets the minimum
	// deterministic bar for ("primary", "subagent"). This is an
	// eligibility pre-filter (currently context-window based), NOT a
	// quality recommendation — the capability probe provides the
	// authoritative agentic-capable signal. Empty means below the bar or
	// unknown. Additive/omitempty so older clients ignore it.
	EligibleRoles []string `json:"eligible_roles,omitempty"`
	// RecommendedRoles ⊆ EligibleRoles, gated on passing the capability probe
	// (subagent ← gates passed, primary ← complex stage passed). Empty when
	// un-probed or not recommended. Populated from the published registry.
	RecommendedRoles []string `json:"recommended_roles,omitempty"`
	// Warnings are non-blocking caveats to surface in the picker (e.g. a small
	// context window in the 64K–128K band). Populated from the published registry.
	Warnings []string `json:"warnings,omitempty"`
	// VisionProbe carries the probe-tested vision capability for this model.
	// nil = never probed (fall back to config-based detection).
	// Non-nil = probe ground truth (true = can see images, false = cannot).
	VisionProbe *bool `json:"vision_probe,omitempty"`
}

ModelInfo represents information about an available model

func CanonicalToModelInfo 

func CanonicalToModelInfo(m modelcontract.CanonicalModel) ModelInfo

CanonicalToModelInfo projects a canonical model down to the ModelInfo shape existing consumers expect. Known-true capabilities are surfaced as Tags so callers that inspect tags (e.g. the CLI "Supports tools" line) work unchanged. Exported for the registry publisher (cmd/refresh_provider_catalog).

func GetAvailableModels 

func GetAvailableModels() ([]ModelInfo, error)

GetAvailableModels returns available models for the current provider. Returns an error if no provider can be determined; callers should surface this to the user and offer interactive provider selection.

func GetModelsForProvider 

func GetModelsForProvider(clientType ClientType) ([]ModelInfo, error)

GetModelsForProvider returns available models for a specific provider

func GetModelsForProviderCtx 

func GetModelsForProviderCtx(ctx context.Context, clientType ClientType) ([]ModelInfo, error)

GetModelsForProviderCtx returns available models for a specific provider with context support. It checks the model registry first (if enabled), falling back to direct per-provider API calls.

type ModelSelection 

type ModelSelection struct {
	// contains filtered or unexported fields
}

ModelSelection represents a model selection system This is a stub implementation for backward compatibility The actual model selection logic has been moved to configuration-based system

func NewModelSelection 

func NewModelSelection(config interface{}) *ModelSelection

NewModelSelection creates a new ModelSelection instance This is a stub for backward compatibility - the actual model selection is now handled through the configuration system

type ModelsListInterface 

type ModelsListInterface interface {
	ListAvailableModels() ([]ModelInfo, error)
	GetDefaultModel() string
	IsModelAvailable(modelID string) bool
}

ModelsListInterface defines methods for listing available models

type OllamaLocalClient 

type OllamaLocalClient struct {
	*TPSBase
	// contains filtered or unexported fields
}

OllamaLocalClient handles local Ollama API requests

func NewOllamaLocalClient 

func NewOllamaLocalClient(model string) (*OllamaLocalClient, error)

NewOllamaLocalClient creates a new local Ollama client

func (*OllamaLocalClient) CheckConnection 

func (c *OllamaLocalClient) CheckConnection() error

CheckConnection verifies local Ollama is accessible

func (*OllamaLocalClient) GetCachedModel added in v0.17.7 

func (c *OllamaLocalClient) GetCachedModel(name string) (int, bool)

GetCachedModel returns a cached context length when the model list is fresh. Matches by ID or Name; callers typically pass the Ollama tag (e.g. "llama3:8b") which ListModels stores in both fields for symmetry with lmStudioListModelsWrapper.

func (*OllamaLocalClient) GetModel 

func (c *OllamaLocalClient) GetModel() string

GetModel returns the current model

func (*OllamaLocalClient) GetModelContextLimit 

func (c *OllamaLocalClient) GetModelContextLimit() (int, error)

GetModelContextLimit returns the context limit for the model.

Resolution layers (most-specific to least):

  1. Cached list-models entry whose context Ollama reported via /api/show.
  2. Fresh /api/show lookup for the current model (2s timeout; log on failure so a misconfigured Ollama server doesn't silently degrade to the wrong default context length).
  3. Static DefaultContextLimit from config (set at construction).
  4. Hardcoded 32000 fallback.

Replaces the previous substring match on "qwen3-coder"/"gpt-oss", which missed most models and silently returned 32k for everything else.

func (*OllamaLocalClient) GetProvider 

func (c *OllamaLocalClient) GetProvider() string

GetProvider returns the provider name

func (*OllamaLocalClient) GetVisionModel 

func (c *OllamaLocalClient) GetVisionModel() string

GetVisionModel returns empty string as vision is not supported

func (*OllamaLocalClient) ListModels 

func (c *OllamaLocalClient) ListModels(ctx context.Context) ([]ModelInfo, error)

ListModels returns available local models.

func (*OllamaLocalClient) SendChatRequest 

func (c *OllamaLocalClient) SendChatRequest(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool) (*ChatResponse, error)

SendChatRequest sends a chat request to local Ollama

func (*OllamaLocalClient) SendChatRequestStream 

func (c *OllamaLocalClient) SendChatRequestStream(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool, callback StreamCallback) (*ChatResponse, error)

SendChatRequestStream streams responses from local Ollama as they arrive

func (*OllamaLocalClient) SendVisionRequest 

func (c *OllamaLocalClient) SendVisionRequest(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool) (*ChatResponse, error)

SendVisionRequest handles vision/OCR requests for Ollama Delegates to SendChatRequest since the image handling is done in buildChatRequest

func (*OllamaLocalClient) SetDebug 

func (c *OllamaLocalClient) SetDebug(debug bool)

SetDebug enables or disables debug mode

func (*OllamaLocalClient) SetModel 

func (c *OllamaLocalClient) SetModel(model string) error

SetModel updates the active model after validating it exists locally

func (*OllamaLocalClient) SupportsConversationalVision added in v0.16.19 

func (c *OllamaLocalClient) SupportsConversationalVision() bool

SupportsConversationalVision returns true only for multimodal chat models. OCR-only models (e.g. glm-ocr) accept images but produce extraction output that doesn't help free-form conversational turns — the tool path (analyze_image_content) is the right channel for them. Inline embedding is only useful for chat models like llama3.2-vision.

func (*OllamaLocalClient) SupportsVision 

func (c *OllamaLocalClient) SupportsVision() bool

SupportsVision returns true for OCR-capable models

func (*OllamaLocalClient) VisionCapabilities added in v0.16.20 

func (c *OllamaLocalClient) VisionCapabilities() VisionCapabilities

VisionCapabilities returns the local-Ollama vision limits used by llama3.2-vision / glm-ocr family models.

Conservative defaults reflect the documented Ollama API constraints: the older base64 payload cap is ~3.5MB (we use 5MB), llama3.2-vision works best at 1024px on the longest side, and we cap to a handful of images per request since local context windows are tight. Detail tiers are intentionally left nil — Ollama picks automatically. The returned values are static (don't depend on c.model) so the table is safe to share across clients; per-model overrides can be added later. SP-103-D3 / AUDIT-GAP-2.

type Provider 

type Provider interface {
	// Core functionality
	SendChatRequest(ctx context.Context, req *ProviderChatRequest) (*ChatResponse, error)
	CheckConnection(ctx context.Context) error

	// Model management
	GetModel() string
	SetModel(model string) error
	GetAvailableModels(ctx context.Context) ([]ModelDetails, error)
	GetModelContextLimit() (int, error)

	// Provider information
	GetName() string
	GetType() ClientType
	GetEndpoint() string

	// Feature support
	SupportsVision() bool
	// SupportsConversationalVision reports whether the provider is suitable as
	// the inline multimodal target for chat-format vision messages. OCR-only
	// models accept image input but produce extraction outputs unsuitable for
	// free-form multimodal conversation. Default: SupportsVision().
	SupportsConversationalVision() bool
	SupportsTools() bool
	SupportsStreaming() bool
	SupportsReasoning() bool
	// VisionCapabilities returns the per-provider vision limits (max bytes
	// per image, max images per request, max dimension, supported detail
	// tiers). Zero-valued fields mean "unknown" and should be filled from
	// VisionCapabilitiesDefault() at the call site. SP-103-D3 / AUDIT-GAP-2.
	VisionCapabilities() VisionCapabilities

	// Configuration
	SetDebug(debug bool)
	IsDebug() bool
}

Provider defines the interface all LLM providers must implement

func CreateProviderFromClient 

func CreateProviderFromClient(clientType ClientType, client ClientInterface) Provider

CreateProviderFromClient creates a Provider from an existing ClientInterface

type ProviderAdapter 

type ProviderAdapter struct {
	// contains filtered or unexported fields
}

ProviderAdapter adapts the existing ClientInterface to the new Provider interface

func NewProviderAdapter 

func NewProviderAdapter(clientType ClientType, client ClientInterface) *ProviderAdapter

NewProviderAdapter creates an adapter for existing clients

func (*ProviderAdapter) CheckConnection 

func (a *ProviderAdapter) CheckConnection(ctx context.Context) error

CheckConnection verifies connectivity

func (*ProviderAdapter) GetAvailableModels 

func (a *ProviderAdapter) GetAvailableModels(ctx context.Context) ([]ModelDetails, error)

GetAvailableModels returns available models for this provider

func (*ProviderAdapter) GetEndpoint 

func (a *ProviderAdapter) GetEndpoint() string

GetEndpoint returns the API endpoint from the underlying client. Falls back to an empty string if the client doesn't expose GetEndpoint.

func (*ProviderAdapter) GetModel 

func (a *ProviderAdapter) GetModel() string

GetModel returns the current model

func (*ProviderAdapter) GetModelContextLimit 

func (a *ProviderAdapter) GetModelContextLimit() (int, error)

GetModelContextLimit returns the context window size

func (*ProviderAdapter) GetName 

func (a *ProviderAdapter) GetName() string

GetName returns the provider name

func (*ProviderAdapter) GetType 

func (a *ProviderAdapter) GetType() ClientType

GetType returns the provider type

func (*ProviderAdapter) IsDebug 

func (a *ProviderAdapter) IsDebug() bool

IsDebug returns whether debug mode is enabled

func (*ProviderAdapter) SendChatRequest 

func (a *ProviderAdapter) SendChatRequest(ctx context.Context, req *ProviderChatRequest) (*ChatResponse, error)

SendChatRequest adapts the old interface to the new one.

Note: This uses the same global per-provider rate limiter as APIClient.sendRequest(). Both paths share one bucket per provider to coordinate across all agents, preventing cascading 429s when multiple subagents run concurrently. Do NOT add additional rate limiting at this layer without coordinating with pkg/agent/api_client.go.

func (*ProviderAdapter) SetDebug 

func (a *ProviderAdapter) SetDebug(debug bool)

SetDebug enables or disables debug mode

func (*ProviderAdapter) SetModel 

func (a *ProviderAdapter) SetModel(model string) error

SetModel sets the current model

func (*ProviderAdapter) SupportsConversationalVision added in v0.16.19 

func (a *ProviderAdapter) SupportsConversationalVision() bool

SupportsConversationalVision returns whether the provider handles inline multimodal chat messages. Delegates to the underlying client; falls back to SupportsVision() if the client doesn't implement the new method.

func (*ProviderAdapter) SupportsReasoning 

func (a *ProviderAdapter) SupportsReasoning() bool

SupportsReasoning returns whether the provider supports reasoning

func (*ProviderAdapter) SupportsStreaming 

func (a *ProviderAdapter) SupportsStreaming() bool

SupportsStreaming returns whether the provider supports streaming

func (*ProviderAdapter) SupportsTools 

func (a *ProviderAdapter) SupportsTools() bool

SupportsTools returns whether the provider supports tools

func (*ProviderAdapter) SupportsVision 

func (a *ProviderAdapter) SupportsVision() bool

SupportsVision returns whether the provider supports vision

func (*ProviderAdapter) VisionCapabilities added in v0.16.20 

func (a *ProviderAdapter) VisionCapabilities() VisionCapabilities

VisionCapabilities returns the per-provider vision limits by delegating to the wrapped client. If the client does not implement VisionCapabilities() (e.g. legacy / mock clients), returns the zero value; callers should pass that through VisionCapabilitiesOrDefault() to get a safe usable configuration. SP-103-D3 / AUDIT-GAP-2.

type ProviderChatRequest 

type ProviderChatRequest struct {
	Messages []Message
	Tools    []Tool
	Options  *RequestOptions
}

ProviderChatRequest represents a unified request structure for providers This extends the basic ChatRequest with provider-specific options

type ProviderInterface 

type ProviderInterface interface {
	SendChatRequest(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool) (*ChatResponse, error)
	SendChatRequestStream(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool, callback StreamCallback) (*ChatResponse, error)
	CheckConnection() error
	SetDebug(debug bool)
	SetModel(model string) error
	GetModel() string
	GetProvider() string
	GetModelContextLimit() (int, error)
	ListModels(ctx context.Context) ([]ModelInfo, error)
	SupportsVision() bool
	SendVisionRequest(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool) (*ChatResponse, error)
}

ProviderInterface defines the interface that all providers must implement. Mirrors ClientInterface's ctx-bearing send methods — see SP-034.

type RequestOptions 

type RequestOptions struct {
	Temperature      *float64
	MaxTokens        *int
	TopP             *float64
	FrequencyPenalty *float64
	PresencePenalty  *float64
	StopSequences    []string
	Stream           bool
	ReasoningEffort  string // For reasoning models
	DisableThinking  *bool  // Disable thinking/reasoning mode for thinking-capable models
}

RequestOptions contains optional parameters for requests

type SSEReader 

type SSEReader struct {
	// contains filtered or unexported fields
}

SSEReader reads Server-Sent Events from a reader

func NewSSEReader 

func NewSSEReader(r io.Reader, onEvent func(event, data string) error) *SSEReader

NewSSEReader creates a new SSE reader

func (*SSEReader) Read 

func (r *SSEReader) Read() error

Read processes the SSE stream

func (*SSEReader) ReadWithTimeout 

func (r *SSEReader) ReadWithTimeout(timeout time.Duration) error

ReadWithTimeout processes the SSE stream with a timeout. Each line read is performed in a short-lived goroutine so that the select can respond to the timeout without blocking the main loop. The background goroutine will exit when the underlying HTTP response body is closed (which triggers ReadString to return an error).

type StreamCallback 

type StreamCallback func(content string, contentType string)

StreamCallback is called for each content chunk received. contentType is "assistant_text" for regular content or "reasoning" for thinking/reasoning content.

type StreamingChatResponse 

type StreamingChatResponse struct {
	ID      string            `json:"id"`
	Object  string            `json:"object"`
	Created int64             `json:"created"`
	Model   string            `json:"model"`
	Choices []StreamingChoice `json:"choices"`
	Usage   *StreamingUsage   `json:"usage,omitempty"`
}

StreamingChatResponse represents a streaming response chunk

func ParseSSEData 

func ParseSSEData(data string) (*StreamingChatResponse, error)

ParseSSEData parses SSE data into a streaming response

type StreamingChoice 

type StreamingChoice struct {
	Index        int            `json:"index"`
	Delta        StreamingDelta `json:"delta"`
	FinishReason *string        `json:"finish_reason"`
}

StreamingChoice represents a streaming response choice

type StreamingDelta 

type StreamingDelta struct {
	Role             string              `json:"role,omitempty"`
	Content          string              `json:"content,omitempty"`
	Reasoning        string              `json:"reasoning,omitempty"` // GLM reasoning field
	ReasoningContent string              `json:"reasoning_content,omitempty"`
	ReasoningDetails json.RawMessage     `json:"reasoning_details,omitempty"` // Can be string (Minimax) or array (GLM models)
	ToolCalls        []StreamingToolCall `json:"tool_calls,omitempty"`
}

StreamingDelta contains incremental updates

type StreamingResponseBuilder 

type StreamingResponseBuilder struct {
	// contains filtered or unexported fields
}

StreamingResponseBuilder accumulates streaming chunks into a complete response

func NewStreamingResponseBuilder 

func NewStreamingResponseBuilder(callback StreamCallback) *StreamingResponseBuilder

NewStreamingResponseBuilder creates a new streaming response builder

func (*StreamingResponseBuilder) GetResponse 

func (b *StreamingResponseBuilder) GetResponse() *ChatResponse

GetResponse returns the accumulated response

func (*StreamingResponseBuilder) GetTokenGenerationDuration 

func (b *StreamingResponseBuilder) GetTokenGenerationDuration() time.Duration

GetTokenGenerationDuration returns the duration from first token to last token

func (*StreamingResponseBuilder) ProcessChunk 

func (b *StreamingResponseBuilder) ProcessChunk(chunk *StreamingChatResponse) error

ProcessChunk processes a streaming chunk and updates the builder state

type StreamingToolCall 

type StreamingToolCall struct {
	Index    int                        `json:"index"`
	ID       string                     `json:"id,omitempty"`
	Type     string                     `json:"type,omitempty"`
	Function *StreamingToolCallFunction `json:"function,omitempty"`
}

StreamingToolCall represents an incremental tool call update

type StreamingToolCallFunction 

type StreamingToolCallFunction struct {
	Name      string `json:"name,omitempty"`
	Arguments string `json:"arguments,omitempty"`
}

StreamingToolCallFunction contains function details

type StreamingUsage 

type StreamingUsage struct {
	PromptTokens        int     `json:"prompt_tokens"`
	CompletionTokens    int     `json:"completion_tokens"`
	TotalTokens         int     `json:"total_tokens"`
	EstimatedCost       float64 `json:"estimated_cost"`
	Cost                float64 `json:"cost,omitempty"` // OpenRouter returns cost directly
	ImageTokens         int     `json:"image_tokens,omitempty"`
	PromptTokensDetails struct {
		CachedTokens     int  `json:"cached_tokens"`
		CacheWriteTokens *int `json:"cache_write_tokens"`
	} `json:"prompt_tokens_details,omitempty"`
}

StreamingUsage is the usage block on a streaming chunk (providers send it on the final data object). Named (not anonymous) so the flexible cost fallback in ParseSSEData can allocate one when a provider reports cost without the standard token block.

type TPSBase 

type TPSBase struct {
	// contains filtered or unexported fields
}

TPSBase provides a default implementation of TPS tracking methods Other providers can embed this struct to get TPS functionality

func NewTPSBase 

func NewTPSBase() *TPSBase

NewTPSBase creates a new TPS base with tracker

func (*TPSBase) GetAverageTPS 

func (t *TPSBase) GetAverageTPS() float64

GetAverageTPS returns the average TPS across all requests

func (*TPSBase) GetLastTPS 

func (t *TPSBase) GetLastTPS() float64

GetLastTPS returns the most recent TPS measurement

func (*TPSBase) GetTPSStats 

func (t *TPSBase) GetTPSStats() map[string]float64

GetTPSStats returns comprehensive TPS statistics

func (*TPSBase) GetTracker 

func (t *TPSBase) GetTracker() *TPSTracker

GetTracker returns the underlying TPS tracker

func (*TPSBase) ResetTPSStats 

func (t *TPSBase) ResetTPSStats()

ResetTPSStats clears all TPS tracking data

type TPSTracker 

type TPSTracker struct {
	// contains filtered or unexported fields
}

func NewTPSTracker 

func NewTPSTracker() *TPSTracker

NewTPSTracker creates a new TPS tracker

func (*TPSTracker) GetAverageTPS 

func (t *TPSTracker) GetAverageTPS() float64

GetAverageTPS returns the average TPS across all recorded requests

func (*TPSTracker) GetCurrentTPS 

func (t *TPSTracker) GetCurrentTPS() float64

GetCurrentTPS returns the most recent TPS value

func (*TPSTracker) GetSmoothTPS 

func (t *TPSTracker) GetSmoothTPS() float64

GetSmoothTPS returns an exponentially smoothed TPS value

func (*TPSTracker) GetStats 

func (t *TPSTracker) GetStats() map[string]interface{}

GetStats returns comprehensive TPS statistics

func (*TPSTracker) RecordRequest 

func (t *TPSTracker) RecordRequest(duration time.Duration, completionTokens int) float64

RecordRequest records the timing and token usage of an API request

func (*TPSTracker) Reset 

func (t *TPSTracker) Reset()

Reset clears all TPS tracking data

type Tool 

type Tool = core.Tool

type ToolCall 

type ToolCall = core.ToolCall

func RecoverMistralToolCalls 

func RecoverMistralToolCalls(content string) (calls []ToolCall, remaining string, ok bool)

RecoverMistralToolCalls recovers tool calls from the Mistral-family text format, where the model emits `[TOOL_CALLS]…` inside the message content instead of the structured tool_calls field. It returns the recovered calls and the content with the marker and its payload stripped. ok is false when no marker is present or nothing parseable follows it (content is returned unchanged in that case).

Two payload shapes are handled: 

[TOOL_CALLS][{"name":"f","arguments":{…}}, …]   (JSON array — Mistral native)
[TOOL_CALLS]f{…}g{…}                             (name + JSON object, repeated)

type ToolCallFunction 

type ToolCallFunction = core.ToolCallFunction

type ToolFunction 

type ToolFunction = core.ToolFunction

type ToolParameter 

type ToolParameter = core.ToolParameter

type ToolParameters 

type ToolParameters = core.ToolParameters

type UnifiedProviderWrapper 

type UnifiedProviderWrapper struct {
	*TPSBase
	// contains filtered or unexported fields
}

UnifiedProviderWrapper wraps any provider that implements ProviderInterface

func NewUnifiedProviderWrapper 

func NewUnifiedProviderWrapper(provider ProviderInterface) *UnifiedProviderWrapper

NewUnifiedProviderWrapper creates a wrapper for any provider

func (*UnifiedProviderWrapper) CheckConnection 

func (w *UnifiedProviderWrapper) CheckConnection() error

Forward all other methods to the provider

func (*UnifiedProviderWrapper) GetLastRequestTPS 

func (w *UnifiedProviderWrapper) GetLastRequestTPS() float64

GetLastRequestTPS returns the TPS for the last API request

func (*UnifiedProviderWrapper) GetModel 

func (w *UnifiedProviderWrapper) GetModel() string

func (*UnifiedProviderWrapper) GetModelContextLimit 

func (w *UnifiedProviderWrapper) GetModelContextLimit() (int, error)

func (*UnifiedProviderWrapper) GetProvider 

func (w *UnifiedProviderWrapper) GetProvider() string

func (*UnifiedProviderWrapper) GetTPSStatistics 

func (w *UnifiedProviderWrapper) GetTPSStatistics() (float64, float64, int)

GetTPSStatistics returns tokens per second statistics

func (*UnifiedProviderWrapper) GetVisionModel 

func (w *UnifiedProviderWrapper) GetVisionModel() string

func (*UnifiedProviderWrapper) ListModels 

func (w *UnifiedProviderWrapper) ListModels(ctx context.Context) ([]ModelInfo, error)

func (*UnifiedProviderWrapper) ResetTPSStatistics 

func (w *UnifiedProviderWrapper) ResetTPSStatistics()

ResetTPSStatistics resets the TPS tracking

func (*UnifiedProviderWrapper) SendChatRequest 

func (w *UnifiedProviderWrapper) SendChatRequest(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool) (*ChatResponse, error)

SendChatRequest converts types and forwards to provider

func (*UnifiedProviderWrapper) SendChatRequestStream 

func (w *UnifiedProviderWrapper) SendChatRequestStream(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool, callback StreamCallback) (*ChatResponse, error)

SendChatRequestStream sends a streaming chat request (not yet implemented for unified providers)

func (*UnifiedProviderWrapper) SendVisionRequest 

func (w *UnifiedProviderWrapper) SendVisionRequest(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool) (*ChatResponse, error)

func (*UnifiedProviderWrapper) SetDebug 

func (w *UnifiedProviderWrapper) SetDebug(debug bool)

func (*UnifiedProviderWrapper) SetModel 

func (w *UnifiedProviderWrapper) SetModel(model string) error

func (*UnifiedProviderWrapper) SupportsConversationalVision added in v0.16.19 

func (w *UnifiedProviderWrapper) SupportsConversationalVision() bool

SupportsConversationalVision reports whether inline multimodal turns should embed the image. Delegates to the underlying provider if it implements the method; otherwise falls back to SupportsVision().

func (*UnifiedProviderWrapper) SupportsVision 

func (w *UnifiedProviderWrapper) SupportsVision() bool

func (*UnifiedProviderWrapper) VisionCapabilities added in v0.16.20 

func (w *UnifiedProviderWrapper) VisionCapabilities() VisionCapabilities

VisionCapabilities returns the per-provider vision limits by delegating to the wrapped provider. If the provider does not implement VisionCapabilities() (e.g. legacy third-party providers), returns the zero value; callers should run the result through VisionCapabilitiesOrDefault() to fill in safe defaults. SP-103-D3 / AUDIT-GAP-2.

type VisionCapabilities added in v0.16.20 

type VisionCapabilities struct {
	// MaxImageBytes is the largest single image the provider accepts
	// inline (oversized images must be resized before embedding).
	MaxImageBytes int
	// MaxImageCount is the max number of inline images per request.
	MaxImageCount int
	// MaxImageDimension is the longest-side cap (px). Providers differ
	// widely: Anthropic auto-resizes to 1568px, OpenAI keeps native
	// at low/auto, Gemini up to 3072 in some configs. Oversized
	// images should be resized to MaxImageDimension before embedding.
	MaxImageDimension int
	// DetailTiers lists supported detail levels (e.g. "low","high" or
	// "low","high","auto"). Empty means the provider picks automatically.
	DetailTiers []string
}

VisionCapabilities describes per-provider vision limits. Populated from provider/model metadata and read once at construction. Fields with zero values mean "unknown — use default".

Why per provider? Real provider limits diverge sharply:

  • Anthropic caps single-image base64 payloads at ~5MB and auto-resizes to 1568px on the longest side, accepting up to 100 images per turn.
  • OpenAI's gpt-4o accepts ~20MB images and supports low/high/auto detail tiers, with up to 500 images in some endpoints.
  • Local Ollama (llama3.2-vision) caps out around 3.5MB and works best at 1024px on the longest side with only a handful of images.

A single 1536px + 5MB + N-image cap is a poor fit for any of them.

The defaults used when a field is zero are intentionally safe (not permissive): they keep the call working while image-resize / batch-splitting code paths evolve. See VisionCapabilitiesDefault.

func VisionCapabilitiesDefault added in v0.16.20 

func VisionCapabilitiesDefault() VisionCapabilities

VisionCapabilitiesDefault returns the package-wide safe fallback when a provider-specific capability table is empty or missing. The defaults are chosen to be conservative (works on every supported provider) rather than best-of-breed:

  • 5_000_000 bytes ≈ Anthropic's official single-image cap.
  • 20 images per request — well under every provider limit.
  • 1536 px on the longest side — matches the historical hard-coded resize cap so existing behavior is preserved when a provider returns zero fields.

Callers that need provider-tuned behavior should read the actual VisionCapabilities() from the provider, not the defaults.

func VisionCapabilitiesOrDefault added in v0.16.20 

func VisionCapabilitiesOrDefault(caps VisionCapabilities) VisionCapabilities

VisionCapabilitiesOrDefault returns caps with zero-valued fields filled in from VisionCapabilitiesDefault(). Non-zero fields are preserved untouched. Use this at the call site (resize / batch-split) so partially-populated tables still produce safe behavior.

Example: 

caps := client.VisionCapabilities()         // may be all zeros
caps  = api.VisionCapabilitiesOrDefault(caps) // now safe to read
if len(images) > caps.MaxImageCount { ... }

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.