api

package
v0.16.13 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 22, 2026 License: MIT Imports: 27 Imported by: 0

Documentation

Overview

Package api provides API types used across all providers.

Seed canonical types (github.com/sprout-foundry/seed/core) are imported via type aliases. Sprout consumes these types — it does not define them.

Index

Constants

View Source 
const (
	// DefaultBufferTokens is the safety buffer for estimation errors
	DefaultBufferTokens = 1000
	// MinOutputTokens is the minimum output tokens to reserve
	MinOutputTokens = 512
	// ToolTokenEstimate is the approximate token count per tool definition
	ToolTokenEstimate = 200
	// SystemInstructionBuffer accounts for system prompt overhead
	SystemInstructionBuffer = 500
	// MessageOverheadTokens accounts for role/message wrapper overhead
	MessageOverheadTokens = 4
	// ToolCallOverheadTokens accounts for assistant tool_call wrapper overhead
	ToolCallOverheadTokens = 12
	// ToolCallIDOverheadTokens accounts for tool response tool_call_id overhead
	ToolCallIDOverheadTokens = 8
	// ImageMessageOverheadTokens conservatively accounts for multimodal image parts
	ImageMessageOverheadTokens = 256
)

Token estimation constants

Variables

This section is empty.

Functions

func CalculateOutputBudget 

func CalculateOutputBudget(contextLimit int, inputTokens int) (int, bool)

CalculateOutputBudget calculates the safe output token budget given context constraints. It returns the maximum tokens that can be requested for completion. If the input exceeds the context limit, returns 0 and an error message.

func ClassifyEligibleRoles 

func ClassifyEligibleRoles(m ModelInfo) []string

ClassifyEligibleRoles returns the agentic roles a model meets the minimum deterministic bar for, based on its context window. Returns nil when the model is below the subagent threshold or its context length is unknown.

func CostFromJSON 

func CostFromJSON(body []byte) (float64, bool)

CostFromJSON probes raw response JSON for a cost value under any known candidate field name. Used as a fallback when the typed decode (estimated_cost/cost) found nothing, so a provider that names the field differently still surfaces a cost. Returns (0,false) when none match.

func EstimateInputTokens 

func EstimateInputTokens(messages []Message, tools []Tool) int

EstimateInputTokens estimates total input tokens for messages and tools. This includes a buffer for system instructions and message formatting overhead.

func EstimateTokens 

func EstimateTokens(text string) int

EstimateTokens provides a token estimation based on OpenAI's tiktoken approach. This is the centralized implementation that all providers should use for consistency.

func FormatHTTPResponseError 

func FormatHTTPResponseError(statusCode int, headers http.Header, body []byte) error

FormatHTTPResponseError converts an HTTP error response into a concise, user-facing error that avoids dumping full HTML or JSON payloads.

func GetCanonicalModelsForProvider 

func GetCanonicalModelsForProvider(ctx context.Context, clientType ClientType) ([]modelcontract.CanonicalModel, error)

GetCanonicalModelsForProvider returns canonical models for a provider — from its adapter where one exists, otherwise by projecting the legacy ModelInfo path up to canonical. Used by the registry publisher to emit the canonical per-provider file.

func GetProviderName 

func GetProviderName(clientType ClientType) string

GetProviderName returns the human-readable name for a provider

func IsProviderAvailable 

func IsProviderAvailable(provider ClientType) bool

IsProviderAvailable checks if a provider can be used. Uses credentials.HasProviderCredential to avoid hardcoding env var strings.

func ModelPricingPerMillion 

func ModelPricingPerMillion(entry map[string]any) (inputPerMillion, outputPerMillion float64)

ModelPricingPerMillion extracts a model's input/output price (USD per million tokens) from a raw /models listing entry, probing candidate field names/units so listings that name pricing differently still surface a price. Returns (0,0) when no candidate matches.

func UsageCost 

func UsageCost(u ChatUsage) float64

UsageCost returns the canonical cost from a typed ChatUsage, preferring the provider-reported Cost over EstimatedCost. Both are populated by the normal decode (and by the flexible fallback below) so callers don't need to know which field a given provider used.

Types

type BaseProvider 

type BaseProvider struct {
	// contains filtered or unexported fields
}

BaseProvider implements common functionality for all providers

func NewBaseProvider 

func NewBaseProvider(name string, clientType ClientType, endpoint string, apiKey string) *BaseProvider

NewBaseProvider creates a base provider with common settings

func (*BaseProvider) EstimateCost 

func (p *BaseProvider) EstimateCost(promptTokens, completionTokens int, model string) float64

EstimateCost calculates the estimated cost for a response

func (*BaseProvider) GetEndpoint 

func (p *BaseProvider) GetEndpoint() string

GetEndpoint returns the API endpoint

func (*BaseProvider) GetModel 

func (p *BaseProvider) GetModel() string

GetModel returns the current model

func (*BaseProvider) GetName 

func (p *BaseProvider) GetName() string

GetName returns the provider name

func (*BaseProvider) GetType 

func (p *BaseProvider) GetType() ClientType

GetType returns the provider type

func (*BaseProvider) IsDebug 

func (p *BaseProvider) IsDebug() bool

IsDebug returns whether debug mode is enabled

func (*BaseProvider) MakeAuthRequest 

func (p *BaseProvider) MakeAuthRequest(ctx context.Context, method, url string, body io.Reader) (*http.Request, error)

MakeAuthRequest creates an HTTP request with authentication

func (*BaseProvider) SetDebug 

func (p *BaseProvider) SetDebug(debug bool)

SetDebug enables or disables debug mode

func (*BaseProvider) SetModel 

func (p *BaseProvider) SetModel(model string) error

SetModel sets the current model

func (*BaseProvider) SupportsReasoning 

func (p *BaseProvider) SupportsReasoning() bool

SupportsReasoning returns whether the provider supports reasoning

func (*BaseProvider) SupportsStreaming 

func (p *BaseProvider) SupportsStreaming() bool

SupportsStreaming returns whether the provider supports streaming

func (*BaseProvider) SupportsTools 

func (p *BaseProvider) SupportsTools() bool

SupportsTools returns whether the provider supports tools

func (*BaseProvider) SupportsVision 

func (p *BaseProvider) SupportsVision() bool

SupportsVision returns whether the provider supports vision

type ChatChoice 

type ChatChoice = core.ChatChoice

type ChatRequest 

type ChatRequest = core.ChatRequest

type ChatResponse 

type ChatResponse = core.ChatResponse

type ChatUsage 

type ChatUsage = core.ChatUsage

type Choice 

type Choice = ChatChoice

type ClientInterface 

type ClientInterface interface {
	SendChatRequest(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool) (*ChatResponse, error)
	SendChatRequestStream(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool, callback StreamCallback) (*ChatResponse, error)
	CheckConnection() error
	SetDebug(debug bool)
	SetModel(model string) error
	GetModel() string
	GetProvider() string
	GetModelContextLimit() (int, error)
	ListModels(ctx context.Context) ([]ModelInfo, error)
	SupportsVision() bool
	GetVisionModel() string
	SendVisionRequest(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool) (*ChatResponse, error)
	// TPS (Tokens Per Second) tracking methods
	GetLastTPS() float64
	GetAverageTPS() float64
	GetTPSStats() map[string]float64
	ResetTPSStats()
}

ClientInterface defines the common interface for all API clients.

The ctx on Send* methods is forwarded to the underlying HTTP request (via http.NewRequestWithContext) so callers can abort in-flight LLM calls when the user clicks Stop. See SP-034 for the design.

func NewDeepInfraClientWrapper 

func NewDeepInfraClientWrapper(model string) (ClientInterface, error)

NewDeepInfraClientWrapper creates a DeepInfra client wrapper

func NewOpenRouterClientWrapper 

func NewOpenRouterClientWrapper(model string) (ClientInterface, error)

NewOpenRouterClientWrapper is deprecated - use factory.CreateProviderClient instead

func NewUnifiedClient 

func NewUnifiedClient(clientType ClientType) (ClientInterface, error)

NewUnifiedClient creates a client with default model for the provider

func NewUnifiedClientWithModel 

func NewUnifiedClientWithModel(clientType ClientType, model string) (ClientInterface, error)

NewUnifiedClientWithModel creates a client with a specific model

type ClientType 

type ClientType string

ClientType represents the type of client to use 

const (
	OllamaClientType      ClientType = "ollama" // Alias for ollama-local
	OllamaLocalClientType ClientType = "ollama-local"
	OllamaCloudClientType ClientType = "ollama-cloud"
	TestClientType        ClientType = "test"   // Mock provider for CI/testing
	EditorClientType      ClientType = "editor" // Editor-only mode, no AI provider
)

Special providers that don't have configs (or have special behavior) 

const (
	OpenAIClientType     ClientType = "openai"
	OpenRouterClientType ClientType = "openrouter"
	ZAIClientType        ClientType = "zai"
	DeepInfraClientType  ClientType = "deepinfra"
	DeepSeekClientType   ClientType = "deepseek"
	LMStudioClientType   ClientType = "lmstudio"
	MistralClientType    ClientType = "mistral"
	MinimaxClientType    ClientType = "minimax"
	ChutesClientType     ClientType = "chutes"
	CerebrasClientType   ClientType = "cerebras"
)

Built-in provider names (ClientType constants) are maintained here to provide type-safe constants. These are kept in sync with provider_gen.go (generated by go generate from provider configs). When adding a new provider, update both the provider config and add a ClientType constant here.

Use providers.AllProviderNames(), providers.KnownProviders(), or providers.ProviderDisplayNames() to get the list of providers without hardcoding them in this package.

func DetermineProvider 

func DetermineProvider(explicitProvider string, lastUsedProvider ClientType) (ClientType, error)

DetermineProvider provides unified provider detection with clear precedence: 1. Command-line flag (if provided) 2. Environment variable (SPROUT_PROVIDER, with LEDIT_PROVIDER backward-compat) 3. Config file (last_used_provider) 4. First available provider based on API keys (uses priority order + dynamic discovery) 5. Fallback to Ollama

func ParseProviderName 

func ParseProviderName(name string) (ClientType, error)

ParseProviderName converts a string provider name to ClientType Handles special aliases and validates provider names.

type HTTPClient 

type HTTPClient interface {
	Do(req *http.Request) (*http.Response, error)
}

HTTPClient interface for testing

type HarmonyFormatter 

type HarmonyFormatter struct {
	// contains filtered or unexported fields
}

HarmonyFormatter handles conversion from OpenAI format to harmony format

func NewHarmonyFormatter 

func NewHarmonyFormatter() *HarmonyFormatter

NewHarmonyFormatter creates a new harmony formatter

func NewHarmonyFormatterWithReasoning 

func NewHarmonyFormatterWithReasoning(reasoning string) *HarmonyFormatter

NewHarmonyFormatterWithReasoning creates a harmony formatter with specific reasoning level

func (*HarmonyFormatter) AddReturnToken 

func (h *HarmonyFormatter) AddReturnToken(response string) string

AddReturnToken adds the completion token to a harmony response

func (*HarmonyFormatter) ConvertReturnToEnd 

func (h *HarmonyFormatter) ConvertReturnToEnd(conversation string) string

ConvertReturnToEnd converts <|return|> tokens to <|end|> for conversation history

func (*HarmonyFormatter) FormatMessagesForCompletion 

func (h *HarmonyFormatter) FormatMessagesForCompletion(messages []Message, tools []Tool, opts *HarmonyOptions) string

FormatMessagesForCompletion converts OpenAI-style messages to harmony format

func (*HarmonyFormatter) StripReturnToken 

func (h *HarmonyFormatter) StripReturnToken(response string) string

StripReturnToken removes <|return|> tokens from model responses

type HarmonyOptions 

type HarmonyOptions struct {
	ReasoningLevel string // "low", "medium", "high" - empty disables explicit reasoning tag
	EnableAnalysis bool   // Whether to enable analysis channel guidance
}

HarmonyOptions configures the harmony formatting

type ImageData 

type ImageData = core.ImageData

type Message 

type Message = core.Message

type ModelDetails 

type ModelDetails struct {
	ID              string
	Name            string
	ContextLength   int
	InputCostPer1K  float64
	OutputCostPer1K float64
	Features        []string // e.g., "vision", "tools", "reasoning"
	IsDefault       bool
}

ModelDetails represents detailed information about a model from a provider

type ModelInfo 

type ModelInfo struct {
	ID            string   `json:"id"`
	Name          string   `json:"name,omitempty"`
	Description   string   `json:"description,omitempty"`
	Provider      string   `json:"provider,omitempty"`
	Size          string   `json:"size,omitempty"`
	Cost          float64  `json:"cost,omitempty"`
	InputCost     float64  `json:"input_cost,omitempty"`
	OutputCost    float64  `json:"output_cost,omitempty"`
	ContextLength int      `json:"context_length,omitempty"`
	Tags          []string `json:"tags,omitempty"`
	// EligibleRoles lists the agentic roles a model meets the minimum
	// deterministic bar for ("primary", "subagent"). This is an
	// eligibility pre-filter (currently context-window based), NOT a
	// quality recommendation — the capability probe provides the
	// authoritative agentic-capable signal. Empty means below the bar or
	// unknown. Additive/omitempty so older clients ignore it.
	EligibleRoles []string `json:"eligible_roles,omitempty"`
	// RecommendedRoles ⊆ EligibleRoles, gated on passing the capability probe
	// (subagent ← gates passed, primary ← complex stage passed). Empty when
	// un-probed or not recommended. Populated from the published registry.
	RecommendedRoles []string `json:"recommended_roles,omitempty"`
	// Warnings are non-blocking caveats to surface in the picker (e.g. a small
	// context window in the 64K–128K band). Populated from the published registry.
	Warnings []string `json:"warnings,omitempty"`
}

ModelInfo represents information about an available model

func CanonicalToModelInfo 

func CanonicalToModelInfo(m modelcontract.CanonicalModel) ModelInfo

CanonicalToModelInfo projects a canonical model down to the ModelInfo shape existing consumers expect. Known-true capabilities are surfaced as Tags so callers that inspect tags (e.g. the CLI "Supports tools" line) work unchanged. Exported for the registry publisher (cmd/refresh_provider_catalog).

func GetAvailableModels 

func GetAvailableModels() ([]ModelInfo, error)

GetAvailableModels returns available models for the current provider

func GetModelsForProvider 

func GetModelsForProvider(clientType ClientType) ([]ModelInfo, error)

GetModelsForProvider returns available models for a specific provider

func GetModelsForProviderCtx 

func GetModelsForProviderCtx(ctx context.Context, clientType ClientType) ([]ModelInfo, error)

GetModelsForProviderCtx returns available models for a specific provider with context support. It checks the model registry first (if enabled), falling back to direct per-provider API calls.

type ModelSelection 

type ModelSelection struct {
	// contains filtered or unexported fields
}

ModelSelection represents a model selection system This is a stub implementation for backward compatibility The actual model selection logic has been moved to configuration-based system

func NewModelSelection 

func NewModelSelection(config interface{}) *ModelSelection

NewModelSelection creates a new ModelSelection instance This is a stub for backward compatibility - the actual model selection is now handled through the configuration system

type ModelsListInterface 

type ModelsListInterface interface {
	ListAvailableModels() ([]ModelInfo, error)
	GetDefaultModel() string
	IsModelAvailable(modelID string) bool
}

ModelsListInterface defines methods for listing available models

type OllamaLocalClient 

type OllamaLocalClient struct {
	*TPSBase
	// contains filtered or unexported fields
}

OllamaLocalClient handles local Ollama API requests

func NewOllamaLocalClient 

func NewOllamaLocalClient(model string) (*OllamaLocalClient, error)

NewOllamaLocalClient creates a new local Ollama client

func (*OllamaLocalClient) CheckConnection 

func (c *OllamaLocalClient) CheckConnection() error

CheckConnection verifies local Ollama is accessible

func (*OllamaLocalClient) GetModel 

func (c *OllamaLocalClient) GetModel() string

GetModel returns the current model

func (*OllamaLocalClient) GetModelContextLimit 

func (c *OllamaLocalClient) GetModelContextLimit() (int, error)

GetModelContextLimit returns the context limit for the model

func (*OllamaLocalClient) GetProvider 

func (c *OllamaLocalClient) GetProvider() string

GetProvider returns the provider name

func (*OllamaLocalClient) GetVisionModel 

func (c *OllamaLocalClient) GetVisionModel() string

GetVisionModel returns empty string as vision is not supported

func (*OllamaLocalClient) ListModels 

func (c *OllamaLocalClient) ListModels(ctx context.Context) ([]ModelInfo, error)

ListModels returns available local models

func (*OllamaLocalClient) SendChatRequest 

func (c *OllamaLocalClient) SendChatRequest(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool) (*ChatResponse, error)

SendChatRequest sends a chat request to local Ollama

func (*OllamaLocalClient) SendChatRequestStream 

func (c *OllamaLocalClient) SendChatRequestStream(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool, callback StreamCallback) (*ChatResponse, error)

SendChatRequestStream streams responses from local Ollama as they arrive

func (*OllamaLocalClient) SendVisionRequest 

func (c *OllamaLocalClient) SendVisionRequest(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool) (*ChatResponse, error)

SendVisionRequest handles vision/OCR requests for Ollama Delegates to SendChatRequest since the image handling is done in buildChatRequest

func (*OllamaLocalClient) SetDebug 

func (c *OllamaLocalClient) SetDebug(debug bool)

SetDebug enables or disables debug mode

func (*OllamaLocalClient) SetModel 

func (c *OllamaLocalClient) SetModel(model string) error

SetModel updates the active model after validating it exists locally

func (*OllamaLocalClient) SupportsVision 

func (c *OllamaLocalClient) SupportsVision() bool

SupportsVision returns true for OCR-capable models

type Provider 

type Provider interface {
	// Core functionality
	SendChatRequest(ctx context.Context, req *ProviderChatRequest) (*ChatResponse, error)
	CheckConnection(ctx context.Context) error

	// Model management
	GetModel() string
	SetModel(model string) error
	GetAvailableModels(ctx context.Context) ([]ModelDetails, error)
	GetModelContextLimit() (int, error)

	// Provider information
	GetName() string
	GetType() ClientType
	GetEndpoint() string

	// Feature support
	SupportsVision() bool
	SupportsTools() bool
	SupportsStreaming() bool
	SupportsReasoning() bool

	// Configuration
	SetDebug(debug bool)
	IsDebug() bool
}

Provider defines the interface all LLM providers must implement

func CreateProviderFromClient 

func CreateProviderFromClient(clientType ClientType, client ClientInterface) Provider

CreateProviderFromClient creates a Provider from an existing ClientInterface

type ProviderAdapter 

type ProviderAdapter struct {
	// contains filtered or unexported fields
}

ProviderAdapter adapts the existing ClientInterface to the new Provider interface

func NewProviderAdapter 

func NewProviderAdapter(clientType ClientType, client ClientInterface) *ProviderAdapter

NewProviderAdapter creates an adapter for existing clients

func (*ProviderAdapter) CheckConnection 

func (a *ProviderAdapter) CheckConnection(ctx context.Context) error

CheckConnection verifies connectivity

func (*ProviderAdapter) GetAvailableModels 

func (a *ProviderAdapter) GetAvailableModels(ctx context.Context) ([]ModelDetails, error)

GetAvailableModels returns available models for this provider

func (*ProviderAdapter) GetEndpoint 

func (a *ProviderAdapter) GetEndpoint() string

GetEndpoint returns the API endpoint

func (*ProviderAdapter) GetModel 

func (a *ProviderAdapter) GetModel() string

GetModel returns the current model

func (*ProviderAdapter) GetModelContextLimit 

func (a *ProviderAdapter) GetModelContextLimit() (int, error)

GetModelContextLimit returns the context window size

func (*ProviderAdapter) GetName 

func (a *ProviderAdapter) GetName() string

GetName returns the provider name

func (*ProviderAdapter) GetType 

func (a *ProviderAdapter) GetType() ClientType

GetType returns the provider type

func (*ProviderAdapter) IsDebug 

func (a *ProviderAdapter) IsDebug() bool

IsDebug returns whether debug mode is enabled

func (*ProviderAdapter) SendChatRequest 

func (a *ProviderAdapter) SendChatRequest(ctx context.Context, req *ProviderChatRequest) (*ChatResponse, error)

SendChatRequest adapts the old interface to the new one.

Note: This uses the same global per-provider rate limiter as APIClient.sendRequest(). Both paths share one bucket per provider to coordinate across all agents, preventing cascading 429s when multiple subagents run concurrently. Do NOT add additional rate limiting at this layer without coordinating with pkg/agent/api_client.go.

func (*ProviderAdapter) SetDebug 

func (a *ProviderAdapter) SetDebug(debug bool)

SetDebug enables or disables debug mode

func (*ProviderAdapter) SetModel 

func (a *ProviderAdapter) SetModel(model string) error

SetModel sets the current model

func (*ProviderAdapter) SupportsReasoning 

func (a *ProviderAdapter) SupportsReasoning() bool

SupportsReasoning returns whether the provider supports reasoning

func (*ProviderAdapter) SupportsStreaming 

func (a *ProviderAdapter) SupportsStreaming() bool

SupportsStreaming returns whether the provider supports streaming

func (*ProviderAdapter) SupportsTools 

func (a *ProviderAdapter) SupportsTools() bool

SupportsTools returns whether the provider supports tools

func (*ProviderAdapter) SupportsVision 

func (a *ProviderAdapter) SupportsVision() bool

SupportsVision returns whether the provider supports vision

type ProviderChatRequest 

type ProviderChatRequest struct {
	Messages []Message
	Tools    []Tool
	Options  *RequestOptions
}

ProviderChatRequest represents a unified request structure for providers This extends the basic ChatRequest with provider-specific options

type ProviderInterface 

type ProviderInterface interface {
	SendChatRequest(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool) (*ChatResponse, error)
	SendChatRequestStream(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool, callback StreamCallback) (*ChatResponse, error)
	CheckConnection() error
	SetDebug(debug bool)
	SetModel(model string) error
	GetModel() string
	GetProvider() string
	GetModelContextLimit() (int, error)
	ListModels(ctx context.Context) ([]ModelInfo, error)
	SupportsVision() bool
	SendVisionRequest(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool) (*ChatResponse, error)
}

ProviderInterface defines the interface that all providers must implement. Mirrors ClientInterface's ctx-bearing send methods — see SP-034.

type RequestOptions 

type RequestOptions struct {
	Temperature      *float64
	MaxTokens        *int
	TopP             *float64
	FrequencyPenalty *float64
	PresencePenalty  *float64
	StopSequences    []string
	Stream           bool
	ReasoningEffort  string // For reasoning models
	DisableThinking  *bool  // Disable thinking/reasoning mode for thinking-capable models
}

RequestOptions contains optional parameters for requests

type SSEReader 

type SSEReader struct {
	// contains filtered or unexported fields
}

SSEReader reads Server-Sent Events from a reader

func NewSSEReader 

func NewSSEReader(r io.Reader, onEvent func(event, data string) error) *SSEReader

NewSSEReader creates a new SSE reader

func (*SSEReader) Read 

func (r *SSEReader) Read() error

Read processes the SSE stream

func (*SSEReader) ReadWithTimeout 

func (r *SSEReader) ReadWithTimeout(timeout time.Duration) error

ReadWithTimeout processes the SSE stream with a timeout. Each line read is performed in a short-lived goroutine so that the select can respond to the timeout without blocking the main loop. The background goroutine will exit when the underlying HTTP response body is closed (which triggers ReadString to return an error).

type StreamCallback 

type StreamCallback func(content string, contentType string)

StreamCallback is called for each content chunk received. contentType is "assistant_text" for regular content or "reasoning" for thinking/reasoning content.

type StreamingChatResponse 

type StreamingChatResponse struct {
	ID      string            `json:"id"`
	Object  string            `json:"object"`
	Created int64             `json:"created"`
	Model   string            `json:"model"`
	Choices []StreamingChoice `json:"choices"`
	Usage   *StreamingUsage   `json:"usage,omitempty"`
}

StreamingChatResponse represents a streaming response chunk

func ParseSSEData 

func ParseSSEData(data string) (*StreamingChatResponse, error)

ParseSSEData parses SSE data into a streaming response

type StreamingChoice 

type StreamingChoice struct {
	Index        int            `json:"index"`
	Delta        StreamingDelta `json:"delta"`
	FinishReason *string        `json:"finish_reason"`
}

StreamingChoice represents a streaming response choice

type StreamingDelta 

type StreamingDelta struct {
	Role             string              `json:"role,omitempty"`
	Content          string              `json:"content,omitempty"`
	Reasoning        string              `json:"reasoning,omitempty"` // GLM reasoning field
	ReasoningContent string              `json:"reasoning_content,omitempty"`
	ReasoningDetails json.RawMessage     `json:"reasoning_details,omitempty"` // Can be string (Minimax) or array (GLM models)
	ToolCalls        []StreamingToolCall `json:"tool_calls,omitempty"`
}

StreamingDelta contains incremental updates

type StreamingResponseBuilder 

type StreamingResponseBuilder struct {
	// contains filtered or unexported fields
}

StreamingResponseBuilder accumulates streaming chunks into a complete response

func NewStreamingResponseBuilder 

func NewStreamingResponseBuilder(callback StreamCallback) *StreamingResponseBuilder

NewStreamingResponseBuilder creates a new streaming response builder

func (*StreamingResponseBuilder) GetResponse 

func (b *StreamingResponseBuilder) GetResponse() *ChatResponse

GetResponse returns the accumulated response

func (*StreamingResponseBuilder) GetTokenGenerationDuration 

func (b *StreamingResponseBuilder) GetTokenGenerationDuration() time.Duration

GetTokenGenerationDuration returns the duration from first token to last token

func (*StreamingResponseBuilder) ProcessChunk 

func (b *StreamingResponseBuilder) ProcessChunk(chunk *StreamingChatResponse) error

ProcessChunk processes a streaming chunk and updates the builder state

type StreamingToolCall 

type StreamingToolCall struct {
	Index    int                        `json:"index"`
	ID       string                     `json:"id,omitempty"`
	Type     string                     `json:"type,omitempty"`
	Function *StreamingToolCallFunction `json:"function,omitempty"`
}

StreamingToolCall represents an incremental tool call update

type StreamingToolCallFunction 

type StreamingToolCallFunction struct {
	Name      string `json:"name,omitempty"`
	Arguments string `json:"arguments,omitempty"`
}

StreamingToolCallFunction contains function details

type StreamingUsage 

type StreamingUsage struct {
	PromptTokens        int     `json:"prompt_tokens"`
	CompletionTokens    int     `json:"completion_tokens"`
	TotalTokens         int     `json:"total_tokens"`
	EstimatedCost       float64 `json:"estimated_cost"`
	Cost                float64 `json:"cost,omitempty"` // OpenRouter returns cost directly
	PromptTokensDetails struct {
		CachedTokens     int  `json:"cached_tokens"`
		CacheWriteTokens *int `json:"cache_write_tokens"`
	} `json:"prompt_tokens_details,omitempty"`
}

StreamingUsage is the usage block on a streaming chunk (providers send it on the final data object). Named (not anonymous) so the flexible cost fallback in ParseSSEData can allocate one when a provider reports cost without the standard token block.

type TPSBase 

type TPSBase struct {
	// contains filtered or unexported fields
}

TPSBase provides a default implementation of TPS tracking methods Other providers can embed this struct to get TPS functionality

func NewTPSBase 

func NewTPSBase() *TPSBase

NewTPSBase creates a new TPS base with tracker

func (*TPSBase) GetAverageTPS 

func (t *TPSBase) GetAverageTPS() float64

GetAverageTPS returns the average TPS across all requests

func (*TPSBase) GetLastTPS 

func (t *TPSBase) GetLastTPS() float64

GetLastTPS returns the most recent TPS measurement

func (*TPSBase) GetTPSStats 

func (t *TPSBase) GetTPSStats() map[string]float64

GetTPSStats returns comprehensive TPS statistics

func (*TPSBase) GetTracker 

func (t *TPSBase) GetTracker() *TPSTracker

GetTracker returns the underlying TPS tracker

func (*TPSBase) ResetTPSStats 

func (t *TPSBase) ResetTPSStats()

ResetTPSStats clears all TPS tracking data

type TPSTracker 

type TPSTracker struct {
	// contains filtered or unexported fields
}

func NewTPSTracker 

func NewTPSTracker() *TPSTracker

NewTPSTracker creates a new TPS tracker

func (*TPSTracker) GetAverageTPS 

func (t *TPSTracker) GetAverageTPS() float64

GetAverageTPS returns the average TPS across all recorded requests

func (*TPSTracker) GetCurrentTPS 

func (t *TPSTracker) GetCurrentTPS() float64

GetCurrentTPS returns the most recent TPS value

func (*TPSTracker) GetSmoothTPS 

func (t *TPSTracker) GetSmoothTPS() float64

GetSmoothTPS returns an exponentially smoothed TPS value

func (*TPSTracker) GetStats 

func (t *TPSTracker) GetStats() map[string]interface{}

GetStats returns comprehensive TPS statistics

func (*TPSTracker) RecordRequest 

func (t *TPSTracker) RecordRequest(duration time.Duration, completionTokens int) float64

RecordRequest records the timing and token usage of an API request

func (*TPSTracker) Reset 

func (t *TPSTracker) Reset()

Reset clears all TPS tracking data

type Tool 

type Tool = core.Tool

type ToolCall 

type ToolCall = core.ToolCall

func RecoverMistralToolCalls 

func RecoverMistralToolCalls(content string) (calls []ToolCall, remaining string, ok bool)

RecoverMistralToolCalls recovers tool calls from the Mistral-family text format, where the model emits `[TOOL_CALLS]…` inside the message content instead of the structured tool_calls field. It returns the recovered calls and the content with the marker and its payload stripped. ok is false when no marker is present or nothing parseable follows it (content is returned unchanged in that case).

Two payload shapes are handled: 

[TOOL_CALLS][{"name":"f","arguments":{…}}, …]   (JSON array — Mistral native)
[TOOL_CALLS]f{…}g{…}                             (name + JSON object, repeated)

type ToolCallFunction 

type ToolCallFunction = core.ToolCallFunction

type ToolFunction 

type ToolFunction = core.ToolFunction

type ToolParameter 

type ToolParameter = core.ToolParameter

type ToolParameters 

type ToolParameters = core.ToolParameters

type UnifiedProviderWrapper 

type UnifiedProviderWrapper struct {
	*TPSBase
	// contains filtered or unexported fields
}

UnifiedProviderWrapper wraps any provider that implements ProviderInterface

func NewUnifiedProviderWrapper 

func NewUnifiedProviderWrapper(provider ProviderInterface) *UnifiedProviderWrapper

NewUnifiedProviderWrapper creates a wrapper for any provider

func (*UnifiedProviderWrapper) CheckConnection 

func (w *UnifiedProviderWrapper) CheckConnection() error

Forward all other methods to the provider

func (*UnifiedProviderWrapper) GetLastRequestTPS 

func (w *UnifiedProviderWrapper) GetLastRequestTPS() float64

GetLastRequestTPS returns the TPS for the last API request

func (*UnifiedProviderWrapper) GetModel 

func (w *UnifiedProviderWrapper) GetModel() string

func (*UnifiedProviderWrapper) GetModelContextLimit 

func (w *UnifiedProviderWrapper) GetModelContextLimit() (int, error)

func (*UnifiedProviderWrapper) GetProvider 

func (w *UnifiedProviderWrapper) GetProvider() string

func (*UnifiedProviderWrapper) GetTPSStatistics 

func (w *UnifiedProviderWrapper) GetTPSStatistics() (float64, float64, int)

GetTPSStatistics returns tokens per second statistics

func (*UnifiedProviderWrapper) GetVisionModel 

func (w *UnifiedProviderWrapper) GetVisionModel() string

func (*UnifiedProviderWrapper) ListModels 

func (w *UnifiedProviderWrapper) ListModels(ctx context.Context) ([]ModelInfo, error)

func (*UnifiedProviderWrapper) ResetTPSStatistics 

func (w *UnifiedProviderWrapper) ResetTPSStatistics()

ResetTPSStatistics resets the TPS tracking

func (*UnifiedProviderWrapper) SendChatRequest 

func (w *UnifiedProviderWrapper) SendChatRequest(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool) (*ChatResponse, error)

SendChatRequest converts types and forwards to provider

func (*UnifiedProviderWrapper) SendChatRequestStream 

func (w *UnifiedProviderWrapper) SendChatRequestStream(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool, callback StreamCallback) (*ChatResponse, error)

SendChatRequestStream sends a streaming chat request (not yet implemented for unified providers)

func (*UnifiedProviderWrapper) SendVisionRequest 

func (w *UnifiedProviderWrapper) SendVisionRequest(ctx context.Context, messages []Message, tools []Tool, reasoning string, disableThinking bool) (*ChatResponse, error)

func (*UnifiedProviderWrapper) SetDebug 

func (w *UnifiedProviderWrapper) SetDebug(debug bool)

func (*UnifiedProviderWrapper) SetModel 

func (w *UnifiedProviderWrapper) SetModel(model string) error

func (*UnifiedProviderWrapper) SupportsVision 

func (w *UnifiedProviderWrapper) SupportsVision() bool

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.