Documentation
¶
Overview ¶
Copyright 2023-2025 Hanzo AI Inc. All Rights Reserved. Portions Copyright 2024 The OpenAgent Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Index ¶
- Constants
- Variables
- func AddPrices(price1 float64, price2 float64) float64
- func CalculateOpenAIModelPrice(model string, modelResult *ModelResult, lang string) error
- func ChatCompletionRequest(model string, messages []openai.ChatCompletionMessage, temperature float32, ...) openai.ChatCompletionRequest
- func CreateVideoDOAI(ctx context.Context, baseURL, apiKey string, req VideoGenRequest) (id, status string, err error)
- func DownloadVideoBytesDOAI(ctx context.Context, baseURL, apiKey, id string) (data []byte, mime string, err error)
- func GetContextLength(typ string) int
- func GetOpenAiClientFromToken(authToken string, providerUrl string) openai.Client
- func GetOpenAiMaxTokens(model string) int
- func GetTokenSize(model string, prompt string) (int, error)
- func InlinesReasoning(upstreamModel string) bool
- func IsVisionModel(subType string) bool
- func NewiFlytekModelProvider(subType string, secretKey string, temperature float32) (*iFlytekModelProvider, error)
- func OpenaiNumTokensFromMessages(messages []openai.ChatCompletionMessage, model string) (int, error)
- func OpenaiRawMessagesToGptVisionMessages(messages []*RawMessage) []openai.ChatCompletionMessage
- func OpenaiRawMessagesToMessages(messages []*RawMessage) []openai.ChatCompletionMessage
- func RefinePrice(price float64) float64
- func RetrieveVideoDOAI(ctx context.Context, baseURL, apiKey, id string) (status, errMsg string, err error)
- func SetContextWindowResolver(f func(model string) int)
- func StripLeadingReasoning(content string) string
- type AgentInfo
- type AgentMessages
- type AlibabacloudModelProvider
- type AmazonBedrockModelProvider
- type AnthropicModelProvider
- type BaichuanModelProvider
- type BaiduCloudModelProvider
- type ChatGLMModelProvider
- type ChatMessage
- type CohereModelProvider
- type DeepSeekProvider
- type DummyModelProvider
- type FireworksModelProvider
- type GeminiModelProvider
- type GeneratedImage
- type GitHubModelProvider
- type GrokModelProvider
- type HuggingFaceModelProvider
- type ImageGenRequest
- type ImageGenResult
- type LocalModelProvider
- type MiniMaxModelProvider
- type MistralModelProvider
- type ModelProvider
- type ModelResult
- type MoonshotModelProvider
- type OpenAiModelProvider
- type OpenRouterModelProvider
- type RawMessage
- type ReasoningStripper
- type SearchResult
- type SiliconFlowProvider
- type Spec
- type StepFunModelProvider
- type TencentCloudClient
- type ThreeDGenRequest
- type ThreeDStatus
- type ToolCall
- type ToolCallResponse
- type Upstream
- type VideoGenRequest
- type VolcengineModelProvider
- type WriterModelProvider
- type YiProvider
Constants ¶
const DefaultContextLength = 131072
DefaultContextLength is the floor used when a model declares no window.
It is deliberately a FLOOR, not a guess. Every model Hanzo serves is at least 128K; a model that wants more declares it. Under-reporting truncates a prompt, over-reporting makes the upstream reject it — so when we do not know, we take the value that is safe on every model in the lineup.
const DryRunPrefix = "$CloudDryRun$"
DryRunPrefix is a special prefix that triggers model providers to estimate token count and price without actually calling the AI model APIs.
const VisionModelForImages = "zen5-omni"
VisionModelForImages is the model to rewrite to when a request carries images and the current model cannot handle them. zen5-omni (kimi-k2.6, 1T multimodal) is the primary; zen3-omni (qwen3.5-397b-a17b) is the cheaper fallback.
Variables ¶
var CohereDefaultTemperature float64 = 0.75
https://docs.cohere.com/docs/command-beta#whats-the-context-window-on-the-command-models
Functions ¶
func CalculateOpenAIModelPrice ¶
func CalculateOpenAIModelPrice(model string, modelResult *ModelResult, lang string) error
func ChatCompletionRequest ¶
func ChatCompletionRequest(model string, messages []openai.ChatCompletionMessage, temperature float32, topP float32, frequencyPenalty float32, presencePenalty float32) openai.ChatCompletionRequest
func CreateVideoDOAI ¶ added in v1.800.0
func CreateVideoDOAI(ctx context.Context, baseURL, apiKey string, req VideoGenRequest) (id, status string, err error)
CreateVideoDOAI POSTs the create request and returns the upstream job id and its initial status ("queued"/"in_progress"). It does NOT wait for completion — that is the caller's job to poll via RetrieveVideoDOAI. baseURL is the provider's ProviderUrl (…/v1); apiKey is the resolved provider key (never logged).
func DownloadVideoBytesDOAI ¶ added in v1.800.0
func DownloadVideoBytesDOAI(ctx context.Context, baseURL, apiKey, id string) (data []byte, mime string, err error)
DownloadVideoBytesDOAI fetches the completed video bytes from /content and returns them raw (NOT base64) with their media type, so the handler can stream them to the client inline. The bytes live ONLY behind this key-authenticated endpoint, so the handler proxies them server-side and never exposes the provider URL/key. A non-video content type (notably the application/json placeholder the async API returns while the job is still in_progress) is rejected rather than handed back as a clip.
func GetContextLength ¶ added in v1.806.6
GetContextLength returns the max context (tokens) for a model.
Configuration is the ONLY source of truth. A model's window is whatever models.yaml declares (following the alias chain: zen5 → its upstream), and nothing else. There is no name-matching heuristic: the previous table dispatched on substrings of the model name ("deepseek", "v3", "32b", …) and every new model silently fell through it — deepseek-v4-pro matched no branch and got 16384, which 402'd every long prompt against a 1M-context model. A table that must be edited for each release is a table that is always one release out of date, so it is gone. Declare the window in models.yaml.
func GetOpenAiMaxTokens ¶
GetOpenAiMaxTokens returns the max tokens for a given OpenAI model.
func InlinesReasoning ¶ added in v1.808.1
InlinesReasoning reports whether an upstream model emits chain-of-thought inline in content (terminated by </think>) rather than in reasoning_content. Gating on it keeps every other upstream's bytes untouched — no buffering, no risk of a false strip.
func IsVisionModel ¶
func NewiFlytekModelProvider ¶
func OpenaiNumTokensFromMessages ¶
func OpenaiNumTokensFromMessages(messages []openai.ChatCompletionMessage, model string) (int, error)
https://github.com/pkoukk/tiktoken-go?tab=readme-ov-file#counting-tokens-for-chat-api-calls https://github.com/hanzoai/go-openai/pull/223#issuecomment-1608689882
func OpenaiRawMessagesToGptVisionMessages ¶
func OpenaiRawMessagesToGptVisionMessages(messages []*RawMessage) []openai.ChatCompletionMessage
func OpenaiRawMessagesToMessages ¶
func OpenaiRawMessagesToMessages(messages []*RawMessage) []openai.ChatCompletionMessage
func RefinePrice ¶
func RetrieveVideoDOAI ¶ added in v1.800.0
func RetrieveVideoDOAI(ctx context.Context, baseURL, apiKey, id string) (status, errMsg string, err error)
RetrieveVideoDOAI performs ONE status poll of an upstream job and returns its current status plus, when the job failed, a short secret-scrubbed reason. It does NOT loop — the handler drives polling from its own client, one HTTP round trip per /v1/videos/{id} request, so the pod never holds a request open across the minutes a generation takes.
func SetContextWindowResolver ¶ added in v1.806.6
SetContextWindowResolver installs the config-backed context-window lookup. Called once from controllers after ModelConfig is initialized.
func StripLeadingReasoning ¶ added in v1.808.1
StripLeadingReasoning removes a leading `<think>…</think>` (or bare `…</think>`, opener consumed by the template) block from a complete content string, returning the clean answer. Content with no </think> is returned unchanged — a no-op on already-clean (e.g. GLM) output.
Types ¶
type AgentInfo ¶
type AgentInfo struct {
AgentClients *agent.AgentClients
AgentMessages *AgentMessages
}
type AgentMessages ¶
type AgentMessages struct {
Messages []*RawMessage
ToolCalls any
}
type AlibabacloudModelProvider ¶
type AlibabacloudModelProvider struct {
// contains filtered or unexported fields
}
func (*AlibabacloudModelProvider) GetPricing ¶
func (p *AlibabacloudModelProvider) GetPricing() string
func (*AlibabacloudModelProvider) QueryText ¶
func (p *AlibabacloudModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)
type AmazonBedrockModelProvider ¶
type AmazonBedrockModelProvider struct {
// contains filtered or unexported fields
}
func NewAmazonBedrockModelProvider ¶
func NewAmazonBedrockModelProvider(subType string, secretKey string, temperature float64) (*AmazonBedrockModelProvider, error)
func (AmazonBedrockModelProvider) GetPricing ¶
func (a AmazonBedrockModelProvider) GetPricing() string
func (*AmazonBedrockModelProvider) QueryText ¶
func (p *AmazonBedrockModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)
type AnthropicModelProvider ¶ added in v1.833.193
type AnthropicModelProvider struct {
// contains filtered or unexported fields
}
func NewAnthropicModelProvider ¶ added in v1.833.193
func (*AnthropicModelProvider) GetPricing ¶ added in v1.833.193
func (p *AnthropicModelProvider) GetPricing() string
func (*AnthropicModelProvider) QueryText ¶ added in v1.833.193
func (p *AnthropicModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)
type BaichuanModelProvider ¶
type BaichuanModelProvider struct {
// contains filtered or unexported fields
}
func (*BaichuanModelProvider) GetPricing ¶
func (p *BaichuanModelProvider) GetPricing() string
func (*BaichuanModelProvider) QueryText ¶
func (p *BaichuanModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)
type BaiduCloudModelProvider ¶
type BaiduCloudModelProvider struct {
// contains filtered or unexported fields
}
func (*BaiduCloudModelProvider) GetPricing ¶
func (p *BaiduCloudModelProvider) GetPricing() string
func (*BaiduCloudModelProvider) QueryText ¶
func (p *BaiduCloudModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)
type ChatGLMModelProvider ¶
type ChatGLMModelProvider struct {
// contains filtered or unexported fields
}
func NewChatGLMModelProvider ¶
func NewChatGLMModelProvider(subType string, clientSecret string) (*ChatGLMModelProvider, error)
func (*ChatGLMModelProvider) GetPricing ¶
func (c *ChatGLMModelProvider) GetPricing() string
func (*ChatGLMModelProvider) QueryText ¶
func (p *ChatGLMModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)
type ChatMessage ¶
type CohereModelProvider ¶
type CohereModelProvider struct {
// contains filtered or unexported fields
}
func NewCohereModelProvider ¶
func NewCohereModelProvider(subType string, secretKey string) (*CohereModelProvider, error)
func (*CohereModelProvider) GetPricing ¶
func (c *CohereModelProvider) GetPricing() string
GetPricing returns the pricing of the model https://cohere.com/pricing
func (*CohereModelProvider) QueryText ¶
func (p *CohereModelProvider) QueryText(message string, writer io.Writer, chat_history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)
type DeepSeekProvider ¶
type DeepSeekProvider struct {
// contains filtered or unexported fields
}
func NewDeepSeekProvider ¶
func (*DeepSeekProvider) GetPricing ¶
func (p *DeepSeekProvider) GetPricing() string
func (*DeepSeekProvider) QueryText ¶
func (p *DeepSeekProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)
type DummyModelProvider ¶
type DummyModelProvider struct {
// contains filtered or unexported fields
}
func NewDummyModelProvider ¶
func NewDummyModelProvider(subType string) (*DummyModelProvider, error)
func (*DummyModelProvider) GetPricing ¶
func (c *DummyModelProvider) GetPricing() string
func (*DummyModelProvider) QueryText ¶
func (p *DummyModelProvider) QueryText(message string, writer io.Writer, chat_history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)
type FireworksModelProvider ¶
type FireworksModelProvider struct {
// contains filtered or unexported fields
}
func NewFireworksProvider ¶
func (*FireworksModelProvider) GetPricing ¶
func (p *FireworksModelProvider) GetPricing() string
func (*FireworksModelProvider) QueryText ¶
func (p *FireworksModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)
type GeminiModelProvider ¶
type GeminiModelProvider struct {
// contains filtered or unexported fields
}
func NewGeminiModelProvider ¶
func (*GeminiModelProvider) GetPricing ¶
func (p *GeminiModelProvider) GetPricing() string
func (*GeminiModelProvider) QueryText ¶
func (p *GeminiModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)
type GeneratedImage ¶ added in v1.790.1
GeneratedImage is one image from a generation call. Exactly one of URL or B64JSON is set, matching what the upstream returned (fal returns a URL).
type GitHubModelProvider ¶
type GitHubModelProvider struct {
*LocalModelProvider
}
func (*GitHubModelProvider) GetPricing ¶
func (p *GitHubModelProvider) GetPricing() string
type GrokModelProvider ¶
type GrokModelProvider struct {
// contains filtered or unexported fields
}
func NewGrokModelProvider ¶
func (*GrokModelProvider) GetPricing ¶
func (p *GrokModelProvider) GetPricing() string
func (*GrokModelProvider) QueryText ¶
func (p *GrokModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)
type HuggingFaceModelProvider ¶
type HuggingFaceModelProvider struct {
// contains filtered or unexported fields
}
func NewHuggingFaceModelProvider ¶
func NewHuggingFaceModelProvider(subType string, secretKey string, temperature float32) (*HuggingFaceModelProvider, error)
func (*HuggingFaceModelProvider) GetPricing ¶
func (p *HuggingFaceModelProvider) GetPricing() string
func (*HuggingFaceModelProvider) QueryText ¶
func (p *HuggingFaceModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)
type ImageGenRequest ¶ added in v1.790.1
type ImageGenRequest struct {
// UpstreamModel is the do-ai/fal model_id (e.g. "fal-ai/flux/schnell").
UpstreamModel string
Prompt string
// N is the number of images (fal "num_images"); clamped to [1,10].
N int
// Size is an OpenAI-style "WxH" (e.g. "1024x1024"); passed through to fal
// as image_size when set. Empty means the model default.
Size string
// OutputFormat is fal's format hint (e.g. "jpeg", "png"); optional.
OutputFormat string
}
ImageGenRequest is the normalized, provider-agnostic image request the model layer accepts. Size/OutputFormat are optional; N defaults to 1.
type ImageGenResult ¶ added in v1.790.1
type ImageGenResult struct {
Images []GeneratedImage
}
ImageGenResult is the structured result of an image generation: the images and the count actually produced (for billing via ModelResult.ImageCount).
func GenerateImageDOAI ¶ added in v1.790.1
func GenerateImageDOAI(ctx context.Context, baseURL, apiKey string, req ImageGenRequest) (*ImageGenResult, error)
GenerateImageDOAI runs a synchronous image generation against DigitalOcean's async fal image API: submit, poll to completion, retrieve. baseURL is the provider's ProviderUrl (…/v1); apiKey is the resolved do-ai key. It never logs the key. Returns the produced images or an error (a failed/timed-out job is an error, never a partial/silent success).
func GenerateImageForModel ¶ added in v1.790.2
func GenerateImageForModel(ctx context.Context, baseURL, apiKey string, req ImageGenRequest) (*ImageGenResult, error)
GenerateImageForModel is the ONE image-generation dispatcher shared by both image callers — the OpenAI-compatible /v1/images/generations handler (controllers/images_api.go) and the chat-stream image branch (QueryText below). It routes to the correct upstream shape by model:
- fal-hosted diffusion (fal-ai/*, zen3-image*) → DigitalOcean's ASYNC invoke API (submit → poll → retrieve) via GenerateImageDOAI.
- everything else (SD 3.5 Large, dall-e, gpt-image) → the SYNCHRONOUS OpenAI /images/generations shape via client.Images.Generate.
upstreamModel is the resolved DO catalog id; baseURL and apiKey are the do-ai provider's ProviderUrl and key (never logged). It returns an OpenAI-shaped result carrying a URL or b64_json per what the upstream produced.
type LocalModelProvider ¶
type LocalModelProvider struct {
// contains filtered or unexported fields
}
func NewAzureModelProvider ¶
func NewLocalModelProvider ¶
func NewLocalModelProvider(typ string, subType string, secretKey string, temperature float32, topP float32, frequencyPenalty float32, presencePenalty float32, providerUrl string, compatibleProvider string, inputPricePerThousandTokens float64, outputPricePerThousandTokens float64, Currency string) (*LocalModelProvider, error)
func (*LocalModelProvider) CalculatePrice ¶
func (p *LocalModelProvider) CalculatePrice(modelResult *ModelResult, lang string) error
func (*LocalModelProvider) GetPricing ¶
func (p *LocalModelProvider) GetPricing() string
func (*LocalModelProvider) QueryText ¶
func (p *LocalModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)
type MiniMaxModelProvider ¶
type MiniMaxModelProvider struct {
// contains filtered or unexported fields
}
func NewMiniMaxModelProvider ¶
func (*MiniMaxModelProvider) GetPricing ¶
func (p *MiniMaxModelProvider) GetPricing() string
func (*MiniMaxModelProvider) QueryText ¶
func (p *MiniMaxModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)
type MistralModelProvider ¶
type MistralModelProvider struct {
// contains filtered or unexported fields
}
func NewMistralProvider ¶
func NewMistralProvider(apiKey, modelName string) (*MistralModelProvider, error)
func (*MistralModelProvider) GetPricing ¶
func (c *MistralModelProvider) GetPricing() string
func (*MistralModelProvider) QueryText ¶
func (c *MistralModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)
type ModelProvider ¶
type ModelProvider interface {
GetPricing() string
QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)
}
func NewGitHubModelProvider ¶
func Open ¶ added in v1.833.195
func Open(s Spec) (ModelProvider, error)
Open reaches the upstream a spec names.
An upstream this build does not speak to answers (nil, nil) — absent, the same answer a row that is not there gives, which the caller reports as "the model provider type: %s is not supported". An error means the upstream is known and could not be reached.
type ModelResult ¶
type ModelResult struct {
PromptTokenCount int
ResponseTokenCount int
TotalTokenCount int
ImageCount int
TotalPrice float64
Currency string
// Unpriced says NO price was stated for this call, so TotalPrice is an absence
// rather than an amount.
//
// A client prices from a table of models it knows, and it cannot know a model
// served from our own hardware or from a vendor's free tier. Reporting nothing
// for those used to mean returning an error, which failed the whole request —
// the call succeeded upstream and the caller got a pricing message instead of an
// answer. Reporting a bare zero would be worse: an empty Currency reads as USD
// downstream, so an unknown model would bill as costing nothing rather than as
// unknown.
//
// Stated as the NEGATIVE deliberately, unlike a route's Priced. The safe zero
// value differs because the risk does: a route that defaulted to priced-at-zero
// would give inference away, while a result that defaulted to unpriced would
// stop billing every client that never sets it. Zero here means priced, which is
// what every existing client already means. The usage warehouse has carried this
// same word all along and can now be told the truth.
Unpriced bool
// CacheReadTokenCount and CacheWriteTokenCount are the prompt tokens a
// provider served from, and wrote to, its prompt cache.
//
// They are part of PromptTokenCount, not additional to it — a provider
// reports the cached portion of the prompt it already charged for, at a
// different rate (a read is typically a tenth of the input price, a write
// a quarter more). So they price the SAME tokens differently; adding them
// to a total would count those tokens twice.
//
// Everything downstream of here was already built to carry them — the
// usage record, the ClickHouse columns, the per-rate pricing, the gen_ai
// span attributes. This struct was the one place they had nowhere to sit,
// so every request reported zero reads and zero writes however well the
// provider cached, and the saving was invisible to billing and to o11y
// alike. Reading them is the whole fix.
CacheReadTokenCount int
CacheWriteTokenCount int
}
func QueryTextWithTools ¶
func QueryTextWithTools(p ModelProvider, question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)
type MoonshotModelProvider ¶
type MoonshotModelProvider struct {
// contains filtered or unexported fields
}
func (*MoonshotModelProvider) GetPricing ¶
func (p *MoonshotModelProvider) GetPricing() string
func (*MoonshotModelProvider) QueryText ¶
func (p *MoonshotModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)
type OpenAiModelProvider ¶
type OpenAiModelProvider struct {
// contains filtered or unexported fields
}
func NewOpenAiModelProvider ¶
func (*OpenAiModelProvider) GetPricing ¶
func (p *OpenAiModelProvider) GetPricing() string
func (*OpenAiModelProvider) QueryText ¶
func (p *OpenAiModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)
type OpenRouterModelProvider ¶
type OpenRouterModelProvider struct {
// contains filtered or unexported fields
}
func (*OpenRouterModelProvider) GetPricing ¶
func (p *OpenRouterModelProvider) GetPricing() string
func (*OpenRouterModelProvider) QueryText ¶
func (p *OpenRouterModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)
type RawMessage ¶
type RawMessage struct {
Text string
Author string
TextTokenCount int
ToolCall openai.ToolCall
ToolCallID string
}
func OpenaiGenerateMessages ¶
func OpenaiGenerateMessages(prompt string, question string, recentMessages []*RawMessage, knowledgeMessages []*RawMessage, model string, maxTokens int, lang string) ([]*RawMessage, error)
type ReasoningStripper ¶ added in v1.808.1
type ReasoningStripper struct {
// contains filtered or unexported fields
}
ReasoningStripper strips the leading inline-reasoning block from a STREAMED content sequence. It holds content deltas until </think> arrives, then emits the answer remainder and passes every later delta straight through. Feed it only content deltas from a reasoning-inlining upstream (see InlinesReasoning); the terminator may split across deltas, which the running buffer handles.
func (*ReasoningStripper) Feed ¶ added in v1.808.1
func (s *ReasoningStripper) Feed(delta string) string
Feed returns the content to forward for one upstream content delta: "" while still inside the reasoning block, the answer text once </think> is seen (or the held buffer if the cap is hit without a terminator), then passthrough.
type SearchResult ¶
type SiliconFlowProvider ¶
type SiliconFlowProvider struct {
// contains filtered or unexported fields
}
func NewSiliconFlowProvider ¶
func (*SiliconFlowProvider) GetPricing ¶
func (p *SiliconFlowProvider) GetPricing() string
func (*SiliconFlowProvider) QueryText ¶
func (p *SiliconFlowProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)
type Spec ¶ added in v1.833.195
type Spec struct {
Upstream Upstream
Model string // the vendor's own name for it, e.g. claude-opus-4-5
ClientID string
Secret string
UserKey string
URL string
APIVersion string
Compatible string // the API this upstream speaks when it is not its own
Temperature float32
TopP float32
TopK int
Frequency float32
Presence float32
Thinking bool
InputPrice float64 // per thousand tokens
OutputPrice float64 // per thousand tokens
Currency string
}
Spec is what a provider row says about reaching its upstream: which upstream, which model, the credentials, and the dials.
type StepFunModelProvider ¶
type StepFunModelProvider struct {
// contains filtered or unexported fields
}
func NewStepFunModelProvider ¶
func (*StepFunModelProvider) GetPricing ¶
func (p *StepFunModelProvider) GetPricing() string
func (*StepFunModelProvider) QueryText ¶
func (p *StepFunModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)
type TencentCloudClient ¶
type TencentCloudClient struct {
// contains filtered or unexported fields
}
func NewTencentCloudProvider ¶
func NewTencentCloudProvider(secretKey, endpoint, subType string, temperature, topP float32) (*TencentCloudClient, error)
func (*TencentCloudClient) GetPricing ¶
func (c *TencentCloudClient) GetPricing() string
func (*TencentCloudClient) QueryText ¶
func (c *TencentCloudClient) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)
type ThreeDGenRequest ¶ added in v1.833.218
type ThreeDGenRequest struct {
UpstreamModel string `json:"model"`
Prompt string `json:"prompt"`
ImageUrl string `json:"image_url,omitempty"`
Format string `json:"format,omitempty"` // "splat" | "ply" | "glb" | "obj"
Quality string `json:"quality,omitempty"`
}
ThreeDGenRequest is the normalized create request for 3D Generation & Gaussian Splats.
type ThreeDStatus ¶ added in v1.833.218
type ThreeDStatus struct {
Id string `json:"id"`
Model string `json:"model"`
Status string `json:"status"` // "queued" | "in_progress" | "completed" | "failed"
Url string `json:"url,omitempty"`
Format string `json:"format,omitempty"`
SplatCount int `json:"splat_count,omitempty"`
Vertices int `json:"vertices,omitempty"`
Error string `json:"error,omitempty"`
}
ThreeDStatus is the status returned by the 3D generation job tracker.
func CreateThreeDJob ¶ added in v1.833.218
func CreateThreeDJob(ctx context.Context, providerUrl, apiKey string, req ThreeDGenRequest) (*ThreeDStatus, error)
CreateThreeDJob posts a 3D generation task to the upstream 3D engine.
type ToolCall ¶
type ToolCall struct {
Name string `json:"name"`
Arguments string `json:"arguments"`
Content string `json:"content"`
}
func GetToolCallsFromWriter ¶
type ToolCallResponse ¶
type Upstream ¶ added in v1.833.198
type Upstream string
Upstream names what serves a call, never what is served: Anthropic serves Claude, OpenAI serves GPT. Some are third-party APIs, some are our own compute — Hanzo and Zen run on our GPUs — and some are a runner a deployment starts itself, like Ollama or Local. All of them answer the same question, which is the code that carries a request out.
The set is closed and each member has one spelling, held by the compiler.
const ( AlibabaCloud Upstream = "Alibaba Cloud" AmazonBedrock Upstream = "Amazon Bedrock" Anthropic Upstream = "Anthropic" Azure Upstream = "Azure" Baichuan Upstream = "Baichuan" BaiduCloud Upstream = "Baidu Cloud" ChatGLM Upstream = "ChatGLM" Cohere Upstream = "Cohere" DeepSeek Upstream = "DeepSeek" DigitalOcean Upstream = "DigitalOcean" Dummy Upstream = "Dummy" Enso Upstream = "Enso" Fireworks Upstream = "Fireworks" Gemini Upstream = "Gemini" GitHub Upstream = "GitHub" Grok Upstream = "Grok" Jina Upstream = "Jina" Hanzo Upstream = "Hanzo" HuggingFace Upstream = "Hugging Face" IFlytek Upstream = "iFlytek" Local Upstream = "Local" MiniMax Upstream = "MiniMax" Mistral Upstream = "Mistral" Moonshot Upstream = "Moonshot" Ollama Upstream = "Ollama" OpenAI Upstream = "OpenAI" OpenRouter Upstream = "OpenRouter" SiliconFlow Upstream = "Silicon Flow" StepFun Upstream = "StepFun" TencentCloud Upstream = "Tencent Cloud" VolcanoEngine Upstream = "Volcano Engine" Word2Vec Upstream = "Word2Vec" Writer Upstream = "Writer" Yi Upstream = "Yi" Zen Upstream = "Zen" )
type VideoGenRequest ¶ added in v1.790.4
type VideoGenRequest struct {
// UpstreamModel is the upstream model id (e.g. "wan2-2-t2v-a14b").
UpstreamModel string
Prompt string
// Size is an optional OpenAI-style "WxH" hint passed through when set.
Size string
// Seconds is an optional clip-length hint (upstream default when 0).
Seconds int
}
VideoGenRequest is the normalized, provider-agnostic create request the model layer accepts. The async /videos API produces exactly one video per create.
type VolcengineModelProvider ¶
type VolcengineModelProvider struct {
// contains filtered or unexported fields
}
func (*VolcengineModelProvider) GetPricing ¶
func (p *VolcengineModelProvider) GetPricing() string
func (*VolcengineModelProvider) QueryText ¶
func (p *VolcengineModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)
type WriterModelProvider ¶
type WriterModelProvider struct {
// contains filtered or unexported fields
}
func NewWriterModelProvider ¶
func (*WriterModelProvider) GetPricing ¶
func (p *WriterModelProvider) GetPricing() string
func (*WriterModelProvider) QueryText ¶
func (p *WriterModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)
type YiProvider ¶
type YiProvider struct {
// contains filtered or unexported fields
}
func NewYiProvider ¶
func (*YiProvider) GetPricing ¶
func (p *YiProvider) GetPricing() string
func (*YiProvider) QueryText ¶
func (p *YiProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)
Source Files
¶
- alibabacloud.go
- amazon_bedrock.go
- anthropic.go
- azure_openai.go
- baichuan.go
- baiducloud.go
- chatglm.go
- cohere.go
- context_length_util.go
- deepseek.go
- doai_3d.go
- doai_image.go
- doai_video.go
- dummy.go
- fireworks.go
- gemini.go
- github.go
- grok.go
- huggingface.go
- iflytek.go
- local.go
- local_gptvision.go
- mcp.go
- minimax.go
- mistral.go
- moonshot.go
- openai.go
- openai_util.go
- openrouter.go
- provider.go
- reasoning.go
- silicon_flow.go
- stepfun.go
- tencentcloud.go
- util.go
- volcengine.go
- writer.go
- yi.go