model

package
v1.833.292 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 3, 2026 License: Apache-2.0 Imports: 45 Imported by: 0

Documentation

Overview

Copyright 2023-2025 Hanzo AI Inc. All Rights Reserved. Portions Copyright 2024 The OpenAgent Authors. All Rights Reserved.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Index

Constants

View Source
const DefaultContextLength = 131072

DefaultContextLength is the floor used when a model declares no window.

It is deliberately a FLOOR, not a guess. Every model Hanzo serves is at least 128K; a model that wants more declares it. Under-reporting truncates a prompt, over-reporting makes the upstream reject it — so when we do not know, we take the value that is safe on every model in the lineup.

View Source
const DryRunPrefix = "$CloudDryRun$"

DryRunPrefix is a special prefix that triggers model providers to estimate token count and price without actually calling the AI model APIs.

View Source
const VisionModelForImages = "zen5-omni"

VisionModelForImages is the model to rewrite to when a request carries images and the current model cannot handle them. zen5-omni (kimi-k2.6, 1T multimodal) is the primary; zen3-omni (qwen3.5-397b-a17b) is the cheaper fallback.

Variables

View Source
var CohereDefaultTemperature float64 = 0.75

https://docs.cohere.com/docs/command-beta#whats-the-context-window-on-the-command-models

Functions

func AddPrices

func AddPrices(price1 float64, price2 float64) float64

func CalculateOpenAIModelPrice

func CalculateOpenAIModelPrice(model string, modelResult *ModelResult, lang string) error

func ChatCompletionRequest

func ChatCompletionRequest(model string, messages []openai.ChatCompletionMessage, temperature float32, topP float32, frequencyPenalty float32, presencePenalty float32) openai.ChatCompletionRequest

func CreateVideoDOAI added in v1.800.0

func CreateVideoDOAI(ctx context.Context, baseURL, apiKey string, req VideoGenRequest) (id, status string, err error)

CreateVideoDOAI POSTs the create request and returns the upstream job id and its initial status ("queued"/"in_progress"). It does NOT wait for completion — that is the caller's job to poll via RetrieveVideoDOAI. baseURL is the provider's ProviderUrl (…/v1); apiKey is the resolved provider key (never logged).

func DownloadVideoBytesDOAI added in v1.800.0

func DownloadVideoBytesDOAI(ctx context.Context, baseURL, apiKey, id string) (data []byte, mime string, err error)

DownloadVideoBytesDOAI fetches the completed video bytes from /content and returns them raw (NOT base64) with their media type, so the handler can stream them to the client inline. The bytes live ONLY behind this key-authenticated endpoint, so the handler proxies them server-side and never exposes the provider URL/key. A non-video content type (notably the application/json placeholder the async API returns while the job is still in_progress) is rejected rather than handed back as a clip.

func GetContextLength added in v1.806.6

func GetContextLength(typ string) int

GetContextLength returns the max context (tokens) for a model.

Configuration is the ONLY source of truth. A model's window is whatever models.yaml declares (following the alias chain: zen5 → its upstream), and nothing else. There is no name-matching heuristic: the previous table dispatched on substrings of the model name ("deepseek", "v3", "32b", …) and every new model silently fell through it — deepseek-v4-pro matched no branch and got 16384, which 402'd every long prompt against a 1M-context model. A table that must be edited for each release is a table that is always one release out of date, so it is gone. Declare the window in models.yaml.

func GetOpenAiClientFromToken

func GetOpenAiClientFromToken(authToken string, providerUrl string) openai.Client

func GetOpenAiMaxTokens

func GetOpenAiMaxTokens(model string) int

GetOpenAiMaxTokens returns the max tokens for a given OpenAI model.

func GetTokenSize

func GetTokenSize(model string, prompt string) (int, error)

func InlinesReasoning added in v1.808.1

func InlinesReasoning(upstreamModel string) bool

InlinesReasoning reports whether an upstream model emits chain-of-thought inline in content (terminated by </think>) rather than in reasoning_content. Gating on it keeps every other upstream's bytes untouched — no buffering, no risk of a false strip.

func IsVisionModel

func IsVisionModel(subType string) bool

func NewiFlytekModelProvider

func NewiFlytekModelProvider(subType string, secretKey string, temperature float32) (*iFlytekModelProvider, error)

func OpenaiRawMessagesToGptVisionMessages

func OpenaiRawMessagesToGptVisionMessages(messages []*RawMessage) []openai.ChatCompletionMessage

func OpenaiRawMessagesToMessages

func OpenaiRawMessagesToMessages(messages []*RawMessage) []openai.ChatCompletionMessage

func RefinePrice

func RefinePrice(price float64) float64

func RetrieveVideoDOAI added in v1.800.0

func RetrieveVideoDOAI(ctx context.Context, baseURL, apiKey, id string) (status, errMsg string, err error)

RetrieveVideoDOAI performs ONE status poll of an upstream job and returns its current status plus, when the job failed, a short secret-scrubbed reason. It does NOT loop — the handler drives polling from its own client, one HTTP round trip per /v1/videos/{id} request, so the pod never holds a request open across the minutes a generation takes.

func SetContextWindowResolver added in v1.806.6

func SetContextWindowResolver(f func(model string) int)

SetContextWindowResolver installs the config-backed context-window lookup. Called once from controllers after ModelConfig is initialized.

func StripLeadingReasoning added in v1.808.1

func StripLeadingReasoning(content string) string

StripLeadingReasoning removes a leading `<think>…</think>` (or bare `…</think>`, opener consumed by the template) block from a complete content string, returning the clean answer. Content with no </think> is returned unchanged — a no-op on already-clean (e.g. GLM) output.

Types

type AgentInfo

type AgentInfo struct {
	AgentClients  *agent.AgentClients
	AgentMessages *AgentMessages
}

type AgentMessages

type AgentMessages struct {
	Messages  []*RawMessage
	ToolCalls any
}

type AlibabacloudModelProvider

type AlibabacloudModelProvider struct {
	// contains filtered or unexported fields
}

func NewAlibabacloudModelProvider

func NewAlibabacloudModelProvider(subType string, apiKey string, temperature float32, topP float32) (*AlibabacloudModelProvider, error)

func (*AlibabacloudModelProvider) GetPricing

func (p *AlibabacloudModelProvider) GetPricing() string

func (*AlibabacloudModelProvider) QueryText

func (p *AlibabacloudModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)

type AmazonBedrockModelProvider

type AmazonBedrockModelProvider struct {
	// contains filtered or unexported fields
}

func NewAmazonBedrockModelProvider

func NewAmazonBedrockModelProvider(subType string, secretKey string, temperature float64) (*AmazonBedrockModelProvider, error)

func (AmazonBedrockModelProvider) GetPricing

func (a AmazonBedrockModelProvider) GetPricing() string

func (*AmazonBedrockModelProvider) QueryText

func (p *AmazonBedrockModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)

type AnthropicModelProvider added in v1.833.193

type AnthropicModelProvider struct {
	// contains filtered or unexported fields
}

func NewAnthropicModelProvider added in v1.833.193

func NewAnthropicModelProvider(subType string, secretKey string, enableThinking bool, budgetTokens int) (*AnthropicModelProvider, error)

func (*AnthropicModelProvider) GetPricing added in v1.833.193

func (p *AnthropicModelProvider) GetPricing() string

func (*AnthropicModelProvider) QueryText added in v1.833.193

func (p *AnthropicModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)

type BaichuanModelProvider

type BaichuanModelProvider struct {
	// contains filtered or unexported fields
}

func NewBaichuanModelProvider

func NewBaichuanModelProvider(subType string, apiKey string, temperature float32, topP float32) (*BaichuanModelProvider, error)

func (*BaichuanModelProvider) GetPricing

func (p *BaichuanModelProvider) GetPricing() string

func (*BaichuanModelProvider) QueryText

func (p *BaichuanModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)

type BaiduCloudModelProvider

type BaiduCloudModelProvider struct {
	// contains filtered or unexported fields
}

func NewBaiduCloudModelProvider

func NewBaiduCloudModelProvider(subType string, apiKey string, temperature float32, topP float32) (*BaiduCloudModelProvider, error)

func (*BaiduCloudModelProvider) GetPricing

func (p *BaiduCloudModelProvider) GetPricing() string

func (*BaiduCloudModelProvider) QueryText

func (p *BaiduCloudModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)

type ChatGLMModelProvider

type ChatGLMModelProvider struct {
	// contains filtered or unexported fields
}

func NewChatGLMModelProvider

func NewChatGLMModelProvider(subType string, clientSecret string) (*ChatGLMModelProvider, error)

func (*ChatGLMModelProvider) GetPricing

func (c *ChatGLMModelProvider) GetPricing() string

func (*ChatGLMModelProvider) QueryText

func (p *ChatGLMModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)

type ChatMessage

type ChatMessage struct {
	Role    string  `json:"role"`
	Message string  `json:"message"`
	User    *string `json:"user,omitempty"`
}

type CohereModelProvider

type CohereModelProvider struct {
	// contains filtered or unexported fields
}

func NewCohereModelProvider

func NewCohereModelProvider(subType string, secretKey string) (*CohereModelProvider, error)

func (*CohereModelProvider) GetPricing

func (c *CohereModelProvider) GetPricing() string

GetPricing returns the pricing of the model https://cohere.com/pricing

func (*CohereModelProvider) QueryText

func (p *CohereModelProvider) QueryText(message string, writer io.Writer, chat_history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)

type DeepSeekProvider

type DeepSeekProvider struct {
	// contains filtered or unexported fields
}

func NewDeepSeekProvider

func NewDeepSeekProvider(subType string, apiKey string, temperature float32, topP float32) (*DeepSeekProvider, error)

func (*DeepSeekProvider) GetPricing

func (p *DeepSeekProvider) GetPricing() string

func (*DeepSeekProvider) QueryText

func (p *DeepSeekProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)

type DummyModelProvider

type DummyModelProvider struct {
	// contains filtered or unexported fields
}

func NewDummyModelProvider

func NewDummyModelProvider(subType string) (*DummyModelProvider, error)

func (*DummyModelProvider) GetPricing

func (c *DummyModelProvider) GetPricing() string

func (*DummyModelProvider) QueryText

func (p *DummyModelProvider) QueryText(message string, writer io.Writer, chat_history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)

type FireworksModelProvider

type FireworksModelProvider struct {
	// contains filtered or unexported fields
}

func NewFireworksProvider

func NewFireworksProvider(subType string, apiKey string, temperature float32, topP float32, frequencyPenalty float32, presencePenalty float32) (*FireworksModelProvider, error)

func (*FireworksModelProvider) GetPricing

func (p *FireworksModelProvider) GetPricing() string

func (*FireworksModelProvider) QueryText

func (p *FireworksModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)

type GeminiModelProvider

type GeminiModelProvider struct {
	// contains filtered or unexported fields
}

func NewGeminiModelProvider

func NewGeminiModelProvider(subType string, secretKey string, temperature float32, topP float32, topK int) (*GeminiModelProvider, error)

func (*GeminiModelProvider) GetPricing

func (p *GeminiModelProvider) GetPricing() string

func (*GeminiModelProvider) QueryText

func (p *GeminiModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)

type GeneratedImage added in v1.790.1

type GeneratedImage struct {
	URL     string
	B64JSON string
}

GeneratedImage is one image from a generation call. Exactly one of URL or B64JSON is set, matching what the upstream returned (fal returns a URL).

type GitHubModelProvider

type GitHubModelProvider struct {
	*LocalModelProvider
}

func (*GitHubModelProvider) GetPricing

func (p *GitHubModelProvider) GetPricing() string

type GrokModelProvider

type GrokModelProvider struct {
	// contains filtered or unexported fields
}

func NewGrokModelProvider

func NewGrokModelProvider(subType string, secretKey string, temperature float32, topP float32) (*GrokModelProvider, error)

func (*GrokModelProvider) GetPricing

func (p *GrokModelProvider) GetPricing() string

func (*GrokModelProvider) QueryText

func (p *GrokModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)

type HuggingFaceModelProvider

type HuggingFaceModelProvider struct {
	// contains filtered or unexported fields
}

func NewHuggingFaceModelProvider

func NewHuggingFaceModelProvider(subType string, secretKey string, temperature float32) (*HuggingFaceModelProvider, error)

func (*HuggingFaceModelProvider) GetPricing

func (p *HuggingFaceModelProvider) GetPricing() string

func (*HuggingFaceModelProvider) QueryText

func (p *HuggingFaceModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)

type ImageGenRequest added in v1.790.1

type ImageGenRequest struct {
	// UpstreamModel is the do-ai/fal model_id (e.g. "fal-ai/flux/schnell").
	UpstreamModel string
	Prompt        string
	// N is the number of images (fal "num_images"); clamped to [1,10].
	N int
	// Size is an OpenAI-style "WxH" (e.g. "1024x1024"); passed through to fal
	// as image_size when set. Empty means the model default.
	Size string
	// OutputFormat is fal's format hint (e.g. "jpeg", "png"); optional.
	OutputFormat string
}

ImageGenRequest is the normalized, provider-agnostic image request the model layer accepts. Size/OutputFormat are optional; N defaults to 1.

type ImageGenResult added in v1.790.1

type ImageGenResult struct {
	Images []GeneratedImage
}

ImageGenResult is the structured result of an image generation: the images and the count actually produced (for billing via ModelResult.ImageCount).

func GenerateImageDOAI added in v1.790.1

func GenerateImageDOAI(ctx context.Context, baseURL, apiKey string, req ImageGenRequest) (*ImageGenResult, error)

GenerateImageDOAI runs a synchronous image generation against DigitalOcean's async fal image API: submit, poll to completion, retrieve. baseURL is the provider's ProviderUrl (…/v1); apiKey is the resolved do-ai key. It never logs the key. Returns the produced images or an error (a failed/timed-out job is an error, never a partial/silent success).

func GenerateImageForModel added in v1.790.2

func GenerateImageForModel(ctx context.Context, baseURL, apiKey string, req ImageGenRequest) (*ImageGenResult, error)

GenerateImageForModel is the ONE image-generation dispatcher shared by both image callers — the OpenAI-compatible /v1/images/generations handler (controllers/images_api.go) and the chat-stream image branch (QueryText below). It routes to the correct upstream shape by model:

  • fal-hosted diffusion (fal-ai/*, zen3-image*) → DigitalOcean's ASYNC invoke API (submit → poll → retrieve) via GenerateImageDOAI.
  • everything else (SD 3.5 Large, dall-e, gpt-image) → the SYNCHRONOUS OpenAI /images/generations shape via client.Images.Generate.

upstreamModel is the resolved DO catalog id; baseURL and apiKey are the do-ai provider's ProviderUrl and key (never logged). It returns an OpenAI-shaped result carrying a URL or b64_json per what the upstream produced.

type LocalModelProvider

type LocalModelProvider struct {
	// contains filtered or unexported fields
}

func NewAzureModelProvider

func NewAzureModelProvider(typ string, subType string, deploymentName string, secretKey string, temperature float32, topP float32, frequencyPenalty float32, presencePenalty float32, providerUrl string, apiVersion string) (*LocalModelProvider, error)

func NewLocalModelProvider

func NewLocalModelProvider(typ string, subType string, secretKey string, temperature float32, topP float32, frequencyPenalty float32, presencePenalty float32, providerUrl string, compatibleProvider string, inputPricePerThousandTokens float64, outputPricePerThousandTokens float64, Currency string) (*LocalModelProvider, error)

func (*LocalModelProvider) CalculatePrice

func (p *LocalModelProvider) CalculatePrice(modelResult *ModelResult, lang string) error

func (*LocalModelProvider) GetPricing

func (p *LocalModelProvider) GetPricing() string

func (*LocalModelProvider) QueryText

func (p *LocalModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)

type MiniMaxModelProvider

type MiniMaxModelProvider struct {
	// contains filtered or unexported fields
}

func NewMiniMaxModelProvider

func NewMiniMaxModelProvider(subType string, groupID string, apiKey string, temperature float32) (*MiniMaxModelProvider, error)

func (*MiniMaxModelProvider) GetPricing

func (p *MiniMaxModelProvider) GetPricing() string

func (*MiniMaxModelProvider) QueryText

func (p *MiniMaxModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)

type MistralModelProvider

type MistralModelProvider struct {
	// contains filtered or unexported fields
}

func NewMistralProvider

func NewMistralProvider(apiKey, modelName string) (*MistralModelProvider, error)

func (*MistralModelProvider) GetPricing

func (c *MistralModelProvider) GetPricing() string

func (*MistralModelProvider) QueryText

func (c *MistralModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)

type ModelProvider

type ModelProvider interface {
	GetPricing() string
	QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)
}

func NewGitHubModelProvider

func NewGitHubModelProvider(typ string, subType string, secretKey string, temperature float32, topP float32, frequencyPenalty float32, presencePenalty float32) (ModelProvider, error)

func Open added in v1.833.195

func Open(s Spec) (ModelProvider, error)

Open reaches the upstream a spec names.

An upstream this build does not speak to answers (nil, nil) — absent, the same answer a row that is not there gives, which the caller reports as "the model provider type: %s is not supported". An error means the upstream is known and could not be reached.

type ModelResult

type ModelResult struct {
	PromptTokenCount   int
	ResponseTokenCount int
	TotalTokenCount    int
	ImageCount         int
	TotalPrice         float64
	Currency           string

	// Unpriced says NO price was stated for this call, so TotalPrice is an absence
	// rather than an amount.
	//
	// A client prices from a table of models it knows, and it cannot know a model
	// served from our own hardware or from a vendor's free tier. Reporting nothing
	// for those used to mean returning an error, which failed the whole request —
	// the call succeeded upstream and the caller got a pricing message instead of an
	// answer. Reporting a bare zero would be worse: an empty Currency reads as USD
	// downstream, so an unknown model would bill as costing nothing rather than as
	// unknown.
	//
	// Stated as the NEGATIVE deliberately, unlike a route's Priced. The safe zero
	// value differs because the risk does: a route that defaulted to priced-at-zero
	// would give inference away, while a result that defaulted to unpriced would
	// stop billing every client that never sets it. Zero here means priced, which is
	// what every existing client already means. The usage warehouse has carried this
	// same word all along and can now be told the truth.
	Unpriced bool

	// CacheReadTokenCount and CacheWriteTokenCount are the prompt tokens a
	// provider served from, and wrote to, its prompt cache.
	//
	// They are part of PromptTokenCount, not additional to it — a provider
	// reports the cached portion of the prompt it already charged for, at a
	// different rate (a read is typically a tenth of the input price, a write
	// a quarter more). So they price the SAME tokens differently; adding them
	// to a total would count those tokens twice.
	//
	// Everything downstream of here was already built to carry them — the
	// usage record, the ClickHouse columns, the per-rate pricing, the gen_ai
	// span attributes. This struct was the one place they had nowhere to sit,
	// so every request reported zero reads and zero writes however well the
	// provider cached, and the saving was invisible to billing and to o11y
	// alike. Reading them is the whole fix.
	CacheReadTokenCount  int
	CacheWriteTokenCount int
}

func QueryTextWithTools

func QueryTextWithTools(p ModelProvider, question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)

type MoonshotModelProvider

type MoonshotModelProvider struct {
	// contains filtered or unexported fields
}

func NewMoonshotModelProvider

func NewMoonshotModelProvider(subType string, secretKey string, temperature float32, topP float32) (*MoonshotModelProvider, error)

func (*MoonshotModelProvider) GetPricing

func (p *MoonshotModelProvider) GetPricing() string

func (*MoonshotModelProvider) QueryText

func (p *MoonshotModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)

type OpenAiModelProvider

type OpenAiModelProvider struct {
	// contains filtered or unexported fields
}

func NewOpenAiModelProvider

func NewOpenAiModelProvider(subType string, secretKey string, providerUrl string, temperature float32, topP float32, frequencyPenalty float32, presencePenalty float32) (*OpenAiModelProvider, error)

func (*OpenAiModelProvider) GetPricing

func (p *OpenAiModelProvider) GetPricing() string

func (*OpenAiModelProvider) QueryText

func (p *OpenAiModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)

type OpenRouterModelProvider

type OpenRouterModelProvider struct {
	// contains filtered or unexported fields
}

func NewOpenRouterModelProvider

func NewOpenRouterModelProvider(subType string, secretKey string, temperature float32, topP float32) (*OpenRouterModelProvider, error)

func (*OpenRouterModelProvider) GetPricing

func (p *OpenRouterModelProvider) GetPricing() string

func (*OpenRouterModelProvider) QueryText

func (p *OpenRouterModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)

type RawMessage

type RawMessage struct {
	Text           string
	Author         string
	TextTokenCount int
	ToolCall       openai.ToolCall
	ToolCallID     string
}

func OpenaiGenerateMessages

func OpenaiGenerateMessages(prompt string, question string, recentMessages []*RawMessage, knowledgeMessages []*RawMessage, model string, maxTokens int, lang string) ([]*RawMessage, error)

type ReasoningStripper added in v1.808.1

type ReasoningStripper struct {
	// contains filtered or unexported fields
}

ReasoningStripper strips the leading inline-reasoning block from a STREAMED content sequence. It holds content deltas until </think> arrives, then emits the answer remainder and passes every later delta straight through. Feed it only content deltas from a reasoning-inlining upstream (see InlinesReasoning); the terminator may split across deltas, which the running buffer handles.

func (*ReasoningStripper) Feed added in v1.808.1

func (s *ReasoningStripper) Feed(delta string) string

Feed returns the content to forward for one upstream content delta: "" while still inside the reasoning block, the answer text once </think> is seen (or the held buffer if the cap is hit without a terminator), then passthrough.

type SearchResult

type SearchResult struct {
	Icon     string `json:"icon,omitempty"`
	SiteName string `json:"site_name,omitempty"`
	Index    int    `json:"index"`
	Title    string `json:"title"`
	URL      string `json:"url"`
}

type SiliconFlowProvider

type SiliconFlowProvider struct {
	// contains filtered or unexported fields
}

func NewSiliconFlowProvider

func NewSiliconFlowProvider(subType string, apiKey string, temperature float32, topP float32) (*SiliconFlowProvider, error)

func (*SiliconFlowProvider) GetPricing

func (p *SiliconFlowProvider) GetPricing() string

func (*SiliconFlowProvider) QueryText

func (p *SiliconFlowProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)

type Spec added in v1.833.195

type Spec struct {
	Upstream Upstream
	Model    string // the vendor's own name for it, e.g. claude-opus-4-5

	ClientID string
	Secret   string
	UserKey  string

	URL        string
	APIVersion string
	Compatible string // the API this upstream speaks when it is not its own

	Temperature float32
	TopP        float32
	TopK        int
	Frequency   float32
	Presence    float32
	Thinking    bool

	InputPrice  float64 // per thousand tokens
	OutputPrice float64 // per thousand tokens
	Currency    string
}

Spec is what a provider row says about reaching its upstream: which upstream, which model, the credentials, and the dials.

type StepFunModelProvider

type StepFunModelProvider struct {
	// contains filtered or unexported fields
}

func NewStepFunModelProvider

func NewStepFunModelProvider(subType string, apiKey string, temperature float32, topP float32) (*StepFunModelProvider, error)

func (*StepFunModelProvider) GetPricing

func (p *StepFunModelProvider) GetPricing() string

func (*StepFunModelProvider) QueryText

func (p *StepFunModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)

type TencentCloudClient

type TencentCloudClient struct {
	// contains filtered or unexported fields
}

func NewTencentCloudProvider

func NewTencentCloudProvider(secretKey, endpoint, subType string, temperature, topP float32) (*TencentCloudClient, error)

func (*TencentCloudClient) GetPricing

func (c *TencentCloudClient) GetPricing() string

func (*TencentCloudClient) QueryText

func (c *TencentCloudClient) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)

type ThreeDGenRequest added in v1.833.218

type ThreeDGenRequest struct {
	UpstreamModel string `json:"model"`
	Prompt        string `json:"prompt"`
	ImageUrl      string `json:"image_url,omitempty"`
	Format        string `json:"format,omitempty"` // "splat" | "ply" | "glb" | "obj"
	Quality       string `json:"quality,omitempty"`
}

ThreeDGenRequest is the normalized create request for 3D Generation & Gaussian Splats.

type ThreeDStatus added in v1.833.218

type ThreeDStatus struct {
	Id         string `json:"id"`
	Model      string `json:"model"`
	Status     string `json:"status"` // "queued" | "in_progress" | "completed" | "failed"
	Url        string `json:"url,omitempty"`
	Format     string `json:"format,omitempty"`
	SplatCount int    `json:"splat_count,omitempty"`
	Vertices   int    `json:"vertices,omitempty"`
	Error      string `json:"error,omitempty"`
}

ThreeDStatus is the status returned by the 3D generation job tracker.

func CreateThreeDJob added in v1.833.218

func CreateThreeDJob(ctx context.Context, providerUrl, apiKey string, req ThreeDGenRequest) (*ThreeDStatus, error)

CreateThreeDJob posts a 3D generation task to the upstream 3D engine.

type ToolCall

type ToolCall struct {
	Name      string `json:"name"`
	Arguments string `json:"arguments"`
	Content   string `json:"content"`
}

func GetToolCallsFromWriter

func GetToolCallsFromWriter(toolMessage string) []ToolCall

type ToolCallResponse

type ToolCallResponse struct {
	Success  bool   `json:"success"`
	Data     any    `json:"data"`
	Error    string `json:"error,omitempty"`
	ToolName string `json:"toolName"`
}

type Upstream added in v1.833.198

type Upstream string

Upstream names what serves a call, never what is served: Anthropic serves Claude, OpenAI serves GPT. Some are third-party APIs, some are our own compute — Hanzo and Zen run on our GPUs — and some are a runner a deployment starts itself, like Ollama or Local. All of them answer the same question, which is the code that carries a request out.

The set is closed and each member has one spelling, held by the compiler.

const (
	AlibabaCloud  Upstream = "Alibaba Cloud"
	AmazonBedrock Upstream = "Amazon Bedrock"
	Anthropic     Upstream = "Anthropic"
	Azure         Upstream = "Azure"
	Baichuan      Upstream = "Baichuan"
	BaiduCloud    Upstream = "Baidu Cloud"
	ChatGLM       Upstream = "ChatGLM"
	Cohere        Upstream = "Cohere"
	DeepSeek      Upstream = "DeepSeek"
	DigitalOcean  Upstream = "DigitalOcean"
	Dummy         Upstream = "Dummy"
	Enso          Upstream = "Enso"
	Fireworks     Upstream = "Fireworks"
	Gemini        Upstream = "Gemini"
	GitHub        Upstream = "GitHub"
	Grok          Upstream = "Grok"
	Jina          Upstream = "Jina"
	Hanzo         Upstream = "Hanzo"
	HuggingFace   Upstream = "Hugging Face"
	IFlytek       Upstream = "iFlytek"
	Local         Upstream = "Local"
	MiniMax       Upstream = "MiniMax"
	Mistral       Upstream = "Mistral"
	Moonshot      Upstream = "Moonshot"
	Ollama        Upstream = "Ollama"
	OpenAI        Upstream = "OpenAI"
	OpenRouter    Upstream = "OpenRouter"
	SiliconFlow   Upstream = "Silicon Flow"
	StepFun       Upstream = "StepFun"
	TencentCloud  Upstream = "Tencent Cloud"
	VolcanoEngine Upstream = "Volcano Engine"
	Word2Vec      Upstream = "Word2Vec"
	Writer        Upstream = "Writer"
	Yi            Upstream = "Yi"
	Zen           Upstream = "Zen"
)

type VideoGenRequest added in v1.790.4

type VideoGenRequest struct {
	// UpstreamModel is the upstream model id (e.g. "wan2-2-t2v-a14b").
	UpstreamModel string
	Prompt        string
	// Size is an optional OpenAI-style "WxH" hint passed through when set.
	Size string
	// Seconds is an optional clip-length hint (upstream default when 0).
	Seconds int
}

VideoGenRequest is the normalized, provider-agnostic create request the model layer accepts. The async /videos API produces exactly one video per create.

type VolcengineModelProvider

type VolcengineModelProvider struct {
	// contains filtered or unexported fields
}

func NewVolcengineModelProvider

func NewVolcengineModelProvider(subType string, endpointID string, apiKey string, temperature float32, topP float32) (*VolcengineModelProvider, error)

func (*VolcengineModelProvider) GetPricing

func (p *VolcengineModelProvider) GetPricing() string

func (*VolcengineModelProvider) QueryText

func (p *VolcengineModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)

type WriterModelProvider

type WriterModelProvider struct {
	// contains filtered or unexported fields
}

func NewWriterModelProvider

func NewWriterModelProvider(subType string, apiKey string, temperature float32, topP float32) (*WriterModelProvider, error)

func (*WriterModelProvider) GetPricing

func (p *WriterModelProvider) GetPricing() string

func (*WriterModelProvider) QueryText

func (p *WriterModelProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)

type YiProvider

type YiProvider struct {
	// contains filtered or unexported fields
}

func NewYiProvider

func NewYiProvider(subType string, apiKey string, temperature float32, topP float32) (*YiProvider, error)

func (*YiProvider) GetPricing

func (p *YiProvider) GetPricing() string

func (*YiProvider) QueryText

func (p *YiProvider) QueryText(question string, writer io.Writer, history []*RawMessage, prompt string, knowledgeMessages []*RawMessage, agentInfo *AgentInfo, lang string) (*ModelResult, error)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL