models

package
v1.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 1, 2026 License: AGPL-3.0 Imports: 34 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	ChatCompletionsCompatDeepSeek = "deepseek"
	ChatCompletionsCompatMiniMax  = "minimax"
	ChatCompletionsCompatKimi     = "kimi"
)

ChatCompletionsCompatDeepSeek enables DeepSeek request compatibility while still using the generic OpenAI Chat Completions provider.

View Source 
const (
	// DefaultOutputReserveTokens is the completion allowance reserved for a
	// turn when the provider's own cap is not mirrored exactly.
	DefaultOutputReserveTokens = 8192
	// DefaultReasoningOutputReserveTokens is the allowance when thinking is
	// active; reasoning output shares the completion cap on every provider.
	DefaultReasoningOutputReserveTokens = 32000
)
View Source 
const (
	GenerationLimitsProviderDefault = "provider_default"
	GenerationLimitsEstimated       = "estimated"
	GenerationLimitsProviderIgnores = "provider_ignores"
	GenerationLimitsWindowClamped   = "window_clamped"
	GenerationLimitsModelCapped     = "model_capped"
)
View Source 
const (
	DefaultProviderRequestTimeout      = 2 * time.Minute
	DefaultProviderProbeTimeout        = 60 * time.Second
	DefaultProviderTLSHandshakeTimeout = 30 * time.Second
)
View Source 
const (
	// PromptCacheTTL5m enables prompt caching with a short (~5 minute)
	// TTL. Recommended default.
	PromptCacheTTL5m = "5m"
	// PromptCacheTTL1h enables prompt caching with a 1-hour TTL. Some
	// vendors (e.g. Anthropic) bill 1h cache writes at a higher rate
	// than the default short TTL.
	PromptCacheTTL1h = "1h"
	// PromptCacheTTLOff disables prompt caching for the provider. Not
	// recommended: every request rebuilds the prefix without cache.
	PromptCacheTTLOff = "off"
)

Prompt cache TTL options accepted in Provider config.

The values are vendor-neutral; ApplyPromptCache dispatches to the vendor-specific decoration based on the resolved client type. Today only Anthropic Messages implements caching, but the public API stays stable when other vendors gain similar support.

View Source 
const (
	CompatVision      = "vision"
	CompatToolCall    = "tool-call"
	CompatImageOutput = "image-output"
	CompatReasoning   = "reasoning"
	// CompatFileInput marks models whose provider API accepts documents (PDF)
	// as native input parts. Distinct from CompatVision: a model can have
	// vision yet lack a provider-side PDF ingestion pipeline (and vice versa
	// never occurs), so the two are routed independently.
	CompatFileInput = "file-input"
)
View Source 
const (
	ReasoningEffortMinimal = reasoning.EffortMinimal
	ReasoningEffortLow     = reasoning.EffortLow
	ReasoningEffortMedium  = reasoning.EffortMedium
	ReasoningEffortHigh    = reasoning.EffortHigh
	ReasoningEffortXHigh   = reasoning.EffortXHigh
	ReasoningEffortMax     = reasoning.EffortMax

	// ReasoningEffortDisable is the single representation of "no reasoning" —
	// both what a user picks and what a model advertises.
	ReasoningEffortDisable = reasoning.EffortDisable
	// ReasoningEffortNone is OpenAI's wire spelling of the same state, produced
	// by adaptors and never declared or stored.
	ReasoningEffortNone = reasoning.EffortNone
)

Reasoning effort tokens. The vocabulary lives in internal/reasoning; these are forwarding aliases so existing call sites keep compiling while they migrate.

View Source 
const (
	ThinkingModeAdaptive = reasoning.ModeAdaptive
	ThinkingModeToggle   = reasoning.ModeToggle
	ThinkingModeAlways   = reasoning.ModeAlways
	ThinkingModeNone     = reasoning.ModeNone
)

Thinking mode tokens. Semantics live in internal/reasoning; these are forwarding aliases so existing call sites keep compiling while they migrate.

View Source 
const (
	ReasoningReplayContent   = "reasoning_content"
	ReasoningReplayDetails   = "reasoning_details"
	ReasoningReplayReasoning = "reasoning"
)

Reasoning replay fields, matching Twilight's sdk.ReasoningReplayField.

View Source 
const ChatCompletionsCompatConfigKey = "chat_completions_compat"

ChatCompletionsCompatConfigKey is the provider config key holding the explicit Chat Completions compatibility mode.

View Source 
const DefaultPromptCacheTTL = PromptCacheTTL5m

DefaultPromptCacheTTL is the value used when a provider does not explicitly configure a cache policy.

View Source 
const MinCacheablePrefixTokens = 256

MinCacheablePrefixTokens is the floor below which a message-level cache breakpoint is withheld: Anthropic ignores breakpoints on prefixes under its per-model minimum (1024–4096 real tokens), and our byte-heuristic estimate undercounts CJK text, so the floor sits far below the provider minimum to never withhold a viable breakpoint — it only stops the bookkeeping from claiming a cached prefix the provider provably ignored.

View Source 
const ModelConfigSchemaVersion = 2

ModelConfigSchemaVersion is the shape every ModelConfig is written in. A stored config without schema_version is version 1; config_migration.go turns older versions into this one when a row is read, and the next write persists the result. There is no database migration.

Variables

View Source 
var (
	ErrCompactionModelNotConfigured = errors.New("no chat model configured for compaction")
	ErrCompactionModelNotChat       = errors.New("compaction model is not a chat model")
	ErrCompactionModelDisabled      = errors.New("compaction model is disabled")
	ErrCompactionProviderDisabled   = errors.New("compaction model provider is disabled")
	ErrCompactionWindowUnknown      = errors.New("compaction model does not declare a context window")
)

Compaction model resolution failures. A run that must compact reports them as the reason it cannot continue.

View Source 
var (
	ErrModelIDAlreadyExists = errors.New("model_id already exists")
	ErrModelIDAmbiguous     = errors.New("model_id is ambiguous across providers")
	// ErrInvalidModelType reports a model type outside the supported set.
	ErrInvalidModelType = errors.New("invalid model type")
)
View Source 
var ErrEmptyEmbeddingVector = errors.New("embedding provider returned no vector values")

ErrEmptyEmbeddingVector reports that the provider returned an empty vector.

View Source 
var ErrStreamSilent = errors.New("model stream sent nothing within its silence window")

ErrStreamSilent marks a stream that sent nothing for as long as its caller allows.

Functions

func AnthropicThinkingBudget 

func AnthropicThinkingBudget(effort string, contextWindow, maxOutput int) int

AnthropicThinkingBudget is the legacy budget_tokens sent for an effort, fitted so the answer allowance plus the budget stays within half of the configured window but never below the minimum the API accepts; the model construction and the budget plan share it so budget_tokens always stays below the requested max_tokens. A known output cap bounds the budget the same way, so the capped max_tokens still leaves room for the answer.

func ApplyPromptCache 

func ApplyPromptCache(
	model *sdk.Model,
	ttl string,
	system string,
	messages []sdk.Message,
	tools []toolexec.Tool,
) (string, []sdk.Message, []toolexec.Tool)

ApplyPromptCache returns a request payload decorated with provider-specific prompt cache breakpoints. The dispatch is keyed off the resolved client type, so the call site does not need to know which vendor (if any) supports caching for the active model.

For models whose vendor does not implement caching, or when the requested TTL is "off", the inputs are returned unchanged.

func ApplyPromptCacheWithPlan 

func ApplyPromptCacheWithPlan(
	model *sdk.Model,
	ttl string,
	plan contextfrag.CachePlan,
	system string,
	messages []sdk.Message,
	tools []toolexec.Tool,
) (string, []sdk.Message, []toolexec.Tool, bool, int)

ApplyPromptCacheWithPlan decorates the provider request using the placement-derived cache plan: when the plan marks a stable leading message span, the final message of that span receives a cache breakpoint so the stable prefix is cached across turns. A zero plan preserves the legacy system-and-tools-only layout. The 4th return value reports whether this call prepended a system message to messages (Anthropic's system->message cache promotion), so callers can tell that apart from an unrelated leading system-role message already present in the input. The 5th return value is the honest count of leading messages actually covered by an applied message-level breakpoint: it matches plan.StableMessageCount unless placement had to fall back to an earlier message (or found none), so callers can keep their own bookkeeping of the cached prefix truthful instead of trusting a claim that was never actually cached.

func ApplyReasoningToRequest added in v0.21.0 

func ApplyReasoningToRequest(req *sdk.Request, cfg SDKModelConfig)

ApplyReasoningToRequest translates the resolved decision into the provider effort and, for OpenCode Go, the catalog-declared thinking control.

func DefaultProviderUserAgent 

func DefaultProviderUserAgent() string

DefaultProviderUserAgent is the project-level User-Agent for outbound model/provider traffic.

func EnforcesMaxOutputTokens 

func EnforcesMaxOutputTokens(clientType ClientType) bool

EnforcesMaxOutputTokens reports whether the client honors the configured maximum output token limit. A client that does not gets no max_tokens; its output reserve stays an estimate.

func FetchProviderByID 

func FetchProviderByID(ctx context.Context, queries dbstore.Queries, providerID string) (sqlc.Provider, error)

FetchProviderByID fetches a provider by ID.

func InferEmbeddingDimensions 

func InferEmbeddingDimensions(ctx context.Context, clientType, baseURL, apiKey, modelID string, timeout time.Duration, httpClient *http.Client) (int, error)

InferEmbeddingDimensions probes the embedding endpoint and returns the vector length produced by the provider for a minimal input.

func IsCompactionModelUnavailable 

func IsCompactionModelUnavailable(err error) bool

IsCompactionModelUnavailable reports whether the resolution failure means "this bot cannot compact right now" rather than an infrastructure error.

func IsImageOnlyChatModel 

func IsImageOnlyChatModel(model GetResponse, provider sqlc.Provider) bool

IsImageOnlyChatModel reports whether a chat-typed model is actually a dedicated image generator that cannot summarize text.

func IsLLMClientType 

func IsLLMClientType(clientType ClientType) bool

IsLLMClientType returns true if the client type belongs to the LLM domain (chat/embedding), excluding speech-only types (any type ending in "-speech").

func IsReasoningDisabled 

func IsReasoningDisabled(effort string) bool

IsReasoningDisabled reports whether an effort value means "no reasoning".

func IsValidClientType 

func IsValidClientType(clientType ClientType) bool

IsValidClientType returns true if the given client type is supported.

func IsValidModelType 

func IsValidModelType(modelType ModelType) bool

func IsValidReasoningEffort 

func IsValidReasoningEffort(effort string) bool

IsValidReasoningEffort reports whether effort can be stored in ModelConfig.

func LatestSessionModelID 

func LatestSessionModelID(ctx context.Context, queries dbstore.Queries, sessionID string) string

LatestSessionModelID returns the models.id UUID of the most recent history message in the session that recorded one, or "" when the session has no model-bearing history yet.

func ListGoogleEmbeddingModels added in v1.0.0 

func ListGoogleEmbeddingModels(ctx context.Context, baseURL, apiKey string, timeout time.Duration, httpClient *http.Client) ([]sdk.EmbeddingModel, error)

ListGoogleEmbeddingModels lists the models a Google Generative AI endpoint serves for embedContent. The chat provider's model list leaves them out.

func NearestEffortToMedium 

func NearestEffortToMedium(levels []string) string

NearestEffortToMedium picks the tier closest to medium from levels, breaking ties toward the weaker tier.

func NewProviderHTTPClient 

func NewProviderHTTPClient(timeout time.Duration) *http.Client

NewProviderHTTPClient returns an HTTP client for model/provider traffic. When timeout is zero or negative, the caller is expected to enforce limits via context deadlines, which keeps streaming responses unbounded by the client's global timeout while still using the relaxed TLS handshake window.

func NewSDKChatModel 

func NewSDKChatModel(cfg SDKModelConfig) *sdk.Model

NewSDKChatModel builds a Twilight AI SDK Model from the resolved model config.

func NewSDKEmbeddingModel 

func NewSDKEmbeddingModel(clientType, baseURL, apiKey, modelID string, timeout time.Duration, httpClient *http.Client) *sdk.EmbeddingModel

NewSDKEmbeddingModel creates a Twilight AI SDK EmbeddingModel for the given provider configuration. It dispatches to the native Google embedding provider when clientType is "google-generative-ai", and falls back to the OpenAI-compatible /embeddings endpoint for all other provider types.

func NewSDKProvider 

func NewSDKProvider(baseURL, apiKey, codexAccountID string, clientType ClientType, timeout time.Duration, httpClient *http.Client) sdk.Provider

NewSDKProvider creates a Twilight AI SDK Provider for the given client type. It is exported so that other packages (e.g. providers) can reuse it for testing.

func NormalizeAdvertisedEfforts 

func NormalizeAdvertisedEfforts(efforts []string) []string

NormalizeAdvertisedEfforts forwards the catalog-boundary normalizer while callers migrate to the reasoning leaf package.

func NormalizePromptCacheTTL 

func NormalizePromptCacheTTL(s string) string

NormalizePromptCacheTTL coerces an arbitrary user-provided value to one of the accepted TTL constants. Empty or unrecognized values fall back to the recommended short-TTL default.

func PromptCacheKey added in v0.20.0 

func PromptCacheKey(model *sdk.Model, ttl, sessionID string) string

PromptCacheKey returns the per-session cache-routing key for OpenAI-family clients. prompt_cache_key groups requests that share a prefix lineage onto cache-warm backends, and the session is exactly that lineage; providers that cache via explicit breakpoints get no key.

func ReasoningEffortParam added in v0.21.0 

func ReasoningEffortParam(cfg SDKModelConfig) (string, bool)

ReasoningEffortParam resolves the per-request reasoning effort value for the resolved thinking decision. ok reports whether an effort should be sent at all; ApplyReasoningToRequest sets Request.ReasoningEffort from it.

func RequestConfigNeedsClientType added in v1.0.0 

func RequestConfigNeedsClientType(raw []byte, modelID string) bool

RequestConfigNeedsClientType reports whether DecodeRequestModelConfig reads the provider's client type for this request: it goes through the version 1 upgrade, and that upgrade depends on the protocol.

func ResolveChatCompletionsCompat 

func ResolveChatCompletionsCompat(baseURL, compat string) string

ResolveChatCompletionsCompat returns the compatibility mode for a provider. An explicit config value always wins, including one that matches no known mode (e.g. "none"), which disables inference. With no explicit value, the mode is inferred from official endpoint origins so provider rows created before the config existed keep their protocol adaptations. Origins match by exact origin or path prefix (covering /v1, /beta, ...), never by substring, so lookalike domains and proxies that merely embed an official hostname are not classified.

func ResolveClientType 

func ResolveClientType(model *sdk.Model) string

ResolveClientType infers the client type string from an SDK Model's provider name.

func ResolveEnable 

func ResolveEnable(override *bool, current bool) bool

ResolveEnable returns the effective enable flag: when the override is nil, the current value is preserved; otherwise the override wins. Used by Service.Create (current=true default) and Service.UpdateByID (current=stored).

func ResolveModelClientType added in v0.21.0 

func ResolveModelClientType(clientType, modelID string) string

ResolveModelClientType uses Twilight's routing table for multi-protocol providers, so prompt caching and media handling follow the actual wire protocol. Reasoning policy keeps the provider identity: sharing a protocol does not mean Go models accept Claude's adaptive flag or OpenAI's effort normalization.

func StreamResult added in v1.0.0 

func StreamResult(ctx context.Context, model *sdk.Model, req sdk.Request, silence time.Duration) (sdk.ModelResult, error)

StreamResult answers req through model as a stream and returns what the stream completed with. A long answer is bounded by silence rather than by a deadline on its first byte: a stream that sends nothing for silence is cut off as an incomplete one, which a retry may ask again for.

func ValidateCompatibilities 

func ValidateCompatibilities(compatibilities []string) error

ValidateCompatibilities validates capability tokens supplied by a client.

func WithModelSession added in v0.21.0 

func WithModelSession(ctx context.Context, sessionID string) context.Context

WithModelSession scopes provider session metadata to the owning conversation. Only providers that require it forward this value to the remote endpoint.

Types

type AddRequest 

type AddRequest struct {
	ModelID    string      `json:"model_id"`
	Name       string      `json:"name,omitempty"`
	ProviderID string      `json:"provider_id"`
	Type       ModelType   `json:"type"`
	Enable     *bool       `json:"enable,omitempty"`
	Config     ModelConfig `json:"config"`
	// contains filtered or unexported fields
}

AddRequest is the payload for creating a new model. Enable is a pointer so the server can default to true when the field is absent from the request.

func (*AddRequest) UnmarshalJSON added in v1.0.0 

func (r *AddRequest) UnmarshalJSON(data []byte) error

UnmarshalJSON reads config in any schema version; see DecodeRequestModelConfig.

type AddResponse 

type AddResponse struct {
	ID      string `json:"id"`
	ModelID string `json:"model_id"`
}

type ClientType 

type ClientType string
const (
	ClientTypeOpenAIResponses         ClientType = "openai-responses"
	ClientTypeOpenAICompletions       ClientType = "openai-completions"
	ClientTypeAnthropicMessages       ClientType = "anthropic-messages"
	ClientTypeGoogleGenerativeAI      ClientType = "google-generative-ai"
	ClientTypeOpenAICodex             ClientType = "openai-codex"
	ClientTypeGitHubCopilot           ClientType = "github-copilot"
	ClientTypeOpenCodeGo              ClientType = "opencode-go"
	ClientTypeEdgeSpeech              ClientType = "edge-speech"
	ClientTypeOpenAISpeech            ClientType = "openai-speech"
	ClientTypeOpenAITranscription     ClientType = "openai-transcription"
	ClientTypeOpenRouterSpeech        ClientType = "openrouter-speech"
	ClientTypeOpenRouterTranscription ClientType = "openrouter-transcription"
	ClientTypeElevenLabsSpeech        ClientType = "elevenlabs-speech"
	ClientTypeElevenLabsTranscription ClientType = "elevenlabs-transcription"
	ClientTypeDeepgramSpeech          ClientType = "deepgram-speech"
	ClientTypeDeepgramTranscription   ClientType = "deepgram-transcription"
	ClientTypeMiniMaxSpeech           ClientType = "minimax-speech"
	ClientTypeVolcengineSpeech        ClientType = "volcengine-speech"
	ClientTypeAlibabaSpeech           ClientType = "alibabacloud-speech"
	ClientTypeAlibabaTranscription    ClientType = "alibabacloud-transcription"
	ClientTypeMicrosoftSpeech         ClientType = "microsoft-speech"
	ClientTypeGoogleSpeech            ClientType = "google-speech"
	ClientTypeGoogleTranscription     ClientType = "google-transcription"
	ClientTypeOpenRouterVideo         ClientType = "openrouter-video"
	ClientTypeModelArkVideo           ClientType = "modelark-video"
	ClientTypeVolcengineVideo         ClientType = "volcengine-video"
)

type CompactionModelResolution 

type CompactionModelResolution struct {
	Model    GetResponse
	Provider sqlc.Provider
	// WindowTokens is the summarizer model's declared context window.
	WindowTokens int
}

CompactionModelResolution is the resolved summarizer identity. Credentials stay with the caller: orchestration surfaces own auth context, the compaction engine only receives a completed contract.

func ResolveCompactionModel 

func ResolveCompactionModel(
	ctx context.Context,
	modelsService *Service,
	queries dbstore.Queries,
	candidates ...string,
) (CompactionModelResolution, error)

ResolveCompactionModel picks the first non-empty candidate model id and validates that it can act as a summarizer: a chat-type, enabled model on an enabled LLM provider that declares a context window (the summary budget derives from it, so an unknown window fails closed). A client that ignores max_tokens (Codex) can still summarize: a pass commits a summary only once it fits the room it was asked for. Callers compose the candidate chain: a run passes its actually-resolved model first; the session info surface prefers the session's latest model before the bot default.

type ConfigContext added in v1.0.0 

type ConfigContext struct {
	ModelID string
	Type    ModelType
	// ClientType is the provider's client type when the caller knows it. The
	// version 1 upgrade reads it to keep a switch that only the protocol could
	// turn off resolving as it did; empty leaves that answer as not stated.
	ClientType string
}

ConfigContext is what an upgrade may need beyond the config itself. Version 1 left the thinking mode empty on rows imported before it existed, and the mode was then inferred from the model id.

type CountResponse 

type CountResponse struct {
	Count int64 `json:"count"`
}

type DeleteRequest 

type DeleteRequest struct {
	ID      string `json:"id,omitempty"`
	ModelID string `json:"model_id,omitempty"`
}

type DeleteResponse 

type DeleteResponse struct {
	Message string `json:"message"`
}

type FieldError added in v0.22.0 

type FieldError struct {
	Field    string
	Required bool
	Err      error
}

FieldError is a model write rejected because of one request field. Field is the name the request uses for it, with dots for a nested key; Required says the field is missing or empty rather than holding a bad value. Err is the private reason.

func (*FieldError) Error added in v0.22.0 

func (e *FieldError) Error() string

func (*FieldError) Unwrap added in v0.22.0 

func (e *FieldError) Unwrap() error

type GenerationLimits 

type GenerationLimits struct {
	MaxOutputTokens int
	Requested       bool
	Resolution      string
}

GenerationLimits is the single authority for one turn's output allowance: the context budget plan reserves MaxOutputTokens and, when Requested, the provider request carries the same value as max_tokens. Unrequested limits are Memoh's reserve only; the provider keeps its own default because the model's real cap is unknown and an explicit value could be rejected. A configured model cap makes every limit explicit and keeps it within the cap.

func ResolveGenerationLimits 

func ResolveGenerationLimits(clientType ClientType, reasoning *ReasoningConfig, contextWindow, maxOutput int) GenerationLimits

ResolveGenerationLimits derives the output allowance from the client type, the resolved thinking decision, the configured context window and the model's maximum output tokens (zero when unknown). Anthropic mirrors the SDK's own defaults, so the reserved and requested values cannot diverge; every other client is estimated without a request unless the model's cap is known. Codex ignores the cap and never requests.

type GetRequest 

type GetRequest struct {
	ID string `json:"id"`
}

type GetResponse 

type GetResponse struct {
	ID      string `json:"id"`
	ModelID string `json:"model_id"`
	Model
	// Reasoning is the model's resolved thinking options, filled by the API layer
	// (it depends on the provider's client type). Clients render this rather than
	// deriving their own answer from ThinkingMode and ReasoningEfforts — the
	// duplication that let the web picker and the wire disagree.
	Reasoning *reasoning.Options `json:"reasoning,omitempty"`
}

func SelectMemoryModel 

func SelectMemoryModel(ctx context.Context, modelsService *Service, queries dbstore.Queries) (GetResponse, sqlc.Provider, error)

SelectMemoryModel selects a chat model for memory operations. It only considers models from enabled providers.

func SelectMemoryModelForBot 

func SelectMemoryModelForBot(ctx context.Context, modelsService *Service, queries dbstore.Queries, chatModelID string) (GetResponse, sqlc.Provider, error)

SelectMemoryModelForBot selects a chat model for memory operations. If botID is provided, it attempts to use the bot's configured chat model first, falling back to the first enabled chat model globally. Models or providers that the user has disabled are skipped so memory extraction/decision/compact never quietly run on a row hidden from the UI.

func (GetResponse) MarshalJSON added in v1.0.0 

func (r GetResponse) MarshalJSON() ([]byte, error)

MarshalJSON adds the version 1 reading of a chat model's config next to its current fields. Clients built before version 2 (an older desktop build, a cached web bundle) read capabilities and efforts only from those keys, and copy them back when they save, which DecodeRequestModelConfig reads as their edit (see mergeLegacyEdit). Current clients ignore the keys and drop them before saving. The keys exist only in API responses, never in the row.

type LegacyModelConfig added in v1.0.0 

type LegacyModelConfig struct {
	Description         *string  `json:"description,omitempty"`
	Dimensions          *int     `json:"dimensions,omitempty"`
	Compatibilities     []string `json:"compatibilities,omitempty"`
	ContextWindow       *int     `json:"context_window,omitempty"`
	ReasoningEfforts    []string `json:"reasoning_efforts,omitempty"`
	ThinkingMode        string   `json:"thinking_mode,omitempty"`
	CatalogAvailable    *bool    `json:"catalog_available,omitempty"`
	ReasoningDialect    string   `json:"reasoning_dialect,omitempty"`
	ReasoningOffSupport string   `json:"reasoning_off_support,omitempty"`
	ReasoningDefaultOn  *bool    `json:"reasoning_default_on,omitempty"`
	ThinkingBudgetMin   *int     `json:"thinking_budget_min,omitempty"`
	ThinkingBudgetMax   *int     `json:"thinking_budget_max,omitempty"`
	MaxOutputTokens     *int     `json:"max_output_tokens,omitempty"`
}

LegacyModelConfig is the version 1 shape. Capabilities were a token list, and reasoning was a thinking mode plus flat fields; the effort list could carry "enabled" for a switch without levels and "disable" for a model that can be turned off. Besides reading old rows, the provider template import still merges discovered capabilities in this vocabulary; see MergeLegacyConfig.

func (LegacyModelConfig) Upgrade added in v1.0.0 

func (v1 LegacyModelConfig) Upgrade(ctx ConfigContext) ModelConfig

Upgrade maps a version 1 config onto version 2. Every model type keeps its description, limits, dimensions and catalog flag. Chat models also turn their capability tokens into modalities and tool calling, and their thinking fields into Reasoning; ModelConfig.ThinkingMode and ReasoningEfforts read Reasoning back as the version 1 runtime saw it.

type ListRequest 

type ListRequest struct {
	Type ModelType `json:"type,omitempty"`
}

type Modality added in v1.0.0 

type Modality string

Modality is a kind of content a model accepts or produces. The values match Twilight's sdk.Modality. 

const (
	ModalityText  Modality = "text"
	ModalityImage Modality = "image"
	ModalityAudio Modality = "audio"
	ModalityVideo Modality = "video"
	ModalityPDF   Modality = "pdf"
)

type Model 

type Model struct {
	ModelID    string      `json:"model_id"`
	Name       string      `json:"name"`
	ProviderID string      `json:"provider_id"`
	Type       ModelType   `json:"type"`
	Enable     bool        `json:"enable"`
	Config     ModelConfig `json:"config"`
}

func (*Model) HasCompatibility 

func (m *Model) HasCompatibility(c string) bool

HasCompatibility reports whether the model has a capability; see ModelConfig.Supports.

func (*Model) ReasoningOptions 

func (m *Model) ReasoningOptions(clientType string) reasoning.Options

ReasoningOptions reports what a caller may select for this model on the given client type: the selectable tiers, whether off is reachable, and the default. It is the single source every surface reads — the web picker, /reasoning, and the API all render this rather than deriving their own answer.

func (*Model) ResolveThinkingMode 

func (m *Model) ResolveThinkingMode() string

ResolveThinkingMode returns the model's thinking mode.

func (*Model) Validate 

func (m *Model) Validate() error

Validate checks a model write. Every failure is a *FieldError naming the request field that holds the offending value.

type ModelConfig 

type ModelConfig struct {
	// SchemaVersion is ModelConfigSchemaVersion on every config this version
	// writes.
	SchemaVersion int     `json:"schema_version"`
	Description   *string `json:"description,omitempty"`

	Modalities *ModelModalities `json:"modalities,omitempty"`
	// ContextWindow is the context window in tokens, input and output together.
	ContextWindow *int `json:"context_window,omitempty"`
	// MaxOutputTokens is the most output tokens the model can produce in one
	// response; nil means unknown. When set, every turn requests an explicit
	// limit within it.
	MaxOutputTokens *int `json:"max_output_tokens,omitempty"`
	ToolCall        bool `json:"tool_call,omitempty"`
	// StructuredOutput reports support for a JSON schema response format.
	StructuredOutput bool `json:"structured_output,omitempty"`
	// Reasoning is nil when the model does not reason.
	Reasoning *ModelReasoning `json:"reasoning,omitempty"`

	// Dimensions is the embedding vector size.
	Dimensions *int `json:"dimensions,omitempty"`

	// CatalogAvailable is false when a managed catalog stopped listing the model.
	CatalogAvailable *bool `json:"catalog_available,omitempty"`
}

ModelConfig holds the JSONB config stored per model, in the shape of Twilight's sdk.Model and sdk.EmbeddingModel. Chat models use Modalities, ContextWindow, MaxOutputTokens, ToolCall, StructuredOutput and Reasoning; embedding models use Dimensions.

func DecodeModelConfig added in v1.0.0 

func DecodeModelConfig(raw []byte, ctx ConfigContext) (ModelConfig, error)

DecodeModelConfig reads a stored config of any schema version into the current shape. Empty input decodes to an empty config. A version newer than this build decodes as the current one, keeping the fields it knows.

func DecodeRequestModelConfig added in v1.0.0 

func DecodeRequestModelConfig(raw []byte, ctx ConfigContext) (ModelConfig, error)

DecodeRequestModelConfig reads a config sent by an API client. A stored config without schema_version is version 1, but a client may send the current shape and leave the version out, so a request without one is read as version 1 only when it uses a key that only version 1 has. A request with both the current version and version 1 keys comes from an older client editing a current config; see mergeLegacyEdit.

func MergeLegacyConfig added in v1.0.0 

func MergeLegacyConfig(existing ModelConfig, merged LegacyModelConfig, ctx ConfigContext) ModelConfig

MergeLegacyConfig upgrades a config merged in the version 1 vocabulary and keeps what version 1 cannot express from existing: structured output, modalities other than text, image and PDF, the reasoning replay, and the default effort and budget.

The version 1 reading of a config is lossy for shapes only version 2 can state, such as an off answer without a switch, or a switch with a budget that cannot be turned off; upgrading that reading back would rewrite them. So the parts the merge did not change keep the existing config's version 2 form: its reasoning when the thinking fields came out the same, and its modalities and tool calling when the capability tokens did.

func (ModelConfig) Accepts added in v1.0.0 

func (c ModelConfig) Accepts(modality Modality) bool

Accepts reports whether the model takes the modality as input.

func (ModelConfig) ContextBudgetMaxTokens 

func (c ModelConfig) ContextBudgetMaxTokens() int

ContextBudgetMaxTokens returns the configured model context window, or zero when budget enforcement is unavailable for this model.

func (ModelConfig) Legacy added in v1.0.0 

func (c ModelConfig) Legacy() LegacyModelConfig

Legacy reads the config back in the version 1 vocabulary, as the version 1 runtime would have seen it.

func (ModelConfig) MarshalJSON added in v1.0.0 

func (c ModelConfig) MarshalJSON() ([]byte, error)

MarshalJSON always writes the current schema version, so a config built without one is never read back as version 1.

func (ModelConfig) MaxOutputTokensLimit added in v0.22.0 

func (c ModelConfig) MaxOutputTokensLimit() int

MaxOutputTokensLimit returns the configured maximum output tokens, or zero when the model's cap is unknown.

func (ModelConfig) Produces added in v1.0.0 

func (c ModelConfig) Produces(modality Modality) bool

Produces reports whether the model returns the modality as output.

func (ModelConfig) ReasoningDefaultOn 

func (c ModelConfig) ReasoningDefaultOn() *bool

ReasoningDefaultOn reports whether the model reasons when a request sets no control; nil means not stated.

func (ModelConfig) ReasoningDialect 

func (c ModelConfig) ReasoningDialect() string

ReasoningDialect returns the wire dialect for the tier, budget and toggle shapes. Adaptive is a thinking mode, not a dialect, so it reads as empty.

func (ModelConfig) ReasoningEfforts 

func (c ModelConfig) ReasoningEfforts() []string

ReasoningEfforts returns the advertised effort list: the effort levels, then for a toggle the enabled token when there is no level or budget to pick, and the disable token unless the model rejects being turned off. A declared Off without a toggle adds nothing: the reasoning package reads it directly.

func (ModelConfig) ReasoningOffSupport 

func (c ModelConfig) ReasoningOffSupport() string

ReasoningOffSupport returns how the model answers an explicit disable.

func (ModelConfig) ReasoningReplay added in v1.0.0 

func (c ModelConfig) ReasoningReplay() *ReasoningReplay

ReasoningReplay returns the model's declared replay, or nil.

func (ModelConfig) StoredForm added in v1.0.0 

func (c ModelConfig) StoredForm(modelID string) ModelConfig

StoredForm returns the config as Create and Update store it: normalized, and with the dialect the model's id implies when none is declared. A caller that compares a stored row with a config it is about to write must compare the stored form, or the write-time changes read as a difference on every pass.

func (ModelConfig) Supports added in v1.0.0 

func (c ModelConfig) Supports(capability string) bool

Supports reports whether the model has a capability: vision and file-input are image and PDF input, image-output is image output, tool-call is tool calling, and reasoning is any reasoning at all.

func (ModelConfig) ThinkingBudgetMax 

func (c ModelConfig) ThinkingBudgetMax() *int

ThinkingBudgetMax returns the upper bound of the reasoning token budget.

func (ModelConfig) ThinkingBudgetMin 

func (c ModelConfig) ThinkingBudgetMin() *int

ThinkingBudgetMin returns the lower bound of the reasoning token budget.

func (ModelConfig) ThinkingMode 

func (c ModelConfig) ThinkingMode() string

ThinkingMode returns the model's thinking mode: none without reasoning, adaptive for the adaptive dialect, always when no request can influence reasoning, and toggle otherwise.

func (ModelConfig) Validate added in v1.0.0 

func (c ModelConfig) Validate(modelType ModelType) error

Validate checks configuration values for a resolved model type. Catalogs can validate configs before a tenant provider or database model exists.

type ModelModalities added in v1.0.0 

type ModelModalities struct {
	Input  []Modality `json:"input,omitempty"`
	Output []Modality `json:"output,omitempty"`
}

ModelModalities lists what a model accepts and produces.

type ModelReasoning added in v1.0.0 

type ModelReasoning struct {
	// Toggle reports a request field that turns reasoning on and off, separate
	// from any effort level.
	Toggle bool `json:"toggle,omitempty"`
	// Efforts lists the effort levels the model accepts, weakest to strongest.
	// Turning reasoning off is Off, not an effort.
	Efforts []string `json:"efforts,omitempty"`
	// Budget is the reasoning token budget range; nil means no budget control.
	Budget *ReasoningBudget `json:"budget,omitempty"`
	// Off is how the model answers a request to stop reasoning: accepted,
	// low_effort_only, rejected, or empty when not stated.
	Off string `json:"off,omitempty"`
	// Dialect is the request shape that carries the controls: tier, budget,
	// toggle, adaptive, or empty when not stated.
	Dialect string `json:"dialect,omitempty"`
	// Default is what the model does when a request sets no reasoning control.
	Default *ReasoningDefault `json:"default,omitempty"`
	// Replay is how a Chat Completions host reads reasoning back on replayed
	// messages; nil leaves the provider's compatibility mode in charge.
	Replay *ReasoningReplay `json:"replay,omitempty"`
}

ModelReasoning mirrors Twilight's sdk.ModelReasoning: the reasoning controls a model exposes (Toggle, Efforts, Budget, Off), the request shape that carries them (Dialect), what the model does when asked for nothing (Default), and how a Chat Completions host reads reasoning back (Replay).

func (*ModelReasoning) CanTurnOff added in v1.0.0 

func (r *ModelReasoning) CanTurnOff() bool

CanTurnOff reports whether a request can stop the model reasoning. A stated Off decides; when it is not stated, a toggle is taken to switch it off.

func (*ModelReasoning) Controllable added in v1.0.0 

func (r *ModelReasoning) Controllable() bool

Controllable reports whether a request can influence reasoning at all.

type ModelType 

type ModelType string
const (
	ModelTypeChat          ModelType = "chat"
	ModelTypeEmbedding     ModelType = "embedding"
	ModelTypeSpeech        ModelType = "speech"
	ModelTypeTranscription ModelType = "transcription"
	ModelTypeVideo         ModelType = "video"
)

type ReasoningBudget added in v1.0.0 

type ReasoningBudget struct {
	Min *int `json:"min,omitempty"`
	Max *int `json:"max,omitempty"`
}

ReasoningBudget bounds a reasoning token budget. A nil bound is not stated.

type ReasoningConfig 

type ReasoningConfig = reasoning.Config

ReasoningConfig is the resolved extended-thinking decision for one call, produced by internal/reasoning and translated here into each provider's wire shape. Anthropic 4.6+ (Adaptive) sends thinking{type:"adaptive"} plus an effort string and never a token budget; legacy Anthropic (<=4.5, non-adaptive) sends thinking{type:"enabled", budget_tokens:N} derived from the effort. OpenAI-style providers only ever receive an effort string.

type ReasoningDefault added in v1.0.0 

type ReasoningDefault struct {
	Enabled *bool  `json:"enabled,omitempty"`
	Effort  string `json:"effort,omitempty"`
	Budget  *int   `json:"budget,omitempty"`
}

ReasoningDefault is what a model does when a request sets no reasoning control. A zero field is not stated.

type ReasoningReplay added in v1.0.0 

type ReasoningReplay struct {
	Field    string `json:"field,omitempty"`
	Required bool   `json:"required,omitempty"`
}

ReasoningReplay is the assistant message field that carries a step's reasoning back to a Chat Completions host, and whether every tool call replay must carry it.

type SDKModelConfig 

type SDKModelConfig struct {
	ModelID        string
	ClientType     string
	APIKey         string //nolint:gosec // carries provider credential material at runtime
	CodexAccountID string
	BaseURL        string
	// ChatCompletionsCompat selects narrow compatibility behavior for
	// OpenAI-compatible /chat/completions backends.
	ChatCompletionsCompat string
	HTTPClient            *http.Client
	ReasoningConfig       *ReasoningConfig
	// ReasoningDialect and the budget bounds come from the model's catalog entry.
	// They say how this model spells its thinking control, which is not derivable
	// from the tiers it advertises.
	ReasoningDialect  string
	ThinkingBudgetMin *int
	ThinkingBudgetMax *int
	// ReasoningOffSupport declares whether this model accepts an explicit
	// thinking{type:"disabled"}. See anthropicAcceptsExplicitOff.
	ReasoningOffSupport string
	// ReasoningReplay is the model's declared replay for Chat Completions hosts.
	// It overrides the replay the compatibility mode implies; nil keeps it.
	ReasoningReplay *ReasoningReplay
	// ContextWindow is the configured context window the turn budgets against;
	// legacy Anthropic thinking budgets are fitted to it. Zero means unknown.
	ContextWindow int
	// MaxOutputTokens is the model's configured output cap; the legacy
	// Anthropic thinking budget is fitted below it. Zero means unknown.
	MaxOutputTokens int
}

SDKModelConfig holds provider and model information resolved from DB, used to construct a Twilight AI SDK Model instance.

type Service 

type Service struct {
	// contains filtered or unexported fields
}

Service provides CRUD operations for models.

func NewService 

func NewService(log *slog.Logger, queries dbstore.Queries) *Service

NewService creates a new models service.

func (*Service) Count 

func (s *Service) Count(ctx context.Context) (int64, error)

Count returns the total number of models.

func (*Service) CountByType 

func (s *Service) CountByType(ctx context.Context, modelType ModelType) (int64, error)

CountByType returns the number of models of a specific type.

func (*Service) Create 

func (s *Service) Create(ctx context.Context, req AddRequest) (AddResponse, error)

Create adds a new model to the database.

func (*Service) DeleteByID 

func (s *Service) DeleteByID(ctx context.Context, id string) error

DeleteByID deletes a model by its internal UUID.

func (*Service) DeleteByModelID 

func (s *Service) DeleteByModelID(ctx context.Context, modelID string) error

DeleteByModelID deletes a model by its model_id field.

func (*Service) GetByID 

func (s *Service) GetByID(ctx context.Context, id string) (GetResponse, error)

GetByID retrieves a model by its internal UUID.

func (*Service) GetByModelID 

func (s *Service) GetByModelID(ctx context.Context, modelID string) (GetResponse, error)

GetByModelID retrieves a model by its model_id field.

func (*Service) GetByProviderAndModelID 

func (s *Service) GetByProviderAndModelID(ctx context.Context, providerID, modelID string) (GetResponse, error)

GetByProviderAndModelID retrieves a model by provider and model_id.

func (*Service) List 

func (s *Service) List(ctx context.Context) ([]GetResponse, error)

List returns all models.

func (*Service) ListByProviderClientType 

func (s *Service) ListByProviderClientType(ctx context.Context, clientType ClientType) ([]GetResponse, error)

ListByProviderClientType returns models whose provider has the given client_type.

func (*Service) ListByProviderID 

func (s *Service) ListByProviderID(ctx context.Context, providerID string) ([]GetResponse, error)

ListByProviderID returns models filtered by provider ID.

func (*Service) ListByProviderIDAndType 

func (s *Service) ListByProviderIDAndType(ctx context.Context, providerID string, modelType ModelType) ([]GetResponse, error)

ListByProviderIDAndType returns models filtered by provider ID and type.

func (*Service) ListByType 

func (s *Service) ListByType(ctx context.Context, modelType ModelType) ([]GetResponse, error)

ListByType returns models filtered by type.

func (*Service) ListEnabled 

func (s *Service) ListEnabled(ctx context.Context) ([]GetResponse, error)

ListEnabled returns all models from enabled providers.

func (*Service) ListEnabledByProviderClientType 

func (s *Service) ListEnabledByProviderClientType(ctx context.Context, clientType ClientType) ([]GetResponse, error)

ListEnabledByProviderClientType returns models from enabled providers with the given client_type.

func (*Service) ListEnabledByType 

func (s *Service) ListEnabledByType(ctx context.Context, modelType ModelType) ([]GetResponse, error)

ListEnabledByType returns models from enabled providers filtered by type.

func (*Service) ResolveReasoningOptions 

func (s *Service) ResolveReasoningOptions(ctx context.Context, id string) (reasoning.Options, error)

ResolveReasoningOptions returns the selectable reasoning contract for a model using the client type of its persisted provider. Keeping provider lookup here gives commands and settings writes the same capability answer.

func (*Service) Test 

func (s *Service) Test(ctx context.Context, id string) (TestResponse, error)

Test probes a model's provider endpoint using the Twilight AI SDK to verify connectivity, authentication, and model availability. Every outcome other than ok and model_not_supported carries the SDK error as Cause.

func (*Service) UpdateByID 

func (s *Service) UpdateByID(ctx context.Context, id string, req UpdateRequest) (GetResponse, error)

UpdateByID updates a model by its internal UUID.

func (*Service) UpdateByModelID 

func (s *Service) UpdateByModelID(ctx context.Context, modelID string, req UpdateRequest) (GetResponse, error)

UpdateByModelID updates a model by its model_id field.

func (*Service) UpdateByProviderAndModelID 

func (s *Service) UpdateByProviderAndModelID(ctx context.Context, providerID, modelID string, req UpdateRequest) (GetResponse, error)

UpdateByProviderAndModelID updates a model within one provider namespace.

type TestResponse 

type TestResponse struct {
	Status    TestStatus `json:"status"`
	Reachable bool       `json:"reachable"`
	LatencyMs int64      `json:"latency_ms,omitempty"`
	// Code is the error catalog code for why the test did not pass. It is
	// empty on success and when no catalog code describes the failure.
	Code string `json:"code,omitempty"`
	// Cause is why the test did not pass. It never leaves the process; the
	// handler records it and derives Code from it.
	Cause error `json:"-"`
}

TestResponse is returned by POST /models/:id/test.

type TestStatus 

type TestStatus string

TestStatus represents the outcome of probing a model. 

const (
	TestStatusOK                TestStatus = "ok"
	TestStatusAuthError         TestStatus = "auth_error"
	TestStatusModelNotSupported TestStatus = "model_not_supported"
	TestStatusError             TestStatus = "error"
)

type UpdateRequest 

type UpdateRequest struct {
	ModelID    string      `json:"model_id"`
	Name       string      `json:"name,omitempty"`
	ProviderID string      `json:"provider_id"`
	Type       ModelType   `json:"type"`
	Enable     *bool       `json:"enable,omitempty"`
	Config     ModelConfig `json:"config"`
	// contains filtered or unexported fields
}

UpdateRequest is the payload for updating an existing model. Enable is a pointer so callers can omit it to preserve the current enable state while still rewriting the other fields.

func (*UpdateRequest) UnmarshalJSON added in v1.0.0 

func (r *UpdateRequest) UnmarshalJSON(data []byte) error

UnmarshalJSON reads config in any schema version; see DecodeRequestModelConfig.

Source Files

View all Source files

Directories

Path Synopsis
Package modelretry decides whether a failed model call is made again, and when.
 Package modelretry decides whether a failed model call is made again, and when.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.