models

package
v0.20.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 15, 2026 License: AGPL-3.0 Imports: 29 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	ChatCompletionsCompatDeepSeek = "deepseek"
	ChatCompletionsCompatMiniMax  = "minimax"
	ChatCompletionsCompatKimi     = "kimi"
)

ChatCompletionsCompatDeepSeek enables DeepSeek request compatibility while still using the generic OpenAI Chat Completions provider.

View Source 
const (
	// DefaultOutputReserveTokens is the completion allowance reserved for a
	// turn when the provider's own cap is not mirrored exactly.
	DefaultOutputReserveTokens = 8192
	// DefaultReasoningOutputReserveTokens is the allowance when thinking is
	// active; reasoning output shares the completion cap on every provider.
	DefaultReasoningOutputReserveTokens = 32000
)
View Source 
const (
	GenerationLimitsProviderDefault = "provider_default"
	GenerationLimitsEstimated       = "estimated"
	GenerationLimitsProviderIgnores = "provider_ignores"
	GenerationLimitsWindowClamped   = "window_clamped"
)
View Source 
const (
	DefaultProviderRequestTimeout      = 2 * time.Minute
	DefaultProviderProbeTimeout        = 60 * time.Second
	DefaultProviderTLSHandshakeTimeout = 30 * time.Second
)
View Source 
const (
	// PromptCacheTTL5m enables prompt caching with a short (~5 minute)
	// TTL. Recommended default.
	PromptCacheTTL5m = "5m"
	// PromptCacheTTL1h enables prompt caching with a 1-hour TTL. Some
	// vendors (e.g. Anthropic) bill 1h cache writes at a higher rate
	// than the default short TTL.
	PromptCacheTTL1h = "1h"
	// PromptCacheTTLOff disables prompt caching for the provider. Not
	// recommended: every request rebuilds the prefix without cache.
	PromptCacheTTLOff = "off"
)

Prompt cache TTL options accepted in Provider config.

The values are vendor-neutral; ApplyPromptCache dispatches to the vendor-specific decoration based on the resolved client type. Today only Anthropic Messages implements caching, but the public API stays stable when other vendors gain similar support.

View Source 
const (
	CompatVision      = "vision"
	CompatToolCall    = "tool-call"
	CompatImageOutput = "image-output"
	CompatReasoning   = "reasoning"
	// CompatFileInput marks models whose provider API accepts documents (PDF)
	// as native input parts. Distinct from CompatVision: a model can have
	// vision yet lack a provider-side PDF ingestion pipeline (and vice versa
	// never occurs), so the two are routed independently.
	CompatFileInput = "file-input"
)
View Source 
const (
	ReasoningEffortMinimal = reasoning.EffortMinimal
	ReasoningEffortLow     = reasoning.EffortLow
	ReasoningEffortMedium  = reasoning.EffortMedium
	ReasoningEffortHigh    = reasoning.EffortHigh
	ReasoningEffortXHigh   = reasoning.EffortXHigh
	ReasoningEffortMax     = reasoning.EffortMax

	// ReasoningEffortDisable is the single representation of "no reasoning" —
	// both what a user picks and what a model advertises.
	ReasoningEffortDisable = reasoning.EffortDisable
	// ReasoningEffortNone is OpenAI's wire spelling of the same state, produced
	// by adaptors and never declared or stored.
	ReasoningEffortNone = reasoning.EffortNone
)

Reasoning effort tokens. The vocabulary lives in internal/reasoning; these are forwarding aliases so existing call sites keep compiling while they migrate.

View Source 
const (
	ThinkingModeAdaptive     = reasoning.ModeAdaptive
	ThinkingModeToggle       = reasoning.ModeToggle
	ThinkingModeAlways       = reasoning.ModeAlways
	ThinkingModeOnlyAdaptive = reasoning.ModeOnlyAdaptive
	ThinkingModeNone         = reasoning.ModeNone
)

Thinking mode tokens. Semantics live in internal/reasoning; these are forwarding aliases so existing call sites keep compiling while they migrate.

View Source 
const ChatCompletionsCompatConfigKey = "chat_completions_compat"

ChatCompletionsCompatConfigKey is the provider config key holding the explicit Chat Completions compatibility mode.

View Source 
const DefaultPromptCacheTTL = PromptCacheTTL5m

DefaultPromptCacheTTL is the value used when a provider does not explicitly configure a cache policy.

View Source 
const MinCacheablePrefixTokens = 256

MinCacheablePrefixTokens is the floor below which a message-level cache breakpoint is withheld: Anthropic ignores breakpoints on prefixes under its per-model minimum (1024–4096 real tokens), and our byte-heuristic estimate undercounts CJK text, so the floor sits far below the provider minimum to never withhold a viable breakpoint — it only stops the bookkeeping from claiming a cached prefix the provider provably ignored.

Variables

View Source 
var (
	ErrCompactionModelNotConfigured     = errors.New("no compaction or chat model configured")
	ErrCompactionModelNotChat           = errors.New("compaction model is not a chat model")
	ErrCompactionModelDisabled          = errors.New("compaction model is disabled")
	ErrCompactionProviderDisabled       = errors.New("compaction model provider is disabled")
	ErrCompactionOutputLimitUnsupported = errors.New("compaction model provider does not enforce the output limit")
	ErrCompactionWindowUnknown          = errors.New("compaction model does not declare a context window")
)

Compaction model resolution failures. Automatic triggers map them to a silent skip; manual surfaces translate them into user-facing errors.

View Source 
var (
	ErrModelIDAlreadyExists = errors.New("model_id already exists")
	ErrModelIDAmbiguous     = errors.New("model_id is ambiguous across providers")
)

Functions

func AnthropicThinkingBudget 

func AnthropicThinkingBudget(effort string, contextWindow int) int

AnthropicThinkingBudget is the legacy budget_tokens sent for an effort, fitted so the answer allowance plus the budget stays within half of the configured window but never below the minimum the API accepts; the model construction and the budget plan share it so budget_tokens always stays below the requested max_tokens.

func ApplyPromptCache 

func ApplyPromptCache(
	model *sdk.Model,
	ttl string,
	system string,
	messages []sdk.Message,
	tools []sdk.Tool,
) (string, []sdk.Message, []sdk.Tool)

ApplyPromptCache returns a request payload decorated with provider-specific prompt cache breakpoints. The dispatch is keyed off the resolved client type, so the call site does not need to know which vendor (if any) supports caching for the active model.

For models whose vendor does not implement caching, or when the requested TTL is "off", the inputs are returned unchanged.

func ApplyPromptCacheWithPlan 

func ApplyPromptCacheWithPlan(
	model *sdk.Model,
	ttl string,
	plan contextfrag.CachePlan,
	system string,
	messages []sdk.Message,
	tools []sdk.Tool,
) (string, []sdk.Message, []sdk.Tool, bool, int)

ApplyPromptCacheWithPlan decorates the provider request using the placement-derived cache plan: when the plan marks a stable leading message span, the final message of that span receives a cache breakpoint so the stable prefix is cached across turns. A zero plan preserves the legacy system-and-tools-only layout. The 4th return value reports whether this call prepended a system message to messages (Anthropic's system->message cache promotion), so callers can tell that apart from an unrelated leading system-role message already present in the input. The 5th return value is the honest count of leading messages actually covered by an applied message-level breakpoint: it matches plan.StableMessageCount unless placement had to fall back to an earlier message (or found none), so callers can keep their own bookkeeping of the cached prefix truthful instead of trusting a claim that was never actually cached.

func BuildReasoningOptions 

func BuildReasoningOptions(cfg SDKModelConfig) []sdk.GenerateOption

BuildReasoningOptions returns per-request SDK generation options for reasoning/thinking. It only ever sets an effort string (output_config.effort for Anthropic, reasoning.effort for OpenAI); the adaptive thinking flag is set at provider construction time in NewSDKChatModel. No token budgets are sent.

func DefaultProviderUserAgent 

func DefaultProviderUserAgent() string

DefaultProviderUserAgent is the project-level User-Agent for outbound model/provider traffic.

func EnforcesMaxOutputTokens 

func EnforcesMaxOutputTokens(clientType ClientType) bool

EnforcesMaxOutputTokens reports whether the client honors the configured maximum output token limit. Compaction relies on the cap to keep summaries inside their output reserve, so clients that ignore it are not eligible summarizers.

func FetchProviderByID 

func FetchProviderByID(ctx context.Context, queries dbstore.Queries, providerID string) (sqlc.Provider, error)

FetchProviderByID fetches a provider by ID.

func InferEmbeddingDimensions 

func InferEmbeddingDimensions(ctx context.Context, clientType, baseURL, apiKey, modelID string, timeout time.Duration, httpClient *http.Client) (int, error)

InferEmbeddingDimensions probes the embedding endpoint and returns the vector length produced by the provider for a minimal input.

func IsCompactionModelUnavailable 

func IsCompactionModelUnavailable(err error) bool

IsCompactionModelUnavailable reports whether the resolution failure means "this bot cannot compact right now" rather than an infrastructure error.

func IsImageOnlyChatModel 

func IsImageOnlyChatModel(model GetResponse, provider sqlc.Provider) bool

IsImageOnlyChatModel reports whether a chat-typed model is actually a dedicated image generator that cannot summarize text.

func IsLLMClientType 

func IsLLMClientType(clientType ClientType) bool

IsLLMClientType returns true if the client type belongs to the LLM domain (chat/embedding), excluding speech-only types (any type ending in "-speech").

func IsReasoningDisabled 

func IsReasoningDisabled(effort string) bool

IsReasoningDisabled reports whether an effort value means "no reasoning".

func IsValidClientType 

func IsValidClientType(clientType ClientType) bool

IsValidClientType returns true if the given client type is supported.

func IsValidModelType 

func IsValidModelType(modelType ModelType) bool

func IsValidReasoningEffort 

func IsValidReasoningEffort(effort string) bool

IsValidReasoningEffort reports whether effort can be stored in ModelConfig.

func LatestSessionModelID 

func LatestSessionModelID(ctx context.Context, queries dbstore.Queries, sessionID string) string

LatestSessionModelID returns the models.id UUID of the most recent history message in the session that recorded one, or "" when the session has no model-bearing history yet.

func NearestEffortToMedium 

func NearestEffortToMedium(levels []string) string

NearestEffortToMedium picks the tier closest to medium from levels, breaking ties toward the weaker tier.

func NewProviderHTTPClient 

func NewProviderHTTPClient(timeout time.Duration) *http.Client

NewProviderHTTPClient returns an HTTP client for model/provider traffic. When timeout is zero or negative, the caller is expected to enforce limits via context deadlines, which keeps streaming responses unbounded by the client's global timeout while still using the relaxed TLS handshake window.

func NewSDKChatModel 

func NewSDKChatModel(cfg SDKModelConfig) *sdk.Model

NewSDKChatModel builds a Twilight AI SDK Model from the resolved model config.

func NewSDKEmbeddingModel 

func NewSDKEmbeddingModel(clientType, baseURL, apiKey, modelID string, timeout time.Duration, httpClient *http.Client) *sdk.EmbeddingModel

NewSDKEmbeddingModel creates a Twilight AI SDK EmbeddingModel for the given provider configuration. It dispatches to the native Google embedding provider when clientType is "google-generative-ai", and falls back to the OpenAI-compatible /embeddings endpoint for all other provider types.

func NewSDKProvider 

func NewSDKProvider(baseURL, apiKey, codexAccountID string, clientType ClientType, timeout time.Duration, httpClient *http.Client) sdk.Provider

NewSDKProvider creates a Twilight AI SDK Provider for the given client type. It is exported so that other packages (e.g. providers) can reuse it for testing.

func NormalizeAdvertisedEfforts 

func NormalizeAdvertisedEfforts(efforts []string) []string

NormalizeAdvertisedEfforts forwards the catalog-boundary normalizer while callers migrate to the reasoning leaf package.

func NormalizePromptCacheTTL 

func NormalizePromptCacheTTL(s string) string

NormalizePromptCacheTTL coerces an arbitrary user-provided value to one of the accepted TTL constants. Empty or unrecognized values fall back to the recommended short-TTL default.

func PromptCacheKey added in v0.20.0 

func PromptCacheKey(model *sdk.Model, ttl, sessionID string) string

PromptCacheKey returns the per-session cache-routing key for OpenAI-family clients. prompt_cache_key groups requests that share a prefix lineage onto cache-warm backends, and the session is exactly that lineage; providers that cache via explicit breakpoints get no key.

func ResolveChatCompletionsCompat 

func ResolveChatCompletionsCompat(baseURL, compat string) string

ResolveChatCompletionsCompat returns the compatibility mode for a provider. An explicit config value always wins, including one that matches no known mode (e.g. "none"), which disables inference. With no explicit value, the mode is inferred from official endpoint origins so provider rows created before the config existed keep their protocol adaptations. Origins match by exact origin or path prefix (covering /v1, /beta, ...), never by substring, so lookalike domains and proxies that merely embed an official hostname are not classified.

func ResolveClientType 

func ResolveClientType(model *sdk.Model) string

ResolveClientType infers the client type string from an SDK Model's provider name.

func ResolveEnable 

func ResolveEnable(override *bool, current bool) bool

ResolveEnable returns the effective enable flag: when the override is nil, the current value is preserved; otherwise the override wins. Used by Service.Create (current=true default) and Service.UpdateByID (current=stored).

func ValidateCompatibilities 

func ValidateCompatibilities(compatibilities []string) error

ValidateCompatibilities validates capability tokens supplied by a client.

Types

type AddRequest 

type AddRequest struct {
	ModelID    string      `json:"model_id"`
	Name       string      `json:"name,omitempty"`
	ProviderID string      `json:"provider_id"`
	Type       ModelType   `json:"type"`
	Enable     *bool       `json:"enable,omitempty"`
	Config     ModelConfig `json:"config"`
}

AddRequest is the payload for creating a new model. Enable is a pointer so the server can default to true when the field is absent from the request.

type AddResponse 

type AddResponse struct {
	ID      string `json:"id"`
	ModelID string `json:"model_id"`
}

type ClientType 

type ClientType string
const (
	ClientTypeOpenAIResponses         ClientType = "openai-responses"
	ClientTypeOpenAICompletions       ClientType = "openai-completions"
	ClientTypeAnthropicMessages       ClientType = "anthropic-messages"
	ClientTypeGoogleGenerativeAI      ClientType = "google-generative-ai"
	ClientTypeOpenAICodex             ClientType = "openai-codex"
	ClientTypeGitHubCopilot           ClientType = "github-copilot"
	ClientTypeEdgeSpeech              ClientType = "edge-speech"
	ClientTypeOpenAISpeech            ClientType = "openai-speech"
	ClientTypeOpenAITranscription     ClientType = "openai-transcription"
	ClientTypeOpenRouterSpeech        ClientType = "openrouter-speech"
	ClientTypeOpenRouterTranscription ClientType = "openrouter-transcription"
	ClientTypeElevenLabsSpeech        ClientType = "elevenlabs-speech"
	ClientTypeElevenLabsTranscription ClientType = "elevenlabs-transcription"
	ClientTypeDeepgramSpeech          ClientType = "deepgram-speech"
	ClientTypeDeepgramTranscription   ClientType = "deepgram-transcription"
	ClientTypeMiniMaxSpeech           ClientType = "minimax-speech"
	ClientTypeVolcengineSpeech        ClientType = "volcengine-speech"
	ClientTypeAlibabaSpeech           ClientType = "alibabacloud-speech"
	ClientTypeAlibabaTranscription    ClientType = "alibabacloud-transcription"
	ClientTypeMicrosoftSpeech         ClientType = "microsoft-speech"
	ClientTypeGoogleSpeech            ClientType = "google-speech"
	ClientTypeGoogleTranscription     ClientType = "google-transcription"
	ClientTypeOpenRouterVideo         ClientType = "openrouter-video"
	ClientTypeModelArkVideo           ClientType = "modelark-video"
	ClientTypeVolcengineVideo         ClientType = "volcengine-video"
)

type CompactionModelResolution 

type CompactionModelResolution struct {
	Model    GetResponse
	Provider sqlc.Provider
	// WindowTokens is the summarizer model's declared context window.
	WindowTokens int
}

CompactionModelResolution is the resolved summarizer identity. Credentials stay with the caller: orchestration surfaces own auth context, the compaction engine only receives a completed contract.

func ResolveCompactionModel 

func ResolveCompactionModel(
	ctx context.Context,
	modelsService *Service,
	queries dbstore.Queries,
	candidates ...string,
) (CompactionModelResolution, error)

ResolveCompactionModel picks the first non-empty candidate model id and validates that it can act as a summarizer: a chat-type, enabled model on an enabled LLM provider that honors output caps and declares a context window (the summary budget derives from it, so an unknown window fails closed). Callers compose the candidate chain: automatic triggers pass the override and the turn's actually-resolved model; manual surfaces prefer the session's latest model before the bot default.

type CountResponse 

type CountResponse struct {
	Count int64 `json:"count"`
}

type DeleteRequest 

type DeleteRequest struct {
	ID      string `json:"id,omitempty"`
	ModelID string `json:"model_id,omitempty"`
}

type DeleteResponse 

type DeleteResponse struct {
	Message string `json:"message"`
}

type GenerationLimits 

type GenerationLimits struct {
	MaxOutputTokens int
	Requested       bool
	Resolution      string
}

GenerationLimits is the single authority for one turn's output allowance: the context budget plan reserves MaxOutputTokens and, when Requested, the provider request carries the same value as max_tokens. Unrequested limits are Memoh's reserve only; the provider keeps its own default because the model's real cap is unknown and an explicit value could be rejected.

func ResolveGenerationLimits 

func ResolveGenerationLimits(clientType ClientType, reasoning *ReasoningConfig, contextWindow int) GenerationLimits

ResolveGenerationLimits derives the output allowance from the client type, the resolved thinking decision, and the configured context window. Anthropic mirrors the SDK's own defaults, so the reserved and requested values cannot diverge; every other client is estimated without a request.

type GetRequest 

type GetRequest struct {
	ID string `json:"id"`
}

type GetResponse 

type GetResponse struct {
	ID      string `json:"id"`
	ModelID string `json:"model_id"`
	Model
	// Reasoning is the model's resolved thinking options, filled by the API layer
	// (it depends on the provider's client type). Clients render this rather than
	// deriving their own answer from ThinkingMode and ReasoningEfforts — the
	// duplication that let the web picker and the wire disagree.
	Reasoning *reasoning.Options `json:"reasoning,omitempty"`
}

func SelectMemoryModel 

func SelectMemoryModel(ctx context.Context, modelsService *Service, queries dbstore.Queries) (GetResponse, sqlc.Provider, error)

SelectMemoryModel selects a chat model for memory operations. It only considers models from enabled providers.

func SelectMemoryModelForBot 

func SelectMemoryModelForBot(ctx context.Context, modelsService *Service, queries dbstore.Queries, chatModelID string) (GetResponse, sqlc.Provider, error)

SelectMemoryModelForBot selects a chat model for memory operations. If botID is provided, it attempts to use the bot's configured chat model first, falling back to the first enabled chat model globally. Models or providers that the user has disabled are skipped so memory extraction/decision/compact never quietly run on a row hidden from the UI.

type ListRequest 

type ListRequest struct {
	Type ModelType `json:"type,omitempty"`
}

type Model 

type Model struct {
	ModelID    string      `json:"model_id"`
	Name       string      `json:"name"`
	ProviderID string      `json:"provider_id"`
	Type       ModelType   `json:"type"`
	Enable     bool        `json:"enable"`
	Config     ModelConfig `json:"config"`
}

func (*Model) HasCompatibility 

func (m *Model) HasCompatibility(c string) bool

HasCompatibility checks whether the model config includes the given capability.

func (*Model) ReasoningOptions 

func (m *Model) ReasoningOptions(clientType string) reasoning.Options

ReasoningOptions reports what a caller may select for this model on the given client type: the selectable tiers, whether off is reachable, and the default. It is the single source every surface reads — the web picker, /reasoning, and the API all render this rather than deriving their own answer.

func (*Model) ResolveThinkingMode 

func (m *Model) ResolveThinkingMode() string

ResolveThinkingMode returns the effective ThinkingMode, bridging legacy data: unknown + reasoning compat → toggle; unknown without it → none.

func (*Model) Validate 

func (m *Model) Validate() error

type ModelConfig 

type ModelConfig struct {
	Description      *string  `json:"description,omitempty"`
	Dimensions       *int     `json:"dimensions,omitempty"`
	Compatibilities  []string `json:"compatibilities,omitempty"`
	ContextWindow    *int     `json:"context_window,omitempty"`
	ReasoningEfforts []string `json:"reasoning_efforts,omitempty"`
	ThinkingMode     string   `json:"thinking_mode,omitempty"`
	CatalogAvailable *bool    `json:"catalog_available,omitempty"`
	// ReasoningDialect declares the wire shape of this model's thinking control,
	// which cannot be inferred from the tiers it advertises: Gemini 2.5 takes a
	// token budget while 3.x takes a named level, and the two are mutually
	// exclusive on the same request. Declared per model because the alternative is
	// sniffing the model id, and an id is not a capability. Empty leaves provider
	// policy in charge; Google's adaptor deliberately sends no thinking control so
	// pre-dialect rows retain their safe pre-upgrade request shape.
	ReasoningDialect string `json:"reasoning_dialect,omitempty"`
	// ReasoningOffSupport declares how the model answers an explicit request to
	// stop thinking. Anthropic's per-model table splits models that share a
	// thinking mode and an identical tier list, so this cannot be derived — see the
	// reasoning package's OffSupport constants.
	ReasoningOffSupport string `json:"reasoning_off_support,omitempty"`
	// ReasoningDefaultOn reports whether omitting the thinking field leaves the
	// model thinking. Separate from off-ability: Claude 4.6 can be turned off *and*
	// defaults to off, while Opus 5 can be turned off but defaults to on, so
	// omitting the field there keeps thinking running — billed, and invisible to a
	// user who believes they turned it off. nil means unknown.
	ReasoningDefaultOn *bool `json:"reasoning_default_on,omitempty"`
	// ThinkingBudgetMin/Max bound the budget dialect. The range is per model
	// family, not per vendor: Gemini 2.5 Pro is 128..32768 and cannot be turned
	// off, while Flash starts at 0 and can.
	ThinkingBudgetMin *int `json:"thinking_budget_min,omitempty"`
	ThinkingBudgetMax *int `json:"thinking_budget_max,omitempty"`
}

ModelConfig holds the JSONB config stored per model.

ReasoningEfforts is the model's effort-level list (a.k.a. effort_levels in the design doc); the JSON key stays "reasoning_efforts" for backward compatibility. ThinkingMode is the discovered thinking behavior; empty = unknown (legacy data), resolved via SupportsReasoning / ResolveThinkingMode.

func (ModelConfig) ContextBudgetMaxTokens 

func (c ModelConfig) ContextBudgetMaxTokens() int

ContextBudgetMaxTokens returns the configured model context window, or zero when budget enforcement is unavailable for this model.

type ModelType 

type ModelType string
const (
	ModelTypeChat          ModelType = "chat"
	ModelTypeEmbedding     ModelType = "embedding"
	ModelTypeSpeech        ModelType = "speech"
	ModelTypeTranscription ModelType = "transcription"
	ModelTypeVideo         ModelType = "video"
)

type ReasoningConfig 

type ReasoningConfig = reasoning.Config

ReasoningConfig is the resolved extended-thinking decision for one call, produced by internal/reasoning and translated here into each provider's wire shape. Anthropic 4.6+ (Adaptive) sends thinking{type:"adaptive"} plus an effort string and never a token budget; legacy Anthropic (<=4.5, non-adaptive) sends thinking{type:"enabled", budget_tokens:N} derived from the effort. OpenAI-style providers only ever receive an effort string.

type SDKModelConfig 

type SDKModelConfig struct {
	ModelID        string
	ClientType     string
	APIKey         string //nolint:gosec // carries provider credential material at runtime
	CodexAccountID string
	BaseURL        string
	// ChatCompletionsCompat selects narrow compatibility behavior for
	// OpenAI-compatible /chat/completions backends.
	ChatCompletionsCompat string
	HTTPClient            *http.Client
	ReasoningConfig       *ReasoningConfig
	// ReasoningDialect and the budget bounds come from the model's catalog entry.
	// They say how this model spells its thinking control, which is not derivable
	// from the tiers it advertises.
	ReasoningDialect  string
	ThinkingBudgetMin *int
	ThinkingBudgetMax *int
	// ReasoningOffSupport declares whether this model accepts an explicit
	// thinking{type:"disabled"}. See anthropicAcceptsExplicitOff.
	ReasoningOffSupport string
	// ReasoningDefaultOn reports whether omitting the thinking field leaves the
	// model thinking. nil means unknown.
	ReasoningDefaultOn *bool
	// ContextWindow is the configured context window the turn budgets against;
	// legacy Anthropic thinking budgets are fitted to it. Zero means unknown.
	ContextWindow int
}

SDKModelConfig holds provider and model information resolved from DB, used to construct a Twilight AI SDK Model instance.

type Service 

type Service struct {
	// contains filtered or unexported fields
}

Service provides CRUD operations for models.

func NewService 

func NewService(log *slog.Logger, queries dbstore.Queries) *Service

NewService creates a new models service.

func (*Service) Count 

func (s *Service) Count(ctx context.Context) (int64, error)

Count returns the total number of models.

func (*Service) CountByType 

func (s *Service) CountByType(ctx context.Context, modelType ModelType) (int64, error)

CountByType returns the number of models of a specific type.

func (*Service) Create 

func (s *Service) Create(ctx context.Context, req AddRequest) (AddResponse, error)

Create adds a new model to the database.

func (*Service) DeleteByID 

func (s *Service) DeleteByID(ctx context.Context, id string) error

DeleteByID deletes a model by its internal UUID.

func (*Service) DeleteByModelID 

func (s *Service) DeleteByModelID(ctx context.Context, modelID string) error

DeleteByModelID deletes a model by its model_id field.

func (*Service) GetByID 

func (s *Service) GetByID(ctx context.Context, id string) (GetResponse, error)

GetByID retrieves a model by its internal UUID.

func (*Service) GetByModelID 

func (s *Service) GetByModelID(ctx context.Context, modelID string) (GetResponse, error)

GetByModelID retrieves a model by its model_id field.

func (*Service) GetByProviderAndModelID 

func (s *Service) GetByProviderAndModelID(ctx context.Context, providerID, modelID string) (GetResponse, error)

GetByProviderAndModelID retrieves a model by provider and model_id.

func (*Service) List 

func (s *Service) List(ctx context.Context) ([]GetResponse, error)

List returns all models.

func (*Service) ListByProviderClientType 

func (s *Service) ListByProviderClientType(ctx context.Context, clientType ClientType) ([]GetResponse, error)

ListByProviderClientType returns models whose provider has the given client_type.

func (*Service) ListByProviderID 

func (s *Service) ListByProviderID(ctx context.Context, providerID string) ([]GetResponse, error)

ListByProviderID returns models filtered by provider ID.

func (*Service) ListByProviderIDAndType 

func (s *Service) ListByProviderIDAndType(ctx context.Context, providerID string, modelType ModelType) ([]GetResponse, error)

ListByProviderIDAndType returns models filtered by provider ID and type.

func (*Service) ListByType 

func (s *Service) ListByType(ctx context.Context, modelType ModelType) ([]GetResponse, error)

ListByType returns models filtered by type.

func (*Service) ListEnabled 

func (s *Service) ListEnabled(ctx context.Context) ([]GetResponse, error)

ListEnabled returns all models from enabled providers.

func (*Service) ListEnabledByProviderClientType 

func (s *Service) ListEnabledByProviderClientType(ctx context.Context, clientType ClientType) ([]GetResponse, error)

ListEnabledByProviderClientType returns models from enabled providers with the given client_type.

func (*Service) ListEnabledByType 

func (s *Service) ListEnabledByType(ctx context.Context, modelType ModelType) ([]GetResponse, error)

ListEnabledByType returns models from enabled providers filtered by type.

func (*Service) ResolveReasoningOptions 

func (s *Service) ResolveReasoningOptions(ctx context.Context, id string) (reasoning.Options, error)

ResolveReasoningOptions returns the selectable reasoning contract for a model using the client type of its persisted provider. Keeping provider lookup here gives commands and settings writes the same capability answer.

func (*Service) Test 

func (s *Service) Test(ctx context.Context, id string) (TestResponse, error)

Test probes a model's provider endpoint using the Twilight AI SDK to verify connectivity, authentication, and model availability.

func (*Service) UpdateByID 

func (s *Service) UpdateByID(ctx context.Context, id string, req UpdateRequest) (GetResponse, error)

UpdateByID updates a model by its internal UUID.

func (*Service) UpdateByModelID 

func (s *Service) UpdateByModelID(ctx context.Context, modelID string, req UpdateRequest) (GetResponse, error)

UpdateByModelID updates a model by its model_id field.

func (*Service) UpdateByProviderAndModelID 

func (s *Service) UpdateByProviderAndModelID(ctx context.Context, providerID, modelID string, req UpdateRequest) (GetResponse, error)

UpdateByProviderAndModelID updates a model within one provider namespace.

type TestResponse 

type TestResponse struct {
	Status    TestStatus `json:"status"`
	Reachable bool       `json:"reachable"`
	LatencyMs int64      `json:"latency_ms,omitempty"`
	Message   string     `json:"message,omitempty"`
}

TestResponse is returned by POST /models/:id/test.

type TestStatus 

type TestStatus string

TestStatus represents the outcome of probing a model. 

const (
	TestStatusOK                TestStatus = "ok"
	TestStatusAuthError         TestStatus = "auth_error"
	TestStatusModelNotSupported TestStatus = "model_not_supported"
	TestStatusError             TestStatus = "error"
)

type UpdateRequest 

type UpdateRequest struct {
	ModelID    string      `json:"model_id"`
	Name       string      `json:"name,omitempty"`
	ProviderID string      `json:"provider_id"`
	Type       ModelType   `json:"type"`
	Enable     *bool       `json:"enable,omitempty"`
	Config     ModelConfig `json:"config"`
}

UpdateRequest is the payload for updating an existing model. Enable is a pointer so callers can omit it to preserve the current enable state while still rewriting the other fields.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.