Documentation
¶
Overview ¶
Package profile declares how to reach a model: endpoint, credential, wire dialect, capabilities, and the models served.
A model is named by (profile, model id), not by a model id alone. The same id is served by several backends at several prices with several credentials — gpt-oss-20b by Vertex AI and by an operator's vLLM — so the id cannot pick the transport, the credential or the price, and a profile does (docs/design.md §5).
This package owns the schema and the built-in profiles. Where an operator's profiles are read from is each product's decision; the struct carries json and yaml tags so either format decodes into it, and DecodeJSON is the strict JSON reader.
A profile is checked at two moments, and both fail before any request is sent (R1): Validate checks its shape, and Resolve checks it against the environment — variables set, credentials found.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ReservedBodyKeys = []string{
"model", "messages", "tools", "tool_choice", "stream", "stream_options",
"max_tokens", "max_completion_tokens", "temperature", "top_p", "stop",
"seed", "presence_penalty", "frequency_penalty", "response_format", "n",
}
ReservedBodyKeys are the request fields an adapter sets itself, which ExtraBody may not override.
Functions ¶
Types ¶
type Capabilities ¶
type Capabilities struct {
// ResponseSchema: the server enforces a JSON-schema response format.
// Many self-hosted servers accept the field and ignore it unless a
// guided-decoding backend is enabled — declare it only when it is.
ResponseSchema *bool `json:"response_schema,omitempty" yaml:"response_schema,omitempty"`
// ReasoningEcho: reasoning from one turn is carried into the next.
ReasoningEcho *bool `json:"reasoning_echo,omitempty" yaml:"reasoning_echo,omitempty"`
// ParallelToolCalls: the model may call several tools in one turn.
ParallelToolCalls *bool `json:"parallel_tool_calls,omitempty" yaml:"parallel_tool_calls,omitempty"`
// ServerTools: the provider runs built-in tools (search, code).
ServerTools *bool `json:"server_tools,omitempty" yaml:"server_tools,omitempty"`
// Streaming: the server streams responses.
Streaming *bool `json:"streaming,omitempty" yaml:"streaming,omitempty"`
// ForcedToolChoice: the model accepts a forced tool call — "required",
// or one named tool. Unset means it does. Declared false (gpt-oss on
// Vertex AI, per Google's function-calling notes), a forced choice is
// sent as "auto" instead and the response is marked; the alternative
// is a request the server rejects or silently ignores.
ForcedToolChoice *bool `json:"forced_tool_choice,omitempty" yaml:"forced_tool_choice,omitempty"`
}
Capabilities are declared, never probed. Nil means not declared, which a product reads as absent.
type Dialect ¶
type Dialect string
Dialect is the wire format a profile's server speaks.
const ( // OpenAIChat is POST /v1/chat/completions: Vertex AI partner models, // xAI, vLLM, SGLang, Ollama, llama.cpp, NIM and the managed long // tail. OpenAIChat Dialect = "openai-chat" // OpenAIResponses is POST /v1/responses: OpenAI first-party, xAI. OpenAIResponses Dialect = "openai-responses" // Anthropic is the Messages API: first-party, Vertex, Bedrock. Anthropic Dialect = "anthropic" // Gemini is genai's generateContent: the Developer API and Vertex. Gemini Dialect = "gemini" )
type Model ¶
type Model struct {
ID string `json:"id" yaml:"id"`
Tier Tier `json:"tier,omitempty" yaml:"tier,omitempty"`
ContextWindow int `json:"context_window,omitempty" yaml:"context_window,omitempty"`
// Capabilities override the profile's for this model only.
Capabilities Capabilities `json:"capabilities,omitzero" yaml:"capabilities,omitempty"`
// ExtraBody adds vendor fields to this model's requests, over the
// profile's ExtraBody.
ExtraBody map[string]any `json:"extra_body,omitempty" yaml:"extra_body,omitempty"`
}
Model is one model a profile serves.
type Options ¶
type Options struct {
// Getenv replaces os.Getenv, for ${VAR} expansion and credentials.
Getenv func(string) string
// Auth is passed to auth.Config.Resolve.
Auth auth.Options
}
Options adjusts Resolve.
type Profile ¶
type Profile struct {
// Name identifies the profile: what --provider names and what a
// product's config refers to. Lowercase letters, digits and '-'.
Name string `json:"name" yaml:"name"`
// Extends names a built-in profile this one starts from; every field
// set here overrides the built-in's.
Extends string `json:"extends,omitempty" yaml:"extends,omitempty"`
// Dialect is the server's wire format.
Dialect Dialect `json:"dialect,omitempty" yaml:"dialect,omitempty"`
// BaseURL is the API root. It may contain {param} placeholders,
// filled from Params.
BaseURL string `json:"base_url,omitempty" yaml:"base_url,omitempty"`
// Params fill BaseURL's placeholders. A value may be ${VAR} or
// ${VAR:-default}, read from the environment at Resolve.
Params map[string]string `json:"params,omitempty" yaml:"params,omitempty"`
// Auth is the credential.
Auth auth.Config `json:"auth,omitzero" yaml:"auth,omitempty"`
// Backend is the identity prices are keyed on, beside the model id.
// Defaults to Name.
Backend string `json:"backend,omitempty" yaml:"backend,omitempty"`
// OpenModels admits any model id, not only those in Models: a
// self-hosted server serves whatever weights the operator loaded.
OpenModels *bool `json:"open_models,omitempty" yaml:"open_models,omitempty"`
// Capabilities are what the server honors. Products refuse at
// startup a workload that needs one the profile lacks.
Capabilities Capabilities `json:"capabilities,omitzero" yaml:"capabilities,omitempty"`
// Usage overrides how reported usage fields are trusted.
Usage UsageOverrides `json:"usage,omitzero" yaml:"usage,omitempty"`
// ReasoningFormat says where the server puts reasoning.
ReasoningFormat ReasoningFormat `json:"reasoning_format,omitempty" yaml:"reasoning_format,omitempty"`
// ExtraBody adds vendor-specific fields to every request: switches a
// server takes that the dialect does not model, such as
// chat_template_kwargs {enable_thinking: true} for Gemma 4 on vLLM,
// reasoning_effort, or top_k. A field the adapter sets itself (model,
// messages, tools, …) cannot be set here.
ExtraBody map[string]any `json:"extra_body,omitempty" yaml:"extra_body,omitempty"`
// MetricsURL is the server's Prometheus endpoint, for the opt-in
// KV-cache sampler (docs/design.md §8).
MetricsURL string `json:"metrics_url,omitempty" yaml:"metrics_url,omitempty"`
// Models lists the models the profile serves.
Models []Model `json:"models,omitempty" yaml:"models,omitempty"`
// Tiers names the model for each tier this profile can fill.
Tiers map[Tier]string `json:"tiers,omitempty" yaml:"tiers,omitempty"`
// contains filtered or unexported fields
}
Profile is one declared way to reach models.
func DecodeJSON ¶
DecodeJSON reads a JSON array of profiles, refusing unknown fields: a misspelled key ("base_ulr") is an error, not a silently missing value.
func Expand ¶
Expand applies Extends: the result is the named built-in with every field p sets laid over it. Params and Tiers merge key by key; Models and Auth replace as a whole; each capability overrides on its own. A profile without Extends is returned unchanged.
func Find ¶
Find returns the profile named name: one of declared if any has that name, else a built-in. A name declared twice is an error, since the second would silently shadow the first.
func (Profile) BackendName ¶
BackendName is Backend, defaulting to Name.
func (Profile) CapabilitiesFor ¶ added in v0.2.0
func (p Profile) CapabilitiesFor(id string) Capabilities
CapabilitiesFor is the profile's capabilities with the listed model's own overrides laid over them. One profile serves many models, and what a server honors can differ between two models it serves (Kimi wants reasoning echoed, gpt-oss refuses a forced tool choice), so a capability is declared per profile and refined per model.
func (Profile) DefaultSmallModel ¶
DefaultSmallModel is the profile's small-tier model, or "".
func (Profile) ExtraBodyFor ¶ added in v0.4.0
ExtraBodyFor is the profile's ExtraBody with the listed model's laid over it, key by key. Nil when neither sets any.
type ReasoningFormat ¶
type ReasoningFormat string
ReasoningFormat says where a server puts a model's reasoning.
const ( // ReasoningField (the default): in reasoning_content or reasoning, // apart from the answer, or not at all. ReasoningField ReasoningFormat = "" // ThinkTags: inline at the start of the answer as <think>…</think> // (Ollama; vLLM and SGLang without a reasoning parser). The adapter // splits the block out, so it never reaches the product as answer // text. ThinkTags ReasoningFormat = "think_tags" )
type Reliability ¶
type Reliability string
Reliability says whether a usage field a server reports can be trusted.
const ( // Reported (the default) records the field when the server sends it. Reported Reliability = "" // Unreliable records the field as not reported even when the server // sends it — for a server build known to send nonsense, such as the // vLLM V1 builds that null or zero prompt_tokens_details. Unreliable Reliability = "unreliable" )
type Resolved ¶
type Resolved struct {
Profile Profile
// Params are the profile's params with every ${VAR} read and the
// derived ones (vertex-maas's {host}) added.
Params map[string]string
BaseURL string
MetricsURL string
Credential *auth.Credential
}
Resolved is a profile checked against the environment: every placeholder filled, every variable read, the credential found.
type UsageOverrides ¶
type UsageOverrides struct {
CachedTokens Reliability `json:"cached_tokens,omitempty" yaml:"cached_tokens,omitempty"`
}
UsageOverrides adjust how reported usage is trusted.