provider

package
v0.0.92 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 3, 2026 License: MIT Imports: 41 Imported by: 0

Documentation

Overview

Package provider — Anthropic prompt-cache breakpoint helpers.

caching.go stamps Anthropic prompt-cache markers on a request before it goes on the wire. The cache is opt-in and prefix-based: a cache_control marker on a block makes the API cache everything up to and including that block, and a later request with a byte-identical prefix reads it back at ~10% of the input price. Setting a marker is always safe: prefixes shorter than the model's minimum cacheable length (1024–4096 tokens) are silently not cached, with no error and no extra cost.

Anthropic caches in the order tools → system → messages and allows at most 4 breakpoints per request. We set 3:

  1. The last tool definition. Tool schemas are static per agent, so this prefix survives every request of a session.

  2. The last system block. Static too, and separate from (1) so editing the prompt (e.g. a hot reload) does not invalidate the tools cache.

  3. The last cacheable block of the last message. Turn N's history becomes turn N+1's cached prefix; this is where the savings come from in agentic loops, which re-send the whole history on every tool round-trip. Thinking and redacted-thinking blocks cannot carry a marker, so this one walks backwards past them to the nearest eligible block.

Call applyAnthropicCacheControl as the LAST step of wire construction so every section is final.

Package provider — provider-neutral extension point for prompt caching.

cache_apply.go provides the shared helper that any provider's wire builder may call to check whether the user has opted out of prompt caching. Today only the Anthropic provider honors it; the openai, gemini, mistral, xai, ollama, and azure builders ignore the option entirely.

Index

Constants

View Source
const DefaultAzureAPIVersion = "2025-04-01-preview"

DefaultAzureAPIVersion is the api-version used when neither the caller nor OPENAI_API_VERSION supplies one.

Variables

View Source
var KnownModels = mustLoadKnownModels()

KnownModels lists recognized model names per provider. The check is prefix-based: a model is valid if it starts with any entry. Ollama models are not validated here (they are dynamic).

OpenAI and Anthropic IDs are loaded from embedded llm-prices snapshots under modeldata/. Gemini and Mistral IDs are maintained in model_catalog.go. Update context-window metadata in modeldata/context-windows.json in the same change when official limits change.

View Source
var KnownProviderPrefixes = []string{
	"agentgateway/",
	"anthropic/",
	"openai/",
	"gemini/",
	"google/",
	"mistral/",
	"xai/",
	"grok/",
	"ollama/",
	"ollama1/",
	"ollama2/",
	"ollama3/",
	"ollama-cloud/",
	"azure/",
	"opencode/",
	"openrouter/",
}

KnownProviderPrefixes lists known vendor and gateway prefixes that may wrap model names in layered routing configurations (e.g. agentgateway/openai/...).

View Source
var OllamaModelPrefixes = []string{}

OllamaModelPrefixes are model prefixes that previously auto-routed to Ollama. Bare Ollama model names are intentionally not auto-detected; use the explicit ollama/ prefix, or the :cloud tag for Ollama cloud models.

Functions

func APIKeyEnvVar added in v0.0.72

func APIKeyEnvVar(providerName string) string

APIKeyEnvVar returns the environment variable a provider's key is read from.

Kept here rather than in a front-end so every caller — the CLI, the public pimodels package, ping — agrees on where a key comes from. A second copy of this mapping is how "works in the CLI, not when embedded" bugs start.

func APIKeyFromEnv added in v0.0.72

func APIKeyFromEnv(providerName string) string

APIKeyFromEnv returns the configured key for a provider, or "" when unset. Providers that need no key (ollama on localhost) are fine with the empty string; the provider constructors decide, not this helper.

func AzureAPIKey added in v0.0.52

func AzureAPIKey(apiKey string) string

AzureAPIKey resolves the Azure credential from an explicit value, then the three accepted environment variables, in the order NewAzureOpenAI uses.

Exported so `pi ping` authenticates exactly as a real run does rather than reimplementing the fallback chain and disagreeing about which var wins.

func AzureAPIVersion added in v0.0.52

func AzureAPIVersion(apiVersion string) string

AzureAPIVersion resolves the api-version, falling back to OPENAI_API_VERSION and then DefaultAzureAPIVersion.

func AzureEndpoint added in v0.0.52

func AzureEndpoint(endpoint string) string

AzureEndpoint resolves the Azure resource endpoint, falling back to AZURE_OPENAI_ENDPOINT.

func AzureProbePaths added in v0.0.52

func AzureProbePaths(deployment, apiVersion, endpoint string) []string

AzureProbePaths returns the paths `pi ping` should GET for an Azure endpoint, most informative first, built from the same rules NewAzureOpenAI applies to real traffic. The caller tries them in order and keeps the first that answers 2xx.

There are two because neither alone is both reliable and meaningful:

  • /openai/models?api-version=… is the data-plane list route. A 200 proves the resource is reachable AND that the Api-Key is accepted, which is the part worth knowing. It does not name the deployment.
  • /openai/deployments/{deployment}?api-version=… names the deployment, but it is the legacy Deployments_Get route and current api-versions may not serve it at all. It is the fallback, not the primary.

Neither one establishes that the deployment can serve a completion — only the model ping that follows does. Pinging the bare host, which is what ping did before, proved nothing beyond a TLS handshake to Azure's front door.

A compat proxy gets /models alone, since deployment paths and api-version break routing there for the same reason they are skipped on real requests.

func BackendName added in v0.0.80

func BackendName(info Info, apiKey, baseURL string) string

BackendName returns a safe description of the selected request backend. It intentionally exposes no credential material.

func BuildHTTPClient added in v0.0.9

func BuildHTTPClient(opts *LLMOptions, timeout time.Duration) (*http.Client, error)

BuildHTTPClient creates an *http.Client with the TLS trust, extra headers, connect timeout and request timeout from opts. Returns a client with the default transport when opts asks for no customization.

func BuildTransport added in v0.0.9

func BuildTransport(opts *LLMOptions) (http.RoundTripper, error)

BuildTransport creates an http.RoundTripper carrying the TLS trust, connect timeout and extra headers from opts. It returns (nil, nil) when opts asks for no customization, so callers can leave the SDK's own client in place.

Every variant starts from a clone of http.DefaultTransport rather than a fresh &http.Transport{}: the default carries ProxyFromEnvironment, connection pooling and HTTP/2, and building from scratch silently drops HTTPS_PROXY — which is exactly the environment where a custom CA or a TLS skip is needed in the first place.

func CatalogFor added in v0.0.83

func CatalogFor(provider string) []string

CatalogFor returns known model prefixes for a provider: XDG cache first (if present), else the embedded modeldata/models-<provider>.json snapshot merged with the hard-coded KnownModels list, else the hard-coded list alone.

The merge is a union: the hard-coded list carries curated compatibility entries (e.g. bare "codestral", "pixtral", "ministral") that the live catalog does not, and the live catalog adds dated models the hard-coded list predates. A union is a superset, so nothing that validated before stops validating.

func CheckOllama added in v0.0.4

func CheckOllama(baseURL string) error

CheckOllama verifies that the Ollama server at baseURL is reachable. It first checks TCP connectivity on the port, then issues a GET to the root endpoint (Ollama returns "Ollama is running").

func ContextWindowSize added in v0.0.18

func ContextWindowSize(modelName string) int64

ContextWindowSize returns the context window size for a model (in tokens). Returns 0 if the model is unknown.

This searches every vendor except Azure. Prefer ContextWindowSizeFor when the provider is known — an Azure deployment shares its name with an OpenAI model but not necessarily its window, so only the provider-aware lookup separates them.

func ContextWindowSizeFor added in v0.0.52

func ContextWindowSizeFor(providerName, modelName string) int64

ContextWindowSizeFor returns the context window for a model as served by a specific provider, falling back to the vendor-agnostic table when that provider publishes no window of its own.

The fallback is what keeps a newly provisioned Azure deployment usable when an OpenAI entry matches the name it was provisioned from — that beats returning 0, since zero disables auto-compaction and lets the session grow unchecked until the API rejects it. A name that matches nothing anywhere still returns 0; there is no window to guess.

func IsAzureCompatProxyEndpoint added in v0.0.52

func IsAzureCompatProxyEndpoint(endpoint string) bool

IsAzureCompatProxyEndpoint reports whether the endpoint is an OpenAI-compatible gateway rather than a native Azure resource, in which case deployment-path rewriting and the api-version parameter must be skipped.

func IsOllamaCloudEndpoint added in v0.0.75

func IsOllamaCloudEndpoint(baseURL string) bool

IsOllamaCloudEndpoint reports whether an already-resolved endpoint is ollama.com's hosted API.

Callers need this to tell a reachability problem from a credential problem: the TCP-and-GET health check in CheckOllama is a statement about a daemon on a host someone controls, and running it against api.ollama.com turns a missing OLLAMA_API_KEY into a misleading "ollama not reachable".

It matches on host, not on a string compare with ollamaCloudURL, so an OLLAMA_HOST pointed at the cloud API by hand is recognized too.

func IsOllamaCloudModel added in v0.0.75

func IsOllamaCloudModel(modelName string) bool

IsOllamaCloudModel reports whether a model name is tagged for ollama.com's hosted service. Ollama publishes both forms — a bare ":cloud" and the ":<size>-cloud" that most of the catalog uses — so a check for one alone silently misses the other.

This is the single fact that decides local versus cloud, and it is the model's name that decides it. Nothing about the caller's environment enters into it.

func ModelReleaseDate added in v0.0.92

func ModelReleaseDate(providerName, modelName string) string

ModelReleaseDate returns the models.dev release date for a model, or "" when the provider or model is unknown. It uses the same prefix matching as CostFor.

func ModelTextOutput added in v0.0.92

func ModelTextOutput(providerName, modelName string) (text bool, ok bool)

ModelTextOutput reports whether a model can emit text output, per the models.dev catalog. ok is false when the provider or model is unknown.

func NewAgentGateway added in v0.0.87

func NewAgentGateway(ctx context.Context, modelName, apiKey, baseURL string, opts *LLMOptions) (model.LLM, error)

NewAgentGateway creates an agentgateway model.LLM. agentgateway is an OpenAI-compatible local gateway, so the client is the OpenAI one pointed at the gateway's base URL. No API key is required: NewOpenAI substitutes a dummy key whenever a base URL is set, and a gateway that needs one can still be given AGENTGATEWAY_API_KEY.

func NewAnthropic

func NewAnthropic(_ context.Context, modelName, apiKey, baseURL, thinkingLevel string, llmOpts *LLMOptions) (model.LLM, error)

NewAnthropic creates an Anthropic model.LLM. If baseURL is non-empty, it overrides the default API endpoint. When baseURL is set, the API key is optional (for Ollama compatibility). thinkingLevel controls extended thinking: "none", "low", "medium", "high". llmOpts.AdvisorModel enables the advisor tool (beta). See specs/features/LLM/001-claude-models/advisor-tool.md

func NewAzureOpenAI added in v0.0.18

func NewAzureOpenAI(_ context.Context, deploymentName, apiKey, endpoint, apiVersion string, llmOpts *LLMOptions) (model.LLM, error)

NewAzureOpenAI creates an Azure OpenAI model.LLM. It uses AZURE_OPENAI_API_KEY, AZURE_OPENAI_ENDPOINT, and OPENAI_API_VERSION (defaults to 2025-04-01-preview). The deploymentName is the Azure deployment name (not the model ID).

func NewGemini

func NewGemini(ctx context.Context, modelName, baseURL string, opts *LLMOptions) (model.LLM, error)

NewGemini creates a Gemini model.LLM using ADK Go's native Gemini support. It reads the API key from GEMINI_API_KEY or GOOGLE_API_KEY env vars. If neither is set, it falls back to Application Default Credentials. If baseURL is non-empty, it overrides the default API endpoint.

func NewLLM

func NewLLM(ctx context.Context, info Info, apiKey, baseURL, thinkingLevel string, opts *LLMOptions) (model.LLM, error)

NewLLM creates a model.LLM for the given provider info, API key, optional base URL, thinking level, and options.

func NewMistral added in v0.0.18

func NewMistral(_ context.Context, modelName, apiKey, baseURL, thinkingLevel string, llmOpts *LLMOptions) (model.LLM, error)

NewMistral creates a Mistral model.LLM. If baseURL is empty, the default Mistral API endpoint is used. thinkingLevel controls reasoning: "none", "low", "medium", "high", "max". It reaches the wire as `reasoning_effort` on the models that accept it and is omitted everywhere else; empty leaves the model's default in force.

func NewOllama added in v0.0.7

func NewOllama(_ context.Context, r OllamaRouting, thinkingLevel string, opts *LLMOptions) (model.LLM, error)

NewOllama creates an Ollama model.LLM using the native Ollama Go client. The server is chosen by ResolveOllamaEndpoint from the routing facts in r. thinkingLevel controls extended thinking: "none", "low", "medium", "high".

func NewOpenAI

func NewOpenAI(_ context.Context, modelName, apiKey, baseURL string, llmOpts *LLMOptions) (model.LLM, error)

NewOpenAI creates an OpenAI model.LLM. If baseURL is non-empty, it overrides the default API endpoint. When apiKey is recognized as a codex ChatGPT OAuth token and no explicit baseURL is provided, the client is pointed at the ChatGPT backend (/codex/responses) and the `chatgpt-account-id` + `originator` headers pi-mono sends are injected — the platform /v1/responses endpoint rejects codex tokens with 401 "Missing scopes: api.responses.write".

func NewOpenCode added in v0.0.47

func NewOpenCode(ctx context.Context, modelName, apiKey, baseURL, thinkingLevel string, opts *LLMOptions) (model.LLM, error)

NewOpenCode creates an OpenCode Go model.LLM, routing the model to the client that serves its API family based on the hardcoded catalog.

  • "chat" and "responses" → NewOpenAI (OpenAI-compatible; NewOpenAI picks the chat/completions vs responses endpoint internally from the model ID)
  • "messages" → NewAnthropic (Anthropic messages, x-api-key)

Unknown models return an error. The opencode/ prefix must already be stripped.

func NewOpenRouter added in v0.0.75

func NewOpenRouter(_ context.Context, modelName, apiKey, baseURL, thinkingLevel string, llmOpts *LLMOptions) (model.LLM, error)

NewOpenRouter creates an OpenRouter model.LLM. If baseURL is empty, the default OpenRouter API endpoint is used. thinkingLevel maps to OpenRouter's unified `reasoning.effort` request parameter; empty or "none" leaves reasoning at the model's default. See https://openrouter.ai/docs/app-attribution for the attribution headers sent on every request.

func NewXAI added in v0.0.71

func NewXAI(_ context.Context, modelName, apiKey, baseURL, thinkingLevel string, llmOpts *LLMOptions) (model.LLM, error)

NewXAI creates an xAI model.LLM. If baseURL is empty, the default xAI API endpoint is used. thinkingLevel controls reasoning effort: "none", "low", "medium", "high", "max".

func OllamaContextWindowSize added in v0.0.34

func OllamaContextWindowSize(ctx context.Context, baseURL, modelName string) int64

OllamaContextWindowSize queries the Ollama server for the context window size of the given model. Returns 0 if the size cannot be determined.

func OllamaListModels added in v0.0.7

func OllamaListModels(ctx context.Context, baseURL string) ([]string, error)

OllamaListModels lists available models from the Ollama server.

func OpenRouterContextWindowSize added in v0.0.76

func OpenRouterContextWindowSize(ctx context.Context, baseURL, modelName string) int64

OpenRouterContextWindowSize queries OpenRouter's /models listing for the context window of the given model, preferring top_provider.context_length (what the best endpoint for this model enforces) over the model-level context_length. Returns 0 if the size cannot be determined — mirroring OllamaContextWindowSize, so callers can treat 0 as "no live answer" and fall back to the static catalog.

OpenRouter has no per-model endpoint (it answers 404), so every lookup scans the full ~700KB listing; results are cached per base URL for an hour.

func RefreshPricing added in v0.0.92

func RefreshPricing(ctx context.Context) error

RefreshPricing fetches a fresh pricing snapshot from models.dev and persists it to the XDG cache, regardless of the current snapshot's age. It is the force-refresh path used by the /model-price-refresh command.

func ResolveOllamaEndpoint added in v0.0.75

func ResolveOllamaEndpoint(r OllamaRouting) string

ResolveOllamaEndpoint picks the server a model should be sent to.

The order is:

  1. An explicit BaseURL (OLLAMA_HOST, --url) always wins — that is how someone points at another machine, a container, or an authenticated proxy.
  2. The ollama/ prefix means the local daemon. The prefix is the one way a user states the destination outright, so a tag cannot overrule it: ollama/deepseek-v4-flash:0731-cloud is a request for the model of that name on localhost, and answering it with api.ollama.com ignores what was asked for.
  3. A cloud-tagged model with a key goes to api.ollama.com.
  4. A cloud-tagged model with no key goes to the local daemon, which has proxied cloud models on the user's `ollama signin` identity since 0.12. api.ollama.com rejects an unauthenticated request with 401 before it looks at the model, so the alternative is not a different result but a guaranteed failure.
  5. Everything else is local.

Rule 4 is the only one that reads the key, and it only ever routes *away* from the cloud. That is deliberate, and it is not the bug this function was written to kill: routing used to follow the presence of a key in the other direction, which made OLLAMA_API_KEY a global switch — exporting it once for a :cloud model silently sent every *local* model to api.ollama.com too, where a privately pulled name like qwen3.8:27b-mlx does not exist. A key still never promotes an untagged model to the cloud; its absence only declines to send a request that cannot be served.

Who receives the key is deliberately not decided here. It still goes to whatever endpoint is chosen whenever one is set, unchanged, because an authenticated daemon may be reached over loopback as easily as over the network and this function cannot tell the two apart.

func ShouldFilterModel added in v0.0.92

func ShouldFilterModel(providerName, modelName string, ref time.Time) bool

ShouldFilterModel reports whether a model should be hidden from listings: it is deprecated, released more than a year before the given reference date, and carries no price. Such models are stale and offer nothing to a user.

func StripKnownProviderPrefixes added in v0.0.91

func StripKnownProviderPrefixes(modelName string) string

StripKnownProviderPrefixes removes known provider prefix wrappers from a model name iteratively (e.g. "agentgateway/openai/gpt-5.6-luna" -> "gpt-5.6-luna"). It preserves the casing of the un-prefixed model identifier.

func ValidateModel added in v0.0.18

func ValidateModel(info Info) error

ValidateModel checks whether the model name is recognized for its provider. Returns an error with suggestions if the model is unknown. Ollama and custom endpoint models are always considered valid (they are dynamic).

Types

type Info

type Info struct {
	Provider string
	Model    string
	Ollama   bool // true when model is served by Ollama
	// LocalOllama records that the model was named with the explicit ollama/
	// prefix, which the CLI help documents as "Ollama, local". It keeps a
	// cloud-looking tag on such a name from routing to api.ollama.com.
	LocalOllama bool
	Custom      bool // true when using an explicit custom OpenAI-compatible endpoint
	// BaseURL is the endpoint finally selected for this model, recorded so a
	// session transcript identifies the backend and not just the model name.
	// The same name served by ollama, by a gateway, and by a vendor API behaves
	// differently, and a transcript without it cannot be reproduced.
	BaseURL string
}

Info describes a provider and the model to use.

func Resolve

func Resolve(modelName string) (Info, error)

Resolve determines the provider from a model name. Ollama models are routed to the native "ollama" provider.

func ResolveWithBaseURL added in v0.0.31

func ResolveWithBaseURL(modelName, baseURL string) (Info, error)

ResolveWithBaseURL determines the provider from a model name. When baseURL is provided and the model is not otherwise recognized, it routes the model to an OpenAI-compatible custom endpoint. The optional openai/ prefix is stripped so `--model openai/foo` sends `foo` to the custom endpoint.

type LLMOptions added in v0.0.9

type LLMOptions struct {
	ExtraHeaders    map[string]string
	InsecureSkipTLS bool
	// CACertPath is a PEM bundle to trust in addition to the system roots —
	// the answer for a TLS-intercepting corporate proxy, which otherwise
	// forces InsecureSkipTLS and drops verification for every endpoint.
	// Ignored when InsecureSkipTLS is set.
	CACertPath string
	// DisableSystemCAs narrows trust to CACertPath alone. Only useful when
	// the endpoint must not be reachable through any public root.
	DisableSystemCAs bool
	// ConnectTimeout bounds connection establishment only. Zero leaves the Go
	// default in place. Distinct from the per-client request timeout, which
	// has to stay generous for streaming completions.
	ConnectTimeout time.Duration
	AdvisorModel   string // Advisor model (e.g., "claude-opus-4-7")
	AdvisorMaxUses int    // Max advisor calls per request (0 = unlimited)
	AdvisorCaching bool   // Enable ephemeral prompt caching for advisor
	// DisablePromptCaching turns OFF the cache_control breakpoints the
	// Anthropic provider stamps on every request. Caching is on by default
	// because it only ever lowers the bill: requests whose prefix is below
	// Anthropic's minimum cacheable length (1024-4096 tokens) are simply
	// not cached, with no error and no extra cost. Other providers ignore
	// this flag today.
	DisablePromptCaching bool
	// TraceHTTP records every LLM request and response — method, URL, full
	// headers and body — to the session log and to OTel span events. Set by
	// --trace-http. Credentials are masked (see httplog.Redact), but prompts
	// and completions are not: the log holds the entire conversation in
	// cleartext, which is the point and also the reason it is off by default.
	TraceHTTP bool
	// EnableXAITools opts into xAI server-side tools (web search, X search,
	// and code interpreter) for xAI Responses API requests.
	EnableXAITools bool
	// MaxOutputTokens caps a reply on the OpenAI-compatible paths, in tokens.
	// Zero uses defaultOaiMaxOutputTokens; set it for a backend whose models
	// stop below that and reject the request rather than clamping it. A
	// per-request genai MaxOutputTokens still wins over both.
	MaxOutputTokens int64
	// RateLimit paces outbound requests so a turn stays inside the provider's
	// per-minute quota rather than being rejected by it. A zero value sends at
	// whatever rate the caller manages. Resolved from config by
	// Config.ResolveRateLimits.
	RateLimit ratelimit.Limits
	// RateLimitScope names the budget RateLimit applies to. Every client
	// aimed at the same budget must pass the same scope, because the quota is
	// enforced per account and not per client — see ratelimit.Shared. Left
	// empty, NewLLM fills it in from the provider, model and base URL.
	RateLimitScope string
}

LLMOptions holds optional configuration for LLM provider creation.

type ListModelsOptions added in v0.0.39

type ListModelsOptions struct {
	APIKey   string
	BaseURL  string // override the default API endpoint
	Insecure bool   // skip TLS certificate verification
}

ListModelsOptions controls how models are fetched from a provider.

type ModelInfo added in v0.0.39

type ModelInfo struct {
	ID            string   `json:"id"`
	OwnedBy       string   `json:"owned_by,omitempty"`
	ContextWindow int64    `json:"context_window,omitempty"` // max_context_length
	Capabilities  []string `json:"capabilities,omitempty"`   // e.g. completion_chat, vision
	ReleaseDate   string   `json:"release_date,omitempty"`   // models.dev catalog date, when known
}

ModelInfo holds a model ID and optional metadata returned by a provider's model listing API.

func ListModels added in v0.0.39

func ListModels(ctx context.Context, providerName string, opts ListModelsOptions) ([]ModelInfo, error)

ListModels calls the given provider's model listing API and returns available model IDs. The switch below is the list of supported providers; an unsupported one returns an error.

func RefreshCatalog added in v0.0.83

func RefreshCatalog(ctx context.Context, providerName string, opts ListModelsOptions) ([]ModelInfo, error)

RefreshCatalog fetches live models for a provider (via ListModels) and persists them to the XDG cache. Writes atomically (temp file + rename). Returns the fetched []ModelInfo. On fetch error returns the error (caller falls back to cache/embedded).

type ModelWindow added in v0.0.52

type ModelWindow struct {
	Name          string
	ContextWindow int64
}

ModelWindow pairs a model or deployment name with its context window.

func AzureDeployments added in v0.0.52

func AzureDeployments() []ModelWindow

AzureDeployments returns the cataloged Azure OpenAI deployments and the context window each was provisioned with, sorted by name.

Azure has no listing endpoint reachable with only an API key — enumerating deployments needs ARM credentials and the resource ID — so this is the embedded catalog, not a live query. It is therefore a description of one subscription's deployments and may not match another's.

type OllamaRouting added in v0.0.78

type OllamaRouting struct {
	// Model is the Ollama model name, tag included.
	Model string
	// BaseURL is an explicit endpoint (OLLAMA_HOST or --url), empty if unset.
	BaseURL string
	// APIKey is OLLAMA_API_KEY, empty if unset.
	APIKey string
	// ForceLocal records that the caller named the model with the explicit
	// ollama/ prefix, which the CLI help documents as "Ollama, local".
	ForceLocal bool
}

OllamaRouting carries the facts that decide which Ollama server a model reaches. They travel together because every caller that builds an Ollama client — the CLI, the TUI's /model switch, the ACP server, ping — has to reach the same answer; each one deciding for itself is how the key-as-destination bug survived in three places after ResolveOllamaEndpoint stopped making that mistake.

type PricingModel added in v0.0.92

type PricingModel struct {
	Input       float64       `json:"input,omitempty"`
	Output      float64       `json:"output,omitempty"`
	CacheRead   float64       `json:"cache_read,omitempty"`
	CacheWrite  float64       `json:"cache_write,omitempty"`
	Tiers       []PricingTier `json:"tiers,omitempty"`
	ReleaseDate string        `json:"release_date,omitempty"`
	Deprecated  bool          `json:"deprecated,omitempty"`
	TextOutput  bool          `json:"text_output,omitempty"`
}

PricingModel holds a model's per-million-token rates in USD. All fields are optional; a provider may omit cache rates or reasoning. Tiers carry the context-over threshold at which the higher rate applies. ReleaseDate, Deprecated, and TextOutput come from the models.dev catalog and are used to annotate and filter model listings.

func CostFor added in v0.0.92

func CostFor(providerName, modelName string) (PricingModel, bool)

CostFor returns the per-million-token USD rates for a model served by a provider, using prefix matching so a dated model ID (gpt-5.6-sol) resolves against its base entry (gpt-5.6). ok is false when the provider or model is unknown.

type PricingTier added in v0.0.92

type PricingTier struct {
	ContextOver int64   `json:"context_over"`
	Input       float64 `json:"input,omitempty"`
	Output      float64 `json:"output,omitempty"`
	CacheRead   float64 `json:"cache_read,omitempty"`
	CacheWrite  float64 `json:"cache_write,omitempty"`
}

PricingTier is a context-length tier: rates that apply once the prompt exceeds contextOver tokens.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL