Documentation
¶
Overview ¶
Package runtime provides a provider-agnostic AI runtime for the ai-sdk.
It glues together the domain interfaces (chat, embed, etc.), provider implementations, and the models.dev catalog so that applications can resolve a model reference such as "openai/gpt-5.4" into a working chat provider without hardcoding every provider themselves.
The runtime is intentionally layered above pkg/core: it imports domain interfaces and provider implementations, then delegates the actual chat/embed orchestration to core.GenerateText / core.StreamText.
Key abstractions:
- ProviderClass: a pluggable factory that turns a ProviderConfig into a ProviderSet of domain implementations. Built-in classes include "openai-compatible", which uses pkg/provider/openai with arbitrary base URLs.
- Catalog: the in-memory view of the models.dev provider/model metadata, loaded from a network snapshot, an embedded fallback, or caller-supplied JSON.
- Runtime: the public entry point. It resolves model references to provider instances, caches them, and exposes Chat/ChatStream calls.
Applications can add custom provider classes by calling RegisterClass before constructing a Runtime, or by populating ProviderConfig entries with a known class name.
Package runtime provides built-in OAuth PKCE authentication for providers.
This implementation is written from scratch using only the Go standard library. It performs a local-callback PKCE flow and caches the resulting access token on disk. Callers must supply OpenBrowser if they want the system browser opened automatically; otherwise the authorization URL is printed to stderr.
Index ¶
- Constants
- Variables
- func ClassNames() []string
- func ClearClasses()
- func RegisterAuthResolver(t AuthType, r AuthResolver)
- func RegisterBuiltinClasses()
- func RegisterClass(c ProviderClass)
- func ResolveAPIKey(cfg ProviderConfig) (string, error)
- type AuthConfig
- type AuthResolver
- type AuthResolverFunc
- type AuthResult
- type AuthType
- type Capability
- type Catalog
- func (c *Catalog) API(providerID string) (string, bool)
- func (c *Catalog) APIKeyEnv(providerID string) (string, bool)
- func (c *Catalog) Fetch(ctx context.Context) error
- func (c *Catalog) FetchedAt() time.Time
- func (c *Catalog) Load(ctx context.Context) error
- func (c *Catalog) LoadFromJSON(data []byte) error
- func (c *Catalog) MergeProviders(providers map[string]CatalogProvider)
- func (c *Catalog) Model(providerID, modelID string) (CatalogModel, bool)
- func (c *Catalog) Models(providerID string) ([]CatalogModel, error)
- func (c *Catalog) NPM(providerID string) (string, bool)
- func (c *Catalog) Provider(id string) (CatalogProvider, bool)
- type CatalogModel
- type CatalogOptions
- type CatalogProvider
- type Config
- type CostConfig
- type ModelConfig
- type ModelInfo
- type ModelRef
- type OAuthPKCEResolver
- type ProviderClass
- type ProviderConfig
- type ProviderSet
- type ReasoningOption
- type Runtime
- func (r *Runtime) Catalog() *Catalog
- func (r *Runtime) Chat(ctx context.Context, ref string, opts core.GenerateOptions) (core.GenerateResult, error)
- func (r *Runtime) ChatProvider(ctx context.Context, ref string) (chat.Provider, string, error)
- func (r *Runtime) ChatStream(ctx context.Context, ref string, opts core.GenerateOptions) (core.StreamResult, error)
- func (r *Runtime) LoadCatalog(ctx context.Context) error
- func (r *Runtime) Models(providerID string) ([]ModelInfo, error)
- func (r *Runtime) ParseModelRef(ref string) (ModelRef, error)
- func (r *Runtime) ResolveAuth(ctx context.Context, providerID string) (AuthResult, error)
Constants ¶
const DefaultCatalogURL = "https://models.dev/api.json"
DefaultCatalogURL is the public models.dev provider API endpoint.
Variables ¶
var ErrCapabilityNotSupported = fmt.Errorf("runtime: capability not supported")
ErrCapabilityNotSupported is returned when the resolved provider cannot satisfy the requested capability.
ErrCatalogUnavailable is returned when the catalog cannot be loaded from any source.
var ErrClassNotFound = fmt.Errorf("runtime: provider class not found")
ErrClassNotFound is returned when a provider's class is not registered.
var ErrProviderNotFound = fmt.Errorf("runtime: provider not found")
ErrProviderNotFound is returned when a model reference cannot be mapped to a configured or catalog provider.
var NPMClassMapping = map[string]string{
"@ai-sdk/openai": "openai",
"@ai-sdk/anthropic": "anthropic",
"@ai-sdk/azure": "azure",
"@ai-sdk/cohere": "cohere",
"@ai-sdk/deepseek": "deepseek",
"@ai-sdk/gemini": "gemini",
"@ai-sdk/google": "gemini",
"@ai-sdk/groq": "groq",
"@ai-sdk/mistral": "mistral",
"@ai-sdk/ollama": "ollama",
"@ai-sdk/perplexity": "perplexity",
"@ai-sdk/togetherai": "togetherai",
"@ai-sdk/xai": "xai",
"@ai-sdk/openai-compatible": "openai-compatible",
}
NPMClassMapping maps models.dev npm package identifiers to the provider class names registered by RegisterBuiltinClasses. This lets the Runtime select a class automatically for known providers.
Compatibility rules:
- "@ai-sdk/google" maps to "gemini" because models.dev publishes Google as provider "google" with npm package "@ai-sdk/google", while the native class in this SDK is registered as "gemini".
Functions ¶
func ClassNames ¶
func ClassNames() []string
ClassNames returns the names of all registered classes in sorted order.
func ClearClasses ¶
func ClearClasses()
ClearClasses removes all registered classes. It is intended for tests.
func RegisterAuthResolver ¶
func RegisterAuthResolver(t AuthType, r AuthResolver)
RegisterAuthResolver registers a resolver for an auth type. It overwrites any existing resolver for that type.
func RegisterBuiltinClasses ¶
func RegisterBuiltinClasses()
RegisterBuiltinClasses registers the provider classes and auth resolvers shipped with the ai-sdk. Call this once at program startup before constructing a Runtime.
func RegisterClass ¶
func RegisterClass(c ProviderClass)
RegisterClass registers a provider class. Calling RegisterClass with the same name twice is a no-op (the first registration wins).
func ResolveAPIKey ¶
func ResolveAPIKey(cfg ProviderConfig) (string, error)
ResolveAPIKey extracts an API key from ProviderConfig, preferring the environment variable named by Auth.APIKeyEnv, then a literal APIKey.
Types ¶
type AuthConfig ¶
type AuthConfig struct {
Type AuthType `json:"type,omitempty"`
APIKeyEnv string `json:"api_key_env,omitempty"`
APIKey string `json:"api_key,omitempty"`
AuthorizeURL string `json:"authorize_url,omitempty"`
TokenURL string `json:"token_url,omitempty"`
ClientID string `json:"client_id,omitempty"`
IDP string `json:"idp,omitempty"`
TokenAuthMethod string `json:"token_auth_method,omitempty"`
}
AuthConfig describes how to obtain a bearer token or API key for a provider. Not all fields are used by every AuthType.
type AuthResolver ¶
type AuthResolver interface {
// Resolve returns a credential for the given provider configuration.
Resolve(ctx context.Context, cfg ProviderConfig) (AuthResult, error)
}
AuthResolver resolves an AuthConfig into a credential at call time. Custom resolvers can be registered per class to handle bespoke auth flows (OAuth device code, short-lived token exchange, etc.).
func GetAuthResolver ¶
func GetAuthResolver(t AuthType) (AuthResolver, bool)
GetAuthResolver returns the registered resolver for t, or false if none exists.
type AuthResolverFunc ¶
type AuthResolverFunc func(ctx context.Context, cfg ProviderConfig) (AuthResult, error)
AuthResolverFunc adapts a function to the AuthResolver interface.
func (AuthResolverFunc) Resolve ¶
func (f AuthResolverFunc) Resolve(ctx context.Context, cfg ProviderConfig) (AuthResult, error)
Resolve implements AuthResolver.
type AuthResult ¶
AuthResult is the resolved credential for a provider. It may be empty for providers that require no authentication.
func (AuthResult) IsEmpty ¶
func (r AuthResult) IsEmpty() bool
IsEmpty reports whether the result carries no token or headers.
type Capability ¶
type Capability string
Capability names a model capability a ProviderClass may satisfy.
const ( CapabilityChat Capability = "chat" CapabilityEmbed Capability = "embed" )
Standard capabilities.
type Catalog ¶
type Catalog struct {
// contains filtered or unexported fields
}
Catalog is the in-memory view of the models.dev provider/model metadata.
func NewCatalog ¶
func NewCatalog(opts CatalogOptions) *Catalog
NewCatalog creates an empty catalog. Call Load or Fetch before use.
func (*Catalog) APIKeyEnv ¶
APIKeyEnv returns the first declared environment variable name for a provider's API key, if any.
func (*Catalog) Load ¶
Load populates the catalog from the freshest available source: network if allowed, otherwise a disk cache, otherwise an embedded snapshot or caller-supplied override.
func (*Catalog) LoadFromJSON ¶
LoadFromJSON populates the catalog directly from JSON bytes, ignoring network and cache. This is useful for embedded snapshots or tests.
func (*Catalog) MergeProviders ¶
func (c *Catalog) MergeProviders(providers map[string]CatalogProvider)
MergeProviders overlays additional providers onto the current catalog. Existing entries are merged field-by-field; new entries are added.
func (*Catalog) Model ¶
func (c *Catalog) Model(providerID, modelID string) (CatalogModel, bool)
Model looks up a specific model advertised under a provider.
func (*Catalog) Models ¶
func (c *Catalog) Models(providerID string) ([]CatalogModel, error)
Models returns the models advertised for a provider in deterministic order.
type CatalogModel ¶
type CatalogModel struct {
ID string `json:"id"`
Name string `json:"name,omitzero"`
Family string `json:"family,omitzero"`
Attachment bool `json:"attachment,omitzero"`
Reasoning bool `json:"reasoning,omitzero"`
ToolCall bool `json:"tool_call,omitzero"`
Structured bool `json:"structured_output,omitzero"`
Temperature bool `json:"temperature,omitzero"`
Modalities struct {
Input []string `json:"input,omitzero"`
Output []string `json:"output,omitzero"`
} `json:"modalities"`
Limit struct {
Context int `json:"context,omitzero"`
Output int `json:"output,omitzero"`
} `json:"limit"`
Cost struct {
Input float64 `json:"input,omitzero"`
Output float64 `json:"output,omitzero"`
CacheRead float64 `json:"cache_read,omitzero"`
CacheWrite float64 `json:"cache_write,omitzero"`
} `json:"cost"`
ReasoningOptions []ReasoningOption `json:"reasoning_options,omitzero"`
}
CatalogModel mirrors the per-provider model entries from models.dev. The runtime uses these as metadata; it does not enforce that every provider exposes every advertised model.
func (CatalogModel) ContextWindow ¶
func (m CatalogModel) ContextWindow() int
ContextWindow returns the model's context limit if known.
func (CatalogModel) MaxOutputTokens ¶
func (m CatalogModel) MaxOutputTokens() int
MaxOutputTokens returns the model's output limit if known.
type CatalogOptions ¶
type CatalogOptions struct {
// URL is the endpoint to fetch. Defaults to DefaultCatalogURL.
URL string
// CachePath is a file path used to store the fetched JSON. Empty
// disables disk caching.
CachePath string
// TTL is the maximum age of a cached file before a network refresh.
// Zero or negative disables freshness checks (always fetch).
TTL time.Duration
// HTTPClient is used for network requests. Nil means a default
// client with a 30s timeout.
HTTPClient *http.Client
}
CatalogOptions configures where and how catalog data is loaded.
type CatalogProvider ¶
type CatalogProvider struct {
ID string `json:"id"`
NPM string `json:"npm,omitzero"`
API string `json:"api,omitzero"`
Env []string `json:"env,omitzero"`
Models map[string]CatalogModel `json:"models,omitzero"`
}
CatalogProvider mirrors models.dev's provider entry.
type Config ¶
type Config struct {
// DefaultProvider is the provider ID used when a model reference
// does not include a provider prefix.
DefaultProvider string `json:"default_provider,omitempty"`
// Providers are the configured provider instances. The map key is
// the local provider ID and must match ProviderConfig.ID.
Providers map[string]ProviderConfig `json:"providers,omitempty"`
// CatalogURL overrides the default models.dev endpoint. Empty means
// use DefaultCatalogURL.
CatalogURL string `json:"catalog_url,omitempty"`
// CatalogCachePath enables on-disk caching of the fetched catalog.
CatalogCachePath string `json:"catalog_cache_path,omitempty"`
}
Config is the declarative runtime configuration. Applications supply this (e.g. from JSON/YAML) and the Runtime turns it into working provider instances.
func (Config) ProviderByID ¶
func (c Config) ProviderByID(id string) (ProviderConfig, bool)
ProviderByID returns the provider config with the given id, or false.
type CostConfig ¶
type CostConfig struct {
Input float64 `json:"input,omitempty"`
Output float64 `json:"output,omitempty"`
CacheRead float64 `json:"cache_read,omitempty"`
CacheWrite float64 `json:"cache_write,omitempty"`
}
CostConfig carries per-token pricing metadata.
type ModelConfig ¶
type ModelConfig struct {
ID string `json:"id"`
Name string `json:"name,omitempty"`
URL string `json:"url,omitempty"`
ContextWindow int `json:"context_window,omitempty"`
MaxOutputTokens int `json:"max_output_tokens,omitempty"`
Reasoning bool `json:"reasoning,omitempty"`
ToolCall bool `json:"tool_call,omitempty"`
StructuredOutput bool `json:"structured_output,omitempty"`
Temperature bool `json:"temperature,omitempty"`
Cost CostConfig `json:"cost"`
Extra map[string]any `json:"extra,omitempty"`
}
ModelConfig is a configured model entry for a provider. It is merged with catalog metadata by the runtime.
type ModelInfo ¶
type ModelInfo struct {
ID string
ProviderID string
Name string
URL string
ContextWindow int
MaxOutputTokens int
Reasoning bool
ReasoningOptions []ReasoningOption
ToolCall bool
StructuredOutput bool
Temperature bool
Cost CostConfig
Extra map[string]any
}
ModelInfo is the runtime's normalised view of a model, combining catalog data and configured overrides.
type ModelRef ¶
ModelRef is a parsed model reference of the form "provider/model" or just "model" when a default provider is configured.
type OAuthPKCEResolver ¶
type OAuthPKCEResolver struct {
// CacheDir overrides the directory used to store access tokens.
// Defaults to os.UserConfigDir()/ai-sdk/oauth.
CacheDir string
// OpenBrowser wraps browser.OpenURL. Tests can override it.
OpenBrowser func(url string) error
}
OAuthPKCEResolver is the built-in AuthResolver for AuthTypeOAuthPKCE. It is registered by RegisterBuiltinClasses so that providers configured with auth.type = "oauth_pkce" can obtain bearer tokens automatically.
func (*OAuthPKCEResolver) Resolve ¶
func (r *OAuthPKCEResolver) Resolve(ctx context.Context, cfg ProviderConfig) (AuthResult, error)
Resolve implements AuthResolver.
type ProviderClass ¶
type ProviderClass interface {
// Name returns the stable class identifier used in ProviderConfig.Class.
Name() string
// Supports reports whether this class can satisfy cap.
Supports(cap Capability) bool
// New builds a ProviderSet from cfg for the given model. ctx may be
// used for short-lived setup requests (e.g. discovery or token
// exchange), but must not be stored.
New(ctx context.Context, cfg ProviderConfig, model ModelInfo) (ProviderSet, error)
}
ProviderClass is a factory for provider instances. Each class knows how to turn a ProviderConfig (base URL, auth, options) and a resolved model into one or more domain providers.
func GetClass ¶
func GetClass(name string) (ProviderClass, bool)
GetClass returns a registered class by name.
func MustRegisterClass ¶
func MustRegisterClass(c ProviderClass) ProviderClass
MustRegisterClass is like RegisterClass but returns the class so it can be chained in var initializations.
type ProviderConfig ¶
type ProviderConfig struct {
// ID is the local identifier for this provider instance (e.g. "openai",
// "my-maas"). It must be unique within a Runtime.
ID string
// Class selects the registered ProviderClass factory. Examples:
// "openai-compatible", "anthropic-messages".
Class string
// BaseURL is the provider's API base URL. Individual models may
// override this via their own URL field.
BaseURL string
// Auth describes how to obtain credentials for this provider.
Auth AuthConfig
// Headers are extra HTTP headers merged into every request.
Headers map[string]string
// Insecure disables TLS certificate verification.
Insecure bool
// Timeout for provider HTTP requests in milliseconds. Zero means the
// class default.
Timeout int
// Options carries class-specific options as a generic map. Classes
// read only the keys they understand.
Options map[string]any
// Models allows per-provider model overrides/advertisements. These
// augment or override the catalog when the runtime resolves model
// references.
Models []ModelConfig
}
ProviderConfig is the minimal information needed to construct a provider instance. The Class field selects which ProviderClass factory is invoked.
type ProviderSet ¶
ProviderSet is a collection of domain providers produced by a single ProviderClass instance.
func (ProviderSet) Has ¶
func (s ProviderSet) Has(cap Capability) bool
Has reports whether the set satisfies cap.
type ReasoningOption ¶
ReasoningOption describes one tunable a reasoning model accepts, as published by models.dev. The common shape is {"type":"effort","values":["low", "medium","high"]}, which lets callers offer exactly the effort levels the model supports instead of guessing.
type Runtime ¶
type Runtime struct {
// contains filtered or unexported fields
}
Runtime resolves model references to provider instances and exposes a high-level chat/embed API. It caches provider instances so repeated calls to the same provider/model/URL reuse the same underlying HTTP client.
func NewRuntime ¶
NewRuntime creates a Runtime from the supplied configuration. It does not load the catalog automatically; call LoadCatalog (or supply a pre-populated Catalog) before making calls.
func NewRuntimeWithCatalog ¶
NewRuntimeWithCatalog creates a Runtime that uses an already-populated catalog.
func (*Runtime) Catalog ¶
Catalog returns the runtime's catalog. It may be nil until LoadCatalog is called.
func (*Runtime) Chat ¶
func (r *Runtime) Chat(ctx context.Context, ref string, opts core.GenerateOptions) (core.GenerateResult, error)
Chat performs a non-streaming chat completion for the given model reference. The model field inside opts is overwritten with the resolved model ID.
func (*Runtime) ChatProvider ¶
ChatProvider resolves a model reference to a chat.Provider. It returns the provider instance and the resolved model ID that should be passed to requests.
func (*Runtime) ChatStream ¶
func (r *Runtime) ChatStream(ctx context.Context, ref string, opts core.GenerateOptions) (core.StreamResult, error)
ChatStream performs a streaming chat completion for the given model reference.
func (*Runtime) LoadCatalog ¶
LoadCatalog fetches or loads the models.dev catalog according to the runtime configuration. If cfg.CatalogURL or cfg.CatalogCachePath are set, they override the catalog defaults.
func (*Runtime) Models ¶
Models returns the resolved model information for a provider, merged from configured overrides and the catalog.
func (*Runtime) ParseModelRef ¶
ParseModelRef splits a model reference into provider and model IDs. If the reference has no "/" separator and a default provider is configured, the default provider is used.
func (*Runtime) ResolveAuth ¶
ResolveAuth resolves credentials for a configured provider using its registered auth resolver.