runtime

package
v0.1.20 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 30, 2026 License: Apache-2.0 Imports: 35 Imported by: 0

Documentation

Overview

Package runtime provides a provider-agnostic AI runtime for the ai-sdk.

It glues together the domain interfaces (chat, embed, etc.), provider implementations, and the models.dev catalog so that applications can resolve a model reference such as "openai/gpt-5.4" into a working chat provider without hardcoding every provider themselves.

The runtime is intentionally layered above pkg/core: it imports domain interfaces and provider implementations, then delegates the actual chat/embed orchestration to core.GenerateText / core.StreamText.

Key abstractions:

  • ProviderClass: a pluggable factory that turns a ProviderConfig into a ProviderSet of domain implementations. Built-in classes include "openai-compatible", which uses pkg/provider/openai with arbitrary base URLs.
  • Catalog: the in-memory view of the models.dev provider/model metadata, loaded from a network snapshot, an embedded fallback, or caller-supplied JSON.
  • Runtime: the public entry point. It resolves model references to provider instances, caches them, and exposes Chat/ChatStream calls.

Applications can add custom provider classes by calling RegisterClass before constructing a Runtime, or by populating ProviderConfig entries with a known class name.

Package runtime provides built-in OAuth PKCE authentication for providers.

This implementation is written from scratch using only the Go standard library. It performs a local-callback PKCE flow and caches the resulting access token on disk. Callers must supply OpenBrowser if they want the system browser opened automatically; otherwise the authorization URL is printed to stderr.

Index

Constants

View Source
const DefaultCatalogURL = "https://models.dev/api.json"

DefaultCatalogURL is the public models.dev provider API endpoint.

Variables

View Source
var ErrCapabilityNotSupported = fmt.Errorf("runtime: capability not supported")

ErrCapabilityNotSupported is returned when the resolved provider cannot satisfy the requested capability.

View Source
var ErrCatalogUnavailable = errors.New("runtime: model catalog unavailable")

ErrCatalogUnavailable is returned when the catalog cannot be loaded from any source.

View Source
var ErrClassNotFound = fmt.Errorf("runtime: provider class not found")

ErrClassNotFound is returned when a provider's class is not registered.

View Source
var ErrProviderNotFound = fmt.Errorf("runtime: provider not found")

ErrProviderNotFound is returned when a model reference cannot be mapped to a configured or catalog provider.

View Source
var NPMClassMapping = map[string]string{
	"@ai-sdk/openai":            "openai",
	"@ai-sdk/anthropic":         "anthropic",
	"@ai-sdk/azure":             "azure",
	"@ai-sdk/cohere":            "cohere",
	"@ai-sdk/deepseek":          "deepseek",
	"@ai-sdk/gemini":            "gemini",
	"@ai-sdk/google":            "gemini",
	"@ai-sdk/groq":              "groq",
	"@ai-sdk/mistral":           "mistral",
	"@ai-sdk/ollama":            "ollama",
	"@ai-sdk/perplexity":        "perplexity",
	"@ai-sdk/togetherai":        "togetherai",
	"@ai-sdk/xai":               "xai",
	"@ai-sdk/openai-compatible": "openai-compatible",
}

NPMClassMapping maps models.dev npm package identifiers to the provider class names registered by RegisterBuiltinClasses. This lets the Runtime select a class automatically for known providers.

Compatibility rules:

  • "@ai-sdk/google" maps to "gemini" because models.dev publishes Google as provider "google" with npm package "@ai-sdk/google", while the native class in this SDK is registered as "gemini".

Functions

func ClassNames

func ClassNames() []string

ClassNames returns the names of all registered classes in sorted order.

func ClearClasses

func ClearClasses()

ClearClasses removes all registered classes. It is intended for tests.

func RegisterAuthResolver

func RegisterAuthResolver(t AuthType, r AuthResolver)

RegisterAuthResolver registers a resolver for an auth type. It overwrites any existing resolver for that type.

func RegisterBuiltinClasses

func RegisterBuiltinClasses()

RegisterBuiltinClasses registers the provider classes and auth resolvers shipped with the ai-sdk. Call this once at program startup before constructing a Runtime.

func RegisterClass

func RegisterClass(c ProviderClass)

RegisterClass registers a provider class. Calling RegisterClass with the same name twice is a no-op (the first registration wins).

func ResolveAPIKey

func ResolveAPIKey(cfg ProviderConfig) (string, error)

ResolveAPIKey extracts an API key from ProviderConfig, preferring the environment variable named by Auth.APIKeyEnv, then a literal APIKey.

Types

type AuthConfig

type AuthConfig struct {
	Type            AuthType `json:"type,omitempty"`
	APIKeyEnv       string   `json:"api_key_env,omitempty"`
	APIKey          string   `json:"api_key,omitempty"`
	AuthorizeURL    string   `json:"authorize_url,omitempty"`
	TokenURL        string   `json:"token_url,omitempty"`
	ClientID        string   `json:"client_id,omitempty"`
	IDP             string   `json:"idp,omitempty"`
	TokenAuthMethod string   `json:"token_auth_method,omitempty"`
}

AuthConfig describes how to obtain a bearer token or API key for a provider. Not all fields are used by every AuthType.

type AuthResolver

type AuthResolver interface {
	// Resolve returns a credential for the given provider configuration.
	Resolve(ctx context.Context, cfg ProviderConfig) (AuthResult, error)
}

AuthResolver resolves an AuthConfig into a credential at call time. Custom resolvers can be registered per class to handle bespoke auth flows (OAuth device code, short-lived token exchange, etc.).

func GetAuthResolver

func GetAuthResolver(t AuthType) (AuthResolver, bool)

GetAuthResolver returns the registered resolver for t, or false if none exists.

type AuthResolverFunc

type AuthResolverFunc func(ctx context.Context, cfg ProviderConfig) (AuthResult, error)

AuthResolverFunc adapts a function to the AuthResolver interface.

func (AuthResolverFunc) Resolve

Resolve implements AuthResolver.

type AuthResult

type AuthResult struct {
	Token   string
	Headers map[string]string
}

AuthResult is the resolved credential for a provider. It may be empty for providers that require no authentication.

func (AuthResult) IsEmpty

func (r AuthResult) IsEmpty() bool

IsEmpty reports whether the result carries no token or headers.

type AuthType

type AuthType string

AuthType names a provider authentication strategy.

const (
	AuthTypeNone      AuthType = "none"
	AuthTypeAPIKey    AuthType = "api_key"
	AuthTypeOAuthPKCE AuthType = "oauth_pkce"
)

Built-in auth types.

type Capability

type Capability string

Capability names a model capability a ProviderClass may satisfy.

const (
	CapabilityChat  Capability = "chat"
	CapabilityEmbed Capability = "embed"
)

Standard capabilities.

type Catalog

type Catalog struct {
	// contains filtered or unexported fields
}

Catalog is the in-memory view of the models.dev provider/model metadata.

func NewCatalog

func NewCatalog(opts CatalogOptions) *Catalog

NewCatalog creates an empty catalog. Call Load or Fetch before use.

func (*Catalog) API

func (c *Catalog) API(providerID string) (string, bool)

API returns the provider's default API base URL from the catalog.

func (*Catalog) APIKeyEnv

func (c *Catalog) APIKeyEnv(providerID string) (string, bool)

APIKeyEnv returns the first declared environment variable name for a provider's API key, if any.

func (*Catalog) Fetch

func (c *Catalog) Fetch(ctx context.Context) error

Fetch forces a network refresh and writes the result to disk cache.

func (*Catalog) FetchedAt

func (c *Catalog) FetchedAt() time.Time

FetchedAt returns the time the current snapshot was obtained.

func (*Catalog) Load

func (c *Catalog) Load(ctx context.Context) error

Load populates the catalog from the freshest available source: network if allowed, otherwise a disk cache, otherwise an embedded snapshot or caller-supplied override.

func (*Catalog) LoadFromJSON

func (c *Catalog) LoadFromJSON(data []byte) error

LoadFromJSON populates the catalog directly from JSON bytes, ignoring network and cache. This is useful for embedded snapshots or tests.

func (*Catalog) MergeProviders

func (c *Catalog) MergeProviders(providers map[string]CatalogProvider)

MergeProviders overlays additional providers onto the current catalog. Existing entries are merged field-by-field; new entries are added.

func (*Catalog) Model

func (c *Catalog) Model(providerID, modelID string) (CatalogModel, bool)

Model looks up a specific model advertised under a provider.

func (*Catalog) Models

func (c *Catalog) Models(providerID string) ([]CatalogModel, error)

Models returns the models advertised for a provider in deterministic order.

func (*Catalog) NPM

func (c *Catalog) NPM(providerID string) (string, bool)

NPM returns the provider's npm package identifier from the catalog.

func (*Catalog) Provider

func (c *Catalog) Provider(id string) (CatalogProvider, bool)

Provider returns a provider by id, normalised to lower case.

type CatalogModel

type CatalogModel struct {
	ID          string `json:"id"`
	Name        string `json:"name,omitzero"`
	Family      string `json:"family,omitzero"`
	Attachment  bool   `json:"attachment,omitzero"`
	Reasoning   bool   `json:"reasoning,omitzero"`
	ToolCall    bool   `json:"tool_call,omitzero"`
	Structured  bool   `json:"structured_output,omitzero"`
	Temperature bool   `json:"temperature,omitzero"`
	Modalities  struct {
		Input  []string `json:"input,omitzero"`
		Output []string `json:"output,omitzero"`
	} `json:"modalities"`
	Limit struct {
		Context int `json:"context,omitzero"`
		Output  int `json:"output,omitzero"`
	} `json:"limit"`
	Cost struct {
		Input      float64 `json:"input,omitzero"`
		Output     float64 `json:"output,omitzero"`
		CacheRead  float64 `json:"cache_read,omitzero"`
		CacheWrite float64 `json:"cache_write,omitzero"`
	} `json:"cost"`
	ReasoningOptions []ReasoningOption `json:"reasoning_options,omitzero"`
}

CatalogModel mirrors the per-provider model entries from models.dev. The runtime uses these as metadata; it does not enforce that every provider exposes every advertised model.

func (CatalogModel) ContextWindow

func (m CatalogModel) ContextWindow() int

ContextWindow returns the model's context limit if known.

func (CatalogModel) MaxOutputTokens

func (m CatalogModel) MaxOutputTokens() int

MaxOutputTokens returns the model's output limit if known.

type CatalogOptions

type CatalogOptions struct {
	// URL is the endpoint to fetch. Defaults to DefaultCatalogURL.
	URL string

	// CachePath is a file path used to store the fetched JSON. Empty
	// disables disk caching.
	CachePath string

	// TTL is the maximum age of a cached file before a network refresh.
	// Zero or negative disables freshness checks (always fetch).
	TTL time.Duration

	// HTTPClient is used for network requests. Nil means a default
	// client with a 30s timeout.
	HTTPClient *http.Client
}

CatalogOptions configures where and how catalog data is loaded.

type CatalogProvider

type CatalogProvider struct {
	ID     string                  `json:"id"`
	NPM    string                  `json:"npm,omitzero"`
	API    string                  `json:"api,omitzero"`
	Env    []string                `json:"env,omitzero"`
	Models map[string]CatalogModel `json:"models,omitzero"`
}

CatalogProvider mirrors models.dev's provider entry.

type Config

type Config struct {
	// DefaultProvider is the provider ID used when a model reference
	// does not include a provider prefix.
	DefaultProvider string `json:"default_provider,omitempty"`

	// Providers are the configured provider instances. The map key is
	// the local provider ID and must match ProviderConfig.ID.
	Providers map[string]ProviderConfig `json:"providers,omitempty"`

	// CatalogURL overrides the default models.dev endpoint. Empty means
	// use DefaultCatalogURL.
	CatalogURL string `json:"catalog_url,omitempty"`

	// CatalogCachePath enables on-disk caching of the fetched catalog.
	CatalogCachePath string `json:"catalog_cache_path,omitempty"`
}

Config is the declarative runtime configuration. Applications supply this (e.g. from JSON/YAML) and the Runtime turns it into working provider instances.

func (Config) ProviderByID

func (c Config) ProviderByID(id string) (ProviderConfig, bool)

ProviderByID returns the provider config with the given id, or false.

type CostConfig

type CostConfig struct {
	Input      float64 `json:"input,omitempty"`
	Output     float64 `json:"output,omitempty"`
	CacheRead  float64 `json:"cache_read,omitempty"`
	CacheWrite float64 `json:"cache_write,omitempty"`
}

CostConfig carries per-token pricing metadata.

type ModelConfig

type ModelConfig struct {
	ID               string         `json:"id"`
	Name             string         `json:"name,omitempty"`
	URL              string         `json:"url,omitempty"`
	ContextWindow    int            `json:"context_window,omitempty"`
	MaxOutputTokens  int            `json:"max_output_tokens,omitempty"`
	Reasoning        bool           `json:"reasoning,omitempty"`
	ToolCall         bool           `json:"tool_call,omitempty"`
	StructuredOutput bool           `json:"structured_output,omitempty"`
	Temperature      bool           `json:"temperature,omitempty"`
	Cost             CostConfig     `json:"cost"`
	Extra            map[string]any `json:"extra,omitempty"`
}

ModelConfig is a configured model entry for a provider. It is merged with catalog metadata by the runtime.

type ModelInfo

type ModelInfo struct {
	ID               string
	ProviderID       string
	Name             string
	URL              string
	ContextWindow    int
	MaxOutputTokens  int
	Reasoning        bool
	ReasoningOptions []ReasoningOption
	ToolCall         bool
	StructuredOutput bool
	Temperature      bool
	Cost             CostConfig
	Extra            map[string]any
}

ModelInfo is the runtime's normalised view of a model, combining catalog data and configured overrides.

type ModelRef

type ModelRef struct {
	ProviderID string
	ModelID    string
}

ModelRef is a parsed model reference of the form "provider/model" or just "model" when a default provider is configured.

type OAuthPKCEResolver

type OAuthPKCEResolver struct {
	// CacheDir overrides the directory used to store access tokens.
	// Defaults to os.UserConfigDir()/ai-sdk/oauth.
	CacheDir string

	// OpenBrowser wraps browser.OpenURL. Tests can override it.
	OpenBrowser func(url string) error
}

OAuthPKCEResolver is the built-in AuthResolver for AuthTypeOAuthPKCE. It is registered by RegisterBuiltinClasses so that providers configured with auth.type = "oauth_pkce" can obtain bearer tokens automatically.

func (*OAuthPKCEResolver) Resolve

Resolve implements AuthResolver.

type ProviderClass

type ProviderClass interface {
	// Name returns the stable class identifier used in ProviderConfig.Class.
	Name() string

	// Supports reports whether this class can satisfy cap.
	Supports(cap Capability) bool

	// New builds a ProviderSet from cfg for the given model. ctx may be
	// used for short-lived setup requests (e.g. discovery or token
	// exchange), but must not be stored.
	New(ctx context.Context, cfg ProviderConfig, model ModelInfo) (ProviderSet, error)
}

ProviderClass is a factory for provider instances. Each class knows how to turn a ProviderConfig (base URL, auth, options) and a resolved model into one or more domain providers.

func GetClass

func GetClass(name string) (ProviderClass, bool)

GetClass returns a registered class by name.

func MustRegisterClass

func MustRegisterClass(c ProviderClass) ProviderClass

MustRegisterClass is like RegisterClass but returns the class so it can be chained in var initializations.

type ProviderConfig

type ProviderConfig struct {
	// ID is the local identifier for this provider instance (e.g. "openai",
	// "my-maas"). It must be unique within a Runtime.
	ID string

	// Class selects the registered ProviderClass factory. Examples:
	// "openai-compatible", "anthropic-messages".
	Class string

	// BaseURL is the provider's API base URL. Individual models may
	// override this via their own URL field.
	BaseURL string

	// Auth describes how to obtain credentials for this provider.
	Auth AuthConfig

	// Headers are extra HTTP headers merged into every request.
	Headers map[string]string

	// Insecure disables TLS certificate verification.
	Insecure bool

	// Timeout for provider HTTP requests in milliseconds. Zero means the
	// class default.
	Timeout int

	// Options carries class-specific options as a generic map. Classes
	// read only the keys they understand.
	Options map[string]any

	// Models allows per-provider model overrides/advertisements. These
	// augment or override the catalog when the runtime resolves model
	// references.
	Models []ModelConfig
}

ProviderConfig is the minimal information needed to construct a provider instance. The Class field selects which ProviderClass factory is invoked.

type ProviderSet

type ProviderSet struct {
	Chat  chat.Provider
	Embed embed.Provider
}

ProviderSet is a collection of domain providers produced by a single ProviderClass instance.

func (ProviderSet) Has

func (s ProviderSet) Has(cap Capability) bool

Has reports whether the set satisfies cap.

type ReasoningOption

type ReasoningOption struct {
	Type   string   `json:"type"`
	Values []string `json:"values,omitzero"`
}

ReasoningOption describes one tunable a reasoning model accepts, as published by models.dev. The common shape is {"type":"effort","values":["low", "medium","high"]}, which lets callers offer exactly the effort levels the model supports instead of guessing.

type Runtime

type Runtime struct {
	// contains filtered or unexported fields
}

Runtime resolves model references to provider instances and exposes a high-level chat/embed API. It caches provider instances so repeated calls to the same provider/model/URL reuse the same underlying HTTP client.

func NewRuntime

func NewRuntime(cfg Config) *Runtime

NewRuntime creates a Runtime from the supplied configuration. It does not load the catalog automatically; call LoadCatalog (or supply a pre-populated Catalog) before making calls.

func NewRuntimeWithCatalog

func NewRuntimeWithCatalog(cfg Config, catalog *Catalog) *Runtime

NewRuntimeWithCatalog creates a Runtime that uses an already-populated catalog.

func (*Runtime) Catalog

func (r *Runtime) Catalog() *Catalog

Catalog returns the runtime's catalog. It may be nil until LoadCatalog is called.

func (*Runtime) Chat

Chat performs a non-streaming chat completion for the given model reference. The model field inside opts is overwritten with the resolved model ID.

func (*Runtime) ChatProvider

func (r *Runtime) ChatProvider(ctx context.Context, ref string) (chat.Provider, string, error)

ChatProvider resolves a model reference to a chat.Provider. It returns the provider instance and the resolved model ID that should be passed to requests.

func (*Runtime) ChatStream

func (r *Runtime) ChatStream(ctx context.Context, ref string, opts core.GenerateOptions) (core.StreamResult, error)

ChatStream performs a streaming chat completion for the given model reference.

func (*Runtime) LoadCatalog

func (r *Runtime) LoadCatalog(ctx context.Context) error

LoadCatalog fetches or loads the models.dev catalog according to the runtime configuration. If cfg.CatalogURL or cfg.CatalogCachePath are set, they override the catalog defaults.

func (*Runtime) Models

func (r *Runtime) Models(providerID string) ([]ModelInfo, error)

Models returns the resolved model information for a provider, merged from configured overrides and the catalog.

func (*Runtime) ParseModelRef

func (r *Runtime) ParseModelRef(ref string) (ModelRef, error)

ParseModelRef splits a model reference into provider and model IDs. If the reference has no "/" separator and a default provider is configured, the default provider is used.

func (*Runtime) ResolveAuth

func (r *Runtime) ResolveAuth(ctx context.Context, providerID string) (AuthResult, error)

ResolveAuth resolves credentials for a configured provider using its registered auth resolver.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL