provider

package
v0.1.8-rc.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 23, 2026 License: Apache-2.0 Imports: 41 Imported by: 0

Documentation

Overview

Package provider implements the multi-provider system for sol. This matches opencode's provider architecture for supporting multiple LLM providers. Provider information is fetched dynamically from models.dev API.

Index

Constants

View Source 
const (
	CapText          = "text"
	CapVision        = "vision"
	CapImageGen      = "image_gen"
	CapSearch        = "search"
	CapEmbedding     = "embedding"
	CapSpeech        = "speech"
	CapTranscription = "transcription"
	CapReranking     = "reranking"
)

Capability constants. These are the strings exposed over the API and used as keys in the UI capability matrix. Keep them lowercase snake_case.

Two axes:

  • Kind-derived (CapEmbedding/CapSpeech/CapTranscription/CapReranking, plus CapText/CapImageGen which double as kinds). True when the provider has at least one model of that kind.
  • Modality-derived (CapVision). True when the provider has at least one language model that accepts the corresponding extra input modality.
View Source 
const (
	// ModelsDevURL is the URL for the models.dev API
	ModelsDevURL = "https://models.dev/api.json"

	// RefreshInterval is the cadence of the background catalog refresh
	// when StartPeriodicRefresh is in use. Also the freshness window
	// that the lazy LoadProviders() path uses to skip re-fetching.
	RefreshInterval = 12 * time.Hour
)
View Source 
const OpenRouterBaseURL = "https://openrouter.ai/api/v1"

OpenRouterBaseURL is OpenRouter's OpenAI-compatible API root. Used as the default when a configured provider leaves the base URL blank.

View Source 
const OpenRouterEmbeddingModelsURL = "https://openrouter.ai/api/v1/embeddings/models"

OpenRouterEmbeddingModelsURL is OpenRouter's public embeddings catalog. Unlike chat models (which models.dev mirrors and OpenRouter also lists at /api/v1/models), OpenRouter keeps embedding models on this separate endpoint — so they appear in neither models.dev nor the main /models list. No API key is required to read it.

Variables

View Source 
var AttachmentOverlay = map[string]AttachmentPolicy{
	"openai": {
		SupportsURL:         true,
		SupportsFileURL:     false,
		MaxURLImages:        50,
		MaxURLBytesPerImage: 18 * 1024 * 1024,
		MaxInlineBytesTotal: 40 * 1024 * 1024,
	},
	"anthropic": {
		SupportsURL:         true,
		SupportsFileURL:     true,
		MaxURLImages:        50,
		MaxURLBytesPerImage: 4*1024*1024 + 512*1024,
		MaxInlineBytesTotal: 24 * 1024 * 1024,
	},
	"google": {
		SupportsURL:         true,
		SupportsFileURL:     true,
		MaxURLImages:        100,
		MaxURLBytesPerImage: 20 * 1024 * 1024,
		MaxInlineBytesTotal: 80 * 1024 * 1024,
	},
	"vertex": {
		SupportsURL:         true,
		SupportsFileURL:     true,
		MaxURLImages:        100,
		MaxURLBytesPerImage: 20 * 1024 * 1024,
		MaxInlineBytesTotal: 80 * 1024 * 1024,
	},
	"bedrock": {
		SupportsURL:         true,
		SupportsFileURL:     true,
		MaxURLImages:        50,
		MaxURLBytesPerImage: 4*1024*1024 + 512*1024,
		MaxInlineBytesTotal: 24 * 1024 * 1024,
	},
	"xai": {
		SupportsURL:         true,
		SupportsFileURL:     false,
		MaxURLImages:        50,
		MaxURLBytesPerImage: 18 * 1024 * 1024,
		MaxInlineBytesTotal: 40 * 1024 * 1024,
	},
	"openaicompat": {
		SupportsURL:         true,
		SupportsFileURL:     false,
		MaxURLImages:        5,
		MaxURLBytesPerImage: 3*1024*1024 + 512*1024,
		MaxInlineBytesTotal: 5 * 1024 * 1024,
	},
}

AttachmentOverlay is the per-provider override map, keyed by provider_id. Values are applied as-is — not merged with DefaultAttachmentPolicy — so each entry must be complete.

Covered: every provider in goai whose message conversion accepts a message.FilePart carrying a URL data variant.

Sizing references (as of 2026-Q2 docs):

Our caps sit *below* each provider's hard limit to leave headroom for JSON framing, tool schemas, prompt bytes, etc. `MaxInlineBytesTotal` is the base64-encoded byte budget (roughly 1.33× the raw byte count).

View Source 
var DefaultAttachmentPolicy = AttachmentPolicy{
	MaxInlineBytesTotal: 5 * 1024 * 1024,
	SupportsURL:         false,
}

DefaultAttachmentPolicy is the fallback — inline-only, 5 MiB request cap.

Functions

func AllProviders 

func AllProviders() (map[string]*ModelsDevProvider, error)

AllProviders returns the full provider catalog: models.dev data enriched with hand-maintained extras (search-only provider stubs, goai's typed kind lists, name-based + OpenRouter embedding classification). Callers that want the live models.dev map unmodified should call LoadProviders directly.

Deprecated models (Status == "deprecated" per models.dev) are dropped before the goai merge so they don't surface in pickers / catalogs. Runtime lookups via GetModelInfo go through LoadProviders directly and so still resolve — agents configured on a now-deprecated model keep running, they're just hidden from the agent-create dropdown going forward.

Merge semantics, in order:

  1. Start from models.dev's LoadProviders() result.
  2. Synthesize stubs for search-only providers absent from models.dev (e.g. brave). Their search capability is derived from SearchBackend.
  3. Drop deprecated entries from every provider in the merged set.
  4. Layer goai: for each provider goai has typed-list coverage for (see goaiProviderFactories), stamp ModelInfo.Kind on existing entries and synthesize new ModelInfo for goai-listed IDs that models.dev doesn't ship (e.g. openai's whisper-1, tts-1). Goai is authoritative for Kind; modalities, cost, and limits come from whichever source has them (models.dev wins, kind-derived defaults fall in for the rest).
  5. Name-based embedding classification (any model id/name with "embed").
  6. Merge OpenRouter's dedicated embeddings catalog (openrouter.go).

The returned map's inner *ModelsDevProvider pointers are clones whenever we touched the provider, so callers can mutate top-level safely. Caches are unchanged.

func CreateEmbeddingModel 

func CreateEmbeddingModel(providerID, modelID string, opts Options) model.EmbeddingModel

CreateEmbeddingModel creates an embedding model from provider and model IDs.

func CreateImageModel 

func CreateImageModel(providerID, modelID string, opts Options) model.ImageModel

CreateImageModel creates an image generation model from provider and model IDs.

func CreateModel 

func CreateModel(providerID, modelID string, opts Options) stream.Model

CreateModel creates a language model from provider and model IDs. This is the main entry point for getting a model instance.

func CreateProxyModel 

func CreateProxyModel(fullModelID string, opts ProxyOptions) stream.Model

CreateProxyModel creates a model that proxies LLM calls through an Airlock-compatible endpoint. The full model string (e.g., "anthropic/claude-sonnet-4-20250514") is passed to the proxy which resolves credentials and forwards to the real provider.

func CreateSpeechModel 

func CreateSpeechModel(providerID, modelID string, opts Options) model.SpeechModel

CreateSpeechModel creates a text-to-speech model from provider and model IDs.

func CreateTranscriptionModel 

func CreateTranscriptionModel(providerID, modelID string, opts Options) model.TranscriptionModel

CreateTranscriptionModel creates a speech-to-text model from provider and model IDs.

func GetContextLimit 

func GetContextLimit(providerID, modelID string) int

GetContextLimit returns the context limit for a model. Returns 0 if the model is not found or has no limit defined.

func GetDisplayName 

func GetDisplayName(providerID string) string

GetDisplayName returns the display name for a provider. Uses dynamic data from models.dev when available.

func GetEnvVarName 

func GetEnvVarName(providerID string) string

GetEnvVarName returns the primary environment variable name for a provider's API key. Uses dynamic data from models.dev when available.

func GetOutputLimit 

func GetOutputLimit(providerID, modelID string) int

GetOutputLimit returns the output limit for a model. Returns 0 if the model is not found or has no limit defined.

func ListProviders 

func ListProviders() []string

ListProviders returns all available provider IDs

func LoadProviders 

func LoadProviders() (map[string]*ModelsDevProvider, error)

LoadProviders loads providers from cache or fetches from models.dev This is called lazily on first access

func MaxOutputTokens 

func MaxOutputTokens(modelID string) int

MaxOutputTokens returns the appropriate max output tokens for a model. This matches opencode's defaults for reasoning vs non-reasoning models.

func ParseModel 

func ParseModel(model string) (providerID, modelID string)

ParseModel parses a "provider/model" string into provider and model IDs. This matches opencode's parseModel function. Examples:

  • "openai/gpt-4o-mini" → ("openai", "gpt-4o-mini")
  • "anthropic/claude-3-5-sonnet" → ("anthropic", "claude-3-5-sonnet")
  • "gpt-4o-mini" → ("openai", "gpt-4o-mini") // defaults to openai

func ProviderOptions 

func ProviderOptions(providerID, modelID, sessionID string) map[string]any

ProviderOptions returns the provider-specific options for a model. This matches opencode's ProviderTransform.options() function.

func SearchBackend 

func SearchBackend(providerID string) string

SearchBackend returns the sol/websearch backend client name for a provider, or "" if the provider doesn't offer web search.

func StartPeriodicRefresh added in v0.1.2 

func StartPeriodicRefresh(ctx context.Context)

StartPeriodicRefresh launches a background ticker that refreshes the models.dev catalog every RefreshInterval. Idempotent — calling more than once spawns at most one ticker (subsequent calls are no-ops).

Pre-flight steps before the ticker runs:

  1. LoadProviders() is invoked synchronously so state.providers is populated from disk cache (or builtin fallback) before the function returns. Callers can then serve catalog requests immediately, even while the network fetch below is in flight.

  2. A non-blocking refresh kicks off in a goroutine — picks up any models.dev changes since the on-disk cache was baked. Failures here are logged but non-fatal: the cache from step 1 stays authoritative.

The ticker honors ctx.Done() and exits cleanly on shutdown.

func SupportsInputModality 

func SupportsInputModality(providerID, modelID, modality string) bool

SupportsInputModality returns true if the model supports the given input type. Returns true if model info is unavailable (optimistic fallback).

Types

type AttachmentPolicy 

type AttachmentPolicy struct {
	// MaxInlineBytesTotal caps the total base64-encoded attachment bytes in
	// a single request body. Parts that don't fit are evicted (oldest first)
	// and replaced with a text placeholder that tells the LLM how to
	// re-attach.
	MaxInlineBytesTotal int

	// SupportsURL is true when the provider accepts http(s):// URLs for
	// image file parts (a FilePart with an image/* type and a URL data
	// variant). Anthropic + Google also accept non-image files by URL (see
	// SupportsFileURL).
	SupportsURL bool

	// SupportsFileURL is true when the provider accepts http(s):// URLs for
	// non-image file parts (a FilePart with a URL data variant). Anthropic +
	// Google support PDFs and other file types by URL; OpenAI currently does
	// not on the chat path.
	SupportsFileURL bool

	// MaxURLImages caps how many image/file parts per request can be sent
	// as URLs before we fall back to inline. Shared across both kinds to
	// keep the provider from rejecting oversized fan-out fetches.
	MaxURLImages int

	// MaxURLBytesPerImage is the per-object size ceiling when sending URLs.
	// Objects above this are fetched and inlined (or evicted) rather than
	// referenced.
	MaxURLBytesPerImage int
}

AttachmentPolicy describes how a given provider/model accepts image and file attachments. Used by airlock's attachref resolver to decide between URL mode (presigned S3 link) and inline mode (base64-encoded bytes), and to enforce per-request size caps.

Defaults are conservative: inline-only, 5 MiB per-request budget. Provider overrides (see AttachmentOverlay) relax these where the upstream API is known to support URL references.

func PolicyFor 

func PolicyFor(providerID, modelID string) AttachmentPolicy

PolicyFor returns the attachment policy for a given provider/model pair. modelID is accepted for future model-level overrides but currently unused.

type CapabilitySet 

type CapabilitySet struct {
	Text          bool
	Vision        bool
	ImageGen      bool
	Search        bool
	Embedding     bool
	Speech        bool
	Transcription bool
	Reranking     bool
}

CapabilitySet is the set of high-level capabilities a model or provider offers. Derived from models.dev modalities, goai-supplied kind, and the overlay's extras.

func CapabilitiesFromModel 

func CapabilitiesFromModel(m ModelInfo) CapabilitySet

CapabilitiesFromModel derives the capability set for a single model from its kind (goai-sourced) plus its modality list (models.dev-sourced). Search is never set at the model level — it's a provider capability (see ProviderCapabilities).

Embedding is special-cased and detected by NAME: models.dev exposes no embedding modality or type, so an embedding model is indistinguishable from a text→text model by its catalog entry. Any model whose id/name contains "embed" is treated as embedding-only — it must not also report text, or it would surface as a chat model and land in the text/build/exec pickers. AllProviders() stamps Kind=embedding for the same models (the picker-grouping axis); this name check keeps the pure function correct on its own. A false positive (a chat model named "…embed…") just errors when called the wrong way at runtime — an acceptable trade for zero hand-maintained lists.

Other kind-derived caps fire whenever Kind is set; modality-derived caps fire independently — a Kind=Language model with image input still gets Vision. Modality-only classification is the fallback for catalog entries from providers without goai typed-list coverage.

func ProviderCapabilities 

func ProviderCapabilities(p *ModelsDevProvider) CapabilitySet

ProviderCapabilities unions per-model capabilities across every model in the provider, then ORs in "search" when the provider has a web-search backend (SearchBackend). Search is a provider feature, not derivable from any single model's modalities, so it's the one capability sourced outside the models.

A provider with no models is valid — e.g. brave, a search-only stub — in which case the result is just its search capability.

func (CapabilitySet) List 

func (c CapabilitySet) List() []string

List returns the set as a sorted slice of capability strings in the canonical UI order.

type ModelCapabilities 

type ModelCapabilities struct {
	// IsReasoningModel indicates if the model is a reasoning model
	IsReasoningModel bool

	// SystemMessageMode determines how system messages should be handled.
	// "system" - use standard system role
	// "developer" - convert to developer role (for reasoning models)
	// "remove" - remove system messages entirely
	SystemMessageMode string

	// DefaultReasoningEffort is the default reasoning effort level for this model.
	// Empty string means no default (non-reasoning model or special case like gpt-5-pro).
	DefaultReasoningEffort string

	// DefaultMaxOutputTokens is the recommended max output tokens for this model.
	// 0 means use the standard default.
	DefaultMaxOutputTokens int
}

ModelCapabilities describes capabilities of a model. This matches opencode's getResponsesModelConfig() in openai-responses-language-model.ts

func GetModelCapabilities 

func GetModelCapabilities(modelID string) ModelCapabilities

GetModelCapabilities returns capabilities for a given model ID. This matches opencode's getResponsesModelConfig() function.

type ModelCost 

type ModelCost struct {
	Input      float64 `json:"input"`
	Output     float64 `json:"output"`
	CacheRead  float64 `json:"cache_read,omitempty"`
	CacheWrite float64 `json:"cache_write,omitempty"`
}

ModelCost represents the cost structure for a model

type ModelInfo 

type ModelInfo struct {
	ID   string `json:"id"`
	Name string `json:"name"`
	// Kind classifies the model's primary purpose. Set by AllProviders()
	// from goai's typed lists (EmbeddingModels / ImageModels /
	// SpeechModels / TranscriptionModels / RerankingModels). Empty when
	// goai has no entry — clients treat empty as KindLanguage.
	Kind        ModelKind        `json:"kind,omitempty"`
	Family      string           `json:"family,omitempty"`
	ReleaseDate string           `json:"release_date,omitempty"`
	Attachment  bool             `json:"attachment,omitempty"`
	Reasoning   bool             `json:"reasoning,omitempty"`
	Temperature bool             `json:"temperature,omitempty"`
	ToolCall    bool             `json:"tool_call,omitempty"`
	Modalities  *ModelModalities `json:"modalities,omitempty"`
	Cost        *ModelCost       `json:"cost,omitempty"`
	Limit       *ModelLimit      `json:"limit,omitempty"`
	Status      string           `json:"status,omitempty"` // alpha, beta, deprecated
	// models.dev's `experimental` field was historically a bool but now
	// ships as an object (or absent) depending on the model. We don't
	// currently consume it; capturing it as RawMessage keeps the struct
	// schema-flexible — future consumers can json.Unmarshal it into
	// whatever typed shape they need without breaking the catalog parse
	// when upstream changes the wire shape again.
	Experimental json.RawMessage `json:"experimental,omitempty"`
}

ModelInfo represents a model from models.dev, optionally enriched with goai-supplied Kind classification by AllProviders().

func GetModelInfo 

func GetModelInfo(providerID, modelID string) (*ModelInfo, bool)

GetModelInfo returns model info for a specific provider and model

type ModelKind 

type ModelKind string

ModelKind classifies a model by its primary purpose. Sourced from goai's per-provider typed lists (Models / EmbeddingModels / ImageModels / SpeechModels / TranscriptionModels / RerankingModels). Empty means sol has no goai typed-list coverage for the provider — currently true for the openai-compat bucket (groq, xai, cerebras, fireworks, deepseek, perplexity, togetherai, etc.), all of which ship language models, so callers can safely treat empty as KindLanguage when filtering. 

const (
	KindLanguage      ModelKind = "language"
	KindEmbedding     ModelKind = "embedding"
	KindImage         ModelKind = "image"
	KindSpeech        ModelKind = "speech"        // text-to-speech
	KindTranscription ModelKind = "transcription" // speech-to-text
	KindReranking     ModelKind = "reranking"
)

type ModelLimit 

type ModelLimit struct {
	Context int `json:"context"`
	Input   int `json:"input,omitempty"`
	Output  int `json:"output"`
}

ModelLimit represents token limits for a model

type ModelModalities 

type ModelModalities struct {
	Input  []string `json:"input"`  // e.g. ["text", "image", "pdf", "audio", "video"]
	Output []string `json:"output"` // e.g. ["text"]
}

ModelModalities describes what input/output types a model supports.

func GetModalities 

func GetModalities(providerID, modelID string) *ModelModalities

GetModalities returns the model's input/output modalities. Returns nil if the model is not found or has no modalities defined.

func (*ModelModalities) SupportsInput 

func (m *ModelModalities) SupportsInput(modality string) bool

SupportsInput returns true if the model accepts the given input modality.

type ModelsDevProvider 

type ModelsDevProvider struct {
	ID     string               `json:"id"`
	Name   string               `json:"name"`
	API    string               `json:"api,omitempty"`
	NPM    string               `json:"npm,omitempty"`
	Env    []string             `json:"env"`
	Models map[string]ModelInfo `json:"models"`
}

ModelsDevProvider represents a provider from models.dev

func GetProviderInfo 

func GetProviderInfo(providerID string) (*ModelsDevProvider, bool)

GetProviderInfo returns provider info from the models.dev data

type Options 

type Options struct {
	APIKey  string
	BaseURL string
}

Options holds configuration for creating a provider.

type ProxyOptions 

type ProxyOptions struct {
	BaseURL string // Proxy server URL (e.g., "http://localhost:8080")
	Token   string // Bearer token for authentication
}

ProxyOptions holds configuration for creating a proxy-backed model.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.