provider

package
v0.36.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 14, 2026 License: MIT Imports: 26 Imported by: 0

Documentation

Index

Constants

View Source
const DefaultFreePoolURL = "https://raw.githubusercontent.com/prasenjeet-symon/ogcode/main/keys.json"

DefaultFreePoolURL is the public raw GitHub URL for the community key pool. It is served from the ogcode repo itself (repo-root keys.json on the default branch) so the pool lives alongside the code. It can be overridden via the OGCODE_FREE_KEYS_URL env var so forks or self-hosters can point at their own key pool (and so tests can point at a local server).

View Source
const DefaultOllamaBaseURL = "http://localhost:11434/v1"

DefaultOllamaBaseURL is the default local Ollama OpenAI-compatible endpoint.

View Source
const FreePoolTimeout = freePoolTimeout

FreePoolTimeout is the exported form for callers outside the provider package.

View Source
const OllamaCloudCatalogURL = "https://ollama.com/api/tags"

OllamaCloudCatalogURL lists the models hosted on Ollama Cloud.

This endpoint is UNDOCUMENTED: it is the same /api/tags path a local Ollama instance serves for its own models, served by ollama.com for the hosted catalog. It is public and needs no authentication. Because it is undocumented, a failure here is normal operating condition, never an error worth surfacing — callers must fall back to the static lists.

Variables

View Source
var AnthropicModels = []CatalogModel{

	{ID: "claude-opus-4-7", Name: "Claude Opus 4.7", ActiveByDefault: true, InputPricePerM: 15, OutputPricePerM: 75, SupportsImages: true, ContextWindow: 200000, MaxOutputTokens: 32000, Thinking: "adaptive"},
	{ID: "claude-opus-4-6", Name: "Claude Opus 4.6", ActiveByDefault: true, InputPricePerM: 15, OutputPricePerM: 75, SupportsImages: true, ContextWindow: 200000, MaxOutputTokens: 32000, Thinking: "adaptive"},
	{ID: "claude-sonnet-4-6", Name: "Claude Sonnet 4.6", ActiveByDefault: true, InputPricePerM: 3, OutputPricePerM: 15, SupportsImages: true, ContextWindow: 200000, MaxOutputTokens: 32000, Thinking: "adaptive"},
	{ID: "claude-haiku-4-5-20251001", Name: "Claude Haiku 4.5", ActiveByDefault: true, InputPricePerM: 0.80, OutputPricePerM: 4, SupportsImages: true, ContextWindow: 200000, MaxOutputTokens: 64000},

	{ID: "claude-opus-4-5-20251101", Name: "Claude Opus 4.5", ActiveByDefault: false, InputPricePerM: 15, OutputPricePerM: 75, SupportsImages: true, ContextWindow: 200000, MaxOutputTokens: 64000},
	{ID: "claude-opus-4-1-20250805", Name: "Claude Opus 4.1", ActiveByDefault: false, InputPricePerM: 15, OutputPricePerM: 75, SupportsImages: true, ContextWindow: 200000, MaxOutputTokens: 32000},
	{ID: "claude-sonnet-4-5-20250929", Name: "Claude Sonnet 4.5", ActiveByDefault: false, InputPricePerM: 3, OutputPricePerM: 15, SupportsImages: true, ContextWindow: 200000, MaxOutputTokens: 64000},

	{ID: "claude-opus-4-20250514", Name: "Claude Opus 4", ActiveByDefault: false, InputPricePerM: 15, OutputPricePerM: 75, SupportsImages: true, ContextWindow: 200000, MaxOutputTokens: 32000},
	{ID: "claude-sonnet-4-20250514", Name: "Claude Sonnet 4", ActiveByDefault: false, InputPricePerM: 3, OutputPricePerM: 15, SupportsImages: true, ContextWindow: 200000, MaxOutputTokens: 64000},
}

AnthropicModels is the authoritative list of Anthropic models. Maintained by contributors — see file header for instructions. All listed Claude models are multimodal and accept image input. All current Claude models expose a 200k-token context window by default.

View Source
var OpenAIModels = []CatalogModel{

	{ID: "gpt-5", Name: "GPT-5", ActiveByDefault: true, InputPricePerM: 10, OutputPricePerM: 30, SupportsImages: true, ContextWindow: 272000},
	{ID: "gpt-5-mini", Name: "GPT-5 Mini", ActiveByDefault: true, InputPricePerM: 1.50, OutputPricePerM: 6, SupportsImages: true, ContextWindow: 272000},
	{ID: "gpt-5-nano", Name: "GPT-5 Nano", ActiveByDefault: false, InputPricePerM: 0.10, OutputPricePerM: 0.40, SupportsImages: true, ContextWindow: 272000},

	{ID: "gpt-4.1", Name: "GPT-4.1", ActiveByDefault: true, InputPricePerM: 2, OutputPricePerM: 8, SupportsImages: true, ContextWindow: 1000000},
	{ID: "gpt-4.1-mini", Name: "GPT-4.1 Mini", ActiveByDefault: true, InputPricePerM: 0.40, OutputPricePerM: 1.60, SupportsImages: true, ContextWindow: 1000000},
	{ID: "gpt-4.1-nano", Name: "GPT-4.1 Nano", ActiveByDefault: false, InputPricePerM: 0.10, OutputPricePerM: 0.40, SupportsImages: true, ContextWindow: 1000000},

	{ID: "gpt-4o", Name: "GPT-4o", ActiveByDefault: false, InputPricePerM: 2.50, OutputPricePerM: 10, SupportsImages: true, ContextWindow: 128000},
	{ID: "gpt-4o-mini", Name: "GPT-4o Mini", ActiveByDefault: false, InputPricePerM: 0.15, OutputPricePerM: 0.60, SupportsImages: true, ContextWindow: 128000},

	{ID: "o4-mini", Name: "o4 Mini", ActiveByDefault: true, InputPricePerM: 1.10, OutputPricePerM: 4.40, SupportsImages: true, ContextWindow: 200000},
	{ID: "o3", Name: "o3", ActiveByDefault: true, InputPricePerM: 10, OutputPricePerM: 40, SupportsImages: true, ContextWindow: 200000},
	{ID: "o3-mini", Name: "o3 Mini", ActiveByDefault: false, InputPricePerM: 1.10, OutputPricePerM: 4.40, ContextWindow: 200000},
	{ID: "o1", Name: "o1", ActiveByDefault: false, InputPricePerM: 15, OutputPricePerM: 60, SupportsImages: true, ContextWindow: 200000},
	{ID: "o1-mini", Name: "o1 Mini", ActiveByDefault: false, InputPricePerM: 1.50, OutputPricePerM: 6, ContextWindow: 128000},
}

OpenAIModels is the authoritative list of OpenAI models. Maintained by contributors — see file header for instructions. SupportsImages marks multimodal models. The o*-mini reasoning models are text-only; the GPT-4o/4.1/5 families and o1/o3/o4-mini accept images. ContextWindow values are conservative (biased low where the published figure is uncertain): understating only makes compaction trigger slightly early, never overflow.

View Source
var PrimaryOllamaBaseURL = DefaultOllamaBaseURL

PrimaryOllamaBaseURL is the first candidate probed. A package-level var rather than a constant so tests can point it at a dead address and exercise the fallback path deterministically.

View Source
var ProviderPriority = []string{
	"anthropic", "openai", "openrouter", "ollama",
	"ogcode-openrouter", "ogcode-cerebras", "ogcode-sambanova",
	"ogcode-github_models", "ogcode-nvidia",
}

ProviderPriority is the stable order used to choose a default provider when a session does not specify a model.

User-configured first-party providers always win. Free-tier providers (keyed "ogcode-<collection>") are appended so the app works out-of-the-box with the community key pool, but never override a user's own credentials.

Functions

func AddFreePoolProviders added in v0.33.0

func AddFreePoolProviders(ctx context.Context, providers map[string]Provider)

AddFreePoolProviders provisions free-tier providers from the shared community key pool (a public GitHub-hosted JSON of OpenAI-compatible provider keys) into the given provider map, keyed "ogcode-<collection>". It gives ogcode a zero-friction out-of-the-box experience: the user can start chatting immediately on a free model without configuring anything.

Free providers never override a user's own credentials: entries already present in the map are skipped, and a user-configured "openai" provider pointing at the same collection's base URL suppresses its free duplicate. The fetch is best-effort (bounded by FreePoolTimeout) and cached locally, so offline launches still work; a failed fetch logs and leaves the map alone.

Both the server (loadProviderMap) and the headless CLI (run/index) call this so the free pool is available everywhere prompts can run.

func CollectionFromBaseURL added in v0.16.0

func CollectionFromBaseURL(baseURL string) string

CollectionFromBaseURL is the exported form of collectionFromBaseURL for use outside the provider package (e.g. server-side provider registration).

func FetchFreePool added in v0.16.0

func FetchFreePool(ctx context.Context) (map[string]FreeProviderDef, error)

FetchFreePool loads the community free-tier key pool. It is safe to call repeatedly — the first call fetches (or reads the cache); subsequent calls return the in-memory copy. The pool is refreshed in the background after the cache TTL expires, but a stale cache is always returned immediately so startup is never blocked on the network.

Returns nil when neither a fetch nor a cache is available (graceful degradation: the caller treats this as "no free providers").

func FreeProviderIDs added in v0.16.0

func FreeProviderIDs(defs map[string]FreeProviderDef) []string

FreeProviderIDs returns the registry IDs for all free-pool providers in a stable priority order (OpenRouter first — the recommended default free provider).

func HasFreeProviders added in v0.16.0

func HasFreeProviders() bool

HasFreeProviders reports whether the free pool is available (loaded and non-empty). Used by the onboarding gate to decide whether to skip the credential wizard.

func IsContextLengthMessage added in v0.21.0

func IsContextLengthMessage(s string) bool

IsContextLengthMessage reports whether a message/body indicates the prompt exceeded the model's context window. Shared by APIError.IsContextLength and the loop's string-matching fallback so the two never drift. The hints must stay a superset of the phrasings contextWindowPatterns matches — the loop learns a window from exactly the bodies this function classifies, so a pattern whose sample body is not classified is dead code (pinned by TestParseContextWindowFromBody_PhrasingsClassifyAsOverflow).

func IsImageRejectionMessage added in v0.29.0

func IsImageRejectionMessage(s string) bool

IsImageRejectionMessage reports whether a message/body indicates the model does not accept image input. Shared by APIError.IsImageRejection, the probe's classifyProbeError, and the loop's string-matching fallback so they never drift.

func OllamaBinaryInstalled added in v0.16.0

func OllamaBinaryInstalled() bool

OllamaBinaryInstalled reports whether the `ollama` executable is on $PATH. Uses exec.LookPath (cross-platform) rather than probing a fixed set of install directories.

func OllamaRunning added in v0.16.0

func OllamaRunning(baseURL string) bool

OllamaRunning probes the Ollama server at the given base URL (or the default localhost endpoint when empty) with a short timeout. Returns true when the server responds with HTTP 200. The probe is best-effort: any transport error or non-200 status is treated as "not running".

func ParseContextWindowFromBody added in v0.35.0

func ParseContextWindowFromBody(body string) int

ParseContextWindowFromBody extracts the model's context window from a context-overflow error body — the one message a provider reliably states the figure in ("maximum context length is 8192 tokens", "195000 tokens > 200000 maximum"). Returns 0 when nothing plausible is found; the caller knows the request that just overflowed, so the sanity check that the window exceeds the prompt size lives at the call site, not here. Shared seam: the loop calls this on exactly the errors isContextLengthError classified, so the phrasings below must stay a superset of IsContextLengthMessage's hints.

func PreferLiveOllamaEndpoint added in v0.26.0

func PreferLiveOllamaEndpoint(configured string, st OllamaStatus) string

PreferLiveOllamaEndpoint resolves which endpoint to actually use when a base URL was configured (a persisted config row) and detection found a different live one.

A configured endpoint wins while it is still answering. A configured endpoint that has gone dead yields to whatever detection found, so a row persisted from an earlier launch cannot permanently shadow a working endpoint — the "local Ollama was uninstalled, but a router is up" case.

An explicit OLLAMA_BASE_URL must never be passed here: it is authoritative by definition and callers should use it directly.

func ProbeImageSupport added in v0.6.0

func ProbeImageSupport(ctx context.Context, p Provider, modelID string) (supports bool, definitive bool, err error)

ProbeImageSupport sends a single minimal image to the model and reports whether it was accepted. Return values:

  • (true, true, nil): the model accepted the image and responded.
  • (false, true, nil): the provider rejected the request for an image/modality reason.
  • (false, false, err): inconclusive (network/auth/rate-limit/etc.) — do NOT cache; retry later.

func ResetFreePoolForTest added in v0.16.0

func ResetFreePoolForTest()

ResetFreePoolForTest clears the singleton free pool state. Test-only — used to isolate server/provider tests from the global pool so they don't pick up providers loaded by the freepool unit tests in the same process.

func ValidateCredentials added in v0.10.0

func ValidateCredentials(ctx context.Context, providerID, apiKey, baseURL string) error

ValidateCredentials makes a minimal chat request with the given credentials to confirm the provider accepts them. It returns nil when the credentials work, or an error describing the failure. Used by the settings/onboarding "test key" flow. The caller is responsible for any timeout via ctx.

Types

type APIError added in v0.21.0

type APIError struct {
	Provider   string
	StatusCode int
	RetryAfter time.Duration // parsed from the Retry-After header; 0 if absent
	Body       string
}

APIError is a structured error returned by a provider when the LLM API responds with a non-2xx status. It carries the HTTP status code and any Retry-After hint so the agent loop can classify the failure and back off precisely instead of sniffing the error string. Error() preserves the historical "<provider> API error <code>: <body>" format, so existing logs and the loop's string-matching fallbacks keep working for non-HTTP (stream/network) errors.

func NewAPIError added in v0.21.0

func NewAPIError(providerID string, resp *http.Response, body string) *APIError

NewAPIError builds an APIError from a non-2xx HTTP response, parsing the Retry-After header. The caller supplies the already-read body.

func (*APIError) Error added in v0.21.0

func (e *APIError) Error() string

func (*APIError) IsContextLength added in v0.21.0

func (e *APIError) IsContextLength() bool

IsContextLength reports whether the error is a context-window overflow. These arrive as 400s whose body mentions the context length. A bare 400 with an empty body counts too, but only from Ollama, which is the one endpoint that answers an overflowing prompt that way. Every other provider explains its 400s, so treating a body-less one as overflow there just mislabels an unrelated rejection — and sends the user off compacting a conversation that was never too big.

func (*APIError) IsImageRejection added in v0.29.0

func (e *APIError) IsImageRejection() bool

IsImageRejection reports whether the error is a 400 caused by the model not accepting image input. The body of such a response mentions image/modality/ vision support. Used by the agent loop to make these failures resumable (resume strips the offending images) instead of fatal, since a non-vision model producing a tool image is a capability mismatch, not a malformed request — switching models or retrying without images fixes it.

func (*APIError) IsTransient added in v0.21.0

func (e *APIError) IsTransient() bool

IsTransient reports whether the status is worth retrying: 429 (rate limited), 529 (Anthropic overloaded), or any 5xx.

type AnthropicProvider

type AnthropicProvider struct {
	// contains filtered or unexported fields
}

AnthropicProvider implements Provider for the Anthropic Messages API.

func NewAnthropicProvider

func NewAnthropicProvider() *AnthropicProvider

func (*AnthropicProvider) ID

func (p *AnthropicProvider) ID() string

func (*AnthropicProvider) Models

func (p *AnthropicProvider) Models() []ModelInfo

func (*AnthropicProvider) StreamChat

func (p *AnthropicProvider) StreamChat(ctx context.Context, req StreamRequest) (<-chan StreamEvent, error)

type BaseURLReporter added in v0.26.1

type BaseURLReporter interface {
	BaseURL() string
}

BaseURLReporter is implemented by providers that talk to a configurable endpoint. Provider ID alone cannot answer the caching question, because newOpenAICompatible serves "openai", "openrouter" and "ollama" from one implementation with a swappable base URL — and that URL can be repointed at any OpenAI-shaped service. The endpoint is the thing that caches, not the slot it is configured in.

type CacheObserver added in v0.26.1

type CacheObserver struct {
	// contains filtered or unexported fields
}

CacheObserver resolves the verdict for endpoints StaticCacheVerdict cannot answer, by watching what comes back on the wire. Observed evidence outranks any table we could ship: it survives a user repointing a base URL at a different service, and needs no per-model catalog upkeep.

The zero value is ready to use and reports CacheUnknown.

func NewCacheObserver added in v0.26.1

func NewCacheObserver(p Provider) *CacheObserver

NewCacheObserver seeds an observer from the static verdict. When identity already settles the question the observer is final immediately and never changes its mind, so no observation budget is spent.

func SettledCacheObserver added in v0.26.1

func SettledCacheObserver(v CacheVerdict) *CacheObserver

SettledCacheObserver returns an observer that already holds v and will not change its mind. Used to carry a verdict established in an earlier turn into a new one without re-running the observation window.

func (*CacheObserver) Observe added in v0.26.1

func (o *CacheObserver) Observe(cacheReadTokens, cacheWriteTokens int, repeatedPrefix bool)

Observe folds in one step's reported cache usage.

repeatedPrefix reports whether this request actually shared a prefix with the previous one — false on the first step of a turn, and after a compaction has rewritten the history. Steps without a repeated prefix carry no evidence either way and are ignored, so a turn that compacts early cannot mislead the observer into declaring a caching endpoint non-caching.

func (*CacheObserver) Verdict added in v0.26.1

func (o *CacheObserver) Verdict() CacheVerdict

Verdict returns the current answer. It stays CacheUnknown until there is enough evidence, and callers must treat unknown as "do not offer compaction yet" rather than as either verdict.

type CacheVerdict added in v0.26.1

type CacheVerdict string

CacheVerdict reports whether re-sending a request's prefix to an endpoint is cheap. It drives whether the agent is offered a way to compact its own context mid-turn: compaction pays for itself only when every step re-pays full price for the whole accumulated prefix.

const (
	// CacheUnknown means the endpoint has not answered yet. Callers should keep
	// observing rather than assume either way.
	CacheUnknown CacheVerdict = "unknown"
	// CacheSupported means a repeated prefix is served from a cache — billed at
	// a discount, or not billed at all. Compacting here is a net loss: it
	// invalidates the cache, so the next request re-establishes the whole prefix
	// at full price.
	CacheSupported CacheVerdict = "caching"
	// CacheAbsent means every step re-pays full price for the entire prefix.
	// This is where in-turn compaction is worth its round trip.
	CacheAbsent CacheVerdict = "none"
)

func StaticCacheVerdict added in v0.26.1

func StaticCacheVerdict(p Provider) CacheVerdict

StaticCacheVerdict answers from provider identity where identity is actually sufficient, and returns CacheUnknown everywhere else so the caller falls back to observing what the endpoint reports.

Ollama is CacheAbsent unconditionally, local endpoint or not. An earlier version keyed this on the base URL and got it backwards, because a local Ollama is a router as much as a runtime: a model named with the cloud marker (":cloud" or "-cloud", see cloudModelID) is forwarded to the hosted backend and billed per token, while the request still goes to http://localhost:11434. The URL says local; the invoice says otherwise. Reading the model name here instead would work, but there is no case where withholding compaction from Ollama is the better trade: a genuinely local model reuses its KV cache but runs in a context window small enough that reclaiming space is worth more than the re-prefill it costs.

type CatalogModel added in v0.2.3

type CatalogModel struct {
	ID              string
	Name            string
	ActiveByDefault bool
	InputPricePerM  float64 // USD per 1M input tokens (0 = unknown)
	OutputPricePerM float64 // USD per 1M output tokens (0 = unknown)
	SupportsImages  bool    // whether the model accepts image input
	ContextWindow   int     // total context length in tokens (0 = unknown → byte-size fallback)
	// MaxOutputTokens is the most output the model produces in one response.
	// 0 means unknown: no explicit limit is sent and the provider's own default
	// applies. NEVER guess this upward — a value above the model's real ceiling
	// makes every request fail, so understate it when a published figure is not
	// at hand. Only Anthropic entries carry a value today: the OpenAI-compatible
	// path (also used for OpenRouter and Ollama) would send it as `max_tokens`,
	// which the o-series and GPT-5 reasoning models reject in favour of
	// `max_completion_tokens`, so those deliberately stay at 0.
	MaxOutputTokens int
	// Thinking names the reasoning mode to request for this model, for providers
	// that must ask for it explicitly. "adaptive" is what Claude 4.6 and later
	// accept: the model decides when and how deeply to think, and reasons
	// between tool calls on its own with no beta header.
	//
	// Empty means no thinking configuration is sent. Claude 4.5 and earlier
	// accept only a fixed `budget_tokens` that has to fit inside `max_tokens` —
	// a tradeoff the agent loop does not currently make, since it leaves
	// `max_tokens` at the provider default — and Haiku 4.5 cannot reason
	// between tool calls at all, which is where an agent loop would spend it.
	Thinking string
}

CatalogModel is a statically-known model for a provider that does not expose a live /v1/models discovery endpoint.

func CatalogModelByID added in v0.31.0

func CatalogModelByID(id string) (CatalogModel, bool)

CatalogModelByID finds a model across the static catalogs above.

Only Anthropic and OpenAI are covered. OpenRouter and Ollama discover their model lists at runtime and carry no pricing here, so callers must treat a false return as "unknown", not "free".

type ContentPart

type ContentPart struct {
	Type string `json:"type"`
	Text string `json:"text,omitempty"`
}

type FreeProviderDef added in v0.16.0

type FreeProviderDef struct {
	Collection   string   `json:"collection"`   // grouping label ("Cerebras", "SambaNova", …)
	BaseURL      string   `json:"baseURL"`      // OpenAI-compatible API base URL
	Keys         []string `json:"keys"`         // pool of API keys (round-robin / random)
	DefaultModel string   `json:"defaultModel"` // suggested default model ID
}

FreeProviderDef describes one OpenAI-compatible free-tier provider sourced from the shared community key pool. The pool is a JSON file hosted on a public GitHub repo so keys can be rotated centrally without a binary release.

func FreeProviderList added in v0.16.0

func FreeProviderList() []FreeProviderDef

FreeProviderList returns the active free-pool provider collection names in priority order, for the UI. Empty when the pool is not available.

type MessageImage added in v0.6.0

type MessageImage struct {
	MediaType string `json:"mediaType"`
	Data      string `json:"data"`
}

MessageImage is an image attached to a message, carried provider-neutrally. Data is base64-encoded image bytes; MediaType is e.g. "image/jpeg".

type ModelInfo

type ModelInfo struct {
	ID              string  `json:"id"`
	Name            string  `json:"name"`
	ProviderID      string  `json:"providerId"`
	Default         bool    `json:"default"`
	ActiveByDefault bool    `json:"activeByDefault"`
	InputPricePerM  float64 `json:"inputPricePerM"`
	OutputPricePerM float64 `json:"outputPricePerM"`
	SupportsImages  bool    `json:"supportsImages"`
	// ContextWindow is the model's total context length in tokens (0 = unknown).
	// Used to size the compaction trigger; when 0 the loop falls back to a fixed
	// byte-size heuristic.
	ContextWindow int `json:"contextWindow,omitempty"`
	// MaxOutputTokens is the most output the model will produce in one response
	// (0 = unknown, leave the request's limit to the provider's own default).
	MaxOutputTokens int `json:"maxOutputTokens,omitempty"`
	// Collection is an optional grouping label for dynamically-fetched models
	// from OpenAI-compatible providers (e.g. "DeepSeek", "Gemini") so the UI can
	// group them instead of collapsing everything under the OpenAI provider id.
	Collection string `json:"collection,omitempty"`
}

func FetchOllamaCloudCatalog added in v0.26.0

func FetchOllamaCloudCatalog(ctx context.Context, baseURL string) ([]ModelInfo, error)

FetchOllamaCloudCatalog returns the models hosted on Ollama Cloud, named for the endpoint they will be requested through: bare names when baseURL is ollama.com itself, "-cloud"-suffixed when going through a local or proxied instance.

Cloud models do not need to be pulled — a signed-in instance resolves them remotely on first use — so every entry is immediately usable. That is the difference from a locally-listed model, which must exist on disk.

The result is sorted smallest-first (entries with no size reported sort last), so a caller that needs to pick a cheap default can take the head of the list without carrying size data of its own.

type ModelMessage

type ModelMessage struct {
	Role       string          `json:"role"`
	Content    json.RawMessage `json:"content,omitempty"`
	ToolCalls  json.RawMessage `json:"tool_calls,omitempty"`
	ToolCallID string          `json:"tool_call_id,omitempty"`
	Name       string          `json:"name,omitempty"`
	// Images carries image attachments for a tool-result message. Providers
	// render these per their API: Anthropic embeds them in the tool_result
	// content block; OpenAI-family inject a follow-up user message.
	Images []MessageImage `json:"images,omitempty"`
	// ReasoningParts carries thinking/reasoning blocks from a previous assistant
	// turn. Anthropic requires these to be forwarded back as "thinking" content
	// blocks with their signatures intact; OpenAI-family providers handle
	// reasoning tokens server-side and should ignore this field.
	ReasoningParts []ReasoningPart `json:"reasoningParts,omitempty"`
}

type ModelRefresher added in v0.1.3

type ModelRefresher interface {
	RefreshModels()
}

ModelRefresher is an optional interface that providers can implement to support dynamic model list refreshing.

type OllamaStatus added in v0.16.0

type OllamaStatus struct {
	// Installed reports whether the `ollama` binary is found on $PATH (via
	// exec.LookPath — cross-platform, unlike hardcoded install paths).
	Installed bool `json:"installed"`
	// Running reports whether the Ollama server responded to a health probe
	// (GET http://localhost:11434 with a short timeout). This is the reliable
	// signal that the endpoint is actually usable right now.
	Running bool `json:"running"`
	// BaseURL is the detected/expected Ollama base URL. It honours
	// OLLAMA_BASE_URL when set, otherwise defaults to the localhost endpoint.
	BaseURL string `json:"baseUrl"`
}

OllamaStatus describes the runtime detection state of a local Ollama install. It is computed by DetectOllama and surfaced to the frontend so the onboarding gate can treat a running instance as already configured.

func DetectOllama added in v0.16.0

func DetectOllama() OllamaStatus

DetectOllama performs a combined detection: binary presence + liveness probe. This is the single source of truth used by both the server (loadProviderMap / provider config endpoint) and the CLI (index command) so detection logic is never duplicated.

Resolution order:

  1. OLLAMA_BASE_URL, when set, is authoritative — we never probe elsewhere when the user has named a target.
  2. The default local endpoint (localhost:11434).
  3. The fallback candidates, so a machine with no local Ollama install still finds a router or proxy serving remote instances.

type OpenAIProvider

type OpenAIProvider struct {
	// contains filtered or unexported fields
}

OpenAIProvider implements Provider for the OpenAI Chat Completions API. Also used for OpenRouter and Ollama (same API format, different base URL). When configured for an OpenAI-compatible third party (DeepSeek, Gemini, Groq, …) via a custom base URL, the `collection` field tags dynamically-fetched models so the UI can group them instead of collapsing them under "openai".

func NewEmbedProvider added in v0.2.1

func NewEmbedProvider(providerID, apiKey, model string) (*OpenAIProvider, error)

NewEmbedProvider creates an OpenAIProvider configured for embedding. providerID must be "openai", "openrouter", or "ollama". If apiKey is non-empty it overrides the env var key. If model is non-empty it is stored as the provider model (used for embedding). Deprecated: Use NewEmbedProviderWithConfig for full control over baseURL.

func NewEmbedProviderWithConfig added in v0.8.1

func NewEmbedProviderWithConfig(providerID, apiKey, model, baseURL string) (*OpenAIProvider, error)

NewEmbedProviderWithConfig creates an OpenAIProvider configured for embedding with optional apiKey, model, and baseURL overrides. Env-var values are used as the base; non-empty parameters override them.

func NewFreePoolProvider added in v0.16.0

func NewFreePoolProvider(def FreeProviderDef) (*OpenAIProvider, error)

NewFreePoolProvider creates an OpenAI-compatible Provider instance for a free-tier entry from the key pool. The provider ID is keyed by the pool's collection (e.g. "ogcode-cerebras") so multiple free providers coexist in the registry as separately selectable instances — but every model they serve is tagged with the shared freePoolCollection ("ogcode") label so they all group together in the UI, apart from the user's own providers.

func NewOllamaProvider

func NewOllamaProvider() *OpenAIProvider

NewOllamaProvider creates an OpenAI-compatible provider for Ollama. When OLLAMA_BASE_URL points to a cloud endpoint (not localhost), the model list is fetched dynamically from /v1/models. For local Ollama, a static fallback list is used.

func NewOpenAIProvider

func NewOpenAIProvider() *OpenAIProvider

func NewOpenRouterProvider

func NewOpenRouterProvider() *OpenAIProvider

NewOpenRouterProvider creates an OpenAI-compatible provider for OpenRouter.

func (*OpenAIProvider) BaseURL added in v0.16.0

func (p *OpenAIProvider) BaseURL() string

BaseURL returns the API base URL the provider is configured to use. Exposed so callers (e.g. the free-pool registration) can compare endpoints without reaching into the unexported field directly.

func (*OpenAIProvider) Embed added in v0.2.1

func (p *OpenAIProvider) Embed(ctx context.Context, inputs []string) ([][]float32, error)

func (*OpenAIProvider) EmbedModel added in v0.2.1

func (p *OpenAIProvider) EmbedModel() string

func (*OpenAIProvider) ID

func (p *OpenAIProvider) ID() string

func (*OpenAIProvider) Models

func (p *OpenAIProvider) Models() []ModelInfo

func (*OpenAIProvider) RefreshModels added in v0.1.3

func (p *OpenAIProvider) RefreshModels()

RefreshModels clears the cached model list so the next call to Models() will re-fetch from the endpoint (for cloud providers). Not safe to call concurrently with Models().

func (*OpenAIProvider) StreamChat

func (p *OpenAIProvider) StreamChat(ctx context.Context, req StreamRequest) (<-chan StreamEvent, error)

type Provider

type Provider interface {
	ID() string
	Models() []ModelInfo
	StreamChat(ctx context.Context, req StreamRequest) (<-chan StreamEvent, error)
}

func NewProviderWithConfig added in v0.2.1

func NewProviderWithConfig(providerID, apiKey, baseURL string) (Provider, error)

NewProviderWithConfig creates a Provider with explicit credentials, used when credentials come from the DB rather than environment variables. providerID must be "anthropic", "openai", "openrouter", or "ollama". Env-var values are used as the base; apiKey and baseURL override them when non-empty.

type ReasoningPart added in v0.17.1

type ReasoningPart struct {
	Text      string `json:"text"`
	Signature string `json:"signature,omitempty"`
	// RedactedData is the opaque payload of a redacted_thinking block. When
	// set, the block carries no readable text and must be re-sent as a
	// redacted_thinking block rather than a thinking block.
	RedactedData string `json:"redactedData,omitempty"`
}

ReasoningPart represents a thinking/reasoning block from a model's response. Anthropic models return these with a cryptographic signature that must be forwarded back unchanged on subsequent turns.

type Registry

type Registry struct {
	// contains filtered or unexported fields
}

func NewRegistry

func NewRegistry() *Registry

func (*Registry) ContextWindow added in v0.21.0

func (r *Registry) ContextWindow(modelID string) int

ContextWindow returns the model's total context length in tokens, or 0 when unknown (dynamically-fetched models without catalog metadata). Callers treat 0 as "fall back to a size heuristic".

func (*Registry) Default added in v0.10.0

func (r *Registry) Default() Provider

Default returns the highest-priority registered provider, or nil if the registry has no providers.

func (*Registry) DefaultUsable added in v0.33.0

func (r *Registry) DefaultUsable() Provider

DefaultUsable returns the provider a fresh prompt should run on, applying the same priority as Default but refusing to hand back an installed-but-stopped Ollama when anything else is available. A registered ollama provider only means "the binary exists (or a base URL was saved)" — if the daemon is down and OLLAMA_API_KEY is unset, the first prompt would die with connection refused, and ollama's presence in ProviderPriority would otherwise shadow the community free pool and every usable provider behind it.

An ollama provider is considered usable when OLLAMA_API_KEY is set or the daemon answers a probe. Non-ollama providers are always considered usable; unknown ollama implementations are assumed usable, while the concrete *OpenAIProvider is probed. When ollama is the only registered provider it is returned even if unreachable — ollama-only users otherwise lose their only option, and the provider surfaces the real connection error itself.

func (*Registry) Get

func (r *Registry) Get(id string) Provider

func (*Registry) IsCustomModel added in v0.32.0

func (r *Registry) IsCustomModel(modelID string) bool

IsCustomModel reports whether modelID was registered as a user-added custom model (rather than a built-in catalog or dynamically-fetched model). Custom models are not present in any provider's curated catalog, so capability lookups that trust the catalog must treat them as unknown and probe instead.

func (*Registry) List

func (r *Registry) List() []string

func (*Registry) ListModels

func (r *Registry) ListModels() []ModelInfo

func (*Registry) MaxOutputTokens added in v0.26.1

func (r *Registry) MaxOutputTokens(modelID string) int

MaxOutputTokens returns the model's output ceiling in tokens, or 0 when unknown. Callers treat 0 as "send no explicit limit and let the provider apply its own default" — overstating a ceiling makes every request fail, so unknown must never be guessed upward.

func (*Registry) ModelSupportsImages added in v0.6.0

func (r *Registry) ModelSupportsImages(modelID string) bool

ModelSupportsImages reports whether the given model accepts image input. Unknown models default to false.

func (*Registry) RefreshModels added in v0.1.3

func (r *Registry) RefreshModels()

RefreshModels clears cached model lists for all providers that support it, forcing re-fetch on next Models() call.

func (*Registry) Register

func (r *Registry) Register(p Provider)

func (*Registry) RegisterCustomModel

func (r *Registry) RegisterCustomModel(modelID, providerID string)

func (*Registry) ReplaceProviders added in v0.10.0

func (r *Registry) ReplaceProviders(providers map[string]Provider)

ReplaceProviders atomically swaps the set of registered providers. Custom model routing (RegisterCustomModel) is preserved. Used to apply provider credential changes from the settings/onboarding UI without a server restart.

func (*Registry) ResolveProvider

func (r *Registry) ResolveProvider(modelID string) Provider

func (*Registry) UnregisterCustomModel

func (r *Registry) UnregisterCustomModel(modelID string)

type StreamEvent

type StreamEvent struct {
	Type         StreamEventType `json:"type"`
	Text         string          `json:"text,omitempty"`
	Signature    string          `json:"signature,omitempty"`
	RedactedData string          `json:"redactedData,omitempty"`
	ToolCallID   string          `json:"toolCallId,omitempty"`
	ToolName     string          `json:"toolName,omitempty"`
	ToolInput    json.RawMessage `json:"toolInput,omitempty"`
	FinishReason *string         `json:"finishReason,omitempty"`
	Usage        *TokenUsage     `json:"usage,omitempty"`
	Error        string          `json:"error,omitempty"`
}

type StreamEventType

type StreamEventType string
const (
	EventTextDelta     StreamEventType = "text-delta"
	EventToolCallStart StreamEventType = "tool-call-start"
	EventToolCallDelta StreamEventType = "tool-call-delta"
	EventToolCallEnd   StreamEventType = "tool-call-end"
	// EventReasoningStart opens a thinking block. It carries no content: it
	// marks the boundary between one block and the next, so blocks are stored
	// and replayed separately rather than concatenated.
	EventReasoningStart     StreamEventType = "reasoning-start"
	EventReasoning          StreamEventType = "reasoning"
	EventReasoningSignature StreamEventType = "reasoning-signature"
	// EventReasoningRedacted carries a safety-redacted thinking block. It has
	// no readable text — only an opaque payload that must be round-tripped
	// verbatim as a redacted_thinking block, so it is its own event rather
	// than a signature on an empty reasoning block.
	EventReasoningRedacted StreamEventType = "reasoning-redacted"
	EventFinish            StreamEventType = "finish"
	EventUsage             StreamEventType = "usage"
	EventError             StreamEventType = "error"
)

type StreamRequest

type StreamRequest struct {
	Model       string           `json:"model"`
	System      []string         `json:"system"`
	Messages    []ModelMessage   `json:"messages"`
	Tools       []ToolDefinition `json:"tools"`
	Temperature float64          `json:"temperature,omitempty"`
	MaxTokens   int              `json:"maxTokens,omitempty"`
	// Thinking asks for the model's reasoning mode, where the provider and the
	// model support one. Only the agent loop sets it. The short utility calls —
	// titles, the auto-mode risk gate, compaction — run on tight max_tokens
	// budgets that thinking would spend before reaching an answer, and none of
	// them is the kind of work reasoning improves.
	Thinking bool `json:"thinking,omitempty"`
}

type TokenUsage

type TokenUsage struct {
	InputTokens      int `json:"inputTokens,omitempty"`
	OutputTokens     int `json:"outputTokens,omitempty"`
	ReasoningTokens  int `json:"reasoningTokens,omitempty"`
	CacheReadTokens  int `json:"cacheReadTokens,omitempty"`
	CacheWriteTokens int `json:"cacheWriteTokens,omitempty"`
}

TokenUsage carries per-message token accounting from a provider. Fields are non-zero where the provider reports them.

type ToolDefinition

type ToolDefinition struct {
	Name        string          `json:"name"`
	Description string          `json:"description"`
	Parameters  json.RawMessage `json:"parameters"`
}

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL