Documentation
¶
Overview ¶
Package llm talks to the AI model a workspace connects (design expansion, phase 3): Anthropic, OpenAI, Google Gemini, Azure OpenAI and servers that speak the OpenAI API (Ollama, vLLM, LM Studio). It is plain HTTP with the standard library: each provider is a small adapter that turns a Request into the provider's JSON and its answer (or error) back into a Response.
Every adapter passes the shared contract suite in llm/contracttest against recorded answers; tests never reach the network.
Requests have a timeout and a response size limit, and the client the caller passes decides which addresses may be dialed (internal/netguard). What goes into a Request is the caller's business: internal/ai builds prompts from templates with the person's content delimited, and never puts contact data in them.
Index ¶
- Constants
- Variables
- func FirstJSON(s string) (json.RawMessage, bool)
- func ImageInput(provider, model string) bool
- func ModelName(provider, model string) string
- func Valid(kind string) bool
- type Config
- type Error
- type ErrorKind
- type Image
- type KnownModel
- type Message
- type Model
- type Part
- type Provider
- type ProviderInfo
- type Request
- type Response
Constants ¶
const ( CapText = "text" CapImageInput = "image_input" CapStructuredOutput = "structured_output" CapOwnNetwork = "own_network" CapDataRegion = "data_region" )
Capabilities a provider card shows.
const ( Anthropic = "anthropic" OpenAI = "openai" Google = "google" Azure = "azure" Compatible = "compatible" )
Provider kinds.
const ( DefaultTimeout = 60 * time.Second DefaultMaxResponseBytes = 4 << 20 DefaultMaxTokens = 2048 // DefaultAzureAPIVersion is Azure OpenAI's generally available API // version for chat completions with images and JSON output. DefaultAzureAPIVersion = "2024-10-21" )
Defaults for requests.
const ( StopEnd = "end" StopLength = "length" StopRefusal = "refusal" StopOther = "other" )
Stop reasons, normalized.
Variables ¶
var Kinds = []string{Anthropic, OpenAI, Google, Azure, Compatible}
Kinds lists the providers in the catalog's order.
var PriceList = []KnownModel{ {Provider: Anthropic, ID: "claude-sonnet-5-5", Name: "Claude Sonnet 5.5", Input: 2, Output: 10, ImageInput: true}, {Provider: Anthropic, ID: "claude-haiku-4-5", Name: "Claude Haiku 4.5", Input: 1, Output: 5, ImageInput: true}, {Provider: Anthropic, ID: "claude-sonnet-4-5", Name: "Claude Sonnet 4.5", Input: 3, Output: 15, ImageInput: true}, {Provider: Anthropic, ID: "claude-opus-4-5", Name: "Claude Opus 4.5", Input: 5, Output: 25, ImageInput: true}, {Provider: OpenAI, ID: "gpt-5", Name: "GPT-5", Input: 1.25, Output: 10, ImageInput: true}, {Provider: OpenAI, ID: "gpt-5-mini", Name: "GPT-5 mini", Input: 0.25, Output: 2, ImageInput: true}, {Provider: OpenAI, ID: "gpt-4.1", Name: "GPT-4.1", Input: 2, Output: 8, ImageInput: true}, {Provider: OpenAI, ID: "gpt-4o-mini", Name: "GPT-4o mini", Input: 0.15, Output: 0.6, ImageInput: true}, {Provider: Google, ID: "gemini-2.5-pro", Name: "Gemini 2.5 Pro", Input: 1.25, Output: 10, ImageInput: true}, {Provider: Google, ID: "gemini-2.5-flash", Name: "Gemini 2.5 Flash", Input: 0.3, Output: 2.5, ImageInput: true}, {Provider: Google, ID: "gemini-2.5-flash-lite", Name: "Gemini 2.5 Flash-Lite", Input: 0.1, Output: 0.4, ImageInput: true}, }
PriceList holds the prices pre-filled when a model is picked. People edit them when their contract differs; a model that isn't listed starts at 0.
PriceListDate is the date of the price list below, the one ReputeCast's release ships with.
var Providers = []ProviderInfo{ {Kind: Anthropic, Name: "Anthropic", NeedsKey: true, Capabilities: []string{CapText, CapImageInput, CapStructuredOutput}, DefaultModel: "claude-sonnet-5-5"}, {Kind: OpenAI, Name: "OpenAI", NeedsKey: true, Capabilities: []string{CapText, CapImageInput, CapStructuredOutput}, DefaultModel: "gpt-5"}, {Kind: Google, Name: "Google Gemini", NeedsKey: true, Capabilities: []string{CapText, CapImageInput, CapStructuredOutput}, DefaultModel: "gemini-2.5-flash"}, {Kind: Azure, Name: "Azure OpenAI", NeedsKey: true, NeedsBaseURL: true, Capabilities: []string{CapText, CapImageInput, CapDataRegion}}, {Kind: Compatible, Name: "OpenAI-compatible", NeedsBaseURL: true, Capabilities: []string{CapText, CapOwnNetwork}}, }
Providers is the catalog, in order.
Functions ¶
func FirstJSON ¶
func FirstJSON(s string) (json.RawMessage, bool)
FirstJSON returns the first JSON object in s: models asked for JSON sometimes wrap it in a code fence or a sentence.
func ImageInput ¶
ImageInput reports whether a model reads images: the hosted providers' current models do; an open model does when its name says so.
Types ¶
type Config ¶
type Config struct {
Kind string
// BaseURL overrides the provider's API root: the Azure resource endpoint
// (https://acme.openai.azure.com) or a compatible server's address with
// its /v1 (http://ollama.acme.internal:11434/v1). Empty is the
// provider's public API.
BaseURL string
// APIKey is optional for compatible servers.
APIKey string
Model string
// Deployment and APIVersion are Azure OpenAI's.
Deployment string
APIVersion string
// HTTP is the client to use (the SSRF guard's in production).
HTTP *http.Client
// Timeout bounds one request, retries included (DefaultTimeout).
Timeout time.Duration
// MaxResponseBytes caps the answer read (DefaultMaxResponseBytes).
MaxResponseBytes int64
// Retries is how many times an overloaded or unavailable provider is
// asked again (1 by default: "returned an error twice"); RetryWait is
// the pause before each retry (1 s).
Retries *int
RetryWait time.Duration
}
Config says how to reach a model.
type Error ¶
type Error struct {
Kind ErrorKind
Status int
// Code is the provider's error type ("authentication_error",
// "rate_limit_exceeded", "RESOURCE_EXHAUSTED").
Code string
// Message is the provider's message ("invalid x-api-key").
Message string
RetryAfter time.Duration
// Attempts is how many times the request was sent.
Attempts int
// Blocked is set when the SSRF guard refused the address.
Blocked bool
Err error
}
Error is a failed call, with what the provider said.
type ErrorKind ¶
type ErrorKind string
ErrorKind classifies what went wrong.
const ( // ErrAuth: the provider rejected the key (401, 403, Gemini's // API_KEY_INVALID). ErrAuth ErrorKind = "auth" // ErrRateLimit: too many requests; RetryAfter says when to try again. ErrRateLimit ErrorKind = "rate_limit" // ErrOverloaded: the provider is overloaded (Anthropic 529, 503). ErrOverloaded ErrorKind = "overloaded" ErrUnavailable ErrorKind = "unavailable" // ErrBadRequest: the provider refused the request itself (400, 404, 422): // an unknown model, a request it can't take. ErrBadRequest ErrorKind = "bad_request" // ErrTimeout: no answer within the timeout. ErrTimeout ErrorKind = "timeout" // ErrNetwork: the provider couldn't be reached (DNS, refused connection, // an address the SSRF guard blocks). ErrNetwork ErrorKind = "network" // ErrTooLarge: the answer was over the size limit. ErrTooLarge ErrorKind = "too_large" // ErrInvalid: the answer couldn't be read. ErrInvalid ErrorKind = "invalid" )
type KnownModel ¶
type KnownModel struct {
Provider string
ID string
Name string
Input float64
Output float64
ImageInput bool
}
KnownModel is a model of the price list: its name, its prices in dollars per million tokens, and whether it reads images.
func Known ¶
func Known(provider, model string) (KnownModel, bool)
Known returns the price list's entry for a model, matching dated snapshots ("claude-haiku-4-5-20251001") and Azure deployments named after their model.
type Provider ¶
type Provider interface {
// Generate sends one request.
Generate(ctx context.Context, req Request) (Response, error)
// Models lists the models the key can use.
Models(ctx context.Context) ([]Model, error)
}
Provider is a connected model.
type ProviderInfo ¶
type ProviderInfo struct {
Kind string
Name string
// NeedsKey is false for compatible servers (local servers often have none).
NeedsKey bool
// NeedsBaseURL: Azure's endpoint, a compatible server's address.
NeedsBaseURL bool
Capabilities []string
// DefaultModel is offered first in the connect dialog.
DefaultModel string
}
ProviderInfo describes a provider in the catalog.
type Request ¶
type Request struct {
// System is the instruction the model follows.
System string
Messages []Message
// MaxTokens bounds the answer (DefaultMaxTokens).
MaxTokens int
// JSON asks for one JSON object; providers with structured output get
// Schema (a JSON Schema) named SchemaName.
JSON bool
Schema map[string]any
SchemaName string
// Task is the request as data, for in-process models (the demo's
// recorded model and tests). Adapters never send it.
Task any
}
Request is one call to the model.
type Response ¶
type Response struct {
Text string
Model string
InputTokens int
OutputTokens int
StopReason string
// Refused is true when the provider says the model declined (Anthropic's
// "refusal" stop reason, OpenAI's refusal field, Gemini's safety block);
// Text is then the model's own words, when it gave any.
Refused bool
}
Response is the model's answer.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package contracttest is the shared test suite every AI provider adapter passes (design expansion, plan §8: each new provider kind gets one, like the sending adapters' internal/providers/contracttest).
|
Package contracttest is the shared test suite every AI provider adapter passes (design expansion, plan §8: each new provider kind gets one, like the sending adapters' internal/providers/contracttest). |