llm

package
v1.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 30, 2026 License: AGPL-3.0 Imports: 13 Imported by: 0

Documentation

Overview

Package llm talks to the AI model a workspace connects (design expansion, phase 3): Anthropic, OpenAI, Google Gemini, Azure OpenAI and servers that speak the OpenAI API (Ollama, vLLM, LM Studio). It is plain HTTP with the standard library: each provider is a small adapter that turns a Request into the provider's JSON and its answer (or error) back into a Response.

Every adapter passes the shared contract suite in llm/contracttest against recorded answers; tests never reach the network.

Requests have a timeout and a response size limit, and the client the caller passes decides which addresses may be dialed (internal/netguard). What goes into a Request is the caller's business: internal/ai builds prompts from templates with the person's content delimited, and never puts contact data in them.

Index

Constants

View Source
const (
	CapText             = "text"
	CapImageInput       = "image_input"
	CapStructuredOutput = "structured_output"
	CapOwnNetwork       = "own_network"
	CapDataRegion       = "data_region"
)

Capabilities a provider card shows.

View Source
const (
	Anthropic  = "anthropic"
	OpenAI     = "openai"
	Google     = "google"
	Azure      = "azure"
	Compatible = "compatible"
)

Provider kinds.

View Source
const (
	DefaultTimeout          = 60 * time.Second
	DefaultMaxResponseBytes = 4 << 20
	DefaultMaxTokens        = 2048
	// DefaultAzureAPIVersion is Azure OpenAI's generally available API
	// version for chat completions with images and JSON output.
	DefaultAzureAPIVersion = "2024-10-21"
)

Defaults for requests.

View Source
const (
	StopEnd     = "end"
	StopLength  = "length"
	StopRefusal = "refusal"
	StopOther   = "other"
)

Stop reasons, normalized.

Variables

Kinds lists the providers in the catalog's order.

View Source
var PriceList = []KnownModel{
	{Provider: Anthropic, ID: "claude-sonnet-5-5", Name: "Claude Sonnet 5.5", Input: 2, Output: 10, ImageInput: true},
	{Provider: Anthropic, ID: "claude-haiku-4-5", Name: "Claude Haiku 4.5", Input: 1, Output: 5, ImageInput: true},
	{Provider: Anthropic, ID: "claude-sonnet-4-5", Name: "Claude Sonnet 4.5", Input: 3, Output: 15, ImageInput: true},
	{Provider: Anthropic, ID: "claude-opus-4-5", Name: "Claude Opus 4.5", Input: 5, Output: 25, ImageInput: true},
	{Provider: OpenAI, ID: "gpt-5", Name: "GPT-5", Input: 1.25, Output: 10, ImageInput: true},
	{Provider: OpenAI, ID: "gpt-5-mini", Name: "GPT-5 mini", Input: 0.25, Output: 2, ImageInput: true},
	{Provider: OpenAI, ID: "gpt-4.1", Name: "GPT-4.1", Input: 2, Output: 8, ImageInput: true},
	{Provider: OpenAI, ID: "gpt-4o-mini", Name: "GPT-4o mini", Input: 0.15, Output: 0.6, ImageInput: true},
	{Provider: Google, ID: "gemini-2.5-pro", Name: "Gemini 2.5 Pro", Input: 1.25, Output: 10, ImageInput: true},
	{Provider: Google, ID: "gemini-2.5-flash", Name: "Gemini 2.5 Flash", Input: 0.3, Output: 2.5, ImageInput: true},
	{Provider: Google, ID: "gemini-2.5-flash-lite", Name: "Gemini 2.5 Flash-Lite", Input: 0.1, Output: 0.4, ImageInput: true},
}

PriceList holds the prices pre-filled when a model is picked. People edit them when their contract differs; a model that isn't listed starts at 0.

View Source
var PriceListDate = time.Date(2026, time.October, 1, 0, 0, 0, 0, time.UTC)

PriceListDate is the date of the price list below, the one ReputeCast's release ships with.

View Source
var Providers = []ProviderInfo{
	{Kind: Anthropic, Name: "Anthropic", NeedsKey: true, Capabilities: []string{CapText, CapImageInput, CapStructuredOutput}, DefaultModel: "claude-sonnet-5-5"},
	{Kind: OpenAI, Name: "OpenAI", NeedsKey: true, Capabilities: []string{CapText, CapImageInput, CapStructuredOutput}, DefaultModel: "gpt-5"},
	{Kind: Google, Name: "Google Gemini", NeedsKey: true, Capabilities: []string{CapText, CapImageInput, CapStructuredOutput}, DefaultModel: "gemini-2.5-flash"},
	{Kind: Azure, Name: "Azure OpenAI", NeedsKey: true, NeedsBaseURL: true, Capabilities: []string{CapText, CapImageInput, CapDataRegion}},
	{Kind: Compatible, Name: "OpenAI-compatible", NeedsBaseURL: true, Capabilities: []string{CapText, CapOwnNetwork}},
}

Providers is the catalog, in order.

Functions

func FirstJSON

func FirstJSON(s string) (json.RawMessage, bool)

FirstJSON returns the first JSON object in s: models asked for JSON sometimes wrap it in a code fence or a sentence.

func ImageInput

func ImageInput(provider, model string) bool

ImageInput reports whether a model reads images: the hosted providers' current models do; an open model does when its name says so.

func ModelName

func ModelName(provider, model string) string

ModelName is how people call a model: the price list's name, else the id.

func Valid

func Valid(kind string) bool

Valid reports whether kind is a provider this package can talk to.

Types

type Config

type Config struct {
	Kind string
	// BaseURL overrides the provider's API root: the Azure resource endpoint
	// (https://acme.openai.azure.com) or a compatible server's address with
	// its /v1 (http://ollama.acme.internal:11434/v1). Empty is the
	// provider's public API.
	BaseURL string
	// APIKey is optional for compatible servers.
	APIKey string
	Model  string
	// Deployment and APIVersion are Azure OpenAI's.
	Deployment string
	APIVersion string
	// HTTP is the client to use (the SSRF guard's in production).
	HTTP *http.Client
	// Timeout bounds one request, retries included (DefaultTimeout).
	Timeout time.Duration
	// MaxResponseBytes caps the answer read (DefaultMaxResponseBytes).
	MaxResponseBytes int64
	// Retries is how many times an overloaded or unavailable provider is
	// asked again (1 by default: "returned an error twice"); RetryWait is
	// the pause before each retry (1 s).
	Retries   *int
	RetryWait time.Duration
}

Config says how to reach a model.

type Error

type Error struct {
	Kind   ErrorKind
	Status int
	// Code is the provider's error type ("authentication_error",
	// "rate_limit_exceeded", "RESOURCE_EXHAUSTED").
	Code string
	// Message is the provider's message ("invalid x-api-key").
	Message    string
	RetryAfter time.Duration
	// Attempts is how many times the request was sent.
	Attempts int
	// Blocked is set when the SSRF guard refused the address.
	Blocked bool
	Err     error
}

Error is a failed call, with what the provider said.

func AsError

func AsError(err error) (*Error, bool)

AsError returns err as an *Error.

func (*Error) Error

func (e *Error) Error() string

func (*Error) Unwrap

func (e *Error) Unwrap() error

type ErrorKind

type ErrorKind string

ErrorKind classifies what went wrong.

const (
	// ErrAuth: the provider rejected the key (401, 403, Gemini's
	// API_KEY_INVALID).
	ErrAuth ErrorKind = "auth"
	// ErrRateLimit: too many requests; RetryAfter says when to try again.
	ErrRateLimit ErrorKind = "rate_limit"
	// ErrOverloaded: the provider is overloaded (Anthropic 529, 503).
	ErrOverloaded ErrorKind = "overloaded"
	// ErrUnavailable: a server error (5xx).
	ErrUnavailable ErrorKind = "unavailable"
	// ErrBadRequest: the provider refused the request itself (400, 404, 422):
	// an unknown model, a request it can't take.
	ErrBadRequest ErrorKind = "bad_request"
	// ErrTimeout: no answer within the timeout.
	ErrTimeout ErrorKind = "timeout"
	// ErrNetwork: the provider couldn't be reached (DNS, refused connection,
	// an address the SSRF guard blocks).
	ErrNetwork ErrorKind = "network"
	// ErrTooLarge: the answer was over the size limit.
	ErrTooLarge ErrorKind = "too_large"
	// ErrInvalid: the answer couldn't be read.
	ErrInvalid ErrorKind = "invalid"
)

type Image

type Image struct {
	MediaType string // image/png, image/jpeg, image/gif
	Data      []byte
}

Image is an image the model should look at.

type KnownModel

type KnownModel struct {
	Provider   string
	ID         string
	Name       string
	Input      float64
	Output     float64
	ImageInput bool
}

KnownModel is a model of the price list: its name, its prices in dollars per million tokens, and whether it reads images.

func Known

func Known(provider, model string) (KnownModel, bool)

Known returns the price list's entry for a model, matching dated snapshots ("claude-haiku-4-5-20251001") and Azure deployments named after their model.

type Message

type Message struct {
	Role  string // user or assistant
	Parts []Part
}

Message is a turn of the conversation.

type Model

type Model struct {
	ID   string `json:"id"`
	Name string `json:"name"`
}

Model is one model a provider offers.

type Part

type Part struct {
	Text  string
	Image *Image
}

Part is a piece of a message: text or an image.

type Provider

type Provider interface {
	// Generate sends one request.
	Generate(ctx context.Context, req Request) (Response, error)
	// Models lists the models the key can use.
	Models(ctx context.Context) ([]Model, error)
}

Provider is a connected model.

func New

func New(cfg Config) (Provider, error)

New returns the adapter of cfg.Kind.

type ProviderInfo

type ProviderInfo struct {
	Kind string
	Name string
	// NeedsKey is false for compatible servers (local servers often have none).
	NeedsKey bool
	// NeedsBaseURL: Azure's endpoint, a compatible server's address.
	NeedsBaseURL bool
	Capabilities []string
	// DefaultModel is offered first in the connect dialog.
	DefaultModel string
}

ProviderInfo describes a provider in the catalog.

func Info

func Info(kind string) ProviderInfo

Info returns a provider's catalog entry.

type Request

type Request struct {
	// System is the instruction the model follows.
	System   string
	Messages []Message
	// MaxTokens bounds the answer (DefaultMaxTokens).
	MaxTokens int
	// JSON asks for one JSON object; providers with structured output get
	// Schema (a JSON Schema) named SchemaName.
	JSON       bool
	Schema     map[string]any
	SchemaName string
	// Task is the request as data, for in-process models (the demo's
	// recorded model and tests). Adapters never send it.
	Task any
}

Request is one call to the model.

type Response

type Response struct {
	Text         string
	Model        string
	InputTokens  int
	OutputTokens int
	StopReason   string
	// Refused is true when the provider says the model declined (Anthropic's
	// "refusal" stop reason, OpenAI's refusal field, Gemini's safety block);
	// Text is then the model's own words, when it gave any.
	Refused bool
}

Response is the model's answer.

Directories

Path Synopsis
Package contracttest is the shared test suite every AI provider adapter passes (design expansion, plan §8: each new provider kind gets one, like the sending adapters' internal/providers/contracttest).
Package contracttest is the shared test suite every AI provider adapter passes (design expansion, plan §8: each new provider kind gets one, like the sending adapters' internal/providers/contracttest).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL