Documentation
¶
Overview ¶
Package ai defines provider-neutral types for the AI proxy (backyard.ai). Providers live in sub-packages (anthropic, openai, bedrock, gemini) and the router package maps client-facing aliases to provider models.
Index ¶
- Constants
- Variables
- func KeyedUserID(key []byte, user string) string
- func OpaqueUserID(user string) string
- func ParseDataURL(s string) (data []byte, mediaType string, ok bool)
- func StripCodeFences(s string) string
- func SystemWithJSON(system string) string
- func TextOf(parts []Part) string
- func ValidEffort(e string) bool
- type ChatRequest
- type ChatResponse
- type EmbedRequest
- type EmbedResponse
- type Image
- type ImageRequest
- type ImageResolver
- type ImageResponse
- type Message
- type ModelInfo
- type Part
- type Provider
- type Service
- type UpstreamError
- type Usage
Constants ¶
const ( StopEndTurn = "end_turn" StopMaxTokens = "max_tokens" StopStopSequence = "stop_sequence" StopRefusal = "refusal" StopOther = "other" )
Normalized stop reasons.
const JSONInstruction = "Respond with a single valid JSON object and nothing else: " +
"no Markdown code fences, no commentary before or after it."
JSONInstruction is appended to the system prompt when a request asks for a JSON response (ChatRequest.JSON). Providers with a native JSON mode use it as well: OpenAI requires the word "JSON" to appear in the prompt.
Variables ¶
var ( // ErrUnsupported: the provider/model does not support this operation. ErrUnsupported = errors.New("ai: operation not supported by this model") // ErrUnknownModel: alias not configured (or raw models disabled). ErrUnknownModel = errors.New("ai: unknown model") // ErrDisabled: AI is not configured on this server. ErrDisabled = errors.New("ai: disabled") // ErrBadRequest: invalid request (empty messages, bad image, …). ErrBadRequest = errors.New("ai: bad request") // ErrNotLocal is returned by an ImageResolver for an absolute URL that // does not point at content the server holds itself; the URL is then // handled like any other remote image URL. ErrNotLocal = errors.New("ai: not a local image") )
Functions ¶
func KeyedUserID ¶
KeyedUserID is OpaqueUserID with a secret key (HMAC-SHA256): without the key, a provider cannot confirm a guessed email address by hashing it. An empty key falls back to OpaqueUserID. The empty user maps to "".
func OpaqueUserID ¶
OpaqueUserID maps an end-user identifier (often an email address) to a stable opaque id suitable for provider abuse-monitoring fields such as Anthropic's metadata.user_id or OpenAI's user. Providers ask callers not to send personal data there. The empty string maps to the empty string.
func ParseDataURL ¶
ParseDataURL decodes a base64 "data:" URL (data:image/png;base64,....). ok is false when s is not a base64 data URL or does not decode.
func StripCodeFences ¶
StripCodeFences extracts a JSON document from a model answer that was asked for JSON. It removes surrounding Markdown code fences (```json … ```) and, when the text still is not valid JSON, falls back to the outermost {…} or […] span if that span is valid JSON. Otherwise the trimmed input is returned unchanged, so callers never lose content.
func SystemWithJSON ¶
SystemWithJSON returns system with JSONInstruction appended.
func ValidEffort ¶
ValidEffort reports whether e is a known effort level (low, medium, high, xhigh, max). The empty string is not a valid level.
Types ¶
type ChatRequest ¶
type ChatRequest struct {
Model string
System string
Messages []Message
MaxTokens int
Temperature *float64
JSON bool // ask for a single JSON object response
Effort string // optional reasoning effort hint (anthropic)
User string // end-user id for provider abuse monitoring (optional)
// ResolveImage, when set, loads images the request references by a URL
// of the calling site's own content (see ImageResolver). It is used by
// the Service (router) before the request reaches a provider; providers
// never see it and ignore it if set.
ResolveImage ImageResolver
}
ChatRequest is a provider-level chat request (Model is the provider model id).
type ChatResponse ¶
type ChatResponse struct {
Content string `json:"content"`
Model string `json:"model"` // provider model id (router replaces with alias)
StopReason string `json:"stop_reason"`
Usage Usage `json:"usage"`
}
ChatResponse is the complete result of a chat request.
type EmbedRequest ¶
EmbedRequest asks for embeddings.
type EmbedResponse ¶
EmbedResponse holds one vector per input, in order.
type ImageRequest ¶
type ImageRequest struct {
Model string
Prompt string
Size string // e.g. "1024x1024"; provider-specific values passed through
N int
User string
}
ImageRequest asks for generated images.
type ImageResolver ¶
ImageResolver loads an image that a chat request references by URL when that URL points at content the server holds itself — typically the calling site's uploads (/__backyard/files/{id}/{name}, which is what backyard.files.upload returns). The HTTP layer builds one per request, bound to the request's site, user and origin; the router calls it:
- for every site-relative URL ("/…", but not "//host/…"); without a resolver such URLs are rejected as bad requests;
- for every absolute http(s) URL; the resolver returns ErrNotLocal for URLs it does not serve (another origin), which are then passed to providers or downloaded by the SSRF-guarded fetcher as before.
ref is the URL exactly as the client sent it (trimmed). maxBytes is the most the router accepts for this image (the per-image limit, lowered by what is left of the request's image byte budget); the resolver should check the stored size before reading and must not return more. Errors wrapping ErrBadRequest (unknown file, another site's file, too large) become 400 responses; context errors are passed through; any other error (a storage failure) is returned to the caller wrapped. The media type is detected from the bytes (png, jpeg, gif or webp; anything else is a bad request), so a stored content type is neither needed nor trusted.
type ImageResponse ¶
type ImageResponse struct {
Images []Image
}
ImageResponse holds generated images.
type Message ¶
Message is one chat turn. Role is "user" or "assistant" (system prompts are carried in ChatRequest.System).
type ModelInfo ¶
type ModelInfo struct {
Name string `json:"name"`
Kind string `json:"kind"` // chat | image | embedding
Provider string `json:"provider"`
Model string `json:"model,omitempty"` // only if ai.expose_model_ids
Description string `json:"description,omitempty"`
}
ModelInfo describes a client-visible alias.
type Part ¶
type Part struct {
Type string `json:"type"` // "text" | "image"
Text string `json:"text,omitempty"` // type=text
Data []byte `json:"-"` // type=image: raw bytes (decoded from base64 "data")
MediaType string `json:"media_type,omitempty"` // type=image: image/png, image/jpeg, image/gif, image/webp
URL string `json:"url,omitempty"` // type=image: remote image URL (providers may fetch or reject)
}
Part is one piece of message content.
type Provider ¶
type Provider interface {
// Name returns the configured provider key (e.g. "anthropic").
Name() string
// Chat performs a non-streaming completion.
Chat(ctx context.Context, req ChatRequest) (*ChatResponse, error)
// ChatStream streams text deltas to onDelta and returns the final response
// (Content contains the full text). If onDelta returns an error the stream
// is aborted and that error returned.
ChatStream(ctx context.Context, req ChatRequest, onDelta func(text string) error) (*ChatResponse, error)
// Image generates images or returns ErrUnsupported.
Image(ctx context.Context, req ImageRequest) (*ImageResponse, error)
// Embed computes embeddings or returns ErrUnsupported.
Embed(ctx context.Context, req EmbedRequest) (*EmbedResponse, error)
}
Provider is an upstream AI API. Implementations must be safe for concurrent use.
type Service ¶
type Service interface {
// Chat resolves req.Model (an alias, "" = default, or "provider/model" if
// raw models are allowed) and dispatches. The response's Model is the alias.
Chat(ctx context.Context, alias string, req ChatRequest) (*ChatResponse, error)
ChatStream(ctx context.Context, alias string, req ChatRequest, onDelta func(string) error) (*ChatResponse, error)
Image(ctx context.Context, alias string, req ImageRequest) (*ImageResponse, string, error) // returns alias used
Embed(ctx context.Context, alias string, req EmbedRequest) (*EmbedResponse, string, error)
// Models lists configured aliases (sorted by name) and the default chat alias.
Models() (defaultAlias string, models []ModelInfo)
// Enabled reports whether any model is configured.
Enabled() bool
}
Service is what the HTTP layer uses: alias resolution + provider dispatch. Implemented by router.Router.
type UpstreamError ¶
type UpstreamError struct {
Provider string
Status int // HTTP status from the provider, 0 if not applicable
Message string
// RetryDelay is how long the provider asked callers to wait before
// retrying (its Retry-After, retry-after-ms or rate-limit reset
// headers, or Google's RetryInfo), 0 when it did not say.
RetryDelay time.Duration
}
UpstreamError wraps an error returned by a provider API.
func (*UpstreamError) Error ¶
func (e *UpstreamError) Error() string
func (*UpstreamError) RetryAfter ¶
func (e *UpstreamError) RetryAfter() time.Duration
RetryAfter returns RetryDelay (0 when the provider gave no retry time). The HTTP layer looks for this method on any error in a chain to set the Retry-After of a throttled (429) response.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package anthropic implements ai.Provider for Claude using the official Anthropic Go SDK (github.com/anthropics/anthropic-sdk-go).
|
Package anthropic implements ai.Provider for Claude using the official Anthropic Go SDK (github.com/anthropics/anthropic-sdk-go). |
|
Package bedrock implements ai.Provider for Amazon Bedrock using the AWS SDK for Go v2 (bedrockruntime).
|
Package bedrock implements ai.Provider for Amazon Bedrock using the AWS SDK for Go v2 (bedrockruntime). |
|
Package gemini implements ai.Provider for Google's Gemini models over raw HTTP, either through the Gemini API in Google AI Studio (api_key, https://generativelanguage.googleapis.com/v1beta) or through Vertex AI (project_id + location with Application Default Credentials; location "global" uses https://aiplatform.googleapis.com).
|
Package gemini implements ai.Provider for Google's Gemini models over raw HTTP, either through the Gemini API in Google AI Studio (api_key, https://generativelanguage.googleapis.com/v1beta) or through Vertex AI (project_id + location with Application Default Credentials; location "global" uses https://aiplatform.googleapis.com). |
|
internal
|
|
|
claudemodel
Package claudemodel encodes per-model request rules for Claude models that are shared by the anthropic provider (first-party, Vertex AI, Bedrock Mantle) and the bedrock provider (Converse API).
|
Package claudemodel encodes per-model request rules for Claude models that are shared by the anthropic provider (first-party, Vertex AI, Bedrock Mantle) and the bedrock provider (Converse API). |
|
httpx
Package httpx holds HTTP helpers shared by the raw-HTTP AI providers (openai, gemini): JSON requests, bounded response decoding and conversion of upstream error bodies into *ai.UpstreamError.
|
Package httpx holds HTTP helpers shared by the raw-HTTP AI providers (openai, gemini): JSON requests, bounded response decoding and conversion of upstream error bodies into *ai.UpstreamError. |
|
imagefetch
Package imagefetch downloads remote images referenced by URL in chat requests for providers that only accept inline bytes (Gemini, Bedrock, Claude on Vertex AI/Bedrock, Ollama and other self-hosted OpenAI-compatible servers).
|
Package imagefetch downloads remote images referenced by URL in chat requests for providers that only accept inline bytes (Gemini, Bedrock, Claude on Vertex AI/Bedrock, Ollama and other self-hosted OpenAI-compatible servers). |
|
sse
Package sse is a small, allocation-conscious reader for the text/event-stream format (https://html.spec.whatwg.org/multipage/server-sent-events.html) as emitted by OpenAI-compatible and Gemini streaming endpoints.
|
Package sse is a small, allocation-conscious reader for the text/event-stream format (https://html.spec.whatwg.org/multipage/server-sent-events.html) as emitted by OpenAI-compatible and Gemini streaming endpoints. |
|
Package openai implements ai.Provider for the OpenAI Chat Completions, Images and Embeddings APIs and for compatible servers: provider types "openai" (api.openai.com or any OpenAI-compatible base_url such as vLLM, LiteLLM, LM Studio or OpenRouter), "ollama" (OpenAI-compatible endpoint of a local Ollama) and "azure-openai" (Azure OpenAI / Azure AI Foundry, API key or Entra ID authentication).
|
Package openai implements ai.Provider for the OpenAI Chat Completions, Images and Embeddings APIs and for compatible servers: provider types "openai" (api.openai.com or any OpenAI-compatible base_url such as vLLM, LiteLLM, LM Studio or OpenRouter), "ollama" (OpenAI-compatible endpoint of a local Ollama) and "azure-openai" (Azure OpenAI / Azure AI Foundry, API key or Entra ID authentication). |
|
Package router maps client-facing model aliases to configured providers and implements ai.Service.
|
Package router maps client-facing model aliases to configured providers and implements ai.Service. |