ai

package
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 15, 2026 License: Apache-2.0 Imports: 9 Imported by: 0

Documentation

Overview

Package ai defines provider-neutral types for the AI proxy (backyard.ai). Providers live in sub-packages (anthropic, openai, bedrock, gemini) and the router package maps client-facing aliases to provider models.

Index

Constants

View Source
const (
	StopEndTurn      = "end_turn"
	StopMaxTokens    = "max_tokens"
	StopStopSequence = "stop_sequence"
	StopRefusal      = "refusal"
	StopOther        = "other"
)

Normalized stop reasons.

View Source
const JSONInstruction = "Respond with a single valid JSON object and nothing else: " +
	"no Markdown code fences, no commentary before or after it."

JSONInstruction is appended to the system prompt when a request asks for a JSON response (ChatRequest.JSON). Providers with a native JSON mode use it as well: OpenAI requires the word "JSON" to appear in the prompt.

Variables

View Source
var (
	// ErrUnsupported: the provider/model does not support this operation.
	ErrUnsupported = errors.New("ai: operation not supported by this model")
	// ErrUnknownModel: alias not configured (or raw models disabled).
	ErrUnknownModel = errors.New("ai: unknown model")
	// ErrDisabled: AI is not configured on this server.
	ErrDisabled = errors.New("ai: disabled")
	// ErrBadRequest: invalid request (empty messages, bad image, …).
	ErrBadRequest = errors.New("ai: bad request")
	// ErrNotLocal is returned by an ImageResolver for an absolute URL that
	// does not point at content the server holds itself; the URL is then
	// handled like any other remote image URL.
	ErrNotLocal = errors.New("ai: not a local image")
)

Functions

func KeyedUserID

func KeyedUserID(key []byte, user string) string

KeyedUserID is OpaqueUserID with a secret key (HMAC-SHA256): without the key, a provider cannot confirm a guessed email address by hashing it. An empty key falls back to OpaqueUserID. The empty user maps to "".

func OpaqueUserID

func OpaqueUserID(user string) string

OpaqueUserID maps an end-user identifier (often an email address) to a stable opaque id suitable for provider abuse-monitoring fields such as Anthropic's metadata.user_id or OpenAI's user. Providers ask callers not to send personal data there. The empty string maps to the empty string.

func ParseDataURL

func ParseDataURL(s string) (data []byte, mediaType string, ok bool)

ParseDataURL decodes a base64 "data:" URL (data:image/png;base64,....). ok is false when s is not a base64 data URL or does not decode.

func StripCodeFences

func StripCodeFences(s string) string

StripCodeFences extracts a JSON document from a model answer that was asked for JSON. It removes surrounding Markdown code fences (```json … ```) and, when the text still is not valid JSON, falls back to the outermost {…} or […] span if that span is valid JSON. Otherwise the trimmed input is returned unchanged, so callers never lose content.

func SystemWithJSON

func SystemWithJSON(system string) string

SystemWithJSON returns system with JSONInstruction appended.

func TextOf

func TextOf(parts []Part) string

TextOf concatenates the text parts of a message (images are skipped).

func ValidEffort

func ValidEffort(e string) bool

ValidEffort reports whether e is a known effort level (low, medium, high, xhigh, max). The empty string is not a valid level.

Types

type ChatRequest

type ChatRequest struct {
	Model       string
	System      string
	Messages    []Message
	MaxTokens   int
	Temperature *float64
	JSON        bool   // ask for a single JSON object response
	Effort      string // optional reasoning effort hint (anthropic)
	User        string // end-user id for provider abuse monitoring (optional)

	// ResolveImage, when set, loads images the request references by a URL
	// of the calling site's own content (see ImageResolver). It is used by
	// the Service (router) before the request reaches a provider; providers
	// never see it and ignore it if set.
	ResolveImage ImageResolver
}

ChatRequest is a provider-level chat request (Model is the provider model id).

type ChatResponse

type ChatResponse struct {
	Content    string `json:"content"`
	Model      string `json:"model"` // provider model id (router replaces with alias)
	StopReason string `json:"stop_reason"`
	Usage      Usage  `json:"usage"`
}

ChatResponse is the complete result of a chat request.

type EmbedRequest

type EmbedRequest struct {
	Model string
	Input []string
	User  string
}

EmbedRequest asks for embeddings.

type EmbedResponse

type EmbedResponse struct {
	Embeddings [][]float64
	Usage      Usage
}

EmbedResponse holds one vector per input, in order.

type Image

type Image struct {
	Data      []byte
	MediaType string
}

Image is one generated image.

type ImageRequest

type ImageRequest struct {
	Model  string
	Prompt string
	Size   string // e.g. "1024x1024"; provider-specific values passed through
	N      int
	User   string
}

ImageRequest asks for generated images.

type ImageResolver

type ImageResolver func(ctx context.Context, ref string, maxBytes int64) ([]byte, error)

ImageResolver loads an image that a chat request references by URL when that URL points at content the server holds itself — typically the calling site's uploads (/__backyard/files/{id}/{name}, which is what backyard.files.upload returns). The HTTP layer builds one per request, bound to the request's site, user and origin; the router calls it:

  • for every site-relative URL ("/…", but not "//host/…"); without a resolver such URLs are rejected as bad requests;
  • for every absolute http(s) URL; the resolver returns ErrNotLocal for URLs it does not serve (another origin), which are then passed to providers or downloaded by the SSRF-guarded fetcher as before.

ref is the URL exactly as the client sent it (trimmed). maxBytes is the most the router accepts for this image (the per-image limit, lowered by what is left of the request's image byte budget); the resolver should check the stored size before reading and must not return more. Errors wrapping ErrBadRequest (unknown file, another site's file, too large) become 400 responses; context errors are passed through; any other error (a storage failure) is returned to the caller wrapped. The media type is detected from the bytes (png, jpeg, gif or webp; anything else is a bad request), so a stored content type is neither needed nor trusted.

type ImageResponse

type ImageResponse struct {
	Images []Image
}

ImageResponse holds generated images.

type Message

type Message struct {
	Role    string `json:"role"`
	Content []Part `json:"content"`
}

Message is one chat turn. Role is "user" or "assistant" (system prompts are carried in ChatRequest.System).

type ModelInfo

type ModelInfo struct {
	Name        string `json:"name"`
	Kind        string `json:"kind"` // chat | image | embedding
	Provider    string `json:"provider"`
	Model       string `json:"model,omitempty"` // only if ai.expose_model_ids
	Description string `json:"description,omitempty"`
}

ModelInfo describes a client-visible alias.

type Part

type Part struct {
	Type      string `json:"type"`                 // "text" | "image"
	Text      string `json:"text,omitempty"`       // type=text
	Data      []byte `json:"-"`                    // type=image: raw bytes (decoded from base64 "data")
	MediaType string `json:"media_type,omitempty"` // type=image: image/png, image/jpeg, image/gif, image/webp
	URL       string `json:"url,omitempty"`        // type=image: remote image URL (providers may fetch or reject)
}

Part is one piece of message content.

type Provider

type Provider interface {
	// Name returns the configured provider key (e.g. "anthropic").
	Name() string
	// Chat performs a non-streaming completion.
	Chat(ctx context.Context, req ChatRequest) (*ChatResponse, error)
	// ChatStream streams text deltas to onDelta and returns the final response
	// (Content contains the full text). If onDelta returns an error the stream
	// is aborted and that error returned.
	ChatStream(ctx context.Context, req ChatRequest, onDelta func(text string) error) (*ChatResponse, error)
	// Image generates images or returns ErrUnsupported.
	Image(ctx context.Context, req ImageRequest) (*ImageResponse, error)
	// Embed computes embeddings or returns ErrUnsupported.
	Embed(ctx context.Context, req EmbedRequest) (*EmbedResponse, error)
}

Provider is an upstream AI API. Implementations must be safe for concurrent use.

type Service

type Service interface {
	// Chat resolves req.Model (an alias, "" = default, or "provider/model" if
	// raw models are allowed) and dispatches. The response's Model is the alias.
	Chat(ctx context.Context, alias string, req ChatRequest) (*ChatResponse, error)
	ChatStream(ctx context.Context, alias string, req ChatRequest, onDelta func(string) error) (*ChatResponse, error)
	Image(ctx context.Context, alias string, req ImageRequest) (*ImageResponse, string, error) // returns alias used
	Embed(ctx context.Context, alias string, req EmbedRequest) (*EmbedResponse, string, error)
	// Models lists configured aliases (sorted by name) and the default chat alias.
	Models() (defaultAlias string, models []ModelInfo)
	// Enabled reports whether any model is configured.
	Enabled() bool
}

Service is what the HTTP layer uses: alias resolution + provider dispatch. Implemented by router.Router.

type UpstreamError

type UpstreamError struct {
	Provider string
	Status   int // HTTP status from the provider, 0 if not applicable
	Message  string
	// RetryDelay is how long the provider asked callers to wait before
	// retrying (its Retry-After, retry-after-ms or rate-limit reset
	// headers, or Google's RetryInfo), 0 when it did not say.
	RetryDelay time.Duration
}

UpstreamError wraps an error returned by a provider API.

func (*UpstreamError) Error

func (e *UpstreamError) Error() string

func (*UpstreamError) RetryAfter

func (e *UpstreamError) RetryAfter() time.Duration

RetryAfter returns RetryDelay (0 when the provider gave no retry time). The HTTP layer looks for this method on any error in a chain to set the Retry-After of a throttled (429) response.

type Usage

type Usage struct {
	InputTokens  int `json:"input_tokens"`
	OutputTokens int `json:"output_tokens"`
}

Usage reports token consumption.

Directories

Path Synopsis
Package anthropic implements ai.Provider for Claude using the official Anthropic Go SDK (github.com/anthropics/anthropic-sdk-go).
Package anthropic implements ai.Provider for Claude using the official Anthropic Go SDK (github.com/anthropics/anthropic-sdk-go).
Package bedrock implements ai.Provider for Amazon Bedrock using the AWS SDK for Go v2 (bedrockruntime).
Package bedrock implements ai.Provider for Amazon Bedrock using the AWS SDK for Go v2 (bedrockruntime).
Package gemini implements ai.Provider for Google's Gemini models over raw HTTP, either through the Gemini API in Google AI Studio (api_key, https://generativelanguage.googleapis.com/v1beta) or through Vertex AI (project_id + location with Application Default Credentials; location "global" uses https://aiplatform.googleapis.com).
Package gemini implements ai.Provider for Google's Gemini models over raw HTTP, either through the Gemini API in Google AI Studio (api_key, https://generativelanguage.googleapis.com/v1beta) or through Vertex AI (project_id + location with Application Default Credentials; location "global" uses https://aiplatform.googleapis.com).
internal
claudemodel
Package claudemodel encodes per-model request rules for Claude models that are shared by the anthropic provider (first-party, Vertex AI, Bedrock Mantle) and the bedrock provider (Converse API).
Package claudemodel encodes per-model request rules for Claude models that are shared by the anthropic provider (first-party, Vertex AI, Bedrock Mantle) and the bedrock provider (Converse API).
httpx
Package httpx holds HTTP helpers shared by the raw-HTTP AI providers (openai, gemini): JSON requests, bounded response decoding and conversion of upstream error bodies into *ai.UpstreamError.
Package httpx holds HTTP helpers shared by the raw-HTTP AI providers (openai, gemini): JSON requests, bounded response decoding and conversion of upstream error bodies into *ai.UpstreamError.
imagefetch
Package imagefetch downloads remote images referenced by URL in chat requests for providers that only accept inline bytes (Gemini, Bedrock, Claude on Vertex AI/Bedrock, Ollama and other self-hosted OpenAI-compatible servers).
Package imagefetch downloads remote images referenced by URL in chat requests for providers that only accept inline bytes (Gemini, Bedrock, Claude on Vertex AI/Bedrock, Ollama and other self-hosted OpenAI-compatible servers).
sse
Package sse is a small, allocation-conscious reader for the text/event-stream format (https://html.spec.whatwg.org/multipage/server-sent-events.html) as emitted by OpenAI-compatible and Gemini streaming endpoints.
Package sse is a small, allocation-conscious reader for the text/event-stream format (https://html.spec.whatwg.org/multipage/server-sent-events.html) as emitted by OpenAI-compatible and Gemini streaming endpoints.
Package openai implements ai.Provider for the OpenAI Chat Completions, Images and Embeddings APIs and for compatible servers: provider types "openai" (api.openai.com or any OpenAI-compatible base_url such as vLLM, LiteLLM, LM Studio or OpenRouter), "ollama" (OpenAI-compatible endpoint of a local Ollama) and "azure-openai" (Azure OpenAI / Azure AI Foundry, API key or Entra ID authentication).
Package openai implements ai.Provider for the OpenAI Chat Completions, Images and Embeddings APIs and for compatible servers: provider types "openai" (api.openai.com or any OpenAI-compatible base_url such as vLLM, LiteLLM, LM Studio or OpenRouter), "ollama" (OpenAI-compatible endpoint of a local Ollama) and "azure-openai" (Azure OpenAI / Azure AI Foundry, API key or Entra ID authentication).
Package router maps client-facing model aliases to configured providers and implements ai.Service.
Package router maps client-facing model aliases to configured providers and implements ai.Service.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL