Documentation
¶
Overview ¶
Package router maps client-facing model aliases to configured providers and implements ai.Service.
Clients choose aliases from ai.models ("smart", "fast", "image", "embed"); an empty alias selects the default of the requested kind (ai.default_model, ai.default_image_model, ai.default_embedding_model). "provider/model-id" is accepted only when ai.allow_raw_models is true (and the model id must look like one: no empty, "." or ".." path segments, no query or whitespace characters). The router validates and normalises requests (role:"system" messages are hoisted into the system prompt, consecutive same-role turns are merged, images are checked), applies the max_tokens caps, prepends ai.system_prompt, applies per-alias effort, bounds every upstream call with ai.request_timeout (streaming and image generation: at least 10 minutes) and reports the alias as the response model.
Limits on max_tokens: the cap is ai.max_tokens, lowered by a per-alias max_tokens when that is smaller (an alias cannot exceed the global cap; raise ai.max_tokens for long-output aliases and give the others their own lower caps); a request without max_tokens gets min(4096, cap). An alias's effort overrides the request's. When an upstream reports no token usage, the router fills in an estimate (about four bytes per token, a flat cost per image) so per-user token limits cannot be bypassed.
Image URLs of the calling site's own content (site-relative upload URLs such as /__backyard/files/{id}/{name}, and absolute URLs of the site's origin) are loaded through the request's ai.ImageResolver, when the HTTP layer supplies one, and sent as inline bytes; without a resolver site-relative URLs are rejected.
Untrusted input is contained: image URLs that a provider needs as inline bytes are downloaded by an SSRF-guarded fetcher within a per-request byte budget (MaxRemoteImageBytes, shared with images loaded through the resolver); end-user ids are replaced by keyed opaque ids (Options.UserIDKey) before they reach a provider; generated images get their media type from their content, and non-raster formats are refused.
Index ¶
- Constants
- func New(ctx context.Context, cfg config.AIConfig, opts Options) (ai.Service, error)
- type Options
- type Router
- func (r *Router) Chat(ctx context.Context, alias string, req ai.ChatRequest) (*ai.ChatResponse, error)
- func (r *Router) ChatStream(ctx context.Context, alias string, req ai.ChatRequest, ...) (*ai.ChatResponse, error)
- func (r *Router) Embed(ctx context.Context, alias string, req ai.EmbedRequest) (*ai.EmbedResponse, string, error)
- func (r *Router) Enabled() bool
- func (r *Router) Image(ctx context.Context, alias string, req ai.ImageRequest) (*ai.ImageResponse, string, error)
- func (r *Router) Models() (string, []ai.ModelInfo)
Constants ¶
const ( KindChat = "chat" KindImage = "image" KindEmbedding = "embedding" )
Model kinds.
const ( // MaxImageBytes is the largest inline image accepted in a chat message. MaxImageBytes = 20 << 20 // MaxImagesPerRequest bounds the number of images in one chat request. MaxImagesPerRequest = 50 // MaxImageCount bounds ImageRequest.N. MaxImageCount = 10 // MaxImagePromptChars bounds ImageRequest.Prompt. MaxImagePromptChars = 32000 // MaxEmbedInputs bounds the number of strings in one embedding request. MaxEmbedInputs = 2048 // MaxEmbedBytes bounds the total size of an embedding request's input. MaxEmbedBytes = 8 << 20 // MaxRemoteImageBytes bounds the total size of the image URLs one chat // request makes Backyard load: the site's own uploads (through the // request's ai.ImageResolver) and downloads for providers that need // inline bytes (Bedrock, Gemini, Claude on Vertex AI/Bedrock, Ollama and // other self-hosted OpenAI-compatible servers). MaxRemoteImageBytes = 64 << 20 )
Request limits enforced before calling a provider.
Variables ¶
This section is empty.
Functions ¶
func New ¶
New builds providers from cfg.Providers and validates cfg.Models. When AI is disabled it returns a Service whose Enabled() is false and whose methods return ai.ErrDisabled.
Only providers referenced by a model alias are built (all of them when allow_raw_models is set; failures of unreferenced providers are then logged and the provider is skipped). Construction does no network I/O: cloud credentials (Vertex AI, Bedrock Mantle, Entra ID) are discovered on first use.
Types ¶
type Options ¶
type Options struct {
HTTPClient *http.Client // used by HTTP-based providers (tests inject fakes)
Logger *slog.Logger
// Providers are prebuilt providers by key. They take precedence over
// building a provider from cfg.Providers and may also add keys that are
// not in cfg.Providers (tests, embedding Backyard with custom providers).
Providers map[string]ai.Provider
// UserIDKey keys the hash that turns end-user ids (emails) into the
// opaque ids sent to providers for abuse monitoring (ai.KeyedUserID).
// Pass the server secret so providers cannot confirm a guessed email;
// when empty an unkeyed hash (ai.OpaqueUserID) is used.
UserIDKey []byte
}
Options are optional dependencies.
type Router ¶
type Router struct {
// contains filtered or unexported fields
}
Router implements ai.Service. It is immutable after New and safe for concurrent use.
func (*Router) Chat ¶
func (r *Router) Chat(ctx context.Context, alias string, req ai.ChatRequest) (*ai.ChatResponse, error)
Chat implements ai.Service.
func (*Router) ChatStream ¶
func (r *Router) ChatStream(ctx context.Context, alias string, req ai.ChatRequest, onDelta func(string) error) (*ai.ChatResponse, error)
ChatStream implements ai.Service.
func (*Router) Embed ¶
func (r *Router) Embed(ctx context.Context, alias string, req ai.EmbedRequest) (*ai.EmbedResponse, string, error)
Embed implements ai.Service.
Source Files