core

package
v0.1.95 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 20, 2026 License: MIT Imports: 21 Imported by: 0

Documentation

Overview

Package core provides core types and interfaces for the LLM gateway.

Package core defines the core interfaces and types for the LLM gateway.

Index

Constants

View Source 
const (
	// BatchActionCreate represents POST /v1/batches.
	BatchActionCreate = "create"
	// BatchActionList represents GET /v1/batches.
	BatchActionList = "list"
	// BatchActionGet represents GET /v1/batches/{id}.
	BatchActionGet = "get"
	// BatchActionCancel represents POST /v1/batches/{id}/cancel.
	BatchActionCancel = "cancel"
	// BatchActionResults represents GET /v1/batches/{id}/results.
	BatchActionResults = "results"
	// BatchActionDelete represents DELETE /v1/messages/batches/{id}
	// (Anthropic Message Batches dialect; the OpenAI dialect has no delete).
	BatchActionDelete = "delete"
)
View Source 
const (
	RequestOriginExternal  RequestOrigin = "external"
	RequestOriginGuardrail RequestOrigin = "guardrail"
	// RequestOriginPlugin marks gateway-internal inference issued by a plugin
	// instance through its Host (for example an LLM-based guardrail).
	RequestOriginPlugin RequestOrigin = "plugin"

	RequestDialectAnthropicMessages RequestDialect = "anthropic_messages"
)
View Source 
const (
	// MaxConversationInitialItems caps the items array accepted by
	// POST /v1/conversations.
	MaxConversationInitialItems = 20
	// MaxConversationMetadataPairs caps the final metadata object after create
	// or update.
	MaxConversationMetadataPairs = 16
)

Conversation limits mirror the OpenAI Conversations API so the gateway keeps an OpenAI-compatible public contract.

View Source 
const (
	ExtraContentVendorGoogle    = "google"
	ExtraContentVendorAnthropic = "anthropic"
)

Vendors that own an extra_content object.

View Source 
const (
	// ThinkingBlocksField holds the raw thinking/redacted_thinking blocks of an
	// assistant turn as a JSON array. Anthropic requires them back verbatim
	// (with signatures) when a thinking-enabled tool-use turn continues.
	ThinkingBlocksField = "thinking_blocks"
	// ToolResultIsErrorField marks a tool message as a failed tool call
	// (Anthropic tool_result.is_error).
	ToolResultIsErrorField = "is_error"
)

Members of extra_content.anthropic, set by the Anthropic Messages ingress and consumed by the Anthropic provider.

View Source 
const (
	// FileActionCreate represents POST /v1/files.
	FileActionCreate = "create"
	// FileActionList represents GET /v1/files.
	FileActionList = "list"
	// FileActionGet represents GET /v1/files/{id}.
	FileActionGet = "get"
	// FileActionDelete represents DELETE /v1/files/{id}.
	FileActionDelete = "delete"
	// FileActionContent represents GET /v1/files/{id}/content.
	FileActionContent = "content"
)
View Source 
const (
	RealtimeIntentTranscription = "transcription"
	RealtimeIntentTranslation   = "translation"
)

Realtime session intents. An empty intent opens a conversation (speech-to-speech) session; the named intents select a provider's specialized realtime surface, which differs in endpoint, session config, and event namespace. The model routes the request inside the gateway either way.

View Source 
const (
	// ModelPricingSourceModelRegistry identifies pricing data from the model registry.
	ModelPricingSourceModelRegistry = "model_registry"
	// ModelPricingSourceConfigYAML identifies pricing data from config.yaml.
	ModelPricingSourceConfigYAML = "config_yaml"
)
View Source 
const ConversationDeletedObject = "conversation.deleted"

ConversationDeletedObject is the value of the "object" field returned by DELETE /v1/conversations/{id}.

View Source 
const ConversationObject = "conversation"

ConversationObject is the value of the "object" field on a conversation.

View Source 
const ExtraContentField = "extra_content"

ExtraContentField names the member that carries provider state through the canonical chat types: opaque data a provider returned and expects back verbatim on a later turn, such as Gemini thought signatures or Anthropic thinking blocks. It lives on messages, tool calls, and Responses items as one object keyed by vendor, for example {"google": {...}}.

The rules are the same for every provider:

  • translation layers copy the member through untouched;
  • the router drops every vendor the selected provider does not own;
  • only the owning provider adapter reads or writes inside its object.
View Source 
const GatewayCachePointField = "_gomodel_cache_point"

GatewayCachePointField is the internal marker shared by provider cache planners and native request translators. It is never forwarded verbatim.

View Source 
const IdempotencyKeyHeader = "Idempotency-Key"

IdempotencyKeyHeader is the request header a client uses to mark retries of one logical request, spelled in canonical textproto form.

View Source 
const RequestIDHeader = "X-Request-Id"

RequestIDHeader is the request/response header carrying the gateway request id, spelled in canonical textproto form ("X-Request-Id") so Header.Get/Set need not canonicalize (and copy) the key on every call. Clients may send it in any case; lookups are case-insensitive.

View Source 
const UserPathHeader = "X-GoModel-User-Path"

Variables

View Source 
var ErrEmbeddedInSuccess = errors.New("provider embedded an error in a success response")

ErrEmbeddedInSuccess is wrapped by provider errors that arrived inside a 2xx response body; test with errors.Is.

View Source 
var ErrMessagesTokenCountUnsupported = errors.New("provider has no token counting endpoint")

ErrMessagesTokenCountUnsupported reports that the provider owning a model has no token counting endpoint.

View Source 
var ErrNativeBatchDeleteUnsupported = errors.New("native batch deletion is not supported by this provider")

ErrNativeBatchDeleteUnsupported reports that a provider's upstream batch API has no delete operation. Callers fall back to gateway-local deletion.

View Source 
var ErrNoChoices = errors.New("provider returned no choices")

ErrNoChoices is wrapped by NewNoChoicesProviderError so observers can tell an empty 200 response apart from other 502s.

View Source 
var ReservedAudioTranscriptionFormFields = map[string]bool{
	"model":                     true,
	"file":                      true,
	"provider":                  true,
	"language":                  true,
	"prompt":                    true,
	"response_format":           true,
	"temperature":               true,
	"timestamp_granularities":   true,
	"timestamp_granularities[]": true,
}

ReservedAudioTranscriptionFormFields are the multipart fields the gateway consumes and re-emits itself. They never travel in Fields, so a client cannot overwrite a gateway-controlled part (model, file, routing metadata) through the passthrough path.

Functions

func ApplyBodySelectorHints 

func ApplyBodySelectorHints(env *WhiteBoxPrompt, model, provider string, stream bool)

ApplyBodySelectorHints records selector hints parsed from a request body. The hints are intentionally sparse and best-effort; canonical request decode remains authoritative for translated JSON requests.

func ApplyBodyStreamHint added in v0.1.84 

func ApplyBodyStreamHint(env *WhiteBoxPrompt, stream bool)

ApplyBodyStreamHint records independently decoded streaming intent without claiming that the request body or its model selector was fully parsed.

func ApplyPartialBodyStreamHint added in v0.1.84 

func ApplyPartialBodyStreamHint(env *WhiteBoxPrompt, stream bool)

ApplyPartialBodyStreamHint records streaming intent observed in an incomplete body while preserving that the final value remains uncertain.

func CacheFileRouteInfo 

func CacheFileRouteInfo(env *WhiteBoxPrompt, req *FileRouteInfo)

CacheFileRouteInfo stores sparse file route metadata on the request semantics.

func CachePassthroughRouteInfo 

func CachePassthroughRouteInfo(env *WhiteBoxPrompt, req *PassthroughRouteInfo)

CachePassthroughRouteInfo stores typed passthrough route metadata on the request semantics.

func CloneOptionalJSONObject added in v0.1.67 

func CloneOptionalJSONObject(raw json.RawMessage) (json.RawMessage, error)

CloneOptionalJSONObject validates and clones an optional raw JSON object. Empty and null values are treated as absent.

func CloneRawJSON 

func CloneRawJSON(raw json.RawMessage) json.RawMessage

func DecodeCanonicalSelector 

func DecodeCanonicalSelector(body []byte, env *WhiteBoxPrompt) (model, provider string, ok bool)

DecodeCanonicalSelector decodes a canonical request body using the codec resolved by canonicalOperationCodecFor for env, then extracts the model and provider via semanticSelectorFromCanonicalRequest. It returns ok=false for a nil env, missing codec, or decode failure.

func DispatchDecodedBatchItem 

func DispatchDecodedBatchItem[T any](decoded *DecodedBatchItemRequest, handlers DecodedBatchItemHandlers[T]) (T, error)

DispatchDecodedBatchItem routes a decoded batch item to the matching typed handler based on its canonical request payload.

func EnsureModel 

func EnsureModel(model *string, requested string)

EnsureModel sets *model to the requested model when a provider response omits it, keeping responses OpenAI-compatible.

func EqualRealtimeIntent added in v0.1.82 

func EqualRealtimeIntent(a, b string) bool

EqualRealtimeIntent compares two session intents, accepting padded and differently cased input (Postel) so every provider compares intents the same way.

func ExtractTextContent 

func ExtractTextContent(content any) string

ExtractTextContent returns the textual portion of request content. Structured content parts are reduced to their text components only.

func GetAuthKeyID 

func GetAuthKeyID(ctx context.Context) string

GetAuthKeyID retrieves the managed auth key id from the context.

func GetCredentialAllowedModels added in v0.1.84 

func GetCredentialAllowedModels(ctx context.Context) []string

GetCredentialAllowedModels retrieves the credential-bound model allowlist. Nil means the credential does not restrict models.

func GetEffectiveUserPath 

func GetEffectiveUserPath(ctx context.Context) string

GetEffectiveUserPath retrieves the effective user path override from context.

func GetEnforceReturningUsageData 

func GetEnforceReturningUsageData(ctx context.Context) bool

GetEnforceReturningUsageData reports whether the request should ask providers to include usage in streaming responses when possible.

func GetFailoverUsed 

func GetFailoverUsed(ctx context.Context) bool

GetFailoverUsed reports whether the request was served by a failover model.

func GetGuardrailsHash 

func GetGuardrailsHash(ctx context.Context) string

GetGuardrailsHash retrieves the guardrails hash from the context. Returns empty string when no guardrails are active or the hash has not been set.

func GetRequestID 

func GetRequestID(ctx context.Context) string

GetRequestID retrieves the request ID from the context. Returns empty string if not found.

func HasStructuredContent 

func HasStructuredContent(content any) bool

HasStructuredContent reports whether the content uses the array form.

func IdempotencyKey added in v0.1.93 

func IdempotencyKey(ctx context.Context) string

IdempotencyKey returns the idempotency key for provider calls made with ctx: an override set with WithIdempotencyKey, otherwise the client's Idempotency-Key header captured in the request snapshot. A header value that is not a plain token of at most 255 visible ASCII characters is ignored, so it is safe to forward as-is.

func IsCredentialHeader 

func IsCredentialHeader(name string) bool

IsCredentialHeader reports whether the header name carries credentials. Matching is case-insensitive and ignores surrounding whitespace. It runs for every header of every audited request, so the fold happens in a stack buffer rather than through strings.ToLower.

func IsJSONNull 

func IsJSONNull(trimmed []byte) bool

CloneRawJSON returns a detached copy of a raw JSON value. IsJSONNull reports whether trimmed JSON data is empty or the null literal.

func IsModelInteractionPath 

func IsModelInteractionPath(path string) bool

IsModelInteractionPath reports whether a path is a model/provider interaction route.

func KeepExtraContentVendor added in v0.1.90 

func KeepExtraContentVendor(raw json.RawMessage, keep string) json.RawMessage

KeepExtraContentVendor reduces a raw extra_content value to the keep vendor's object. It returns nil when nothing should remain: the vendor is absent, keep is empty, or the value is not an object.

func MarkPassthroughStreamUncertain added in v0.1.74 

func MarkPassthroughStreamUncertain(env *WhiteBoxPrompt)

MarkPassthroughStreamUncertain records that bounded opaque-body inspection stopped before it could determine explicit streaming intent.

func MergeLabels 

func MergeLabels(sets ...[]string) []string

MergeLabels combines label sets in order into one list, trimming whitespace and dropping empty values and duplicates. Returns nil when nothing remains.

func NormalizeEmbeddingEncoding 

func NormalizeEmbeddingEncoding(resp *EmbeddingResponse, encodingFormat string)

NormalizeEmbeddingEncoding reconciles a response's embedding encoding with the encoding_format the client requested, keeping responses OpenAI-compatible regardless of provider quirks.

The OpenAI Python and JS/LangChain SDKs request encoding_format="base64" by default and decode it client-side. Some OpenAI-compatible servers (notably LM Studio) ignore encoding_format and always return float arrays, which makes those SDKs mis-decode the floats as packed bytes and produce corrupted, wrong-dimension vectors. Following Postel's Law, GoModel accepts whatever the upstream returns and re-encodes each vector into the format the caller asked for: base64 (little-endian float32, matching OpenAI) or a float array.

An empty or unrecognized format is treated as "float" (the OpenAI default when the field is omitted). Vectors already in the requested form, and values that don't parse as either shape, are left untouched.

func NormalizeMessageContent 

func NormalizeMessageContent(content any) (any, error)

NormalizeMessageContent validates dynamic content and returns its canonical form.

func NormalizeOperationPath 

func NormalizeOperationPath(raw string) string

NormalizeOperationPath returns a stable path-only form for model-facing endpoints.

func NormalizeUserPath 

func NormalizeUserPath(raw string) (string, error)

NormalizeUserPath canonicalizes one user hierarchy path from request ingress.

func ParseProviderPassthroughPath 

func ParseProviderPassthroughPath(path string) (provider string, endpoint string, ok bool)

ParseProviderPassthroughPath extracts provider and endpoint from /p/{provider}/{endpoint...}.

func PluginNoStore added in v0.1.91 

func PluginNoStore(ctx context.Context) bool

PluginNoStore reports whether a plugin that ran for the request asked for its response not to be stored in the response cache.

func PrimaryRouteSaturated 

func PrimaryRouteSaturated(ctx context.Context) error

PrimaryRouteSaturated returns the rate-limit rejection recorded for the resolved primary route, or nil when the route has capacity.

func RedactSensitiveURLQuery added in v0.1.71 

func RedactSensitiveURLQuery(raw string) string

RedactSensitiveURLQuery removes credential and authentication-transaction values from an absolute or relative URL while preserving ordinary query parameters. Malformed sensitive queries fail closed by dropping the query.

func RequestLabelsFromContext 

func RequestLabelsFromContext(ctx context.Context) []string

RequestLabelsFromContext returns the labels extracted for this request. Callers must treat the returned slice as read-only.

func ResolveBatchItemEndpoint 

func ResolveBatchItemEndpoint(defaultEndpoint, itemURL string) string

ResolveBatchItemEndpoint prefers an inline item URL and otherwise falls back to the batch default endpoint.

func ResponsesBlocksFromContentParts added in v0.1.86 

func ResponsesBlocksFromContentParts(parts []ContentPart) []any

ResponsesBlocksFromContentParts converts typed parts into generic content blocks with their type kept verbatim, or returns nil when a part cannot be encoded. ContentPart is the Chat content type: its MarshalJSON rewrites "input_text" to "text", which the Responses API rejects, so code that places typed parts into Responses input must convert them with this helper instead of serializing them directly.

func RewriteTokensSavedFromContext 

func RewriteTokensSavedFromContext(ctx context.Context) int

RewriteTokensSavedFromContext retrieves the request's rewrite savings estimate, or zero when no rewriter reported savings.

func SessionIDFromContext added in v0.1.63 

func SessionIDFromContext(ctx context.Context) string

SessionIDFromContext retrieves the detected client session id, or "" when the request carries no session signal.

func SpeechResponseContentType 

func SpeechResponseContentType(format string) string

SpeechResponseContentType maps a text-to-speech response_format to its MIME type. An unset format defaults to mp3, matching OpenAI's default.

func TaggingStripHeadersFromContext 

func TaggingStripHeadersFromContext(ctx context.Context) map[string]struct{}

TaggingStripHeadersFromContext returns the canonical header names marked as do-not-pass by the tagging configuration. Callers must treat the returned map as read-only.

func TranscriptionResponseContentType 

func TranscriptionResponseContentType(format string) string

TranscriptionResponseContentType maps a transcription response_format to its MIME type. json and verbose_json (and an unset format) are JSON; text, srt and vtt are plain text.

func UnmarshalMessageContent 

func UnmarshalMessageContent(data []byte) (any, error)

UnmarshalMessageContent decodes supported chat message content payloads. Chat content accepts plain strings, null, or arrays of supported content parts.

func UserPathAncestors 

func UserPathAncestors(path string) []string

UserPathAncestors returns deepest-to-root path fallback candidates.

func UserPathChild added in v0.1.77 

func UserPathChild(base, path string) (string, bool)

UserPathChild resolves the direct child of base that contains path. It is used by per-child policies so deeper descendants share their direct child's quota. The base path itself has no child and therefore does not match.

func UserPathContains added in v0.1.87 

func UserPathContains(base, path string) bool

UserPathContains reports whether path equals base or descends from it. Root ("/") contains every non-empty path; an empty path is contained by nothing. Both values are trimmed but otherwise expected to be canonical.

func UserPathFromContext 

func UserPathFromContext(ctx context.Context) string

UserPathFromContext returns the canonical request user path when available.

func UserPathHeaderName 

func UserPathHeaderName(raw string) string

UserPathHeaderName canonicalizes the configured user-path header name.

func UserPathHeaderNameFromContext 

func UserPathHeaderNameFromContext(ctx context.Context) string

UserPathHeaderNameFromContext returns the request-scoped user-path header name, falling back to the default public header.

func ValidFilePayload added in v0.1.87 

func ValidFilePayload(file *FileContent) bool

ValidFilePayload reports whether a file part carries an attachment: inline file_data, a remote file_url, or a provider file_id.

func ValidInputAudioPayload 

func ValidInputAudioPayload(data, format string) bool

ValidInputAudioPayload reports whether an input_audio payload satisfies the contract: data is always required, and format may be omitted only when data is a data: URI that already carries an explicit media type (used by providers such as Xiaomi MiMo ASR).

func ValidateImageEditRequest added in v0.1.81 

func ValidateImageEditRequest(req *ImageEditRequest) error

ValidateImageEditRequest enforces the fields every image edit provider requires before the request is routed.

func ValidateImageGenerationRequest added in v0.1.81 

func ValidateImageGenerationRequest(req *ImageGenerationRequest) error

ValidateImageGenerationRequest enforces the fields every image provider requires before the request is routed.

func WithAccessScope added in v0.1.87 

func WithAccessScope(ctx context.Context, scope AccessScope) context.Context

WithAccessScope returns a context carrying the credential's access scope. The path is canonicalized; an unparseable path is kept verbatim so it can never widen to global by accident.

func WithAuthKeyID 

func WithAuthKeyID(ctx context.Context, id string) context.Context

WithAuthKeyID returns a new context with the authenticated managed auth key id attached.

func WithBatchPreparationMetadata 

func WithBatchPreparationMetadata(ctx context.Context, metadata *BatchPreparationMetadata) context.Context

WithBatchPreparationMetadata returns a new context with batch preprocessing metadata attached.

func WithCredentialAllowedModels added in v0.1.84 

func WithCredentialAllowedModels(ctx context.Context, allowed []string) context.Context

WithCredentialAllowedModels returns a new context carrying the model allowlist bound to the authenticated credential. An empty list clears it.

func WithEffectiveUserPath 

func WithEffectiveUserPath(ctx context.Context, userPath string) context.Context

WithEffectiveUserPath returns a new context with an effective user path override attached.

func WithEnforceReturningUsageData 

func WithEnforceReturningUsageData(ctx context.Context, enforce bool) context.Context

WithEnforceReturningUsageData returns a new context with the streaming usage policy attached.

func WithFailoverUsed 

func WithFailoverUsed(ctx context.Context) context.Context

WithFailoverUsed returns a new context marked as having used a failover model.

func WithGuardrailsHash 

func WithGuardrailsHash(ctx context.Context, hash string) context.Context

WithGuardrailsHash returns a new context with the guardrails hash attached. The hash is the SHA-256 of all applied guardrail rule IDs and their versions, computed post-patch in the translated inference handlers.

func WithIdempotencyKey added in v0.1.93 

func WithIdempotencyKey(ctx context.Context, key string) context.Context

WithIdempotencyKey overrides the idempotency key for provider calls made with ctx. An empty key clears it, which is how an attempt that sends a different request body (a failover to another model) opts out.

func WithPrimaryRouteSaturated 

func WithPrimaryRouteSaturated(ctx context.Context, err error) context.Context

WithPrimaryRouteSaturated marks the resolved primary route as rate-saturated. The stored error is the 429 the client would have received; dispatch uses it as the synthetic primary failure that triggers the failover sweep, and it surfaces unchanged when no failover target can take the request.

func WithRequestDialect added in v0.1.67 

func WithRequestDialect(ctx context.Context, dialect RequestDialect) context.Context

WithRequestDialect returns a context carrying the translated ingress dialect.

func WithRequestID 

func WithRequestID(ctx context.Context, requestID string) context.Context

WithRequestID returns a new context with the request ID attached.

func WithRequestLabels 

func WithRequestLabels(ctx context.Context, labels []string) context.Context

WithRequestLabels returns a new context with the request labels attached. Labels are extracted at ingress from configured tagging headers.

func WithRequestOrigin 

func WithRequestOrigin(ctx context.Context, origin RequestOrigin) context.Context

WithRequestOrigin returns a new context with the logical request origin attached.

func WithRequestSnapshot 

func WithRequestSnapshot(ctx context.Context, snapshot *RequestSnapshot) context.Context

WithRequestSnapshot returns a new context with the request snapshot attached.

func WithRewriteTokensSaved 

func WithRewriteTokensSaved(ctx context.Context, tokensSaved int) context.Context

WithRewriteTokensSaved returns a new context carrying the total prompt tokens that applied request rewriters estimate they removed. Non-positive totals leave the context unchanged.

func WithSessionID added in v0.1.63 

func WithSessionID(ctx context.Context, sessionID string) context.Context

WithSessionID returns a new context with the detected client session id attached.

func WithTaggingStripHeaders 

func WithTaggingStripHeaders(ctx context.Context, headers map[string]struct{}) context.Context

WithTaggingStripHeaders returns a new context carrying the canonical tagging header names that must not be forwarded to upstream providers.

func WithUserPathHeaderName 

func WithUserPathHeaderName(ctx context.Context, headerName string) context.Context

WithUserPathHeaderName returns a new context with a non-default configured user-path request header name attached. The default header is intentionally a no-op and does not clear an existing value.

func WithWhiteBoxPrompt 

func WithWhiteBoxPrompt(ctx context.Context, prompt *WhiteBoxPrompt) context.Context

WithWhiteBoxPrompt returns a new context with the white-box prompt attached.

func WithWorkflow 

func WithWorkflow(ctx context.Context, workflow *Workflow) context.Context

WithWorkflow returns a new context with the workflow attached.

Types

type AccessScope added in v0.1.87 

type AccessScope struct {
	UserPath string
}

AccessScope is the user-path subtree the authenticated credential may act on. It is derived from the credential alone (a managed key's or extension identity's bound user path), never from request headers, so it is the value ownership and admin-scoping checks must consult. The zero value is global: the master key, unauthenticated requests, and credentials without a bound user path can reach every user path, including rows that carry none.

func AccessScopeFromContext added in v0.1.87 

func AccessScopeFromContext(ctx context.Context) AccessScope

AccessScopeFromContext returns the credential's access scope. A context without one is global.

func (AccessScope) Allows added in v0.1.87 

func (s AccessScope) Allows(userPath string) bool

Allows reports whether userPath lies inside the scope: the scope root itself or any descendant. A non-global scope never admits an empty path, so rows written without a user path stay visible to global scopes only.

func (AccessScope) Global added in v0.1.87 

func (s AccessScope) Global() bool

Global reports whether the scope places no user-path restriction.

type AudioProvider 

type AudioProvider interface {
	CreateSpeech(ctx context.Context, req *AudioSpeechRequest) (*AudioResponse, error)
	CreateTranscription(ctx context.Context, req *AudioTranscriptionRequest) (*AudioResponse, error)
}

AudioProvider is implemented by providers that support OpenAI-compatible audio endpoints: text-to-speech (CreateSpeech) and speech-to-text (CreateTranscription). It is optional so providers without audio support can omit it.

type AudioResponse 

type AudioResponse struct {
	ContentType string
	Data        []byte
	// Stream carries a body the provider is still producing, so the gateway can
	// relay it as it arrives instead of holding the whole generation: chunked
	// speech audio and transcription server-sent events both trade badly for
	// time-to-first-byte when buffered. It is mutually exclusive with Data, and
	// the caller owns draining and closing it.
	Stream io.ReadCloser
}

AudioResponse wraps an opaque audio or transcription payload with its content type. Speech returns binary audio; transcription returns JSON or text depending on response_format. In both cases the gateway proxies the bytes verbatim.

type AudioSpeechRequest 

type AudioSpeechRequest struct {
	Model          string  `json:"model"`
	Input          string  `json:"input"`
	Voice          string  `json:"voice"`
	Instructions   string  `json:"instructions,omitempty"`
	ResponseFormat string  `json:"response_format,omitempty"`
	Speed          float64 `json:"speed,omitempty"`

	// Provider is gateway routing metadata, stripped before dispatching upstream.
	Provider string `json:"provider,omitempty"`

	ExtraFields UnknownJSONFields `json:"-" swaggerignore:"true"`
}

AudioSpeechRequest is an OpenAI-compatible POST /v1/audio/speech (text-to-speech) request. Only the fields the gateway needs to route, validate, and audit are typed; every other member (stream_format, model-specific extras, ...) is preserved verbatim in ExtraFields and forwarded upstream so new provider parameters work without a gateway change (ADR-0011 rule 1).

func DecodeAudioSpeechRequest 

func DecodeAudioSpeechRequest(body []byte, _ *WhiteBoxPrompt) (*AudioSpeechRequest, error)

DecodeAudioSpeechRequest decodes a JSON text-to-speech request body. The semantic envelope is unused: audio responses are binary and not response-cached.

func (AudioSpeechRequest) MarshalJSON added in v0.1.92 

func (r AudioSpeechRequest) MarshalJSON() ([]byte, error)

func (*AudioSpeechRequest) UnmarshalJSON added in v0.1.92 

func (r *AudioSpeechRequest) UnmarshalJSON(data []byte) error

type AudioTranscriptionRequest 

type AudioTranscriptionRequest struct {
	Model                  string
	Filename               string
	FileContentType        string
	File                   []byte
	FileReader             io.Reader
	Language               string
	Prompt                 string
	ResponseFormat         string
	Temperature            string
	TimestampGranularities []string
	// Fields carries the form values the gateway does not consume itself
	// (include[], chunking_strategy, stream, provider-native extras, ...). They
	// are forwarded upstream verbatim (ADR-0011 rule 1). Repeated names keep
	// their relative order; ordering across different names is not preserved
	// (multipart forms have no cross-field order semantics).
	Fields []FormField

	// Provider is gateway routing metadata, stripped before dispatching upstream.
	Provider string
}

AudioTranscriptionRequest is an OpenAI-compatible POST /v1/audio/transcriptions (speech-to-text) request. The upstream call is multipart/form-data, so the audio bytes and form fields are transport data rather than a JSON body.

type AudioTranslationProvider added in v0.1.68 

type AudioTranslationProvider interface {
	CreateTranslation(ctx context.Context, req *AudioTranscriptionRequest) (*AudioResponse, error)
}

AudioTranslationProvider is implemented by audio providers that support translating spoken audio into English through POST /v1/audio/translations. It is separate from AudioProvider because transcription support does not imply translation support for every upstream provider.

type AvailabilityChecker 

type AvailabilityChecker interface {
	// CheckAvailability verifies the provider's backend service is reachable.
	// Returns nil if available, error otherwise. Initialization logs failures but
	// keeps the provider registered so later refreshes can retry discovery.
	//
	// The caller owns the deadline: implementations honor ctx and must not
	// impose a timeout of their own, so startup and the request path can each
	// probe on the budget that suits them.
	CheckAvailability(ctx context.Context) error
}

AvailabilityChecker is an optional interface for providers that can report backend reachability during startup diagnostics.

type BatchCreateHintAwareProvider 

type BatchCreateHintAwareProvider interface {
	CreateBatchWithHints(ctx context.Context, req *BatchRequest) (*BatchResponse, map[string]string, error)
}

BatchCreateHintAwareProvider is an optional native batch extension for providers that need gateway persistence for per-item endpoint hints.

type BatchError 

type BatchError struct {
	Type    string `json:"type"`
	Message string `json:"message"`
}

BatchError represents a normalized error for a failed batch item.

type BatchFileTransport 

type BatchFileTransport interface {
	GetFileContent(ctx context.Context, providerType, id string) (*FileContentResponse, error)
	CreateFile(ctx context.Context, providerType string, req *FileCreateRequest) (*FileObject, error)
}

BatchFileTransport is the minimal provider-native file API surface needed to preprocess file-backed batch requests.

type BatchItemRewriteFunc 

type BatchItemRewriteFunc func(context.Context, BatchRequestItem, *DecodedBatchItemRequest) (json.RawMessage, error)

BatchItemRewriteFunc rewrites a decoded batch item body for provider submission. The original batch item is provided so callers can preserve non-semantic JSON structure when needed.

type BatchListResponse 

type BatchListResponse struct {
	Object  string          `json:"object"`
	Data    []BatchResponse `json:"data"`
	HasMore bool            `json:"has_more"`
	FirstID string          `json:"first_id,omitempty"`
	LastID  string          `json:"last_id,omitempty"`
}

BatchListResponse is returned by GET /v1/batches.

type BatchPreparationMetadata 

type BatchPreparationMetadata struct {
	OriginalInputFileID  string
	RewrittenInputFileID string
}

BatchPreparationMetadata captures request-scoped batch preprocessing effects that are useful for persistence and debugging but should not be exposed as public API fields automatically.

func GetBatchPreparationMetadata 

func GetBatchPreparationMetadata(ctx context.Context) *BatchPreparationMetadata

GetBatchPreparationMetadata retrieves batch preprocessing metadata from the context.

func (*BatchPreparationMetadata) RecordInputFileRewrite 

func (m *BatchPreparationMetadata) RecordInputFileRewrite(original, rewritten string)

RecordInputFileRewrite tracks the first user-supplied provider file id and the latest derived provider file id submitted upstream.

type BatchRequest 

type BatchRequest struct {
	InputFileID      string             `json:"input_file_id,omitempty"`
	Endpoint         string             `json:"endpoint,omitempty"`
	CompletionWindow string             `json:"completion_window,omitempty"`
	Metadata         map[string]string  `json:"metadata,omitempty"`
	Requests         []BatchRequestItem `json:"requests,omitempty"`
	ExtraFields      UnknownJSONFields  `json:"-" swaggerignore:"true"`
}

BatchRequest is OpenAI-compatible for core fields and extends with inline requests.

OpenAI-compatible fields:

  • input_file_id
  • endpoint
  • completion_window
  • metadata

Gateway extension:

  • requests (inline payloads for providers that support native inline batch bodies)

func DecodeBatchRequest 

func DecodeBatchRequest(body []byte, env *WhiteBoxPrompt) (*BatchRequest, error)

DecodeBatchRequest decodes and caches the canonical batch request for a semantic envelope.

func (BatchRequest) MarshalJSON 

func (r BatchRequest) MarshalJSON() ([]byte, error)

func (*BatchRequest) UnmarshalJSON 

func (r *BatchRequest) UnmarshalJSON(data []byte) error

type BatchRequestCounts 

type BatchRequestCounts struct {
	Total     int `json:"total"`
	Completed int `json:"completed"`
	Failed    int `json:"failed"`
}

BatchRequestCounts is OpenAI-compatible aggregate batch status.

type BatchRequestItem 

type BatchRequestItem struct {
	CustomID    string            `json:"custom_id,omitempty"`
	Method      string            `json:"method,omitempty"`
	URL         string            `json:"url"`
	Body        json.RawMessage   `json:"body" swaggertype:"object"`
	ExtraFields UnknownJSONFields `json:"-" swaggerignore:"true"`
}

BatchRequestItem represents one sub-request in an inline batch.

func (BatchRequestItem) MarshalJSON 

func (r BatchRequestItem) MarshalJSON() ([]byte, error)

func (*BatchRequestItem) UnmarshalJSON 

func (r *BatchRequestItem) UnmarshalJSON(data []byte) error

type BatchResponse 

type BatchResponse struct {
	ID               string             `json:"id"`
	Object           string             `json:"object"`
	Provider         string             `json:"provider,omitempty"`
	ProviderBatchID  string             `json:"provider_batch_id,omitempty"`
	Endpoint         string             `json:"endpoint"`
	InputFileID      string             `json:"input_file_id,omitempty"`
	CompletionWindow string             `json:"completion_window,omitempty"`
	Status           string             `json:"status"`
	CreatedAt        int64              `json:"created_at"`
	InProgressAt     *int64             `json:"in_progress_at,omitempty"`
	CompletedAt      *int64             `json:"completed_at,omitempty"`
	FailedAt         *int64             `json:"failed_at,omitempty"`
	CancellingAt     *int64             `json:"cancelling_at,omitempty"`
	CancelledAt      *int64             `json:"cancelled_at,omitempty"`
	RequestCounts    BatchRequestCounts `json:"request_counts"`
	Metadata         map[string]string  `json:"metadata,omitempty"`

	// Gateway extension: optional usage/result snapshots persisted by the gateway.
	Usage   BatchUsageSummary `json:"usage"`
	Results []BatchResultItem `json:"results,omitempty"`
}

BatchResponse uses OpenAI-compatible batch fields and includes provider mapping plus optional cached results.

type BatchResultHintAwareProvider 

type BatchResultHintAwareProvider interface {
	GetBatchResultsWithHints(ctx context.Context, id string, endpointByCustomID map[string]string) (*BatchResultsResponse, error)
	ClearBatchResultHints(batchID string)
}

BatchResultHintAwareProvider is an optional native batch extension for providers that need persisted per-item endpoint hints to shape results.

type BatchResultItem 

type BatchResultItem struct {
	Index      int         `json:"index"`
	CustomID   string      `json:"custom_id,omitempty"`
	URL        string      `json:"url"`
	StatusCode int         `json:"status_code"`
	Model      string      `json:"model,omitempty"`
	Provider   string      `json:"provider,omitempty"`
	Response   any         `json:"response,omitempty"`
	Error      *BatchError `json:"error,omitempty"`
}

BatchResultItem represents one sub-response in a batch.

type BatchResultsResponse 

type BatchResultsResponse struct {
	Object  string            `json:"object"`
	BatchID string            `json:"batch_id"`
	Data    []BatchResultItem `json:"data"`
}

BatchResultsResponse is returned by GET /v1/batches/{id}/results.

type BatchRewriteResult 

type BatchRewriteResult struct {
	Request              *BatchRequest
	RequestEndpointHints map[string]string
	OriginalInputFileID  string
	RewrittenInputFileID string
}

BatchRewriteResult captures the normalized request plus any gateway-only metadata derived while rewriting inline or file-backed batch sources.

func RewriteBatchSource 

func RewriteBatchSource(
	ctx context.Context,
	providerType string,
	req *BatchRequest,
	fileTransport BatchFileTransport,
	operations []Operation,
	rewrite BatchItemRewriteFunc,
) (*BatchRewriteResult, error)

RewriteBatchSource normalizes both inline and file-backed batch sources using the same typed per-item rewrite callback.

type BatchRouteInfo 

type BatchRouteInfo struct {
	Action   string
	BatchID  string
	After    string
	LimitRaw string
	Limit    int
	HasLimit bool
}

BatchRouteInfo is sparse canonical metadata the gateway can derive for /v1/batches* routes. The full create payload remains in BatchRequest when the gateway lazily decodes JSON bodies.

func BatchRouteMetadata 

func BatchRouteMetadata(env *WhiteBoxPrompt, method, path string, routeParams map[string]string, queryParams map[string][]string) (*BatchRouteInfo, error)

BatchRouteMetadata returns sparse batch route semantics, caching them on the envelope when present.

func DeriveBatchRouteInfoFromTransport 

func DeriveBatchRouteInfoFromTransport(method, path string, routeParams map[string]string, queryParams map[string][]string) *BatchRouteInfo

DeriveBatchRouteInfoFromTransport derives sparse batch route info from transport metadata.

type BatchUsageSummary 

type BatchUsageSummary struct {
	InputTokens  int `json:"input_tokens"`
	OutputTokens int `json:"output_tokens"`
	TotalTokens  int `json:"total_tokens"`

	InputCost  *float64 `json:"input_cost,omitempty"`
	OutputCost *float64 `json:"output_cost,omitempty"`
	TotalCost  *float64 `json:"total_cost,omitempty"`
}

BatchUsageSummary aggregates usage and cost for successful batch items.

type BodyMode 

type BodyMode string

BodyMode describes the transport shape expected for an endpoint. 

const (
	BodyModeNone      BodyMode = "none"
	BodyModeJSON      BodyMode = "json"
	BodyModeMultipart BodyMode = "multipart"
	BodyModeOpaque    BodyMode = "opaque"
)

type CapabilitySet 

type CapabilitySet struct {
	SemanticExtraction bool
	AliasResolution    bool
	Guardrails         bool
	RequestPatching    bool
	UsageTracking      bool
	ResponseCaching    bool
	Streaming          bool
	Passthrough        bool
}

CapabilitySet advertises the gateway behaviors that are valid for a request. This is intentionally small and pragmatic for the initial workflow slice.

func CapabilitiesForEndpoint 

func CapabilitiesForEndpoint(desc EndpointDescriptor) CapabilitySet

CapabilitiesForEndpoint returns the current capability set for one endpoint.

type ChatRequest 

type ChatRequest struct {
	Temperature       *float64          `json:"temperature,omitempty"`
	TopP              *float64          `json:"top_p,omitempty"`
	MaxTokens         *int              `json:"max_tokens,omitempty"`
	Model             string            `json:"model"`
	Provider          string            `json:"provider,omitempty"` // Gateway routing hint; stripped before upstream execution.
	Messages          []Message         `json:"messages"`
	Tools             []map[string]any  `json:"tools,omitempty"`
	ToolChoice        any               `json:"tool_choice,omitempty"` // string or object
	ParallelToolCalls *bool             `json:"parallel_tool_calls,omitempty"`
	Stream            bool              `json:"stream,omitempty"`
	StreamOptions     *StreamOptions    `json:"stream_options,omitempty"`
	Reasoning         *Reasoning        `json:"reasoning,omitempty"`
	User              string            `json:"user,omitempty"`
	ServiceTier       string            `json:"service_tier,omitempty"`
	ExtraFields       UnknownJSONFields `json:"-" swaggerignore:"true"`
	// PromptCachePlan carries gateway-internal, post-routing cache metadata.
	// It is never serialized to clients or upstream OpenAI-compatible APIs.
	PromptCachePlan *PromptCachePlan `json:"-" swaggerignore:"true"`
}

ChatRequest represents the incoming chat completion request

func DecodeChatRequest 

func DecodeChatRequest(body []byte, env *WhiteBoxPrompt) (*ChatRequest, error)

DecodeChatRequest decodes and caches the canonical chat request for a semantic envelope.

func (ChatRequest) MarshalJSON 

func (r ChatRequest) MarshalJSON() ([]byte, error)

func (*ChatRequest) UnmarshalJSON 

func (r *ChatRequest) UnmarshalJSON(data []byte) error

UnmarshalJSON decodes the typed fields via an alias (so new fields are picked up automatically) and captures every other member in ExtraFields.

func (*ChatRequest) WithStreaming 

func (r *ChatRequest) WithStreaming() *ChatRequest

WithStreaming returns a shallow copy of the request with Stream set to true. This avoids mutating the caller's request object.

type ChatResponse 

type ChatResponse struct {
	ID                string   `json:"id"`
	Object            string   `json:"object"`
	Model             string   `json:"model"`
	Provider          string   `json:"provider"`
	SystemFingerprint string   `json:"system_fingerprint,omitempty"`
	Choices           []Choice `json:"choices"`
	Usage             Usage    `json:"usage"`
	Created           int64    `json:"created"`
	// ExtraFields keeps provider response members the gateway does not model
	// (for example OpenRouter's top-level extras) so they reach the client.
	ExtraFields UnknownJSONFields `json:"-" swaggerignore:"true"`
}

ChatResponse represents the chat completion response

func (ChatResponse) MarshalJSON added in v0.1.86 

func (r ChatResponse) MarshalJSON() ([]byte, error)

func (*ChatResponse) UnmarshalJSON added in v0.1.86 

func (r *ChatResponse) UnmarshalJSON(data []byte) error

type Choice 

type Choice struct {
	Message      ResponseMessage `json:"message"`
	FinishReason string          `json:"finish_reason"`
	Index        int             `json:"index"`
	Logprobs     json.RawMessage `json:"logprobs,omitempty" swaggertype:"object"`
	// StopSequence is the matched stop sequence when the provider reports one
	// natively (Anthropic stop_reason "stop_sequence"). OpenAI's finish_reason
	// "stop" conflates natural stops with stop-parameter hits, so this is an
	// extension field: present only when the provider knows the answer, in the
	// same spirit as the relayed reasoning_content extension.
	StopSequence string            `json:"stop_sequence,omitempty"`
	ExtraFields  UnknownJSONFields `json:"-" swaggerignore:"true"`
}

Choice represents a single completion choice

func (Choice) MarshalJSON added in v0.1.86 

func (c Choice) MarshalJSON() ([]byte, error)

func (*Choice) UnmarshalJSON added in v0.1.86 

func (c *Choice) UnmarshalJSON(data []byte) error

type CompletionTokensDetails 

type CompletionTokensDetails struct {
	ReasoningTokens          int `json:"reasoning_tokens"`
	AudioTokens              int `json:"audio_tokens"`
	AcceptedPredictionTokens int `json:"accepted_prediction_tokens"`
	RejectedPredictionTokens int `json:"rejected_prediction_tokens"`
}

CompletionTokensDetails holds extended output token breakdown (OpenAI/xAI).

type ContentPart 

type ContentPart struct {
	Type        string             `json:"type"`
	Text        string             `json:"text,omitempty"`
	VideoURL    *VideoURLContent   `json:"video_url,omitempty"`
	ImageURL    *ImageURLContent   `json:"image_url,omitempty"`
	InputAudio  *InputAudioContent `json:"input_audio,omitempty"`
	File        *FileContent       `json:"file,omitempty"`
	ExtraFields UnknownJSONFields  `json:"-" swaggerignore:"true"`
}

ContentPart represents a single OpenAI-compatible multimodal chat content part.

func NormalizeContentParts 

func NormalizeContentParts(content any) ([]ContentPart, bool)

NormalizeContentParts converts dynamic JSON-decoded content into typed parts.

func (ContentPart) MarshalJSON 

func (p ContentPart) MarshalJSON() ([]byte, error)

func (*ContentPart) UnmarshalJSON 

func (p *ContentPart) UnmarshalJSON(data []byte) error

type Conversation 

type Conversation struct {
	ID        string            `json:"id"`
	Object    string            `json:"object"` // "conversation"
	CreatedAt int64             `json:"created_at"`
	Metadata  map[string]string `json:"metadata"`
}

Conversation is the OpenAI-compatible conversation resource returned by the /v1/conversations endpoints.

type ConversationCreateRequest 

type ConversationCreateRequest struct {
	Items    []json.RawMessage `json:"items,omitempty" swaggertype:"array,object"`
	Metadata map[string]string `json:"metadata,omitempty"`
}

ConversationCreateRequest is the accepted body for POST /v1/conversations. Items are stored as opaque JSON so the gateway accepts any item shape the client sends without constraining future item-list support.

func DecodeConversationCreateRequest 

func DecodeConversationCreateRequest(data []byte) (*ConversationCreateRequest, error)

DecodeConversationCreateRequest parses a conversation create body. An empty body is treated as an empty request (a conversation with no items/metadata).

type ConversationDeleteResponse 

type ConversationDeleteResponse struct {
	ID      string `json:"id"`
	Object  string `json:"object"` // "conversation.deleted"
	Deleted bool   `json:"deleted"`
}

ConversationDeleteResponse is returned by DELETE /v1/conversations/{id}.

type ConversationItemCreateRequest added in v0.1.58 

type ConversationItemCreateRequest struct {
	Items []json.RawMessage `json:"items" binding:"required" swaggertype:"array,object"`
}

ConversationItemCreateRequest is accepted by POST /v1/conversations/{id}/items.

func DecodeConversationItemCreateRequest added in v0.1.58 

func DecodeConversationItemCreateRequest(data []byte) (*ConversationItemCreateRequest, error)

DecodeConversationItemCreateRequest parses a conversation item batch.

type ConversationItemListParams added in v0.1.58 

type ConversationItemListParams struct {
	After   string
	Include []string
	Limit   int
	Order   string
}

ConversationItemListParams contains query parameters accepted by GET /v1/conversations/{id}/items.

type ConversationItemListResponse added in v0.1.58 

type ConversationItemListResponse struct {
	Object  string            `json:"object"`
	Data    []json.RawMessage `json:"data" swaggertype:"array,object"`
	FirstID *string           `json:"first_id" extensions:"x-nullable"`
	LastID  *string           `json:"last_id" extensions:"x-nullable"`
	HasMore bool              `json:"has_more"`
}

ConversationItemListResponse is returned when creating or listing conversation items. Data stays raw so new OpenAI item variants can pass through the gateway without waiting for a typed Go model.

type ConversationUpdateRequest 

type ConversationUpdateRequest struct {
	Metadata *map[string]string `json:"metadata" binding:"required"`
}

ConversationUpdateRequest is the accepted body for POST /v1/conversations/{id}. Metadata is a pointer so the handler can tell an absent field apart from an explicit empty object: OpenAI requires metadata on update.

func DecodeConversationUpdateRequest 

func DecodeConversationUpdateRequest(data []byte) (*ConversationUpdateRequest, error)

DecodeConversationUpdateRequest parses a conversation update body. An empty body decodes to a request with no metadata, which the handler rejects.

type DecodedBatchItemHandlers 

type DecodedBatchItemHandlers[T any] struct {
	Chat       func(*ChatRequest) (T, error)
	Responses  func(*ResponsesRequest) (T, error)
	Embeddings func(*EmbeddingRequest) (T, error)
	Default    func(*DecodedBatchItemRequest) (T, error)
}

DecodedBatchItemHandlers contains operation-specific handlers for a decoded batch item request. Downstream consumers can use this instead of switching on operation names directly.

type DecodedBatchItemRequest 

type DecodedBatchItemRequest struct {
	Endpoint  string
	Method    string
	Operation Operation
	Request   any
}

DecodedBatchItemRequest is the canonical decode result for known JSON batch subrequests.

func DecodeKnownBatchItemRequest 

func DecodeKnownBatchItemRequest(defaultEndpoint string, item BatchRequestItem) (*DecodedBatchItemRequest, error)

DecodeKnownBatchItemRequest normalizes and decodes a known JSON batch subrequest.

func MaybeDecodeKnownBatchItemRequest 

func MaybeDecodeKnownBatchItemRequest(defaultEndpoint string, item BatchRequestItem, operations ...Operation) (*DecodedBatchItemRequest, bool, error)

MaybeDecodeKnownBatchItemRequest selectively decodes a known JSON batch subrequest only when it targets one of the requested operations. Non-POST, body-less, or unmatched items are reported as not handled.

func (*DecodedBatchItemRequest) RequestedModelSelector 

func (decoded *DecodedBatchItemRequest) RequestedModelSelector() (RequestedModelSelector, error)

RequestedModelSelector returns the raw selector requested by the decoded batch item, preserving whether the provider came from the explicit field.

type EmbeddingData 

type EmbeddingData struct {
	Object    string          `json:"object"`
	Embedding json.RawMessage `json:"embedding" swaggertype:"object"`
	Index     int             `json:"index"`
}

EmbeddingData represents a single embedding data point. Embedding is json.RawMessage to support both float arrays and base64-encoded strings.

type EmbeddingRequest 

type EmbeddingRequest struct {
	Model          string            `json:"model"`
	Provider       string            `json:"provider,omitempty"` // Gateway routing hint; stripped before upstream execution.
	Input          any               `json:"input"`
	EncodingFormat string            `json:"encoding_format,omitempty"`
	Dimensions     *int              `json:"dimensions,omitempty"`
	ExtraFields    UnknownJSONFields `json:"-" swaggerignore:"true"`
}

EmbeddingRequest represents the incoming embeddings request (OpenAI-compatible).

func DecodeEmbeddingRequest 

func DecodeEmbeddingRequest(body []byte, env *WhiteBoxPrompt) (*EmbeddingRequest, error)

DecodeEmbeddingRequest decodes and caches the canonical embeddings request for a semantic envelope.

func (EmbeddingRequest) MarshalJSON 

func (r EmbeddingRequest) MarshalJSON() ([]byte, error)

func (*EmbeddingRequest) UnmarshalJSON 

func (r *EmbeddingRequest) UnmarshalJSON(data []byte) error

type EmbeddingResponse 

type EmbeddingResponse struct {
	Object   string          `json:"object"`
	Data     []EmbeddingData `json:"data"`
	Model    string          `json:"model"`
	Provider string          `json:"provider"`
	Usage    EmbeddingUsage  `json:"usage"`
}

EmbeddingResponse represents the embeddings response (OpenAI-compatible).

type EmbeddingUsage 

type EmbeddingUsage struct {
	PromptTokens int `json:"prompt_tokens"`
	TotalTokens  int `json:"total_tokens"`
}

EmbeddingUsage represents token usage information for embeddings.

type EndpointDescriptor 

type EndpointDescriptor struct {
	ModelInteraction bool
	IngressManaged   bool
	Dialect          string
	Operation        Operation
	BodyMode         BodyMode
}

EndpointDescriptor centralizes the transport-facing classification of model and provider routes.

func DescribeEndpoint 

func DescribeEndpoint(method, path string) EndpointDescriptor

DescribeEndpoint classifies a request path and method for ADR-0002 ingress handling.

func DescribeEndpointPath 

func DescribeEndpointPath(path string) EndpointDescriptor

DescribeEndpointPath classifies a request path for ADR-0002 ingress handling.

type ErrorType 

type ErrorType string

ErrorType represents the type of error that occurred 

const (
	// ErrorTypeProvider indicates an upstream provider error (5xx)
	ErrorTypeProvider ErrorType = "provider_error"
	// ErrorTypeRateLimit indicates a rate limit error (429)
	ErrorTypeRateLimit ErrorType = "rate_limit_error"
	// ErrorTypeInvalidRequest indicates a client error (4xx)
	ErrorTypeInvalidRequest ErrorType = "invalid_request_error"
	// ErrorTypeAuthentication indicates an authentication error (401)
	ErrorTypeAuthentication ErrorType = "authentication_error"
	// ErrorTypeNotFound indicates a not found error (404)
	ErrorTypeNotFound ErrorType = "not_found_error"
	// ErrorTypePermission indicates an authenticated caller lacks the
	// permission for the operation (403)
	ErrorTypePermission ErrorType = "permission_error"
	// ErrorTypeInternal indicates a failure inside the gateway itself (500),
	// as opposed to one an upstream provider reported
	ErrorTypeInternal ErrorType = "internal_error"
)

type ExecutionMode 

type ExecutionMode string

ExecutionMode describes how the gateway intends to execute a request. 

const (
	ExecutionModeTranslated  ExecutionMode = "translated"
	ExecutionModePassthrough ExecutionMode = "passthrough"
	ExecutionModeNativeBatch ExecutionMode = "native_batch"
	ExecutionModeNativeFile  ExecutionMode = "native_file"
)

type FileContent added in v0.1.87 

type FileContent struct {
	FileData    string            `json:"file_data,omitempty"`
	FileURL     string            `json:"file_url,omitempty"`
	FileID      string            `json:"file_id,omitempty"`
	Filename    string            `json:"filename,omitempty"`
	ExtraFields UnknownJSONFields `json:"-" swaggerignore:"true"`
}

FileContent carries a document attachment for file parts, mirroring the OpenAI Chat Completions file input. FileData is inline content as a data: URL (PDF or plain text); FileURL is a remote http(s) document (the Responses API input_file.file_url); FileID references a provider-side uploaded file. Anthropic document blocks translate to and from this part.

func (FileContent) MarshalJSON added in v0.1.87 

func (c FileContent) MarshalJSON() ([]byte, error)

func (*FileContent) UnmarshalJSON added in v0.1.87 

func (c *FileContent) UnmarshalJSON(data []byte) error

type FileContentResponse 

type FileContentResponse struct {
	ID          string
	Filename    string
	ContentType string
	Data        []byte
}

FileContentResponse wraps raw file bytes with response metadata.

type FileCreateRequest 

type FileCreateRequest struct {
	Purpose       string    `json:"purpose"`
	Filename      string    `json:"filename,omitempty"`
	Content       []byte    `json:"-"`
	ContentReader io.Reader `json:"-"`
}

FileCreateRequest represents an OpenAI-compatible file upload request. The actual request is multipart/form-data; Content is not serialized.

type FileDeleteResponse 

type FileDeleteResponse struct {
	ID      string `json:"id"`
	Object  string `json:"object"`
	Deleted bool   `json:"deleted"`
}

FileDeleteResponse is returned by DELETE /v1/files/{id}.

type FileListResponse 

type FileListResponse struct {
	Object  string       `json:"object"`
	Data    []FileObject `json:"data"`
	HasMore bool         `json:"has_more,omitempty"`
}

FileListResponse is returned by GET /v1/files.

type FileMultipartMetadataReader 

type FileMultipartMetadataReader interface {
	Value(name string) string
	Filename(name string) (string, bool)
}

FileMultipartMetadataReader exposes the small subset of multipart form data needed for sparse file-create semantics.

type FileObject 

type FileObject struct {
	ID            string  `json:"id"`
	Object        string  `json:"object"`
	Bytes         int64   `json:"bytes"`
	CreatedAt     int64   `json:"created_at"`
	Filename      string  `json:"filename"`
	Purpose       string  `json:"purpose"`
	Status        string  `json:"status,omitempty"`
	StatusDetails *string `json:"status_details,omitempty"`

	// Gateway enrichment for multi-provider deployments.
	Provider string `json:"provider,omitempty"`
}

FileObject represents an OpenAI-compatible file object.

type FileRouteInfo 

type FileRouteInfo struct {
	Action   string
	Provider string
	Purpose  string
	Filename string
	FileID   string
	After    string
	LimitRaw string
	Limit    int
	HasLimit bool
}

FileRouteInfo is sparse canonical metadata the gateway can derive for /v1/files* routes. It intentionally excludes file bytes, which remain transport data rather than semantic data.

func DeriveFileRouteInfoFromTransport 

func DeriveFileRouteInfoFromTransport(method, path string, routeParams map[string]string, queryParams map[string][]string) *FileRouteInfo

DeriveFileRouteInfoFromTransport derives sparse file route info from transport metadata.

func EnrichFileCreateRouteInfo 

func EnrichFileCreateRouteInfo(req *FileRouteInfo, reader FileMultipartMetadataReader) *FileRouteInfo

EnrichFileCreateRouteInfo enriches req with provider, purpose, and filename metadata extracted from a multipart reader for file-create requests. It returns req unchanged when req is nil, req.Action is not FileActionCreate, or reader is nil.

func FileRouteMetadata 

func FileRouteMetadata(env *WhiteBoxPrompt, method, path string, routeParams map[string]string, queryParams map[string][]string) (*FileRouteInfo, error)

FileRouteMetadata returns sparse file route semantics, caching them on the envelope when present.

type FormField added in v0.1.81 

type FormField struct {
	Name  string
	Value string
}

FormField is a single multipart form value forwarded upstream unchanged.

type FunctionCall 

type FunctionCall struct {
	Name        string            `json:"name"`
	Arguments   string            `json:"arguments"`
	ExtraFields UnknownJSONFields `json:"-" swaggerignore:"true"`
}

FunctionCall contains the function name and serialized arguments payload.

func (FunctionCall) MarshalJSON 

func (f FunctionCall) MarshalJSON() ([]byte, error)

FunctionCall.MarshalJSON marshals a FunctionCall to JSON, including unknown JSON members from ExtraFields. alias inherits FunctionCall's fields and tags but drops MarshalJSON so json.Marshal does not recurse; ExtraFields (json:"-") is merged separately.

func (*FunctionCall) UnmarshalJSON 

func (f *FunctionCall) UnmarshalJSON(data []byte) error

FunctionCall.UnmarshalJSON unmarshals a FunctionCall from JSON, preserving unknown JSON members in ExtraFields.

type GatewayError 

type GatewayError struct {
	Type       ErrorType `json:"type"`
	Message    string    `json:"message"`
	StatusCode int       `json:"status_code"`
	Provider   string    `json:"provider,omitempty"`
	Param      *string   `json:"param" extensions:"x-nullable"`
	Code       *string   `json:"code" extensions:"x-nullable"`
	// Original error for debugging (not exposed to clients)
	Err error `json:"-"`
	// ResponseBody and ResponseHeaders carry the raw upstream error response so
	// failed provider attempts can be audited. Never serialized to API clients.
	ResponseBody    []byte      `json:"-"`
	ResponseHeaders http.Header `json:"-"`
}

GatewayError is the base error type for all gateway errors

func NewAuthenticationError 

func NewAuthenticationError(provider string, message string) *GatewayError

NewAuthenticationError creates a new authentication error (401)

func NewEmptyProviderResponseError 

func NewEmptyProviderResponseError(provider string) *GatewayError

NewEmptyProviderResponseError reports that a provider returned no response body (502).

func NewInvalidRequestError 

func NewInvalidRequestError(message string, err error) *GatewayError

NewInvalidRequestError creates a new invalid request error (400)

func NewInvalidRequestErrorWithStatus 

func NewInvalidRequestErrorWithStatus(statusCode int, message string, err error) *GatewayError

NewInvalidRequestErrorWithStatus creates a new invalid request error with a specific status code

func NewModelNotFoundError 

func NewModelNotFoundError(model string) *GatewayError

NewModelNotFoundError reports a model the gateway cannot route. It mirrors OpenAI's contract for unknown models — HTTP 404 with code "model_not_found" — so clients that key on the status or code behave the same as against OpenAI.

func NewNoChoicesProviderError added in v0.1.91 

func NewNoChoicesProviderError(provider string) *GatewayError

NewNoChoicesProviderError reports a chat completion that succeeded upstream but carried no choices (502), so failover treats it as a failed attempt.

func NewNotFoundError 

func NewNotFoundError(message string) *GatewayError

NewNotFoundError creates a new not found error (404)

func NewPermissionError added in v0.1.87 

func NewPermissionError(message string) *GatewayError

NewPermissionError creates a new permission error (403) for an authenticated caller whose credential does not cover the operation.

func NewProviderError 

func NewProviderError(provider string, statusCode int, message string, err error) *GatewayError

NewProviderError creates a new provider error (upstream 5xx)

func NewRateLimitError 

func NewRateLimitError(provider string, message string) *GatewayError

NewRateLimitError creates a new rate limit error (429)

func ParseEmbeddedProviderError added in v0.1.86 

func ParseEmbeddedProviderError(provider string, body []byte) *GatewayError

ParseEmbeddedProviderError converts a bare {"error": ...} payload delivered with a 2xx status (OpenRouter is the canonical offender) into the GatewayError the provider effectively returned. An HTTP status carried in error.code is preserved, anything else maps to 502. Returns nil for normal success payloads. The returned error wraps ErrEmbeddedInSuccess.

func ParseProviderError 

func ParseProviderError(provider string, statusCode int, body []byte, originalErr error) *GatewayError

ParseProviderError parses an error response from a provider and returns an appropriate GatewayError

func ValidateConversationMetadata 

func ValidateConversationMetadata(metadata map[string]string) *GatewayError

ValidateConversationMetadata enforces the OpenAI metadata limits (at most 16 pairs, keys up to 64 characters, values up to 512 characters). It returns nil when the metadata is acceptable.

func (*GatewayError) Error 

func (e *GatewayError) Error() string

Error implements the error interface

func (*GatewayError) HTTPStatusCode 

func (e *GatewayError) HTTPStatusCode() int

HTTPStatusCode returns the appropriate HTTP status code for this error

func (*GatewayError) ToJSON 

func (e *GatewayError) ToJSON() map[string]any

ToJSON converts the error to a JSON-compatible map

func (*GatewayError) Unwrap 

func (e *GatewayError) Unwrap() error

Unwrap implements the error unwrapping interface

func (*GatewayError) WithCode 

func (e *GatewayError) WithCode(code string) *GatewayError

WithCode annotates the error with a machine-readable error code.

func (*GatewayError) WithParam 

func (e *GatewayError) WithParam(param string) *GatewayError

WithParam annotates the error with the offending parameter name.

func (*GatewayError) WithResponseBody added in v0.1.93 

func (e *GatewayError) WithResponseBody(body []byte) *GatewayError

WithResponseBody attaches a bounded copy of the raw upstream response body for auditing. Providers that report failures inside a 2xx body use this to retain the same evidence ParseProviderError captures for non-2xx responses.

type ImageData added in v0.1.81 

type ImageData struct {
	URL           string `json:"url,omitempty"`
	B64JSON       string `json:"b64_json,omitempty"`
	RevisedPrompt string `json:"revised_prompt,omitempty"`
}

ImageData is one generated image: either a hosted URL or inline base64 bytes, depending on the model and response_format.

type ImageEditProvider added in v0.1.81 

type ImageEditProvider interface {
	CreateImageEdit(ctx context.Context, req *ImageEditRequest) (*ImageGenerationResponse, error)
}

ImageEditProvider is implemented by providers that support the OpenAI-compatible image edit endpoint (POST /v1/images/edits). It is separate from ImageProvider because the upstream call is multipart/form-data and not every provider that generates images accepts that shape; the router discovers support by interface assertion.

type ImageEditRequest added in v0.1.81 

type ImageEditRequest struct {
	Model  string
	Prompt string
	// Images holds the source image(s) to edit. DALL·E 2 accepts exactly one;
	// gpt-image-1 accepts several (sent upstream as image[]).
	Images []ImageFile
	// Mask optionally marks the area to edit (transparent pixels are replaced).
	Mask *ImageFile
	// Fields carries the remaining form fields. Repeated names are kept and
	// preserve their relative order; ordering across different names is not
	// preserved (multipart forms have no cross-field order semantics).
	Fields []FormField

	// Provider is gateway routing metadata, stripped before dispatching upstream.
	Provider string
}

ImageEditRequest is an OpenAI-compatible POST /v1/images/edits request. The upstream call is multipart/form-data, so the image bytes and form fields are transport data rather than a JSON body. Only the fields the gateway needs to route and validate are typed; every other form field (n, size, quality, response_format, background, output_format, input_fidelity, user, ...) is preserved in Fields and forwarded upstream verbatim so new provider parameters work without a gateway change.

func (*ImageEditRequest) Field added in v0.1.81 

func (r *ImageEditRequest) Field(name string) (string, bool)

Field returns the first value of the named form field and whether it was set.

type ImageFile added in v0.1.81 

type ImageFile struct {
	Filename    string
	ContentType string
	Data        []byte
}

ImageFile is one uploaded image part of a multipart image request.

type ImageGenerationRequest added in v0.1.81 

type ImageGenerationRequest struct {
	Model          string `json:"model" binding:"required"`
	Prompt         string `json:"prompt" binding:"required"`
	N              *int   `json:"n,omitempty" minimum:"1"`
	ResponseFormat string `json:"response_format,omitempty"`
	Size           string `json:"size,omitempty"`
	Quality        string `json:"quality,omitempty"`
	User           string `json:"user,omitempty"`
	// Stream is typed only so the gateway can reject it: streaming image
	// generation returns server-sent events, which this endpoint does not relay.
	Stream bool `json:"stream,omitempty"`

	// Provider is gateway routing metadata, stripped before dispatching upstream.
	Provider string `json:"provider,omitempty"`

	ExtraFields UnknownJSONFields `json:"-" swaggerignore:"true"`
}

ImageGenerationRequest is an OpenAI-compatible POST /v1/images/generations request. Only the fields the gateway needs to route, validate, and audit are typed; every other field (quality, size, style, background, output_format, moderation, ...) is preserved verbatim in ExtraFields and forwarded upstream so new provider parameters work without a gateway change.

func DecodeImageGenerationRequest added in v0.1.81 

func DecodeImageGenerationRequest(body []byte, _ *WhiteBoxPrompt) (*ImageGenerationRequest, error)

DecodeImageGenerationRequest decodes a JSON image generation request body. The semantic envelope is unused: image responses are not response-cached.

func (*ImageGenerationRequest) ImageCount added in v0.1.81 

func (r *ImageGenerationRequest) ImageCount() int

ImageCount returns the number of images the request asks for, defaulting to one when n is omitted (OpenAI's default).

func (ImageGenerationRequest) MarshalJSON added in v0.1.81 

func (r ImageGenerationRequest) MarshalJSON() ([]byte, error)

func (*ImageGenerationRequest) UnmarshalJSON added in v0.1.81 

func (r *ImageGenerationRequest) UnmarshalJSON(data []byte) error

type ImageGenerationResponse added in v0.1.81 

type ImageGenerationResponse struct {
	Created      int64       `json:"created"`
	Data         []ImageData `json:"data"`
	Background   string      `json:"background,omitempty"`
	OutputFormat string      `json:"output_format,omitempty"`
	Quality      string      `json:"quality,omitempty"`
	Size         string      `json:"size,omitempty"`
	Usage        *ImageUsage `json:"usage,omitempty"`

	// Provider is a gateway addition, stamped like every other routed response,
	// so clients can tell which provider served the request.
	Provider string `json:"provider,omitempty"`
}

ImageGenerationResponse is the OpenAI-compatible images response envelope. Providers that report output parameters (gpt-image-1 echoes background, output_format, quality and size) or token usage have them passed through; providers that do not simply omit them.

type ImageProvider added in v0.1.81 

type ImageProvider interface {
	CreateImage(ctx context.Context, req *ImageGenerationRequest) (*ImageGenerationResponse, error)
}

ImageProvider is implemented by providers that support the OpenAI-compatible image generation endpoint (POST /v1/images/generations). It is optional so providers without image models can omit it; the router discovers support by interface assertion.

type ImageTokenDetails added in v0.1.81 

type ImageTokenDetails struct {
	TextTokens  int `json:"text_tokens"`
	ImageTokens int `json:"image_tokens"`
}

ImageTokenDetails splits image input tokens between text and image inputs.

type ImageURLContent 

type ImageURLContent struct {
	URL         string            `json:"url"`
	Detail      string            `json:"detail,omitempty"`
	MediaType   string            `json:"media_type,omitempty"`
	ExtraFields UnknownJSONFields `json:"-" swaggerignore:"true"`
}

ImageURLContent contains an image reference for image_url parts.

func (ImageURLContent) MarshalJSON 

func (c ImageURLContent) MarshalJSON() ([]byte, error)

func (*ImageURLContent) UnmarshalJSON 

func (c *ImageURLContent) UnmarshalJSON(data []byte) error

type ImageUsage added in v0.1.81 

type ImageUsage struct {
	InputTokens        int                `json:"input_tokens"`
	OutputTokens       int                `json:"output_tokens"`
	TotalTokens        int                `json:"total_tokens"`
	InputTokensDetails *ImageTokenDetails `json:"input_tokens_details,omitempty"`
}

ImageUsage is the token usage block returned by token-billed image models (gpt-image-1). DALL·E models omit it entirely.

type InputAudioContent 

type InputAudioContent struct {
	Data        string            `json:"data"`
	Format      string            `json:"format,omitempty"`
	ExtraFields UnknownJSONFields `json:"-" swaggerignore:"true"`
}

InputAudioContent contains inline audio payload metadata.

func (InputAudioContent) MarshalJSON 

func (a InputAudioContent) MarshalJSON() ([]byte, error)

func (*InputAudioContent) UnmarshalJSON 

func (a *InputAudioContent) UnmarshalJSON(data []byte) error

type Message 

type Message struct {
	Role        string `json:"role"`
	ToolCallID  string `json:"tool_call_id,omitempty"`
	ContentNull bool   `json:"-"`
	// Content accepts either a plain string or an array of ContentPart values.
	// This preserves OpenAI-compatible multimodal chat payloads.
	Content MessageContent `json:"content"`
	//nolint:govet // Intentional duplicate json tag for Swagger docs: content is null OR string OR []ContentPart.
	// ContentSchema documents that `content` accepts either a plain string
	// or an array of ContentPart values.
	ContentSchema []ContentPart     `` /* 166-byte string literal not displayed */
	ToolCalls     []ToolCall        `json:"tool_calls,omitempty"`
	ExtraFields   UnknownJSONFields `json:"-" swaggerignore:"true"`
}

Message represents a single message in the chat.

func (Message) MarshalJSON 

func (m Message) MarshalJSON() ([]byte, error)

Message.MarshalJSON emits validated chat request message content, preserves null handling, and includes unknown JSON members from ExtraFields.

func (*Message) UnmarshalJSON 

func (m *Message) UnmarshalJSON(data []byte) error

Message.UnmarshalJSON validates chat request message content, preserves unknown JSON members in ExtraFields, and keeps null content handling intact.

type MessageContent 

type MessageContent any

MessageContent stores message content as either text or structured parts.

type MessagesTokenCounter added in v0.1.91 

type MessagesTokenCounter interface {
	CountMessagesTokens(ctx context.Context, model string, body []byte) (int, error)
}

MessagesTokenCounter is implemented by providers that count the input tokens of an Anthropic Messages request exactly through an endpoint of their own. It is optional: the router answers ErrMessagesTokenCountUnsupported for a route whose provider lacks it, and the Messages API estimates instead. The body is the client's original request; the provider forwards the fields its endpoint accepts with model replaced by the resolved one.

type Model 

type Model struct {
	ID      string `json:"id"`
	Object  string `json:"object"`
	OwnedBy string `json:"owned_by"`
	Created int64  `json:"created"`
	// Metadata holds optional enrichment data (display name, pricing, capabilities, etc.).
	// May be nil if the model was not found in the external registry.
	Metadata *ModelMetadata `json:"metadata,omitempty"`
}

Model represents a single model in the models list

type ModelCategory 

type ModelCategory string

ModelCategory represents a model's functional category for UI grouping. 

const (
	CategoryAll            ModelCategory = "all"
	CategoryTextGeneration ModelCategory = "text_generation"
	CategoryEmbedding      ModelCategory = "embedding"
	CategoryImage          ModelCategory = "image"
	CategoryAudio          ModelCategory = "audio"
	CategoryVideo          ModelCategory = "video"
	CategoryUtility        ModelCategory = "utility"
)

func AllCategories 

func AllCategories() []ModelCategory

AllCategories returns the ordered list of categories for UI rendering.

func CategoriesForModes 

func CategoriesForModes(modes []string) []ModelCategory

CategoriesForModes returns deduplicated ModelCategory values for the given mode strings. Unrecognized modes are silently skipped.

type ModelLookup 

type ModelLookup interface {
	// Supports returns true if the registry has a provider for the given model
	Supports(model string) bool

	// GetProvider returns the provider for the given model, or nil if not found
	GetProvider(model string) Provider

	// GetProviderType returns the provider type string for the given model.
	// Returns empty string if the model is not found.
	GetProviderType(model string) string

	// ListModels returns all models in the registry
	ListModels() []Model

	// ModelCount returns the number of registered models
	ModelCount() int

	// GetProviderName maps a model selector back to the concrete configured
	// provider instance name. Implementations that have no such mapping return
	// an empty string. Same shape as the optional ProviderNameResolver
	// interface used elsewhere for provider-side type assertions.
	GetProviderName(model string) string

	// GetProviderNameForType maps a provider type such as "openai" to the
	// concrete configured instance name chosen for routing, e.g.
	// "openai-primary". Returns empty when no mapping exists.
	GetProviderNameForType(providerType string) string

	// GetProviderTypeForName maps a concrete configured instance name back to
	// its provider type. Returns empty when no mapping exists.
	GetProviderTypeForName(providerName string) string
}

ModelLookup defines the interface for looking up models and their providers. This abstraction allows the Router to be decoupled from the concrete ModelRegistry implementation.

Implementations normalize on write: every provider name, provider type, and model ID they return is whitespace-trimmed, so callers compare the values directly.

type ModelMetadata 

type ModelMetadata struct {
	DisplayName     string                  `json:"display_name,omitempty" yaml:"display_name,omitempty"`
	Description     string                  `json:"description,omitempty" yaml:"description,omitempty"`
	Family          string                  `json:"family,omitempty" yaml:"family,omitempty"`
	Modes           []string                `json:"modes,omitempty" yaml:"modes,omitempty"`
	Categories      []ModelCategory         `json:"categories,omitempty" yaml:"categories,omitempty"`
	Tags            []string                `json:"tags,omitempty" yaml:"tags,omitempty"`
	ContextWindow   *int                    `json:"context_window,omitempty" yaml:"context_window,omitempty"`
	MaxOutputTokens *int                    `json:"max_output_tokens,omitempty" yaml:"max_output_tokens,omitempty"`
	Capabilities    map[string]bool         `json:"capabilities,omitempty" yaml:"capabilities,omitempty"`
	Rankings        map[string]ModelRanking `json:"rankings,omitempty" yaml:"rankings,omitempty"`
	Pricing         *ModelPricing           `json:"pricing,omitempty" yaml:"pricing,omitempty"`
	PricingSources  map[string]string       `json:"pricing_sources,omitempty" yaml:"-"`
}

ModelMetadata holds enriched metadata from the external model registry. YAML tags mirror the JSON field names so operators can declare metadata overrides in config.yaml in the same shape that appears in /v1/models output.

func (*ModelMetadata) Clone 

func (m *ModelMetadata) Clone() *ModelMetadata

Clone returns a deep copy so callers can safely mutate the result without affecting the original. Slices, maps, and pointer fields are re-allocated.

type ModelPricing 

type ModelPricing struct {
	Currency               string   `json:"currency" yaml:"currency"`
	InputPerMtok           *float64 `json:"input_per_mtok,omitempty" yaml:"input_per_mtok,omitempty"`
	OutputPerMtok          *float64 `json:"output_per_mtok,omitempty" yaml:"output_per_mtok,omitempty"`
	CachedInputPerMtok     *float64 `json:"cached_input_per_mtok,omitempty" yaml:"cached_input_per_mtok,omitempty"`
	CacheWritePerMtok      *float64 `json:"cache_write_per_mtok,omitempty" yaml:"cache_write_per_mtok,omitempty"`
	ReasoningOutputPerMtok *float64 `json:"reasoning_output_per_mtok,omitempty" yaml:"reasoning_output_per_mtok,omitempty"`
	BatchInputPerMtok      *float64 `json:"batch_input_per_mtok,omitempty" yaml:"batch_input_per_mtok,omitempty"`
	BatchOutputPerMtok     *float64 `json:"batch_output_per_mtok,omitempty" yaml:"batch_output_per_mtok,omitempty"`
	AudioInputPerMtok      *float64 `json:"audio_input_per_mtok,omitempty" yaml:"audio_input_per_mtok,omitempty"`
	AudioOutputPerMtok     *float64 `json:"audio_output_per_mtok,omitempty" yaml:"audio_output_per_mtok,omitempty"`
	// OutputImagePerMtok prices generated image tokens, which providers bill at a
	// different rate from text output (OpenAI gpt-image-1: $40/Mtok image output
	// and no text output rate at all; Gemini 3 Pro Image: $120/Mtok image output
	// versus $12/Mtok text). It is the output rate on the image endpoints.
	OutputImagePerMtok *float64 `json:"output_image_per_mtok,omitempty" yaml:"output_image_per_mtok,omitempty"`
	// PerImage prices a returned image as a flat unit, for models that report no
	// token usage at all (DALL·E, Imagen, grok-imagine). It is an alternative
	// expression of the same charge as OutputImagePerMtok, never an addition to
	// it: catalog entries carry both, so only one may be applied (see
	// usage.CalculateGranularCost).
	PerImage          *float64           `json:"per_image,omitempty" yaml:"per_image,omitempty"`
	InputPerImage     *float64           `json:"input_per_image,omitempty" yaml:"input_per_image,omitempty"`
	PerSecondInput    *float64           `json:"per_second_input,omitempty" yaml:"per_second_input,omitempty"`
	PerSecondOutput   *float64           `json:"per_second_output,omitempty" yaml:"per_second_output,omitempty"`
	PerCharacterInput *float64           `json:"per_character_input,omitempty" yaml:"per_character_input,omitempty"`
	PerRequest        *float64           `json:"per_request,omitempty" yaml:"per_request,omitempty"`
	PerPage           *float64           `json:"per_page,omitempty" yaml:"per_page,omitempty"`
	Tiers             []ModelPricingTier `json:"tiers,omitempty" yaml:"tiers,omitempty"`
	// TimeWindows carry rates that replace the base prices during recurring
	// UTC windows (see ModelPricingTimeWindow). Base prices are the standard
	// (peak) rates; use AtTime to resolve the rates in effect at a moment.
	TimeWindows []ModelPricingTimeWindow `json:"time_windows,omitempty" yaml:"time_windows,omitempty"`
}

ModelPricing holds pricing information for cost calculation.

func (*ModelPricing) AtTime added in v0.1.87 

func (p *ModelPricing) AtTime(t time.Time) *ModelPricing

AtTime returns the pricing in effect at t: the first time window containing t (evaluated in UTC) replaces the base rates it publishes. Without windows, without a match, or for a zero t the receiver itself is returned, so the standard rates always apply when the moment of use is unknown.

func (*ModelPricing) Clone 

func (p *ModelPricing) Clone() *ModelPricing

Clone returns a deep copy so callers can safely mutate the result without affecting the original. Pointer fields, Tiers, and TimeWindows are re-allocated.

func (*ModelPricing) DropTimeWindowRatesOverriddenBy added in v0.1.87 

func (p *ModelPricing) DropTimeWindowRatesOverriddenBy(override *ModelPricing)

DropTimeWindowRatesOverriddenBy removes from every time window the rates whose base field the override sets, so an operator's rate is not undercut by a catalog discount that was published against a different base price. Windows left without any rate are dropped.

func (*ModelPricing) FieldSources 

func (p *ModelPricing) FieldSources(source string) map[string]string

FieldSources returns non-empty pricing field names mapped to source. Callers should pass a non-empty source string. Tiered pricing is reported as the coarse "tiers" key rather than per-tier entries.

func (*ModelPricing) TimeWindowAt added in v0.1.87 

func (p *ModelPricing) TimeWindowAt(t time.Time) (ModelPricingTimeWindow, bool)

TimeWindowAt returns the first time window containing t, if any.

type ModelPricingTier 

type ModelPricingTier struct {
	UpToTokens    *float64 `json:"up_to_tokens,omitempty" yaml:"up_to_tokens,omitempty"`
	UpToMtok      *float64 `json:"up_to_mtok,omitempty" yaml:"up_to_mtok,omitempty"`
	InputPerMtok  *float64 `json:"input_per_mtok,omitempty" yaml:"input_per_mtok,omitempty"`
	OutputPerMtok *float64 `json:"output_per_mtok,omitempty" yaml:"output_per_mtok,omitempty"`
}

ModelPricingTier represents a volume-based pricing tier.

type ModelPricingTimeWindow added in v0.1.87 

type ModelPricingTimeWindow struct {
	Label     string                      `json:"label" yaml:"label"`
	UTCRanges []ModelPricingUTCRange      `json:"utc_ranges" yaml:"utc_ranges"`
	Pricing   ModelPricingTimeWindowRates `json:"pricing" yaml:"pricing"`
}

ModelPricingTimeWindow is a recurring window during which some per-token rates replace the base prices — DeepSeek's off-peak hours, for example. The base prices stay the standard (peak) rates, so a consumer that ignores the windows never understates cost.

func (ModelPricingTimeWindow) Contains added in v0.1.87 

func (w ModelPricingTimeWindow) Contains(t time.Time) bool

Contains reports whether t (evaluated in UTC) falls in any of the window's ranges.

type ModelPricingTimeWindowRates added in v0.1.87 

type ModelPricingTimeWindowRates struct {
	InputPerMtok       *float64 `json:"input_per_mtok,omitempty" yaml:"input_per_mtok,omitempty"`
	OutputPerMtok      *float64 `json:"output_per_mtok,omitempty" yaml:"output_per_mtok,omitempty"`
	CachedInputPerMtok *float64 `json:"cached_input_per_mtok,omitempty" yaml:"cached_input_per_mtok,omitempty"`
	CacheWritePerMtok  *float64 `json:"cache_write_per_mtok,omitempty" yaml:"cache_write_per_mtok,omitempty"`
}

ModelPricingTimeWindowRates are the base fields a time window can replace. Absent fields keep their base price.

type ModelPricingUTCRange added in v0.1.87 

type ModelPricingUTCRange struct {
	Days  []string `json:"days,omitempty" yaml:"days,omitempty"`
	Start string   `json:"start" yaml:"start"`
	End   string   `json:"end" yaml:"end"`
}

ModelPricingUTCRange is a half-open daily UTC range [start, end). An end at or before the start wraps past midnight into the next day. Days optionally limits the range to weekdays ("mon" … "sun"); a wrapping range starts on a listed day and spills into the following one.

func (ModelPricingUTCRange) Contains added in v0.1.87 

func (r ModelPricingUTCRange) Contains(t time.Time) bool

Contains reports whether t (evaluated in UTC) falls in the range. A range with an unparseable bound never matches, so malformed catalog data falls back to the base rates rather than a discount.

type ModelRanking 

type ModelRanking struct {
	Elo  *float64 `json:"elo,omitempty" yaml:"elo,omitempty"`
	Rank *int     `json:"rank,omitempty" yaml:"rank,omitempty"`
	AsOf string   `json:"as_of,omitempty" yaml:"as_of,omitempty"`
}

ModelRanking holds one benchmark or leaderboard entry for a model.

func CloneModelRanking 

func CloneModelRanking(r ModelRanking) ModelRanking

CloneModelRanking returns a deep copy so the caller can mutate pointer fields (Elo, Rank) without affecting the original.

type ModelSelector 

type ModelSelector struct {
	Model    string
	Provider string
}

ModelSelector is a normalized model routing selector. Model is always the raw upstream model ID (without provider prefix).

func ParseModelSelector 

func ParseModelSelector(model, provider string) (ModelSelector, error)

ParseModelSelector normalizes model/provider routing input.

Accepted forms:

  • model only: "gpt-4o"
  • model with prefix: "openai/gpt-4o"
  • explicit provider field: provider="openai", model="gpt-4o"
  • explicit provider with raw slash model: provider="groq", model="openai/gpt-oss-120b"

When provider is explicit, it is authoritative. A matching leading "provider/" prefix on the model is stripped once as redundant qualification.

func (ModelSelector) QualifiedModel 

func (s ModelSelector) QualifiedModel() string

QualifiedModel returns "provider/model" when Provider is set, or only model otherwise.

type ModelsResponse 

type ModelsResponse struct {
	Object string  `json:"object"`
	Data   []Model `json:"data"`
}

ModelsResponse represents the response from the /v1/models endpoint

type NativeBatchDeleteProvider added in v0.1.53 

type NativeBatchDeleteProvider interface {
	DeleteBatch(ctx context.Context, id string) error
}

NativeBatchDeleteProvider is an optional native batch extension for providers whose upstream supports deleting an ended batch (the Anthropic Message Batches dialect exposes DELETE; the OpenAI batch API does not).

type NativeBatchDeleteRoutableProvider added in v0.1.53 

type NativeBatchDeleteRoutableProvider interface {
	DeleteBatch(ctx context.Context, providerType, id string) error
}

NativeBatchDeleteRoutableProvider extends routing with native batch deletion.

type NativeBatchHintRoutableProvider 

type NativeBatchHintRoutableProvider interface {
	CreateBatchWithHints(ctx context.Context, providerType string, req *BatchRequest) (*BatchResponse, map[string]string, error)
	GetBatchResultsWithHints(ctx context.Context, providerType, id string, endpointByCustomID map[string]string) (*BatchResultsResponse, error)
	ClearBatchResultHints(providerType, batchID string)
}

NativeBatchHintRoutableProvider is an optional routing extension for providers that can consume persisted per-item endpoint hints.

type NativeBatchProvider 

type NativeBatchProvider interface {
	CreateBatch(ctx context.Context, req *BatchRequest) (*BatchResponse, error)
	GetBatch(ctx context.Context, id string) (*BatchResponse, error)
	ListBatches(ctx context.Context, limit int, after string) (*BatchListResponse, error)
	CancelBatch(ctx context.Context, id string) (*BatchResponse, error)
	GetBatchResults(ctx context.Context, id string) (*BatchResultsResponse, error)
}

NativeBatchProvider is implemented by providers that support native discounted batching. This is intentionally separate from Provider so unsupported providers can still implement regular synchronous APIs without batch capabilities.

type NativeBatchProviderTypeLister 

type NativeBatchProviderTypeLister interface {
	NativeBatchProviderTypes() []string
}

NativeBatchProviderTypeLister exposes registered provider types that support native batch operations.

type NativeBatchRoutableProvider 

type NativeBatchRoutableProvider interface {
	CreateBatch(ctx context.Context, providerType string, req *BatchRequest) (*BatchResponse, error)
	GetBatch(ctx context.Context, providerType, id string) (*BatchResponse, error)
	ListBatches(ctx context.Context, providerType string, limit int, after string) (*BatchListResponse, error)
	CancelBatch(ctx context.Context, providerType, id string) (*BatchResponse, error)
	GetBatchResults(ctx context.Context, providerType, id string) (*BatchResultsResponse, error)
}

NativeBatchRoutableProvider extends routing with native batch operations.

type NativeFileProvider 

type NativeFileProvider interface {
	CreateFile(ctx context.Context, req *FileCreateRequest) (*FileObject, error)
	ListFiles(ctx context.Context, purpose string, limit int, after string) (*FileListResponse, error)
	GetFile(ctx context.Context, id string) (*FileObject, error)
	DeleteFile(ctx context.Context, id string) (*FileDeleteResponse, error)
	GetFileContent(ctx context.Context, id string) (*FileContentResponse, error)
}

NativeFileProvider is implemented by providers that support OpenAI-compatible files APIs.

type NativeFileProviderTypeLister 

type NativeFileProviderTypeLister interface {
	NativeFileProviderTypes() []string
}

NativeFileProviderTypeLister exposes registered provider types that support native file operations. This is an internal capability inventory and must not depend on the public model catalog.

type NativeFileRoutableProvider 

type NativeFileRoutableProvider interface {
	CreateFile(ctx context.Context, providerType string, req *FileCreateRequest) (*FileObject, error)
	ListFiles(ctx context.Context, providerType, purpose string, limit int, after string) (*FileListResponse, error)
	GetFile(ctx context.Context, providerType, id string) (*FileObject, error)
	DeleteFile(ctx context.Context, providerType, id string) (*FileDeleteResponse, error)
	GetFileContent(ctx context.Context, providerType, id string) (*FileContentResponse, error)
}

NativeFileRoutableProvider extends routing with provider-native file operations.

type NativeResponseLifecycleProvider 

type NativeResponseLifecycleProvider interface {
	GetResponse(ctx context.Context, id string, params ResponseRetrieveParams) (*ResponsesResponse, error)
	ListResponseInputItems(ctx context.Context, id string, params ResponseInputItemsParams) (*ResponseInputItemListResponse, error)
	CancelResponse(ctx context.Context, id string) (*ResponsesResponse, error)
	DeleteResponse(ctx context.Context, id string) (*ResponseDeleteResponse, error)
}

NativeResponseLifecycleProvider is implemented by providers that support OpenAI-compatible Responses lifecycle operations.

type NativeResponseLifecycleRoutableProvider 

type NativeResponseLifecycleRoutableProvider interface {
	GetResponse(ctx context.Context, providerType, id string, params ResponseRetrieveParams) (*ResponsesResponse, error)
	ListResponseInputItems(ctx context.Context, providerType, id string, params ResponseInputItemsParams) (*ResponseInputItemListResponse, error)
	CancelResponse(ctx context.Context, providerType, id string) (*ResponsesResponse, error)
	DeleteResponse(ctx context.Context, providerType, id string) (*ResponseDeleteResponse, error)
}

NativeResponseLifecycleRoutableProvider extends routing with provider-native Responses lifecycle operations.

type NativeResponseProviderTypeLister 

type NativeResponseProviderTypeLister interface {
	NativeResponseProviderTypes() []string
}

NativeResponseProviderTypeLister exposes registered provider types that support native Responses lifecycle operations.

type NativeResponseUtilityProvider 

type NativeResponseUtilityProvider interface {
	CountResponseInputTokens(ctx context.Context, req *ResponsesRequest) (*ResponseInputTokensResponse, error)
	CompactResponse(ctx context.Context, req *ResponsesRequest) (*ResponseCompactResponse, error)
}

NativeResponseUtilityProvider is implemented by providers that support OpenAI-compatible Responses utility operations.

type NativeResponseUtilityRoutableProvider 

type NativeResponseUtilityRoutableProvider interface {
	CountResponseInputTokens(ctx context.Context, providerType string, req *ResponsesRequest) (*ResponseInputTokensResponse, error)
	CompactResponse(ctx context.Context, providerType string, req *ResponsesRequest) (*ResponseCompactResponse, error)
}

NativeResponseUtilityRoutableProvider extends routing with provider-native Responses utility operations.

type OpenAIErrorEnvelope 

type OpenAIErrorEnvelope struct {
	Error OpenAIErrorObject `json:"error" binding:"required"`
}

OpenAIErrorEnvelope documents the public OpenAI-compatible error response.

type OpenAIErrorObject 

type OpenAIErrorObject struct {
	Type    ErrorType `json:"type" binding:"required"`
	Message string    `json:"message" binding:"required"`
	Param   *string   `json:"param" binding:"required" extensions:"x-nullable"`
	Code    *string   `json:"code" binding:"required" extensions:"x-nullable"`
	// Provider names the upstream provider that produced the error. It is
	// omitted for errors raised by the gateway itself.
	Provider string `json:"provider,omitempty"`
}

OpenAIErrorObject is the error object exposed in public API responses.

type Operation 

type Operation string

Operation identifies the gateway operation represented by an endpoint. 

const (
	OperationChatCompletions     Operation = "chat_completions"
	OperationResponses           Operation = "responses"
	OperationConversations       Operation = "conversations"
	OperationEmbeddings          Operation = "embeddings"
	OperationBatches             Operation = "batches"
	OperationFiles               Operation = "files"
	OperationAudioSpeech         Operation = "audio_speech"
	OperationAudioTranscriptions Operation = "audio_transcriptions"
	OperationAudioTranslations   Operation = "audio_translations"
	OperationImageGenerations    Operation = "image_generations"
	OperationImageEdits          Operation = "image_edits"
	OperationRealtime            Operation = "realtime"
	OperationProviderPassthrough Operation = "provider_passthrough"
	OperationMCP                 Operation = "mcp"
)

type PassthroughProvider 

type PassthroughProvider interface {
	Passthrough(ctx context.Context, req *PassthroughRequest) (*PassthroughResponse, error)
}

PassthroughProvider supports opaque provider-native forwarding.

type PassthroughRequest 

type PassthroughRequest struct {
	Method    string
	Endpoint  string
	Operation string // optional semantic GenAI operation derived at ingress
	Model     string // optional model derived from the opaque request body
	Stream    bool   // explicit streaming intent derived from the request body
	// StreamUncertain means ingress intentionally stopped its bounded body peek
	// before it could prove whether this opaque request streams.
	StreamUncertain bool
	Body            io.ReadCloser
	Headers         http.Header
	ProviderName    string // optional: concrete configured provider instance name for name-based routing
}

PassthroughRequest is the transport-oriented request for opaque provider-native forwarding.

type PassthroughResponse 

type PassthroughResponse struct {
	StatusCode int
	Headers    map[string][]string
	Body       io.ReadCloser
}

PassthroughResponse is the raw upstream response for opaque forwarding. Body is an io.ReadCloser returned by the upstream provider, and callers are responsible for closing it when they are finished with the response body.

type PassthroughRouteInfo 

type PassthroughRouteInfo struct {
	Provider           string // resolved provider type (e.g. "anthropic")
	ProviderName       string // original configured provider instance name (e.g. "teste")
	RawEndpoint        string
	NormalizedEndpoint string
	SemanticOperation  string
	GenAIOperation     string // standard GenAI operation, if this is an inference call
	Stream             bool   // explicit streaming intent derived from the request body
	StreamUncertain    bool   // bounded opaque-body inspection could not determine stream intent
	AuditPath          string
	Model              string
}

PassthroughRouteInfo is typed passthrough metadata derived from ingress transport and later enrichment stages.

RawEndpoint reflects the route-relative provider endpoint from the inbound path. NormalizedEndpoint, SemanticOperation, and AuditPath are optional and may be filled by later gateway enrichment before execution. Once cached on WhiteBoxPrompt or Workflow, it should be treated as immutable by later request stages.

type PassthroughSemanticEnricher 

type PassthroughSemanticEnricher interface {
	ProviderType() string
	Enrich(snapshot *RequestSnapshot, prompt *WhiteBoxPrompt, info *PassthroughRouteInfo) *PassthroughRouteInfo
}

PassthroughSemanticEnricher derives provider-specific passthrough metadata from ingress transport and best-effort prompt state before execution workflow resolution runs.

type PromptCachePlan added in v0.1.69 

type PromptCachePlan struct {
	Key string
}

PromptCachePlan identifies a provider-native cached prefix materialization.

type PromptTokensDetails 

type PromptTokensDetails struct {
	CachedTokens int `json:"cached_tokens"`
	AudioTokens  int `json:"audio_tokens"`
	TextTokens   int `json:"text_tokens"`
	ImageTokens  int `json:"image_tokens"`
}

PromptTokensDetails holds extended input token breakdown (OpenAI/xAI).

type Provider 

type Provider interface {
	// ChatCompletion executes a chat completion request
	ChatCompletion(ctx context.Context, req *ChatRequest) (*ChatResponse, error)

	// StreamChatCompletion returns a raw SSE stream (caller must close)
	StreamChatCompletion(ctx context.Context, req *ChatRequest) (io.ReadCloser, error)

	// ListModels returns the list of available models
	ListModels(ctx context.Context) (*ModelsResponse, error)

	// Responses executes a Responses API request (OpenAI-compatible)
	Responses(ctx context.Context, req *ResponsesRequest) (*ResponsesResponse, error)

	// StreamResponses returns a raw SSE stream for Responses API (caller must close)
	StreamResponses(ctx context.Context, req *ResponsesRequest) (io.ReadCloser, error)

	// Embeddings sends an embeddings request to the provider
	Embeddings(ctx context.Context, req *EmbeddingRequest) (*EmbeddingResponse, error)
}

Provider defines the interface for LLM providers

type ProviderNameResolver 

type ProviderNameResolver interface {
	GetProviderName(model string) string
}

ProviderNameResolver is an optional interface for components that can map a routed model selector back to the concrete configured provider instance name.

type ProviderNameTypeResolver 

type ProviderNameTypeResolver interface {
	GetProviderTypeForName(providerName string) string
}

ProviderNameTypeResolver is an optional interface for components that can map a concrete configured provider instance name such as "openai_primary" back to its provider type, such as "openai".

type ProviderTypeNameResolver 

type ProviderTypeNameResolver interface {
	GetProviderNameForType(providerType string) string
}

ProviderTypeNameResolver is an optional interface for components that can map a provider type such as "openai" to the concrete configured provider instance name used for routing, such as "openai_primary".

type RealtimeCallProvider 

type RealtimeCallProvider interface {
	RealtimeCallTarget(ctx context.Context, req *RealtimeRequest) (*RealtimeHTTPTarget, error)
	RealtimeClientSecretTarget(ctx context.Context, req *RealtimeRequest) (*RealtimeHTTPTarget, error)
}

RealtimeCallProvider is implemented by providers that expose OpenAI-compatible realtime HTTP signaling endpoints: SDP exchange for WebRTC calls and ephemeral client secrets for browser clients. It is optional, like RealtimeProvider; websocket-only realtime providers simply omit it.

type RealtimeCallRouter 

type RealtimeCallRouter interface {
	RealtimeCallTarget(ctx context.Context, req *RealtimeRequest) (*RealtimeHTTPTarget, error)
	RealtimeClientSecretTarget(ctx context.Context, req *RealtimeRequest) (*RealtimeHTTPTarget, error)
}

RealtimeCallRouter resolves realtime HTTP signaling targets for a request. The Router implements it by routing on the model, mirroring RealtimeRouter.

type RealtimeHTTPTarget 

type RealtimeHTTPTarget struct {
	URL     string
	Headers http.Header
}

RealtimeHTTPTarget describes an upstream HTTPS endpoint for realtime call signaling (WebRTC SDP exchange, ephemeral client secrets). Like the websocket target, it carries only the dial URL and the credential headers to inject; headers must never be logged.

type RealtimeIntentProvider added in v0.1.82 

type RealtimeIntentProvider interface {
	SupportsRealtimeIntent(intent string) bool
}

RealtimeIntentProvider is implemented by realtime providers that serve one or more specialized session intents. It is optional: a provider that omits it serves conversation sessions only, and the gateway rejects a specialized intent for its models instead of silently opening a conversation session in its place. A provider reports an intent only when every realtime surface it exposes honors it — websocket, WebRTC calls, and client secrets alike.

type RealtimeProvider 

type RealtimeProvider interface {
	RealtimeTarget(ctx context.Context, req *RealtimeRequest) (*RealtimeTarget, error)
}

RealtimeProvider is implemented by providers that expose an OpenAI-compatible realtime websocket endpoint. It is optional, like AudioProvider, so providers without realtime support simply omit it.

type RealtimeRequest 

type RealtimeRequest struct {
	Model    string
	Provider string
	CallID   string
	Intent   string
}

RealtimeRequest carries the resolved parameters for opening a realtime (speech-to-speech) websocket session. The model selects the provider; the optional Provider hint mirrors the audio endpoints. CallID, when set, attaches to an existing WebRTC/SIP call as a sideband websocket instead of opening a fresh model session. Intent, when set, asks the provider for one of the specialized session types above.

func (*RealtimeRequest) HasIntent added in v0.1.82 

func (r *RealtimeRequest) HasIntent(intent string) bool

HasIntent reports whether the request asks for the given session intent.

type RealtimeRouter 

type RealtimeRouter interface {
	RealtimeTarget(ctx context.Context, req *RealtimeRequest) (*RealtimeTarget, error)
}

RealtimeRouter resolves a realtime target for a request. The Router implements it by routing on the model (optionally constrained by a provider hint), so it backs both the typed /v1/realtime route and the /p/{provider}/v1/realtime passthrough upgrade.

type RealtimeTarget 

type RealtimeTarget struct {
	URL          string
	Headers      http.Header
	Subprotocols []string
	// PinSessionModel, when non-empty, names the model the gateway must force
	// into the client's session.update frames. Providers set it when the
	// upstream URL carries no model — as in OpenAI transcription sessions —
	// so the session payload, not the URL, selects the model; pinning keeps
	// that selection on the model the caller was authorized for. Providers
	// whose URL fixes the model leave it empty and no frame mapping happens.
	PinSessionModel string
	// MeterInputAudio asks the gateway to bill the session from the input audio
	// it relays, at the model's per-second input rate, when the session reports
	// no usage of its own. Providers set it for session types that may report
	// nothing: OpenAI translation sessions emit only transcript and audio
	// deltas and never a usage event, and a transcription session whose model
	// omits usage from its completed event reports nothing either. Such a
	// session is then recorded with its real duration instead of as free.
	//
	// It is a fallback, not a second meter: a session that recorded even one
	// usage entry of its own is already billed, and metering the same audio on
	// top of it would double-count the session. Session types that always
	// report their own usage leave it false and are never metered.
	MeterInputAudio bool
}

RealtimeTarget describes the upstream websocket a provider exposes for realtime sessions. Realtime is a transport concern, not a translation concern: the provider's event schema is the wire format, so the gateway only needs the dial URL and the credential headers to inject. Headers must never be logged.

type Reasoning 

type Reasoning struct {
	// Effort controls how much reasoning effort the model should use.
	// Valid values are "low", "medium", "high", "xhigh", and "max".
	// "xhigh" and "max" are supported by newer models such as Claude Opus 4.8;
	// providers downgrade unsupported levels to their nearest equivalent.
	Effort string `json:"effort,omitempty"`
}

Reasoning configures reasoning behavior for models that support extended thinking. This is used with OpenAI's o-series models and other reasoning-capable models.

type RequestDialect added in v0.1.67 

type RequestDialect string

RequestDialect identifies the external request shape translated at ingress.

func RequestDialectFromContext added in v0.1.67 

func RequestDialectFromContext(ctx context.Context) RequestDialect

RequestDialectFromContext returns the translated ingress dialect, if any.

type RequestModelResolution 

type RequestModelResolution struct {
	Requested        RequestedModelSelector
	ResolvedSelector ModelSelector
	ProviderType     string
	ProviderName     string
	AliasApplied     bool
	// Slowdown is the extra-time factor selected for this request. A value of
	// 0.5 adds 50% of measured inference time; zero disables slowdown.
	Slowdown float64
}

RequestModelResolution captures the requested model selector at ingress and the concrete selector chosen for execution after alias resolution.

func (*RequestModelResolution) RequestedQualifiedModel 

func (r *RequestModelResolution) RequestedQualifiedModel() string

RequestedQualifiedModel returns the canonical requested selector.

func (*RequestModelResolution) ResolvedQualifiedModel 

func (r *RequestModelResolution) ResolvedQualifiedModel() string

ResolvedQualifiedModel returns the concrete qualified model selected for execution.

type RequestOrigin 

type RequestOrigin string

RequestOrigin identifies whether a request came from an external caller or an internal gateway-owned workflow.

func GetRequestOrigin 

func GetRequestOrigin(ctx context.Context) RequestOrigin

GetRequestOrigin retrieves the request origin from context. When unset, external traffic is assumed.

type RequestSnapshot 

type RequestSnapshot struct {
	// Method is the inbound HTTP method.
	Method string
	// Path is the request URL path as received at ingress.
	Path string
	// UserPath is the canonical business hierarchy path sourced from the
	// configured user-path request header when provided.
	UserPath string

	// ContentType is the inbound Content-Type header value.
	ContentType string

	// BodyNotCaptured reports that the request body exceeded the capture limit,
	// so CapturedBody is omitted and the live body stream remains on the request.
	BodyNotCaptured bool
	// RequestID is the canonical request id propagated through context, headers,
	// providers, and audit records for this request.
	RequestID string
	// contains filtered or unexported fields
}

RequestSnapshot is the transport-level capture of an inbound request. It preserves the request as received at the HTTP boundary so later stages can extract semantics without losing fidelity while keeping mutable state behind defensive-copy accessors by default.

func GetRequestSnapshot 

func GetRequestSnapshot(ctx context.Context) *RequestSnapshot

GetRequestSnapshot retrieves the request snapshot from the context.

func NewRequestSnapshot 

func NewRequestSnapshot(method, path string, routeParams map[string]string, queryParams, headers map[string][]string, contentType string, capturedBody []byte, bodyNotCaptured bool, requestID string, traceMetadata map[string]string, userPath ...string) *RequestSnapshot

NewRequestSnapshot constructs a RequestSnapshot and defensively copies its mutable map and byte-slice inputs.

func NewRequestSnapshotWithOwnedMaps 

func NewRequestSnapshotWithOwnedMaps(method, path string, routeParams map[string]string, queryParams, headers map[string][]string, contentType string, capturedBody []byte, bodyNotCaptured bool, requestID string, traceMetadata map[string]string, userPath ...string) *RequestSnapshot

NewRequestSnapshotWithOwnedMaps constructs a RequestSnapshot that takes ownership of routeParams, queryParams, traceMetadata, and capturedBody (callers must not mutate them afterwards) while still defensively cloning headers, which is typically the live request header map mutated downstream.

Use this on the ingress hot path, where the route/query/trace maps and body are freshly built for the snapshot and would otherwise be cloned for no benefit.

func (*RequestSnapshot) CapturedBody 

func (s *RequestSnapshot) CapturedBody() []byte

CapturedBody returns a defensive copy of the captured request body bytes.

func (*RequestSnapshot) CapturedBodyView 

func (s *RequestSnapshot) CapturedBodyView() []byte

CapturedBodyView returns the captured request body bytes without cloning. Callers must treat the returned slice as read-only.

func (*RequestSnapshot) GetHeaders 

func (s *RequestSnapshot) GetHeaders() map[string][]string

GetHeaders returns a defensive copy of the captured request headers.

func (*RequestSnapshot) GetQueryParams 

func (s *RequestSnapshot) GetQueryParams() map[string][]string

GetQueryParams returns a defensive copy of the captured query parameters.

func (*RequestSnapshot) GetRouteParams 

func (s *RequestSnapshot) GetRouteParams() map[string]string

GetRouteParams returns a defensive copy of the captured route parameters.

func (*RequestSnapshot) GetTraceMetadata 

func (s *RequestSnapshot) GetTraceMetadata() map[string]string

GetTraceMetadata returns a defensive copy of the captured trace metadata.

func (*RequestSnapshot) HeadersView 

func (s *RequestSnapshot) HeadersView() map[string][]string

HeadersView returns the captured request headers without cloning. Callers must treat the returned map as read-only.

func (*RequestSnapshot) WithOwnedCapturedBody 

func (s *RequestSnapshot) WithOwnedCapturedBody(capturedBody []byte, bodyNotCaptured bool) *RequestSnapshot

WithOwnedCapturedBody returns a shallow-cloned snapshot with request body capture state replaced. capturedBody is taken as owned by the snapshot and must not be mutated after this call.

func (*RequestSnapshot) WithUserPath 

func (s *RequestSnapshot) WithUserPath(userPath string) *RequestSnapshot

WithUserPath returns a shallow-cloned snapshot with UserPath and the captured default user-path header rewritten to the provided canonical value.

func (*RequestSnapshot) WithUserPathHeader 

func (s *RequestSnapshot) WithUserPathHeader(userPath, headerName string) *RequestSnapshot

WithUserPathHeader returns a shallow-cloned snapshot with UserPath and the captured configured user-path header rewritten to the provided canonical value.

type RequestedModelSelector 

type RequestedModelSelector struct {
	Model            string
	ProviderHint     string
	ExplicitProvider bool
}

RequestedModelSelector captures the raw selector as provided by a caller before alias resolution or provider routing.

func BatchItemRequestedModelSelector 

func BatchItemRequestedModelSelector(defaultEndpoint string, item BatchRequestItem) (RequestedModelSelector, error)

BatchItemRequestedModelSelector derives the raw requested selector for a known JSON batch subrequest.

func NewRequestedModelSelector 

func NewRequestedModelSelector(model, providerHint string) RequestedModelSelector

NewRequestedModelSelector normalizes raw selector input while preserving whether the provider came from an explicit field rather than model syntax.

func (RequestedModelSelector) Normalize 

func (s RequestedModelSelector) Normalize() (ModelSelector, error)

Normalize returns the canonical routing selector for this request input.

func (RequestedModelSelector) RequestedQualifiedModel 

func (s RequestedModelSelector) RequestedQualifiedModel() string

RequestedQualifiedModel returns the canonical requested selector string used for audit and workflow reporting.

type ResolvedWorkflowPolicy 

type ResolvedWorkflowPolicy struct {
	VersionID string
	Version   int
	// ScopeProvider is the configured provider instance name stored on the matched workflow.
	ScopeProvider string
	ScopeModel    string
	ScopeUserPath string
	Name          string
	WorkflowHash  string
	Features      WorkflowFeatures
	// GuardrailsHash is the prompt-phase plugin chain hash; it feeds the
	// response cache key.
	GuardrailsHash string
	// ChainHashes holds the per-phase plugin chain hashes ("prompt",
	// "response", "stream"), for diagnostics and the admin views.
	ChainHashes map[string]string
}

ResolvedWorkflowPolicy is the request-scoped runtime projection of one matched persisted workflow version.

type ResponseCacheVeto added in v0.1.91 

type ResponseCacheVeto interface {
	NoStore() bool
}

ResponseCacheVeto is implemented by the per-request plugin state (Workflow.PluginState) so the response cache can ask whether a plugin decision asked for the response not to be stored.

type ResponseCompactRequest 

type ResponseCompactRequest ResponseInputTokensRequest

ResponseCompactRequest documents the request body accepted by POST /v1/responses/compact. It accepts exactly the same members as ResponseInputTokensRequest, so it is defined from that struct: the field set is written once and both utility endpoints stay in lockstep.

func (ResponseCompactRequest) MarshalJSON 

func (r ResponseCompactRequest) MarshalJSON() ([]byte, error)

MarshalJSON preserves the dynamic input payload while omitting Swagger-only schema fields.

func (*ResponseCompactRequest) UnmarshalJSON 

func (r *ResponseCompactRequest) UnmarshalJSON(data []byte) error

UnmarshalJSON preserves the dynamic input payload for gateway utility requests.

type ResponseCompactResponse 

type ResponseCompactResponse struct {
	ID        string                `json:"id"`
	Object    string                `json:"object"`
	CreatedAt int64                 `json:"created_at"`
	Output    []ResponsesOutputItem `json:"output"`
	Usage     *ResponsesUsage       `json:"usage,omitempty"`
	Error     *ResponsesError       `json:"error,omitempty"`
	Metadata  map[string]string     `json:"metadata,omitempty"`
	Provider  string                `json:"provider,omitempty"`
}

ResponseCompactResponse is returned by POST /v1/responses/compact.

type ResponseDeleteResponse 

type ResponseDeleteResponse struct {
	ID      string `json:"id"`
	Object  string `json:"object"`
	Deleted bool   `json:"deleted"`
}

ResponseDeleteResponse is returned by DELETE /v1/responses/{id}.

type ResponseInputItemListResponse 

type ResponseInputItemListResponse struct {
	Object  string            `json:"object"`
	Data    []json.RawMessage `json:"data" swaggertype:"array,object"`
	FirstID string            `json:"first_id,omitempty"`
	LastID  string            `json:"last_id,omitempty"`
	HasMore bool              `json:"has_more"`
}

ResponseInputItemListResponse is returned by GET /v1/responses/{id}/input_items.

type ResponseInputItemsParams 

type ResponseInputItemsParams struct {
	After   string
	Include []string
	Limit   int
	Order   string
}

ResponseInputItemsParams contains query parameters accepted by GET /v1/responses/{id}/input_items.

type ResponseInputTokensRequest 

type ResponseInputTokensRequest struct {
	Model              string            `json:"model,omitempty"`
	Provider           string            `json:"provider,omitempty"` // Gateway routing hint; stripped before upstream execution.
	Input              any               `json:"input,omitempty"`    // string or []ResponsesInputElement — see docs for array form
	Instructions       string            `json:"instructions,omitempty"`
	Tools              []map[string]any  `json:"tools,omitempty"`
	ToolChoice         any               `json:"tool_choice,omitempty"` // string or object
	ParallelToolCalls  *bool             `json:"parallel_tool_calls,omitempty"`
	Temperature        *float64          `json:"temperature,omitempty"`
	TopP               *float64          `json:"top_p,omitempty"`
	TopLogprobs        *int              `json:"top_logprobs,omitempty"`
	MaxOutputTokens    *int              `json:"max_output_tokens,omitempty"`
	Metadata           map[string]string `json:"metadata,omitempty"`
	Reasoning          *Reasoning        `json:"reasoning,omitempty"`
	Text               any               `json:"text,omitempty"`
	Include            []string          `json:"include,omitempty"`
	Truncation         string            `json:"truncation,omitempty"`
	Store              *bool             `json:"store,omitempty"`
	PreviousResponseID string            `json:"previous_response_id,omitempty"`
	// Conversation accepts either a conversation ID string or an object with id.
	Conversation         *ResponsesConversationRef `json:"conversation,omitempty"`
	Prompt               any                       `json:"prompt,omitempty"`
	PromptCacheRetention string                    `json:"prompt_cache_retention,omitempty"`
	ContextManagement    any                       `json:"context_management,omitempty"`
	User                 string                    `json:"user,omitempty"`
	ServiceTier          string                    `json:"service_tier,omitempty"`
	SafetyIdentifier     string                    `json:"safety_identifier,omitempty"`
	ExtraFields          UnknownJSONFields         `json:"-" swaggerignore:"true"`
}

ResponseInputTokensRequest documents the request body accepted by POST /v1/responses/input_tokens.

func (ResponseInputTokensRequest) MarshalJSON 

func (r ResponseInputTokensRequest) MarshalJSON() ([]byte, error)

MarshalJSON preserves the dynamic input payload while omitting Swagger-only schema fields.

func (*ResponseInputTokensRequest) UnmarshalJSON 

func (r *ResponseInputTokensRequest) UnmarshalJSON(data []byte) error

UnmarshalJSON preserves the dynamic input payload for gateway utility requests.

type ResponseInputTokensResponse 

type ResponseInputTokensResponse struct {
	Object      string `json:"object"`
	InputTokens int    `json:"input_tokens"`
}

ResponseInputTokensResponse is returned by POST /v1/responses/input_tokens.

type ResponseMessage 

type ResponseMessage struct {
	Role    string         `json:"role"`
	Content MessageContent `json:"content"`
	//nolint:govet // Intentional duplicate json tag for Swagger docs: content is null OR string OR []ContentPart.
	ContentSchema []ContentPart     `` /* 166-byte string literal not displayed */
	ToolCalls     []ToolCall        `json:"tool_calls,omitempty"`
	ExtraFields   UnknownJSONFields `json:"-" swaggerignore:"true"`
}

ResponseMessage represents a single assistant message in a chat response.

func (ResponseMessage) MarshalJSON 

func (m ResponseMessage) MarshalJSON() ([]byte, error)

ResponseMessage.MarshalJSON preserves OpenAI-compatible null content for tool-call response messages and includes unknown JSON members from ExtraFields.

func (*ResponseMessage) UnmarshalJSON 

func (m *ResponseMessage) UnmarshalJSON(data []byte) error

ResponseMessage.UnmarshalJSON validates chat response message content, preserves unknown JSON members in ExtraFields, and keeps tool-call null content handling intact.

type ResponseRetrieveParams 

type ResponseRetrieveParams struct {
	Include            []string
	IncludeObfuscation *bool
	StartingAfter      *int
	Stream             bool
}

ResponseRetrieveParams contains query parameters accepted by GET /v1/responses/{id}.

type ResponsesContentItem 

type ResponsesContentItem struct {
	Type       string             `json:"type"` // "output_text", "input_image", "input_audio", etc.
	Text       string             `json:"text,omitempty"`
	ImageURL   *ImageURLContent   `json:"image_url,omitempty"`
	InputAudio *InputAudioContent `json:"input_audio,omitempty"`
	// input_file items carry their file fields flat, unlike chat file parts.
	FileData string `json:"file_data,omitempty"`
	FileURL  string `json:"file_url,omitempty"`
	FileID   string `json:"file_id,omitempty"`
	Filename string `json:"filename,omitempty"`
	// Providers can return structured annotation objects here (for example
	// citations from native tools), so keep the payload shape liberal.
	Annotations []json.RawMessage `json:"annotations,omitempty" swaggertype:"array,object"`
}

ResponsesContentItem represents a content item in the output.

func (ResponsesContentItem) MarshalJSON added in v0.1.84 

func (c ResponsesContentItem) MarshalJSON() ([]byte, error)

MarshalJSON keeps `text` and `annotations` on the wire for output_text parts. Both are required by the OpenAI schema: OpenAI always returns `annotations: []` there and OpenAI SDK consumers (for example LangChain's Responses converter) index into it without a nil check, and an empty assistant answer must still serialize `text: ""` rather than drop the member. Input parts keep both fields only when a caller supplied them.

type ResponsesConversationRef 

type ResponsesConversationRef struct {
	ID  string          `json:"id,omitempty"`
	Raw json.RawMessage `json:"-" swaggerignore:"true"`
}

ResponsesConversationRef represents the Responses API conversation request field. OpenAI accepts either a conversation ID string or an object with id. Raw preserves the original string/object shape across JSON round trips.

func (ResponsesConversationRef) MarshalJSON 

func (c ResponsesConversationRef) MarshalJSON() ([]byte, error)

MarshalJSON preserves whether the conversation was originally supplied as a string or object. The ID field is authoritative so callers can update or clear a decoded reference without leaking the original raw value.

func (*ResponsesConversationRef) UnmarshalJSON 

func (c *ResponsesConversationRef) UnmarshalJSON(data []byte) error

UnmarshalJSON accepts the documented Responses conversation union: a string ID or an object with an id field.

type ResponsesError 

type ResponsesError struct {
	Code    string `json:"code"`
	Message string `json:"message"`
}

ResponsesError represents an error in the response.

type ResponsesIncompleteDetails added in v0.1.92 

type ResponsesIncompleteDetails struct {
	Reason string `json:"reason"`
}

ResponsesIncompleteDetails carries the reason a response stopped early, following the OpenAI Responses contract ("max_output_tokens", "content_filter", or "interrupted" for a cut upstream stream).

type ResponsesInputElement 

type ResponsesInputElement struct {
	Type string `json:"type,omitempty"` // "message", "function_call", "function_call_output"

	// Message fields (type="" or "message")
	Role    string `json:"role,omitempty"`
	Status  string `json:"status,omitempty"`
	Content any    `json:"content,omitempty"` // Can be string or []ContentPart
	//nolint:govet // Intentional duplicate json tag for Swagger docs: content is string OR []ContentPart.
	ContentSchema []ContentPart `` /* 146-byte string literal not displayed */

	// Function call fields (type="function_call")
	CallID    string `json:"call_id,omitempty"`
	Name      string `json:"name,omitempty"`
	Arguments string `json:"arguments,omitempty"`

	// Function call output fields (type="function_call_output") — CallID shared above
	Output      string            `json:"output,omitempty"`
	Raw         json.RawMessage   `json:"-" swaggerignore:"true"`
	ExtraFields UnknownJSONFields `json:"-" swaggerignore:"true"`
}

ResponsesInputElement represents a single item in the Responses API input array. It is a discriminated union keyed on Type:

  • "" or "message": a chat-style message with Role and Content
  • "function_call": a tool invocation with CallID, Name, and Arguments
  • "function_call_output": a tool result with CallID and Output

Unknown JSON members encountered during unmarshaling are preserved in ExtraFields (UnknownJSONFields) and marshaled back out unchanged so extensions can round-trip; Swagger ignores ExtraFields, and typed fields should be preferred when available.

func (ResponsesInputElement) MarshalJSON 

func (e ResponsesInputElement) MarshalJSON() ([]byte, error)

MarshalJSON serializes a ResponsesInputElement, emitting only the fields relevant to its Type variant.

func (*ResponsesInputElement) UnmarshalJSON 

func (e *ResponsesInputElement) UnmarshalJSON(data []byte) error

UnmarshalJSON deserializes a ResponsesInputElement, switching on the "type" field to populate variant-specific fields.

type ResponsesOutputItem 

type ResponsesOutputItem struct {
	ID        string                 `json:"id"`
	Type      string                 `json:"type"` // "message", "function_call", etc.
	Role      string                 `json:"role,omitempty"`
	Status    string                 `json:"status,omitempty"`
	CallID    string                 `json:"call_id,omitempty"`
	Name      string                 `json:"name,omitempty"`
	Arguments string                 `json:"arguments,omitempty"`
	Content   []ResponsesContentItem `json:"content,omitempty"`
	// Preserve fields belonging to newer or variant-specific output items, such
	// as reasoning.summary, reasoning.encrypted_content, hosted-tool payloads,
	// and provider extensions. Conversation replay depends on these fields.
	ExtraFields UnknownJSONFields `json:"-" swaggerignore:"true"`
}

ResponsesOutputItem represents an item in the output array.

func (ResponsesOutputItem) MarshalJSON added in v0.1.58 

func (i ResponsesOutputItem) MarshalJSON() ([]byte, error)

MarshalJSON emits typed output fields together with every unknown field retained during decoding.

func (*ResponsesOutputItem) UnmarshalJSON added in v0.1.58 

func (i *ResponsesOutputItem) UnmarshalJSON(data []byte) error

UnmarshalJSON preserves variant-specific Responses output item fields. This is required for lossless Responses passthrough and for replaying reasoning and hosted-tool items from a gateway-managed conversation.

type ResponsesRequest 

type ResponsesRequest struct {
	Model              string            `json:"model"`
	Provider           string            `json:"provider,omitempty"` // Gateway routing hint; stripped before upstream execution.
	Input              any               `json:"input"`              // string or []ResponsesInputElement — see docs for array form
	Instructions       string            `json:"instructions,omitempty"`
	Tools              []map[string]any  `json:"tools,omitempty"`
	ToolChoice         any               `json:"tool_choice,omitempty"` // string or object
	ParallelToolCalls  *bool             `json:"parallel_tool_calls,omitempty"`
	Temperature        *float64          `json:"temperature,omitempty"`
	TopP               *float64          `json:"top_p,omitempty"`
	TopLogprobs        *int              `json:"top_logprobs,omitempty"`
	MaxOutputTokens    *int              `json:"max_output_tokens,omitempty"`
	Stream             bool              `json:"stream,omitempty"`
	StreamOptions      *StreamOptions    `json:"stream_options,omitempty"`
	Metadata           map[string]string `json:"metadata,omitempty"`
	Reasoning          *Reasoning        `json:"reasoning,omitempty"`
	Text               any               `json:"text,omitempty"`
	Include            []string          `json:"include,omitempty"`
	Truncation         string            `json:"truncation,omitempty"`
	Store              *bool             `json:"store,omitempty"`
	PreviousResponseID string            `json:"previous_response_id,omitempty"`
	// Conversation accepts either a conversation ID string or an object with id.
	Conversation         *ResponsesConversationRef `json:"conversation,omitempty"`
	Prompt               any                       `json:"prompt,omitempty"`
	PromptCacheRetention string                    `json:"prompt_cache_retention,omitempty"`
	ContextManagement    any                       `json:"context_management,omitempty"`
	User                 string                    `json:"user,omitempty"`
	ServiceTier          string                    `json:"service_tier,omitempty"`
	SafetyIdentifier     string                    `json:"safety_identifier,omitempty"`
	ExtraFields          UnknownJSONFields         `json:"-" swaggerignore:"true"`
}

ResponsesRequest represents the request body for the Responses API. This is the OpenAI-compatible /v1/responses endpoint. Unknown JSON members encountered during unmarshaling are preserved in ExtraFields (UnknownJSONFields) and emitted again during marshaling so callers can round-trip extensions; Swagger ignores ExtraFields, and typed fields should be preferred when available.

func DecodeResponsesRequest 

func DecodeResponsesRequest(body []byte, env *WhiteBoxPrompt) (*ResponsesRequest, error)

DecodeResponsesRequest decodes and caches the canonical responses request for a semantic envelope.

func (*ResponsesRequest) CompactRequest 

func (r *ResponsesRequest) CompactRequest() *ResponseCompactRequest

CompactRequest reduces a full Responses request for the compact endpoint; see InputTokensRequest.

func (*ResponsesRequest) InputTokensRequest 

func (r *ResponsesRequest) InputTokensRequest() *ResponseInputTokensRequest

InputTokensRequest reduces a full Responses request to the field set shared with the utility endpoints (the streaming controls are dropped — utility endpoints never stream). ExtraFields are cloned. The reduction is defined here, next to the types, so the field knowledge stays in one file; a guard test asserts it covers every non-streaming ResponsesRequest field.

func (ResponsesRequest) MarshalJSON 

func (r ResponsesRequest) MarshalJSON() ([]byte, error)

MarshalJSON preserves dynamic input payloads while supporting Swagger-only schema fields. alias inherits every field and json tag from ResponsesRequest but drops the MarshalJSON method (so json.Marshal does not recurse); ExtraFields is json:"-" and merged in separately. New typed fields round-trip automatically.

func (*ResponsesRequest) UnmarshalJSON 

func (r *ResponsesRequest) UnmarshalJSON(data []byte) error

UnmarshalJSON preserves dynamic input payloads while supporting Swagger-only schema fields. Array inputs are deserialized as []ResponsesInputElement for type-safe downstream handling. The body decodes through an alias embedding so every typed field (present and future) is populated by the JSON package directly — only Input (a raw union) and ExtraFields need explicit handling.

func (*ResponsesRequest) WithStreaming 

func (r *ResponsesRequest) WithStreaming() *ResponsesRequest

WithStreaming returns a shallow copy of the request with Stream set to true. This avoids mutating the caller's request object.

type ResponsesResponse 

type ResponsesResponse struct {
	ID        string                `json:"id"`
	Object    string                `json:"object"` // "response"
	CreatedAt int64                 `json:"created_at"`
	Model     string                `json:"model"`
	Provider  string                `json:"provider"`
	Status    string                `json:"status"` // "completed", "incomplete", "failed", "in_progress"
	Output    []ResponsesOutputItem `json:"output"`
	Usage     *ResponsesUsage       `json:"usage,omitempty"`
	Error     *ResponsesError       `json:"error,omitempty"`
	// IncompleteDetails explains a status of "incomplete": the model hit
	// max_output_tokens, was stopped by a content filter, or the upstream
	// stream was interrupted.
	IncompleteDetails *ResponsesIncompleteDetails `json:"incomplete_details,omitempty"`
	// PreviousResponseID names the response this one was chained from, as
	// OpenAI echoes it; stored snapshots follow it to rebuild the history.
	PreviousResponseID string `json:"previous_response_id,omitempty"`
}

ResponsesResponse represents the response from the Responses API.

type ResponsesUsage 

type ResponsesUsage struct {
	InputTokens             int                      `json:"input_tokens"`
	OutputTokens            int                      `json:"output_tokens"`
	TotalTokens             int                      `json:"total_tokens"`
	PromptTokensDetails     *PromptTokensDetails     `json:"prompt_tokens_details,omitempty"`
	CompletionTokensDetails *CompletionTokensDetails `json:"completion_tokens_details,omitempty"`
	RawUsage                map[string]any           `json:"raw_usage,omitempty"`
}

ResponsesUsage represents token usage for the Responses API.

func (ResponsesUsage) MarshalJSON 

func (u ResponsesUsage) MarshalJSON() ([]byte, error)

MarshalJSON emits the OpenAI Responses usage shape and nothing else. Unlike Chat Completions, which passes provider usage extras through at the top level, the Responses API returns a closed object, so provider-specific members (thoughts_token_count, completion_reasoning_tokens, cache_creation_input_tokens, …) stay in RawUsage for usage records and cost calculation instead of leaking into the client response. Everything with an OpenAI-shaped home is kept: reasoning tokens under output_tokens_details, cached tokens under input_tokens_details.

func (*ResponsesUsage) UnmarshalJSON 

func (u *ResponsesUsage) UnmarshalJSON(data []byte) error

type RoutablePassthrough 

type RoutablePassthrough interface {
	Passthrough(ctx context.Context, providerType string, req *PassthroughRequest) (*PassthroughResponse, error)
}

RoutablePassthrough resolves a provider type before issuing an opaque passthrough request.

type RoutableProvider 

type RoutableProvider interface {
	Provider

	Supports(model string) bool
	GetProviderType(model string) string
}

RoutableProvider extends Provider with routing capability. This is implemented by the Router which uses a model registry to determine if a model is supported.

type RouteHints 

type RouteHints struct {
	Model    string
	Provider string
	Endpoint string
}

RouteHints holds minimal routing-relevant request hints derived from the transport snapshot.

These hints are intentionally smaller than a full semantic interpretation.

Lifecycle:

  • DeriveWhiteBoxPrompt seeds these values directly from transport/body data.
  • Canonical JSON decode may refine them from a cached request object.
  • Selector normalization (ParseModelSelector / RequestedModelSelector.Normalize) canonicalizes model/provider values in place.

Consumers that require canonical selector state should prefer a cached canonical request or normalize the selector before relying on these fields.

type StreamOptions 

type StreamOptions struct {
	// IncludeUsage requests token usage information in streaming responses.
	// When true, the final streaming chunk will include usage statistics.
	IncludeUsage bool `json:"include_usage,omitempty"`
}

StreamOptions controls streaming behavior options. This is used to request usage data in streaming responses.

type ToolCall 

type ToolCall struct {
	ID          string            `json:"id"`
	Type        string            `json:"type"`
	Function    FunctionCall      `json:"function"`
	ExtraFields UnknownJSONFields `json:"-" swaggerignore:"true"`
}

ToolCall represents a single tool invocation emitted by a model.

func (ToolCall) MarshalJSON 

func (t ToolCall) MarshalJSON() ([]byte, error)

ToolCall.MarshalJSON marshals a ToolCall to JSON, including unknown JSON members from ExtraFields. alias inherits ToolCall's fields and tags but drops MarshalJSON so json.Marshal does not recurse; ExtraFields (json:"-") is merged separately.

func (*ToolCall) UnmarshalJSON 

func (t *ToolCall) UnmarshalJSON(data []byte) error

ToolCall.UnmarshalJSON unmarshals a ToolCall from JSON, preserving unknown JSON members in ExtraFields.

type UnknownJSONFields 

type UnknownJSONFields struct {
	// contains filtered or unexported fields
}

UnknownJSONFields stores unknown JSON object members as a single raw object. This avoids allocating a map for every decoded chat-family request while still allowing lookups and round-trip preservation when needed.

func CloneUnknownJSONFields 

func CloneUnknownJSONFields(fields UnknownJSONFields) UnknownJSONFields

CloneUnknownJSONFields returns a detached copy of a raw unknown-field object.

func MergeUnknownJSONFields 

func MergeUnknownJSONFields(base UnknownJSONFields, additions map[string]json.RawMessage) (UnknownJSONFields, error)

MergeUnknownJSONFields returns base with the given raw members added; additions override existing members on key conflict. It lets translation layers inject derived fields (such as a chat response_format mapped from a Responses text format) into a request's passthrough object without a dedicated typed field.

func UnknownJSONFieldsFromMap 

func UnknownJSONFieldsFromMap(fields map[string]json.RawMessage) UnknownJSONFields

UnknownJSONFieldsFromMap converts a raw field map into a compact JSON object.

func (UnknownJSONFields) ExtraContent added in v0.1.90 

func (fields UnknownJSONFields) ExtraContent(vendor string) json.RawMessage

ExtraContent returns the vendor's object under extra_content, or nil when the member is absent, is not an object, or has no entry for the vendor.

func (UnknownJSONFields) HasAny added in v0.1.94 

func (fields UnknownJSONFields) HasAny(keys ...string) bool

HasAny reports whether any of the named members is present, in one pass over the stored object rather than one pass per key. Callers that only test for presence — the prompt-cache planner checks six directive keys on every message, content part and tool call — would otherwise rescan the same raw JSON once per key.

func (UnknownJSONFields) HasForeignExtraContent added in v0.1.90 

func (fields UnknownJSONFields) HasForeignExtraContent(keep string) bool

HasForeignExtraContent reports whether WithoutForeignExtraContent(keep) would remove anything.

func (UnknownJSONFields) IsEmpty 

func (fields UnknownJSONFields) IsEmpty() bool

IsEmpty reports whether the container has no stored fields.

func (UnknownJSONFields) Lookup 

func (fields UnknownJSONFields) Lookup(key string) json.RawMessage

Lookup returns the raw JSON value for key or nil when absent. It scans the stored object on demand so single lookups stay allocation-light, but repeated lookups on the same value are linear in the raw JSON size; reach for HasAny when several keys are only tested for presence.

Keys are matched literally. The scan deliberately does not use a gjson path, which would read "a.b" as a nested lookup and "x*" as a wildcard.

A member whose value the request decoder would reject is reported as absent, so callers never receive bytes they cannot hand back to a JSON decoder. A container can only hold such a value if it was built from an invalid raw map.

func (UnknownJSONFields) WithExtraContent added in v0.1.90 

func (fields UnknownJSONFields) WithExtraContent(vendor string, value json.RawMessage) (UnknownJSONFields, error)

WithExtraContent returns fields with extra_content.<vendor> set to value. Other vendors' objects and every other member are kept.

func (UnknownJSONFields) Without added in v0.1.67 

func (fields UnknownJSONFields) Without(keys ...string) UnknownJSONFields

Without returns the fields without the named members. Other members, including duplicate unknown keys, retain their original JSON representation.

func (UnknownJSONFields) WithoutForeignExtraContent added in v0.1.90 

func (fields UnknownJSONFields) WithoutForeignExtraContent(keep string) UnknownJSONFields

WithoutForeignExtraContent returns fields with every extra_content vendor other than keep removed. The member disappears when nothing remains, so a provider with no state of its own (keep == "") never sees it.

type Usage 

type Usage struct {
	PromptTokens            int                      `json:"prompt_tokens"`
	CompletionTokens        int                      `json:"completion_tokens"`
	TotalTokens             int                      `json:"total_tokens"`
	PromptTokensDetails     *PromptTokensDetails     `json:"prompt_tokens_details,omitempty"`
	CompletionTokensDetails *CompletionTokensDetails `json:"completion_tokens_details,omitempty"`
	RawUsage                map[string]any           `json:"raw_usage,omitempty"`
}

Usage represents token usage information

func (Usage) MarshalJSON 

func (u Usage) MarshalJSON() ([]byte, error)

func (*Usage) UnmarshalJSON 

func (u *Usage) UnmarshalJSON(data []byte) error

type VideoURLContent added in v0.1.93 

type VideoURLContent struct {
	URL         string            `json:"url"`
	Detail      string            `json:"detail,omitempty"`
	ExtraFields UnknownJSONFields `json:"-" swaggerignore:"true"`
}

VideoURLContent contains a video reference and optional processing settings. Provider-specific settings are preserved in ExtraFields.

func (VideoURLContent) MarshalJSON added in v0.1.93 

func (v VideoURLContent) MarshalJSON() ([]byte, error)

func (*VideoURLContent) UnmarshalJSON added in v0.1.93 

func (v *VideoURLContent) UnmarshalJSON(data []byte) error

type WhiteBoxPrompt 

type WhiteBoxPrompt struct {
	RouteType     string
	OperationType string
	RouteHints    RouteHints
	// StreamRequested reports that the inbound request explicitly asked for
	// streaming semantics. This is request intent, not endpoint capability.
	StreamRequested bool
	// JSONBodyParsed reports that the captured request body was parsed as JSON
	// (for selector peeking and/or canonical request decode).
	JSONBodyParsed bool
	// contains filtered or unexported fields
}

WhiteBoxPrompt is the gateway's best-effort semantic extraction from the transport snapshot. It may be partial and should not be treated as authoritative transport state.

The semantics are populated incrementally:

  • transport seeds RouteType/OperationType plus sparse RouteHints
  • route-specific metadata may be cached on demand
  • canonical request decode may cache a parsed request and refine RouteHints
  • selector normalization may rewrite selector hints into canonical form

func DeriveWhiteBoxPrompt 

func DeriveWhiteBoxPrompt(snapshot *RequestSnapshot) *WhiteBoxPrompt

DeriveWhiteBoxPrompt derives best-effort request semantics from the captured transport snapshot. Unknown or invalid bodies are tolerated; the returned envelope may be partial.

func GetWhiteBoxPrompt 

func GetWhiteBoxPrompt(ctx context.Context) *WhiteBoxPrompt

GetWhiteBoxPrompt retrieves the white-box prompt from the context.

func RefreshWhiteBoxPrompt added in v0.1.74 

func RefreshWhiteBoxPrompt(snapshot *RequestSnapshot, previous *WhiteBoxPrompt) *WhiteBoxPrompt

RefreshWhiteBoxPrompt rebuilds request semantics after a deferred body read while retaining passthrough metadata added by provider-owned enrichment. Body-derived model and stream intent from the refreshed snapshot remain authoritative when the complete body parses successfully.

func (*WhiteBoxPrompt) CachedBatchRequest 

func (env *WhiteBoxPrompt) CachedBatchRequest() *BatchRequest

CachedBatchRequest returns the cached canonical batch create request, if present.

func (*WhiteBoxPrompt) CachedBatchRouteInfo 

func (env *WhiteBoxPrompt) CachedBatchRouteInfo() *BatchRouteInfo

CachedBatchRouteInfo returns cached sparse batch route info, if present.

func (*WhiteBoxPrompt) CachedChatRequest 

func (env *WhiteBoxPrompt) CachedChatRequest() *ChatRequest

CachedChatRequest returns the cached canonical chat request, if present.

func (*WhiteBoxPrompt) CachedEmbeddingRequest 

func (env *WhiteBoxPrompt) CachedEmbeddingRequest() *EmbeddingRequest

CachedEmbeddingRequest returns the cached canonical embeddings request, if present.

func (*WhiteBoxPrompt) CachedFileRouteInfo 

func (env *WhiteBoxPrompt) CachedFileRouteInfo() *FileRouteInfo

CachedFileRouteInfo returns cached sparse file route info, if present.

func (*WhiteBoxPrompt) CachedPassthroughRouteInfo 

func (env *WhiteBoxPrompt) CachedPassthroughRouteInfo() *PassthroughRouteInfo

CachedPassthroughRouteInfo returns cached typed passthrough route info, if present.

func (*WhiteBoxPrompt) CachedResponsesRequest 

func (env *WhiteBoxPrompt) CachedResponsesRequest() *ResponsesRequest

CachedResponsesRequest returns the cached canonical responses request, if present.

func (*WhiteBoxPrompt) CanonicalSelectorFromCachedRequest 

func (env *WhiteBoxPrompt) CanonicalSelectorFromCachedRequest() (model, provider string, ok bool)

CanonicalSelectorFromCachedRequest returns model/provider selector hints from any cached canonical JSON request for the current operation kind.

type Workflow 

type Workflow struct {
	RequestID    string
	Endpoint     EndpointDescriptor
	Mode         ExecutionMode
	Capabilities CapabilitySet
	ProviderType string
	Passthrough  *PassthroughRouteInfo
	Resolution   *RequestModelResolution
	Policy       *ResolvedWorkflowPolicy
	// PluginState is the per-request plugin state, created by the plugin
	// runtime the first time a plugin runs for the request and nil otherwise.
	// It lives here rather than in the context so a request that runs no
	// plugin allocates nothing.
	PluginState any
}

Workflow is the request-scoped control-plane result consumed by later execution stages. It carries the resolved execution mode, endpoint capabilities, and any model routing decision already made for the request.

func GetWorkflow 

func GetWorkflow(ctx context.Context) *Workflow

GetWorkflow retrieves the workflow from the context.

func (*Workflow) AuditEnabled 

func (p *Workflow) AuditEnabled() bool

AuditEnabled reports whether audit logging is enabled for the request.

func (*Workflow) BudgetEnabled 

func (p *Workflow) BudgetEnabled() bool

BudgetEnabled reports whether budget checks are enabled for the request.

func (*Workflow) CacheEnabled 

func (p *Workflow) CacheEnabled() bool

CacheEnabled reports whether response caching is enabled for the request.

func (*Workflow) FailoverEnabled 

func (p *Workflow) FailoverEnabled() bool

FailoverEnabled reports whether translated-route failover is enabled for the request.

func (*Workflow) GuardrailsEnabled 

func (p *Workflow) GuardrailsEnabled() bool

GuardrailsEnabled reports whether guardrail processing is enabled for the request.

func (*Workflow) GuardrailsHash 

func (p *Workflow) GuardrailsHash() string

GuardrailsHash returns the matched workflow's guardrails hash.

func (*Workflow) RequestedQualifiedModel 

func (p *Workflow) RequestedQualifiedModel() string

RequestedQualifiedModel returns the requested model selector when present.

func (*Workflow) ResolvedQualifiedModel 

func (p *Workflow) ResolvedQualifiedModel() string

ResolvedQualifiedModel returns the resolved model selector when present.

func (*Workflow) UsageEnabled 

func (p *Workflow) UsageEnabled() bool

UsageEnabled reports whether usage tracking is enabled for the request.

func (*Workflow) WorkflowVersionID 

func (p *Workflow) WorkflowVersionID() string

WorkflowVersionID returns the matched immutable workflow version id.

type WorkflowFeatures 

type WorkflowFeatures struct {
	Cache      bool `json:"cache"`
	Audit      bool `json:"audit"`
	Usage      bool `json:"usage"`
	Budget     bool `json:"budget"`
	Guardrails bool `json:"guardrails"`
	Failover   bool `json:"failover"`
}

WorkflowFeatures stores resolved per-request feature flags sourced from the matched persisted workflow.

func DefaultWorkflowFeatures 

func DefaultWorkflowFeatures() WorkflowFeatures

DefaultWorkflowFeatures returns the permissive runtime default used when no persisted workflow has been attached to the request.

func (WorkflowFeatures) ApplyUpperBound 

func (f WorkflowFeatures) ApplyUpperBound(caps WorkflowFeatures) WorkflowFeatures

ApplyUpperBound returns features with process-level caps applied.

type WorkflowSelector 

type WorkflowSelector struct {
	// Provider is the configured provider instance name used for workflow matching.
	Provider string
	Model    string
	UserPath string
}

WorkflowSelector contains the request facts used to match one persisted workflow version.

func NewWorkflowSelector 

func NewWorkflowSelector(provider, model string, userPath ...string) WorkflowSelector

NewWorkflowSelector trims selector inputs for deterministic matching.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.