protocolserver

package
v0.260806.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 6, 2026 License: MPL-2.0 Imports: 55 Imported by: 0

Documentation

Overview

Package aimodel hosts the AI Model API surface split out of internal/server: the LLM gateway routes (/tingly/:scenario/...) covering OpenAI/Anthropic compatible chat/completions/responses/messages/embeddings/images, MCP tool handling during model requests, protocol dispatch/transform/passthrough, failover/load-balance dispatch, guardrails runtime evaluation, and usage/token tracking.

This package intentionally does not depend on *server.Server. Handlers are built from the narrow ProtocolHandlerDeps struct below, following the same pattern already used by internal/server/module/* (see module/scenario/handler.go).

Index

Constants

View Source 
const (
	ContextKeyRule           = constant.CtxKeyRule
	ContextKeyProvider       = constant.CtxKeyProvider
	ContextKeyModel          = constant.CtxKeyModel
	ContextKeyRequestModel   = constant.CtxKeyRequestModel
	ContextKeyScenario       = constant.CtxKeyScenario
	ContextKeyStreamed       = constant.CtxKeyStreamed
	ContextKeyStartTime      = constant.CtxKeyStartTime
	ContextKeyFirstTokenTime = constant.CtxKeyFirstTokenTime
	ContextKeyCacheHit       = constant.CtxKeyCacheHit
	ContextKeySessionID      = constant.CtxKeySessionID
	ContextKeyAffinityKey    = constant.CtxKeyAffinityKey
	ContextKeyLBServiceID    = constant.CtxKeyLBServiceID
	ContextKeyLBTactic       = constant.CtxKeyLBTactic
)

Gin context key aliases — canonical values live in constant so that routing/ and middleware/ sub-packages can reference them without an import cycle.

View Source 
const ProbeSyntheticRuleUUID = "probe-synthetic"

ProbeSyntheticRuleUUID marks the throwaway rule built for an X-Tingly-Probe-Service request — it has no persisted identity. Owned here (moved from root's handlers.go constant) since protocol_dispatch.go's setProbeUpstreamHeaders is the sole consumer that has moved so far; root's handlers.go (not yet moved) keeps a companion alias.

Variables

View Source 
var GuardrailsSupportedScenarios = []string{
	string(typ.ScenarioAnthropic),
	string(typ.ScenarioClaudeCode),
}

GuardrailsSupportedScenarios lists the scenarios guardrails can gate.

Functions

func ApplyGuardrailsToAnthropicV1BetaNonStreamResponse 

func ApplyGuardrailsToAnthropicV1BetaNonStreamResponse(c *gin.Context, runtime *guardrails.Guardrails, req *anthropic.BetaMessageNewParams, actualModel string, provider *typ.Provider, resp *anthropic.BetaMessage) bool

ApplyGuardrailsToAnthropicV1BetaNonStreamResponse is the beta equivalent of ApplyGuardrailsToAnthropicV1NonStreamResponse.

func ApplyGuardrailsToAnthropicV1BetaRequest 

func ApplyGuardrailsToAnthropicV1BetaRequest(c *gin.Context, runtime *guardrails.Guardrails, req *anthropic.BetaMessageNewParams, actualModel string, provider *typ.Provider)

ApplyGuardrailsToAnthropicV1BetaRequest is the merged request-side entry for Anthropic beta requests. It runs request tool_result filtering first and then request credential masking on the latest raw request state.

func ApplyGuardrailsToAnthropicV1NonStreamResponse 

func ApplyGuardrailsToAnthropicV1NonStreamResponse(c *gin.Context, runtime *guardrails.Guardrails, req *anthropic.MessageNewParams, actualModel string, provider *typ.Provider, resp *anthropic.Message) bool

ApplyGuardrailsToAnthropicV1NonStreamResponse evaluates a fully assembled Anthropic v1 response and rewrites it when guardrails block it.

func ApplyGuardrailsToAnthropicV1Request 

func ApplyGuardrailsToAnthropicV1Request(c *gin.Context, runtime *guardrails.Guardrails, req *anthropic.MessageNewParams, actualModel string, provider *typ.Provider)

ApplyGuardrailsToAnthropicV1Request is the merged request-side entry for Anthropic v1 requests. It runs request tool_result filtering first and then request credential masking on the latest raw request state.

func AttachGuardrailsHooks 

func AttachGuardrailsHooks(c *gin.Context, runtime *guardrails.Guardrails, hc *protocol.HandleContext, actualModel string, provider *typ.Provider, messages []guardrailscore.Message)

AttachGuardrailsHooks wires the shared stream guardrails runtime into a protocol handle context. Provider-specific handlers only need to provide already-normalized message history.

func AuthTypeSortWeight 

func AuthTypeSortWeight(a typ.AuthType) int

AuthTypeSortWeight ranks auth types for /v1/models ordering: oauth (0) -> api_key (1) -> vmodel (2). Any unknown value is treated as api_key so legacy entries cluster with regular providers.

func BuildGuardrailsBaseInput 

func BuildGuardrailsBaseInput(c *gin.Context, actualModel string, provider *typ.Provider, direction guardrailscore.Direction, messages []guardrailscore.Message) guardrailscore.Input

BuildGuardrailsBaseInput creates the shared evaluation envelope; adapters can then add request/response-specific content without rebuilding metadata each time.

func BuildOpenAIContinuationSegment 

func BuildOpenAIContinuationSegment(
	calls []stream.OpenAIToAnthropicToolCall,
	virtualResults []mcp.ToolExecutionResult,
) []openai.ChatCompletionMessageParamUnion

BuildOpenAIContinuationSegment builds the assistant tool_calls + tool result messages appended to the OpenAI message history after a round of virtual MCP tool execution.

func CalculateLatencyFromStart 

func CalculateLatencyFromStart(startTime time.Time) int

CalculateLatencyFromStart calculates the elapsed time in milliseconds since the start time.

func CalculateTPS 

func CalculateTPS(c *gin.Context, outputTokens int, streamed bool) float64

CalculateTPS calculates Tokens Per Second for streaming requests. For non-streaming requests or when TTFT is not available, returns 0.

TPS is the inverse of time per output token (TPOT): (outputTokens - 1) / (currentTime - firstTokenTime).

Parameters:

  • c: Gin context containing timing information
  • outputTokens: Number of output tokens generated
  • streamed: Whether this was a streaming request

Returns:

  • TPS value (tokens per second), or 0 if not applicable

func CalculateTTFT 

func CalculateTTFT(c *gin.Context) int64

CalculateTTFT returns Time To First Token in milliseconds, or 0 when no first-token time was recorded (e.g. non-streaming requests). It must not fall back to total latency, which would make TTFT indistinguishable from it.

func CloneAnthropicBetaRequest 

func CloneAnthropicBetaRequest(template []byte) (*protocol.AnthropicBetaMessagesRequest, error)

CloneAnthropicBetaRequest rebuilds an Anthropic beta request from a marshalled template (produced by AnthropicBetaMessagesRequest.MarshalJSON).

func CloneAnthropicV1Request 

func CloneAnthropicV1Request(template []byte) (*protocol.AnthropicMessagesRequest, error)

CloneAnthropicV1Request rebuilds an Anthropic v1 request from a marshalled template (produced by AnthropicMessagesRequest.MarshalJSON).

func CloneOpenAIChatRequest 

func CloneOpenAIChatRequest(template []byte) (*protocol.OpenAIChatCompletionRequest, error)

CloneOpenAIChatRequest rebuilds an OpenAI chat request from a marshalled template (produced by OpenAIChatCompletionRequest.MarshalJSON).

func CloneResponsesParams 

func CloneResponsesParams(template *responses.ResponseNewParams) (*responses.ResponseNewParams, error)

CloneResponsesParams clones the typed responses.ResponseNewParams. It marshals and unmarshals the SDK struct directly — NOT through ResponseCreateRequest, whose UnmarshalJSON re-runs PreprocessInputData (input-item type injection) that has already been applied to the template.

func CommitFirstChunkIfGate 

func CommitFirstChunkIfGate(w gin.ResponseWriter) bool

CommitFirstChunkIfGate signals "first real chunk arrived" if w is currently a *firstChunkGate (installed by DispatchWithPriorityFailover when a rule has more than one active service), no-op otherwise. Exported so callers outside this package that simulate a streaming success (e.g. the load-balance dry-run simulator) can commit the gate without needing firstChunkGate itself exported. Returns whether a gate was found and committed.

func ConvertAnthropicToOpenAIResponseWithProvider 

func ConvertAnthropicToOpenAIResponseWithProvider(anthropicResp *anthropic.BetaMessage, responseModel string, provider *typ.Provider, model string) map[string]interface{}

ConvertAnthropicToOpenAIResponseWithProvider converts an Anthropic response to OpenAI format and applies provider-specific transformations to the response

func DetectCacheHit 

func DetectCacheHit(usage *protocol.TokenUsage) bool

DetectCacheHit determines if a request was served from cache based on TokenUsage. Returns true if cache was hit, false otherwise.

Detection logic:

  • OpenAI: usage.CacheReadTokens > 0 (cache_read_input_tokens field)
  • Anthropic: usage.CacheReadTokens > 0 (cache_read_input_tokens field)
  • Other providers: Returns false (conservative - assumes cache miss)

Parameters:

  • usage: Token usage information from the response

Returns:

  • true if cache hit detected, false otherwise

func EnsureGuardrailsCredentialMaskState 

func EnsureGuardrailsCredentialMaskState(c *gin.Context) *guardrailscore.CredentialMaskState

func ExecuteAnthropicPreChain 

func ExecuteAnthropicPreChain[T *anthropic.MessageNewParams | *anthropic.BetaMessageNewParams](
	req T,
	scenarioConfig *typ.ScenarioConfig,
	defaultMaxTokens, maxAllowed int,
	isStreaming bool,
) error

ExecuteAnthropicPreChain builds and runs the server-side pre-transform chain for Anthropic requests (req is *anthropic.MessageNewParams or *anthropic.BetaMessageNewParams — the transforms type-switch internally). Currently only MaxTokens validation remains at scenario level; other scenario-level transforms (ThinkingEffort, CleanHeader) are handled via rule flags injection in resolveRuleFlagsWithScenario. Returns an error that should be mapped to HTTP 400.

func ExtractAnthropicBetaMessages 

func ExtractAnthropicBetaMessages(messages []anthropic.BetaMessageParam) []map[string]any

ExtractAnthropicBetaMessages JSON-roundtrips an Anthropic beta message list into the generic map shape MCP tool hooks pass through as message history.

func ExtractOpenAIMessages 

func ExtractOpenAIMessages(messages []openai.ChatCompletionMessageParamUnion) []map[string]any

ExtractOpenAIMessages JSON-roundtrips an OpenAI message list into the generic map shape MCP tool hooks pass through as message history.

func ExtractScenarioFromPath 

func ExtractScenarioFromPath(path string) string

ExtractScenarioFromPath extracts the scenario segment from a request path, e.g. "/tingly/claude_code/v1/messages" -> "claude_code".

func GetCacheHit 

func GetCacheHit(c *gin.Context) (bool, bool)

GetCacheHit retrieves the cache hit status from the context. Returns the cache hit status and true if it exists, false and false otherwise.

func GetFirstTokenTime 

func GetFirstTokenTime(c *gin.Context) (time.Time, bool)

GetFirstTokenTime retrieves the first token time from the context. Returns the timestamp and true if it exists, zero time and false otherwise.

func GetTrackingContext 

func GetTrackingContext(c *gin.Context) (rule *typ.Rule, provider *typ.Provider, actualModel, requestModel, scenario string, streamed bool, startTime time.Time)

GetTrackingContext retrieves tracking metadata from the gin context. Returns zero values if the context keys are not set.

func GetTrackingContextScenario 

func GetTrackingContextScenario(c *gin.Context) (scenario string)

func GetUserIDFromContext 

func GetUserIDFromContext(c *gin.Context) string

GetUserIDFromContext extracts the user ID from gin context. Priority: user_id (from JWT API token or global auth) > enterprise_user_id (from enterprise JWT) > "" This ensures that JWT API token authentication takes precedence for user tracking.

func GuardrailsEnabledForScenario 

func GuardrailsEnabledForScenario(cfg *config.Config, runtime *guardrails.Guardrails, scenario string) bool

GuardrailsEnabledForScenario centralizes feature-flag checks so protocol handlers do not repeat scenario/global guardrails gating logic.

func GuardrailsSupportsScenario 

func GuardrailsSupportsScenario(scenario string) bool

GuardrailsSupportsScenario reports whether scenario is one guardrails can gate.

func HasDeclaredMCPAnthropicBetaTools 

func HasDeclaredMCPAnthropicBetaTools(req *anthropic.BetaMessageNewParams) bool

HasDeclaredMCPAnthropicBetaTools reports whether req declares any MCP-named tool in its Anthropic beta tool list.

func HasDeclaredMCPAnthropicV1Tools 

func HasDeclaredMCPAnthropicV1Tools(req *anthropic.MessageNewParams) bool

HasDeclaredMCPAnthropicV1Tools reports whether req declares any MCP-named tool in its Anthropic v1 tool list. Moved here from root's protocol_dispatch.go (which still hosts its call sites, Step 8) because it is a pure function with zero *Server dependency, same class as its Beta sibling below.

func HasDeclaredMCPTools 

func HasDeclaredMCPTools(req *openai.ChatCompletionNewParams) bool

HasDeclaredMCPTools reports whether req declares any MCP-named tool in its OpenAI Chat tool list.

func HasNativeAdvisorBeta 

func HasNativeAdvisorBeta(req *protocol.AnthropicBetaMessagesRequest) bool

func IsValidRuleScenario 

func IsValidRuleScenario(scenario typ.RuleScenario) bool

IsValidRuleScenario checks if the given scenario is a valid RuleScenario

func MCPEnabled 

func MCPEnabled(cfg *config.Config) bool

MCPEnabled centralizes the MCP feature-flag check so gateway handlers do not repeat scenario-flag lookups. Mirrors GuardrailsEnabledForScenario.

func PrimaryAuthTypeForRule 

func PrimaryAuthTypeForRule(cfg *config.Config, rule typ.Rule) typ.AuthType

PrimaryAuthTypeForRule returns the AuthType of the first active service's provider in a rule. It is used by /v1/models endpoints so the frontend can order picker entries oauth -> api_key -> vmodel.

Returns AuthTypeAPIKey as the fallback for empty/unresolvable rules so they land in the middle group rather than at the head or tail.

func ReattachGuardrailsHooks 

func ReattachGuardrailsHooks(
	c *gin.Context,
	runtime *guardrails.Guardrails,
	hc *protocol.HandleContext,
	actualModel string,
	provider *typ.Provider,
	messages []guardrailscore.Message,
	baseEventHooks int,
	baseErrorHooks int,
)

ReattachGuardrailsHooks resets per-round guardrails state and re-registers fresh hooks on hc for the next MCP loop round. It truncates OnStreamEventHooks back to baseEventHooks (the count before guardrails was first attached) so previous-round guardrails hooks don't accumulate.

func ResolveOpenAIEndpoint 

func ResolveOpenAIEndpoint(provider *typ.Provider, flags typ.RuleFlags, incoming IncomingAPIType) (protocol.APIType, error)

ResolveOpenAIEndpoint picks an OpenAI endpoint using the optional per-rule override first, then the provider's declared OpenAIEndpointMode.

Precedence:

  1. Rule flag (flags.OpenAIEndpointOverride). Overrides provider settings.
  2. provider.OpenAIEndpointMode: EndpointModeUnknown / zero value → Chat EndpointModeChat → Chat EndpointModeResponses → Responses EndpointModeBoth → mirror incoming

Rule override is honored unconditionally (per design intent). When an override conflicts with the provider's declared mode, a warning is logged but the override takes effect. This allows explicit routing control for debugging and special cases.

Defaulting unknown providers to Chat (not "mirror incoming") is intentional: most OpenAI-compatible vendors implement only /chat/completions. Providers that genuinely support Responses must declare it via template or OAuth.

When an incoming Responses request routes to Chat, Responses-only fields (previous_response_id, include, background, truncation, reasoning) are silently dropped by ConvertOpenAIResponsesToChat — the same posture as Anthropic→Chat downgrades. The user accepts this by declaring the mode.

Pure function: no Server state, no probe lookups, no I/O.

func ResolveRuleFlags 

func ResolveRuleFlags(c *gin.Context, rule *typ.Rule) typ.RuleFlags

ResolveRuleFlags returns the effective flags for this request: a copy of the rule's persisted flags, with: - cursor_compat_auto folded into cursor_compat when the inbound request carries Cursor headers Returns the zero value when no rule is bound.

All flag folding/injection happens here (not at each handler call site) so that RulePreBaseTransforms and downstream consumers read the same merged value.

func ResolveRuleFlagsWithScenario 

func ResolveRuleFlagsWithScenario(
	c *gin.Context,
	rule *typ.Rule,
	scenarioType typ.RuleScenario,
	scenarioConfig *typ.ScenarioConfig,
	sourceAPI, targetAPI protocol.APIType,
	provider *typ.Provider,
) typ.RuleFlags

ResolveRuleFlagsWithScenario extends resolveRuleFlags to also inject scenario-level flags and auto-apply CleanHeader for protocol transformation scenarios.

This is the main entry point that merges:

  1. Rule-level flags (from the rule definition)
  2. Scenario flags (from the scenario configuration)
  3. Auto-applied flags (like CleanHeader for protocol transformation)
  4. Provider-driven suppressions (CleanHeader is cleared for Claude OAuth providers; the billing header must reach Anthropic's billing backend unchanged).

Side effect: it also attaches the resolved CustomUserAgent to the request context (applyCustomUserAgent) so callers don't have to repeat that at each handler. The User-Agent is the one rule flag that has to reach a deep component (the outbound transport) via ctx, so this central merge point is where it gets applied.

func RoundtripAnthropicBetaResponseViaOpenAI 

func RoundtripAnthropicBetaResponseViaOpenAI(anthropicResp *anthropic.BetaMessage, responseModel string, provider *typ.Provider, actualModel string) (*anthropic.BetaMessage, error)

func RoundtripOpenAIMapViaAnthropic 

func RoundtripOpenAIMapViaAnthropic(openaiResp map[string]interface{}, responseModel string, provider *typ.Provider, actualModel string) (map[string]interface{}, error)

func RoundtripOpenAIResponseViaAnthropic 

func RoundtripOpenAIResponseViaAnthropic(openaiResp *openai.ChatCompletion, responseModel string, provider *typ.Provider, actualModel string) (map[string]interface{}, error)

func RulePreBaseTransforms 

func RulePreBaseTransforms(flags typ.RuleFlags) []transform.Transform

RulePreBaseTransforms builds the per-rule list of pre-Base transforms for the chain's preBase slot. Pre-Base transforms act on the *inbound* request shape — they run before BaseTransform's protocol conversion, so the type-switch inside each transform sees what the client actually sent.

Returns nil when no rule-level flag requires a pre-Base stage so callers can pass the result straight to BuildTransformChain's preBase parameter.

func RulePreVendorTransforms 

func RulePreVendorTransforms(flags typ.RuleFlags) []transform.Transform

RulePreVendorTransforms builds the per-rule list of pre-Vendor transforms for the chain's preVendor slot (after Consistency, before Vendor). These act on the *target* request shape — they run after BaseTransform's protocol conversion, so the type-switch inside each transform matches the upstream-bound form, but still before Vendor finalizes the request.

Returns nil when no rule-level flag requires a chain stage so callers can pass the result straight to a `preVendorTransforms []transform.Transform` parameter.

Takes already-resolved flags so callers that need other fields off RuleFlags (CustomUserAgent, SkipUsage) can resolve once and share.

func SendErrorResponse 

func SendErrorResponse(c *gin.Context, err error, desc string)

SendErrorResponse registers the error into gin context for logging middleware and sends JSON response.

func SetCacheHit 

func SetCacheHit(c *gin.Context, isHit bool)

SetCacheHit records whether this request was a cache hit.

func SetEnterpriseRateLimitReporter 

func SetEnterpriseRateLimitReporter(reporter func(context.Context, string, string, string, string) error)

SetEnterpriseRateLimitReporter sets callback for enterprise 429 events.

func SetTrackingContext 

func SetTrackingContext(c *gin.Context, rule *typ.Rule, provider *typ.Provider, actualModel, requestModel string, streamed bool)

SetTrackingContext sets all tracking metadata in the gin context. This should be called once at the beginning of request processing to avoid explicit parameter passing throughout the handler chain.

Parameters:

  • c: Gin context
  • rule: The load balancer rule that was selected
  • provider: The provider that was selected
  • actualModel: The actual model name used (may differ from requested)
  • requestModel: The original model name requested by the user
  • streamed: Whether this is a streaming request

func ShouldIncludeRuleInModelList 

func ShouldIncludeRuleInModelList(requestedScenario typ.RuleScenario, ruleScenario typ.RuleScenario) bool

ShouldIncludeRuleInModelList reports whether a rule should appear in the model list for the requested scenario. Each scenario — base or profiled — is an isolated scope: it only lists rules bound to that exact scenario. Transport compatibility does not grant cross-scenario visibility.

func ShouldRoundtripResponse 

func ShouldRoundtripResponse(c *gin.Context, target string) bool

func ShouldStripUsage 

func ShouldStripUsage(extra map[string]interface{}) bool

ShouldStripUsage merges the cursor_compat and skip_usage hints carried in reqCtx.Extra. The dispatch layer ORs both together so a rule that only flips skip_usage still strips the usage block, and cursor_compat keeps its historical behavior of suppressing usage as a side effect.

Extracted so the wiring is unit-testable independent of the surrounding transform/forward machinery.

func ShouldUseGenericMCPForProvider 

func ShouldUseGenericMCPForProvider(cfg *config.Config, provider *typ.Provider) bool

ShouldUseGenericMCPForProvider is the pure-Config form of Handler.shouldUseGenericMCPForProvider, exported so callers that only have a *config.Config (e.g. tests constructing a bare *Server without a wired aiHandler) can check the same provider-limits logic directly.

func UpdateTrackingForFailover 

func UpdateTrackingForFailover(c *gin.Context, provider *typ.Provider, model string)

UpdateTrackingForFailover updates provider and model tracking during failover retry. This ensures that logging/middleware shows the final successful service after failover, not the initially-selected failed service.

Parameters:

  • c: Gin context
  • provider: The new provider being tried in this failover attempt
  • model: The new model being tried in this failover attempt

Types

type AffinityStore 

type AffinityStore struct {
	// contains filtered or unexported fields
}

AffinityStore maps (ruleUUID, sessionID) -> locked Service. It directly implements routing.AffinityStore interface.

func NewAffinityStore 

func NewAffinityStore(ttl time.Duration) *AffinityStore

NewAffinityStore creates a new affinity store with the given TTL

func (*AffinityStore) CountByService 

func (s *AffinityStore) CountByService(serviceID string) int

CountByService returns the number of sessions locked to the given serviceID that were created within the capacity active window (last 30 minutes). This provides a "recently active users" count for seat utilization calculation.

func (*AffinityStore) Delete 

func (s *AffinityStore) Delete(ruleUUID, sessionID string)

Delete removes an affinity entry for the given rule and session

func (*AffinityStore) DeleteByRule 

func (s *AffinityStore) DeleteByRule(ruleUUID string)

DeleteByRule removes all affinity entries for a given rule UUID

func (*AffinityStore) GC 

func (s *AffinityStore) GC()

GC removes expired entries from the store

func (*AffinityStore) Get 

func (s *AffinityStore) Get(ruleUUID, sessionID string) (*routing.AffinityEntry, bool)

Get retrieves an affinity entry for the given rule and session

func (*AffinityStore) Set 

func (s *AffinityStore) Set(ruleUUID, sessionID string, entry *routing.AffinityEntry)

Set stores an affinity entry for the given rule and session

func (*AffinityStore) StartGC 

func (s *AffinityStore) StartGC()

StartGC starts a background goroutine that periodically cleans up expired entries

func (*AffinityStore) UpdateMessageID 

func (s *AffinityStore) UpdateMessageID(ruleUUID, sessionID, messageID string)

UpdateMessageID updates the message ID for an existing affinity entry

type AnthropicModel 

type AnthropicModel struct {
	ID             string             `json:"id"`
	CreatedAt      string             `json:"created_at"`
	DisplayName    string             `json:"display_name"`
	Type           string             `json:"type"`
	Capabilities   *ModelCapabilities `json:"capabilities,omitempty"`
	MaxInputTokens int                `json:"max_input_tokens,omitempty"`
	MaxTokens      int                `json:"max_tokens,omitempty"`
	// Description is a tingly-box extension (not in Anthropic's wire format)
	// consumed by the frontend to show model description in the model picker.
	Description string `json:"description,omitempty"`
	// AuthType is a tingly-box extension (not in Anthropic's wire format)
	// consumed by the frontend to order model picker entries:
	// oauth -> api_key -> vmodel.
	AuthType string `json:"auth_type,omitempty"`
}

AnthropicModel maps to Anthropic's native /v1/models response format.

type AnthropicModelsResponse 

type AnthropicModelsResponse struct {
	Data    []AnthropicModel `json:"data"`
	FirstID string           `json:"first_id"`
	HasMore bool             `json:"has_more"`
	LastID  string           `json:"last_id"`
}

type CapabilitySupport 

type CapabilitySupport struct {
	Supported bool `json:"supported"`
}

CapabilitySupport indicates whether a capability is supported.

type ContextManagementCapability 

type ContextManagementCapability struct {
	Supported             bool              `json:"supported"`
	ClearThinking20251015 CapabilitySupport `json:"clear_thinking_20251015,omitempty"`
	ClearToolUses20250919 CapabilitySupport `json:"clear_tool_uses_20250919,omitempty"`
	Compact20260112       CapabilitySupport `json:"compact_20260112,omitempty"`
}

ContextManagementCapability describes context management support.

type EffortCapability 

type EffortCapability struct {
	Supported bool              `json:"supported"`
	Low       CapabilitySupport `json:"low,omitempty"`
	Medium    CapabilitySupport `json:"medium,omitempty"`
	High      CapabilitySupport `json:"high,omitempty"`
	XHigh     CapabilitySupport `json:"xhigh,omitempty"`
	Max       CapabilitySupport `json:"max,omitempty"`
}

EffortCapability describes reasoning_effort support and levels.

type EndpointOverride 

type EndpointOverride string

EndpointOverride is the typed value of the openai_endpoint_override rule flag. It forces an OpenAI request onto a specific endpoint, overriding the provider's declared OpenAIEndpointMode default (provider declarations trump conflicting overrides — see ResolveOpenAIEndpoint). 

const (
	OverrideAuto      EndpointOverride = "auto"
	OverrideChat      EndpointOverride = "chat"
	OverrideResponses EndpointOverride = "responses"
)

func ParseEndpointOverride 

func ParseEndpointOverride(s string) EndpointOverride

ParseEndpointOverride coerces a raw rule-flag string to a known EndpointOverride. Empty, "auto" and any unrecognized value map to OverrideAuto so misconfigured rules degrade safely.

type ErrorDetail 

type ErrorDetail struct {
	Message string `json:"message"`
	Type    string `json:"type"`
	Code    string `json:"code,omitempty"`
}

ErrorDetail represents error details

type ErrorResponse 

type ErrorResponse struct {
	Error ErrorDetail `json:"error"`
}

ErrorResponse represents an error response

type GuardrailsState 

type GuardrailsState struct {
	// contains filtered or unexported fields
}

GuardrailsState is the single source of truth for the guardrails runtime pointer: the mutex-guarded swap/read primitives, and the lifecycle operations (activation refresh, credential cache refresh) driven by config edits, hot-reload, and server construction. It is owned by protocolserver (the gateway evaluates guardrails on live requests); the host server's admin handlers reach it through the narrow GuardrailsRuntime interface implemented by *Server via thin forwarding methods.

The gateway-facing half — building the evaluation envelope and applying guardrails during a live model request — lives in guardrails_runtime_ai.go. Those functions take the current runtime snapshot as an explicit parameter (via Current() below) rather than reaching into shared state directly.

func NewGuardrailsState 

func NewGuardrailsState(cfg *config.Config) *GuardrailsState

NewGuardrailsState builds the guardrails runtime holder for cfg. The runtime pointer starts nil; call Set (or the host's init path) to activate.

func (*GuardrailsState) Current 

func (g *GuardrailsState) Current() *guardrails.Guardrails

Current returns the active runtime snapshot (nil when guardrails are off).

func (*GuardrailsState) EnabledForScenario 

func (g *GuardrailsState) EnabledForScenario(scenario string) bool

EnabledForScenario centralizes feature-flag checks so protocol handlers do not repeat scenario/global guardrails gating logic.

func (*GuardrailsState) RefreshCredentialCache 

func (g *GuardrailsState) RefreshCredentialCache() error

Credential cache and activation state live alongside the runtime gate because they are shared by request masking, history rendering, and runtime reloads.

func (*GuardrailsState) RefreshCredentialCacheOrWarn 

func (g *GuardrailsState) RefreshCredentialCacheOrWarn(context string)

func (*GuardrailsState) Set 

func (g *GuardrailsState) Set(runtime *guardrails.Guardrails, context string)

Set swaps in a new runtime, carrying over the previous history store and credential cache when the incoming runtime lacks them, then refreshes activation and credential state. context labels warning logs.

func (*GuardrailsState) SetRef 

func (g *GuardrailsState) SetRef(runtime *guardrails.Guardrails)

func (*GuardrailsState) SupportedScenarios 

func (g *GuardrailsState) SupportedScenarios() []string

SupportedScenarios returns a copy of the scenarios guardrails can gate.

type IncomingAPIType 

type IncomingAPIType string

IncomingAPIType describes which OpenAI-style endpoint the client originally hit on this gateway. Only consulted when the provider declares EndpointModeBoth; otherwise the provider's declared mode dictates the upstream endpoint regardless of what the client sent. 

const (
	IncomingAPIChat      IncomingAPIType = "chat"
	IncomingAPIResponses IncomingAPIType = "responses"
)

type LoadBalancer 

type LoadBalancer struct {
	// contains filtered or unexported fields
}

LoadBalancer is the tactic-selection engine: it narrows a rule's services to the healthy active set and delegates the final pick to the rule's configured tactic (rule.LBTactic.Instantiate()). It also serves as the stats/admin surface consumed by LoadBalancerAPI.

func NewLoadBalancer 

func NewLoadBalancer(cfg *config.Config, healthFilter *routing.HealthFilter) *LoadBalancer

NewLoadBalancer creates a new load balancer

func (*LoadBalancer) ClearAllStats 

func (lb *LoadBalancer) ClearAllStats()

ClearAllStats clears all statistics (both in-memory and persisted in config)

func (*LoadBalancer) ClearServiceStats 

func (lb *LoadBalancer) ClearServiceStats(provider, model string)

ClearServiceStats clears statistics for a specific provider:model — both the persisted stats store and the in-memory ServiceStats on every active rule service that matches (stats are global per provider:model, shared across rules). The previous implementation only touched an internal map that was never populated, so the clear was a no-op.

func (*LoadBalancer) GetAllServiceStats 

func (lb *LoadBalancer) GetAllServiceStats() map[string]*loadbalance.ServiceStats

GetAllServiceStats returns all service statistics from all active rules. Stats are keyed by provider:model since stats are global (shared across rules).

func (*LoadBalancer) GetRuleSummary 

func (lb *LoadBalancer) GetRuleSummary(rule *typ.Rule) map[string]interface{}

GetRuleSummary returns a summary of rule configuration and statistics

func (*LoadBalancer) GetServiceStats 

func (lb *LoadBalancer) GetServiceStats(provider, model string) *loadbalance.ServiceStats

GetServiceStats returns statistics for a specific service

func (*LoadBalancer) HealthFilter 

func (lb *LoadBalancer) HealthFilter() *routing.HealthFilter

HealthFilter returns the health filter for the load balancer

func (*LoadBalancer) PreviewService 

func (lb *LoadBalancer) PreviewService(rule *typ.Rule) (*loadbalance.Service, error)

PreviewService selects exactly like SelectService but never claims a breaker probe slot. Read-only surfaces (the admin current-service preview) must use it — they never dispatch, so a claimed half-open probe would get no recorded outcome and would block real traffic from probing the recovering service until the stale-probe reclaim kicks in.

func (*LoadBalancer) SelectService 

func (lb *LoadBalancer) SelectService(rule *typ.Rule) (*loadbalance.Service, error)

SelectService selects the best service for a rule based on the configured tactic, claiming the picked service's breaker probe slot (the dispatch path records the outcome, releasing it).

type MetricsData 

type MetricsData struct {
	InputTokens  int     // Number of input/prompt tokens
	OutputTokens int     // Number of output/completion tokens
	LatencyMs    int64   // Total request latency in milliseconds
	TTFTMs       int64   // Time To First Token in milliseconds (0 if not available/applicable)
	CacheHit     bool    // Whether this request hit the cache
	TPS          float64 // Per-request output TPS after TTFT (0 when unavailable)
}

MetricsData encapsulates all metrics collected for a request. This structure is used to pass comprehensive metrics to updateServiceStats without requiring frequent function signature changes.

type ModelCapabilities 

type ModelCapabilities struct {
	Batch             CapabilitySupport            `json:"batch"`
	Citations         CapabilitySupport            `json:"citations"`
	CodeExecution     CapabilitySupport            `json:"code_execution"`
	ContextManagement *ContextManagementCapability `json:"context_management,omitempty"`
	Effort            *EffortCapability            `json:"effort,omitempty"`
	ImageInput        CapabilitySupport            `json:"image_input"`
	PDFInput          CapabilitySupport            `json:"pdf_input"`
	StructuredOutputs CapabilitySupport            `json:"structured_outputs"`
	Thinking          *ThinkingCapability          `json:"thinking,omitempty"`
}

ModelCapabilities maps to Anthropic's ModelCapabilities in /v1/models.

type OpenAIModel 

type OpenAIModel struct {
	ID          string `json:"id"`
	Object      string `json:"object"`
	Created     int64  `json:"created"`
	OwnedBy     string `json:"owned_by"`
	Description string `json:"description,omitempty"` // Model description
	Context     int    `json:"context,omitempty"`     // Max context window
	MaxOutput   int    `json:"max_output,omitempty"`  // Max output tokens
	// AuthType reflects the primary backing provider's auth type. It is
	// non-standard (OpenAI's models API has no such field) and consumed by
	// the tingly-box frontend to order model picker entries:
	// oauth -> api_key -> vmodel.
	AuthType string `json:"auth_type,omitempty"`
}

OpenAIModel represents a model in OpenAI's models API format

type OpenAIModelsResponse 

type OpenAIModelsResponse struct {
	Object string        `json:"object"`
	Data   []OpenAIModel `json:"data"`
}

OpenAIModelsResponse represents OpenAI's models API response format

type ProtocolHandler 

type ProtocolHandler struct {
	// contains filtered or unexported fields
}

ProtocolHandler is the aggregate handler for the AI Model API. Individual method files (openai_*.go, anthropic_*.go, protocol_*.go, etc.) will be moved here in later steps and become methods on *ProtocolHandler.

func NewHandler 

func NewHandler(deps ProtocolHandlerDeps) *ProtocolHandler

NewHandler constructs the AI Model API handler from its dependencies.

func (*ProtocolHandler) AnthropicCountTokens 

func (ph *ProtocolHandler) AnthropicCountTokens(c *gin.Context)

AnthropicCountTokens handles Anthropic v1 count_tokens endpoint This is the entry point that delegates to the appropriate implementation (v1 or beta)

func (*ProtocolHandler) AnthropicListModelsForScenario 

func (ph *ProtocolHandler) AnthropicListModelsForScenario(c *gin.Context, scenario typ.RuleScenario)

AnthropicListModelsForScenario handles scenario-scoped model listing for Anthropic format

func (*ProtocolHandler) AnthropicMessagesV1 

func (ph *ProtocolHandler) AnthropicMessagesV1(c *gin.Context, req *protocol.AnthropicMessagesRequest, requestModel string, responseModel string, rule *typ.Rule, provider *typ.Provider)

AnthropicMessagesV1 implements standard v1 messages API.

It runs the provider-independent prologue once, then drives the failover loop whose per-attempt callback re-runs the whole provider-dependent pipeline (pre-chain → guardrails → target resolution → transform → dispatch) against the candidate selected for that attempt. Because the transform is re-run per attempt, failover can rotate across heterogeneous API styles.

func (*ProtocolHandler) AnthropicMessagesV1Beta 

func (ph *ProtocolHandler) AnthropicMessagesV1Beta(c *gin.Context, req *protocol.AnthropicBetaMessagesRequest, requestModel string, responseModel string, rule *typ.Rule, provider *typ.Provider)

AnthropicMessagesV1Beta implements beta messages API.

Like AnthropicMessagesV1, the provider-independent prologue runs once and the per-attempt callback re-runs the provider-dependent pipeline so failover can rotate across heterogeneous API styles.

func (*ProtocolHandler) CallMCPToolWithHooks 

func (ph *ProtocolHandler) CallMCPToolWithHooks(ctx context.Context, toolName, arguments string, messages []map[string]any) (context.Context, coretool.ToolResult, error)

CallMCPToolWithHooks executes response-phase MCP servertool hooks before the runtime call. Returns updated context (with advisor quota decremented), result, and error.

func (*ProtocolHandler) DispatchChainResult 

func (ph *ProtocolHandler) DispatchChainResult(
	c *gin.Context, reqCtx *transform.TransformContext,
	rule *typ.Rule, provider *typ.Provider,
	isStreaming bool, recorder *recording.ProtocolRecorder,
)

dispatchChainResult do request from source to target, and return upstream response from target to source

func (*ProtocolHandler) DispatchGenericAnthropicBetaNonStream 

func (ph *ProtocolHandler) DispatchGenericAnthropicBetaNonStream(
	c *gin.Context,
	reqCtx *transform.TransformContext,
	rule *typ.Rule,
	provider *typ.Provider,
	recorder *recording.ProtocolRecorder,
)

DispatchGenericAnthropicBetaNonStream handles Aβ→Aβ non-streaming with generic processor

func (*ProtocolHandler) DispatchGenericAnthropicBetaStream 

func (ph *ProtocolHandler) DispatchGenericAnthropicBetaStream(
	c *gin.Context,
	reqCtx *transform.TransformContext,
	rule *typ.Rule,
	provider *typ.Provider,
	recorder *recording.ProtocolRecorder,
)

DispatchGenericAnthropicBetaStream handles Aβ→Aβ streaming with generic interceptor

func (*ProtocolHandler) DispatchGenericOpenAIChatNonStream 

func (ph *ProtocolHandler) DispatchGenericOpenAIChatNonStream(
	c *gin.Context,
	reqCtx *transform.TransformContext,
	rule *typ.Rule,
	provider *typ.Provider,
	recorder *recording.ProtocolRecorder,
)

DispatchGenericOpenAIChatNonStream handles O→O non-streaming with generic processor

func (*ProtocolHandler) DispatchGenericOpenAIChatStream 

func (ph *ProtocolHandler) DispatchGenericOpenAIChatStream(
	c *gin.Context,
	reqCtx *transform.TransformContext,
	rule *typ.Rule,
	provider *typ.Provider,
	recorder *recording.ProtocolRecorder,
)

DispatchGenericOpenAIChatStream handles O→O streaming with generic interceptor

func (*ProtocolHandler) DispatchWithPriorityFailover 

func (ph *ProtocolHandler) DispatchWithPriorityFailover(
	c *gin.Context,
	rule *typ.Rule,
	initialProvider *typ.Provider,
	initialModel string,
	attempt dispatchAttempt,
)

dispatchWithPriorityFailover runs `attempt` repeatedly, retrying on retryable buffered failures until either the gate commits (the stream's first real chunk reached the wire, retry impossible) or the candidate pool is exhausted (the last buffered error flushes on the deferred return).

Single-service requests bypass the gate entirely: with no fallback tier, failover is impossible and there is no reason to buffer.

func (*ProtocolHandler) EnsureProtocolRecorder 

func (ph *ProtocolHandler) EnsureProtocolRecorder(c *gin.Context, scenario string, provider *typ.Provider, model string, mode obs.RecordMode, bs []byte) *recording.ProtocolRecorder

EnsureProtocolRecorder returns a ProtocolRecorder for the given scenario, reusing any recorder already stored in the gin context. Returns nil when recording is disabled (no sink) or the request body cannot be read.

GetOrCreateScenarioSink is a ProtocolHandlerDeps callback rather than a direct call because scenario sink lifecycle (creation, mutex, recordDir) still lives on root *Server — see ProtocolHandlerDeps.

func (*ProtocolHandler) FailAttemptSetup 

func (ph *ProtocolHandler) FailAttemptSetup(c *gin.Context, err error)

failAttemptSetup reports an in-attempt setup failure — target/endpoint resolution, the pre-transform chain, or the transform itself — that happens before any upstream call. It always writes a 500-class status, which the failover gate buffers and treats as retryable, so the orchestrator advances to the next candidate (possibly a different API style) instead of terminating the whole request on one misconfigured provider. Genuine client errors are rejected in the prologue, before the gate is installed, so they remain non-retryable and reach the client unchanged.

func (*ProtocolHandler) HandleAnthropicListModels 

func (ph *ProtocolHandler) HandleAnthropicListModels(c *gin.Context)

HandleAnthropicListModels handles Anthropic v1 models endpoint

func (*ProtocolHandler) HandleAnthropicMessages 

func (ph *ProtocolHandler) HandleAnthropicMessages(c *gin.Context)

HandleAnthropicMessages handles Anthropic v1 messages API requests This is the entry point that delegates to the appropriate implementation (v1 or beta)

func (*ProtocolHandler) HandleOpenAIChatCompletions 

func (ph *ProtocolHandler) HandleOpenAIChatCompletions(c *gin.Context)

HandleOpenAIChatCompletions handles OpenAI v1 chat completion requests

func (*ProtocolHandler) HandleOpenAIEmbeddings 

func (ph *ProtocolHandler) HandleOpenAIEmbeddings(c *gin.Context)

HandleOpenAIEmbeddings serves OpenAI-compatible embedding requests.

The endpoint is exposed via the mixin route group, so any scenario whose descriptor declares TransportOpenAI or TransportEmbed can reach it. The canonical home is the dedicated `embed` scenario; `openai` scenario also works because its descriptor is extended with TransportEmbed.

func (*ProtocolHandler) HandleOpenAIImageGeneration 

func (ph *ProtocolHandler) HandleOpenAIImageGeneration(c *gin.Context)

HandleOpenAIImageGeneration serves OpenAI-compatible image generation requests against the upstream POST /v1/images/generations endpoint. The request is forwarded as-is; tingly-box does not probe whether the upstream prefers the dedicated images endpoint or the Responses API — the caller chooses the surface and the corresponding tingly-box route.

Exposed via the mixin route group, so any scenario whose descriptor declares TransportImageGen (or TransportOpenAI as a mixin) can reach it. The canonical home is the dedicated `imagegen` scenario.

func (*ProtocolHandler) HandleOpenAIListModels 

func (ph *ProtocolHandler) HandleOpenAIListModels(c *gin.Context)

HandleOpenAIListModels handles the /v1/models endpoint (OpenAI compatible)

func (*ProtocolHandler) HandleResponsesCreate 

func (ph *ProtocolHandler) HandleResponsesCreate(c *gin.Context)

HandleResponsesCreate handles POST /v1/responses

func (*ProtocolHandler) HandleResponsesGet 

func (ph *ProtocolHandler) HandleResponsesGet(c *gin.Context)

HandleResponsesGet handles GET /v1/responses/{id}

func (*ProtocolHandler) ListModelsByScenario 

func (ph *ProtocolHandler) ListModelsByScenario(c *gin.Context)

ListModelsByScenario handles the /v1/models endpoint for scenario-based routing

func (*ProtocolHandler) NonstreamAnthropicV1 

func (ph *ProtocolHandler) NonstreamAnthropicV1(
	c *gin.Context,
	reqCtx *transform.TransformContext,
	rule *typ.Rule,
	provider *typ.Provider,
	recorder *recording.ProtocolRecorder,
)

NonstreamAnthropicV1 handles A→A non-streaming with generic processor

func (*ProtocolHandler) OpenAIChatCompletion 

func (ph *ProtocolHandler) OpenAIChatCompletion(c *gin.Context, req *protocol.OpenAIChatCompletionRequest, responseModel string, provider *typ.Provider, scenarioType typ.RuleScenario, rule *typ.Rule)

OpenAIChatCompletion runs the provider-independent prologue once, then drives the failover loop whose per-attempt callback re-runs the provider-dependent pipeline (align → cap → target resolution → transform → dispatch) so failover can rotate across heterogeneous API styles.

func (*ProtocolHandler) OpenAIListModelsForScenario 

func (ph *ProtocolHandler) OpenAIListModelsForScenario(c *gin.Context, scenario typ.RuleScenario)

OpenAIListModelsForScenario handles scenario-scoped model listing for OpenAI format

func (*ProtocolHandler) RegisterRoutes 

func (ph *ProtocolHandler) RegisterRoutes(engine *gin.Engine, modelAuth gin.HandlerFunc)

RegisterRoutes mounts the LLM gateway surface (/tingly/:scenario[/v1]/...) onto engine. modelAuth gates every model endpoint (model API-token trust domain — distinct from the host's user/admin auth).

The host server owns the gin engine and the auth middleware; this package owns the route shape and the handlers behind it.

func (*ProtocolHandler) ReportHealthStatus 

func (ph *ProtocolHandler) ReportHealthStatus(provider *typ.Provider, model string, err error, errorCode string)

reportHealthStatus reports the health status of a service based on request outcome. It uses the health monitor to track service health for load balancing decisions.

func (*ProtocolHandler) ResponsesCreate 

func (ph *ProtocolHandler) ResponsesCreate(c *gin.Context, scenarioType typ.RuleScenario, provider *typ.Provider, rule *typ.Rule, req *protocol.ResponseCreateRequest, responseModel string, maxAllowed int)

ResponsesCreate runs the provider-independent prologue once, then drives the failover loop whose per-attempt callback re-runs the provider-dependent pipeline (target resolution → transform → dispatch) so failover can rotate across heterogeneous API styles.

func (*ProtocolHandler) RunGenericAnthropicBetaNonStream 

func (ph *ProtocolHandler) RunGenericAnthropicBetaNonStream(
	ctx context.Context,
	provider *typ.Provider,
	req *anthropic.BetaMessageNewParams,
	recorder *recording.ProtocolRecorder,
) (*anthropic.BetaMessage, *mcp.TokenUsage, error)

func (*ProtocolHandler) RunGenericAnthropicV1NonStream 

func (ph *ProtocolHandler) RunGenericAnthropicV1NonStream(
	ctx context.Context,
	provider *typ.Provider,
	req *anthropic.MessageNewParams,
	recorder *recording.ProtocolRecorder,
) (*anthropic.Message, *mcp.TokenUsage, error)

func (*ProtocolHandler) RunGenericOpenAIChatNonStream 

func (ph *ProtocolHandler) RunGenericOpenAIChatNonStream(
	ctx context.Context,
	provider *typ.Provider,
	req *openai.ChatCompletionNewParams,
	recorder *recording.ProtocolRecorder,
) (*openai.ChatCompletion, *mcp.TokenUsage, error)

func (*ProtocolHandler) SetupAnthropicEndpoints 

func (ph *ProtocolHandler) SetupAnthropicEndpoints(group *gin.RouterGroup, modelAuth gin.HandlerFunc)

SetupAnthropicEndpoints registers the Anthropic-only endpoint set on group.

func (*ProtocolHandler) SetupMixinEndpoints 

func (ph *ProtocolHandler) SetupMixinEndpoints(group *gin.RouterGroup, modelAuth gin.HandlerFunc)

SetupMixinEndpoints registers the protocol-mixed endpoint set (OpenAI + Anthropic surfaces on one group), each gated by modelAuth.

func (*ProtocolHandler) SetupOpenAIEndpoints 

func (ph *ProtocolHandler) SetupOpenAIEndpoints(group *gin.RouterGroup, modelAuth gin.HandlerFunc)

SetupOpenAIEndpoints registers the OpenAI-only endpoint set on group.

func (*ProtocolHandler) StreamAnthropicBeta 

func (ph *ProtocolHandler) StreamAnthropicBeta(c *gin.Context, req *anthropic.BetaMessageNewParams, streamResp *anthropicstream.Stream[anthropic.BetaRawMessageStreamEventUnion], actualModel string, responseModel string, provider *typ.Provider, recorder *recording.ProtocolRecorder)

StreamAnthropicBeta processes the Anthropic beta streaming response. The resolved model is passed in as actualModel rather than read from the request, so the handler no longer depends on req.Model.

func (*ProtocolHandler) StreamAnthropicBetaToOpenAIChatWithMCP 

func (ph *ProtocolHandler) StreamAnthropicBetaToOpenAIChatWithMCP(
	c *gin.Context,
	provider *typ.Provider,
	req *anthropic.BetaMessageNewParams,
	actualModel string,
	responseModel string,
	disableStreamUsage bool,
	recorder *recording.ProtocolRecorder,
)

func (*ProtocolHandler) StreamAnthropicV1 

func (ph *ProtocolHandler) StreamAnthropicV1(
	c *gin.Context,
	reqCtx *transform.TransformContext,
	rule *typ.Rule,
	provider *typ.Provider,
	recorder *recording.ProtocolRecorder,
)

StreamAnthropicV1 handles A→A streaming with generic interceptor

func (*ProtocolHandler) StreamOpenAIChatToAnthropicBetaWithMCP 

func (ph *ProtocolHandler) StreamOpenAIChatToAnthropicBetaWithMCP(
	c *gin.Context,
	provider *typ.Provider,
	req *openai.ChatCompletionNewParams,
	actualModel string,
	responseModel string,
	recorder *recording.ProtocolRecorder,
)

func (*ProtocolHandler) StreamOpenAIChatToAnthropicV1WithMCP 

func (ph *ProtocolHandler) StreamOpenAIChatToAnthropicV1WithMCP(
	c *gin.Context,
	provider *typ.Provider,
	req *openai.ChatCompletionNewParams,
	actualModel string,
	responseModel string,
	recorder *recording.ProtocolRecorder,
)

func (*ProtocolHandler) TransformAnthropicBeta 

func (ph *ProtocolHandler) TransformAnthropicBeta(c *gin.Context, req *protocol.AnthropicBetaMessagesRequest, target protocol.APIType, provider *typ.Provider, isStreaming bool, protocolRecorder *recording.ProtocolRecorder, scenarioType typ.RuleScenario, preBaseTransforms []transform.Transform, preVendorTransforms []transform.Transform) (*transform.TransformContext, error)

func (*ProtocolHandler) TransformAnthropicV1 

func (ph *ProtocolHandler) TransformAnthropicV1(c *gin.Context, req *protocol.AnthropicMessagesRequest, target protocol.APIType, provider *typ.Provider, isStreaming bool, protocolRecorder *recording.ProtocolRecorder, scenarioType typ.RuleScenario, preBaseTransforms []transform.Transform, preVendorTransforms []transform.Transform) (*transform.TransformContext, error)

func (*ProtocolHandler) TransformOpenAIChat 

func (ph *ProtocolHandler) TransformOpenAIChat(c *gin.Context, req *protocol.OpenAIChatCompletionRequest, target protocol.APIType, provider *typ.Provider, isStreaming bool, protocolRecorder *recording.ProtocolRecorder, scenarioType typ.RuleScenario, preBaseTransforms []transform.Transform, preVendorTransforms []transform.Transform) (*transform.TransformContext, error)

func (*ProtocolHandler) TransformOpenAIResponses 

func (ph *ProtocolHandler) TransformOpenAIResponses(c *gin.Context, req *protocol.ResponseCreateRequest, target protocol.APIType, provider *typ.Provider, isStreaming bool, protocolRecorder *recording.ProtocolRecorder, scenarioType typ.RuleScenario, maxAllowed int, preBaseTransforms []transform.Transform, preVendorTransforms []transform.Transform) (*transform.TransformContext, error)

type ProtocolHandlerDeps 

type ProtocolHandlerDeps struct {
	Config *config.Config

	// TokenTracker records usage to the OTel meter pipeline (may be nil if
	// OTel setup failed at startup — callers must nil-check).
	TokenTracker *tracker.TokenTracker

	// HealthMonitor reports per-service health outcomes (success / rate
	// limit / auth error / general error) back into the load-balance health
	// filter (may be nil — callers must nil-check).
	HealthMonitor *loadbalance.HealthMonitor

	// ClientPool caches upstream provider clients (OpenAI/Anthropic/Google).
	ClientPool *client.ClientPool

	// LoadBalancer selects the active service for a rule (tier/random/etc
	// tactics). Used by failover to pick the next candidate on retry.
	LoadBalancer *LoadBalancer

	// TemplateManager resolves per-provider model metadata (max tokens,
	// description, context window) from the provider template catalog.
	TemplateManager *data.TemplateManager

	// RoutingSelector runs the full selection pipeline (health → smart →
	// affinity → strategy) for a scenario/rule/request, used by the top-level
	// OpenAI/Anthropic entry handlers (Step 10).
	RoutingSelector *routing.SimpleSelector

	// VisionProxyService rewrites image content into text descriptions for
	// scenarios/rules configured with a vision proxy plugin. Concrete type,
	// not a callback: internal/visionproxy.Service takes *config.Config
	// directly and has no *server.Server dependency.
	VisionProxyService *visionproxy.Service

	// MCPRuntime holds the virtual tool registry and advisor state for
	// external MCP tools invoked during a live model request.
	MCPRuntime *mcpruntime.Runtime

	// GetServertoolPipeline returns the current virtual-tool-provider pipeline
	// (advisor quota hooks, etc.). A callback rather than a plain field
	// because root's config hot-reload (registerAdviserFromConfig) reassigns
	// *Server.servertoolPipeline in place — a plain field copied at
	// NewHandler time would silently go stale on the next reload. May return
	// nil — callers fall back to servertool.NewDefaultExecutor.
	GetServertoolPipeline func() *servertool.Pipeline

	// AffinityStore holds session→service affinity entries (sticky routing).
	// Owned by the host server (constructed and GC-started there); the
	// gateway updates message IDs on it after each response.
	AffinityStore *AffinityStore

	// GuardrailsState holds the mutex-guarded guardrails runtime pointer and
	// its lifecycle operations. Owned by this package; the host server's
	// admin handlers drive it through *Server's thin forwarding methods.
	GuardrailsState *GuardrailsState

	// GetOrCreateScenarioSink reaches back into root *Server state that has
	// not (yet) moved here: scenario recording sinks. Wiring it as a func
	// keeps this package independent of *server.Server.
	GetOrCreateScenarioSink func(scenario typ.RuleScenario) *obs.Sink

	// GetScenarioRecordMode resolves the effective recording mode for a
	// scenario. Backed by root's s.recordMode/s.scenarioRecordSinks, which
	// have not moved to aimodel (recording lifecycle stays root-owned).
	GetScenarioRecordMode func(scenario typ.RuleScenario) obs.RecordMode
}

ProtocolHandlerDeps declares exactly what the AI Model API handlers need from the host server. It is populated and passed in once, from server.NewServer, after all of *Server's fields have been constructed.

This grows as each subsequent migration step moves a file in and wires up the fields/methods it actually touches on *Server today.

type ThinkingCapability 

type ThinkingCapability struct {
	Supported bool           `json:"supported"`
	Types     *ThinkingTypes `json:"types,omitempty"`
}

ThinkingCapability describes thinking support.

type ThinkingTypes 

type ThinkingTypes struct {
	Adaptive CapabilitySupport `json:"adaptive,omitempty"`
	Enabled  CapabilitySupport `json:"enabled,omitempty"`
}

ThinkingTypes describes supported thinking type configurations.

type TransformRecorder 

type TransformRecorder struct {
	// contains filtered or unexported fields
}

TransformRecorder is a transform.Transform that snapshots the request body at a given stage and stores it on a ProtocolRecorder.

func NewTransformRecorder 

func NewTransformRecorder(c *gin.Context, recorder *recording.ProtocolRecorder, stage TransformStage) *TransformRecorder

NewTransformRecorder builds a recorder transform for the given stage.

func (*TransformRecorder) Apply 

func (t *TransformRecorder) Apply(ctx *transform.TransformContext) error

func (*TransformRecorder) Name 

func (t *TransformRecorder) Name() string

type TransformStage 

type TransformStage int

TransformStage selects which side of the transform pipeline the recorder captures from. 

const (
	// StagePre captures ctx.OriginalRequest before any transformation is
	// applied.
	StagePre TransformStage = iota
	// StagePost captures ctx.Request after base transformation.
	StagePost
)

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.