server

package
v0.260723.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 24, 2026 License: MPL-2.0 Imports: 109 Imported by: 0

Documentation

Overview

Package aimodel hosts the AI Model API surface split out of internal/server: the LLM gateway routes (/tingly/:scenario/...) covering OpenAI/Anthropic compatible chat/completions/responses/messages/embeddings/images, MCP tool handling during model requests, protocol dispatch/transform/passthrough, failover/load-balance dispatch, guardrails runtime evaluation, and usage/token tracking.

This package intentionally does not depend on *server.Server. Handlers are built from the narrow ProtocolHandlerDeps struct below, following the same pattern already used by internal/server/module/* (see module/scenario/handler.go).

Index

Constants

View Source 
const (
	ContextKeyRule           = constant.CtxKeyRule
	ContextKeyProvider       = constant.CtxKeyProvider
	ContextKeyModel          = constant.CtxKeyModel
	ContextKeyRequestModel   = constant.CtxKeyRequestModel
	ContextKeyScenario       = constant.CtxKeyScenario
	ContextKeyStreamed       = constant.CtxKeyStreamed
	ContextKeyStartTime      = constant.CtxKeyStartTime
	ContextKeyFirstTokenTime = constant.CtxKeyFirstTokenTime
	ContextKeyCacheHit       = constant.CtxKeyCacheHit
	ContextKeySessionID      = constant.CtxKeySessionID
	ContextKeyAffinityKey    = constant.CtxKeyAffinityKey
	ContextKeyLBServiceID    = constant.CtxKeyLBServiceID
	ContextKeyLBTactic       = constant.CtxKeyLBTactic
)

Gin context key aliases — canonical values live in constant so that routing/ and middleware/ sub-packages can reference them without an import cycle.

View Source 
const GuardrailsRegistryGitHubURL = "https://raw.githubusercontent.com/tingly-dev/tingly-guardrails-registry/main/index.yaml"

Leave the remote registry unset until the dedicated policy repository is ready. The API will surface this as an unavailable registry instead of coupling guardrails downloads to the main code repository.

View Source 
const ProbeSyntheticRuleUUID = "probe-synthetic"

ProbeSyntheticRuleUUID marks the throwaway rule built for an X-Tingly-Probe-Service request — it has no persisted identity. Owned here (moved from root's handlers.go constant) since protocol_dispatch.go's setProbeUpstreamHeaders is the sole consumer that has moved so far; root's handlers.go (not yet moved) keeps a companion alias.

Variables

View Source 
var GuardrailsSupportedScenarios = []string{
	string(typ.ScenarioAnthropic),
	string(typ.ScenarioClaudeCode),
}

GuardrailsSupportedScenarios lists the scenarios guardrails can gate.

Functions

func ApplyGuardrailsToAnthropicV1BetaNonStreamResponse added in v0.260709.1 

func ApplyGuardrailsToAnthropicV1BetaNonStreamResponse(c *gin.Context, runtime *guardrails.Guardrails, req *anthropic.BetaMessageNewParams, actualModel string, provider *typ.Provider, resp *anthropic.BetaMessage) bool

ApplyGuardrailsToAnthropicV1BetaNonStreamResponse is the beta equivalent of ApplyGuardrailsToAnthropicV1NonStreamResponse.

func ApplyGuardrailsToAnthropicV1BetaRequest added in v0.260709.1 

func ApplyGuardrailsToAnthropicV1BetaRequest(c *gin.Context, runtime *guardrails.Guardrails, req *anthropic.BetaMessageNewParams, actualModel string, provider *typ.Provider)

ApplyGuardrailsToAnthropicV1BetaRequest is the merged request-side entry for Anthropic beta requests. It runs request tool_result filtering first and then request credential masking on the latest raw request state.

func ApplyGuardrailsToAnthropicV1NonStreamResponse added in v0.260709.1 

func ApplyGuardrailsToAnthropicV1NonStreamResponse(c *gin.Context, runtime *guardrails.Guardrails, req *anthropic.MessageNewParams, actualModel string, provider *typ.Provider, resp *anthropic.Message) bool

ApplyGuardrailsToAnthropicV1NonStreamResponse evaluates a fully assembled Anthropic v1 response and rewrites it when guardrails block it.

func ApplyGuardrailsToAnthropicV1Request added in v0.260709.1 

func ApplyGuardrailsToAnthropicV1Request(c *gin.Context, runtime *guardrails.Guardrails, req *anthropic.MessageNewParams, actualModel string, provider *typ.Provider)

ApplyGuardrailsToAnthropicV1Request is the merged request-side entry for Anthropic v1 requests. It runs request tool_result filtering first and then request credential masking on the latest raw request state.

func AttachGuardrailsHooks added in v0.260709.1 

func AttachGuardrailsHooks(c *gin.Context, runtime *guardrails.Guardrails, hc *protocol.HandleContext, actualModel string, provider *typ.Provider, messages []guardrailscore.Message)

AttachGuardrailsHooks wires the shared stream guardrails runtime into a protocol handle context. Provider-specific handlers only need to provide already-normalized message history.

func AuthTypeSortWeight added in v0.260709.1 

func AuthTypeSortWeight(a typ.AuthType) int

AuthTypeSortWeight ranks auth types for /v1/models ordering: oauth (0) -> api_key (1) -> vmodel (2). Any unknown value is treated as api_key so legacy entries cluster with regular providers.

func BuildGuardrailsBaseInput added in v0.260709.1 

func BuildGuardrailsBaseInput(c *gin.Context, actualModel string, provider *typ.Provider, direction guardrailscore.Direction, messages []guardrailscore.Message) guardrailscore.Input

BuildGuardrailsBaseInput creates the shared evaluation envelope; adapters can then add request/response-specific content without rebuilding metadata each time.

func BuildOpenAIContinuationSegment added in v0.260709.1 

func BuildOpenAIContinuationSegment(
	calls []stream.OpenAIToAnthropicToolCall,
	virtualResults []mcp.ToolExecutionResult,
) []openai.ChatCompletionMessageParamUnion

BuildOpenAIContinuationSegment builds the assistant tool_calls + tool result messages appended to the OpenAI message history after a round of virtual MCP tool execution.

func CalculateLatencyFromStart added in v0.260709.1 

func CalculateLatencyFromStart(startTime time.Time) int

CalculateLatencyFromStart calculates the elapsed time in milliseconds since the start time.

func CalculateTPS 

func CalculateTPS(c *gin.Context, outputTokens int, streamed bool) float64

CalculateTPS calculates Tokens Per Second (generation speed) for streaming requests. For non-streaming requests or when TTFT is not available, returns 0.

TPS is calculated as: outputTokens / (currentTime - firstTokenTime) This measures the actual token generation speed after the first token was received.

Parameters:

  • c: Gin context containing timing information
  • outputTokens: Number of output tokens generated
  • streamed: Whether this was a streaming request

Returns:

  • TPS value (tokens per second), or 0 if not applicable

func CalculateTTFT 

func CalculateTTFT(c *gin.Context) int64

CalculateTTFT returns Time To First Token in milliseconds, or 0 when no first-token time was recorded (e.g. non-streaming requests). It must not fall back to total latency, which would make TTFT indistinguishable from it.

func CloneAnthropicBetaRequest added in v0.260709.1 

func CloneAnthropicBetaRequest(template []byte) (*protocol.AnthropicBetaMessagesRequest, error)

CloneAnthropicBetaRequest rebuilds an Anthropic beta request from a marshalled template (produced by AnthropicBetaMessagesRequest.MarshalJSON).

func CloneAnthropicV1Request added in v0.260709.1 

func CloneAnthropicV1Request(template []byte) (*protocol.AnthropicMessagesRequest, error)

CloneAnthropicV1Request rebuilds an Anthropic v1 request from a marshalled template (produced by AnthropicMessagesRequest.MarshalJSON).

func CloneOpenAIChatRequest added in v0.260709.1 

func CloneOpenAIChatRequest(template []byte) (*protocol.OpenAIChatCompletionRequest, error)

CloneOpenAIChatRequest rebuilds an OpenAI chat request from a marshalled template (produced by OpenAIChatCompletionRequest.MarshalJSON).

func CloneResponsesParams added in v0.260709.1 

func CloneResponsesParams(template *responses.ResponseNewParams) (*responses.ResponseNewParams, error)

CloneResponsesParams clones the typed responses.ResponseNewParams. It marshals and unmarshals the SDK struct directly — NOT through ResponseCreateRequest, whose UnmarshalJSON re-runs PreprocessInputData (input-item type injection) that has already been applied to the template.

func CommitFirstChunkIfGate added in v0.260709.1 

func CommitFirstChunkIfGate(w gin.ResponseWriter) bool

CommitFirstChunkIfGate signals "first real chunk arrived" if w is currently a *firstChunkGate (installed by DispatchWithPriorityFailover when a rule has more than one active service), no-op otherwise. Exported so callers outside this package that simulate a streaming success (e.g. the load-balance dry-run simulator) can commit the gate without needing firstChunkGate itself exported. Returns whether a gate was found and committed.

func ConvertAnthropicToOpenAIResponseWithProvider 

func ConvertAnthropicToOpenAIResponseWithProvider(anthropicResp *anthropic.BetaMessage, responseModel string, provider *typ.Provider, model string) map[string]interface{}

ConvertAnthropicToOpenAIResponseWithProvider converts an Anthropic response to OpenAI format and applies provider-specific transformations to the response

func DetectCacheHit added in v0.260709.1 

func DetectCacheHit(usage *protocol.TokenUsage) bool

DetectCacheHit determines if a request was served from cache based on TokenUsage. Returns true if cache was hit, false otherwise.

Detection logic:

  • OpenAI: usage.CacheInputTokens > 0 (cache_read_input_tokens field)
  • Anthropic: usage.CacheInputTokens > 0 (cache_read_input_tokens field)
  • Other providers: Returns false (conservative - assumes cache miss)

Parameters:

  • usage: Token usage information from the response

Returns:

  • true if cache hit detected, false otherwise

func EnsureGuardrailsCredentialMaskState added in v0.260709.1 

func EnsureGuardrailsCredentialMaskState(c *gin.Context) *guardrailscore.CredentialMaskState

func ExecuteAnthropicPreChain added in v0.260716.1 

func ExecuteAnthropicPreChain[T *anthropic.MessageNewParams | *anthropic.BetaMessageNewParams](
	req T,
	scenarioConfig *typ.ScenarioConfig,
	defaultMaxTokens, maxAllowed int,
	isStreaming bool,
) error

ExecuteAnthropicPreChain builds and runs the server-side pre-transform chain for Anthropic requests (req is *anthropic.MessageNewParams or *anthropic.BetaMessageNewParams — the transforms type-switch internally). Currently only MaxTokens validation remains at scenario level; other scenario-level transforms (ThinkingEffort, CleanHeader) are handled via rule flags injection in resolveRuleFlagsWithScenario. Returns an error that should be mapped to HTTP 400.

func ExtractAnthropicBetaMessages added in v0.260709.1 

func ExtractAnthropicBetaMessages(messages []anthropic.BetaMessageParam) []map[string]any

ExtractAnthropicBetaMessages JSON-roundtrips an Anthropic beta message list into the generic map shape MCP tool hooks pass through as message history.

func ExtractOpenAIMessages added in v0.260709.1 

func ExtractOpenAIMessages(messages []openai.ChatCompletionMessageParamUnion) []map[string]any

ExtractOpenAIMessages JSON-roundtrips an OpenAI message list into the generic map shape MCP tool hooks pass through as message history.

func ExtractScenarioFromPath added in v0.260709.1 

func ExtractScenarioFromPath(path string) string

ExtractScenarioFromPath extracts the scenario segment from a request path, e.g. "/tingly/claude_code/v1/messages" -> "claude_code".

func GenerateCurlCommand 

func GenerateCurlCommand(apiBase, apiStyle, token, model string) string

GenerateCurlCommand generates a curl command for testing the provider

func GenerateOpenAPI added in v0.260418.2200 

func GenerateOpenAPI(cfg *config.Config) (string, error)

GenerateOpenAPI creates an OpenAPI v3 schema without starting the server

func GetCacheHit 

func GetCacheHit(c *gin.Context) (bool, bool)

GetCacheHit retrieves the cache hit status from the context. Returns the cache hit status and true if it exists, false and false otherwise.

func GetFirstTokenTime 

func GetFirstTokenTime(c *gin.Context) (time.Time, bool)

GetFirstTokenTime retrieves the first token time from the context. Returns the timestamp and true if it exists, zero time and false otherwise.

func GetShutdownChannel 

func GetShutdownChannel() <-chan struct{}

GetShutdownChannel returns the shutdown channel for the main process to listen on

func GetTrackingContext 

func GetTrackingContext(c *gin.Context) (rule *typ.Rule, provider *typ.Provider, actualModel, requestModel, scenario string, streamed bool, startTime time.Time)

GetTrackingContext retrieves tracking metadata from the gin context. Returns zero values if the context keys are not set.

func GetTrackingContextScenario added in v0.260625.1 

func GetTrackingContextScenario(c *gin.Context) (scenario string)

func GetUserIDFromContext added in v0.260418.2200 

func GetUserIDFromContext(c *gin.Context) string

GetUserIDFromContext extracts the user ID from gin context. Priority: user_id (from JWT API token or global auth) > enterprise_user_id (from enterprise JWT) > "" This ensures that JWT API token authentication takes precedence for user tracking.

func GuardrailsEnabledForScenario added in v0.260709.1 

func GuardrailsEnabledForScenario(cfg *config.Config, runtime *guardrails.Guardrails, scenario string) bool

GuardrailsEnabledForScenario centralizes feature-flag checks so protocol handlers do not repeat scenario/global guardrails gating logic.

func GuardrailsSupportsScenario added in v0.260709.1 

func GuardrailsSupportsScenario(scenario string) bool

GuardrailsSupportsScenario reports whether scenario is one guardrails can gate.

func HasDeclaredMCPAnthropicBetaTools added in v0.260709.1 

func HasDeclaredMCPAnthropicBetaTools(req *anthropic.BetaMessageNewParams) bool

HasDeclaredMCPAnthropicBetaTools reports whether req declares any MCP-named tool in its Anthropic beta tool list.

func HasDeclaredMCPAnthropicV1Tools added in v0.260709.1 

func HasDeclaredMCPAnthropicV1Tools(req *anthropic.MessageNewParams) bool

HasDeclaredMCPAnthropicV1Tools reports whether req declares any MCP-named tool in its Anthropic v1 tool list. Moved here from root's protocol_dispatch.go (which still hosts its call sites, Step 8) because it is a pure function with zero *Server dependency, same class as its Beta sibling below.

func HasDeclaredMCPTools added in v0.260709.1 

func HasDeclaredMCPTools(req *openai.ChatCompletionNewParams) bool

HasDeclaredMCPTools reports whether req declares any MCP-named tool in its OpenAI Chat tool list.

func HasNativeAdvisorBeta added in v0.260709.1 

func HasNativeAdvisorBeta(req *protocol.AnthropicBetaMessagesRequest) bool

func IsValidRuleScenario added in v0.260709.1 

func IsValidRuleScenario(scenario typ.RuleScenario) bool

IsValidRuleScenario checks if the given scenario is a valid RuleScenario

func MCPEnabled added in v0.260709.1 

func MCPEnabled(cfg *config.Config) bool

MCPEnabled centralizes the MCP feature-flag check so gateway handlers do not repeat scenario-flag lookups. Mirrors GuardrailsEnabledForScenario.

func PrimaryAuthTypeForRule added in v0.260709.1 

func PrimaryAuthTypeForRule(cfg *config.Config, rule typ.Rule) typ.AuthType

PrimaryAuthTypeForRule returns the AuthType of the first active service's provider in a rule. It is used by /v1/models endpoints so the frontend can order picker entries oauth -> api_key -> vmodel.

Returns AuthTypeAPIKey as the fallback for empty/unresolvable rules so they land in the middle group rather than at the head or tail.

func ReattachGuardrailsHooks added in v0.260709.1 

func ReattachGuardrailsHooks(
	c *gin.Context,
	runtime *guardrails.Guardrails,
	hc *protocol.HandleContext,
	actualModel string,
	provider *typ.Provider,
	messages []guardrailscore.Message,
	baseEventHooks int,
	baseErrorHooks int,
)

ReattachGuardrailsHooks resets per-round guardrails state and re-registers fresh hooks on hc for the next MCP loop round. It truncates OnStreamEventHooks back to baseEventHooks (the count before guardrails was first attached) so previous-round guardrails hooks don't accumulate.

func ResolveOpenAIEndpoint added in v0.260531.1 

func ResolveOpenAIEndpoint(provider *typ.Provider, flags typ.RuleFlags, incoming IncomingAPIType) (protocol.APIType, error)

ResolveOpenAIEndpoint picks an OpenAI endpoint using the optional per-rule override first, then the provider's declared OpenAIEndpointMode.

Precedence:

  1. Rule flag (flags.OpenAIEndpointOverride). Overrides provider settings.
  2. provider.OpenAIEndpointMode: EndpointModeUnknown / zero value → Chat EndpointModeChat → Chat EndpointModeResponses → Responses EndpointModeBoth → mirror incoming

Rule override is honored unconditionally (per design intent). When an override conflicts with the provider's declared mode, a warning is logged but the override takes effect. This allows explicit routing control for debugging and special cases.

Defaulting unknown providers to Chat (not "mirror incoming") is intentional: most OpenAI-compatible vendors implement only /chat/completions. Providers that genuinely support Responses must declare it via template or OAuth.

When an incoming Responses request routes to Chat, Responses-only fields (previous_response_id, include, background, truncation, reasoning) are silently dropped by ConvertOpenAIResponsesToChat — the same posture as Anthropic→Chat downgrades. The user accepts this by declaring the mode.

Pure function: no Server state, no probe lookups, no I/O.

func ResolveRuleFlags added in v0.260709.1 

func ResolveRuleFlags(c *gin.Context, rule *typ.Rule) typ.RuleFlags

ResolveRuleFlags returns the effective flags for this request: a copy of the rule's persisted flags, with: - cursor_compat_auto folded into cursor_compat when the inbound request carries Cursor headers Returns the zero value when no rule is bound.

All flag folding/injection happens here (not at each handler call site) so that RulePreBaseTransforms and downstream consumers read the same merged value.

func ResolveRuleFlagsWithScenario added in v0.260709.1 

func ResolveRuleFlagsWithScenario(
	c *gin.Context,
	rule *typ.Rule,
	scenarioType typ.RuleScenario,
	scenarioConfig *typ.ScenarioConfig,
	sourceAPI, targetAPI protocol.APIType,
	provider *typ.Provider,
) typ.RuleFlags

ResolveRuleFlagsWithScenario extends resolveRuleFlags to also inject scenario-level flags and auto-apply CleanHeader for protocol transformation scenarios.

This is the main entry point that merges:

  1. Rule-level flags (from the rule definition)
  2. Scenario flags (from the scenario configuration)
  3. Auto-applied flags (like CleanHeader for protocol transformation)
  4. Provider-driven suppressions (CleanHeader is cleared for Claude OAuth providers; the billing header must reach Anthropic's billing backend unchanged).

Side effect: it also attaches the resolved CustomUserAgent to the request context (applyCustomUserAgent) so callers don't have to repeat that at each handler. The User-Agent is the one rule flag that has to reach a deep component (the outbound transport) via ctx, so this central merge point is where it gets applied.

func RoundtripAnthropicBetaResponseViaOpenAI 

func RoundtripAnthropicBetaResponseViaOpenAI(anthropicResp *anthropic.BetaMessage, responseModel string, provider *typ.Provider, actualModel string) (*anthropic.BetaMessage, error)

func RoundtripOpenAIMapViaAnthropic 

func RoundtripOpenAIMapViaAnthropic(openaiResp map[string]interface{}, responseModel string, provider *typ.Provider, actualModel string) (map[string]interface{}, error)

func RoundtripOpenAIResponseViaAnthropic 

func RoundtripOpenAIResponseViaAnthropic(openaiResp *openai.ChatCompletion, responseModel string, provider *typ.Provider, actualModel string) (map[string]interface{}, error)

func RulePreBaseTransforms added in v0.260709.1 

func RulePreBaseTransforms(flags typ.RuleFlags) []transform.Transform

RulePreBaseTransforms builds the per-rule list of pre-Base transforms for the chain's preBase slot. Pre-Base transforms act on the *inbound* request shape — they run before BaseTransform's protocol conversion, so the type-switch inside each transform sees what the client actually sent.

Returns nil when no rule-level flag requires a pre-Base stage so callers can pass the result straight to BuildTransformChain's preBase parameter.

func RulePreVendorTransforms added in v0.260709.1 

func RulePreVendorTransforms(flags typ.RuleFlags) []transform.Transform

RulePreVendorTransforms builds the per-rule list of pre-Vendor transforms for the chain's preVendor slot (after Consistency, before Vendor). These act on the *target* request shape — they run after BaseTransform's protocol conversion, so the type-switch inside each transform matches the upstream-bound form, but still before Vendor finalizes the request.

Returns nil when no rule-level flag requires a chain stage so callers can pass the result straight to a `preVendorTransforms []transform.Transform` parameter.

Takes already-resolved flags so callers that need other fields off RuleFlags (CustomUserAgent, SkipUsage) can resolve once and share.

func RuntimeAuditSink added in v0.260709.1 

func RuntimeAuditSink(log *audit.Logger) remotescenario.AuditFunc

RuntimeAuditSink adapts an audit.Logger into the AuditFunc the scenario runtime hands to plugins. Plugin actions (e.g. claude_code.interactive.start / .done / .error) land here as audit entries with structured details.

func SendErrorResponse added in v0.260418.2200 

func SendErrorResponse(c *gin.Context, err error, desc string)

SendErrorResponse registers the error into gin context for logging middleware and sends JSON response.

func SetCacheHit 

func SetCacheHit(c *gin.Context, isHit bool)

SetCacheHit records whether this request was a cache hit.

func SetEnterpriseRateLimitReporter 

func SetEnterpriseRateLimitReporter(reporter func(context.Context, string, string, string, string) error)

SetEnterpriseRateLimitReporter sets callback for enterprise 429 events.

func SetGlobalServer 

func SetGlobalServer(server *Server)

SetGlobalServer sets the global server instance for web UI control

func SetTrackingContext 

func SetTrackingContext(c *gin.Context, rule *typ.Rule, provider *typ.Provider, actualModel, requestModel string, streamed bool)

SetTrackingContext sets all tracking metadata in the gin context. This should be called once at the beginning of request processing to avoid explicit parameter passing throughout the handler chain.

Parameters:

  • c: Gin context
  • rule: The load balancer rule that was selected
  • provider: The provider that was selected
  • actualModel: The actual model name used (may differ from requested)
  • requestModel: The original model name requested by the user
  • streamed: Whether this is a streaming request

func ShouldIncludeRuleInModelList added in v0.260709.1 

func ShouldIncludeRuleInModelList(requestedScenario typ.RuleScenario, ruleScenario typ.RuleScenario) bool

ShouldIncludeRuleInModelList reports whether a rule should appear in the model list for the requested scenario. Each scenario — base or profiled — is an isolated scope: it only lists rules bound to that exact scenario. Transport compatibility does not grant cross-scenario visibility.

func ShouldRoundtripResponse 

func ShouldRoundtripResponse(c *gin.Context, target string) bool

func ShouldStripUsage added in v0.260709.1 

func ShouldStripUsage(extra map[string]interface{}) bool

ShouldStripUsage merges the cursor_compat and skip_usage hints carried in reqCtx.Extra. The dispatch layer ORs both together so a rule that only flips skip_usage still strips the usage block, and cursor_compat keeps its historical behavior of suppressing usage as a side effect.

Extracted so the wiring is unit-testable independent of the surrounding transform/forward machinery.

func ShouldUseGenericMCPForProvider added in v0.260709.1 

func ShouldUseGenericMCPForProvider(cfg *config.Config, provider *typ.Provider) bool

ShouldUseGenericMCPForProvider is the pure-Config form of Handler.shouldUseGenericMCPForProvider, exported so callers that only have a *config.Config (e.g. tests constructing a bare *Server without a wired aiHandler) can check the same provider-limits logic directly.

func UpdateTrackingForFailover added in v0.260625.1 

func UpdateTrackingForFailover(c *gin.Context, provider *typ.Provider, model string)

UpdateTrackingForFailover updates provider and model tracking during failover retry. This ensures that logging/middleware shows the final successful service after failover, not the initially-selected failed service.

Parameters:

  • c: Gin context
  • provider: The new provider being tried in this failover attempt
  • model: The new model being tried in this failover attempt

func UseIndexHTML added in v0.260709.1 

func UseIndexHTML(c *gin.Context)

func UseWebStaticEndpoints added in v0.260709.1 

func UseWebStaticEndpoints(engine *gin.Engine)

Types

type ActionHistoryEntry 

type ActionHistoryEntry struct {
	Time    time.Time              `json:"time"`
	Level   string                 `json:"level"`
	Message string                 `json:"message"`
	Action  string                 `json:"action,omitempty"`
	Success bool                   `json:"success,omitempty"`
	Details interface{}            `json:"details,omitempty"`
	Fields  map[string]interface{} `json:"fields,omitempty"`
}

ActionHistoryEntry represents an action history entry for API response

type ActionHistoryResponse 

type ActionHistoryResponse struct {
	Total   int                  `json:"total"`
	Actions []ActionHistoryEntry `json:"actions"`
}

ActionHistoryResponse represents the API response for action history

type AnthropicModel 

type AnthropicModel struct {
	ID             string             `json:"id"`
	CreatedAt      string             `json:"created_at"`
	DisplayName    string             `json:"display_name"`
	Type           string             `json:"type"`
	Capabilities   *ModelCapabilities `json:"capabilities,omitempty"`
	MaxInputTokens int                `json:"max_input_tokens,omitempty"`
	MaxTokens      int                `json:"max_tokens,omitempty"`
	// Description is a tingly-box extension (not in Anthropic's wire format)
	// consumed by the frontend to show model description in the model picker.
	Description string `json:"description,omitempty"`
	// AuthType is a tingly-box extension (not in Anthropic's wire format)
	// consumed by the frontend to order model picker entries:
	// oauth -> api_key -> vmodel.
	AuthType string `json:"auth_type,omitempty"`
}

AnthropicModel maps to Anthropic's native /v1/models response format.

type AnthropicModelsResponse 

type AnthropicModelsResponse struct {
	Data    []AnthropicModel `json:"data"`
	FirstID string           `json:"first_id"`
	HasMore bool             `json:"has_more"`
	LastID  string           `json:"last_id"`
}

type CapabilitySupport added in v0.260611.1 

type CapabilitySupport struct {
	Supported bool `json:"supported"`
}

CapabilitySupport indicates whether a capability is supported.

type ContextManagementCapability added in v0.260611.1 

type ContextManagementCapability struct {
	Supported             bool              `json:"supported"`
	ClearThinking20251015 CapabilitySupport `json:"clear_thinking_20251015,omitempty"`
	ClearToolUses20250919 CapabilitySupport `json:"clear_tool_uses_20250919,omitempty"`
	Compact20260112       CapabilitySupport `json:"compact_20260112,omitempty"`
}

ContextManagementCapability describes context management support.

type EffortCapability added in v0.260611.1 

type EffortCapability struct {
	Supported bool              `json:"supported"`
	Low       CapabilitySupport `json:"low,omitempty"`
	Medium    CapabilitySupport `json:"medium,omitempty"`
	High      CapabilitySupport `json:"high,omitempty"`
	XHigh     CapabilitySupport `json:"xhigh,omitempty"`
	Max       CapabilitySupport `json:"max,omitempty"`
}

EffortCapability describes reasoning_effort support and levels.

type EndpointOverride added in v0.260531.1 

type EndpointOverride string

EndpointOverride is the typed value of the openai_endpoint_override rule flag. It forces an OpenAI request onto a specific endpoint, overriding the provider's declared OpenAIEndpointMode default (provider declarations trump conflicting overrides — see ResolveOpenAIEndpoint). 

const (
	OverrideAuto      EndpointOverride = "auto"
	OverrideChat      EndpointOverride = "chat"
	OverrideResponses EndpointOverride = "responses"
)

func ParseEndpointOverride added in v0.260531.1 

func ParseEndpointOverride(s string) EndpointOverride

ParseEndpointOverride coerces a raw rule-flag string to a known EndpointOverride. Empty, "auto" and any unrecognized value map to OverrideAuto so misconfigured rules degrade safely.

type ErrorDetail 

type ErrorDetail struct {
	Message string `json:"message"`
	Type    string `json:"type"`
	Code    string `json:"code,omitempty"`
}

ErrorDetail represents error details

type ErrorResponse 

type ErrorResponse struct {
	Error ErrorDetail `json:"error"`
}

ErrorResponse represents an error response

type GenerateTokenRequest 

type GenerateTokenRequest struct {
	ClientID string `json:"client_id" binding:"required" description:"Client ID for token generation" example:"user123"`
}

GenerateTokenRequest represents the request to generate a token

type GuardrailsDeps added in v0.260709.1 

type GuardrailsDeps struct {
	Config  *config.Config
	Runtime GuardrailsRuntime

	// GuardrailsConfigMu serializes config/policy/group file edits. It is the
	// SAME mutex instance as root server's Server.guardrailsConfigMu (passed
	// in by pointer) so admin edits and any other root-side writer are
	// mutually exclusive.
	GuardrailsConfigMu *sync.Mutex
}

GuardrailsDeps declares exactly what the guardrails admin handlers need from the host server.

type GuardrailsHandler added in v0.260709.1 

type GuardrailsHandler struct {
	// contains filtered or unexported fields
}

GuardrailsHandler is the aggregate handler for the guardrails admin surface (config editor, policy/group CRUD, protected credentials, registry install, history).

func NewGuardrailsHandler added in v0.260709.1 

func NewGuardrailsHandler(deps GuardrailsDeps) *GuardrailsHandler

NewGuardrailsHandler constructs the guardrails admin handler.

func (*GuardrailsHandler) ClearGuardrailsHistory added in v0.260709.1 

func (h *GuardrailsHandler) ClearGuardrailsHistory(c *gin.Context)

ClearGuardrailsHistory deletes all persisted guardrails history rows.

func (*GuardrailsHandler) CreateGuardrailsCredential added in v0.260709.1 

func (h *GuardrailsHandler) CreateGuardrailsCredential(c *gin.Context)

func (*GuardrailsHandler) CreateGuardrailsGroup added in v0.260709.1 

func (h *GuardrailsHandler) CreateGuardrailsGroup(c *gin.Context)

CreateGuardrailsGroup creates a new group and reloads the engine.

func (*GuardrailsHandler) CreateGuardrailsPolicy added in v0.260709.1 

func (h *GuardrailsHandler) CreateGuardrailsPolicy(c *gin.Context)

CreateGuardrailsPolicy creates a new policy and reloads the engine.

func (*GuardrailsHandler) DeleteGuardrailsCredential added in v0.260709.1 

func (h *GuardrailsHandler) DeleteGuardrailsCredential(c *gin.Context)

func (*GuardrailsHandler) DeleteGuardrailsGroup added in v0.260709.1 

func (h *GuardrailsHandler) DeleteGuardrailsGroup(c *gin.Context)

DeleteGuardrailsGroup deletes a group and reloads the engine.

func (*GuardrailsHandler) DeleteGuardrailsPolicy added in v0.260709.1 

func (h *GuardrailsHandler) DeleteGuardrailsPolicy(c *gin.Context)

DeleteGuardrailsPolicy deletes a policy and reloads the engine.

func (*GuardrailsHandler) ExportGuardrailsFragments added in v0.260709.1 

func (h *GuardrailsHandler) ExportGuardrailsFragments(c *gin.Context)

ExportGuardrailsFragments returns the raw imported fragment files selected by the user so the UI can download one or more source files directly.

func (*GuardrailsHandler) GetGuardrailsBuiltins added in v0.260709.1 

func (h *GuardrailsHandler) GetGuardrailsBuiltins(c *gin.Context)

GetGuardrailsBuiltins returns curated builtin policies for the Guardrails UI.

func (*GuardrailsHandler) GetGuardrailsConfig added in v0.260709.1 

func (h *GuardrailsHandler) GetGuardrailsConfig(c *gin.Context)

GetGuardrailsConfig returns the current guardrails config file content and parsed config.

func (*GuardrailsHandler) GetGuardrailsCredential added in v0.260709.1 

func (h *GuardrailsHandler) GetGuardrailsCredential(c *gin.Context)

GetGuardrailsCredential returns a single protected credential, including the current secret, for the local editor dialog.

func (*GuardrailsHandler) GetGuardrailsCredentials added in v0.260709.1 

func (h *GuardrailsHandler) GetGuardrailsCredentials(c *gin.Context)

Credential list responses intentionally mask secrets; the edit dialog uses GetGuardrailsCredential when it needs the underlying value. GetGuardrailsCredentials returns protected credentials without exposing raw secrets.

func (*GuardrailsHandler) GetGuardrailsHistory added in v0.260709.1 

func (h *GuardrailsHandler) GetGuardrailsHistory(c *gin.Context)

GetGuardrailsHistory returns the most recent guardrails history rows.

func (*GuardrailsHandler) GetGuardrailsRegistry added in v0.260709.1 

func (h *GuardrailsHandler) GetGuardrailsRegistry(c *gin.Context)

GetGuardrailsRegistry lists downloadable policies from a remote registry.

func (*GuardrailsHandler) ImportGuardrailsFragment added in v0.260709.1 

func (h *GuardrailsHandler) ImportGuardrailsFragment(c *gin.Context)

ImportGuardrailsFragment appends one or more policies from a fragment file into guardrails/custom/import.yaml and ensures the root config imports it.

func (*GuardrailsHandler) InstallGuardrailsRegistryPolicy added in v0.260709.1 

func (h *GuardrailsHandler) InstallGuardrailsRegistryPolicy(c *gin.Context)

InstallGuardrailsRegistryPolicy downloads a remote policy fragment into guardrails/remote and wires it into root imports.

func (*GuardrailsHandler) ReloadGuardrailsConfig added in v0.260709.1 

func (h *GuardrailsHandler) ReloadGuardrailsConfig(c *gin.Context)

ReloadGuardrailsConfig reloads guardrails from disk and rebuilds the runtime.

func (*GuardrailsHandler) UpdateGuardrailsConfig added in v0.260709.1 

func (h *GuardrailsHandler) UpdateGuardrailsConfig(c *gin.Context)

UpdateGuardrailsConfig saves a new guardrails config and reloads the engine.

func (*GuardrailsHandler) UpdateGuardrailsCredential added in v0.260709.1 

func (h *GuardrailsHandler) UpdateGuardrailsCredential(c *gin.Context)

func (*GuardrailsHandler) UpdateGuardrailsGroup added in v0.260709.1 

func (h *GuardrailsHandler) UpdateGuardrailsGroup(c *gin.Context)

UpdateGuardrailsGroup updates a single group and reloads the engine.

func (*GuardrailsHandler) UpdateGuardrailsPolicy added in v0.260709.1 

func (h *GuardrailsHandler) UpdateGuardrailsPolicy(c *gin.Context)

UpdateGuardrailsPolicy updates a single policy and reloads the engine.

type GuardrailsRuntime added in v0.260709.1 

type GuardrailsRuntime interface {
	CurrentGuardrailsRuntime() *guardrails.Guardrails
	SetGuardrailsRuntime(runtime *guardrails.Guardrails, context string)
	GetGuardrailsSupportedScenarios() []string
	RefreshGuardrailsCredentialCacheOrWarn(context string)
}

GuardrailsRuntime is the narrow slice of the root server's guardrails runtime state (internal/server.guardrails_runtime.go) that the admin surface needs: the current runtime snapshot, the ability to swap it after a config edit, and the small set of gating/derived helpers. Declared as an interface — rather than depending on *server.Server — to avoid an import cycle, since root server already imports this webui package.

type HTTPTimeouts added in v0.260723.1 

type HTTPTimeouts struct {
	ReadHeaderTimeout time.Duration
	ReadTimeout       time.Duration
	WriteTimeout      time.Duration
	IdleTimeout       time.Duration
}

HTTPTimeouts overrides the timeouts Start() arms on the underlying http.Server. Zero fields keep Start()'s hardcoded default for that field — see WithHTTPTimeouts.

type HistoryResponse 

type HistoryResponse struct {
	Success bool        `json:"success" example:"true"`
	Data    interface{} `json:"data"`
}

HistoryResponse represents the response for request history

type IncomingAPIType added in v0.260531.1 

type IncomingAPIType string

IncomingAPIType describes which OpenAI-style endpoint the client originally hit on this gateway. Only consulted when the provider declares EndpointModeBoth; otherwise the provider's declared mode dictates the upstream endpoint regardless of what the client sent. 

const (
	IncomingAPIChat      IncomingAPIType = "chat"
	IncomingAPIResponses IncomingAPIType = "responses"
)

type LBSimulator added in v0.260625.1 

type LBSimulator struct {
	// contains filtered or unexported fields
}

LBSimulator drives the real load-balancing path — routing.ServiceSelector.Select (health → smart → affinity → strategy) followed by dispatchWithPriorityFailover — against programmable fake upstreams over a request sequence, with a deterministic breaker clock.

It is the shared engine behind both the Go scenario tests (internal/server/lb_scenario_test.go) and the `harness lb` CLI tier, so the "how an LB scenario is simulated" logic lives in exactly one place. It lives in package server because it must reach the unexported failover dispatch loop.

func NewLBSimulator added in v0.260625.1 

func NewLBSimulator(rule *typ.Rule, faults map[string][]int) (sim *LBSimulator, cleanup func(), err error)

NewLBSimulator builds the real selection + dispatch stack for rule against a throwaway config, registering one provider per distinct service provider. The faults map keys are serviceIDs (loadbalance.Service.ServiceID(), i.e. "provider/model"); each value is a per-call status sequence (last repeats). Services without a fault entry always return 200.

This is the status-list shorthand over NewLBSimulatorWithSequences: each []int becomes a vmodel.Sequence with ExhaustClamp, preserving the historical "last entry repeats" semantics. Callers that need repeats or a loop/fail exhaustion policy build the SequenceConfig form and call NewLBSimulatorWithSequences directly.

func NewLBSimulatorWithSequences added in v0.260723.1 

func NewLBSimulatorWithSequences(rule *typ.Rule, faults map[string]vmodel.SequenceConfig) (sim *LBSimulator, cleanup func(), err error)

NewLBSimulatorWithSequences is NewLBSimulator's richer form: each fault is a full vmodel.SequenceConfig, so scenarios can express repeats and a loop/clamp/fail exhaustion policy rather than only a clamped status list. Only each step's HTTP status affects LB/failover decisions.

It installs a deterministic breaker clock; the returned cleanup restores the real clock and removes the temp config dir, and must be called.

func (*LBSimulator) Advance added in v0.260625.1 

func (s *LBSimulator) Advance(d time.Duration)

Advance moves the deterministic breaker clock forward, e.g. past OpenDuration to drive a half-open recovery probe.

func (*LBSimulator) BreakerStates added in v0.260625.1 

func (s *LBSimulator) BreakerStates() map[string]string

BreakerStates returns a snapshot of every rule service's breaker state, keyed by serviceID (values: "closed" / "open" / "half_open"). The breaker store is rule-scoped, so reads key on s.rule.UUID; the returned map stays serviceID-keyed (a consumer-facing contract used by scenario tests + the harness CLI).

func (*LBSimulator) HealthStates added in v0.260625.1 

func (s *LBSimulator) HealthStates() map[string]string

HealthStates returns a snapshot of every rule service's health-monitor state, keyed by serviceID (values: "healthy" / "unhealthy"). This is the channel fed by the special status codes (429 → rate-limit, 401/403 → auth), separate from the breaker.

func (*LBSimulator) Pin added in v0.260625.1 

func (s *LBSimulator) Pin(session string) string

Pin returns the serviceID the session is currently affinity-locked to ("" if none).

func (*LBSimulator) PinDetail added in v0.260625.1 

func (s *LBSimulator) PinDetail(session string) (serviceID string, lockedAt, expiresAt time.Time, ok bool)

PinDetail returns the current (non-expired) affinity lock for a session: the serviceID and its LockedAt/ExpiresAt timestamps (on the simulator's fake clock). ok is false when there is no live lock. It reads through the store's strict-TTL Get, so an expired lock reports ok=false — exactly what selection sees. Used to assert strict (non-sliding) TTL: an unrefreshed lock keeps its original timestamps, and a re-lock after expiry carries a fresh LockedAt.

func (*LBSimulator) Request added in v0.260625.1 

func (s *LBSimulator) Request(session string) (LBTrace, error)

Request runs one request for the given session (empty = no affinity) through the real selection + failover path, returning the trace.

func (*LBSimulator) SeedPin added in v0.260625.1 

func (s *LBSimulator) SeedPin(session, provider, model string)

SeedPin manually locks a session to a service (e.g. to reproduce a stale pin).

type LBTrace added in v0.260625.1 

type LBTrace struct {
	Session     string   `json:"session"`
	Attempts    []string `json:"attempts"`     // serviceIDs attempted, in order (failover hops)
	Statuses    []int    `json:"statuses"`     // per-attempt status, parallel to Attempts
	FinalStatus int      `json:"final_status"` // status the client would see
	PinAfter    string   `json:"pin_after"`    // affinity pin after this request ("" if none)
	// State snapshots taken AFTER this request, keyed by serviceID.
	BreakerAfter map[string]string `json:"breaker_after"` // closed/open/half_open
	HealthAfter  map[string]string `json:"health_after"`  // healthy/unhealthy
}

LBTrace is the record of one simulated request.

type LoadBalancer 

type LoadBalancer struct {
	// contains filtered or unexported fields
}

LoadBalancer is the tactic-selection engine: it narrows a rule's services to the healthy active set and delegates the final pick to the rule's configured tactic (rule.LBTactic.Instantiate()). It also serves as the stats/admin surface consumed by LoadBalancerAPI.

func NewLoadBalancer 

func NewLoadBalancer(cfg *config.Config, healthFilter *typ.HealthFilter) *LoadBalancer

NewLoadBalancer creates a new load balancer

func (*LoadBalancer) ClearAllStats 

func (lb *LoadBalancer) ClearAllStats()

ClearAllStats clears all statistics (both in-memory and persisted in config)

func (*LoadBalancer) ClearServiceStats 

func (lb *LoadBalancer) ClearServiceStats(provider, model string)

ClearServiceStats clears statistics for a specific provider:model — both the persisted stats store and the in-memory ServiceStats on every active rule service that matches (stats are global per provider:model, shared across rules). The previous implementation only touched an internal map that was never populated, so the clear was a no-op.

func (*LoadBalancer) GetAllServiceStats 

func (lb *LoadBalancer) GetAllServiceStats() map[string]*loadbalance.ServiceStats

GetAllServiceStats returns all service statistics from all active rules. Stats are keyed by provider:model since stats are global (shared across rules).

func (*LoadBalancer) GetRuleSummary 

func (lb *LoadBalancer) GetRuleSummary(rule *typ.Rule) map[string]interface{}

GetRuleSummary returns a summary of rule configuration and statistics

func (*LoadBalancer) GetServiceStats 

func (lb *LoadBalancer) GetServiceStats(provider, model string) *loadbalance.ServiceStats

GetServiceStats returns statistics for a specific service

func (*LoadBalancer) HealthFilter 

func (lb *LoadBalancer) HealthFilter() *typ.HealthFilter

HealthFilter returns the health filter for the load balancer

func (*LoadBalancer) PreviewService added in v0.260716.1 

func (lb *LoadBalancer) PreviewService(rule *typ.Rule) (*loadbalance.Service, error)

PreviewService selects exactly like SelectService but never claims a breaker probe slot. Read-only surfaces (the admin current-service preview) must use it — they never dispatch, so a claimed half-open probe would get no recorded outcome and would block real traffic from probing the recovering service until the stale-probe reclaim kicks in.

func (*LoadBalancer) SelectService 

func (lb *LoadBalancer) SelectService(rule *typ.Rule) (*loadbalance.Service, error)

SelectService selects the best service for a rule based on the configured tactic, claiming the picked service's breaker probe slot (the dispatch path records the outcome, releasing it).

type LoadBalancerAPI 

type LoadBalancerAPI struct {
	// contains filtered or unexported fields
}

LoadBalancerAPI provides REST endpoints for load balancer management

func NewLoadBalancerAPI 

func NewLoadBalancerAPI(loadBalancer LoadBalancerEngine, cfg *config.Config) *LoadBalancerAPI

NewLoadBalancerAPI creates a new load balancer API

func (*LoadBalancerAPI) ClearAllStats 

func (api *LoadBalancerAPI) ClearAllStats(c *gin.Context)

ClearAllStats clears all statistics

func (*LoadBalancerAPI) ClearRuleStats 

func (api *LoadBalancerAPI) ClearRuleStats(c *gin.Context)

ClearRuleStats clears statistics for all services in a rule

func (*LoadBalancerAPI) ClearServiceStats 

func (api *LoadBalancerAPI) ClearServiceStats(c *gin.Context)

ClearServiceStats clears statistics for a specific service

func (*LoadBalancerAPI) GetAllStats 

func (api *LoadBalancerAPI) GetAllStats(c *gin.Context)

GetAllStats returns statistics for all services

func (*LoadBalancerAPI) GetCurrentService 

func (api *LoadBalancerAPI) GetCurrentService(c *gin.Context)

GetCurrentService returns the currently active service for a rule

func (*LoadBalancerAPI) GetRule 

func (api *LoadBalancerAPI) GetRule(c *gin.Context)

GetRule returns a specific rule configuration

func (*LoadBalancerAPI) GetRuleStats 

func (api *LoadBalancerAPI) GetRuleStats(c *gin.Context)

GetRuleStats returns statistics for all services in a rule

func (*LoadBalancerAPI) GetRuleSummary 

func (api *LoadBalancerAPI) GetRuleSummary(c *gin.Context)

GetRuleSummary returns a comprehensive summary of a rule including statistics

func (*LoadBalancerAPI) GetServiceStats 

func (api *LoadBalancerAPI) GetServiceStats(c *gin.Context)

GetServiceStats returns statistics for a specific service

func (*LoadBalancerAPI) GetServicesHealth 

func (api *LoadBalancerAPI) GetServicesHealth(c *gin.Context)

GetServicesHealth returns health status for all services in a rule

func (*LoadBalancerAPI) RegisterRoutes 

func (api *LoadBalancerAPI) RegisterRoutes(loadBalancer *gin.RouterGroup)

RegisterRoutes registers the load balancer API routes

func (*LoadBalancerAPI) ResetServiceHealth 

func (api *LoadBalancerAPI) ResetServiceHealth(c *gin.Context)

ResetServiceHealth manually resets a service's health to healthy

func (*LoadBalancerAPI) UpdateRuleTactic 

func (api *LoadBalancerAPI) UpdateRuleTactic(c *gin.Context)

UpdateRuleTactic updates the load balancing tactic for a rule without resubmitting the whole rule. The tactic name is validated strictly (unknown names are rejected, not silently degraded) and the params decode through Tactic.UnmarshalJSON — the SAME polymorphic path a full rule save uses — so this partial update cannot drift from the canonical parser.

type LoadBalancerEngine added in v0.260709.1 

type LoadBalancerEngine interface {
	// PreviewService is the side-effect-free selection used by read-only
	// endpoints: unlike SelectService it never claims a breaker probe slot.
	PreviewService(rule *typ.Rule) (*loadbalance.Service, error)
	GetServiceStats(provider, model string) *loadbalance.ServiceStats
	GetAllServiceStats() map[string]*loadbalance.ServiceStats
	ClearServiceStats(provider, model string)
	ClearAllStats()
	GetRuleSummary(rule *typ.Rule) map[string]interface{}
	HealthFilter() *typ.HealthFilter
}

LoadBalancerEngine is the narrow slice of the AI Model API's load-balancer engine (internal/server(aimodel).LoadBalancer) that the admin REST surface needs. Declared as an interface here — rather than importing the concrete type — to avoid an import cycle, since the root server package already imports this webui package for static-asset wiring.

type LogEntry 

type LogEntry struct {
	Time    time.Time              `json:"time"`
	Level   string                 `json:"level"`
	Message string                 `json:"message"`
	Data    map[string]interface{} `json:"data,omitempty"`
	Fields  map[string]interface{} `json:"fields,omitempty"`
}

LogEntry represents a log entry for API response

type LogsResponse 

type LogsResponse struct {
	Total int        `json:"total"`
	Logs  []LogEntry `json:"logs"`
}

LogsResponse represents the API response for logs

type MetricsData 

type MetricsData struct {
	InputTokens  int     // Number of input/prompt tokens
	OutputTokens int     // Number of output/completion tokens
	LatencyMs    int64   // Total request latency in milliseconds
	TTFTMs       int64   // Time To First Token in milliseconds (0 if not available/applicable)
	CacheHit     bool    // Whether this request hit the cache
	TPS          float64 // Tokens Per Second - generation speed (0 for non-streaming requests)
}

MetricsData encapsulates all metrics collected for a request. This structure is used to pass comprehensive metrics to updateServiceStats without requiring frequent function signature changes.

type ModelCapabilities added in v0.260611.1 

type ModelCapabilities struct {
	Batch             CapabilitySupport            `json:"batch"`
	Citations         CapabilitySupport            `json:"citations"`
	CodeExecution     CapabilitySupport            `json:"code_execution"`
	ContextManagement *ContextManagementCapability `json:"context_management,omitempty"`
	Effort            *EffortCapability            `json:"effort,omitempty"`
	ImageInput        CapabilitySupport            `json:"image_input"`
	PDFInput          CapabilitySupport            `json:"pdf_input"`
	StructuredOutputs CapabilitySupport            `json:"structured_outputs"`
	Thinking          *ThinkingCapability          `json:"thinking,omitempty"`
}

ModelCapabilities maps to Anthropic's ModelCapabilities in /v1/models.

type ModelRequestDetail added in v0.260604.1 

type ModelRequestDetail struct {
	ModelRequestSummary
	Events []ModelRequestEvent `json:"events"`
}

ModelRequestDetail is a summary plus the full, time-ordered event timeline.

type ModelRequestEvent added in v0.260604.1 

type ModelRequestEvent struct {
	Time    time.Time              `json:"time"`
	Source  string                 `json:"source"`
	Level   string                 `json:"level"`
	Stage   string                 `json:"stage,omitempty"`
	Message string                 `json:"message"`
	Fields  map[string]interface{} `json:"fields,omitempty"`
}

ModelRequestEvent is a single log line belonging to one model request, regardless of which pipeline stage emitted it (HTTP envelope, protocol conversion / upstream client call, or smart-routing evaluation).

type ModelRequestSummary added in v0.260604.1 

type ModelRequestSummary struct {
	RequestID    string    `json:"request_id"`
	Time         time.Time `json:"time"`
	Scenario     string    `json:"scenario,omitempty"`
	RequestModel string    `json:"request_model,omitempty"`
	RoutedModel  string    `json:"routed_model,omitempty"`
	Provider     string    `json:"provider,omitempty"`
	Method       string    `json:"method,omitempty"`
	Path         string    `json:"path,omitempty"`
	Status       int       `json:"status,omitempty"`
	LatencyMs    int64     `json:"latency_ms,omitempty"`
	HasError     bool      `json:"has_error"`
	MaxLevel     string    `json:"max_level,omitempty"`
	EventCount   int       `json:"event_count"`

	// Failover visibility: how many failover hops this request took and the
	// service path it walked ("prov-a/model-a → prov-b/model-b"). Zero/empty
	// when the first attempt served the request. Derived from the failover
	// loop's stage=failover_retry events, so the list view can answer "did
	// this request fail over, and to where" without opening the timeline.
	FailoverHops int    `json:"failover_hops,omitempty"`
	FailoverPath string `json:"failover_path,omitempty"`
}

ModelRequestSummary is the per-request row shown in the Requests view. It is derived by correlating every event that shares a request_id.

type ModelRequestsResponse added in v0.260604.1 

type ModelRequestsResponse struct {
	Total    int                   `json:"total"`
	Requests []ModelRequestSummary `json:"requests"`
}

ModelRequestsResponse is the list response for the Requests view.

type OpenAIChatCompletionResponse 

type OpenAIChatCompletionResponse struct {
	ID      string `json:"id" example:"chatcmpl-123"`
	Object  string `json:"object" example:"chat.completion"`
	Created int64  `json:"created" example:"1677652288"`
	Model   string `json:"model" example:"gpt-3.5-turbo"`
	Choices []struct {
		Index   int `json:"index" example:"0"`
		Message struct {
			Role    string `json:"role" example:"assistant"`
			Content string `json:"content" example:"Hello! How can I help you?"`
		} `json:"message"`
		FinishReason string `json:"finish_reason" example:"stop"`
	} `json:"choices"`
	Usage struct {
		PromptTokens     int `json:"prompt_tokens" example:"10"`
		CompletionTokens int `json:"completion_tokens" example:"20"`
		TotalTokens      int `json:"total_tokens" example:"30"`
	} `json:"usage"`
}

OpenAIChatCompletionResponse represents the OpenAI chat completion response

type OpenAIModel 

type OpenAIModel struct {
	ID          string `json:"id"`
	Object      string `json:"object"`
	Created     int64  `json:"created"`
	OwnedBy     string `json:"owned_by"`
	Description string `json:"description,omitempty"` // Model description
	Context     int    `json:"context,omitempty"`     // Max context window
	MaxOutput   int    `json:"max_output,omitempty"`  // Max output tokens
	// AuthType reflects the primary backing provider's auth type. It is
	// non-standard (OpenAI's models API has no such field) and consumed by
	// the tingly-box frontend to order model picker entries:
	// oauth -> api_key -> vmodel.
	AuthType string `json:"auth_type,omitempty"`
}

OpenAIModel represents a model in OpenAI's models API format

type OpenAIModelsResponse 

type OpenAIModelsResponse struct {
	Object string        `json:"object"`
	Data   []OpenAIModel `json:"data"`
}

OpenAIModelsResponse represents OpenAI's models API response format

type ProbeProviderResponse 

type ProbeProviderResponse struct {
	Success bool                             `json:"success" example:"true"`
	Error   *ErrorDetail                     `json:"error,omitempty"`
	Data    *probe.ProbeProviderResponseData `json:"data,omitempty"`
}

ProbeProviderResponse represents the response from provider probing. The wrapper stays here because it embeds *ErrorDetail (server's global error model). The Data shape lives in internal/probe.

type ProbeRequestDetail 

type ProbeRequestDetail struct {
	Messages    []map[string]interface{} `json:"messages"`
	Model       string                   `json:"model"`
	MaxTokens   int                      `json:"max_tokens"`
	Temperature float64                  `json:"temperature"`
	Provider    string                   `json:"provider"`
	Timestamp   string                   `json:"timestamp"`
}

ProbeRequestDetail represents the mock request data for probing

func NewMockRequest 

func NewMockRequest(provider, model string) ProbeRequestDetail

NewMockRequest creates a new mock request with default values

type ProbeResponse 

type ProbeResponse struct {
	Success bool               `json:"success"`
	Error   *ErrorDetail       `json:"error,omitempty"`
	Data    *ProbeResponseData `json:"data,omitempty"`
}

ProbeResponse represents the overall probe response

type ProbeResponseData 

type ProbeResponseData struct {
	Request     ProbeRequestDetail  `json:"request"`
	Response    ProbeResponseDetail `json:"response"`
	Usage       ProbeUsage          `json:"usage"`
	CurlCommand string              `json:"curl_command,omitempty"`
}

ProbeResponseData represents the response data structure

type ProbeResponseDetail 

type ProbeResponseDetail struct {
	Content      string `json:"content"`
	Model        string `json:"model"`
	Provider     string `json:"provider"`
	FinishReason string `json:"finish_reason"`
	Error        string `json:"error,omitempty"`
}

ProbeResponseDetail represents the API response

type ProbeUsage 

type ProbeUsage struct {
	PromptTokens     int `json:"prompt_tokens"`
	CompletionTokens int `json:"completion_tokens"`
	TotalTokens      int `json:"total_tokens"`
	TimeCost         int `json:"time_cost"`
}

ProbeUsage represents token usage information

type ProtocolHandler added in v0.260709.1 

type ProtocolHandler struct {
	// contains filtered or unexported fields
}

ProtocolHandler is the aggregate handler for the AI Model API. Individual method files (openai_*.go, anthropic_*.go, protocol_*.go, etc.) will be moved here in later steps and become methods on *ProtocolHandler.

func NewHandler added in v0.260709.1 

func NewHandler(deps ProtocolHandlerDeps) *ProtocolHandler

NewHandler constructs the AI Model API handler from its dependencies.

func (*ProtocolHandler) AnthropicCountTokens added in v0.260709.1 

func (ph *ProtocolHandler) AnthropicCountTokens(c *gin.Context)

AnthropicCountTokens handles Anthropic v1 count_tokens endpoint This is the entry point that delegates to the appropriate implementation (v1 or beta)

func (*ProtocolHandler) AnthropicListModelsForScenario added in v0.260709.1 

func (ph *ProtocolHandler) AnthropicListModelsForScenario(c *gin.Context, scenario typ.RuleScenario)

AnthropicListModelsForScenario handles scenario-scoped model listing for Anthropic format

func (*ProtocolHandler) AnthropicMessagesV1 added in v0.260709.1 

func (ph *ProtocolHandler) AnthropicMessagesV1(c *gin.Context, req *protocol.AnthropicMessagesRequest, requestModel string, responseModel string, rule *typ.Rule, provider *typ.Provider)

AnthropicMessagesV1 implements standard v1 messages API.

It runs the provider-independent prologue once, then drives the failover loop whose per-attempt callback re-runs the whole provider-dependent pipeline (pre-chain → guardrails → target resolution → transform → dispatch) against the candidate selected for that attempt. Because the transform is re-run per attempt, failover can rotate across heterogeneous API styles.

func (*ProtocolHandler) AnthropicMessagesV1Beta added in v0.260709.1 

func (ph *ProtocolHandler) AnthropicMessagesV1Beta(c *gin.Context, req *protocol.AnthropicBetaMessagesRequest, requestModel string, responseModel string, rule *typ.Rule, provider *typ.Provider)

AnthropicMessagesV1Beta implements beta messages API.

Like AnthropicMessagesV1, the provider-independent prologue runs once and the per-attempt callback re-runs the provider-dependent pipeline so failover can rotate across heterogeneous API styles.

func (*ProtocolHandler) CallMCPToolWithHooks added in v0.260709.1 

func (ph *ProtocolHandler) CallMCPToolWithHooks(ctx context.Context, toolName, arguments string, messages []map[string]any) (context.Context, coretool.ToolResult, error)

CallMCPToolWithHooks executes response-phase MCP servertool hooks before the runtime call. Returns updated context (with advisor quota decremented), result, and error.

func (*ProtocolHandler) DispatchChainResult added in v0.260709.1 

func (ph *ProtocolHandler) DispatchChainResult(
	c *gin.Context, reqCtx *transform.TransformContext,
	rule *typ.Rule, provider *typ.Provider,
	isStreaming bool, recorder *recording.ProtocolRecorder,
)

dispatchChainResult do request from source to target, and return upstream response from target to source

func (*ProtocolHandler) DispatchGenericAnthropicBetaNonStream added in v0.260709.1 

func (ph *ProtocolHandler) DispatchGenericAnthropicBetaNonStream(
	c *gin.Context,
	reqCtx *transform.TransformContext,
	rule *typ.Rule,
	provider *typ.Provider,
	recorder *recording.ProtocolRecorder,
)

DispatchGenericAnthropicBetaNonStream handles Aβ→Aβ non-streaming with generic processor

func (*ProtocolHandler) DispatchGenericAnthropicBetaStream added in v0.260709.1 

func (ph *ProtocolHandler) DispatchGenericAnthropicBetaStream(
	c *gin.Context,
	reqCtx *transform.TransformContext,
	rule *typ.Rule,
	provider *typ.Provider,
	recorder *recording.ProtocolRecorder,
)

DispatchGenericAnthropicBetaStream handles Aβ→Aβ streaming with generic interceptor

func (*ProtocolHandler) DispatchGenericOpenAIChatNonStream added in v0.260709.1 

func (ph *ProtocolHandler) DispatchGenericOpenAIChatNonStream(
	c *gin.Context,
	reqCtx *transform.TransformContext,
	rule *typ.Rule,
	provider *typ.Provider,
	recorder *recording.ProtocolRecorder,
)

DispatchGenericOpenAIChatNonStream handles O→O non-streaming with generic processor

func (*ProtocolHandler) DispatchGenericOpenAIChatStream added in v0.260709.1 

func (ph *ProtocolHandler) DispatchGenericOpenAIChatStream(
	c *gin.Context,
	reqCtx *transform.TransformContext,
	rule *typ.Rule,
	provider *typ.Provider,
	recorder *recording.ProtocolRecorder,
)

DispatchGenericOpenAIChatStream handles O→O streaming with generic interceptor

func (*ProtocolHandler) DispatchWithPriorityFailover added in v0.260709.1 

func (ph *ProtocolHandler) DispatchWithPriorityFailover(
	c *gin.Context,
	rule *typ.Rule,
	initialProvider *typ.Provider,
	initialModel string,
	attempt dispatchAttempt,
)

dispatchWithPriorityFailover runs `attempt` repeatedly, retrying on retryable buffered failures until either the gate commits (the stream's first real chunk reached the wire, retry impossible) or the candidate pool is exhausted (the last buffered error flushes on the deferred return).

Single-service requests bypass the gate entirely: with no fallback tier, failover is impossible and there is no reason to buffer.

func (*ProtocolHandler) EnsureProtocolRecorder added in v0.260709.1 

func (ph *ProtocolHandler) EnsureProtocolRecorder(c *gin.Context, scenario string, provider *typ.Provider, model string, mode obs.RecordMode, bs []byte) *recording.ProtocolRecorder

EnsureProtocolRecorder returns a ProtocolRecorder for the given scenario, reusing any recorder already stored in the gin context. Returns nil when recording is disabled (no sink) or the request body cannot be read.

GetOrCreateScenarioSink is a ProtocolHandlerDeps callback rather than a direct call because scenario sink lifecycle (creation, mutex, recordDir) still lives on root *Server — see ProtocolHandlerDeps.

func (*ProtocolHandler) FailAttemptSetup added in v0.260709.1 

func (ph *ProtocolHandler) FailAttemptSetup(c *gin.Context, err error)

failAttemptSetup reports an in-attempt setup failure — target/endpoint resolution, the pre-transform chain, or the transform itself — that happens before any upstream call. It always writes a 500-class status, which the failover gate buffers and treats as retryable, so the orchestrator advances to the next candidate (possibly a different API style) instead of terminating the whole request on one misconfigured provider. Genuine client errors are rejected in the prologue, before the gate is installed, so they remain non-retryable and reach the client unchanged.

func (*ProtocolHandler) HandleAnthropicListModels added in v0.260709.1 

func (ph *ProtocolHandler) HandleAnthropicListModels(c *gin.Context)

HandleAnthropicListModels handles Anthropic v1 models endpoint

func (*ProtocolHandler) HandleAnthropicMessages added in v0.260709.1 

func (ph *ProtocolHandler) HandleAnthropicMessages(c *gin.Context)

HandleAnthropicMessages handles Anthropic v1 messages API requests This is the entry point that delegates to the appropriate implementation (v1 or beta)

func (*ProtocolHandler) HandleOpenAIChatCompletions added in v0.260709.1 

func (ph *ProtocolHandler) HandleOpenAIChatCompletions(c *gin.Context)

HandleOpenAIChatCompletions handles OpenAI v1 chat completion requests

func (*ProtocolHandler) HandleOpenAIEmbeddings added in v0.260709.1 

func (ph *ProtocolHandler) HandleOpenAIEmbeddings(c *gin.Context)

HandleOpenAIEmbeddings serves OpenAI-compatible embedding requests.

The endpoint is exposed via the mixin route group, so any scenario whose descriptor declares TransportOpenAI or TransportEmbed can reach it. The canonical home is the dedicated `embed` scenario; `openai` scenario also works because its descriptor is extended with TransportEmbed.

func (*ProtocolHandler) HandleOpenAIImageGeneration added in v0.260709.1 

func (ph *ProtocolHandler) HandleOpenAIImageGeneration(c *gin.Context)

HandleOpenAIImageGeneration serves OpenAI-compatible image generation requests against the upstream POST /v1/images/generations endpoint. The request is forwarded as-is; tingly-box does not probe whether the upstream prefers the dedicated images endpoint or the Responses API — the caller chooses the surface and the corresponding tingly-box route.

Exposed via the mixin route group, so any scenario whose descriptor declares TransportImageGen (or TransportOpenAI as a mixin) can reach it. The canonical home is the dedicated `imagegen` scenario.

func (*ProtocolHandler) HandleOpenAIListModels added in v0.260709.1 

func (ph *ProtocolHandler) HandleOpenAIListModels(c *gin.Context)

HandleOpenAIListModels handles the /v1/models endpoint (OpenAI compatible)

func (*ProtocolHandler) HandleResponsesCreate added in v0.260709.1 

func (ph *ProtocolHandler) HandleResponsesCreate(c *gin.Context)

HandleResponsesCreate handles POST /v1/responses

func (*ProtocolHandler) HandleResponsesGet added in v0.260709.1 

func (ph *ProtocolHandler) HandleResponsesGet(c *gin.Context)

HandleResponsesGet handles GET /v1/responses/{id}

func (*ProtocolHandler) ListModelsByScenario added in v0.260709.1 

func (ph *ProtocolHandler) ListModelsByScenario(c *gin.Context)

ListModelsByScenario handles the /v1/models endpoint for scenario-based routing

func (*ProtocolHandler) NonstreamAnthropicV1 added in v0.260709.1 

func (ph *ProtocolHandler) NonstreamAnthropicV1(
	c *gin.Context,
	reqCtx *transform.TransformContext,
	rule *typ.Rule,
	provider *typ.Provider,
	recorder *recording.ProtocolRecorder,
)

NonstreamAnthropicV1 handles A→A non-streaming with generic processor

func (*ProtocolHandler) OpenAIChatCompletion added in v0.260709.1 

func (ph *ProtocolHandler) OpenAIChatCompletion(c *gin.Context, req *protocol.OpenAIChatCompletionRequest, responseModel string, provider *typ.Provider, scenarioType typ.RuleScenario, rule *typ.Rule)

OpenAIChatCompletion runs the provider-independent prologue once, then drives the failover loop whose per-attempt callback re-runs the provider-dependent pipeline (align → cap → target resolution → transform → dispatch) so failover can rotate across heterogeneous API styles.

func (*ProtocolHandler) OpenAIListModelsForScenario added in v0.260709.1 

func (ph *ProtocolHandler) OpenAIListModelsForScenario(c *gin.Context, scenario typ.RuleScenario)

OpenAIListModelsForScenario handles scenario-scoped model listing for OpenAI format

func (*ProtocolHandler) ResponsesCreate added in v0.260709.1 

func (ph *ProtocolHandler) ResponsesCreate(c *gin.Context, scenarioType typ.RuleScenario, provider *typ.Provider, rule *typ.Rule, req *protocol.ResponseCreateRequest, responseModel string, maxAllowed int)

ResponsesCreate runs the provider-independent prologue once, then drives the failover loop whose per-attempt callback re-runs the provider-dependent pipeline (target resolution → transform → dispatch) so failover can rotate across heterogeneous API styles.

func (*ProtocolHandler) RunGenericAnthropicBetaNonStream added in v0.260709.1 

func (ph *ProtocolHandler) RunGenericAnthropicBetaNonStream(
	ctx context.Context,
	provider *typ.Provider,
	req *anthropic.BetaMessageNewParams,
	recorder *recording.ProtocolRecorder,
) (*anthropic.BetaMessage, *mcp.TokenUsage, error)

func (*ProtocolHandler) RunGenericAnthropicV1NonStream added in v0.260709.1 

func (ph *ProtocolHandler) RunGenericAnthropicV1NonStream(
	ctx context.Context,
	provider *typ.Provider,
	req *anthropic.MessageNewParams,
	recorder *recording.ProtocolRecorder,
) (*anthropic.Message, *mcp.TokenUsage, error)

func (*ProtocolHandler) RunGenericOpenAIChatNonStream added in v0.260709.1 

func (ph *ProtocolHandler) RunGenericOpenAIChatNonStream(
	ctx context.Context,
	provider *typ.Provider,
	req *openai.ChatCompletionNewParams,
	recorder *recording.ProtocolRecorder,
) (*openai.ChatCompletion, *mcp.TokenUsage, error)

func (*ProtocolHandler) StreamAnthropicBeta added in v0.260709.1 

func (ph *ProtocolHandler) StreamAnthropicBeta(c *gin.Context, req *anthropic.BetaMessageNewParams, streamResp *anthropicstream.Stream[anthropic.BetaRawMessageStreamEventUnion], actualModel string, responseModel string, provider *typ.Provider, recorder *recording.ProtocolRecorder)

StreamAnthropicBeta processes the Anthropic beta streaming response. The resolved model is passed in as actualModel rather than read from the request, so the handler no longer depends on req.Model.

func (*ProtocolHandler) StreamAnthropicBetaToOpenAIChatWithMCP added in v0.260709.1 

func (ph *ProtocolHandler) StreamAnthropicBetaToOpenAIChatWithMCP(
	c *gin.Context,
	provider *typ.Provider,
	req *anthropic.BetaMessageNewParams,
	actualModel string,
	responseModel string,
	disableStreamUsage bool,
	recorder *recording.ProtocolRecorder,
)

func (*ProtocolHandler) StreamAnthropicV1 added in v0.260709.1 

func (ph *ProtocolHandler) StreamAnthropicV1(
	c *gin.Context,
	reqCtx *transform.TransformContext,
	rule *typ.Rule,
	provider *typ.Provider,
	recorder *recording.ProtocolRecorder,
)

StreamAnthropicV1 handles A→A streaming with generic interceptor

func (*ProtocolHandler) StreamOpenAIChatToAnthropicBetaWithMCP added in v0.260709.1 

func (ph *ProtocolHandler) StreamOpenAIChatToAnthropicBetaWithMCP(
	c *gin.Context,
	provider *typ.Provider,
	req *openai.ChatCompletionNewParams,
	actualModel string,
	responseModel string,
	recorder *recording.ProtocolRecorder,
)

func (*ProtocolHandler) StreamOpenAIChatToAnthropicV1WithMCP added in v0.260709.1 

func (ph *ProtocolHandler) StreamOpenAIChatToAnthropicV1WithMCP(
	c *gin.Context,
	provider *typ.Provider,
	req *openai.ChatCompletionNewParams,
	actualModel string,
	responseModel string,
	recorder *recording.ProtocolRecorder,
)

func (*ProtocolHandler) TransformAnthropicBeta added in v0.260709.1 

func (ph *ProtocolHandler) TransformAnthropicBeta(c *gin.Context, req *protocol.AnthropicBetaMessagesRequest, target protocol.APIType, provider *typ.Provider, isStreaming bool, protocolRecorder *recording.ProtocolRecorder, scenarioType typ.RuleScenario, preBaseTransforms []transform.Transform, preVendorTransforms []transform.Transform) (*transform.TransformContext, error)

func (*ProtocolHandler) TransformAnthropicV1 added in v0.260709.1 

func (ph *ProtocolHandler) TransformAnthropicV1(c *gin.Context, req *protocol.AnthropicMessagesRequest, target protocol.APIType, provider *typ.Provider, isStreaming bool, protocolRecorder *recording.ProtocolRecorder, scenarioType typ.RuleScenario, preBaseTransforms []transform.Transform, preVendorTransforms []transform.Transform) (*transform.TransformContext, error)

func (*ProtocolHandler) TransformOpenAIChat added in v0.260709.1 

func (ph *ProtocolHandler) TransformOpenAIChat(c *gin.Context, req *protocol.OpenAIChatCompletionRequest, target protocol.APIType, provider *typ.Provider, isStreaming bool, protocolRecorder *recording.ProtocolRecorder, scenarioType typ.RuleScenario, preBaseTransforms []transform.Transform, preVendorTransforms []transform.Transform) (*transform.TransformContext, error)

func (*ProtocolHandler) TransformOpenAIResponses added in v0.260709.1 

func (ph *ProtocolHandler) TransformOpenAIResponses(c *gin.Context, req *protocol.ResponseCreateRequest, target protocol.APIType, provider *typ.Provider, isStreaming bool, protocolRecorder *recording.ProtocolRecorder, scenarioType typ.RuleScenario, maxAllowed int, preBaseTransforms []transform.Transform, preVendorTransforms []transform.Transform) (*transform.TransformContext, error)

type ProtocolHandlerDeps added in v0.260709.1 

type ProtocolHandlerDeps struct {
	Config *config.Config

	// TokenTracker records usage to the OTel meter pipeline (may be nil if
	// OTel setup failed at startup — callers must nil-check).
	TokenTracker *tracker.TokenTracker

	// HealthMonitor reports per-service health outcomes (success / rate
	// limit / auth error / general error) back into the load-balance health
	// filter (may be nil — callers must nil-check).
	HealthMonitor *loadbalance.HealthMonitor

	// ClientPool caches upstream provider clients (OpenAI/Anthropic/Google).
	ClientPool *client.ClientPool

	// LoadBalancer selects the active service for a rule (tier/random/etc
	// tactics). Used by failover to pick the next candidate on retry.
	LoadBalancer *LoadBalancer

	// TemplateManager resolves per-provider model metadata (max tokens,
	// description, context window) from the provider template catalog.
	TemplateManager *data.TemplateManager

	// RoutingSelector runs the full selection pipeline (health → smart →
	// affinity → strategy) for a scenario/rule/request, used by the top-level
	// OpenAI/Anthropic entry handlers (Step 10).
	RoutingSelector *routing.SimpleSelector

	// VisionProxyService rewrites image content into text descriptions for
	// scenarios/rules configured with a vision proxy plugin. Concrete type,
	// not a callback: internal/visionproxy.Service takes *config.Config
	// directly and has no *server.Server dependency.
	VisionProxyService *visionproxy.Service

	// MCPRuntime holds the virtual tool registry and advisor state for
	// external MCP tools invoked during a live model request.
	MCPRuntime *mcpruntime.Runtime

	// GetServertoolPipeline returns the current virtual-tool-provider pipeline
	// (advisor quota hooks, etc.). A callback rather than a plain field
	// because root's config hot-reload (registerAdviserFromConfig) reassigns
	// *Server.servertoolPipeline in place — a plain field copied at
	// NewHandler time would silently go stale on the next reload. May return
	// nil — callers fall back to servertool.NewDefaultExecutor.
	GetServertoolPipeline func() *servertool.Pipeline

	// The callbacks below reach back into root *Server state that has not
	// (yet) moved to aimodel: OTel usage-tracking wiring, the affinity
	// store, scenario recording sinks, and the guardrails runtime pointer.
	// Wiring them as funcs keeps this package independent of *server.Server
	// while letting already-moved gateway logic still reach that state.
	TrackUsageWithTokenUsage func(c *gin.Context, usage *protocol.TokenUsage, err error)
	TrackUsageFromContext    func(c *gin.Context, inputTokens, outputTokens int, err error)
	UpdateAffinityMessageID  func(c *gin.Context, rule *typ.Rule, messageID string)
	GetOrCreateScenarioSink  func(scenario typ.RuleScenario) *obs.Sink
	CurrentGuardrailsRuntime func() *guardrails.Guardrails

	// GetScenarioRecordMode resolves the effective recording mode for a
	// scenario. Backed by root's s.recordMode/s.scenarioRecordSinks, which
	// have not moved to aimodel (recording lifecycle stays root-owned).
	GetScenarioRecordMode func(scenario typ.RuleScenario) obs.RecordMode
}

ProtocolHandlerDeps declares exactly what the AI Model API handlers need from the host server. It is populated and passed in once, from server.NewServer, after all of *Server's fields have been constructed.

This grows as each subsequent migration step moves a file in and wires up the fields/methods it actually touches on *Server today.

type RequestConfig 

type RequestConfig struct {
	RequestModel  string `json:"request_model" example:"gpt-3.5-turbo"`
	ResponseModel string `json:"response_model" example:"gpt-3.5-turbo"`
	Provider      string `json:"provider" example:"openai"`
	DefaultModel  string `json:"default_model" example:"gpt-3.5-turbo"`
}

RequestConfig represents a request configuration in defaults response

type Server 

type Server struct {
	// contains filtered or unexported fields
}

Server represents the HTTP server

func GetGlobalServer 

func GetGlobalServer() *Server

GetGlobalServer gets the global server instance

func NewServer 

func NewServer(cfg *config.Config, opts ...ServerOption) *Server

NewServer creates a new HTTP server instance with functional options

func (*Server) Cancel added in v0.260414.2000 

func (s *Server) Cancel() context.CancelFunc

func (*Server) Context added in v0.260414.2000 

func (s *Server) Context() context.Context

func (*Server) CurrentGuardrailsRuntime added in v0.260709.1 

func (s *Server) CurrentGuardrailsRuntime() *guardrails.Guardrails

CurrentGuardrailsRuntime returns the active guardrails runtime snapshot.

func (*Server) EnsureProtocolRecorder added in v0.260531.1 

func (s *Server) EnsureProtocolRecorder(c *gin.Context, scenario string, provider *typ.Provider, model string, mode obs.RecordMode, bs []byte) *recording.ProtocolRecorder

EnsureProtocolRecorder delegates to the AI Model API handler, which owns the ProtocolRecorder type. Kept as a thin root wrapper since callers (anthropic_message.go and its tests) have not moved to aimodel yet.

func (*Server) GetGuardrailsSupportedScenarios added in v0.260709.1 

func (s *Server) GetGuardrailsSupportedScenarios() []string

GetGuardrailsSupportedScenarios returns the scenarios guardrails can gate.

func (*Server) GetLoadBalancer 

func (s *Server) GetLoadBalancer() *LoadBalancer

GetLoadBalancer returns the load balancer instance

func (*Server) GetOrCreateScenarioSink 

func (s *Server) GetOrCreateScenarioSink(scenario typ.RuleScenario) *obs.Sink

GetOrCreateScenarioSink gets or creates a recording sink for the specified scenario The sink is created on-demand and cached for subsequent use

func (*Server) GetRouter 

func (s *Server) GetRouter() *gin.Engine

GetRouter returns the Gin engine for testing purposes

func (*Server) GetScenarioRecordMode added in v0.260514.1 

func (s *Server) GetScenarioRecordMode(scenario typ.RuleScenario) obs.RecordMode

func (*Server) GetUserToken 

func (s *Server) GetUserToken(c *gin.Context)

GetUserToken returns the current user token (masked) Requires authentication

func (*Server) GetVirtualModelService added in v0.260716.1 

func (s *Server) GetVirtualModelService() *virtualserver.Service

GetVirtualModelService returns the in-process virtual-model service, so embedding callers (e.g. the duo harness child) can register additional virtual models before Start.

func (*Server) HealthMonitor 

func (s *Server) HealthMonitor() *loadbalance.HealthMonitor

HealthMonitor returns the server's health monitor

func (*Server) IsRemoteCoderRunning 

func (s *Server) IsRemoteCoderRunning() bool

IsRemoteCoderRunning returns whether the remote control service is running

func (*Server) RefreshGuardrailsCredentialCacheOrWarn added in v0.260709.1 

func (s *Server) RefreshGuardrailsCredentialCacheOrWarn(context string)

RefreshGuardrailsCredentialCacheOrWarn rebuilds the protected-credential cache, logging (rather than returning) any failure.

func (*Server) ResetModelToken 

func (s *Server) ResetModelToken(c *gin.Context)

ResetModelToken generates a new secure random model token and updates the configuration Requires authentication

func (*Server) ResetUserToken 

func (s *Server) ResetUserToken(c *gin.Context)

ResetUserToken generates a new secure random token and updates the configuration Requires authentication

func (*Server) SetGuardrailsRuntime added in v0.260709.1 

func (s *Server) SetGuardrailsRuntime(runtime *guardrails.Guardrails, context string)

SetGuardrailsRuntime swaps in a new guardrails runtime, preserving history and credential-cache state carried over from the previous runtime.

func (*Server) SetupAnthropicEndpoints 

func (s *Server) SetupAnthropicEndpoints(group *gin.RouterGroup)

func (*Server) SetupMixinEndpoints 

func (s *Server) SetupMixinEndpoints(group *gin.RouterGroup)

func (*Server) SetupOpenAIEndpoints 

func (s *Server) SetupOpenAIEndpoints(group *gin.RouterGroup)

func (*Server) Start 

func (s *Server) Start(port int) error

Start starts the HTTP server

func (*Server) StartDynamicCallbackServer 

func (s *Server) StartDynamicCallbackServer(sessionID string, port int) error

StartDynamicCallbackServer starts a temporary callback server for OAuth Implements CallbackServerManager interface for oauth module

func (*Server) StartRemoteCoder 

func (s *Server) StartRemoteCoder() error

StartRemoteCoder starts the remote control service if not already running

func (*Server) Stop 

func (s *Server) Stop(ctx context.Context) error

Stop gracefully stops the HTTP server

func (*Server) StopDynamicCallbackServer 

func (s *Server) StopDynamicCallbackServer(sessionID string)

StopDynamicCallbackServer stops a temporary callback server for OAuth Implements CallbackServerManager interface for oauth module

func (*Server) StopRemoteCoder 

func (s *Server) StopRemoteCoder()

StopRemoteCoder stops the remote control service if running

func (*Server) StopServer 

func (s *Server) StopServer(c *gin.Context)

func (*Server) SyncRemoteCoderBots 

func (s *Server) SyncRemoteCoderBots(ctx context.Context) error

SyncRemoteCoderBots syncs bots with the remote control bot manager

func (*Server) TrackUsage 

func (s *Server) TrackUsage(ctx context.Context, inputTokens, outputTokens int, err error)

TrackUsage implements the UsageTracker interface. It extracts the gin.Context from the provided context and calls trackUsageFromContext. The gin.Context should be stored in the context with the key "gin_context".

func (*Server) UsageStore 

func (s *Server) UsageStore() *db.UsageStore

UsageStore returns the server's usage store instance for internal integrations.

func (*Server) UseAIEndpoints 

func (s *Server) UseAIEndpoints()

func (*Server) UseLoadBalanceEndpoints 

func (s *Server) UseLoadBalanceEndpoints()

func (*Server) UseTokenManagementEndpoints added in v0.260418.2200 

func (s *Server) UseTokenManagementEndpoints()

UseTokenManagementEndpoints registers the token management API endpoints.

func (*Server) UseUIEndpoints 

func (s *Server) UseUIEndpoints(ctx context.Context)

Init sets up Server routes and templates on the main server engine

func (*Server) UseVirtualModelEndpoints 

func (s *Server) UseVirtualModelEndpoints()

UseVirtualModelEndpoints sets up the direct virtual-model entrypoints, split per protocol: 

/virtual/openai/v1/{models,chat/completions,responses}
/virtual/anthropic/v1/{models,messages}

These bypass the provider/rule/scenario pipeline and call the in-process handler directly — useful when a client wants a fixed URL pointed at the vmodel registry without configuring a provider. The protocol split ensures /models returns only the model IDs the chosen protocol can actually dispatch.

The canonical path for virtual models in normal use is still /v1/messages and /v1/chat/completions, where the dispatcher short-circuits to the same handler when it resolves to a vmodel provider (see HandleAnthropicMessages and HandleOpenAIChatCompletions).

func (*Server) UseWebAPIEndpoints added in v0.260709.1 

func (s *Server) UseWebAPIEndpoints(manager *swagger.RouteManager)

UseWebAPIEndpoints configures API routes for web UI using swagger manager

func (*Server) ValidateAuthToken 

func (s *Server) ValidateAuthToken(c *gin.Context)

ValidateAuthToken validates an authentication token without requiring auth This is used during login flow to verify a token before establishing session

type ServerActionResponse 

type ServerActionResponse struct {
	Success bool   `json:"success" example:"true"`
	Message string `json:"message" example:"Server stopped successfully"`
}

ServerActionResponse represents the response for server actions (start/stop/restart)

type ServerOption 

type ServerOption func(*Server)

ServerOption defines a functional option for Server configuration

func WithAuthMiddleware 

func WithAuthMiddleware(userAuth, modelAuth gin.HandlerFunc) ServerOption

WithAuthMiddleware sets custom auth middlewares for WebUI and Model API endpoints This allows TBE to inject its own JWT auth middleware instead of using tingly-box's default UserAuthMiddleware and ModelAuthMiddleware

Usage in TBE: 

server := NewServer(cfg,
    WithAuthMiddleware(tbeUserAuth, tbeModelAuth),
)

func WithDebug 

func WithDebug(enabled bool) ServerOption

WithDebug enables or disables debug mode for the server

func WithDefault 

func WithDefault() ServerOption

WithDefault applies all default server options

func WithGuardrails 

func WithGuardrails(runtime *guardrails.Guardrails) ServerOption

WithGuardrails sets a guardrails runtime for stream evaluation.

func WithHTTPTimeouts added in v0.260723.1 

func WithHTTPTimeouts(t HTTPTimeouts) ServerOption

WithHTTPTimeouts overrides the http.Server timeouts Start() otherwise hardcodes (ReadHeaderTimeout: 10s, ReadTimeout: 30s, WriteTimeout: 10m, IdleTimeout: 120s). Only non-zero fields in HTTPTimeouts are applied; the rest keep Start()'s defaults. Production callers have no reason to use this — it exists so tests can arm a real http.Server with a short WriteTimeout/ReadTimeout to exercise deadline-dependent behavior (e.g. ClearServerIOTimeouts, see internal/server/middleware/io_timeout_test.go) without hand-rolling a parallel http.Server outside the real Start() path.

func WithHost 

func WithHost(host string) ServerOption

func WithModelAuthMiddleware 

func WithModelAuthMiddleware(modelAuth gin.HandlerFunc) ServerOption

WithModelAuthMiddleware sets a custom model auth middleware for Model API endpoints Use this if you only want to replace ModelAuthMiddleware but keep UserAuthMiddleware

func WithMultiLogger 

func WithMultiLogger(logger *pkgobs.MultiLogger) ServerOption

WithMultiLogger sets the multi-mode logger for the server

func WithOpenBrowser 

func WithOpenBrowser(enabled bool) ServerOption

WithOpenBrowser enables or disables automatic browser opening

func WithRecordDir 

func WithRecordDir(dir string) ServerOption

WithRecordDir sets the scenario-level record directory

func WithRecordMode 

func WithRecordMode(mode obs.RecordMode) ServerOption

WithRecordMode sets the record mode for request/response recording mode: empty string = disabled, "all" = record all, "response" = response only, "scenario" = record scenario only

func WithRecording 

func WithRecording(enabled bool) ServerOption

WithRecording enables dual-stage recording for protocol conversion scenarios

func WithRecordingCAS added in v0.260514.1 

func WithRecordingCAS(enabled bool) ServerOption

WithRecordingCAS toggles content-addressed dedup alongside the default gzip recording. When enabled, each session is written twice: once as a gzip JSONL.gz (default), and once as content-addressed slim JSONL plus a per-record blob tree. Useful for cross-session prompt analysis and replay.

func WithTemplateManager added in v0.260409.1540 

func WithTemplateManager(tm *data.TemplateManager) ServerOption

WithTemplateManager allows TBE to inject a custom TemplateManager. This follows the same pattern as WithAuthMiddleware for consistency.

func WithUI 

func WithUI(enabled bool) ServerOption

WithUI enables or disables the UI for the server

func WithUserAuthMiddleware 

func WithUserAuthMiddleware(userAuth gin.HandlerFunc) ServerOption

WithUserAuthMiddleware sets a custom user auth middleware for WebUI endpoints Use this if you only want to replace UserAuthMiddleware but keep ModelAuthMiddleware

func WithVersion 

func WithVersion(version string) ServerOption

type ServiceHealthResponse 

type ServiceHealthResponse struct {
	Rule   string                 `json:"rule" example:"gpt-4"`
	Health map[string]interface{} `json:"health"`
}

ServiceHealthResponse represents the health check response for services

type StatusResponse 

type StatusResponse struct {
	Success bool `json:"success" example:"true"`
	Data    struct {
		ServerRunning    bool `json:"server_running" example:"true"`
		Port             int  `json:"port" example:"12580"`
		ProvidersTotal   int  `json:"providers_total" example:"3"`
		ProvidersEnabled int  `json:"providers_enabled" example:"2"`
		RequestCount     int  `json:"request_count" example:"100"`
	} `json:"data"`
}

StatusResponse represents the server status API response

type SystemLogEntry 

type SystemLogEntry struct {
	Time    time.Time              `json:"time"`
	Level   string                 `json:"level"`
	Message string                 `json:"message"`
	Fields  map[string]interface{} `json:"fields,omitempty"`
}

SystemLogEntry represents a system log entry for API response

type SystemLogLevelRequest 

type SystemLogLevelRequest struct {
	Level string `json:"level" binding:"required"`
}

SystemLogLevelRequest represents a request to set the log level

type SystemLogLevelResponse added in v0.260716.1 

type SystemLogLevelResponse struct {
	Message string `json:"message,omitempty"`
	Level   string `json:"level"`
}

type SystemLogsResponse 

type SystemLogsResponse struct {
	Total int              `json:"total"`
	Logs  []SystemLogEntry `json:"logs"`
}

SystemLogsResponse represents the API response for system logs

type ThinkingCapability added in v0.260611.1 

type ThinkingCapability struct {
	Supported bool           `json:"supported"`
	Types     *ThinkingTypes `json:"types,omitempty"`
}

ThinkingCapability describes thinking support.

type ThinkingTypes added in v0.260611.1 

type ThinkingTypes struct {
	Adaptive CapabilitySupport `json:"adaptive,omitempty"`
	Enabled  CapabilitySupport `json:"enabled,omitempty"`
}

ThinkingTypes describes supported thinking type configurations.

type TokenResponse 

type TokenResponse struct {
	Token string `json:"token" example:"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..."`
	Type  string `json:"type" example:"Bearer"`
}

TokenResponse represents the token response

type TransformRecorder added in v0.260531.1 

type TransformRecorder struct {
	// contains filtered or unexported fields
}

TransformRecorder is a transform.Transform that snapshots the request body at a given stage and stores it on a ProtocolRecorder.

func NewTransformRecorder added in v0.260531.1 

func NewTransformRecorder(c *gin.Context, recorder *recording.ProtocolRecorder, stage TransformStage) *TransformRecorder

NewTransformRecorder builds a recorder transform for the given stage.

func (*TransformRecorder) Apply added in v0.260531.1 

func (t *TransformRecorder) Apply(ctx *transform.TransformContext) error

func (*TransformRecorder) Name added in v0.260531.1 

func (t *TransformRecorder) Name() string

type TransformStage added in v0.260531.1 

type TransformStage int

TransformStage selects which side of the transform pipeline the recorder captures from. 

const (
	// StagePre captures ctx.OriginalRequest before any transformation is
	// applied.
	StagePre TransformStage = iota
	// StagePost captures ctx.Request after base transformation.
	StagePost
)

type WebDeps added in v0.260709.1 

type WebDeps struct {
	// MemoryLogMW backs the HTTP request log API (GetLogs/GetLogStats/ClearLogs).
	MemoryLogMW *middleware.MultiModeMemoryLogMiddleware

	// MultiLogger backs the system log, model-request trace and action
	// history APIs.
	MultiLogger *obs.MultiLogger

	// Config backs token generation/retrieval (model token persistence).
	Config *config.Config

	// JWTManager issues the JWT-backed model tokens.
	JWTManager *auth.JWTManager
}

WebDeps declares exactly what the WebUI Management API's control handlers need from the host server. It is populated and passed in once, from server.NewServer, after all of *Server's fields have been constructed.

This grows as each subsequent migration step moves a file in (server_control.go, guardrails_handler.go, etc.) and wires up the fields/methods it actually touches on *Server today.

type WebHandler added in v0.260709.1 

type WebHandler struct {
	// contains filtered or unexported fields
}

WebHandler is the aggregate handler for the WebUI Management API's server-control surface (status/start/stop, logs, guardrails admin, token management, etc). Individual method files will be moved here in later steps and become methods on *WebHandler.

func NewWebHandler added in v0.260709.1 

func NewWebHandler(deps WebDeps) *WebHandler

NewWebHandler constructs the WebUI control handler from its dependencies.

func (*WebHandler) ClearLogs added in v0.260709.1 

func (h *WebHandler) ClearLogs(c *gin.Context)

ClearLogs clears all log entries

func (*WebHandler) GenerateToken added in v0.260709.1 

func (h *WebHandler) GenerateToken(c *gin.Context)

GenerateToken handles token generation requests

func (*WebHandler) GetActionHistory added in v0.260709.1 

func (h *WebHandler) GetActionHistory(c *gin.Context)

GetActionHistory retrieves user action history from memory Query parameters:

  • limit: maximum number of recent entries to return (default: 100, max: 1000)

func (*WebHandler) GetActionStats added in v0.260709.1 

func (h *WebHandler) GetActionStats(c *gin.Context)

GetActionStats returns statistics about user actions

func (*WebHandler) GetHistory added in v0.260709.1 

func (h *WebHandler) GetHistory(c *gin.Context)

GetHistory returns request history from the action log.

func (*WebHandler) GetLogStats added in v0.260709.1 

func (h *WebHandler) GetLogStats(c *gin.Context)

GetLogStats returns statistics about the logs

func (*WebHandler) GetLogs added in v0.260709.1 

func (h *WebHandler) GetLogs(c *gin.Context)

GetLogs retrieves logs with optional filtering Query parameters:

  • limit: maximum number of entries to return (default: 100)
  • level: filter by log level (debug, info, warn, error)
  • since: RFC3339 timestamp to filter entries after this time

func (*WebHandler) GetModelRequestDetail added in v0.260709.1 

func (h *WebHandler) GetModelRequestDetail(c *gin.Context)

GetModelRequestDetail returns the full event timeline for a single request id.

func (*WebHandler) GetModelRequests added in v0.260709.1 

func (h *WebHandler) GetModelRequests(c *gin.Context)

GetModelRequests returns recent model requests, one row per correlation id, built by joining the HTTP access log, model_request stage logs and smart-routing traces from the in-memory sinks.

Query parameters:

  • limit: maximum number of requests to return (default: 100, max: 1000)
  • scenario / provider / status: optional exact-match filters

func (*WebHandler) GetStatus added in v0.260709.1 

func (h *WebHandler) GetStatus(c *gin.Context)

GetStatus returns server status and statistics.

func (*WebHandler) GetSystemLogLevel added in v0.260709.1 

func (h *WebHandler) GetSystemLogLevel(c *gin.Context)

GetSystemLogLevel returns the current system log level

func (*WebHandler) GetSystemLogStats added in v0.260709.1 

func (h *WebHandler) GetSystemLogStats(c *gin.Context)

GetSystemLogStats returns statistics about the system logs

func (*WebHandler) GetSystemLogs added in v0.260709.1 

func (h *WebHandler) GetSystemLogs(c *gin.Context)

GetSystemLogs retrieves system logs with optional filtering Query parameters:

  • limit: maximum number of recent entries to return (default: 100, max: 1000)

func (*WebHandler) GetToken added in v0.260709.1 

func (h *WebHandler) GetToken(c *gin.Context)

GetToken handles token retrieval requests - generates a token if it doesn't exist

func (*WebHandler) RestartServer added in v0.260709.1 

func (h *WebHandler) RestartServer(c *gin.Context)

RestartServer is a placeholder: restarting via the web UI is not supported.

func (*WebHandler) SetSystemLogLevel added in v0.260709.1 

func (h *WebHandler) SetSystemLogLevel(c *gin.Context)

SetSystemLogLevel sets the minimum log level for system logs

func (*WebHandler) StartServer added in v0.260709.1 

func (h *WebHandler) StartServer(c *gin.Context)

StartServer is a placeholder: starting the server via the web UI is not supported — the server itself must already be running to serve this request, so start would be a no-op even if implemented.

Source Files

View all Source files

Directories

Path Synopsis
Package guardrailspath holds filesystem-layout helpers for the guardrails config/storage directory.
 Package guardrailspath holds filesystem-layout helpers for the guardrails config/storage directory.
Package middleware provides Gin middleware for the tingly-box server.
 Package middleware provides Gin middleware for the tingly-box server.
module
codeximport
configapply
debug
Package debug exposes runtime memory diagnostics for a running instance: a memstats snapshot and a pprof heap profile.
 Package debug exposes runtime memory diagnostics for a running instance: a memstats snapshot and a pprof heap profile.
imbot
Package imbotsettings provides handlers for ImBot settings management.
 Package imbotsettings provides handlers for ImBot settings management.
info
Package versioncheck provides version lookup against the npm registry (with npmmirror as a China-mirror fallback) and semver-style comparison.
 Package versioncheck provides version lookup against the npm registry (with npmmirror as a China-mirror fallback) and semver-style comparison.
mcp
notify
Package notify is the HTTP front end for scenario plugin events.
 Package notify is the HTTP front end for scenario plugin events.
oauth
onboarding
probe
provider
Package provider handles CRUD and model-management HTTP endpoints for AI provider configurations.
 Package provider handles CRUD and model-management HTTP endpoints for AI provider configurations.
providerquota
providertemplate
rule
scenario
sharing
Package apitoken implements CRUD HTTP endpoints for shared API tokens.
 Package apitoken implements CRUD HTTP endpoints for shared API tokens.
skill
statusline
tokenrefresh
usage
virtualmodel
Package virtualmodel exposes management endpoints for the in-process virtual-model providers.
 Package virtualmodel exposes management endpoints for the in-process virtual-model providers.
Package recordingtest provides shared test helpers for exercising internal/server/recording's AttachRecorderHooks/ProtocolRecorder wiring through the production *server.ProtocolHandler entry points.
 Package recordingtest provides shared test helpers for exercising internal/server/recording's AttachRecorderHooks/ProtocolRecorder wiring through the production *server.ProtocolHandler entry points.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.