server

package
v0.260418.2200 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 18, 2026 License: MPL-2.0 Imports: 89 Imported by: 0

Documentation

Overview

Package server since we do refactoring and migrating step by step, some api names are not unified, this will be updated in future

Index

Constants

View Source 
const (
	// DefaultProbeTimeout is the default timeout for each endpoint probe
	DefaultProbeTimeout = 10 * time.Second
	// DefaultCacheTTL is the default time-to-live for cached probe results
	DefaultCacheTTL = 24 * time.Hour
)
View Source 
const (
	ExperimentalFeatureSmartCompact = "smart_compact"
	ExperimentalFeatureRecording    = "recording"
)

Experimental feature flag names

View Source 
const (
	ContextKeyRule           = "tracking_rule"             // *typ.Rule
	ContextKeyProvider       = "tracking_provider"         // *typ.Provider
	ContextKeyModel          = "tracking_model"            // string (actual model used)
	ContextKeyRequestModel   = "tracking_request_model"    // string (model requested by user)
	ContextKeyScenario       = "tracking_scenario"         // string (extracted from request path)
	ContextKeyStreamed       = "tracking_streamed"         // bool
	ContextKeyStartTime      = "tracking_start_time"       // time.Time
	ContextKeyFirstTokenTime = "tracking_first_token_time" // time.Time (for TTFT calculation)
	ContextKeyCacheHit       = "tracking_cache_hit"        // bool (cache hit status)
	ContextKeySessionID      = "tracking_session_id"       // string (resolved session ID for affinity)
)

Gin context keys for tracking metadata. These keys are used to store tracking information in the gin context to avoid explicit parameter passing throughout the handler chain.

View Source 
const GuardrailsRegistryGitHubURL = "https://raw.githubusercontent.com/tingly-dev/tingly-guardrails-registry/main/index.yaml"

Leave the remote registry unset until the dedicated policy repository is ready. The API will surface this as an unavailable registry instead of coupling guardrails downloads to the main code repository.

Variables

This section is empty.

Functions

func CalculateTPS 

func CalculateTPS(c *gin.Context, outputTokens int, streamed bool) float64

CalculateTPS calculates Tokens Per Second (generation speed) for streaming requests. For non-streaming requests or when TTFT is not available, returns 0.

TPS is calculated as: outputTokens / (currentTime - firstTokenTime) This measures the actual token generation speed after the first token was received.

Parameters:

  • c: Gin context containing timing information
  • outputTokens: Number of output tokens generated
  • streamed: Whether this was a streaming request

Returns:

  • TPS value (tokens per second), or 0 if not applicable

func CalculateTTFT 

func CalculateTTFT(c *gin.Context) int64

CalculateTTFT calculates Time To First Token in milliseconds. Returns TTFT if firstTokenTime is set, otherwise returns total latency.

func CleanBetaSystemMessages 

func CleanBetaSystemMessages(blocks []anthropic.BetaTextBlockParam) []anthropic.BetaTextBlockParam

CleanBetaSystemMessages removes billing header messages from beta system blocks

func CleanSystemMessages 

func CleanSystemMessages(blocks []anthropic.TextBlockParam) []anthropic.TextBlockParam

CleanSystemMessages removes billing header messages from system blocks This is used for Claude Code scenario to filter out injected billing headers

func ConvertAnthropicToOpenAIResponseWithProvider 

func ConvertAnthropicToOpenAIResponseWithProvider(anthropicResp *anthropic.BetaMessage, responseModel string, provider *typ.Provider, model string) map[string]interface{}

ConvertAnthropicToOpenAIResponseWithProvider converts an Anthropic response to OpenAI format and applies provider-specific transformations to the response

func FindGuardrailsConfig added in v0.260418.2200 

func FindGuardrailsConfig(configDir string) (string, error)

func ForwardAnthropicV1 

func ForwardAnthropicV1(fc *ForwardContext, wrapper *client.AnthropicClient, req *anthropic.MessageNewParams) (*anthropic.Message, context.CancelFunc, error)

ForwardAnthropicV1 sends a non-streaming Anthropic v1 message request.

func ForwardAnthropicV1Beta 

func ForwardAnthropicV1Beta(fc *ForwardContext, wrapper *client.AnthropicClient, req *anthropic.BetaMessageNewParams) (*anthropic.BetaMessage, context.CancelFunc, error)

ForwardAnthropicV1Beta sends a non-streaming Anthropic v1 beta message request.

func ForwardAnthropicV1BetaStream 

func ForwardAnthropicV1BetaStream(fc *ForwardContext, wrapper *client.AnthropicClient, req *anthropic.BetaMessageNewParams) (*anthropicstream.Stream[anthropic.BetaRawMessageStreamEventUnion], context.CancelFunc, error)

ForwardAnthropicV1BetaStream sends a streaming Anthropic v1 beta message request. Note: Set BaseCtx via WithBaseCtx() to support client cancellation.

func ForwardAnthropicV1Stream 

func ForwardAnthropicV1Stream(fc *ForwardContext, wrapper *client.AnthropicClient, req *anthropic.MessageNewParams) (*anthropicstream.Stream[anthropic.MessageStreamEventUnion], context.CancelFunc, error)

ForwardAnthropicV1Stream sends a streaming Anthropic v1 message request. Note: Set BaseCtx via WithBaseCtx() to support client cancellation.

func ForwardGoogle 

func ForwardGoogle(fc *ForwardContext, wrapper *client.GoogleClient, model string, contents []*genai.Content, config *genai.GenerateContentConfig) (*genai.GenerateContentResponse, context.CancelFunc, error)

ForwardGoogle sends a non-streaming Google Generative AI request.

func ForwardGoogleStream 

func ForwardGoogleStream(fc *ForwardContext, wrapper *client.GoogleClient, model string, contents []*genai.Content, config *genai.GenerateContentConfig) (iter.Seq2[*genai.GenerateContentResponse, error], context.CancelFunc, error)

ForwardGoogleStream sends a streaming Google Generative AI request. Note: Pass request context (c.Request.Context()) as baseCtx in NewForwardContext for client cancellation support.

func ForwardOpenAIChat 

func ForwardOpenAIChat(fc *ForwardContext, wrapper *client.OpenAIClient, req *openai.ChatCompletionNewParams) (*openai.ChatCompletion, context.CancelFunc, error)

ForwardOpenAIChat sends a non-streaming OpenAI chat completion request. IMPORTANT: All transformations (protocol conversion + vendor-specific) should be applied by the transform chain BEFORE calling this function.

func ForwardOpenAIChatStream 

func ForwardOpenAIChatStream(fc *ForwardContext, wrapper *client.OpenAIClient, req *openai.ChatCompletionNewParams) (*openaistream.Stream[openai.ChatCompletionChunk], context.CancelFunc, error)

ForwardOpenAIChatStream sends a streaming OpenAI chat completion request. IMPORTANT: All transformations (protocol conversion + vendor-specific) should be applied by the transform chain BEFORE calling this function. Note: Pass request context (c.Request.Context()) as baseCtx in NewForwardContext for client cancellation support.

func ForwardOpenAIResponses 

func ForwardOpenAIResponses(fc *ForwardContext, wrapper *client.OpenAIClient, params responses.ResponseNewParams) (*responses.Response, context.CancelFunc, error)

ForwardOpenAIResponses sends a non-streaming OpenAI Responses API request.

func ForwardOpenAIResponsesStream 

func ForwardOpenAIResponsesStream(fc *ForwardContext, wrapper *client.OpenAIClient, params responses.ResponseNewParams) (*openaistream.Stream[responses.ResponseStreamEventUnion], context.CancelFunc, error)

ForwardOpenAIResponsesStream sends a streaming OpenAI Responses API request. Note: Pass request context (c.Request.Context()) as baseCtx in NewForwardContext for client cancellation support.

func GenerateCurlCommand 

func GenerateCurlCommand(apiBase, apiStyle, token, model string) string

GenerateCurlCommand generates a curl command for testing the provider

func GenerateOpenAPI added in v0.260418.2200 

func GenerateOpenAPI(cfg *config.Config) (string, error)

GenerateOpenAPI creates an OpenAPI v3 schema without starting the server

func GetCacheHit 

func GetCacheHit(c *gin.Context) (bool, bool)

GetCacheHit retrieves the cache hit status from the context. Returns the cache hit status and true if it exists, false and false otherwise.

func GetFirstTokenTime 

func GetFirstTokenTime(c *gin.Context) (time.Time, bool)

GetFirstTokenTime retrieves the first token time from the context. Returns the timestamp and true if it exists, zero time and false otherwise.

func GetGuardrailsBuiltinDir added in v0.260418.2200 

func GetGuardrailsBuiltinDir(configDir string) string

func GetGuardrailsCacheDir added in v0.260418.2200 

func GetGuardrailsCacheDir(configDir string) string

func GetGuardrailsConfigPath added in v0.260418.2200 

func GetGuardrailsConfigPath(configDir string) string

func GetGuardrailsCustomDir added in v0.260418.2200 

func GetGuardrailsCustomDir(configDir string) string

func GetGuardrailsDBDir added in v0.260418.2200 

func GetGuardrailsDBDir(configDir string) string

func GetGuardrailsDBPath added in v0.260418.2200 

func GetGuardrailsDBPath(configDir string) string

func GetGuardrailsDir added in v0.260418.2200 

func GetGuardrailsDir(configDir string) string

Guardrails config and storage live under one dedicated subdirectory. Keeping these path helpers next to the handlers makes the admin surface self-contained.

func GetGuardrailsHistoryPath added in v0.260418.2200 

func GetGuardrailsHistoryPath(configDir string) string

func GetGuardrailsRegistryCachePath added in v0.260418.2200 

func GetGuardrailsRegistryCachePath(configDir string) string

func GetGuardrailsRemoteDir added in v0.260418.2200 

func GetGuardrailsRemoteDir(configDir string) string

func GetProbeToolChoiceAutoAnthropic 

func GetProbeToolChoiceAutoAnthropic() anthropic.ToolChoiceUnionParam

GetProbeToolChoiceAutoAnthropic returns auto tool choice for testing

func GetProbeToolChoiceOpenAI 

func GetProbeToolChoiceOpenAI() map[string]interface{}

GetProbeToolChoiceOpenAI returns auto tool choice for OpenAI (as JSON map)

func GetProbeToolsAnthropic 

func GetProbeToolsAnthropic() []anthropic.ToolUnionParam

GetProbeToolsAnthropic returns predefined tools in Anthropic format for probe testing

func GetProbeToolsOpenAI 

func GetProbeToolsOpenAI() []openai.ChatCompletionToolUnionParam

GetProbeToolsOpenAI returns predefined tools in OpenAI format (as JSON map)

func GetShutdownChannel 

func GetShutdownChannel() <-chan struct{}

GetShutdownChannel returns the shutdown channel for the main process to listen on

func GetTrackingContext 

func GetTrackingContext(c *gin.Context) (rule *typ.Rule, provider *typ.Provider, actualModel, requestModel, scenario string, streamed bool, startTime time.Time)

GetTrackingContext retrieves tracking metadata from the gin context. Returns zero values if the context keys are not set.

func GetUserIDFromContext added in v0.260418.2200 

func GetUserIDFromContext(c *gin.Context) string

GetUserIDFromContext extracts the user ID from gin context. Priority: user_uuid (from JWT API token) > enterprise_user_id (from enterprise JWT) > "" This ensures that JWT API token authentication takes precedence for user tracking.

func NewGinHandlerWrapper 

func NewGinHandlerWrapper(h gin.HandlerFunc) swagger.Handler

NewGinHandlerWrapper converts gin.HandlerFunc to swagger.Handler

func NewNonStreamRecorderHook 

func NewNonStreamRecorderHook(recorder *ScenarioRecorder, provider *typ.Provider, model string) func()

NewNonStreamRecorderHook creates a hook for non-streaming responses.

func NewRecorderHooks 

func NewRecorderHooks(recorder *ProtocolRecorder) (onStreamEvent func(event interface{}) error, onStreamComplete func(), onStreamError func(err error))

NewRecorderHooks creates hook functions from a ScenarioRecorder for use with HandleContext. This allows decoupling the recorder from the handle context while maintaining recording functionality. Usage is tracked internally in the event hook, so complete hooks don't need usage parameters.

Returns: - onStreamEvent: Hook for each stream event - onStreamComplete: Hook for stream completion - onStreamError: Hook for stream errors

func NewRecorderHooksWithModel 

func NewRecorderHooksWithModel(recorder *ProtocolRecorder, model string, provider *typ.Provider) (onStreamEvent func(event interface{}) error, onStreamComplete func(), onStreamError func(err error))

NewRecorderHooksWithModel creates hook functions with an explicit model parameter. This is preferred when the model is known at hook creation time. Usage is tracked internally in the event hook.

func RoundtripAnthropicBetaResponseViaOpenAI 

func RoundtripAnthropicBetaResponseViaOpenAI(anthropicResp *anthropic.BetaMessage, responseModel string, provider *typ.Provider, actualModel string) (*anthropic.BetaMessage, error)

func RoundtripOpenAIMapViaAnthropic 

func RoundtripOpenAIMapViaAnthropic(openaiResp map[string]interface{}, responseModel string, provider *typ.Provider, actualModel string) (map[string]interface{}, error)

func RoundtripOpenAIResponseViaAnthropic 

func RoundtripOpenAIResponseViaAnthropic(openaiResp *openai.ChatCompletion, responseModel string, provider *typ.Provider, actualModel string) (map[string]interface{}, error)

func SendErrorResponse added in v0.260418.2200 

func SendErrorResponse(c *gin.Context, statusCode int, err error, errType string)

sendErrorResponse registers the error into gin context for logging middleware and sends JSON response.

func SetCacheHit 

func SetCacheHit(c *gin.Context, isHit bool)

SetCacheHit records whether this request was a cache hit.

func SetEnterpriseRateLimitReporter 

func SetEnterpriseRateLimitReporter(reporter func(context.Context, string, string, string, string) error)

SetEnterpriseRateLimitReporter sets callback for enterprise 429 events.

func SetFirstTokenTime 

func SetFirstTokenTime(c *gin.Context)

SetFirstTokenTime records when the first token was received (for TTFT calculation). This should be called when the first chunk is received in streaming requests.

func SetGlobalServer 

func SetGlobalServer(server *Server)

SetGlobalServer sets the global server instance for web UI control

func SetTrackingContext 

func SetTrackingContext(c *gin.Context, rule *typ.Rule, provider *typ.Provider, actualModel, requestModel string, streamed bool)

SetTrackingContext sets all tracking metadata in the gin context. This should be called once at the beginning of request processing to avoid explicit parameter passing throughout the handler chain.

Parameters:

  • c: Gin context
  • rule: The load balancer rule that was selected
  • provider: The provider that was selected
  • actualModel: The actual model name used (may differ from requested)
  • requestModel: The original model name requested by the user
  • streamed: Whether this is a streaming request

func ShouldRoundtripResponse 

func ShouldRoundtripResponse(c *gin.Context, target string) bool

Types

type APITokenInfo added in v0.260418.2200 

type APITokenInfo struct {
	TokenID     string     `json:"token_id"`
	UserUUID    string     `json:"user_uuid"`
	DisplayName string     `json:"display_name"`
	Enabled     bool       `json:"enabled"`
	LastUsedAt  *time.Time `json:"last_used_at,omitempty"`
	CreatedAt   time.Time  `json:"created_at"`
	CreatedBy   string     `json:"created_by,omitempty"`
}

APITokenInfo represents information about an API token (without the actual token)

type ActionHistoryEntry 

type ActionHistoryEntry struct {
	Time    time.Time              `json:"time"`
	Level   string                 `json:"level"`
	Message string                 `json:"message"`
	Action  string                 `json:"action,omitempty"`
	Success bool                   `json:"success,omitempty"`
	Details interface{}            `json:"details,omitempty"`
	Fields  map[string]interface{} `json:"fields,omitempty"`
}

ActionHistoryEntry represents an action history entry for API response

type ActionHistoryResponse 

type ActionHistoryResponse struct {
	Total   int                  `json:"total"`
	Actions []ActionHistoryEntry `json:"actions"`
}

ActionHistoryResponse represents the API response for action history

type AdaptiveProbe 

type AdaptiveProbe struct {
	// contains filtered or unexported fields
}

AdaptiveProbe handles concurrent endpoint probing for model capabilities

func NewAdaptiveProbe 

func NewAdaptiveProbe(s *Server) *AdaptiveProbe

NewAdaptiveProbe creates a new adaptive probe instance

func (*AdaptiveProbe) GetModelCapability 

func (ap *AdaptiveProbe) GetModelCapability(providerUUID, modelID string) (*ModelEndpointCapability, error)

GetModelCapability retrieves cached capability for a model, or triggers a probe if not cached

func (*AdaptiveProbe) GetPreferredEndpoint 

func (ap *AdaptiveProbe) GetPreferredEndpoint(provider *typ.Provider, modelID string) string

GetPreferredEndpoint returns the preferred endpoint for a model

func (*AdaptiveProbe) InvalidateProviderCache 

func (ap *AdaptiveProbe) InvalidateProviderCache(providerUUID string)

InvalidateProviderCache invalidates all cached capabilities for a provider

func (*AdaptiveProbe) ProbeModelEndpoints 

func (ap *AdaptiveProbe) ProbeModelEndpoints(ctx context.Context, req ModelProbeRequest) (*ProbeResult, error)

ProbeModelEndpoints probes both chat and responses endpoints concurrently for a model

func (*AdaptiveProbe) ProbeProviderModels 

func (ap *AdaptiveProbe) ProbeProviderModels(ctx context.Context, provider *typ.Provider, models []string) map[string]*ProbeResult

ProbeProviderModels probes all models for a provider concurrently

type AffinityStore 

type AffinityStore struct {
	// contains filtered or unexported fields
}

AffinityStore maps (ruleUUID, sessionID) -> locked Service. It directly implements routing.AffinityStore interface.

func NewAffinityStore 

func NewAffinityStore(ttl time.Duration) *AffinityStore

NewAffinityStore creates a new affinity store with the given TTL

func (*AffinityStore) CountByService added in v0.260409.1540 

func (s *AffinityStore) CountByService(serviceID string) int

CountByService returns the number of sessions locked to the given serviceID that were created within the capacity active window (last 30 minutes). This provides a "recently active users" count for seat utilization calculation.

func (*AffinityStore) Delete 

func (s *AffinityStore) Delete(ruleUUID, sessionID string)

Delete removes an affinity entry for the given rule and session

func (*AffinityStore) DeleteByRule 

func (s *AffinityStore) DeleteByRule(ruleUUID string)

DeleteByRule removes all affinity entries for a given rule UUID

func (*AffinityStore) GC 

func (s *AffinityStore) GC()

GC removes expired entries from the store

func (*AffinityStore) Get 

func (s *AffinityStore) Get(ruleUUID, sessionID string) (*routing.AffinityEntry, bool)

Get retrieves an affinity entry for the given rule and session

func (*AffinityStore) Set 

func (s *AffinityStore) Set(ruleUUID, sessionID string, entry *routing.AffinityEntry)

Set stores an affinity entry for the given rule and session

func (*AffinityStore) StartGC 

func (s *AffinityStore) StartGC()

StartGC starts a background goroutine that periodically cleans up expired entries

func (*AffinityStore) UpdateMessageID 

func (s *AffinityStore) UpdateMessageID(ruleUUID, sessionID, messageID string)

UpdateMessageID updates the message ID for an existing affinity entry

type AllStatsResponse 

type AllStatsResponse struct {
	Stats map[string]interface{} `json:"stats"`
}

AllStatsResponse represents the response for all statistics

type AnthropicModel 

type AnthropicModel struct {
	ID          string `json:"id"`
	CreatedAt   string `json:"created_at"`
	DisplayName string `json:"display_name"`
	Type        string `json:"type"`
}

AnthropicModel Model types - based on Anthropic's official models API format

type AnthropicModelsResponse 

type AnthropicModelsResponse struct {
	Data    []AnthropicModel `json:"data"`
	FirstID string           `json:"first_id"`
	HasMore bool             `json:"has_more"`
	LastID  string           `json:"last_id"`
}

type CachedModelCapability 

type CachedModelCapability struct {
	Capability ModelEndpointCapability
	ExpiresAt  time.Time
}

CachedModelCapability represents a cached model endpoint capability with expiration

type CleanHeaderTransform 

type CleanHeaderTransform struct{}

CleanHeaderTransform strips injected billing header blocks from system messages. Used by Claude Code scenarios to remove upstream-injected x-anthropic-billing-header blocks. Only added to the chain when CleanHeader flag is true.

func NewCleanHeaderTransform 

func NewCleanHeaderTransform() *CleanHeaderTransform

NewCleanHeaderTransform creates a CleanHeaderTransform.

func (*CleanHeaderTransform) Apply 

func (t *CleanHeaderTransform) Apply(ctx *protocoltransform.TransformContext) error

func (*CleanHeaderTransform) Name 

func (t *CleanHeaderTransform) Name() string

type ClearStatsResponse 

type ClearStatsResponse struct {
	Message string `json:"message" example:"Statistics cleared for rule: gpt-4"`
}

ClearStatsResponse represents the response for clearing statistics

type ConfigInfo 

type ConfigInfo struct {
	ConfigPath string `json:"config_path" example:"/Users/user/.tingly-box/config.json"`
	ConfigDir  string `json:"config_dir" example:"/Users/user/.tingly-box"`
}

ConfigInfo represents configuration information

type ConfigInfoResponse 

type ConfigInfoResponse struct {
	Success bool       `json:"success" example:"true"`
	Data    ConfigInfo `json:"data"`
}

ConfigInfoResponse represents the response for config info endpoint

type CreateProviderRequest 

type CreateProviderRequest struct {
	Name          string `json:"name" binding:"required" description:"Provider name" example:"openai"`
	APIBase       string `json:"api_base" binding:"required" description:"API base URL" example:"https://api.openai.com/v1"`
	APIStyle      string `json:"api_style" description:"API style" example:"openai"`
	Token         string `json:"token" description:"API token" example:"sk-..."`
	NoKeyRequired bool   `json:"no_key_required" description:"Whether provider requires no API key" example:"false"`
	Enabled       bool   `json:"enabled" description:"Whether provider is enabled" example:"true"`
	ProxyURL      string `` /* 153-byte string literal not displayed */
	AuthType      string `json:"auth_type,omitempty" description:"Auth type: api_key or oauth (default: api_key)" example:"api_key"`
}

CreateProviderRequest represents the request to add a new provider

type CreateProviderResponse 

type CreateProviderResponse struct {
	Success bool        `json:"success" example:"true"`
	Message string      `json:"message" example:"Provider added successfully"`
	Data    interface{} `json:"data"`
}

CreateProviderResponse represents the response for adding a provider

type CurrentServiceResponse 

type CurrentServiceResponse struct {
	Rule      string                 `json:"rule" example:"gpt-4"`
	Service   interface{}            `json:"service"`
	ServiceID string                 `json:"service_id" example:"openai:gpt-4"`
	Tactic    string                 `json:"tactic" example:"adaptive"`
	Stats     map[string]interface{} `json:"stats,omitempty"`
}

CurrentServiceResponse represents the current service response

type DeleteProviderResponse 

type DeleteProviderResponse struct {
	Success bool   `json:"success" example:"true"`
	Message string `json:"message" example:"Provider deleted successfully"`
}

DeleteProviderResponse represents the response for deleting a provider

type EndpointProbeStatus 

type EndpointProbeStatus struct {
	Available    bool   `json:"available" example:"true"`
	LatencyMs    int    `json:"latency_ms" example:"234"`
	ErrorMessage string `json:"error_message,omitempty" example:""`
	LastChecked  string `json:"last_checked" example:"2026-01-23T10:30:00Z"`
}

EndpointProbeStatus represents the status of an endpoint probe

type EndpointStatus 

type EndpointStatus struct {
	Available    bool
	LatencyMs    int
	ErrorMessage string
	LastChecked  time.Time
}

EndpointStatus represents the status of a single endpoint

type ErrorDetail 

type ErrorDetail struct {
	Message string `json:"message"`
	Type    string `json:"type"`
	Code    string `json:"code,omitempty"`
}

ErrorDetail represents error details

type ErrorResponse 

type ErrorResponse struct {
	Error ErrorDetail `json:"error"`
}

ErrorResponse represents an error response

type FetchProviderModelsResponse 

type FetchProviderModelsResponse struct {
	Success bool        `json:"success" example:"true"`
	Message string      `json:"message" example:"Successfully fetched 150 models for provider openai"`
	Data    interface{} `json:"data"`
}

FetchProviderModelsResponse represents the response for fetching provider models

type ForwardContext 

type ForwardContext struct {
	// Required dependencies
	Provider *typ.Provider
	BaseCtx  context.Context // Base context (e.g., request context for cancellation support)

	// Optional configuration
	Timeout time.Duration

	// Hooks (chainable - multiple hooks can be added)
	BeforeRequestHooks []func(ctx context.Context, req interface{}) (context.Context, error)
	AfterRequestHooks  []func(ctx context.Context, resp interface{}, err error)
}

ForwardContext provides dependencies for forward functions. It uses the builder pattern for optional configuration and hooks.

func NewForwardContext 

func NewForwardContext(baseCtx context.Context, provider *typ.Provider) *ForwardContext

NewForwardContext creates a new ForwardContext with required dependencies. The timeout is set to the provider's default timeout. baseCtx is the base context for the request:

  • Use context.Background() for non-streaming requests
  • Use c.Request.Context() for streaming requests to support client cancellation

func (*ForwardContext) Complete 

func (fc *ForwardContext) Complete(ctx context.Context, resp interface{}, err error)

Complete calls all AfterRequest hooks (if set) with the response and error. Hooks are called in the order they were added. This should be called after the request completes, regardless of success or failure.

func (*ForwardContext) PrepareContext 

func (fc *ForwardContext) PrepareContext(req interface{}) (context.Context, context.CancelFunc)

PrepareContext prepares the final context for the request. It applies the BeforeRequest hooks and adds the scenario to the context. It also sets up the timeout and returns a cancel function. If BaseCtx is not set, it uses context.Background() as the base.

The order of operations matches the original implementation: 1. Apply BeforeRequest hooks 2. Add timeout

func (*ForwardContext) WithAfterRequest 

func (fc *ForwardContext) WithAfterRequest(hook func(context.Context, interface{}, error)) *ForwardContext

WithAfterRequest adds a hook that is called after the request completes. Multiple hooks can be added and will be called in order. Each hook receives the response and any error that occurred.

func (*ForwardContext) WithBeforeRequest 

func (fc *ForwardContext) WithBeforeRequest(hook func(context.Context, interface{}) (context.Context, error)) *ForwardContext

WithBeforeRequest adds a hook that is called before the request is sent. Multiple hooks can be added and will be called in order. Each hook can modify the context and return an error to abort the request.

func (*ForwardContext) WithTimeout 

func (fc *ForwardContext) WithTimeout(timeout time.Duration) *ForwardContext

WithTimeout sets the timeout for the request. If not set, the provider's default timeout is used.

type GenerateTokenRequest 

type GenerateTokenRequest struct {
	ClientID string `json:"client_id" binding:"required" description:"Client ID for token generation" example:"user123"`
}

GenerateTokenRequest represents the request to generate a token

type HealthInfoResponse 

type HealthInfoResponse struct {
	Status  string `json:"status" example:"healthy"`
	Service string `json:"service" example:"tingly-box"`
	Health  bool   `json:"health" example:"healthy"`
}

HealthInfoResponse represents the health check response

type HistoryResponse 

type HistoryResponse struct {
	Success bool        `json:"success" example:"true"`
	Data    interface{} `json:"data"`
}

HistoryResponse represents the response for request history

type LatestVersionInfo 

type LatestVersionInfo struct {
	CurrentVersion string `json:"current_version" example:"0.260124.1430"`
	LatestVersion  string `json:"latest_version" example:"0.260130.1200"`
	HasUpdate      bool   `json:"has_update" example:"true"`
	ReleaseURL     string `json:"release_url" example:"https://github.com/tingly-dev/tingly-box/releases/tag/v0.260130.1200"`
	ShouldNotify   bool   `json:"should_notify" example:"true"`
}

LatestVersionInfo contains version comparison information

type LatestVersionResponse 

type LatestVersionResponse struct {
	Success bool              `json:"success"`
	Error   string            `json:"error,omitempty"`
	Data    LatestVersionInfo `json:"data,omitempty"`
}

LatestVersionResponse represents the response for version check endpoint

type LoadBalancer 

type LoadBalancer struct {
	// contains filtered or unexported fields
}

LoadBalancer manages load balancing across multiple services

func NewLoadBalancer 

func NewLoadBalancer(cfg *config.Config, healthFilter *typ.HealthFilter) *LoadBalancer

NewLoadBalancer creates a new load balancer

func (*LoadBalancer) ClearAllStats 

func (lb *LoadBalancer) ClearAllStats()

ClearAllStats clears all statistics (both in-memory and persisted in config)

func (*LoadBalancer) ClearServiceStats 

func (lb *LoadBalancer) ClearServiceStats(provider, model string)

ClearServiceStats clears statistics for a specific service

func (*LoadBalancer) GetAllServiceStats 

func (lb *LoadBalancer) GetAllServiceStats() map[string]*loadbalance.ServiceStats

GetAllServiceStats returns all service statistics from all active rules. Stats are keyed by provider:model since stats are global (shared across rules).

func (*LoadBalancer) GetRuleSummary 

func (lb *LoadBalancer) GetRuleSummary(rule *typ.Rule) map[string]interface{}

GetRuleSummary returns a summary of rule configuration and statistics

func (*LoadBalancer) GetServiceStats 

func (lb *LoadBalancer) GetServiceStats(provider, model string) *loadbalance.ServiceStats

GetServiceStats returns statistics for a specific service

func (*LoadBalancer) HealthFilter 

func (lb *LoadBalancer) HealthFilter() *typ.HealthFilter

HealthFilter returns the health filter for the load balancer

func (*LoadBalancer) RecordUsage 

func (lb *LoadBalancer) RecordUsage(provider, model string, inputTokens, outputTokens int)

RecordUsage records usage for a service Deprecated: This method is no longer needed as usage is recorded directly in handlers

func (*LoadBalancer) RegisterTactic 

func (lb *LoadBalancer) RegisterTactic(tacticType loadbalance.TacticType, tactic typ.LoadBalancingTactic)

RegisterTactic registers a custom tactic

func (*LoadBalancer) SelectService 

func (lb *LoadBalancer) SelectService(rule *typ.Rule) (*loadbalance.Service, error)

SelectService selects the best service for a rule based on the configured tactic

func (*LoadBalancer) Stop 

func (lb *LoadBalancer) Stop()

Stop stops the load balancer and cleanup resources

func (*LoadBalancer) UpdateServiceIndex 

func (lb *LoadBalancer) UpdateServiceIndex(rule *typ.Rule, selectedService *loadbalance.Service)

UpdateServiceIndex updates the current service ID for a rule

func (*LoadBalancer) ValidateRule 

func (lb *LoadBalancer) ValidateRule(rule *typ.Rule) error

ValidateRule validates a rule configuration

type LoadBalancerAPI 

type LoadBalancerAPI struct {
	// contains filtered or unexported fields
}

LoadBalancerAPI provides REST endpoints for load balancer management

func NewLoadBalancerAPI 

func NewLoadBalancerAPI(loadBalancer *LoadBalancer, cfg *config.Config) *LoadBalancerAPI

NewLoadBalancerAPI creates a new load balancer API

func (*LoadBalancerAPI) ClearAllStats 

func (api *LoadBalancerAPI) ClearAllStats(c *gin.Context)

ClearAllStats clears all statistics

func (*LoadBalancerAPI) ClearRuleStats 

func (api *LoadBalancerAPI) ClearRuleStats(c *gin.Context)

ClearRuleStats clears statistics for all services in a rule

func (*LoadBalancerAPI) ClearServiceStats 

func (api *LoadBalancerAPI) ClearServiceStats(c *gin.Context)

ClearServiceStats clears statistics for a specific service

func (*LoadBalancerAPI) GetAllStats 

func (api *LoadBalancerAPI) GetAllStats(c *gin.Context)

GetAllStats returns statistics for all services

func (*LoadBalancerAPI) GetCurrentService 

func (api *LoadBalancerAPI) GetCurrentService(c *gin.Context)

GetCurrentService returns the currently active service for a rule

func (*LoadBalancerAPI) GetMetrics 

func (api *LoadBalancerAPI) GetMetrics(c *gin.Context)

GetMetrics returns load balancing metrics

func (*LoadBalancerAPI) GetRule 

func (api *LoadBalancerAPI) GetRule(c *gin.Context)

GetRule returns a specific rule configuration

func (*LoadBalancerAPI) GetRuleStats 

func (api *LoadBalancerAPI) GetRuleStats(c *gin.Context)

GetRuleStats returns statistics for all services in a rule

func (*LoadBalancerAPI) GetRuleSummary 

func (api *LoadBalancerAPI) GetRuleSummary(c *gin.Context)

GetRuleSummary returns a comprehensive summary of a rule including statistics

func (*LoadBalancerAPI) GetServiceHealth 

func (api *LoadBalancerAPI) GetServiceHealth(c *gin.Context)

GetServiceHealth checks the health of all services in a rule

func (*LoadBalancerAPI) GetServiceStats 

func (api *LoadBalancerAPI) GetServiceStats(c *gin.Context)

GetServiceStats returns statistics for a specific service

func (*LoadBalancerAPI) GetServicesHealth 

func (api *LoadBalancerAPI) GetServicesHealth(c *gin.Context)

GetServicesHealth returns health status for all services in a rule

func (*LoadBalancerAPI) RegisterRoutes 

func (api *LoadBalancerAPI) RegisterRoutes(loadBalancer *gin.RouterGroup)

RegisterRoutes registers the load balancer API routes

func (*LoadBalancerAPI) ResetServiceHealth 

func (api *LoadBalancerAPI) ResetServiceHealth(c *gin.Context)

ResetServiceHealth manually resets a service's health to healthy

func (*LoadBalancerAPI) UpdateRuleTactic 

func (api *LoadBalancerAPI) UpdateRuleTactic(c *gin.Context)

UpdateRuleTactic updates the load balancing tactic for a rule

type LogEntry 

type LogEntry struct {
	Time    time.Time              `json:"time"`
	Level   string                 `json:"level"`
	Message string                 `json:"message"`
	Data    map[string]interface{} `json:"data,omitempty"`
	Fields  map[string]interface{} `json:"fields,omitempty"`
}

LogEntry represents a log entry for API response

type LogsResponse 

type LogsResponse struct {
	Total int        `json:"total"`
	Logs  []LogEntry `json:"logs"`
}

LogsResponse represents the API response for logs

type MaxTokensTransform 

type MaxTokensTransform struct {
	DefaultMaxTokens int
	MaxAllowed       int
}

MaxTokensTransform ensures max_tokens is set and properly bounded for Anthropic requests.

It applies three rules:

  1. Fill defaultMaxTokens when the client sends 0.
  2. Cap max_tokens at maxAllowed.
  3. If the thinking budget exceeds maxAllowed, shrink it to max(maxAllowed/10, 1024).

func NewMaxTokensTransform 

func NewMaxTokensTransform(defaultMaxTokens, maxAllowed int) *MaxTokensTransform

NewMaxTokensTransform creates a MaxTokensTransform.

func (*MaxTokensTransform) Apply 

func (t *MaxTokensTransform) Apply(ctx *protocoltransform.TransformContext) error

func (*MaxTokensTransform) Name 

func (t *MaxTokensTransform) Name() string

type MetricsData 

type MetricsData struct {
	InputTokens  int     // Number of input/prompt tokens
	OutputTokens int     // Number of output/completion tokens
	LatencyMs    int64   // Total request latency in milliseconds
	TTFTMs       int64   // Time To First Token in milliseconds (0 if not available/applicable)
	CacheHit     bool    // Whether this request hit the cache
	TPS          float64 // Tokens Per Second - generation speed (0 for non-streaming requests)
}

MetricsData encapsulates all metrics collected for a request. This structure is used to pass comprehensive metrics to updateServiceStats without requiring frequent function signature changes.

type MetricsResponse 

type MetricsResponse struct {
	Metrics       []ServiceMetric `json:"metrics"`
	TotalServices int             `json:"total_services" example:"5"`
}

MetricsResponse represents the metrics response

type ModelEndpointCapability 

type ModelEndpointCapability struct {
	ProviderUUID       string
	ModelID            string
	SupportsChat       bool
	ChatLatencyMs      int
	ChatError          string
	SupportsResponses  bool
	ResponsesLatencyMs int
	ResponsesError     string
	PreferredEndpoint  string // "chat", "responses", or ""
	LastVerified       time.Time
}

ModelEndpointCapability represents the endpoint capability information for a model

type ModelProbeData 

type ModelProbeData struct {
	ProviderUUID      string              `json:"provider_uuid" example:"uuid-123"`
	ModelID           string              `json:"model_id" example:"gpt-4"`
	ChatEndpoint      EndpointProbeStatus `json:"chat_endpoint"`
	ResponsesEndpoint EndpointProbeStatus `json:"responses_endpoint"`
	PreferredEndpoint string              `json:"preferred_endpoint" example:"responses"`
	LastUpdated       string              `json:"last_updated" example:"2026-01-23T10:30:00Z"`
}

ModelProbeData represents the probe result data

type ModelProbeRequest 

type ModelProbeRequest struct {
	ProviderUUID string `json:"provider_uuid" binding:"required" description:"Provider UUID to probe" example:"uuid-123"`
	ModelID      string `json:"model_id" binding:"required" description:"Model ID to probe" example:"gpt-4"`
	ForceRefresh bool   `json:"force_refresh" description:"Force new probe even if cached" example:"false"`
}

ModelProbeRequest represents the request to probe a specific model

type ModelProbeResponse 

type ModelProbeResponse struct {
	Success bool            `json:"success" example:"true"`
	Error   *ErrorDetail    `json:"error,omitempty"`
	Data    *ModelProbeData `json:"data,omitempty"`
}

ModelProbeResponse represents the response from model endpoint probing

type NpmPackage 

type NpmPackage struct {
	Name     string `json:"name"`
	DistTags struct {
		Latest string `json:"latest"`
	} `json:"dist-tags"`
}

NpmPackage represents an npm registry package response

type OAuthAuthorizeRequest 

type OAuthAuthorizeRequest = oauth.OAuthAuthorizeRequest

type OAuthAuthorizeResponse 

type OAuthAuthorizeResponse = oauth.OAuthAuthorizeResponse

type OAuthCallbackDataResponse 

type OAuthCallbackDataResponse = oauth.OAuthCallbackDataResponse

type OAuthCancelRequest 

type OAuthCancelRequest = oauth.OAuthCancelRequest

type OAuthDeviceCodeResponse 

type OAuthDeviceCodeResponse = oauth.OAuthDeviceCodeResponse

type OAuthErrorResponse 

type OAuthErrorResponse = oauth.OAuthErrorResponse

type OAuthMessageResponse 

type OAuthMessageResponse = oauth.OAuthMessageResponse

type OAuthProviderDataResponse 

type OAuthProviderDataResponse = oauth.OAuthProviderDataResponse

type OAuthProviderInfo 

type OAuthProviderInfo = oauth.OAuthProviderInfo

OAuth type aliases for backward compatibility These allow existing code to continue using the old type names while the actual implementation is now in the oauth module

type OAuthProvidersResponse 

type OAuthProvidersResponse = oauth.OAuthProvidersResponse

type OAuthRefreshTokenRequest 

type OAuthRefreshTokenRequest = oauth.OAuthRefreshTokenRequest

type OAuthRefreshTokenResponse 

type OAuthRefreshTokenResponse = oauth.OAuthRefreshTokenResponse

type OAuthSessionStatusResponse 

type OAuthSessionStatusResponse = oauth.OAuthSessionStatusResponse

type OAuthTokenResponse 

type OAuthTokenResponse = oauth.OAuthTokenResponse

type OAuthTokensResponse 

type OAuthTokensResponse = oauth.OAuthTokensResponse

type OAuthUpdateProviderRequest 

type OAuthUpdateProviderRequest = oauth.OAuthUpdateProviderRequest

type OAuthUpdateProviderResponse 

type OAuthUpdateProviderResponse = oauth.OAuthUpdateProviderResponse

type OpenAIChatCompletionResponse 

type OpenAIChatCompletionResponse struct {
	ID      string `json:"id" example:"chatcmpl-123"`
	Object  string `json:"object" example:"chat.completion"`
	Created int64  `json:"created" example:"1677652288"`
	Model   string `json:"model" example:"gpt-3.5-turbo"`
	Choices []struct {
		Index   int `json:"index" example:"0"`
		Message struct {
			Role    string `json:"role" example:"assistant"`
			Content string `json:"content" example:"Hello! How can I help you?"`
		} `json:"message"`
		FinishReason string `json:"finish_reason" example:"stop"`
	} `json:"choices"`
	Usage struct {
		PromptTokens     int `json:"prompt_tokens" example:"10"`
		CompletionTokens int `json:"completion_tokens" example:"20"`
		TotalTokens      int `json:"total_tokens" example:"30"`
	} `json:"usage"`
}

OpenAIChatCompletionResponse represents the OpenAI chat completion response

type OpenAIModel 

type OpenAIModel struct {
	ID      string `json:"id"`
	Object  string `json:"object"`
	Created int64  `json:"created"`
	OwnedBy string `json:"owned_by"`
}

OpenAIModel represents a model in OpenAI's models API format

type OpenAIModelsResponse 

type OpenAIModelsResponse struct {
	Object string        `json:"object"`
	Data   []OpenAIModel `json:"data"`
}

OpenAIModelsResponse represents OpenAI's models API response format

type PassthroughHandler 

type PassthroughHandler struct {
	// contains filtered or unexported fields
}

PassthroughHandler handles pass-through requests It replaces the model name and proxies the request without any transformations

func NewPassthroughHandler 

func NewPassthroughHandler(server *Server) *PassthroughHandler

NewPassthroughHandler creates a new pass-through handler

type PostTransformRecorder 

type PostTransformRecorder struct {
	// contains filtered or unexported fields
}

PostTransformRecorder captures the request after base transformation This records the request after protocol conversion (e.g., Anthropic → OpenAI)

func NewPostTransformRecorder 

func NewPostTransformRecorder(recorder interface{}, c *gin.Context) *PostTransformRecorder

NewPostTransformRecorder creates a new PostTransformRecorder

func (*PostTransformRecorder) Apply 

func (t *PostTransformRecorder) Apply(ctx *transform.TransformContext) error

Apply captures the request after base transformation The transformed request is in ctx.Request (after BaseTransform)

func (*PostTransformRecorder) Name 

func (t *PostTransformRecorder) Name() string

Name returns the name of this transform

type PreTransformRecorder 

type PreTransformRecorder struct {
	// contains filtered or unexported fields
}

PreTransformRecorder captures the original request before any transformations This is the first transform in the chain, recording the request as received from the client

func NewPreTransformRecorder 

func NewPreTransformRecorder(c *gin.Context, recorder interface{}) *PreTransformRecorder

NewPreTransformRecorder creates a new PreTransformRecorder

func (*PreTransformRecorder) Apply 

func (t *PreTransformRecorder) Apply(ctx *transform.TransformContext) error

Apply captures the original request before any transformations The original request is stored in ctx.OriginalRequest

func (*PreTransformRecorder) Name 

func (t *PreTransformRecorder) Name() string

Name returns the name of this transform

type ProbeCache 

type ProbeCache struct {
	// contains filtered or unexported fields
}

ProbeCache provides in-memory caching for model endpoint capability probe results

func NewProbeCache 

func NewProbeCache(ttl time.Duration) *ProbeCache

NewProbeCache creates a new probe cache with the specified TTL

func (*ProbeCache) CleanupExpired 

func (pc *ProbeCache) CleanupExpired()

CleanupExpired removes expired entries from cache

func (*ProbeCache) Clear 

func (pc *ProbeCache) Clear()

Clear removes all cached entries

func (*ProbeCache) Get 

func (pc *ProbeCache) Get(providerUUID, modelID string) *ModelEndpointCapability

Get retrieves cached capability for a model

func (*ProbeCache) Invalidate 

func (pc *ProbeCache) Invalidate(providerUUID, modelID string)

Invalidate removes cached capability for a specific model

func (*ProbeCache) InvalidateProvider 

func (pc *ProbeCache) InvalidateProvider(providerUUID string)

InvalidateProvider removes all cached capabilities for a provider

func (*ProbeCache) Set 

func (pc *ProbeCache) Set(providerUUID, modelID string, capability *ModelEndpointCapability)

Set stores capability in cache

func (*ProbeCache) SetFromProbeResult 

func (pc *ProbeCache) SetFromProbeResult(result *ProbeResult)

SetFromProbeResult stores probe result in cache

func (*ProbeCache) StartCleanupTask 

func (pc *ProbeCache) StartCleanupTask(interval time.Duration)

StartCleanupTask starts a background task to periodically clean up expired entries

type ProbeCacheRequest 

type ProbeCacheRequest struct {
	ProviderUUID string
	ModelID      string
	ForceRefresh bool // Force new probe even if cached
}

ProbeCacheRequest represents a request to probe a model

type ProbeMode 

type ProbeMode string

ProbeMode defines the test mode 

const (
	ProbeV2ModeSimple    ProbeMode = "simple"
	ProbeV2ModeStreaming ProbeMode = "streaming"
	ProbeV2ModeTool      ProbeMode = "tool"
)

type ProbeProviderRequest 

type ProbeProviderRequest struct {
	Name     string `json:"name" binding:"required" description:"Provider name" example:"openai"`
	APIBase  string `json:"api_base" binding:"required" description:"API base URL" example:"https://api.openai.com/v1"`
	APIStyle string `json:"api_style" binding:"required,oneof=openai anthropic" description:"API style" example:"openai"`
	Token    string `json:"token" binding:"required" description:"API token to test" example:"sk-..."`
}

ProbeProviderRequest represents the request to probe/test a provider's API key and connectivity

type ProbeProviderResponse 

type ProbeProviderResponse struct {
	Success bool                       `json:"success" example:"true"`
	Error   *ErrorDetail               `json:"error,omitempty"`
	Data    *ProbeProviderResponseData `json:"data,omitempty"`
}

ProbeProviderResponse represents the response from provider probing

type ProbeProviderResponseData 

type ProbeProviderResponseData struct {
	Provider     string `json:"provider" example:"openai"`
	APIBase      string `json:"api_base" example:"https://api.openai.com/v1"`
	APIStyle     string `json:"api_style" example:"openai"`
	Valid        bool   `json:"valid" example:"true"`
	Message      string `json:"message" example:"API key is valid and accessible"`
	TestResult   string `json:"test_result" example:"models_endpoint_success"`
	ResponseTime int64  `json:"response_time_ms" example:"250"`
	ModelsCount  int    `json:"models_count,omitempty" example:"150"`
}

ProbeProviderResponseData represents the data returned from provider probing

type ProbeRequest 

type ProbeRequest struct {
	Provider string `json:"provider" binding:"required" description:"Provider UUID to test against" example:"550e8400-e29b-41d4-a716-446655440000"`
	Model    string `json:"model" binding:"required" description:"Model name to test against" example:"gpt-4-latest"`
}

ProbeRequest represents the request to probe/test a provider and model

type ProbeRequestDetail 

type ProbeRequestDetail struct {
	Messages    []map[string]interface{} `json:"messages"`
	Model       string                   `json:"model"`
	MaxTokens   int                      `json:"max_tokens"`
	Temperature float64                  `json:"temperature"`
	Provider    string                   `json:"provider"`
	Timestamp   string                   `json:"timestamp"`
}

ProbeRequestDetail represents the mock request data for probing

func NewMockRequest 

func NewMockRequest(provider, model string) ProbeRequestDetail

NewMockRequest creates a new mock request with default values

type ProbeResponse 

type ProbeResponse struct {
	Success bool               `json:"success"`
	Error   *ErrorDetail       `json:"error,omitempty"`
	Data    *ProbeResponseData `json:"data,omitempty"`
}

ProbeResponse represents the overall probe response

type ProbeResponseData 

type ProbeResponseData struct {
	Request     ProbeRequestDetail  `json:"request"`
	Response    ProbeResponseDetail `json:"response"`
	Usage       ProbeUsage          `json:"usage"`
	CurlCommand string              `json:"curl_command,omitempty"`
}

ProbeResponseData represents the response data structure

type ProbeResponseDetail 

type ProbeResponseDetail struct {
	Content      string `json:"content"`
	Model        string `json:"model"`
	Provider     string `json:"provider"`
	FinishReason string `json:"finish_reason"`
	Error        string `json:"error,omitempty"`
}

ProbeResponseDetail represents the API response

type ProbeResult 

type ProbeResult struct {
	ProviderUUID      string
	ModelID           string
	ChatEndpoint      EndpointStatus
	ResponsesEndpoint EndpointStatus
	PreferredEndpoint string
	LastUpdated       time.Time
}

ProbeResult represents the complete probe result for a model

type ProbeTarget 

type ProbeTarget string

ProbeTarget defines the target type for probe 

const (
	ProbeV2TargetRule     ProbeTarget = "rule"
	ProbeV2TargetProvider ProbeTarget = "provider"
)

type ProbeUsage 

type ProbeUsage struct {
	PromptTokens     int `json:"prompt_tokens"`
	CompletionTokens int `json:"completion_tokens"`
	TotalTokens      int `json:"total_tokens"`
	TimeCost         int `json:"time_cost"`
}

ProbeUsage represents token usage information

type ProbeV2Data 

type ProbeV2Data struct {
	Content    string            `json:"content,omitempty"`
	ToolCalls  []ProbeV2ToolCall `json:"tool_calls,omitempty"`
	Usage      *ProbeV2Usage     `json:"usage,omitempty"`
	LatencyMs  int64             `json:"latency_ms"`
	RequestURL string            `json:"request_url,omitempty"`
}

ProbeV2Data represents the probe result data

type ProbeV2Request 

type ProbeV2Request struct {
	// Target type: rule or provider
	TargetType ProbeTarget `json:"target_type" binding:"required"`

	// Rule test (required when target_type is rule)
	Scenario string `json:"scenario,omitempty" example:"anthropic"`
	RuleUUID string `json:"rule_uuid,omitempty" binding:"required_if=TargetType rule"`

	// Provider test (required when target_type is provider)
	ProviderUUID string `json:"provider_uuid,omitempty" binding:"required_if=TargetType provider"`
	Model        string `json:"model,omitempty" binding:"required_if=TargetType provider"`

	// Test mode
	TestMode ProbeMode `json:"test_mode" binding:"required"`

	// Optional custom message (overrides preset)
	Message string `json:"message,omitempty"`
}

ProbeV2Request represents a Probe V3 request

type ProbeV2Response 

type ProbeV2Response struct {
	Success bool         `json:"success"`
	Error   *ErrorDetail `json:"error,omitempty"`
	Data    *ProbeV2Data `json:"data,omitempty"`
}

ProbeV2Response represents a Probe V3 response

type ProbeV2ResponseChunk 

type ProbeV2ResponseChunk struct {
	Type      string           `json:"type"` // content, tool_call, error, done
	Content   string           `json:"content,omitempty"`
	ToolCall  *ProbeV2ToolCall `json:"tool_call,omitempty"`
	Error     string           `json:"error,omitempty"`
	Usage     *ProbeV2Usage    `json:"usage,omitempty"`
	LatencyMs int64            `json:"latency_ms,omitempty"`
}

ProbeV2ResponseChunk represents a streaming response chunk

type ProbeV2Service 

type ProbeV2Service struct {
	// contains filtered or unexported fields
}

ProbeV2Service handles probe V3 operations

func NewProbeV2Service 

func NewProbeV2Service(server *Server) *ProbeV2Service

NewProbeV2Service creates a new Probe V3 service

type ProbeV2ToolCall 

type ProbeV2ToolCall struct {
	ID        string                 `json:"id"`
	Name      string                 `json:"name"`
	Arguments map[string]interface{} `json:"arguments"`
}

ProbeV2ToolCall represents a tool call in the response

type ProbeV2Usage 

type ProbeV2Usage struct {
	PromptTokens     int `json:"prompt_tokens"`
	CompletionTokens int `json:"completion_tokens"`
	TotalTokens      int `json:"total_tokens"`
}

ProbeV2Usage represents token usage

type ProtocolRecorder 

type ProtocolRecorder struct {
	*ScenarioRecorder
	// contains filtered or unexported fields
}

ProtocolRecorder extends ScenarioRecorder for dual-stage recording It provides support for capturing both original and transformed requests, as well as provider and final responses (for protocol conversion scenarios)

func NewScenarioRecorderV2 

func NewScenarioRecorderV2(recorder *ScenarioRecorder, provider *typ.Provider, model string, mode obs.RecordMode) *ProtocolRecorder

NewScenarioRecorderV2 creates a new ProtocolRecorder from an existing ScenarioRecorder

func (*ProtocolRecorder) Record 

func (sr *ProtocolRecorder) Record()

Record writes the V2 record entry to the sink

func (*ProtocolRecorder) RecordError 

func (sr *ProtocolRecorder) RecordError(err error)

RecordError records an error for V2 entries

func (*ProtocolRecorder) SetFinalResponse 

func (sr *ProtocolRecorder) SetFinalResponse(resp *obs.RecordResponse)

SetFinalResponse stores the final response to the client

func (*ProtocolRecorder) SetOriginalRequest 

func (sr *ProtocolRecorder) SetOriginalRequest(req *obs.RecordRequest)

SetOriginalRequest stores the pre-transform request

func (*ProtocolRecorder) SetProviderResponse 

func (sr *ProtocolRecorder) SetProviderResponse(resp *obs.RecordResponse)

SetProviderResponse stores the raw response from the provider

func (*ProtocolRecorder) SetTransformSteps 

func (sr *ProtocolRecorder) SetTransformSteps(steps []string)

SetTransformSteps records the transform steps that were applied

func (*ProtocolRecorder) SetTransformedRequest 

func (sr *ProtocolRecorder) SetTransformedRequest(req *obs.RecordRequest)

SetTransformedRequest stores the post-transform request

type ProviderModelInfo 

type ProviderModelInfo struct {
	Models      []string             `json:"models" example:"gpt-3.5-turbo,gpt-4"`
	StarModels  []string             `json:"star_models" example:"gpt-4"`
	CustomModel []string             `json:"custom_model" example:"custom-gpt-model"`
	APIBase     string               `json:"api_base" example:"https://api.openai.com/v1"`
	LastUpdated string               `json:"last_updated,omitempty" example:"2024-01-15 10:30:00"`
	Quota       *quota.ProviderUsage `json:"quota,omitempty"` // Quota information for this provider
}

ProviderModelInfo represents model information for a specific provider

type ProviderModelsResponse 

type ProviderModelsResponse struct {
	Success bool              `json:"success" example:"true"`
	Message string            `json:"message" example:"Provider models successfully"`
	Data    ProviderModelInfo `json:"data"`
}

ProviderModelsResponse represents the response for getting provider models

type ProviderResponse 

type ProviderResponse struct {
	UUID          string           `json:"uuid" example:"0123456789ABCDEF"`
	Name          string           `json:"name" example:"openai"`
	APIBase       string           `json:"api_base" example:"https://api.openai.com/v1"`
	APIStyle      string           `json:"api_style" example:"openai"`
	Token         string           `json:"token" example:"sk-***...***"` // Only populated for api_key auth type
	NoKeyRequired bool             `json:"no_key_required" example:"false"`
	Enabled       bool             `json:"enabled" example:"true"`
	ProxyURL      string           `json:"proxy_url,omitempty" example:"http://127.0.0.1:7890"`
	AuthType      string           `json:"auth_type,omitempty" example:"api_key"` // api_key or oauth
	OAuthDetail   *typ.OAuthDetail `json:"oauth_detail,omitempty"`                // OAuth credentials (only for oauth auth type)
}

ProviderResponse represents a provider configuration with masked token

type ProvidersResponse 

type ProvidersResponse struct {
	Success bool               `json:"success" example:"true"`
	Data    []ProviderResponse `json:"data"`
}

ProvidersResponse represents the response for listing providers

type RequestBodyResponse added in v0.260418.2200 

type RequestBodyResponse struct {
	ID        string `json:"id"`
	Method    string `json:"method"`
	Path      string `json:"path"`
	Body      string `json:"body"`
	Truncated bool   `json:"truncated"`
}

RequestBodyResponse represents the API response for a stored request body

type RequestConfig 

type RequestConfig struct {
	RequestModel  string `json:"request_model" example:"gpt-3.5-turbo"`
	ResponseModel string `json:"response_model" example:"gpt-3.5-turbo"`
	Provider      string `json:"provider" example:"openai"`
	DefaultModel  string `json:"default_model" example:"gpt-3.5-turbo"`
}

RequestConfig represents a request configuration in defaults response

type RuleStatsResponse 

type RuleStatsResponse struct {
	Rule  string                 `json:"rule" example:"gpt-4"`
	Stats map[string]interface{} `json:"stats"`
}

RuleStatsResponse represents the statistics response for a rule

type RuleSummaryResponse 

type RuleSummaryResponse struct {
	Summary interface{} `json:"summary"`
}

RuleSummaryResponse represents a rule summary response

type SDKProbeBuilder 

type SDKProbeBuilder struct{}

SDKProbeBuilder builds SDK requests for probe operations

func NewSDKProbeBuilder 

func NewSDKProbeBuilder() *SDKProbeBuilder

NewSDKProbeBuilder creates a new SDK probe builder

type ScenarioRecorder 

type ScenarioRecorder struct {
	// contains filtered or unexported fields
}

ScenarioRecorder captures scenario-level request/response recording

func (*ScenarioRecorder) EnableStreaming 

func (sr *ScenarioRecorder) EnableStreaming()

EnableStreaming enables streaming mode for this recorder

func (*ScenarioRecorder) GetStreamChunks 

func (sr *ScenarioRecorder) GetStreamChunks() []map[string]interface{}

GetStreamChunks returns the collected stream chunks

func (*ScenarioRecorder) RecordError 

func (sr *ScenarioRecorder) RecordError(err error)

RecordError records an error for the scenario-level request

func (*ScenarioRecorder) RecordResponse 

func (sr *ScenarioRecorder) RecordResponse(provider *typ.Provider, model string)

RecordResponse records the scenario-level response (tingly-box -> client) This captures the response sent back to the client

func (*ScenarioRecorder) RecordStreamChunk 

func (sr *ScenarioRecorder) RecordStreamChunk(eventType string, chunk interface{})

RecordStreamChunk records a single stream chunk Uses RawJSON() when available to preserve the exact original API response

func (*ScenarioRecorder) SetAssembledResponse 

func (sr *ScenarioRecorder) SetAssembledResponse(response any)

SetAssembledResponse sets the assembled response for streaming Accepts any type (e.g., anthropic.Message) and converts to map for storage

type Server 

type Server struct {
	// contains filtered or unexported fields
}

Server represents the HTTP server

func GetGlobalServer 

func GetGlobalServer() *Server

GetGlobalServer gets the global server instance

func NewServer 

func NewServer(cfg *config.Config, opts ...ServerOption) *Server

NewServer creates a new HTTP server instance with functional options

func (*Server) AnthropicCountTokens 

func (s *Server) AnthropicCountTokens(c *gin.Context)

AnthropicCountTokens handles Anthropic v1 count_tokens endpoint This is the entry point that delegates to the appropriate implementation (v1 or beta)

func (*Server) AnthropicListModels 

func (s *Server) AnthropicListModels(c *gin.Context)

AnthropicListModels handles Anthropic v1 models endpoint

func (*Server) AnthropicListModelsForScenario 

func (s *Server) AnthropicListModelsForScenario(c *gin.Context, scenario typ.RuleScenario)

AnthropicListModelsForScenario handles scenario-scoped model listing for Anthropic format

func (*Server) AnthropicMessagesV1 

func (s *Server) AnthropicMessagesV1(c *gin.Context, req protocol.AnthropicMessagesRequest, proxyModel string, provider *typ.Provider, actualModel string, rule *typ.Rule)

AnthropicMessagesV1 implements standard v1 messages API

func (*Server) AnthropicMessagesV1Beta 

func (s *Server) AnthropicMessagesV1Beta(c *gin.Context, req protocol.AnthropicBetaMessagesRequest, proxyModel string, provider *typ.Provider, actualModel string, rule *typ.Rule)

AnthropicMessagesV1Beta implements beta messages API

func (*Server) ApplyRecording 

func (s *Server) ApplyRecording(scenario typ.RuleScenario) bool

ApplyRecording checks if recording should be applied for a scenario

func (*Server) ApplySmartCompact 

func (s *Server) ApplySmartCompact(scenario typ.RuleScenario) bool

ApplySmartCompact checks if smart_compact should be applied for a scenario

func (*Server) BuildTransformChain 

func (s *Server) BuildTransformChain(c *gin.Context, targetType protocol.APIType, providerURL string, scenarioFlags *typ.ScenarioFlags, recorder *ProtocolRecorder) (*transform.TransformChain, error)

BuildTransformChain builds the appropriate transform chain based on recording configuration

func (*Server) Cancel added in v0.260414.2000 

func (s *Server) Cancel() context.CancelFunc

func (*Server) ClearGuardrailsHistory 

func (s *Server) ClearGuardrailsHistory(c *gin.Context)

ClearGuardrailsHistory deletes all persisted guardrails history rows.

func (*Server) ClearLogs 

func (s *Server) ClearLogs(c *gin.Context)

ClearLogs clears all log entries

func (*Server) ClearRequestBodies added in v0.260418.2200 

func (s *Server) ClearRequestBodies(c *gin.Context)

ClearRequestBodies clears all stored request bodies from memory

func (*Server) Context added in v0.260414.2000 

func (s *Server) Context() context.Context

func (*Server) CreateGuardrailsCredential 

func (s *Server) CreateGuardrailsCredential(c *gin.Context)

func (*Server) CreateGuardrailsGroup 

func (s *Server) CreateGuardrailsGroup(c *gin.Context)

CreateGuardrailsGroup creates a new group and reloads the engine.

func (*Server) CreateGuardrailsPolicy 

func (s *Server) CreateGuardrailsPolicy(c *gin.Context)

CreateGuardrailsPolicy creates a new policy and reloads the engine.

func (*Server) CreateProvider 

func (s *Server) CreateProvider(c *gin.Context)

CreateProvider adds a new provider

func (*Server) DeleteGuardrailsCredential 

func (s *Server) DeleteGuardrailsCredential(c *gin.Context)

func (*Server) DeleteGuardrailsGroup 

func (s *Server) DeleteGuardrailsGroup(c *gin.Context)

DeleteGuardrailsGroup deletes a group and reloads the engine.

func (*Server) DeleteGuardrailsPolicy 

func (s *Server) DeleteGuardrailsPolicy(c *gin.Context)

DeleteGuardrailsPolicy deletes a policy and reloads the engine.

func (*Server) DeleteProvider 

func (s *Server) DeleteProvider(c *gin.Context)

DeleteProvider removes a provider

func (*Server) DetermineProviderAndModel deprecated 

func (s *Server) DetermineProviderAndModel(rule *typ.Rule) (*typ.Provider, *loadbalance.Service, error)

DetermineProviderAndModel resolves the model name and finds the appropriate provider using load balancing

Deprecated: Use routing.SimpleSelector.SelectService() instead. This method is kept for backward compatibility with anthropic_token.go and passthrough.go. Currently used by:

  • internal/server/anthropic_token.go:89
  • internal/server/passthrough.go:81

func (*Server) DetermineProviderAndModelWithScenario deprecated 

func (s *Server) DetermineProviderAndModelWithScenario(scenario typ.RuleScenario, rule *typ.Rule, req interface{}, sessionID string) (*typ.Provider, *loadbalance.Service, error)

DetermineProviderAndModelWithScenario determines the provider and service for a request// based on the rule's configuration, smart routing rules, and optional affinity locking. sessionID is used for affinity locking when SmartAffinity is enabled.

Deprecated: Use routing.SimpleSelector.SelectService() instead for cleaner code. This method is kept for backward compatibility with anthropic_token.go and passthrough.go. See internal/server/anthropic.go for migration example.

func (*Server) ExportGuardrailsFragments added in v0.260418.2200 

func (s *Server) ExportGuardrailsFragments(c *gin.Context)

ExportGuardrailsFragments returns the raw imported fragment files selected by the user so the UI can download one or more source files directly.

func (*Server) ExtractRequestContext deprecated 

func (s *Server) ExtractRequestContext(req interface{}) (*smartrouting.RequestContext, error)

ExtractRequestContext extracts RequestContext from request based on type

Deprecated: Use routing/smart_routing_helper.go ExtractRequestContext() instead.

func (*Server) GenerateToken 

func (s *Server) GenerateToken(c *gin.Context)

GenerateToken handles token generation requests

func (*Server) GetActionHistory 

func (s *Server) GetActionHistory(c *gin.Context)

GetActionHistory retrieves user action history from memory Query parameters:

  • limit: maximum number of recent entries to return (default: 100, max: 1000)

func (*Server) GetActionStats 

func (s *Server) GetActionStats(c *gin.Context)

GetActionStats returns statistics about user actions

func (*Server) GetGuardrailsBuiltins 

func (s *Server) GetGuardrailsBuiltins(c *gin.Context)

GetGuardrailsBuiltins returns curated builtin policies for the Guardrails UI.

func (*Server) GetGuardrailsConfig 

func (s *Server) GetGuardrailsConfig(c *gin.Context)

GetGuardrailsConfig returns the current guardrails config file content and parsed config.

func (*Server) GetGuardrailsCredential 

func (s *Server) GetGuardrailsCredential(c *gin.Context)

GetGuardrailsCredential returns a single protected credential, including the current secret, for the local editor dialog.

func (*Server) GetGuardrailsCredentials 

func (s *Server) GetGuardrailsCredentials(c *gin.Context)

Credential list responses intentionally mask secrets; the edit dialog uses GetGuardrailsCredential when it needs the underlying value. GetGuardrailsCredentials returns protected credentials without exposing raw secrets.

func (*Server) GetGuardrailsHistory 

func (s *Server) GetGuardrailsHistory(c *gin.Context)

GetGuardrailsHistory returns the most recent guardrails history rows.

func (*Server) GetGuardrailsRegistry added in v0.260418.2200 

func (s *Server) GetGuardrailsRegistry(c *gin.Context)

GetGuardrailsRegistry lists downloadable policies from a remote registry.

func (*Server) GetHealthInfo 

func (s *Server) GetHealthInfo(c *gin.Context)

GetHealthInfo handles health check requests This is a lightweight health check that can be called frequently

func (*Server) GetHistory 

func (s *Server) GetHistory(c *gin.Context)

func (*Server) GetInfoConfig 

func (s *Server) GetInfoConfig(c *gin.Context)

func (*Server) GetInfoVersion 

func (s *Server) GetInfoVersion(c *gin.Context)

func (*Server) GetLatestVersion 

func (s *Server) GetLatestVersion(c *gin.Context)

GetLatestVersion checks GitHub releases for the latest version

func (*Server) GetLoadBalancer 

func (s *Server) GetLoadBalancer() *LoadBalancer

GetLoadBalancer returns the load balancer instance

func (*Server) GetLogStats 

func (s *Server) GetLogStats(c *gin.Context)

GetLogStats returns statistics about the logs

func (*Server) GetLogs 

func (s *Server) GetLogs(c *gin.Context)

GetLogs retrieves logs with optional filtering Query parameters:

  • limit: maximum number of entries to return (default: 100)
  • level: filter by log level (debug, info, warn, error)
  • since: RFC3339 timestamp to filter entries after this time

func (*Server) GetOrCreateScenarioRecorderV2 

func (s *Server) GetOrCreateScenarioRecorderV2(c *gin.Context, scenario string, provider *typ.Provider, model string, mode obs.RecordMode) *ProtocolRecorder

GetOrCreateScenarioRecorderV2 gets or creates a V2 scenario recorder for the given scenario

func (*Server) GetOrCreateScenarioSink 

func (s *Server) GetOrCreateScenarioSink(scenario typ.RuleScenario) *obs.Sink

GetOrCreateScenarioSink gets or creates a recording sink for the specified scenario The sink is created on-demand and cached for subsequent use

func (*Server) GetPreferredEndpointForModel 

func (s *Server) GetPreferredEndpointForModel(provider *typ.Provider, modelID string) string

GetPreferredEndpointForModel returns the preferred endpoint (chat or responses) for a model Returns "responses" if the model supports the Responses API, otherwise returns "chat"

func (*Server) GetProvider 

func (s *Server) GetProvider(c *gin.Context)

GetProvider returns details for a specific provider (with masked token)

func (*Server) GetProviderModelsByUUID 

func (s *Server) GetProviderModelsByUUID(c *gin.Context)

func (*Server) GetProviders 

func (s *Server) GetProviders(c *gin.Context)

func (*Server) GetQuotaManager added in v0.260409.1540 

func (s *Server) GetQuotaManager() providerQuotaModule.Manager

GetQuotaManager returns the quota manager (returns interface type for module access)

func (*Server) GetRequestBody added in v0.260418.2200 

func (s *Server) GetRequestBody(c *gin.Context)

GetRequestBody retrieves a stored request body by its reference ID GET /api/logs/request/:id

func (*Server) GetRequestBodyStats added in v0.260418.2200 

func (s *Server) GetRequestBodyStats(c *gin.Context)

GetRequestBodyStats returns statistics about the request body store

func (*Server) GetRouter 

func (s *Server) GetRouter() *gin.Engine

GetRouter returns the Gin engine for testing purposes

func (*Server) GetRoutingSelector added in v0.260414.2000 

func (s *Server) GetRoutingSelector() *routing.SimpleSelector

GetRoutingSelector returns the server's routing selector for service selection.

func (*Server) GetStatus 

func (s *Server) GetStatus(c *gin.Context)

func (*Server) GetSystemLogLevel 

func (s *Server) GetSystemLogLevel(c *gin.Context)

GetSystemLogLevel returns the current system log level

func (*Server) GetSystemLogStats 

func (s *Server) GetSystemLogStats(c *gin.Context)

GetSystemLogStats returns statistics about the system logs

func (*Server) GetSystemLogs 

func (s *Server) GetSystemLogs(c *gin.Context)

GetSystemLogs retrieves system logs with optional filtering Query parameters:

  • limit: maximum number of recent entries to return (default: 100, max: 1000)

func (*Server) GetToken 

func (s *Server) GetToken(c *gin.Context)

GetToken handles token retrieval requests - generates a token if it doesn't exist

func (*Server) GetUserToken 

func (s *Server) GetUserToken(c *gin.Context)

GetUserToken returns the current user token (masked) Requires authentication

func (*Server) HandleAnthropicMessages 

func (s *Server) HandleAnthropicMessages(c *gin.Context)

HandleAnthropicMessages handles Anthropic v1 messages API requests This is the entry point that delegates to the appropriate implementation (v1 or beta)

func (*Server) HandleOpenAIChatCompletions 

func (s *Server) HandleOpenAIChatCompletions(c *gin.Context)

HandleOpenAIChatCompletions handles OpenAI v1 chat completion requests

func (*Server) HandleProbeModel 

func (s *Server) HandleProbeModel(c *gin.Context)

HandleProbeModel tests a rule configuration by sending a sample request to the configured provider

func (*Server) HandleProbeModelEndpoints 

func (s *Server) HandleProbeModelEndpoints(c *gin.Context)

HandleProbeModelEndpoints handles adaptive probe for model endpoints (chat and responses)

func (*Server) HandleProbeProvider 

func (s *Server) HandleProbeProvider(c *gin.Context)

HandleProbeProvider tests a provider's API key and connectivity

func (*Server) HandleProbeV2 

func (s *Server) HandleProbeV2(c *gin.Context)

HandleProbeV2 handles Probe V2 requests (unified endpoint for all test types)

func (*Server) HandleResponsesCreate 

func (s *Server) HandleResponsesCreate(c *gin.Context)

HandleResponsesCreate handles POST /v1/responses

func (*Server) HealthMonitor 

func (s *Server) HealthMonitor() *loadbalance.HealthMonitor

HealthMonitor returns the server's health monitor

func (*Server) ImportGuardrailsFragment added in v0.260418.2200 

func (s *Server) ImportGuardrailsFragment(c *gin.Context)

ImportGuardrailsFragment appends one or more policies from a fragment file into guardrails/custom/import.yaml and ensures the root config imports it.

func (*Server) InstallGuardrailsRegistryPolicy added in v0.260418.2200 

func (s *Server) InstallGuardrailsRegistryPolicy(c *gin.Context)

InstallGuardrailsRegistryPolicy downloads a remote policy fragment into guardrails/remote and wires it into root imports.

func (*Server) InvalidateProviderCache 

func (s *Server) InvalidateProviderCache(providerUUID string)

InvalidateProviderCache invalidates cached capabilities for a provider

func (*Server) IsExperimentalFeatureEnabled 

func (s *Server) IsExperimentalFeatureEnabled(scenario typ.RuleScenario, feature string) bool

IsExperimentalFeatureEnabled checks if an experimental feature is enabled for a scenario

func (*Server) IsFeatureEnabled 

func (s *Server) IsFeatureEnabled(feature string) bool

IsFeatureEnabled checks if a specific feature is enabled

func (*Server) IsRemoteCoderRunning 

func (s *Server) IsRemoteCoderRunning() bool

IsRemoteCoderRunning returns whether the remote control service is running

func (*Server) ListModelsByScenario 

func (s *Server) ListModelsByScenario(c *gin.Context)

ListModelsByScenario handles the /v1/models endpoint for scenario-based routing

func (*Server) NewUsageTracker 

func (s *Server) NewUsageTracker() *UsageTracker

NewUsageTracker creates a new UsageTracker

func (*Server) OpenAIChatCompletion 

func (s *Server) OpenAIChatCompletion(c *gin.Context, req protocol.OpenAIChatCompletionRequest, responseModel string, provider *typ.Provider, scenarioType typ.RuleScenario, rule *typ.Rule)

func (*Server) OpenAIListModels 

func (s *Server) OpenAIListModels(c *gin.Context)

OpenAIListModels handles the /v1/models endpoint (OpenAI compatible)

func (*Server) OpenAIListModelsForScenario 

func (s *Server) OpenAIListModelsForScenario(c *gin.Context, scenario typ.RuleScenario)

OpenAIListModelsForScenario handles scenario-scoped model listing for OpenAI format

func (*Server) PassthroughAnthropic 

func (s *Server) PassthroughAnthropic(c *gin.Context)

PassthroughAnthropic handles Anthropic-style pass-through requests Consolidates: PassthroughAnthropicMessages, PassthroughAnthropicCountTokens

func (*Server) PassthroughOpenAI 

func (s *Server) PassthroughOpenAI(c *gin.Context)

PassthroughOpenAI handles OpenAI-style pass-through requests Consolidates: PassthroughOpenAIChatCompletions, PassthroughOpenAIResponsesCreate, PassthroughOpenAIResponsesGet

func (*Server) RecordScenarioRequest 

func (s *Server) RecordScenarioRequest(c *gin.Context, scenario string) *ProtocolRecorder

RecordScenarioRequest records the scenario-level request (client -> tingly-box) This captures the original request from the client before any transformation

func (*Server) ReloadGuardrailsConfig 

func (s *Server) ReloadGuardrailsConfig(c *gin.Context)

ReloadGuardrailsConfig reloads guardrails from disk and rebuilds the runtime.

func (*Server) ResetModelToken 

func (s *Server) ResetModelToken(c *gin.Context)

ResetModelToken generates a new secure random model token and updates the configuration Requires authentication

func (*Server) ResetUserToken 

func (s *Server) ResetUserToken(c *gin.Context)

ResetUserToken generates a new secure random token and updates the configuration Requires authentication

func (*Server) ResponsesCreate 

func (s *Server) ResponsesCreate(c *gin.Context, scenarioType typ.RuleScenario, provider *typ.Provider, rule *typ.Rule, req protocol.ResponseCreateRequest, responseModel string, maxAllowed int)

func (*Server) ResponsesGet 

func (s *Server) ResponsesGet(c *gin.Context)

ResponsesGet handles GET /v1/responses/{id}

func (*Server) RestartServer 

func (s *Server) RestartServer(c *gin.Context)

func (*Server) SelectServiceFromSmartRouting deprecated 

func (s *Server) SelectServiceFromSmartRouting(matchedServices []*loadbalance.Service, rule *typ.Rule) (*loadbalance.Service, error)

SelectServiceFromSmartRouting selects a service from matched smart routing services Creates a temporary rule with the matched services and uses the configured load balancing tactic

Deprecated: Use routing.SmartRoutingStage instead.

func (*Server) SetSystemLogLevel 

func (s *Server) SetSystemLogLevel(c *gin.Context)

SetSystemLogLevel sets the minimum log level for system logs

func (*Server) SetupAnthropicEndpoints 

func (s *Server) SetupAnthropicEndpoints(group *gin.RouterGroup)

func (*Server) SetupMixinEndpoints 

func (s *Server) SetupMixinEndpoints(group *gin.RouterGroup)

func (*Server) SetupOpenAIEndpoints 

func (s *Server) SetupOpenAIEndpoints(group *gin.RouterGroup)

func (*Server) SetupPassthroughAnthropicEndpoints 

func (s *Server) SetupPassthroughAnthropicEndpoints(group *gin.RouterGroup)

SetupPassthroughAnthropicEndpoints sets up pass-through endpoints for Anthropic-style requests These endpoints bypass request/response transformations and only replace the model name

func (*Server) SetupPassthroughOpenAIEndpoints 

func (s *Server) SetupPassthroughOpenAIEndpoints(group *gin.RouterGroup)

SetupPassthroughOpenAIEndpoints sets up pass-through endpoints for OpenAI-style requests These endpoints bypass request/response transformations and only replace the model name

func (*Server) ShouldRecording 

func (s *Server) ShouldRecording(recorder *ProtocolRecorder) bool

ShouldRecording determines if recording should be used

func (*Server) Start 

func (s *Server) Start(port int) error

Start starts the HTTP server

func (*Server) StartDynamicCallbackServer 

func (s *Server) StartDynamicCallbackServer(sessionID string, port int) error

StartDynamicCallbackServer starts a temporary callback server for OAuth Implements CallbackServerManager interface for oauth module

func (*Server) StartRemoteCoder 

func (s *Server) StartRemoteCoder() error

StartRemoteCoder starts the remote control service if not already running

func (*Server) StartServer 

func (s *Server) StartServer(c *gin.Context)

func (*Server) Stop 

func (s *Server) Stop(ctx context.Context) error

Stop gracefully stops the HTTP server

func (*Server) StopDynamicCallbackServer 

func (s *Server) StopDynamicCallbackServer(sessionID string)

StopDynamicCallbackServer stops a temporary callback server for OAuth Implements CallbackServerManager interface for oauth module

func (*Server) StopRemoteCoder 

func (s *Server) StopRemoteCoder()

StopRemoteCoder stops the remote control service if running

func (*Server) StopServer 

func (s *Server) StopServer(c *gin.Context)

func (*Server) SyncRemoteCoderBots 

func (s *Server) SyncRemoteCoderBots(ctx context.Context) error

SyncRemoteCoderBots syncs bots with the remote control bot manager

func (*Server) ToggleProvider 

func (s *Server) ToggleProvider(c *gin.Context)

ToggleProvider enables/disables a provider

func (*Server) TrackUsage 

func (s *Server) TrackUsage(ctx context.Context, inputTokens, outputTokens int, err error)

TrackUsage implements the UsageTracker interface. It extracts the gin.Context from the provided context and calls trackUsageFromContext. The gin.Context should be stored in the context with the key "gin_context".

func (*Server) UpdateGuardrailsConfig 

func (s *Server) UpdateGuardrailsConfig(c *gin.Context)

UpdateGuardrailsConfig saves a new guardrails config and reloads the engine.

func (*Server) UpdateGuardrailsCredential 

func (s *Server) UpdateGuardrailsCredential(c *gin.Context)

func (*Server) UpdateGuardrailsGroup 

func (s *Server) UpdateGuardrailsGroup(c *gin.Context)

UpdateGuardrailsGroup updates a single group and reloads the engine.

func (*Server) UpdateGuardrailsPolicy 

func (s *Server) UpdateGuardrailsPolicy(c *gin.Context)

UpdateGuardrailsPolicy updates a single policy and reloads the engine.

func (*Server) UpdateProvider 

func (s *Server) UpdateProvider(c *gin.Context)

UpdateProvider updates an existing provider

func (*Server) UpdateProviderModelsByUUID 

func (s *Server) UpdateProviderModelsByUUID(c *gin.Context)

func (*Server) UsageStore 

func (s *Server) UsageStore() *db.UsageStore

UsageStore returns the server's usage store instance for internal integrations.

func (*Server) UseAIEndpoints 

func (s *Server) UseAIEndpoints()

func (*Server) UseIndexHTML 

func (s *Server) UseIndexHTML(c *gin.Context)

func (*Server) UseLoadBalanceEndpoints 

func (s *Server) UseLoadBalanceEndpoints()

func (*Server) UseTokenManagementEndpoints added in v0.260418.2200 

func (s *Server) UseTokenManagementEndpoints()

UseTokenManagementEndpoints registers the token management API endpoints

func (*Server) UseUIEndpoints 

func (s *Server) UseUIEndpoints(ctx context.Context)

Init sets up Server routes and templates on the main server engine

func (*Server) UseVirtualModelEndpoints 

func (s *Server) UseVirtualModelEndpoints()

UseVirtualModelEndpoints sets up virtual model endpoints for testing

func (*Server) ValidateAuthToken 

func (s *Server) ValidateAuthToken(c *gin.Context)

ValidateAuthToken validates an authentication token without requiring auth This is used during login flow to verify a token before establishing session

type ServerActionResponse 

type ServerActionResponse struct {
	Success bool   `json:"success" example:"true"`
	Message string `json:"message" example:"Server stopped successfully"`
}

ServerActionResponse represents the response for server actions (start/stop/restart)

type ServerOption 

type ServerOption func(*Server)

ServerOption defines a functional option for Server configuration

func WithAdaptor 

func WithAdaptor(enabled bool) ServerOption

WithAdaptor enables or disables the adaptor for the server

func WithAuthMiddleware 

func WithAuthMiddleware(userAuth, modelAuth gin.HandlerFunc) ServerOption

WithAuthMiddleware sets custom auth middlewares for WebUI and Model API endpoints This allows TBE to inject its own JWT auth middleware instead of using tingly-box's default UserAuthMiddleware and ModelAuthMiddleware

Usage in TBE: 

server := NewServer(cfg,
    WithAuthMiddleware(tbeUserAuth, tbeModelAuth),
)

func WithDebug 

func WithDebug(enabled bool) ServerOption

WithDebug enables or disables debug mode for the server

func WithDefault 

func WithDefault() ServerOption

WithDefault applies all default server options

func WithExperimentalFeatures 

func WithExperimentalFeatures(features map[string]bool) ServerOption

WithExperimentalFeatures sets the experimental features for the server

func WithGuardrails 

func WithGuardrails(runtime *guardrails.Guardrails) ServerOption

WithGuardrails sets a guardrails runtime for stream evaluation.

func WithHost 

func WithHost(host string) ServerOption

func WithModelAuthMiddleware 

func WithModelAuthMiddleware(modelAuth gin.HandlerFunc) ServerOption

WithModelAuthMiddleware sets a custom model auth middleware for Model API endpoints Use this if you only want to replace ModelAuthMiddleware but keep UserAuthMiddleware

func WithMultiLogger 

func WithMultiLogger(logger *pkgobs.MultiLogger) ServerOption

WithMultiLogger sets the multi-mode logger for the server

func WithOpenBrowser 

func WithOpenBrowser(enabled bool) ServerOption

WithOpenBrowser enables or disables automatic browser opening

func WithRecordDir 

func WithRecordDir(dir string) ServerOption

WithRecordDir sets the scenario-level record directory

func WithRecordMode 

func WithRecordMode(mode obs.RecordMode) ServerOption

WithRecordMode sets the record mode for request/response recording mode: empty string = disabled, "all" = record all, "response" = response only, "scenario" = record scenario only

func WithRecording 

func WithRecording(enabled bool) ServerOption

WithRecording enables dual-stage recording for protocol conversion scenarios

func WithTemplateManager added in v0.260409.1540 

func WithTemplateManager(tm *data.TemplateManager) ServerOption

WithTemplateManager allows TBE to inject a custom TemplateManager. This follows the same pattern as WithAuthMiddleware for consistency.

func WithUI 

func WithUI(enabled bool) ServerOption

WithUI enables or disables the UI for the server

func WithUserAuthMiddleware 

func WithUserAuthMiddleware(userAuth gin.HandlerFunc) ServerOption

WithUserAuthMiddleware sets a custom user auth middleware for WebUI endpoints Use this if you only want to replace UserAuthMiddleware but keep ModelAuthMiddleware

func WithVersion 

func WithVersion(version string) ServerOption

type ServiceHealthResponse 

type ServiceHealthResponse struct {
	Rule   string                 `json:"rule" example:"gpt-4"`
	Health map[string]interface{} `json:"health"`
}

ServiceHealthResponse represents the health check response for services

type ServiceMetric 

type ServiceMetric struct {
	ServiceID            string `json:"service_id" example:"openai:gpt-4"`
	RequestCount         int64  `json:"request_count" example:"100"`
	WindowRequestCount   int64  `json:"window_request_count" example:"50"`
	WindowTokensConsumed int64  `json:"window_tokens_consumed" example:"25000"`
	WindowInputTokens    int64  `json:"window_input_tokens" example:"15000"`
	WindowOutputTokens   int64  `json:"window_output_tokens" example:"10000"`
	LastUsed             string `json:"last_used" example:"2024-01-01T12:00:00Z"`
}

ServiceMetric represents a service metric entry

type ServiceStatsResponse 

type ServiceStatsResponse struct {
	ServiceID string                 `json:"service_id" example:"openai:gpt-4"`
	Stats     map[string]interface{} `json:"stats,omitempty"`
}

ServiceStatsResponse represents the statistics response for a service

type StatusResponse 

type StatusResponse struct {
	Success bool `json:"success" example:"true"`
	Data    struct {
		ServerRunning    bool `json:"server_running" example:"true"`
		Port             int  `json:"port" example:"12580"`
		ProvidersTotal   int  `json:"providers_total" example:"3"`
		ProvidersEnabled int  `json:"providers_enabled" example:"2"`
		RequestCount     int  `json:"request_count" example:"100"`
	} `json:"data"`
}

StatusResponse represents the server status response

type SystemLogEntry 

type SystemLogEntry struct {
	Time    time.Time              `json:"time"`
	Level   string                 `json:"level"`
	Message string                 `json:"message"`
	Fields  map[string]interface{} `json:"fields,omitempty"`
}

SystemLogEntry represents a system log entry for API response

type SystemLogLevelRequest 

type SystemLogLevelRequest struct {
	Level string `json:"level" binding:"required"`
}

SystemLogLevelRequest represents a request to set the log level

type SystemLogsResponse 

type SystemLogsResponse struct {
	Total int              `json:"total"`
	Logs  []SystemLogEntry `json:"logs"`
}

SystemLogsResponse represents the API response for system logs

type ThinkingModeTransform 

type ThinkingModeTransform struct {
	ScenarioConfig *typ.ScenarioConfig
}

ThinkingModeTransform controls how extended thinking is enabled based on scenario config.

Supported modes:

  • Force: always enable thinking with the configured budget.
  • Adaptive: convert any existing thinking config (OfEnabled or OfAdaptive) to OfEnabled.
  • Default: only convert OfEnabled to use the configured budget; leave OfAdaptive untouched.
  • Any other value: disable thinking entirely.

Budget resolution priority: client-provided budget > effort-level mapping > medium fallback. Only added to the chain when ThinkingMode is non-empty.

func NewThinkingModeTransform 

func NewThinkingModeTransform(scenarioConfig *typ.ScenarioConfig) *ThinkingModeTransform

NewThinkingModeTransform creates a ThinkingModeTransform.

func (*ThinkingModeTransform) Apply 

func (t *ThinkingModeTransform) Apply(ctx *protocoltransform.TransformContext) error

func (*ThinkingModeTransform) Name 

func (t *ThinkingModeTransform) Name() string

type ToggleProviderResponse 

type ToggleProviderResponse struct {
	Success bool   `json:"success" example:"true"`
	Message string `json:"message" example:"Provider openai enabled successfully"`
	Data    struct {
		Enabled bool `json:"enabled" example:"true"`
	} `json:"data"`
}

ToggleProviderResponse represents the response for toggling a provider

type TokenCreateRequest added in v0.260418.2200 

type TokenCreateRequest struct {
	DisplayName string `json:"display_name" binding:"required"`
}

TokenCreateRequest represents the request to create a new API token

type TokenCreateResponse added in v0.260418.2200 

type TokenCreateResponse struct {
	Token       string    `json:"token"`
	TokenID     string    `json:"token_id"`
	UserUUID    string    `json:"user_uuid"`
	DisplayName string    `json:"display_name"`
	CreatedAt   time.Time `json:"created_at"`
}

TokenCreateResponse represents the response after creating a new API token

type TokenInfo 

type TokenInfo = oauth.TokenInfo

type TokenListResponse added in v0.260418.2200 

type TokenListResponse struct {
	Tokens []APITokenInfo `json:"tokens"`
	Total  int64          `json:"total"`
}

TokenListResponse represents the response when listing tokens

type TokenResponse 

type TokenResponse struct {
	Token string `json:"token" example:"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..."`
	Type  string `json:"type" example:"Bearer"`
}

TokenResponse represents the token response

type UpdateProviderRequest 

type UpdateProviderRequest struct {
	Name          *string `json:"name,omitempty" description:"New provider name"`
	APIBase       *string `json:"api_base,omitempty" description:"New API base URL"`
	APIStyle      *string `json:"api_style,omitempty" description:"New API style"`
	Token         *string `json:"token,omitempty" description:"New API token"`
	NoKeyRequired *bool   `json:"no_key_required,omitempty" description:"Whether provider requires no API key"`
	Enabled       *bool   `json:"enabled,omitempty" description:"New enabled status"`
	ProxyURL      *string `json:"proxy_url,omitempty" description:"HTTP or SOCKS proxy URL"`
}

UpdateProviderRequest represents the request to update a provider

type UpdateProviderResponse 

type UpdateProviderResponse struct {
	Success bool             `json:"success" example:"true"`
	Message string           `json:"message" example:"Provider updated successfully"`
	Data    ProviderResponse `json:"data"`
}

UpdateProviderResponse represents the response for updating a provider

type UpdateRuleTacticRequest 

type UpdateRuleTacticRequest struct {
	Tactic string `` /* 147-byte string literal not displayed */
}

UpdateRuleTacticRequest represents the request to update rule tactic

type UpdateRuleTacticResponse 

type UpdateRuleTacticResponse struct {
	Message string `json:"message" example:"Tactic updated successfully"`
	Tactic  string `json:"tactic" example:"adaptive"`
}

UpdateRuleTacticResponse represents the response for updating rule tactic

type UsageTracker 

type UsageTracker struct {
	// contains filtered or unexported fields
}

UsageTracker provides usage tracking methods for handlers. It encapsulates the logic for recording token usage to both service stats and detailed usage records.

func (*UsageTracker) RecordUsage 

func (t *UsageTracker) RecordUsage(
	c *gin.Context,
	rule *typ.Rule,
	provider *typ.Provider,
	model, requestModel string,
	inputTokens, outputTokens int,
	streamed bool,
	status, errorCode string,
)

RecordUsage records token usage from a handler. It updates both the service-level stats and the detailed usage records.

Parameters:

  • c: Gin context for accessing request metadata
  • rule: The load balancer rule that was used
  • provider: The provider that handled the request
  • model: The actual model name used (not the requested model)
  • requestModel: The original model name requested by the user
  • inputTokens: Number of input/prompt tokens consumed
  • outputTokens: Number of output/completion tokens consumed
  • streamed: Whether this was a streaming request
  • status: Request status - "success", "error", or "partial"
  • errorCode: Error code if status is not "success"

type ValidationError 

type ValidationError struct {
	Field   string
	Message string
}

ValidationError represents a validation error

func (*ValidationError) Error 

func (e *ValidationError) Error() string

type VersionChecker 

type VersionChecker struct {
	// contains filtered or unexported fields
}

VersionChecker handles version-related operations

func (*VersionChecker) CheckLatestVersion 

func (vc *VersionChecker) CheckLatestVersion() (version, releaseURL string, err error)

CheckLatestVersion checks for the latest version with fallback: 1. npm registry (primary) 2. npmmirror (China mirror, fallback)

type VersionInfo 

type VersionInfo struct {
	Version string `json:"version" example:"1.0.0"`
}

type VersionInfoResponse 

type VersionInfoResponse struct {
	Success bool        `json:"success" example:"true"`
	Message string      `json:"message" example:"Provider models successfully"`
	Data    VersionInfo `json:"data"`
}

Source Files

View all Source files

Directories

Path Synopsis
module
codeximport
configapply
imbot
Package imbotsettings provides handlers for ImBot settings management.
 Package imbotsettings provides handlers for ImBot settings management.
mcp
notify
oauth
provider_quota
providertemplate
rule
scenario
skill
statusline
usage

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.