server

package
v0.260321.800-hotfix Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 21, 2026 License: MPL-2.0 Imports: 67 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// DefaultProbeTimeout is the default timeout for each endpoint probe
	DefaultProbeTimeout = 10 * time.Second
	// DefaultCacheTTL is the default time-to-live for cached probe results
	DefaultCacheTTL = 24 * time.Hour
)
View Source 
const (
	ExperimentalFeatureSmartCompact = "smart_compact"
	ExperimentalFeatureRecording    = "recording"
)

Experimental feature flag names

View Source 
const (
	ContextKeyRule         = "tracking_rule"          // *typ.Rule
	ContextKeyProvider     = "tracking_provider"      // *typ.Provider
	ContextKeyModel        = "tracking_model"         // string (actual model used)
	ContextKeyRequestModel = "tracking_request_model" // string (model requested by user)
	ContextKeyScenario     = "tracking_scenario"      // string (extracted from request path)
	ContextKeyStreamed     = "tracking_streamed"      // bool
	ContextKeyStartTime    = "tracking_start_time"    // time.Time
)

Gin context keys for tracking metadata. These keys are used to store tracking information in the gin context to avoid explicit parameter passing throughout the handler chain.

Variables

This section is empty.

Functions

func ConvertAnthropicToOpenAIResponseWithProvider 

func ConvertAnthropicToOpenAIResponseWithProvider(
	anthropicResp *anthropic.Message,
	responseModel string,
	provider *typ.Provider,
	model string,
) map[string]interface{}

ConvertAnthropicToOpenAIResponseWithProvider converts an Anthropic response to OpenAI format and applies provider-specific transformations to the response

func ForwardAnthropicV1 

func ForwardAnthropicV1(fc *ForwardContext, wrapper *client.AnthropicClient, req anthropic.MessageNewParams) (*anthropic.Message, context.CancelFunc, error)

ForwardAnthropicV1 sends a non-streaming Anthropic v1 message request.

func ForwardAnthropicV1Beta 

func ForwardAnthropicV1Beta(fc *ForwardContext, wrapper *client.AnthropicClient, req anthropic.BetaMessageNewParams) (*anthropic.BetaMessage, context.CancelFunc, error)

ForwardAnthropicV1Beta sends a non-streaming Anthropic v1 beta message request.

func ForwardAnthropicV1BetaStream 

func ForwardAnthropicV1BetaStream(fc *ForwardContext, wrapper *client.AnthropicClient, req anthropic.BetaMessageNewParams) (*anthropicstream.Stream[anthropic.BetaRawMessageStreamEventUnion], context.CancelFunc, error)

ForwardAnthropicV1BetaStream sends a streaming Anthropic v1 beta message request. Note: Set BaseCtx via WithBaseCtx() to support client cancellation.

func ForwardAnthropicV1Stream 

func ForwardAnthropicV1Stream(fc *ForwardContext, wrapper *client.AnthropicClient, req anthropic.MessageNewParams) (*anthropicstream.Stream[anthropic.MessageStreamEventUnion], context.CancelFunc, error)

ForwardAnthropicV1Stream sends a streaming Anthropic v1 message request. Note: Set BaseCtx via WithBaseCtx() to support client cancellation.

func ForwardGoogle 

func ForwardGoogle(fc *ForwardContext, wrapper *client.GoogleClient, model string, contents []*genai.Content, config *genai.GenerateContentConfig) (*genai.GenerateContentResponse, context.CancelFunc, error)

ForwardGoogle sends a non-streaming Google Generative AI request.

func ForwardGoogleStream 

func ForwardGoogleStream(fc *ForwardContext, wrapper *client.GoogleClient, model string, contents []*genai.Content, config *genai.GenerateContentConfig) (iter.Seq2[*genai.GenerateContentResponse, error], context.CancelFunc, error)

ForwardGoogleStream sends a streaming Google Generative AI request. Note: Pass request context (c.Request.Context()) as baseCtx in NewForwardContext for client cancellation support.

func ForwardOpenAIChat 

func ForwardOpenAIChat(fc *ForwardContext, wrapper *client.OpenAIClient, req *openai.ChatCompletionNewParams) (*openai.ChatCompletion, context.CancelFunc, error)

ForwardOpenAIChat sends a non-streaming OpenAI chat completion request.

func ForwardOpenAIChatStream 

func ForwardOpenAIChatStream(fc *ForwardContext, wrapper *client.OpenAIClient, req *openai.ChatCompletionNewParams) (*openaistream.Stream[openai.ChatCompletionChunk], context.CancelFunc, error)

ForwardOpenAIChatStream sends a streaming OpenAI chat completion request. Note: Pass request context (c.Request.Context()) as baseCtx in NewForwardContext for client cancellation support.

func ForwardOpenAIResponses 

func ForwardOpenAIResponses(fc *ForwardContext, wrapper *client.OpenAIClient, params responses.ResponseNewParams) (*responses.Response, context.CancelFunc, error)

ForwardOpenAIResponses sends a non-streaming OpenAI Responses API request.

func ForwardOpenAIResponsesStream 

func ForwardOpenAIResponsesStream(fc *ForwardContext, wrapper *client.OpenAIClient, params responses.ResponseNewParams) (*openaistream.Stream[responses.ResponseStreamEventUnion], context.CancelFunc, error)

ForwardOpenAIResponsesStream sends a streaming OpenAI Responses API request. Note: Pass request context (c.Request.Context()) as baseCtx in NewForwardContext for client cancellation support.

func GenerateCurlCommand 

func GenerateCurlCommand(apiBase, apiStyle, token, model string) string

GenerateCurlCommand generates a curl command for testing the provider

func GetProbeToolChoiceAutoAnthropic 

func GetProbeToolChoiceAutoAnthropic() anthropic.ToolChoiceUnionParam

GetProbeToolChoiceAutoAnthropic returns auto tool choice for testing

func GetProbeToolChoiceOpenAI 

func GetProbeToolChoiceOpenAI() map[string]interface{}

GetProbeToolChoiceOpenAI returns auto tool choice for OpenAI (as JSON map)

func GetProbeToolsAnthropic 

func GetProbeToolsAnthropic() []anthropic.ToolUnionParam

GetProbeToolsAnthropic returns predefined tools in Anthropic format for probe testing

func GetProbeToolsOpenAI 

func GetProbeToolsOpenAI() []openai.ChatCompletionToolUnionParam

GetProbeToolsOpenAI returns predefined tools in OpenAI format (as JSON map)

func GetShutdownChannel 

func GetShutdownChannel() <-chan struct{}

GetShutdownChannel returns the shutdown channel for the main process to listen on

func GetTrackingContext 

func GetTrackingContext(c *gin.Context) (rule *typ.Rule, provider *typ.Provider, actualModel, requestModel, scenario string, streamed bool, startTime time.Time)

GetTrackingContext retrieves tracking metadata from the gin context. Returns zero values if the context keys are not set.

func NewGinHandlerWrapper 

func NewGinHandlerWrapper(h gin.HandlerFunc) swagger.Handler

NewGinHandlerWrapper converts gin.HandlerFunc to swagger.Handler

func NewNonStreamRecorderHook 

func NewNonStreamRecorderHook(recorder *ScenarioRecorder, provider *typ.Provider, model string) func()

NewNonStreamRecorderHook creates a hook for non-streaming responses.

func NewRecorderHooks 

func NewRecorderHooks(recorder *ProtocolRecorder) (onStreamEvent func(event interface{}) error, onStreamComplete func(), onStreamError func(err error))

NewRecorderHooks creates hook functions from a ScenarioRecorder for use with HandleContext. This allows decoupling the recorder from the handle context while maintaining recording functionality. Usage is tracked internally in the event hook, so complete hooks don't need usage parameters.

Returns: - onStreamEvent: Hook for each stream event - onStreamComplete: Hook for stream completion - onStreamError: Hook for stream errors

func NewRecorderHooksWithModel 

func NewRecorderHooksWithModel(recorder *ProtocolRecorder, model string, provider *typ.Provider) (onStreamEvent func(event interface{}) error, onStreamComplete func(), onStreamError func(err error))

NewRecorderHooksWithModel creates hook functions with an explicit model parameter. This is preferred when the model is known at hook creation time. Usage is tracked internally in the event hook.

func RoundtripAnthropicResponseViaOpenAI added in v0.260324.0 

func RoundtripAnthropicResponseViaOpenAI(anthropicResp *anthropic.Message, responseModel string, provider *typ.Provider, actualModel string) (*anthropic.Message, error)

func RoundtripOpenAIMapViaAnthropic 

func RoundtripOpenAIMapViaAnthropic(openaiResp map[string]interface{}, responseModel string, provider *typ.Provider, actualModel string) (map[string]interface{}, error)

func RoundtripOpenAIResponseViaAnthropic 

func RoundtripOpenAIResponseViaAnthropic(openaiResp *openai.ChatCompletion, responseModel string, provider *typ.Provider, actualModel string) (map[string]interface{}, error)

func SSEventOpenAI 

func SSEventOpenAI(c *gin.Context, t string, data any, modelOverride ...string) error

SSEventOpenAI sends an SSE event with the given data If the data is a ResponseStreamEventUnion, it uses the raw JSON to avoid serializing empty fields from the union type If modelOverride is provided and the event contains a response object with a model field, it will be overridden in the JSON output

func SetEnterpriseRateLimitReporter 

func SetEnterpriseRateLimitReporter(reporter func(context.Context, string, string, string, string) error)

SetEnterpriseRateLimitReporter sets callback for enterprise 429 events.

func SetGlobalServer 

func SetGlobalServer(server *Server)

SetGlobalServer sets the global server instance for web UI control

func SetTrackingContext 

func SetTrackingContext(c *gin.Context, rule *typ.Rule, provider *typ.Provider, actualModel, requestModel string, streamed bool)

SetTrackingContext sets all tracking metadata in the gin context. This should be called once at the beginning of request processing to avoid explicit parameter passing throughout the handler chain.

Parameters:

  • c: Gin context
  • rule: The load balancer rule that was selected
  • provider: The provider that was selected
  • actualModel: The actual model name used (may differ from requested)
  • requestModel: The original model name requested by the user
  • streamed: Whether this is a streaming request

func ShouldRoundtripResponse 

func ShouldRoundtripResponse(c *gin.Context, target string) bool

Types

type ActionHistoryEntry 

type ActionHistoryEntry struct {
	Time    time.Time              `json:"time"`
	Level   string                 `json:"level"`
	Message string                 `json:"message"`
	Action  string                 `json:"action,omitempty"`
	Success bool                   `json:"success,omitempty"`
	Details interface{}            `json:"details,omitempty"`
	Fields  map[string]interface{} `json:"fields,omitempty"`
}

ActionHistoryEntry represents an action history entry for API response

type ActionHistoryResponse 

type ActionHistoryResponse struct {
	Total   int                  `json:"total"`
	Actions []ActionHistoryEntry `json:"actions"`
}

ActionHistoryResponse represents the API response for action history

type AdaptiveProbe 

type AdaptiveProbe struct {
	// contains filtered or unexported fields
}

AdaptiveProbe handles concurrent endpoint probing for model capabilities

func NewAdaptiveProbe 

func NewAdaptiveProbe(s *Server) *AdaptiveProbe

NewAdaptiveProbe creates a new adaptive probe instance

func (*AdaptiveProbe) GetModelCapability 

func (ap *AdaptiveProbe) GetModelCapability(providerUUID, modelID string) (*ModelEndpointCapability, error)

GetModelCapability retrieves cached capability for a model, or triggers a probe if not cached

func (*AdaptiveProbe) GetPreferredEndpoint 

func (ap *AdaptiveProbe) GetPreferredEndpoint(provider *typ.Provider, modelID string) string

GetPreferredEndpoint returns the preferred endpoint for a model

func (*AdaptiveProbe) InvalidateProviderCache 

func (ap *AdaptiveProbe) InvalidateProviderCache(providerUUID string)

InvalidateProviderCache invalidates all cached capabilities for a provider

func (*AdaptiveProbe) ProbeModelEndpoints 

func (ap *AdaptiveProbe) ProbeModelEndpoints(ctx context.Context, req ModelProbeRequest) (*ProbeResult, error)

ProbeModelEndpoints probes both chat and responses endpoints concurrently for a model

func (*AdaptiveProbe) ProbeProviderModels 

func (ap *AdaptiveProbe) ProbeProviderModels(ctx context.Context, provider *typ.Provider, models []string) map[string]*ProbeResult

ProbeProviderModels probes all models for a provider concurrently

type AllStatsResponse 

type AllStatsResponse struct {
	Stats map[string]interface{} `json:"stats"`
}

AllStatsResponse represents the response for all statistics

type AnthropicModel 

type AnthropicModel struct {
	ID          string `json:"id"`
	CreatedAt   string `json:"created_at"`
	DisplayName string `json:"display_name"`
	Type        string `json:"type"`
}

AnthropicModel Model types - based on Anthropic's official models API format

type AnthropicModelsResponse 

type AnthropicModelsResponse struct {
	Data    []AnthropicModel `json:"data"`
	FirstID string           `json:"first_id"`
	HasMore bool             `json:"has_more"`
	LastID  string           `json:"last_id"`
}

type CachedModelCapability 

type CachedModelCapability struct {
	Capability ModelEndpointCapability
	ExpiresAt  time.Time
}

CachedModelCapability represents a cached model endpoint capability with expiration

type ClearStatsResponse 

type ClearStatsResponse struct {
	Message string `json:"message" example:"Statistics cleared for rule: gpt-4"`
}

ClearStatsResponse represents the response for clearing statistics

type ConfigInfo 

type ConfigInfo struct {
	ConfigPath string `json:"config_path" example:"/Users/user/.tingly-box/config.json"`
	ConfigDir  string `json:"config_dir" example:"/Users/user/.tingly-box"`
}

ConfigInfo represents configuration information

type ConfigInfoResponse 

type ConfigInfoResponse struct {
	Success bool       `json:"success" example:"true"`
	Data    ConfigInfo `json:"data"`
}

ConfigInfoResponse represents the response for config info endpoint

type CreateProviderRequest 

type CreateProviderRequest struct {
	Name          string `json:"name" binding:"required" description:"Provider name" example:"openai"`
	APIBase       string `json:"api_base" binding:"required" description:"API base URL" example:"https://api.openai.com/v1"`
	APIStyle      string `json:"api_style" description:"API style" example:"openai"`
	Token         string `json:"token" description:"API token" example:"sk-..."`
	NoKeyRequired bool   `json:"no_key_required" description:"Whether provider requires no API key" example:"false"`
	Enabled       bool   `json:"enabled" description:"Whether provider is enabled" example:"true"`
	ProxyURL      string `` /* 153-byte string literal not displayed */
	AuthType      string `json:"auth_type,omitempty" description:"Auth type: api_key or oauth (default: api_key)" example:"api_key"`
}

CreateProviderRequest represents the request to add a new provider

type CreateProviderResponse 

type CreateProviderResponse struct {
	Success bool        `json:"success" example:"true"`
	Message string      `json:"message" example:"Provider added successfully"`
	Data    interface{} `json:"data"`
}

CreateProviderResponse represents the response for adding a provider

type CurrentServiceResponse 

type CurrentServiceResponse struct {
	Rule      string                 `json:"rule" example:"gpt-4"`
	Service   interface{}            `json:"service"`
	ServiceID string                 `json:"service_id" example:"openai:gpt-4"`
	Tactic    string                 `json:"tactic" example:"round_robin"`
	Stats     map[string]interface{} `json:"stats,omitempty"`
}

CurrentServiceResponse represents the current service response

type DeleteProviderResponse 

type DeleteProviderResponse struct {
	Success bool   `json:"success" example:"true"`
	Message string `json:"message" example:"Provider deleted successfully"`
}

DeleteProviderResponse represents the response for deleting a provider

type EndpointProbeStatus 

type EndpointProbeStatus struct {
	Available    bool   `json:"available" example:"true"`
	LatencyMs    int    `json:"latency_ms" example:"234"`
	ErrorMessage string `json:"error_message,omitempty" example:""`
	LastChecked  string `json:"last_checked" example:"2026-01-23T10:30:00Z"`
}

EndpointProbeStatus represents the status of an endpoint probe

type EndpointStatus 

type EndpointStatus struct {
	Available    bool
	LatencyMs    int
	ErrorMessage string
	LastChecked  time.Time
}

EndpointStatus represents the status of a single endpoint

type ErrorDetail 

type ErrorDetail struct {
	Message string `json:"message"`
	Type    string `json:"type"`
	Code    string `json:"code,omitempty"`
}

ErrorDetail represents error details

type ErrorResponse 

type ErrorResponse struct {
	Error ErrorDetail `json:"error"`
}

ErrorResponse represents an error response

type FetchProviderModelsResponse 

type FetchProviderModelsResponse struct {
	Success bool        `json:"success" example:"true"`
	Message string      `json:"message" example:"Successfully fetched 150 models for provider openai"`
	Data    interface{} `json:"data"`
}

FetchProviderModelsResponse represents the response for fetching provider models

type ForwardContext 

type ForwardContext struct {
	// Required dependencies
	Provider *typ.Provider
	BaseCtx  context.Context // Base context (e.g., request context for cancellation support)

	// Optional configuration
	Timeout time.Duration

	// Hooks (chainable - multiple hooks can be added)
	BeforeRequestHooks []func(ctx context.Context, req interface{}) (context.Context, error)
	AfterRequestHooks  []func(ctx context.Context, resp interface{}, err error)
}

ForwardContext provides dependencies for forward functions. It uses the builder pattern for optional configuration and hooks.

func NewForwardContext 

func NewForwardContext(baseCtx context.Context, provider *typ.Provider) *ForwardContext

NewForwardContext creates a new ForwardContext with required dependencies. The timeout is set to the provider's default timeout. baseCtx is the base context for the request:

  • Use context.Background() for non-streaming requests
  • Use c.Request.Context() for streaming requests to support client cancellation

func (*ForwardContext) Complete 

func (fc *ForwardContext) Complete(ctx context.Context, resp interface{}, err error)

Complete calls all AfterRequest hooks (if set) with the response and error. Hooks are called in the order they were added. This should be called after the request completes, regardless of success or failure.

func (*ForwardContext) PrepareContext 

func (fc *ForwardContext) PrepareContext(req interface{}) (context.Context, context.CancelFunc)

PrepareContext prepares the final context for the request. It applies the BeforeRequest hooks and adds the scenario to the context. It also sets up the timeout and returns a cancel function. If BaseCtx is not set, it uses context.Background() as the base.

The order of operations matches the original implementation: 1. Apply BeforeRequest hooks 2. Add timeout

func (*ForwardContext) WithAfterRequest 

func (fc *ForwardContext) WithAfterRequest(hook func(context.Context, interface{}, error)) *ForwardContext

WithAfterRequest adds a hook that is called after the request completes. Multiple hooks can be added and will be called in order. Each hook receives the response and any error that occurred.

func (*ForwardContext) WithBeforeRequest 

func (fc *ForwardContext) WithBeforeRequest(hook func(context.Context, interface{}) (context.Context, error)) *ForwardContext

WithBeforeRequest adds a hook that is called before the request is sent. Multiple hooks can be added and will be called in order. Each hook can modify the context and return an error to abort the request.

func (*ForwardContext) WithTimeout 

func (fc *ForwardContext) WithTimeout(timeout time.Duration) *ForwardContext

WithTimeout sets the timeout for the request. If not set, the provider's default timeout is used.

type GenerateTokenRequest 

type GenerateTokenRequest struct {
	ClientID string `json:"client_id" binding:"required" description:"Client ID for token generation" example:"user123"`
}

GenerateTokenRequest represents the request to generate a token

type HealthInfoResponse 

type HealthInfoResponse struct {
	Status  string `json:"status" example:"healthy"`
	Service string `json:"service" example:"tingly-box"`
	Health  bool   `json:"health" example:"healthy"`
}

HealthInfoResponse represents the health check response

type HistoryResponse 

type HistoryResponse struct {
	Success bool        `json:"success" example:"true"`
	Data    interface{} `json:"data"`
}

HistoryResponse represents the response for request history

type LatestVersionInfo 

type LatestVersionInfo struct {
	CurrentVersion string `json:"current_version" example:"0.260124.1430"`
	LatestVersion  string `json:"latest_version" example:"0.260130.1200"`
	HasUpdate      bool   `json:"has_update" example:"true"`
	ReleaseURL     string `json:"release_url" example:"https://github.com/tingly-dev/tingly-box/releases/tag/v0.260130.1200"`
	ShouldNotify   bool   `json:"should_notify" example:"true"`
}

LatestVersionInfo contains version comparison information

type LatestVersionResponse 

type LatestVersionResponse struct {
	Success bool              `json:"success"`
	Error   string            `json:"error,omitempty"`
	Data    LatestVersionInfo `json:"data,omitempty"`
}

LatestVersionResponse represents the response for version check endpoint

type LoadBalancer 

type LoadBalancer struct {
	// contains filtered or unexported fields
}

LoadBalancer manages load balancing across multiple services

func NewLoadBalancer 

func NewLoadBalancer(cfg *config.Config, healthFilter *typ.HealthFilter) *LoadBalancer

NewLoadBalancer creates a new load balancer

func (*LoadBalancer) ClearAllStats 

func (lb *LoadBalancer) ClearAllStats()

ClearAllStats clears all statistics (both in-memory and persisted in config)

func (*LoadBalancer) ClearServiceStats 

func (lb *LoadBalancer) ClearServiceStats(provider, model string)

ClearServiceStats clears statistics for a specific service

func (*LoadBalancer) GetAllServiceStats 

func (lb *LoadBalancer) GetAllServiceStats() map[string]*loadbalance.ServiceStats

GetAllServiceStats returns all service statistics from all active rules. Stats are keyed by provider:model since stats are global (shared across rules).

func (*LoadBalancer) GetRuleSummary 

func (lb *LoadBalancer) GetRuleSummary(rule *typ.Rule) map[string]interface{}

GetRuleSummary returns a summary of rule configuration and statistics

func (*LoadBalancer) GetServiceStats 

func (lb *LoadBalancer) GetServiceStats(provider, model string) *loadbalance.ServiceStats

GetServiceStats returns statistics for a specific service

func (*LoadBalancer) HealthFilter 

func (lb *LoadBalancer) HealthFilter() *typ.HealthFilter

HealthFilter returns the health filter for the load balancer

func (*LoadBalancer) RecordUsage 

func (lb *LoadBalancer) RecordUsage(provider, model string, inputTokens, outputTokens int)

RecordUsage records usage for a service Deprecated: This method is no longer needed as usage is recorded directly in handlers

func (*LoadBalancer) RegisterTactic 

func (lb *LoadBalancer) RegisterTactic(tacticType loadbalance.TacticType, tactic typ.LoadBalancingTactic)

RegisterTactic registers a custom tactic

func (*LoadBalancer) SelectService 

func (lb *LoadBalancer) SelectService(rule *typ.Rule) (*loadbalance.Service, error)

SelectService selects the best service for a rule based on the configured tactic

func (*LoadBalancer) Stop 

func (lb *LoadBalancer) Stop()

Stop stops the load balancer and cleanup resources

func (*LoadBalancer) UpdateServiceIndex 

func (lb *LoadBalancer) UpdateServiceIndex(rule *typ.Rule, selectedService *loadbalance.Service)

UpdateServiceIndex updates the current service ID for a rule

func (*LoadBalancer) ValidateRule 

func (lb *LoadBalancer) ValidateRule(rule *typ.Rule) error

ValidateRule validates a rule configuration

type LoadBalancerAPI 

type LoadBalancerAPI struct {
	// contains filtered or unexported fields
}

LoadBalancerAPI provides REST endpoints for load balancer management

func NewLoadBalancerAPI 

func NewLoadBalancerAPI(loadBalancer *LoadBalancer, cfg *config.Config) *LoadBalancerAPI

NewLoadBalancerAPI creates a new load balancer API

func (*LoadBalancerAPI) ClearAllStats 

func (api *LoadBalancerAPI) ClearAllStats(c *gin.Context)

ClearAllStats clears all statistics

func (*LoadBalancerAPI) ClearRuleStats 

func (api *LoadBalancerAPI) ClearRuleStats(c *gin.Context)

ClearRuleStats clears statistics for all services in a rule

func (*LoadBalancerAPI) ClearServiceStats 

func (api *LoadBalancerAPI) ClearServiceStats(c *gin.Context)

ClearServiceStats clears statistics for a specific service

func (*LoadBalancerAPI) GetAllStats 

func (api *LoadBalancerAPI) GetAllStats(c *gin.Context)

GetAllStats returns statistics for all services

func (*LoadBalancerAPI) GetCurrentService 

func (api *LoadBalancerAPI) GetCurrentService(c *gin.Context)

GetCurrentService returns the currently active service for a rule

func (*LoadBalancerAPI) GetMetrics 

func (api *LoadBalancerAPI) GetMetrics(c *gin.Context)

GetMetrics returns load balancing metrics

func (*LoadBalancerAPI) GetRule 

func (api *LoadBalancerAPI) GetRule(c *gin.Context)

GetRule returns a specific rule configuration

func (*LoadBalancerAPI) GetRuleStats 

func (api *LoadBalancerAPI) GetRuleStats(c *gin.Context)

GetRuleStats returns statistics for all services in a rule

func (*LoadBalancerAPI) GetRuleSummary 

func (api *LoadBalancerAPI) GetRuleSummary(c *gin.Context)

GetRuleSummary returns a comprehensive summary of a rule including statistics

func (*LoadBalancerAPI) GetServiceHealth 

func (api *LoadBalancerAPI) GetServiceHealth(c *gin.Context)

GetServiceHealth checks the health of all services in a rule

func (*LoadBalancerAPI) GetServiceStats 

func (api *LoadBalancerAPI) GetServiceStats(c *gin.Context)

GetServiceStats returns statistics for a specific service

func (*LoadBalancerAPI) GetServicesHealth 

func (api *LoadBalancerAPI) GetServicesHealth(c *gin.Context)

GetServicesHealth returns health status for all services in a rule

func (*LoadBalancerAPI) RegisterRoutes 

func (api *LoadBalancerAPI) RegisterRoutes(loadBalancer *gin.RouterGroup)

RegisterRoutes registers the load balancer API routes

func (*LoadBalancerAPI) ResetServiceHealth 

func (api *LoadBalancerAPI) ResetServiceHealth(c *gin.Context)

ResetServiceHealth manually resets a service's health to healthy

func (*LoadBalancerAPI) UpdateRuleTactic 

func (api *LoadBalancerAPI) UpdateRuleTactic(c *gin.Context)

UpdateRuleTactic updates the load balancing tactic for a rule

type LogEntry 

type LogEntry struct {
	Time    time.Time              `json:"time"`
	Level   string                 `json:"level"`
	Message string                 `json:"message"`
	Data    map[string]interface{} `json:"data,omitempty"`
	Fields  map[string]interface{} `json:"fields,omitempty"`
}

LogEntry represents a log entry for API response

type LogsResponse 

type LogsResponse struct {
	Total int        `json:"total"`
	Logs  []LogEntry `json:"logs"`
}

LogsResponse represents the API response for logs

type MetricsResponse 

type MetricsResponse struct {
	Metrics       []ServiceMetric `json:"metrics"`
	TotalServices int             `json:"total_services" example:"5"`
}

MetricsResponse represents the metrics response

type ModelEndpointCapability 

type ModelEndpointCapability struct {
	ProviderUUID       string
	ModelID            string
	SupportsChat       bool
	ChatLatencyMs      int
	ChatError          string
	SupportsResponses  bool
	ResponsesLatencyMs int
	ResponsesError     string
	PreferredEndpoint  string // "chat", "responses", or ""
	LastVerified       time.Time
}

ModelEndpointCapability represents the endpoint capability information for a model

type ModelProbeData 

type ModelProbeData struct {
	ProviderUUID      string              `json:"provider_uuid" example:"uuid-123"`
	ModelID           string              `json:"model_id" example:"gpt-4"`
	ChatEndpoint      EndpointProbeStatus `json:"chat_endpoint"`
	ResponsesEndpoint EndpointProbeStatus `json:"responses_endpoint"`
	PreferredEndpoint string              `json:"preferred_endpoint" example:"responses"`
	LastUpdated       string              `json:"last_updated" example:"2026-01-23T10:30:00Z"`
}

ModelProbeData represents the probe result data

type ModelProbeRequest 

type ModelProbeRequest struct {
	ProviderUUID string `json:"provider_uuid" binding:"required" description:"Provider UUID to probe" example:"uuid-123"`
	ModelID      string `json:"model_id" binding:"required" description:"Model ID to probe" example:"gpt-4"`
	ForceRefresh bool   `json:"force_refresh" description:"Force new probe even if cached" example:"false"`
}

ModelProbeRequest represents the request to probe a specific model

type ModelProbeResponse 

type ModelProbeResponse struct {
	Success bool            `json:"success" example:"true"`
	Error   *ErrorDetail    `json:"error,omitempty"`
	Data    *ModelProbeData `json:"data,omitempty"`
}

ModelProbeResponse represents the response from model endpoint probing

type NpmPackage 

type NpmPackage struct {
	Name     string `json:"name"`
	DistTags struct {
		Latest string `json:"latest"`
	} `json:"dist-tags"`
}

NpmPackage represents an npm registry package response

type OAuthAuthorizeRequest 

type OAuthAuthorizeRequest = oauth.OAuthAuthorizeRequest

type OAuthAuthorizeResponse 

type OAuthAuthorizeResponse = oauth.OAuthAuthorizeResponse

type OAuthCallbackDataResponse 

type OAuthCallbackDataResponse = oauth.OAuthCallbackDataResponse

type OAuthCancelRequest 

type OAuthCancelRequest = oauth.OAuthCancelRequest

type OAuthDeviceCodeResponse 

type OAuthDeviceCodeResponse = oauth.OAuthDeviceCodeResponse

type OAuthErrorResponse 

type OAuthErrorResponse = oauth.OAuthErrorResponse

type OAuthMessageResponse 

type OAuthMessageResponse = oauth.OAuthMessageResponse

type OAuthProviderDataResponse 

type OAuthProviderDataResponse = oauth.OAuthProviderDataResponse

type OAuthProviderInfo 

type OAuthProviderInfo = oauth.OAuthProviderInfo

OAuth type aliases for backward compatibility These allow existing code to continue using the old type names while the actual implementation is now in the oauth module

type OAuthProvidersResponse 

type OAuthProvidersResponse = oauth.OAuthProvidersResponse

type OAuthRefreshTokenRequest 

type OAuthRefreshTokenRequest = oauth.OAuthRefreshTokenRequest

type OAuthRefreshTokenResponse 

type OAuthRefreshTokenResponse = oauth.OAuthRefreshTokenResponse

type OAuthSessionStatusResponse 

type OAuthSessionStatusResponse = oauth.OAuthSessionStatusResponse

type OAuthTokenResponse 

type OAuthTokenResponse = oauth.OAuthTokenResponse

type OAuthTokensResponse 

type OAuthTokensResponse = oauth.OAuthTokensResponse

type OAuthUpdateProviderRequest 

type OAuthUpdateProviderRequest = oauth.OAuthUpdateProviderRequest

type OAuthUpdateProviderResponse 

type OAuthUpdateProviderResponse = oauth.OAuthUpdateProviderResponse

type OpenAIChatCompletionResponse 

type OpenAIChatCompletionResponse struct {
	ID      string `json:"id" example:"chatcmpl-123"`
	Object  string `json:"object" example:"chat.completion"`
	Created int64  `json:"created" example:"1677652288"`
	Model   string `json:"model" example:"gpt-3.5-turbo"`
	Choices []struct {
		Index   int `json:"index" example:"0"`
		Message struct {
			Role    string `json:"role" example:"assistant"`
			Content string `json:"content" example:"Hello! How can I help you?"`
		} `json:"message"`
		FinishReason string `json:"finish_reason" example:"stop"`
	} `json:"choices"`
	Usage struct {
		PromptTokens     int `json:"prompt_tokens" example:"10"`
		CompletionTokens int `json:"completion_tokens" example:"20"`
		TotalTokens      int `json:"total_tokens" example:"30"`
	} `json:"usage"`
}

OpenAIChatCompletionResponse represents the OpenAI chat completion response

type OpenAIModel 

type OpenAIModel struct {
	ID      string `json:"id"`
	Object  string `json:"object"`
	Created int64  `json:"created"`
	OwnedBy string `json:"owned_by"`
}

OpenAIModel represents a model in OpenAI's models API format

type OpenAIModelsResponse 

type OpenAIModelsResponse struct {
	Object string        `json:"object"`
	Data   []OpenAIModel `json:"data"`
}

OpenAIModelsResponse represents OpenAI's models API response format

type PassthroughHandler 

type PassthroughHandler struct {
	// contains filtered or unexported fields
}

PassthroughHandler handles pass-through requests It replaces the model name and proxies the request without any transformations

func NewPassthroughHandler 

func NewPassthroughHandler(server *Server) *PassthroughHandler

NewPassthroughHandler creates a new pass-through handler

type PostTransformRecorder 

type PostTransformRecorder struct {
	// contains filtered or unexported fields
}

PostTransformRecorder captures the request after base transformation This records the request after protocol conversion (e.g., Anthropic → OpenAI)

func NewPostTransformRecorder 

func NewPostTransformRecorder(recorder interface{}, c *gin.Context) *PostTransformRecorder

NewPostTransformRecorder creates a new PostTransformRecorder

func (*PostTransformRecorder) Apply 

func (t *PostTransformRecorder) Apply(ctx *transform.TransformContext) error

Apply captures the request after base transformation The transformed request is in ctx.Request (after BaseTransform)

func (*PostTransformRecorder) Name 

func (t *PostTransformRecorder) Name() string

Name returns the name of this transform

type PreTransformRecorder 

type PreTransformRecorder struct {
	// contains filtered or unexported fields
}

PreTransformRecorder captures the original request before any transformations This is the first transform in the chain, recording the request as received from the client

func NewPreTransformRecorder 

func NewPreTransformRecorder(c *gin.Context, recorder interface{}) *PreTransformRecorder

NewPreTransformRecorder creates a new PreTransformRecorder

func (*PreTransformRecorder) Apply 

func (t *PreTransformRecorder) Apply(ctx *transform.TransformContext) error

Apply captures the original request before any transformations The original request is stored in ctx.OriginalRequest

func (*PreTransformRecorder) Name 

func (t *PreTransformRecorder) Name() string

Name returns the name of this transform

type ProbeCache 

type ProbeCache struct {
	// contains filtered or unexported fields
}

ProbeCache provides in-memory caching for model endpoint capability probe results

func NewProbeCache 

func NewProbeCache(ttl time.Duration) *ProbeCache

NewProbeCache creates a new probe cache with the specified TTL

func (*ProbeCache) CleanupExpired 

func (pc *ProbeCache) CleanupExpired()

CleanupExpired removes expired entries from cache

func (*ProbeCache) Clear 

func (pc *ProbeCache) Clear()

Clear removes all cached entries

func (*ProbeCache) Get 

func (pc *ProbeCache) Get(providerUUID, modelID string) *ModelEndpointCapability

Get retrieves cached capability for a model

func (*ProbeCache) Invalidate 

func (pc *ProbeCache) Invalidate(providerUUID, modelID string)

Invalidate removes cached capability for a specific model

func (*ProbeCache) InvalidateProvider 

func (pc *ProbeCache) InvalidateProvider(providerUUID string)

InvalidateProvider removes all cached capabilities for a provider

func (*ProbeCache) Set 

func (pc *ProbeCache) Set(providerUUID, modelID string, capability *ModelEndpointCapability)

Set stores capability in cache

func (*ProbeCache) SetFromProbeResult 

func (pc *ProbeCache) SetFromProbeResult(result *ProbeResult)

SetFromProbeResult stores probe result in cache

func (*ProbeCache) StartCleanupTask 

func (pc *ProbeCache) StartCleanupTask(interval time.Duration)

StartCleanupTask starts a background task to periodically clean up expired entries

type ProbeCacheRequest 

type ProbeCacheRequest struct {
	ProviderUUID string
	ModelID      string
	ForceRefresh bool // Force new probe even if cached
}

ProbeCacheRequest represents a request to probe a model

type ProbeMode 

type ProbeMode string

ProbeMode defines the test mode 

const (
	ProbeV2ModeSimple    ProbeMode = "simple"
	ProbeV2ModeStreaming ProbeMode = "streaming"
	ProbeV2ModeTool      ProbeMode = "tool"
)

type ProbeProviderRequest 

type ProbeProviderRequest struct {
	Name     string `json:"name" binding:"required" description:"Provider name" example:"openai"`
	APIBase  string `json:"api_base" binding:"required" description:"API base URL" example:"https://api.openai.com/v1"`
	APIStyle string `json:"api_style" binding:"required,oneof=openai anthropic" description:"API style" example:"openai"`
	Token    string `json:"token" binding:"required" description:"API token to test" example:"sk-..."`
}

ProbeProviderRequest represents the request to probe/test a provider's API key and connectivity

type ProbeProviderResponse 

type ProbeProviderResponse struct {
	Success bool                       `json:"success" example:"true"`
	Error   *ErrorDetail               `json:"error,omitempty"`
	Data    *ProbeProviderResponseData `json:"data,omitempty"`
}

ProbeProviderResponse represents the response from provider probing

type ProbeProviderResponseData 

type ProbeProviderResponseData struct {
	Provider     string `json:"provider" example:"openai"`
	APIBase      string `json:"api_base" example:"https://api.openai.com/v1"`
	APIStyle     string `json:"api_style" example:"openai"`
	Valid        bool   `json:"valid" example:"true"`
	Message      string `json:"message" example:"API key is valid and accessible"`
	TestResult   string `json:"test_result" example:"models_endpoint_success"`
	ResponseTime int64  `json:"response_time_ms" example:"250"`
	ModelsCount  int    `json:"models_count,omitempty" example:"150"`
}

ProbeProviderResponseData represents the data returned from provider probing

type ProbeRequest 

type ProbeRequest struct {
	Provider string `json:"provider" binding:"required" description:"Provider UUID to test against" example:"550e8400-e29b-41d4-a716-446655440000"`
	Model    string `json:"model" binding:"required" description:"Model name to test against" example:"gpt-4-latest"`
}

ProbeRequest represents the request to probe/test a provider and model

type ProbeRequestDetail 

type ProbeRequestDetail struct {
	Messages    []map[string]interface{} `json:"messages"`
	Model       string                   `json:"model"`
	MaxTokens   int                      `json:"max_tokens"`
	Temperature float64                  `json:"temperature"`
	Provider    string                   `json:"provider"`
	Timestamp   string                   `json:"timestamp"`
}

ProbeRequestDetail represents the mock request data for probing

func NewMockRequest 

func NewMockRequest(provider, model string) ProbeRequestDetail

NewMockRequest creates a new mock request with default values

type ProbeResponse 

type ProbeResponse struct {
	Success bool               `json:"success"`
	Error   *ErrorDetail       `json:"error,omitempty"`
	Data    *ProbeResponseData `json:"data,omitempty"`
}

ProbeResponse represents the overall probe response

type ProbeResponseData 

type ProbeResponseData struct {
	Request     ProbeRequestDetail  `json:"request"`
	Response    ProbeResponseDetail `json:"response"`
	Usage       ProbeUsage          `json:"usage"`
	CurlCommand string              `json:"curl_command,omitempty"`
}

ProbeResponseData represents the response data structure

type ProbeResponseDetail 

type ProbeResponseDetail struct {
	Content      string `json:"content"`
	Model        string `json:"model"`
	Provider     string `json:"provider"`
	FinishReason string `json:"finish_reason"`
	Error        string `json:"error,omitempty"`
}

ProbeResponseDetail represents the API response

type ProbeResult 

type ProbeResult struct {
	ProviderUUID      string
	ModelID           string
	ChatEndpoint      EndpointStatus
	ResponsesEndpoint EndpointStatus
	PreferredEndpoint string
	LastUpdated       time.Time
}

ProbeResult represents the complete probe result for a model

type ProbeTarget 

type ProbeTarget string

ProbeTarget defines the target type for probe 

const (
	ProbeV2TargetRule     ProbeTarget = "rule"
	ProbeV2TargetProvider ProbeTarget = "provider"
)

type ProbeUsage 

type ProbeUsage struct {
	PromptTokens     int `json:"prompt_tokens"`
	CompletionTokens int `json:"completion_tokens"`
	TotalTokens      int `json:"total_tokens"`
	TimeCost         int `json:"time_cost"`
}

ProbeUsage represents token usage information

type ProbeV2Data 

type ProbeV2Data struct {
	Content    string            `json:"content,omitempty"`
	ToolCalls  []ProbeV2ToolCall `json:"tool_calls,omitempty"`
	Usage      *ProbeV2Usage     `json:"usage,omitempty"`
	LatencyMs  int64             `json:"latency_ms"`
	RequestURL string            `json:"request_url,omitempty"`
}

ProbeV2Data represents the probe result data

type ProbeV2Request 

type ProbeV2Request struct {
	// Target type: rule or provider
	TargetType ProbeTarget `json:"target_type" binding:"required"`

	// Rule test (required when target_type is rule)
	Scenario string `json:"scenario,omitempty" example:"anthropic"`
	RuleUUID string `json:"rule_uuid,omitempty" binding:"required_if=TargetType rule"`

	// Provider test (required when target_type is provider)
	ProviderUUID string `json:"provider_uuid,omitempty" binding:"required_if=TargetType provider"`
	Model        string `json:"model,omitempty" binding:"required_if=TargetType provider"`

	// Test mode
	TestMode ProbeMode `json:"test_mode" binding:"required"`

	// Optional custom message (overrides preset)
	Message string `json:"message,omitempty"`
}

ProbeV2Request represents a Probe V3 request

type ProbeV2Response 

type ProbeV2Response struct {
	Success bool         `json:"success"`
	Error   *ErrorDetail `json:"error,omitempty"`
	Data    *ProbeV2Data `json:"data,omitempty"`
}

ProbeV2Response represents a Probe V3 response

type ProbeV2ResponseChunk 

type ProbeV2ResponseChunk struct {
	Type      string           `json:"type"` // content, tool_call, error, done
	Content   string           `json:"content,omitempty"`
	ToolCall  *ProbeV2ToolCall `json:"tool_call,omitempty"`
	Error     string           `json:"error,omitempty"`
	Usage     *ProbeV2Usage    `json:"usage,omitempty"`
	LatencyMs int64            `json:"latency_ms,omitempty"`
}

ProbeV2ResponseChunk represents a streaming response chunk

type ProbeV2Service 

type ProbeV2Service struct {
	// contains filtered or unexported fields
}

ProbeV2Service handles probe V3 operations

func NewProbeV2Service 

func NewProbeV2Service(server *Server) *ProbeV2Service

NewProbeV2Service creates a new Probe V3 service

type ProbeV2ToolCall 

type ProbeV2ToolCall struct {
	ID        string                 `json:"id"`
	Name      string                 `json:"name"`
	Arguments map[string]interface{} `json:"arguments"`
}

ProbeV2ToolCall represents a tool call in the response

type ProbeV2Usage 

type ProbeV2Usage struct {
	PromptTokens     int `json:"prompt_tokens"`
	CompletionTokens int `json:"completion_tokens"`
	TotalTokens      int `json:"total_tokens"`
}

ProbeV2Usage represents token usage

type ProtocolRecorder 

type ProtocolRecorder struct {
	*ScenarioRecorder
	// contains filtered or unexported fields
}

ProtocolRecorder extends ScenarioRecorder for dual-stage recording It provides support for capturing both original and transformed requests, as well as provider and final responses (for protocol conversion scenarios)

func NewScenarioRecorderV2 

func NewScenarioRecorderV2(recorder *ScenarioRecorder, provider *typ.Provider, model string, mode obs.RecordMode) *ProtocolRecorder

NewScenarioRecorderV2 creates a new ProtocolRecorder from an existing ScenarioRecorder

func (*ProtocolRecorder) Record 

func (sr *ProtocolRecorder) Record()

Record writes the V2 record entry to the sink

func (*ProtocolRecorder) RecordError 

func (sr *ProtocolRecorder) RecordError(err error)

RecordError records an error for V2 entries

func (*ProtocolRecorder) SetFinalResponse 

func (sr *ProtocolRecorder) SetFinalResponse(resp *obs.RecordResponse)

SetFinalResponse stores the final response to the client

func (*ProtocolRecorder) SetOriginalRequest 

func (sr *ProtocolRecorder) SetOriginalRequest(req *obs.RecordRequest)

SetOriginalRequest stores the pre-transform request

func (*ProtocolRecorder) SetProviderResponse 

func (sr *ProtocolRecorder) SetProviderResponse(resp *obs.RecordResponse)

SetProviderResponse stores the raw response from the provider

func (*ProtocolRecorder) SetTransformSteps 

func (sr *ProtocolRecorder) SetTransformSteps(steps []string)

SetTransformSteps records the transform steps that were applied

func (*ProtocolRecorder) SetTransformedRequest 

func (sr *ProtocolRecorder) SetTransformedRequest(req *obs.RecordRequest)

SetTransformedRequest stores the post-transform request

type ProviderModelInfo 

type ProviderModelInfo struct {
	Models      []string `json:"models" example:"gpt-3.5-turbo,gpt-4"`
	StarModels  []string `json:"star_models" example:"gpt-4"`
	CustomModel []string `json:"custom_model" example:"custom-gpt-model"`
	APIBase     string   `json:"api_base" example:"https://api.openai.com/v1"`
	LastUpdated string   `json:"last_updated,omitempty" example:"2024-01-15 10:30:00"`
}

ProviderModelInfo represents model information for a specific provider

type ProviderModelsResponse 

type ProviderModelsResponse struct {
	Success bool              `json:"success" example:"true"`
	Message string            `json:"message" example:"Provider models successfully"`
	Data    ProviderModelInfo `json:"data"`
}

ProviderModelsResponse represents the response for getting provider models

type ProviderResponse 

type ProviderResponse struct {
	UUID          string           `json:"uuid" example:"0123456789ABCDEF"`
	Name          string           `json:"name" example:"openai"`
	APIBase       string           `json:"api_base" example:"https://api.openai.com/v1"`
	APIStyle      string           `json:"api_style" example:"openai"`
	Token         string           `json:"token" example:"sk-***...***"` // Only populated for api_key auth type
	NoKeyRequired bool             `json:"no_key_required" example:"false"`
	Enabled       bool             `json:"enabled" example:"true"`
	ProxyURL      string           `json:"proxy_url,omitempty" example:"http://127.0.0.1:7890"`
	AuthType      string           `json:"auth_type,omitempty" example:"api_key"` // api_key or oauth
	OAuthDetail   *typ.OAuthDetail `json:"oauth_detail,omitempty"`                // OAuth credentials (only for oauth auth type)
}

ProviderResponse represents a provider configuration with masked token

type ProvidersResponse 

type ProvidersResponse struct {
	Success bool               `json:"success" example:"true"`
	Data    []ProviderResponse `json:"data"`
}

ProvidersResponse represents the response for listing providers

type RequestConfig 

type RequestConfig struct {
	RequestModel  string `json:"request_model" example:"gpt-3.5-turbo"`
	ResponseModel string `json:"response_model" example:"gpt-3.5-turbo"`
	Provider      string `json:"provider" example:"openai"`
	DefaultModel  string `json:"default_model" example:"gpt-3.5-turbo"`
}

RequestConfig represents a request configuration in defaults response

type RuleStatsResponse 

type RuleStatsResponse struct {
	Rule  string                 `json:"rule" example:"gpt-4"`
	Stats map[string]interface{} `json:"stats"`
}

RuleStatsResponse represents the statistics response for a rule

type RuleSummaryResponse 

type RuleSummaryResponse struct {
	Summary interface{} `json:"summary"`
}

RuleSummaryResponse represents a rule summary response

type SDKProbeBuilder 

type SDKProbeBuilder struct{}

SDKProbeBuilder builds SDK requests for probe operations

func NewSDKProbeBuilder 

func NewSDKProbeBuilder() *SDKProbeBuilder

NewSDKProbeBuilder creates a new SDK probe builder

type ScenarioRecorder 

type ScenarioRecorder struct {
	// contains filtered or unexported fields
}

ScenarioRecorder captures scenario-level request/response recording

func (*ScenarioRecorder) EnableStreaming 

func (sr *ScenarioRecorder) EnableStreaming()

EnableStreaming enables streaming mode for this recorder

func (*ScenarioRecorder) GetStreamChunks 

func (sr *ScenarioRecorder) GetStreamChunks() []map[string]interface{}

GetStreamChunks returns the collected stream chunks

func (*ScenarioRecorder) RecordError 

func (sr *ScenarioRecorder) RecordError(err error)

RecordError records an error for the scenario-level request

func (*ScenarioRecorder) RecordResponse 

func (sr *ScenarioRecorder) RecordResponse(provider *typ.Provider, model string)

RecordResponse records the scenario-level response (tingly-box -> client) This captures the response sent back to the client

func (*ScenarioRecorder) RecordStreamChunk 

func (sr *ScenarioRecorder) RecordStreamChunk(eventType string, chunk interface{})

RecordStreamChunk records a single stream chunk

func (*ScenarioRecorder) SetAssembledResponse 

func (sr *ScenarioRecorder) SetAssembledResponse(response any)

SetAssembledResponse sets the assembled response for streaming Accepts any type (e.g., anthropic.Message) and converts to map for storage

type Server 

type Server struct {
	// contains filtered or unexported fields
}

Server represents the HTTP server

func GetGlobalServer 

func GetGlobalServer() *Server

GetGlobalServer gets the global server instance

func NewServer 

func NewServer(cfg *config.Config, opts ...ServerOption) *Server

NewServer creates a new HTTP server instance with functional options

func (*Server) AnthropicCountTokens 

func (s *Server) AnthropicCountTokens(c *gin.Context)

AnthropicCountTokens handles Anthropic v1 count_tokens endpoint This is the entry point that delegates to the appropriate implementation (v1 or beta)

func (*Server) AnthropicListModels 

func (s *Server) AnthropicListModels(c *gin.Context)

AnthropicListModels handles Anthropic v1 models endpoint

func (*Server) AnthropicListModelsForScenario 

func (s *Server) AnthropicListModelsForScenario(c *gin.Context, scenario typ.RuleScenario)

AnthropicListModelsForScenario handles scenario-scoped model listing for Anthropic format

func (*Server) AnthropicMessages added in v0.260124.900 

func (s *Server) AnthropicMessages(c *gin.Context)

AnthropicMessages handles Anthropic v1 messages API requests This is the entry point that delegates to the appropriate implementation (v1 or beta)

func (*Server) ApplyRecording 

func (s *Server) ApplyRecording(scenario typ.RuleScenario) bool

ApplyRecording checks if recording should be applied for a scenario

func (*Server) ApplySmartCompact 

func (s *Server) ApplySmartCompact(scenario typ.RuleScenario) bool

ApplySmartCompact checks if smart_compact should be applied for a scenario

func (*Server) BuildTransformChain 

func (s *Server) BuildTransformChain(c *gin.Context, targetType transform.TargetAPIStyle, providerURL string, scenarioFlags *typ.ScenarioFlags, recorder *ProtocolRecorder) (*transform.TransformChain, error)

BuildTransformChain builds the appropriate transform chain based on recording configuration

func (*Server) ClearLogs 

func (s *Server) ClearLogs(c *gin.Context)

ClearLogs clears all log entries

func (*Server) CreateProvider 

func (s *Server) CreateProvider(c *gin.Context)

CreateProvider adds a new provider

func (*Server) DeleteProvider 

func (s *Server) DeleteProvider(c *gin.Context)

DeleteProvider removes a provider

func (*Server) DetermineProviderAndModel 

func (s *Server) DetermineProviderAndModel(rule *typ.Rule) (*typ.Provider, *loadbalance.Service, error)

DetermineProviderAndModel resolves the model name and finds the appropriate provider using load balancing

func (*Server) DetermineProviderAndModelWithScenario 

func (s *Server) DetermineProviderAndModelWithScenario(scenario typ.RuleScenario, rule *typ.Rule, req interface{}) (*typ.Provider, *loadbalance.Service, error)

DetermineProviderAndModelWithScenario

func (*Server) ExtractRequestContext 

func (s *Server) ExtractRequestContext(req interface{}) (*smartrouting.RequestContext, error)

ExtractRequestContext extracts RequestContext from request based on type

func (*Server) GenerateToken 

func (s *Server) GenerateToken(c *gin.Context)

GenerateToken handles token generation requests

func (*Server) GetActionHistory 

func (s *Server) GetActionHistory(c *gin.Context)

GetActionHistory retrieves user action history from memory Query parameters:

  • limit: maximum number of recent entries to return (default: 100, max: 1000)

func (*Server) GetActionStats 

func (s *Server) GetActionStats(c *gin.Context)

GetActionStats returns statistics about user actions

func (*Server) GetHealthInfo 

func (s *Server) GetHealthInfo(c *gin.Context)

GetHealthInfo handles health check requests This is a lightweight health check that can be called frequently

func (*Server) GetHistory 

func (s *Server) GetHistory(c *gin.Context)

func (*Server) GetInfoConfig 

func (s *Server) GetInfoConfig(c *gin.Context)

func (*Server) GetInfoVersion 

func (s *Server) GetInfoVersion(c *gin.Context)

func (*Server) GetLatestVersion 

func (s *Server) GetLatestVersion(c *gin.Context)

GetLatestVersion checks GitHub releases for the latest version

func (*Server) GetLoadBalancer 

func (s *Server) GetLoadBalancer() *LoadBalancer

GetLoadBalancer returns the load balancer instance

func (*Server) GetLogStats 

func (s *Server) GetLogStats(c *gin.Context)

GetLogStats returns statistics about the logs

func (*Server) GetLogs 

func (s *Server) GetLogs(c *gin.Context)

GetLogs retrieves logs with optional filtering Query parameters:

  • limit: maximum number of entries to return (default: 100)
  • level: filter by log level (debug, info, warn, error)
  • since: RFC3339 timestamp to filter entries after this time

func (*Server) GetOrCreateScenarioRecorderV2 

func (s *Server) GetOrCreateScenarioRecorderV2(c *gin.Context, scenario string, provider *typ.Provider, model string, mode obs.RecordMode) *ProtocolRecorder

GetOrCreateScenarioRecorderV2 gets or creates a V2 scenario recorder for the given scenario

func (*Server) GetOrCreateScenarioSink 

func (s *Server) GetOrCreateScenarioSink(scenario typ.RuleScenario) *obs.Sink

GetOrCreateScenarioSink gets or creates a recording sink for the specified scenario The sink is created on-demand and cached for subsequent use

func (*Server) GetPreferredEndpointForModel 

func (s *Server) GetPreferredEndpointForModel(provider *typ.Provider, modelID string) string

GetPreferredEndpointForModel returns the preferred endpoint (chat or responses) for a model Returns "responses" if the model supports the Responses API, otherwise returns "chat"

func (*Server) GetProvider 

func (s *Server) GetProvider(c *gin.Context)

GetProvider returns details for a specific provider (with masked token)

func (*Server) GetProviderModelsByUUID 

func (s *Server) GetProviderModelsByUUID(c *gin.Context)

func (*Server) GetProviders 

func (s *Server) GetProviders(c *gin.Context)

func (*Server) GetRouter 

func (s *Server) GetRouter() *gin.Engine

GetRouter returns the Gin engine for testing purposes

func (*Server) GetStatus 

func (s *Server) GetStatus(c *gin.Context)

func (*Server) GetSystemLogLevel 

func (s *Server) GetSystemLogLevel(c *gin.Context)

GetSystemLogLevel returns the current system log level

func (*Server) GetSystemLogStats 

func (s *Server) GetSystemLogStats(c *gin.Context)

GetSystemLogStats returns statistics about the system logs

func (*Server) GetSystemLogs 

func (s *Server) GetSystemLogs(c *gin.Context)

GetSystemLogs retrieves system logs with optional filtering Query parameters:

  • limit: maximum number of recent entries to return (default: 100, max: 1000)

func (*Server) GetToken 

func (s *Server) GetToken(c *gin.Context)

GetToken handles token retrieval requests - generates a token if it doesn't exist

func (*Server) HandleProbeModel 

func (s *Server) HandleProbeModel(c *gin.Context)

HandleProbeModel tests a rule configuration by sending a sample request to the configured provider

func (*Server) HandleProbeModelEndpoints 

func (s *Server) HandleProbeModelEndpoints(c *gin.Context)

HandleProbeModelEndpoints handles adaptive probe for model endpoints (chat and responses)

func (*Server) HandleProbeProvider 

func (s *Server) HandleProbeProvider(c *gin.Context)

HandleProbeProvider tests a provider's API key and connectivity

func (*Server) HandleProbeV2 

func (s *Server) HandleProbeV2(c *gin.Context)

HandleProbeV2 handles Probe V2 requests (unified endpoint for all test types)

func (*Server) HealthMonitor 

func (s *Server) HealthMonitor() *loadbalance.HealthMonitor

HealthMonitor returns the server's health monitor

func (*Server) InvalidateProviderCache 

func (s *Server) InvalidateProviderCache(providerUUID string)

InvalidateProviderCache invalidates cached capabilities for a provider

func (*Server) IsExperimentalFeatureEnabled 

func (s *Server) IsExperimentalFeatureEnabled(scenario typ.RuleScenario, feature string) bool

IsExperimentalFeatureEnabled checks if an experimental feature is enabled for a scenario

func (*Server) IsFeatureEnabled 

func (s *Server) IsFeatureEnabled(feature string) bool

IsFeatureEnabled checks if a specific feature is enabled

func (*Server) IsRemoteCoderRunning 

func (s *Server) IsRemoteCoderRunning() bool

IsRemoteCoderRunning returns whether the remote control service is running

func (*Server) ListModelsByScenario 

func (s *Server) ListModelsByScenario(c *gin.Context)

ListModelsByScenario handles the /v1/models endpoint for scenario-based routing

func (*Server) NewUsageTracker 

func (s *Server) NewUsageTracker() *UsageTracker

NewUsageTracker creates a new UsageTracker

func (*Server) OpenAIChatCompletions added in v0.260124.900 

func (s *Server) OpenAIChatCompletions(c *gin.Context)

OpenAIChatCompletions handles OpenAI v1 chat completion requests

func (*Server) OpenAIListModels 

func (s *Server) OpenAIListModels(c *gin.Context)

OpenAIListModels handles the /v1/models endpoint (OpenAI compatible)

func (*Server) OpenAIListModelsForScenario 

func (s *Server) OpenAIListModelsForScenario(c *gin.Context, scenario typ.RuleScenario)

OpenAIListModelsForScenario handles scenario-scoped model listing for OpenAI format

func (*Server) PassthroughAnthropic 

func (s *Server) PassthroughAnthropic(c *gin.Context)

PassthroughAnthropic handles Anthropic-style pass-through requests Consolidates: PassthroughAnthropicMessages, PassthroughAnthropicCountTokens

func (*Server) PassthroughOpenAI 

func (s *Server) PassthroughOpenAI(c *gin.Context)

PassthroughOpenAI handles OpenAI-style pass-through requests Consolidates: PassthroughOpenAIChatCompletions, PassthroughOpenAIResponsesCreate, PassthroughOpenAIResponsesGet

func (*Server) RecordScenarioRequest 

func (s *Server) RecordScenarioRequest(c *gin.Context, scenario string) *ProtocolRecorder

RecordScenarioRequest records the scenario-level request (client -> tingly-box) This captures the original request from the client before any transformation

func (*Server) ResponsesCreate 

func (s *Server) ResponsesCreate(c *gin.Context)

ResponsesCreate handles POST /v1/responses

func (*Server) ResponsesGet 

func (s *Server) ResponsesGet(c *gin.Context)

ResponsesGet handles GET /v1/responses/{id}

func (*Server) RestartServer 

func (s *Server) RestartServer(c *gin.Context)

func (*Server) SelectServiceFromSmartRouting 

func (s *Server) SelectServiceFromSmartRouting(matchedServices []*loadbalance.Service, rule *typ.Rule) (*loadbalance.Service, error)

SelectServiceFromSmartRouting selects a service from matched smart routing services Creates a temporary rule with the matched services and uses the configured load balancing tactic

func (*Server) SetSystemLogLevel 

func (s *Server) SetSystemLogLevel(c *gin.Context)

SetSystemLogLevel sets the minimum log level for system logs

func (*Server) SetupAnthropicEndpoints 

func (s *Server) SetupAnthropicEndpoints(group *gin.RouterGroup)

func (*Server) SetupMixinEndpoints 

func (s *Server) SetupMixinEndpoints(group *gin.RouterGroup)

func (*Server) SetupOpenAIEndpoints 

func (s *Server) SetupOpenAIEndpoints(group *gin.RouterGroup)

func (*Server) SetupPassthroughAnthropicEndpoints 

func (s *Server) SetupPassthroughAnthropicEndpoints(group *gin.RouterGroup)

SetupPassthroughAnthropicEndpoints sets up pass-through endpoints for Anthropic-style requests These endpoints bypass request/response transformations and only replace the model name

func (*Server) SetupPassthroughOpenAIEndpoints 

func (s *Server) SetupPassthroughOpenAIEndpoints(group *gin.RouterGroup)

SetupPassthroughOpenAIEndpoints sets up pass-through endpoints for OpenAI-style requests These endpoints bypass request/response transformations and only replace the model name

func (*Server) ShouldRecording 

func (s *Server) ShouldRecording(recorder *ProtocolRecorder) bool

ShouldRecording determines if recording should be used

func (*Server) Start 

func (s *Server) Start(port int) error

Start starts the HTTP server

func (*Server) StartDynamicCallbackServer 

func (s *Server) StartDynamicCallbackServer(sessionID string, port int) error

StartDynamicCallbackServer starts a temporary callback server for OAuth Implements CallbackServerManager interface for oauth module

func (*Server) StartRemoteCoder 

func (s *Server) StartRemoteCoder() error

StartRemoteCoder starts the remote control service if not already running

func (*Server) StartServer 

func (s *Server) StartServer(c *gin.Context)

func (*Server) Stop 

func (s *Server) Stop(ctx context.Context) error

Stop gracefully stops the HTTP server

func (*Server) StopDynamicCallbackServer 

func (s *Server) StopDynamicCallbackServer(sessionID string)

StopDynamicCallbackServer stops a temporary callback server for OAuth Implements CallbackServerManager interface for oauth module

func (*Server) StopRemoteCoder 

func (s *Server) StopRemoteCoder()

StopRemoteCoder stops the remote control service if running

func (*Server) StopServer 

func (s *Server) StopServer(c *gin.Context)

func (*Server) SyncRemoteCoderBots 

func (s *Server) SyncRemoteCoderBots(ctx context.Context) error

SyncRemoteCoderBots syncs bots with the remote control bot manager

func (*Server) ToggleProvider 

func (s *Server) ToggleProvider(c *gin.Context)

ToggleProvider enables/disables a provider

func (*Server) TrackUsage 

func (s *Server) TrackUsage(ctx context.Context, inputTokens, outputTokens int, err error)

TrackUsage implements the UsageTracker interface. It extracts the gin.Context from the provided context and calls trackUsageFromContext. The gin.Context should be stored in the context with the key "gin_context".

func (*Server) UpdateProvider 

func (s *Server) UpdateProvider(c *gin.Context)

UpdateProvider updates an existing provider

func (*Server) UpdateProviderModelsByUUID 

func (s *Server) UpdateProviderModelsByUUID(c *gin.Context)

func (*Server) UsageStore 

func (s *Server) UsageStore() *db.UsageStore

UsageStore returns the server's usage store instance for internal integrations.

func (*Server) UseAIEndpoints 

func (s *Server) UseAIEndpoints()

func (*Server) UseIndexHTML 

func (s *Server) UseIndexHTML(c *gin.Context)

func (*Server) UseLoadBalanceEndpoints 

func (s *Server) UseLoadBalanceEndpoints()

func (*Server) UseUIEndpoints 

func (s *Server) UseUIEndpoints()

Init sets up Server routes and templates on the main server engine

func (*Server) UseVirtualModelEndpoints 

func (s *Server) UseVirtualModelEndpoints()

UseVirtualModelEndpoints sets up virtual model endpoints for testing

func (*Server) ValidateAuthToken 

func (s *Server) ValidateAuthToken(c *gin.Context)

ValidateAuthToken validates an authentication token without requiring auth This is used during login flow to verify a token before establishing session

type ServerActionResponse 

type ServerActionResponse struct {
	Success bool   `json:"success" example:"true"`
	Message string `json:"message" example:"Server stopped successfully"`
}

ServerActionResponse represents the response for server actions (start/stop/restart)

type ServerOption 

type ServerOption func(*Server)

ServerOption defines a functional option for Server configuration

func WithAdaptor 

func WithAdaptor(enabled bool) ServerOption

WithAdaptor enables or disables the adaptor for the server

func WithDebug 

func WithDebug(enabled bool) ServerOption

WithDebug enables or disables debug mode for the server

func WithDefault 

func WithDefault() ServerOption

WithDefault applies all default server options

func WithExperimentalFeatures 

func WithExperimentalFeatures(features map[string]bool) ServerOption

WithExperimentalFeatures sets the experimental features for the server

func WithHTTPSCertDir 

func WithHTTPSCertDir(certDir string) ServerOption

WithHTTPSCertDir sets the HTTPS certificate directory

func WithHTTPSEnabled 

func WithHTTPSEnabled(enabled bool) ServerOption

WithHTTPSEnabled enables or disables HTTPS

func WithHTTPSRegenerate 

func WithHTTPSRegenerate(regenerate bool) ServerOption

WithHTTPSRegenerate sets the HTTPS certificate regenerate flag

func WithHost 

func WithHost(host string) ServerOption

func WithMultiLogger 

func WithMultiLogger(logger *pkgobs.MultiLogger) ServerOption

WithMultiLogger sets the multi-mode logger for the server

func WithOpenBrowser 

func WithOpenBrowser(enabled bool) ServerOption

WithOpenBrowser enables or disables automatic browser opening

func WithRecordDir 

func WithRecordDir(dir string) ServerOption

WithRecordDir sets the scenario-level record directory

func WithRecordMode 

func WithRecordMode(mode obs.RecordMode) ServerOption

WithRecordMode sets the record mode for request/response recording mode: empty string = disabled, "all" = record all, "response" = response only, "scenario" = record scenario only

func WithRecording 

func WithRecording(enabled bool) ServerOption

WithRecording enables dual-stage recording for protocol conversion scenarios

func WithUI 

func WithUI(enabled bool) ServerOption

WithUI enables or disables the UI for the server

func WithVersion 

func WithVersion(version string) ServerOption

type ServiceHealthResponse 

type ServiceHealthResponse struct {
	Rule   string                 `json:"rule" example:"gpt-4"`
	Health map[string]interface{} `json:"health"`
}

ServiceHealthResponse represents the health check response for services

type ServiceMetric 

type ServiceMetric struct {
	ServiceID            string `json:"service_id" example:"openai:gpt-4"`
	RequestCount         int64  `json:"request_count" example:"100"`
	WindowRequestCount   int64  `json:"window_request_count" example:"50"`
	WindowTokensConsumed int64  `json:"window_tokens_consumed" example:"25000"`
	WindowInputTokens    int64  `json:"window_input_tokens" example:"15000"`
	WindowOutputTokens   int64  `json:"window_output_tokens" example:"10000"`
	LastUsed             string `json:"last_used" example:"2024-01-01T12:00:00Z"`
}

ServiceMetric represents a service metric entry

type ServiceStatsResponse 

type ServiceStatsResponse struct {
	ServiceID string                 `json:"service_id" example:"openai:gpt-4"`
	Stats     map[string]interface{} `json:"stats,omitempty"`
}

ServiceStatsResponse represents the statistics response for a service

type StatusResponse 

type StatusResponse struct {
	Success bool `json:"success" example:"true"`
	Data    struct {
		ServerRunning    bool `json:"server_running" example:"true"`
		Port             int  `json:"port" example:"12580"`
		ProvidersTotal   int  `json:"providers_total" example:"3"`
		ProvidersEnabled int  `json:"providers_enabled" example:"2"`
		RequestCount     int  `json:"request_count" example:"100"`
	} `json:"data"`
}

StatusResponse represents the server status response

type SystemLogEntry 

type SystemLogEntry struct {
	Time    time.Time              `json:"time"`
	Level   string                 `json:"level"`
	Message string                 `json:"message"`
	Fields  map[string]interface{} `json:"fields,omitempty"`
}

SystemLogEntry represents a system log entry for API response

type SystemLogLevelRequest 

type SystemLogLevelRequest struct {
	Level string `json:"level" binding:"required"`
}

SystemLogLevelRequest represents a request to set the log level

type SystemLogsResponse 

type SystemLogsResponse struct {
	Total int              `json:"total"`
	Logs  []SystemLogEntry `json:"logs"`
}

SystemLogsResponse represents the API response for system logs

type ToggleProviderResponse 

type ToggleProviderResponse struct {
	Success bool   `json:"success" example:"true"`
	Message string `json:"message" example:"Provider openai enabled successfully"`
	Data    struct {
		Enabled bool `json:"enabled" example:"true"`
	} `json:"data"`
}

ToggleProviderResponse represents the response for toggling a provider

type TokenInfo 

type TokenInfo = oauth.TokenInfo

type TokenResponse 

type TokenResponse struct {
	Token string `json:"token" example:"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..."`
	Type  string `json:"type" example:"Bearer"`
}

TokenResponse represents the token response

type UpdateProviderRequest 

type UpdateProviderRequest struct {
	Name          *string `json:"name,omitempty" description:"New provider name"`
	APIBase       *string `json:"api_base,omitempty" description:"New API base URL"`
	APIStyle      *string `json:"api_style,omitempty" description:"New API style"`
	Token         *string `json:"token,omitempty" description:"New API token"`
	NoKeyRequired *bool   `json:"no_key_required,omitempty" description:"Whether provider requires no API key"`
	Enabled       *bool   `json:"enabled,omitempty" description:"New enabled status"`
	ProxyURL      *string `json:"proxy_url,omitempty" description:"HTTP or SOCKS proxy URL"`
}

UpdateProviderRequest represents the request to update a provider

type UpdateProviderResponse 

type UpdateProviderResponse struct {
	Success bool             `json:"success" example:"true"`
	Message string           `json:"message" example:"Provider updated successfully"`
	Data    ProviderResponse `json:"data"`
}

UpdateProviderResponse represents the response for updating a provider

type UpdateRuleTacticRequest 

type UpdateRuleTacticRequest struct {
	Tactic string `` /* 152-byte string literal not displayed */
}

UpdateRuleTacticRequest represents the request to update rule tactic

type UpdateRuleTacticResponse 

type UpdateRuleTacticResponse struct {
	Message string `json:"message" example:"Tactic updated successfully"`
	Tactic  string `json:"tactic" example:"round_robin"`
}

UpdateRuleTacticResponse represents the response for updating rule tactic

type UsageTracker 

type UsageTracker struct {
	// contains filtered or unexported fields
}

UsageTracker provides usage tracking methods for handlers. It encapsulates the logic for recording token usage to both service stats and detailed usage records.

func (*UsageTracker) RecordUsage 

func (t *UsageTracker) RecordUsage(
	c *gin.Context,
	rule *typ.Rule,
	provider *typ.Provider,
	model, requestModel string,
	inputTokens, outputTokens int,
	streamed bool,
	status, errorCode string,
)

RecordUsage records token usage from a handler. It updates both the service-level stats and the detailed usage records.

Parameters:

  • c: Gin context for accessing request metadata
  • rule: The load balancer rule that was used
  • provider: The provider that handled the request
  • model: The actual model name used (not the requested model)
  • requestModel: The original model name requested by the user
  • inputTokens: Number of input/prompt tokens consumed
  • outputTokens: Number of output/completion tokens consumed
  • streamed: Whether this was a streaming request
  • status: Request status - "success", "error", or "partial"
  • errorCode: Error code if status is not "success"

type ValidationError 

type ValidationError struct {
	Field   string
	Message string
}

ValidationError represents a validation error

func (*ValidationError) Error 

func (e *ValidationError) Error() string

type VersionChecker 

type VersionChecker struct {
	// contains filtered or unexported fields
}

VersionChecker handles version-related operations

func (*VersionChecker) CheckLatestVersion 

func (vc *VersionChecker) CheckLatestVersion() (version, releaseURL string, err error)

CheckLatestVersion checks for the latest version with fallback: 1. npm registry (primary) 2. npmmirror (China mirror, fallback)

type VersionInfo 

type VersionInfo struct {
	Version string `json:"version" example:"1.0.0"`
}

type VersionInfoResponse 

type VersionInfoResponse struct {
	Success bool        `json:"success" example:"true"`
	Message string      `json:"message" example:"Provider models successfully"`
	Data    VersionInfo `json:"data"`
}

Source Files

View all Source files

Directories

Path Synopsis
module
configapply
imbotsettings
oauth
providertemplate
rule
scenario
skill
statusline
usage

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.