router

package
v0.5.21 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 2, 2026 License: Apache-2.0 Imports: 11 Imported by: 0

Documentation

Overview

Package router provides intelligent failover routing between AI providers.

Package router provides intelligent failover routing between AI providers. It tracks provider capabilities and statistics to make informed routing decisions.

Package router provides intelligent failover routing for the Superbrain system.

Package router provides intelligent failover routing for the Superbrain system. It manages provider selection based on capabilities, availability, and historical success rates to ensure requests are routed to the most suitable alternative provider.

Index

Constants

This section is empty.

Variables

View Source
var (
	// ErrInvalidRequest indicates the request could not be parsed.
	ErrInvalidRequest = errors.New("invalid request format")

	// ErrUnsupportedProvider indicates the target provider is not supported for adaptation.
	ErrUnsupportedProvider = errors.New("unsupported provider for adaptation")
)
View Source
var (
	// ErrNoFallbackAvailable indicates no suitable fallback provider was found.
	ErrNoFallbackAvailable = errors.New("no fallback provider available")

	// ErrProviderNotConfigured indicates the provider is not in the configured list.
	ErrProviderNotConfigured = errors.New("provider not configured for fallback")
)
View Source
var DefaultProviderCapabilities = map[string]*ProviderCapability{
	"claudecli": {
		Provider:       "claudecli",
		MaxContextSize: 200000,
		SupportsStream: true,
		SupportsCLI:    true,
		SuccessRate:    1.0,
		IsAvailable:    true,
	},
	"geminicli": {
		Provider:       "geminicli",
		MaxContextSize: 1000000,
		SupportsStream: true,
		SupportsCLI:    true,
		SuccessRate:    1.0,
		IsAvailable:    true,
	},
	"gemini": {
		Provider:       "gemini",
		MaxContextSize: 1000000,
		SupportsStream: true,
		SupportsCLI:    false,
		SuccessRate:    1.0,
		IsAvailable:    true,
	},
	"ollama": {
		Provider:       "ollama",
		MaxContextSize: 128000,
		SupportsStream: true,
		SupportsCLI:    false,
		SuccessRate:    1.0,
		IsAvailable:    true,
	},
	"openai": {
		Provider:       "openai",
		MaxContextSize: 128000,
		SupportsStream: true,
		SupportsCLI:    false,
		SuccessRate:    1.0,
		IsAvailable:    true,
	},
	"lmstudio": {
		Provider:       "lmstudio",
		MaxContextSize: 32000,
		SupportsStream: true,
		SupportsCLI:    false,
		SuccessRate:    1.0,
		IsAvailable:    true,
	},
}

DefaultProviderCapabilities returns the default capabilities for known providers.

Functions

This section is empty.

Types

type AdaptedRequest

type AdaptedRequest struct {
	// Payload is the adapted request JSON.
	Payload []byte

	// OriginalModel is the model from the original request.
	OriginalModel string

	// AdaptedModel is the model for the target provider.
	AdaptedModel string

	// PreservedSemantics lists what was preserved during adaptation.
	PreservedSemantics []string
}

AdaptedRequest contains the adapted request and metadata about the adaptation.

type CapabilityRegistry

type CapabilityRegistry struct {
	// contains filtered or unexported fields
}

CapabilityRegistry manages provider capabilities.

func NewCapabilityRegistry

func NewCapabilityRegistry() *CapabilityRegistry

NewCapabilityRegistry creates a new registry with default capabilities.

func (*CapabilityRegistry) GetAllCapabilities

func (r *CapabilityRegistry) GetAllCapabilities() []*ProviderCapability

GetAllCapabilities returns all registered capabilities.

func (*CapabilityRegistry) GetCapability

func (r *CapabilityRegistry) GetCapability(provider string) *ProviderCapability

GetCapability returns the capability for a provider, or nil if not found.

func (*CapabilityRegistry) SetCapability

func (r *CapabilityRegistry) SetCapability(cap *ProviderCapability)

SetCapability sets or updates the capability for a provider.

func (*CapabilityRegistry) UpdateAvailability

func (r *CapabilityRegistry) UpdateAvailability(provider string, available bool)

UpdateAvailability updates the availability status for a provider.

func (*CapabilityRegistry) UpdateLatency

func (r *CapabilityRegistry) UpdateLatency(provider string, latency time.Duration)

UpdateLatency updates the average latency for a provider.

func (*CapabilityRegistry) UpdateSuccessRate

func (r *CapabilityRegistry) UpdateSuccessRate(provider string, rate float64)

UpdateSuccessRate updates the success rate for a provider.

type ChatMessage

type ChatMessage struct {
	// Role is the message author (system, user, assistant).
	Role string `json:"role"`

	// Content is the message text.
	Content string `json:"content"`
}

ChatMessage represents a single message in the conversation.

type ChatRequest

type ChatRequest struct {
	// Model is the model identifier.
	Model string `json:"model"`

	// Messages is the conversation history.
	Messages []ChatMessage `json:"messages"`

	// Stream indicates whether to stream the response.
	Stream bool `json:"stream,omitempty"`

	// Temperature controls randomness (0.0 to 2.0).
	Temperature *float64 `json:"temperature,omitempty"`

	// MaxTokens limits the response length.
	MaxTokens *int `json:"max_tokens,omitempty"`

	// TopP controls nucleus sampling.
	TopP *float64 `json:"top_p,omitempty"`

	// Stop sequences that halt generation.
	Stop []string `json:"stop,omitempty"`

	// ExtraBody contains provider-specific fields.
	ExtraBody map[string]interface{} `json:"extra_body,omitempty"`
}

ChatRequest represents an OpenAI-compatible chat completion request. This is the common format used for request adaptation.

type FallbackAttempt

type FallbackAttempt struct {
	Provider string
	Success  bool
	Error    error
	Latency  time.Duration
}

FallbackAttempt records a single fallback attempt.

type FallbackChain

type FallbackChain struct {
	// contains filtered or unexported fields
}

CreateFallbackChain creates a chain of fallback attempts for a request. This is useful for trying multiple fallback providers in sequence.

func (*FallbackChain) GetAttempts

func (fc *FallbackChain) GetAttempts() []FallbackAttempt

GetAttempts returns all recorded attempts.

func (*FallbackChain) GetNegotiatedFailure

func (fc *FallbackChain) GetNegotiatedFailure() *types.NegotiatedFailureResponse

GetNegotiatedFailure creates a NegotiatedFailureResponse summarizing all attempts.

func (*FallbackChain) Next

Next returns the next fallback provider to try. Returns nil if no more fallbacks are available or max attempts reached.

func (*FallbackChain) RecordAttempt

func (fc *FallbackChain) RecordAttempt(provider string, success bool, err error, latency time.Duration)

RecordAttempt records the result of a fallback attempt.

type FallbackDecision

type FallbackDecision struct {
	// OriginalProvider is the provider that failed.
	OriginalProvider string

	// FallbackProvider is the selected alternative provider.
	FallbackProvider string

	// Reason explains why this fallback was selected.
	Reason string

	// CapabilityMatch indicates how well the fallback matches requirements (0.0 to 1.0).
	CapabilityMatch float64

	// AdaptedRequest contains the request adapted for the fallback provider (if adaptation was performed).
	AdaptedRequest *AdaptedRequest
}

FallbackDecision contains routing decision details.

type FallbackIntegration

type FallbackIntegration struct {
	// contains filtered or unexported fields
}

FallbackIntegration provides a high-level interface for fallback routing that integrates with the Superbrain metadata and audit systems.

func NewFallbackIntegration

func NewFallbackIntegration(cfg *config.FallbackConfig) *FallbackIntegration

NewFallbackIntegration creates a new FallbackIntegration with the given configuration.

func (*FallbackIntegration) AttemptFallback

func (fi *FallbackIntegration) AttemptFallback(ctx context.Context, req *FallbackRequest, aggregator *metadata.Aggregator) *FallbackResponse

AttemptFallback attempts to find and route to a fallback provider. It records the attempt in the metadata aggregator and audit log.

func (*FallbackIntegration) GetProviderCapabilities

func (fi *FallbackIntegration) GetProviderCapabilities() []*ProviderCapability

GetProviderCapabilities returns capabilities for all providers.

func (*FallbackIntegration) GetRouter

func (fi *FallbackIntegration) GetRouter() *FallbackRouter

GetRouter returns the underlying FallbackRouter for advanced usage.

func (*FallbackIntegration) HandleExecutionError

func (fi *FallbackIntegration) HandleExecutionError(err error, diagnosis *types.Diagnosis) bool

HandleExecutionError processes an execution error and determines if fallback should be attempted. Returns true if fallback should be attempted, false if the error is not recoverable.

func (*FallbackIntegration) IsProviderAvailable

func (fi *FallbackIntegration) IsProviderAvailable(provider string) bool

IsProviderAvailable checks if a provider is currently available for routing.

func (*FallbackIntegration) IsProviderConfigured

func (fi *FallbackIntegration) IsProviderConfigured(provider string) bool

IsProviderConfigured checks if a provider is in the configured fallback list.

func (*FallbackIntegration) NewFallbackChain

func (fi *FallbackIntegration) NewFallbackChain(req *FallbackRequest, aggregator *metadata.Aggregator, maxAttempts int) *FallbackChain

NewFallbackChain creates a new FallbackChain.

func (*FallbackIntegration) RecordProviderResult

func (fi *FallbackIntegration) RecordProviderResult(provider string, success bool, latency time.Duration, failureReason string)

RecordProviderResult records the result of a provider execution. This updates the provider statistics for future routing decisions.

func (*FallbackIntegration) SetAuditLogger

func (fi *FallbackIntegration) SetAuditLogger(logger *audit.Logger)

SetAuditLogger sets the audit logger for this integration.

func (*FallbackIntegration) SetProviderAvailability

func (fi *FallbackIntegration) SetProviderAvailability(provider string, available bool)

SetProviderAvailability updates the availability status for a provider.

type FallbackRequest

type FallbackRequest struct {
	// RequestID uniquely identifies the request.
	RequestID string

	// FailedProvider is the provider that failed.
	FailedProvider string

	// Model is the model being used.
	Model string

	// OriginalPayload is the original request payload.
	OriginalPayload []byte

	// Requirements specifies what capabilities the request needs.
	Requirements *RequestRequirements

	// Diagnosis contains the failure diagnosis if available.
	Diagnosis *types.Diagnosis

	// OriginalError is the error from the failed provider.
	OriginalError error
}

FallbackRequest contains the information needed for fallback routing.

type FallbackResponse

type FallbackResponse struct {
	// Success indicates whether a fallback was found.
	Success bool

	// Decision contains the routing decision details.
	Decision *FallbackDecision

	// NegotiatedFailure contains the failure response if no fallback was available.
	NegotiatedFailure *types.NegotiatedFailureResponse

	// Error contains any error that occurred.
	Error error
}

FallbackResponse contains the result of fallback routing.

type FallbackRouter

type FallbackRouter struct {
	// contains filtered or unexported fields
}

FallbackRouter manages intelligent failover between providers.

func NewFallbackRouter

func NewFallbackRouter(cfg *config.FallbackConfig) *FallbackRouter

NewFallbackRouter creates a new fallback router with the given configuration.

func (*FallbackRouter) AdaptRequest

func (r *FallbackRouter) AdaptRequest(originalPayload []byte, targetProvider string) (*AdaptedRequest, error)

AdaptRequest adapts a request for a fallback provider. It preserves the original request semantics (messages, streaming, capabilities) while adapting the model name and provider-specific fields.

func (*FallbackRouter) GetCapabilityRegistry

func (r *FallbackRouter) GetCapabilityRegistry() *CapabilityRegistry

GetCapabilityRegistry returns the capability registry for testing.

func (*FallbackRouter) GetFallback

func (r *FallbackRouter) GetFallback(ctx context.Context, failedProvider string, requirements *RequestRequirements) (*FallbackDecision, error)

GetFallback finds an alternative provider for a failed request. It considers provider capabilities, availability, and success rate.

func (*FallbackRouter) GetFallbackWithAdaptation

func (r *FallbackRouter) GetFallbackWithAdaptation(ctx context.Context, failedProvider string, requirements *RequestRequirements, originalPayload []byte) (*FallbackDecision, error)

GetFallbackWithAdaptation finds an alternative provider and adapts the request. This is a convenience method that combines GetFallback and AdaptRequest.

func (*FallbackRouter) GetProviderCapabilities

func (r *FallbackRouter) GetProviderCapabilities() []*ProviderCapability

GetProviderCapabilities returns capabilities for all providers.

func (*FallbackRouter) GetRequestAdapter

func (r *FallbackRouter) GetRequestAdapter() *RequestAdapter

GetRequestAdapter returns the request adapter for testing or custom configuration.

func (*FallbackRouter) GetStatsTracker

func (r *FallbackRouter) GetStatsTracker() *StatsTracker

GetStatsTracker returns the stats tracker for testing.

func (*FallbackRouter) IsProviderAvailable

func (r *FallbackRouter) IsProviderAvailable(provider string) bool

IsProviderAvailable checks if a provider is currently usable.

func (*FallbackRouter) IsProviderConfigured

func (r *FallbackRouter) IsProviderConfigured(provider string) bool

IsProviderConfigured checks if a provider is in the configured fallback list.

func (*FallbackRouter) SetProviderAvailability

func (r *FallbackRouter) SetProviderAvailability(provider string, available bool)

SetProviderAvailability updates the availability status for a provider.

func (*FallbackRouter) UpdateProviderStats

func (r *FallbackRouter) UpdateProviderStats(provider string, success bool, latency time.Duration)

UpdateProviderStats records success/failure for a provider.

type ProviderCapability

type ProviderCapability struct {
	// Provider is the unique identifier for this provider (e.g., "claudecli", "gemini").
	Provider string

	// MaxContextSize is the maximum number of tokens the provider can handle.
	MaxContextSize int

	// SupportsStream indicates whether the provider supports streaming responses.
	SupportsStream bool

	// SupportsCLI indicates whether the provider is a CLI-based tool.
	SupportsCLI bool

	// SuccessRate is the historical success rate (0.0 to 1.0).
	SuccessRate float64

	// AverageLatency is the average response time for this provider.
	AverageLatency time.Duration

	// IsAvailable indicates whether the provider is currently usable.
	IsAvailable bool
}

ProviderCapability describes what a provider can handle.

type ProviderStats

type ProviderStats struct {
	// Provider is the unique identifier for this provider.
	Provider string

	// TotalRequests is the total number of requests made to this provider.
	TotalRequests int64

	// SuccessCount is the number of successful requests.
	SuccessCount int64

	// FailureCount is the number of failed requests.
	FailureCount int64

	// TotalLatencyMs is the cumulative latency in milliseconds.
	TotalLatencyMs int64

	// LastSuccess is the timestamp of the last successful request.
	LastSuccess time.Time

	// LastFailure is the timestamp of the last failed request.
	LastFailure time.Time

	// FailureReasons maps failure reasons to their occurrence counts.
	FailureReasons map[string]int64
}

ProviderStats tracks historical performance for a provider.

func (*ProviderStats) AverageLatency

func (s *ProviderStats) AverageLatency() time.Duration

AverageLatency calculates the average latency for this provider.

func (*ProviderStats) SuccessRate

func (s *ProviderStats) SuccessRate() float64

SuccessRate calculates the success rate for this provider.

type RequestAdapter

type RequestAdapter struct {
	// contains filtered or unexported fields
}

RequestAdapter adapts requests for fallback providers while preserving semantics.

func NewRequestAdapter

func NewRequestAdapter() *RequestAdapter

NewRequestAdapter creates a new request adapter with default model mappings.

func (*RequestAdapter) AdaptRequest

func (a *RequestAdapter) AdaptRequest(originalPayload []byte, targetProvider string) (*AdaptedRequest, error)

AdaptRequest adapts a request for a fallback provider. It preserves the original request semantics (messages, streaming, capabilities) while adapting the model name and provider-specific fields.

func (*RequestAdapter) GetModelMapping

func (a *RequestAdapter) GetModelMapping(originalModel, targetProvider string) string

GetModelMapping returns the mapped model for a given original model and target provider.

func (*RequestAdapter) SetModelMapping

func (a *RequestAdapter) SetModelMapping(originalModel, targetProvider, mappedModel string)

SetModelMapping sets a custom model mapping.

type RequestRequirements

type RequestRequirements struct {
	// RequiresStream indicates the request needs streaming support.
	RequiresStream bool

	// RequiresCLI indicates the request needs CLI execution.
	RequiresCLI bool

	// MinContextSize is the minimum context size needed.
	MinContextSize int
}

RequestRequirements specifies what capabilities a request needs.

type StatsTracker

type StatsTracker struct {
	// contains filtered or unexported fields
}

StatsTracker manages provider statistics with thread-safe access.

func NewStatsTracker

func NewStatsTracker() *StatsTracker

NewStatsTracker creates a new statistics tracker.

func (*StatsTracker) GetAllStats

func (t *StatsTracker) GetAllStats() map[string]*ProviderStats

GetAllStats returns statistics for all tracked providers.

func (*StatsTracker) GetOrCreateStats

func (t *StatsTracker) GetOrCreateStats(provider string) *ProviderStats

GetOrCreateStats returns existing stats or creates new ones for a provider.

func (*StatsTracker) GetStats

func (t *StatsTracker) GetStats(provider string) *ProviderStats

GetStats returns the statistics for a provider. Returns nil if no stats exist for the provider.

func (*StatsTracker) Reset

func (t *StatsTracker) Reset()

Reset clears all statistics.

func (*StatsTracker) UpdateStats

func (t *StatsTracker) UpdateStats(provider string, success bool, latency time.Duration, failureReason string)

UpdateStats records a request outcome for a provider.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL