Documentation
¶
Overview ¶
Package router implements the Router Service.
Responsibilities:
- Resolving a ModelId to the correct Provider
- Fetching live Credentials from the Credential Pool
- Intelligent retry with credential rotation on rate limits
- Delegating requests to Lua plugins or built-in Go adapters
- Translating backend-specific errors back to OpenAI-compatible ones
Index ¶
- type Service
- func (s *Service) Complete(ctx context.Context, req *models.ChatCompletionRequest, ...) (*models.ChatCompletionResponse, error)
- func (s *Service) CompleteStream(ctx context.Context, req *models.ChatCompletionRequest, w io.Writer, ...) error
- func (s *Service) GetProviderIDForModel(ctx context.Context, modelID models.ModelId) (string, error)
- func (s *Service) ProbeCapabilities(ctx context.Context, modelID models.ModelId) ([]string, error)
- func (s *Service) SetMaxRetries(n int)
- func (s *Service) TestCredential(ctx context.Context, providerID, credentialID string, overrideModel string) TestResult
- func (s *Service) TestModel(ctx context.Context, modelID models.ModelId) TestResult
- type TestResult
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Service ¶
type Service struct {
// contains filtered or unexported fields
}
Service routes validated API requests to the appropriate provider.
func New ¶
func New(providerSvc *provider.Service, credSvc *credential.Service, modelInfoSvc *modelinfo.Service, maxRetries int, logger *slog.Logger) *Service
New constructs a new router Service.
func (*Service) Complete ¶
func (s *Service) Complete( ctx context.Context, req *models.ChatCompletionRequest, token *models.RouterToken, ) (*models.ChatCompletionResponse, error)
Complete routes a non-streaming chat completion request with credential rotation via the shared retry engine and exponential backoff across cycles.
func (*Service) CompleteStream ¶
func (s *Service) CompleteStream( ctx context.Context, req *models.ChatCompletionRequest, w io.Writer, token *models.RouterToken, ) error
CompleteStream routes a streaming chat completion request. Server-Sent Events are written directly to w.
func (*Service) GetProviderIDForModel ¶
func (s *Service) GetProviderIDForModel(ctx context.Context, modelID models.ModelId) (string, error)
GetProviderIDForModel returns the composite provider ID for a given model.
func (*Service) ProbeCapabilities ¶
ProbeCapabilities detects model features with live probe requests. Each probe consumes a small amount of quota. Unprobed capabilities are simply absent from the result.
func (*Service) SetMaxRetries ¶
SetMaxRetries updates the retry limit at runtime.
func (*Service) TestCredential ¶
func (s *Service) TestCredential(ctx context.Context, providerID, credentialID string, overrideModel string) TestResult
TestCredential runs a single probe request pinned to one credential, bypassing pool rotation. The probe model is the provider's first listed model, or overrideModel when given.