Documentation
¶
Overview ¶
Package router implements the Router Service.
Responsibilities:
- Resolving a ModelId to the correct Provider
- Fetching live Credentials from the Credential Pool
- Single-pass delegation to Lua plugins or built-in Go adapters
- Translating backend-specific errors back to OpenAI-compatible ones
Index ¶
- type Service
- func (s *Service) Complete(ctx context.Context, req *models.ChatCompletionRequest, ...) (*models.ChatCompletionResponse, error)
- func (s *Service) CompleteStream(ctx context.Context, req *models.ChatCompletionRequest, w io.Writer, ...) error
- func (s *Service) Embed(ctx context.Context, req *models.EmbeddingsRequest, token *models.RouterToken) (*models.EmbeddingsResponse, error)
- func (s *Service) GenerateImage(ctx context.Context, req *models.ImageGenerationRequest, ...) (*models.ImageGenerationResponse, error)
- func (s *Service) GetProviderIDForModel(ctx context.Context, modelID models.ModelId) (string, error)
- func (s *Service) ProbeCapabilities(ctx context.Context, modelID models.ModelId) ([]string, error)
- func (s *Service) Speech(ctx context.Context, req *models.SpeechRequest, token *models.RouterToken) (*models.SpeechResponse, error)
- func (s *Service) TestCredential(ctx context.Context, providerID, credentialID string, overrideModel string) TestResult
- func (s *Service) TestModel(ctx context.Context, modelID models.ModelId) TestResult
- func (s *Service) Transcribe(ctx context.Context, req *models.TranscriptionRequest, ...) (*models.TranscriptionResponse, error)
- type TestResult
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Service ¶
type Service struct {
// contains filtered or unexported fields
}
Service routes validated API requests to the appropriate provider.
func New ¶
func New(providerSvc *provider.Service, credSvc *credential.Service, modelInfoSvc *modelinfo.Service, logger *slog.Logger) *Service
New constructs a new router Service.
func (*Service) Complete ¶
func (s *Service) Complete( ctx context.Context, req *models.ChatCompletionRequest, token *models.RouterToken, ) (*models.ChatCompletionResponse, error)
Complete routes a non-streaming chat completion request. The backend tries the credential pool in order, at most once per key; the first success wins and the last error is returned as-is.
func (*Service) CompleteStream ¶
func (s *Service) CompleteStream( ctx context.Context, req *models.ChatCompletionRequest, w io.Writer, token *models.RouterToken, ) error
CompleteStream routes a streaming chat completion request. Server-Sent Events are written directly to w.
func (*Service) Embed ¶ added in v0.0.5
func (s *Service) Embed( ctx context.Context, req *models.EmbeddingsRequest, token *models.RouterToken, ) (*models.EmbeddingsResponse, error)
Embed routes a POST /v1/embeddings request in a single backend pass over the credential pool.
func (*Service) GenerateImage ¶ added in v0.0.5
func (s *Service) GenerateImage( ctx context.Context, req *models.ImageGenerationRequest, token *models.RouterToken, ) (*models.ImageGenerationResponse, error)
GenerateImage routes a POST /v1/images/generations request in a single backend pass over the credential pool.
func (*Service) GetProviderIDForModel ¶
func (s *Service) GetProviderIDForModel(ctx context.Context, modelID models.ModelId) (string, error)
GetProviderIDForModel returns the composite provider ID for a given model.
func (*Service) ProbeCapabilities ¶
ProbeCapabilities detects model features with live probe requests. Each probe consumes a small amount of quota. Unprobed capabilities are simply absent from the result.
func (*Service) Speech ¶ added in v0.0.5
func (s *Service) Speech( ctx context.Context, req *models.SpeechRequest, token *models.RouterToken, ) (*models.SpeechResponse, error)
Speech routes a POST /v1/audio/speech request in a single backend pass over the credential pool.
func (*Service) TestCredential ¶
func (s *Service) TestCredential(ctx context.Context, providerID, credentialID string, overrideModel string) TestResult
TestCredential runs a single probe request pinned to one credential, bypassing pool rotation. The probe model is the provider's first listed model, or overrideModel when given.
func (*Service) TestModel ¶
TestModel runs a probe through the normal routing path (credential pool included), as an internal unrestricted call.
func (*Service) Transcribe ¶ added in v0.0.5
func (s *Service) Transcribe( ctx context.Context, req *models.TranscriptionRequest, token *models.RouterToken, ) (*models.TranscriptionResponse, error)
Transcribe routes a POST /v1/audio/transcriptions request in a single backend pass over the credential pool.