router

package
v0.0.4 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 17, 2026 License: MIT Imports: 13 Imported by: 0

Documentation

Overview

Package router implements the Router Service.

Responsibilities:

  • Resolving a ModelId to the correct Provider
  • Fetching live Credentials from the Credential Pool
  • Intelligent retry with credential rotation on rate limits
  • Delegating requests to Lua plugins or built-in Go adapters
  • Translating backend-specific errors back to OpenAI-compatible ones

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Service

type Service struct {
	// contains filtered or unexported fields
}

Service routes validated API requests to the appropriate provider.

func New

func New(providerSvc *provider.Service, credSvc *credential.Service, modelInfoSvc *modelinfo.Service, maxRetries int, logger *slog.Logger) *Service

New constructs a new router Service.

func (*Service) Complete

Complete routes a non-streaming chat completion request with credential rotation via the shared retry engine and exponential backoff across cycles.

func (*Service) CompleteStream

func (s *Service) CompleteStream(
	ctx context.Context,
	req *models.ChatCompletionRequest,
	w io.Writer,
	token *models.RouterToken,
) error

CompleteStream routes a streaming chat completion request. Server-Sent Events are written directly to w.

func (*Service) GetProviderIDForModel

func (s *Service) GetProviderIDForModel(ctx context.Context, modelID models.ModelId) (string, error)

GetProviderIDForModel returns the composite provider ID for a given model.

func (*Service) ProbeCapabilities

func (s *Service) ProbeCapabilities(ctx context.Context, modelID models.ModelId) ([]string, error)

ProbeCapabilities detects model features with live probe requests. Each probe consumes a small amount of quota. Unprobed capabilities are simply absent from the result.

func (*Service) SetMaxRetries

func (s *Service) SetMaxRetries(n int)

SetMaxRetries updates the retry limit at runtime.

func (*Service) TestCredential

func (s *Service) TestCredential(ctx context.Context, providerID, credentialID string, overrideModel string) TestResult

TestCredential runs a single probe request pinned to one credential, bypassing pool rotation. The probe model is the provider's first listed model, or overrideModel when given.

func (*Service) TestModel

func (s *Service) TestModel(ctx context.Context, modelID models.ModelId) TestResult

TestModel runs a probe through the normal routing path (credential pool included), as an internal unrestricted call.

type TestResult

type TestResult struct {
	OK       bool   `json:"ok"`
	Latency  int64  `json:"latency_ms"`
	Error    string `json:"error,omitempty"`
	Response string `json:"response,omitempty"`
}

TestResult reports the outcome of a credential or model probe.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL