router

package
v0.0.7 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 19, 2026 License: MIT Imports: 12 Imported by: 0

Documentation

Overview

Package router implements the Router Service.

Responsibilities:

  • Resolving a ModelId to the correct Provider
  • Fetching live Credentials from the Credential Pool
  • Single-pass delegation to Lua plugins or built-in Go adapters
  • Translating backend-specific errors back to OpenAI-compatible ones

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Service

type Service struct {
	// contains filtered or unexported fields
}

Service routes validated API requests to the appropriate provider.

func New

func New(providerSvc *provider.Service, credSvc *credential.Service, modelInfoSvc *modelinfo.Service, logger *slog.Logger) *Service

New constructs a new router Service.

func (*Service) Complete

Complete routes a non-streaming chat completion request. The backend tries the credential pool in order, at most once per key; the first success wins and the last error is returned as-is.

func (*Service) CompleteStream

func (s *Service) CompleteStream(
	ctx context.Context,
	req *models.ChatCompletionRequest,
	w io.Writer,
	token *models.RouterToken,
) error

CompleteStream routes a streaming chat completion request. Server-Sent Events are written directly to w.

func (*Service) Embed added in v0.0.5

Embed routes a POST /v1/embeddings request in a single backend pass over the credential pool.

func (*Service) GenerateImage added in v0.0.5

GenerateImage routes a POST /v1/images/generations request in a single backend pass over the credential pool.

func (*Service) GetProviderIDForModel

func (s *Service) GetProviderIDForModel(ctx context.Context, modelID models.ModelId) (string, error)

GetProviderIDForModel returns the composite provider ID for a given model.

func (*Service) ProbeCapabilities

func (s *Service) ProbeCapabilities(ctx context.Context, modelID models.ModelId) ([]string, error)

ProbeCapabilities detects model features with live probe requests. Each probe consumes a small amount of quota. Unprobed capabilities are simply absent from the result.

func (*Service) Speech added in v0.0.5

Speech routes a POST /v1/audio/speech request in a single backend pass over the credential pool.

func (*Service) TestCredential

func (s *Service) TestCredential(ctx context.Context, providerID, credentialID string, overrideModel string) TestResult

TestCredential runs a single probe request pinned to one credential, bypassing pool rotation. The probe model is the provider's first listed model, or overrideModel when given.

func (*Service) TestModel

func (s *Service) TestModel(ctx context.Context, modelID models.ModelId) TestResult

TestModel runs a probe through the normal routing path (credential pool included), as an internal unrestricted call.

func (*Service) Transcribe added in v0.0.5

Transcribe routes a POST /v1/audio/transcriptions request in a single backend pass over the credential pool.

type TestResult

type TestResult struct {
	OK       bool   `json:"ok"`
	Latency  int64  `json:"latency_ms"`
	Error    string `json:"error,omitempty"`
	Response string `json:"response,omitempty"`
}

TestResult reports the outcome of a credential or model probe.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL