router

package
v0.11.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 13, 2026 License: MIT Imports: 19 Imported by: 0

Documentation

Overview

Package router implements the Router Service.

Responsibilities:

  • Resolving a ModelId to the correct Provider
  • Enforcing router-token credential rules as an allow-list for plugins
  • Single-pass delegation to Lua plugins or built-in Go adapters

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type BackendAttempt added in v0.3.8

type BackendAttempt struct {
	Resolved *provider.Resolved
	Creds    []*models.Credential
	Allowed  []string
}

BackendAttempt groups one backend pass inputs. Resolved identifies the backend; Creds carries the gated pool; Allowed carries the token allow-list plugins may use (nil means unrestricted).

type Service

type Service struct {
	// contains filtered or unexported fields
}

Service routes validated API requests to the appropriate provider.

func New

func New(providerService *provider.GormService, credentialService *credential.GormService, modelInfoService *modelinfo.GormService, videoJobsService *videojobs.Service, responseService *responses.GormService, batchService *batches.SQLiteService, logger *slog.Logger) *Service

New constructs a new router Service. videoJobsService may be nil (video submits fail closed). responseService and batchService may be nil (responses/batches fail closed). Production always wires all of them.

func (*Service) AppendConversationItems added in v0.3.8

func (s *Service) AppendConversationItems(
	ctx context.Context,
	id string,
	token *models.RouterToken,
	items []models.ResponseInputItem,
) ([]models.ResponseInputItem, error)

AppendConversationItems appends items to a conversation.

func (*Service) CancelBatch added in v0.3.8

func (s *Service) CancelBatch(
	ctx context.Context,
	id string,
	token *models.RouterToken,
) (*models.BatchInfo, error)

CancelBatch marks a non-terminal batch canceled. Synchronously executed batches are terminal, so cancel succeeds only on rows still open.

func (*Service) CancelResponse added in v0.3.8

func (s *Service) CancelResponse(
	ctx context.Context,
	id string,
	token *models.RouterToken,
) (*models.ResponseObject, error)

CancelResponse marks a non-terminal response canceled. Synchronously executed responses are terminal, so cancel succeeds only on rows still open (e.g. requires_action runs surfaced as responses).

func (*Service) CancelRun added in v0.3.8

func (s *Service) CancelRun(
	ctx context.Context,
	threadID, runID string,
	token *models.RouterToken,
) (*models.Run, error)

CancelRun marks a non-terminal run canceled.

func (*Service) CompactResponse added in v0.3.8

func (s *Service) CompactResponse(
	ctx context.Context,
	id string,
	token *models.RouterToken,
) (*models.ResponseObject, error)

CompactResponse collapses a response input chain. The router stores flat inputs, so compaction returns the stored object unchanged.

func (*Service) Complete

Complete routes a non-streaming chat completion request in a single backend pass. Lua plugins run their own credential/proxy iteration and return the first success or the terminal error as-is.

func (*Service) CompleteStream

func (s *Service) CompleteStream(
	ctx context.Context,
	request *models.ChatCompletionRequest,
	w io.Writer,
	token *models.RouterToken,
) error

CompleteStream routes a streaming chat completion request. Server-Sent Events are written directly to w.

func (*Service) CountResponseInputTokens added in v0.3.8

func (s *Service) CountResponseInputTokens(request *models.ResponseRequest) int

CountResponseInputTokens estimates input tokens for a response request with the documented 4-chars-per-token heuristic.

func (*Service) CreateAssistant added in v0.3.8

func (s *Service) CreateAssistant(
	ctx context.Context,
	request *models.AssistantRequest,
) (*models.Assistant, error)

CreateAssistant persists an assistant preset. Model is required: presets execute runs through the chat pipeline.

func (*Service) CreateBatch added in v0.3.8

func (s *Service) CreateBatch(
	ctx context.Context,
	request *models.BatchCreateRequest,
	token *models.RouterToken,
) (*models.BatchInfo, error)

CreateBatch validates the entries, executes each through the chat pipeline in order, and persists the ended batch. Execution is synchronous: the batch returns ended (or partially errored lines), never a deferred worker. Custom IDs must be unique within the batch.

func (*Service) CreateConversation added in v0.3.8

func (s *Service) CreateConversation(
	ctx context.Context,
	model models.ModelId,
	metadata map[string]string,
	items []models.ResponseInputItem,
) (*models.ConversationObject, error)

CreateConversation persists a conversation container. Model is optional: containers carry no model until items referencing one arrive.

func (*Service) CreateResponse added in v0.3.8

func (s *Service) CreateResponse(
	ctx context.Context,
	request *models.ResponseRequest,
	token *models.RouterToken,
) (*models.ResponseObject, error)

CreateResponse runs the request through the chat pipeline and persists the response object, unless Store is explicitly false. Background is accepted and executed inline: the router has no deferred worker, so the object always returns completed or failed.

func (*Service) CreateRun added in v0.3.8

func (s *Service) CreateRun(
	ctx context.Context,
	threadID string,
	request *models.RunRequest,
	token *models.RouterToken,
) (*models.Run, error)

CreateRun executes an assistant over a thread through the chat pipeline. Runs with tool calls park in requires_action; completed runs append the assistant turn to the thread, mirroring OpenAI run semantics.

func (*Service) CreateThread added in v0.3.8

func (s *Service) CreateThread(
	ctx context.Context,
	metadata map[string]string,
) (*models.Thread, error)

CreateThread persists a thread handle.

func (*Service) CreateThreadMessage added in v0.3.8

func (s *Service) CreateThreadMessage(
	ctx context.Context,
	threadID string,
	request *models.ThreadMessageRequest,
) (*models.ThreadMessage, error)

CreateThreadMessage appends a user or assistant message to a thread.

func (*Service) DeleteAssistant added in v0.3.8

func (s *Service) DeleteAssistant(ctx context.Context, id string) error

DeleteAssistant removes an assistant preset. Runs referencing it stay addressable by run id.

func (*Service) DeleteBatch added in v0.3.8

func (s *Service) DeleteBatch(ctx context.Context, id string) error

DeleteBatch removes a batch row. Missing rows are not errors only when the row never existed? No: missing rows are 404, matching the catalog.

func (*Service) DeleteConversation added in v0.3.8

func (s *Service) DeleteConversation(ctx context.Context, id string) error

DeleteConversation removes a conversation container. Items stored as responses with the conversation id stay addressable by response id.

func (*Service) DeleteThread added in v0.3.8

func (s *Service) DeleteThread(ctx context.Context, id string) error

DeleteThread removes a thread handle. Messages and runs referencing it stay addressable by id.

func (*Service) Embed added in v0.0.5

Embed routes a POST /v1/embeddings request in a single backend pass over the credential pool.

func (*Service) GenerateImage added in v0.0.5

GenerateImage routes a POST /v1/images/generations request in a single backend pass over the credential pool.

func (*Service) GetAssistant added in v0.3.8

func (s *Service) GetAssistant(
	ctx context.Context,
	id string,
	token *models.RouterToken,
) (*models.Assistant, error)

GetAssistant returns one stored assistant.

func (*Service) GetBatch added in v0.3.8

func (s *Service) GetBatch(
	ctx context.Context,
	id string,
	token *models.RouterToken,
) (*models.BatchInfo, error)

GetBatch returns one stored batch.

func (*Service) GetBatchResults added in v0.3.8

func (s *Service) GetBatchResults(
	ctx context.Context,
	id string,
	token *models.RouterToken,
) (string, error)

GetBatchResults renders stored results as JSONL.

func (*Service) GetConversation added in v0.3.8

func (s *Service) GetConversation(
	ctx context.Context,
	id string,
	token *models.RouterToken,
) (*models.ConversationObject, error)

GetConversation returns one stored conversation.

func (*Service) GetProviderIDForModel

func (s *Service) GetProviderIDForModel(ctx context.Context, modelID models.ModelId) (string, error)

GetProviderIDForModel returns the composite provider ID for a given model.

func (*Service) GetResponse added in v0.3.8

func (s *Service) GetResponse(
	ctx context.Context,
	id string,
	token *models.RouterToken,
) (*models.ResponseObject, error)

GetResponse returns one stored response. Token rules apply to the stored request model, exactly like video poll/content authorization.

func (*Service) GetRun added in v0.3.8

func (s *Service) GetRun(
	ctx context.Context,
	threadID, runID string,
	token *models.RouterToken,
) (*models.Run, error)

GetRun returns one run, asserting thread membership.

func (*Service) GetThread added in v0.3.8

func (s *Service) GetThread(
	ctx context.Context,
	id string,
	token *models.RouterToken,
) (*models.Thread, error)

GetThread returns one stored thread.

func (*Service) GetThreadMessage added in v0.3.8

func (s *Service) GetThreadMessage(
	ctx context.Context,
	threadID, messageID string,
) (*models.ThreadMessage, error)

GetThreadMessage returns one thread message, asserting thread membership.

func (*Service) ListAssistants added in v0.3.8

func (s *Service) ListAssistants(ctx context.Context) ([]*models.Assistant, error)

ListAssistants returns all stored assistants, oldest first.

func (*Service) ListBatches added in v0.3.8

func (s *Service) ListBatches(ctx context.Context) ([]*models.BatchInfo, error)

ListBatches returns all stored batches, oldest first.

func (*Service) ListConversationItems added in v0.3.8

func (s *Service) ListConversationItems(
	ctx context.Context,
	id string,
	token *models.RouterToken,
) ([]models.ResponseInputItem, error)

ListConversationItems returns the items appended to a conversation.

func (*Service) ListResponseInputItems added in v0.3.8

func (s *Service) ListResponseInputItems(
	ctx context.Context,
	id string,
	token *models.RouterToken,
) ([]models.ResponseInputItem, error)

ListResponseInputItems returns the stored input of a response as items: a stored string becomes one message item, stored items return as-is.

func (*Service) ListRuns added in v0.3.8

func (s *Service) ListRuns(
	ctx context.Context,
	threadID string,
) ([]*models.Run, error)

ListRuns returns all runs of a thread, oldest first.

func (*Service) ListThreadMessages added in v0.3.8

func (s *Service) ListThreadMessages(
	ctx context.Context,
	threadID string,
) ([]*models.ThreadMessage, error)

ListThreadMessages returns all messages of a thread, oldest first.

func (*Service) Moderate added in v0.3.8

func (s *Service) Moderate(
	ctx context.Context,
	request *models.ModerationRequest,
	token *models.RouterToken,
) (*models.ModerationResponse, error)

Moderate routes a POST /v1/moderations request in a single backend pass over the credential pool.

func (*Service) PollVideo added in v0.3.8

func (s *Service) PollVideo(
	ctx context.Context,
	jobID string,
	token *models.RouterToken,
) (*models.VideoGenerationResponse, error)

PollVideo routes a GET /v1/videos/{jobId} request: the stored job row supplies the backend model and provider, the credential pool supplies fresh credentials for the upstream poll.

func (*Service) Speech added in v0.0.5

func (s *Service) Speech(
	ctx context.Context,
	request *models.SpeechRequest,
	token *models.RouterToken,
) (*models.SpeechResponse, error)

Speech routes a POST /v1/audio/speech request in a single backend pass over the credential pool.

func (*Service) SubmitToolOutputs added in v0.3.8

func (s *Service) SubmitToolOutputs(
	ctx context.Context,
	threadID, runID string,
	token *models.RouterToken,
	outputs []models.ToolOutput,
) (*models.Run, error)

SubmitToolOutputs continues a requires_action run: tool outputs join the stashed turns, the pipeline runs again, and completion appends the assistant turn to the thread.

func (*Service) SubmitVideo added in v0.3.8

SubmitVideo routes a POST /v1/videos request in a single backend pass over the credential pool and persists the router-side job row mapping the local job ID to the upstream one.

func (*Service) TestCredential

func (s *Service) TestCredential(ctx context.Context, providerID, credentialID string, overrideModel string) TestResult

TestCredential verifies one credential through the type's check_health handler. Types without the handler report unsupported: no live request runs, so probing never spends quota or triggers traffic side effects. An explicit unhealthy verdict disables the credential at once (DisabledBy="healthcheck"); unknown and handler failures change nothing. overrideModel is accepted for endpoint compatibility and ignored.

func (*Service) TestModel

func (s *Service) TestModel(ctx context.Context, modelID models.ModelId, endpoint string) TestResult

TestModel verifies one model through the type's check_model handler. The plugin owns liveness payloads for every endpoint; the router owns resolution, timeout, latency and verdict mapping only. Types without the handler and built-in Go backends report unsupported with no live request. Manually disabled models stay testable. Probes never mutate state and never disable models; the dashboard disables explicitly on model_unavailable verdicts.

func (*Service) Transcribe added in v0.0.5

Transcribe routes a POST /v1/audio/transcriptions request in a single backend pass over the credential pool.

func (*Service) UpdateAssistant added in v0.3.8

func (s *Service) UpdateAssistant(
	ctx context.Context,
	id string,
	token *models.RouterToken,
	request *models.AssistantRequest,
) (*models.Assistant, error)

UpdateAssistant patches name, description, instructions, tools, response_format and metadata. Model changes re-parse the id.

func (*Service) UpdateThread added in v0.3.8

func (s *Service) UpdateThread(
	ctx context.Context,
	id string,
	metadata map[string]string,
) (*models.Thread, error)

UpdateThread patches thread metadata.

func (*Service) VideoContent added in v0.3.8

func (s *Service) VideoContent(
	ctx context.Context,
	jobID string,
	index int,
	token *models.RouterToken,
) (*models.VideoContentResponse, error)

VideoContent routes a GET /v1/videos/{jobId}/content request.

type TestResult

type TestResult struct {
	OK            bool   `json:"ok"`
	Latency       int64  `json:"latency_ms"`
	Error         string `json:"error,omitempty"`
	Response      string `json:"response,omitempty"`
	QuotaExceeded bool   `json:"quota_exceeded,omitempty"`
	Code          string `json:"code,omitempty"`
	Summary       string `json:"summary,omitempty"`
	Proxy         string `json:"proxy,omitempty"`
}

TestResult reports the outcome of a credential or model probe. QuotaExceeded marks temporary exhaustion: the model is alive, the quota is not. Callers must not disable models over it. Code is the machine-readable wire code for the failure (empty on success); Summary is a short human string for toasts, the full error goes to logs.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL