Documentation
¶
Overview ¶
Package v1 implements the OpenAI-compatible /v1/... API endpoints.
Incoming requests are:
- Authenticated via the Internal Token Service
- Validated against the token's rules (allowed models)
- Routed to the appropriate provider by the Router Service
- Translated back to OpenAI-compatible responses
Index ¶
- func BuildStreamMetricEvent(start time.Time, model models.ModelId, providerID, tokenID string, ...) models.MetricEvent
- func MarshalOpenAIStreamError(streamErr error) ([]byte, error)
- func WireMessage(err error) string
- func WriteOpenAIStreamError(w io.Writer, flusher interface{ ... }, streamErr error)
- type Handler
- type Params
- type StreamMetricsTap
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func BuildStreamMetricEvent ¶ added in v0.4.0
func BuildStreamMetricEvent(start time.Time, model models.ModelId, providerID, tokenID string, duration time.Duration, streamErr error, usage *models.ChatCompletionUsage) models.MetricEvent
BuildStreamMetricEvent builds the shared MetricEvent for one streaming request that committed (status 200 already sent). A mid-stream failure records the classified status and code, the same mapping as unary requests. usage carries the token counts seen in the stream (nil when the upstream sent none).
func MarshalOpenAIStreamError ¶ added in v0.4.0
MarshalOpenAIStreamError renders the OpenAI-shaped error envelope sent as an SSE data frame when a stream fails mid-flight. The message follows the edge contract (WireMessage): plugin messages verbatim, never a wrapped Go error chain. Callers own the write and flush so the Anthropic path (translatedWriter.emit) stays separate.
func WireMessage ¶ added in v0.6.9
WireMessage renders the client-facing message: the raw plugin message for terminal provider errors, the router-authored sentinel text otherwise. Every edge (v1 unary, v1 streams, dashboard chat) renders through it.
func WriteOpenAIStreamError ¶ added in v0.4.0
WriteOpenAIStreamError marshals the error envelope and emits it as an SSE data frame. It shares the exact write sequence of the OpenAI streaming paths; the dashboard path keeps its own guarded write on top of MarshalOpenAIStreamError.
Types ¶
type Handler ¶
type Handler struct {
// contains filtered or unexported fields
}
Handler holds the dependencies for the v1 API.
type Params ¶ added in v0.1.2
type Params struct {
Tokens *token.GormService
RouterSvc *router.Service
MetricsSvc *metrics.Service
ProviderSvc *provider.GormService
ModelInfoSvc *modelinfo.GormService
VirtualSvc *virtual.GormService
Logger *slog.Logger
}
Params carries every dependency of the v1 Handler in one value, so adding a service touches one struct instead of every call site.
type StreamMetricsTap ¶ added in v0.6.9
type StreamMetricsTap struct {
// contains filtered or unexported fields
}
StreamMetricsTap forwards OpenAI SSE bytes unchanged and keeps what metrics need from the data frames: the last usage object and the model that answered (the router stamps it on every frame; a virtual request carries its member). Partial frames stay buffered until their newline.
func NewStreamMetricsTap ¶ added in v0.6.9
func NewStreamMetricsTap(next io.Writer, flusher interface{ Flush() }) *StreamMetricsTap
NewStreamMetricsTap wraps next; flusher receives forwarded Flush calls.
func (*StreamMetricsTap) Flush ¶ added in v0.6.9
func (t *StreamMetricsTap) Flush()
Flush forwards to the downstream flusher.
func (*StreamMetricsTap) Model ¶ added in v0.6.9
func (t *StreamMetricsTap) Model(requested models.ModelId) models.ModelId
Model returns the answering model from the first frame that named one, or requested when none did.
func (*StreamMetricsTap) Usage ¶ added in v0.6.9
func (t *StreamMetricsTap) Usage() *models.ChatCompletionUsage
Usage returns the last usage object seen, or nil.