Documentation
¶
Overview ¶
Package relay provides the core relay types: a stripped-down RelayInfo (no user/token/billing/DB fields) and a clean Adaptor interface that uses context.Context instead of gin.Context.
This is adapted from LingRein's pkg/relay/common and pkg/relay/channel, with all product-layer coupling (Gin, GORM, user management, billing, subscription) removed.
Index ¶
- Constants
- Variables
- func CtxGet(ctx context.Context, key string) (any, bool)
- func GetTimestamp() int64
- func QuotaFromFloat(f float64) int
- type Adaptor
- type AdaptorRegistry
- type ChannelMeta
- type HTTPClient
- type RelayInfo
- func (info *RelayInfo) AppendRequestConversion(format types.RelayFormat)
- func (info *RelayInfo) ConvOptions() *convmeta.Options
- func (info *RelayInfo) EnsureClaudeConvertInfo() *convmeta.ClaudeConvertInfo
- func (info *RelayInfo) GetChannelID() int
- func (info *RelayInfo) GetChannelType() int
- func (info *RelayInfo) GetEstimatePromptTokens() int
- func (info *RelayInfo) GetIsStream() bool
- func (info *RelayInfo) GetOriginModelName() string
- func (info *RelayInfo) GetReasoningEffort() string
- func (info *RelayInfo) GetSendResponseCount() int
- func (info *RelayInfo) GetUpstreamModelName() string
- func (info *RelayInfo) HasChannelMeta() bool
- func (info *RelayInfo) IncrSendResponseCount()
- func (info *RelayInfo) SetReasoningEffort(effort string)
- type TaskAdaptor
- type TaskAdaptorRegistry
- type TaskError
- type TaskInfo
- type TaskSubmitReq
Constants ¶
const MaxTaskDurationSeconds = 600
MaxTaskDurationSeconds is the upper bound for task duration billing.
Variables ¶
var DefaultRegistry = NewAdaptorRegistry()
DefaultRegistry is the global adaptor registry.
var DefaultTaskRegistry = NewTaskAdaptorRegistry()
DefaultTaskRegistry is the global task adaptor registry.
Functions ¶
func QuotaFromFloat ¶
QuotaFromFloat converts a float64 to an int quota value. In library mode this is a simple truncation.
Types ¶
type Adaptor ¶
type Adaptor interface {
// Init initializes the adaptor with the relay info.
Init(info *RelayInfo)
// GetRequestURL returns the upstream URL for this request.
GetRequestURL(info *RelayInfo) (string, error)
// SetupRequestHeader sets auth and content-type headers on the request.
SetupRequestHeader(ctx context.Context, header *http.Header, info *RelayInfo) error
// ConvertOpenAIRequest converts an OpenAI-format chat request to the
// provider's native format.
ConvertOpenAIRequest(ctx context.Context, info *RelayInfo, request *dto.GeneralOpenAIRequest) (any, error)
// ConvertClaudeRequest converts a Claude Messages request to the
// provider's native format.
ConvertClaudeRequest(ctx context.Context, info *RelayInfo, request *dto.ClaudeRequest) (any, error)
// ConvertGeminiRequest converts a Gemini generateContent request to the
// provider's native format.
ConvertGeminiRequest(ctx context.Context, info *RelayInfo, request *dto.GeminiChatRequest) (any, error)
// ConvertImageRequest converts an image generation request.
ConvertImageRequest(ctx context.Context, info *RelayInfo, request dto.ImageRequest) (any, error)
// ConvertEmbeddingRequest converts an embedding request.
ConvertEmbeddingRequest(ctx context.Context, info *RelayInfo, request dto.EmbeddingRequest) (any, error)
// ConvertAudioRequest converts an audio (TTS/ASR) request.
ConvertAudioRequest(ctx context.Context, info *RelayInfo, request dto.AudioRequest) (io.Reader, error)
// ConvertRerankRequest converts a rerank request.
ConvertRerankRequest(ctx context.Context, relayMode int, request dto.RerankRequest) (any, error)
// ConvertOpenAIResponsesRequest converts an OpenAI Responses request.
ConvertOpenAIResponsesRequest(ctx context.Context, info *RelayInfo, request dto.OpenAIResponsesRequest) (any, error)
// DoRequest sends the HTTP request to the upstream provider.
DoRequest(ctx context.Context, info *RelayInfo, requestBody io.Reader) (*http.Response, error)
// DoResponse parses the upstream response and returns usage info.
// For streaming responses, this writes to the provided http.ResponseWriter.
DoResponse(ctx context.Context, resp *http.Response, info *RelayInfo, w http.ResponseWriter) (usage any, err *types.NewAPIError)
// GetModelList returns the models supported by this provider.
GetModelList() []string
// GetChannelName returns the provider name.
GetChannelName() string
}
Adaptor is the provider adapter interface, adapted from LingRein's channel.Adaptor with gin.Context removed. Each provider (OpenAI, Claude, Gemini, ...) implements this interface.
The flow for a relay request is:
- Init(info) — initialize adaptor with request context
- ConvertXxxRequest(ctx, info, request) — convert unified request to provider's native format
- GetRequestURL(info) — build upstream URL
- SetupRequestHeader(ctx, header, info) — set auth/content-type headers
- DoRequest(ctx, info, body) — send HTTP request to upstream
- DoResponse(ctx, resp, info) — parse upstream response, return usage
func GetAdaptor ¶
GetAdaptor returns an adaptor for the given API type from the default registry.
type AdaptorRegistry ¶
type AdaptorRegistry struct {
// contains filtered or unexported fields
}
AdaptorRegistry maps API types to adaptor factories.
func NewAdaptorRegistry ¶
func NewAdaptorRegistry() *AdaptorRegistry
NewAdaptorRegistry creates an empty registry.
func (*AdaptorRegistry) Get ¶
func (r *AdaptorRegistry) Get(apiType int) Adaptor
Get returns a new adaptor instance for the given API type.
func (*AdaptorRegistry) Register ¶
func (r *AdaptorRegistry) Register(apiType int, factory func() Adaptor)
Register associates an API type with an adaptor factory.
type ChannelMeta ¶
type ChannelMeta struct {
ChannelType int // constant.ChannelType*
ApiType int // constant.APIType*
ApiVersion string // Azure API version, Vertex region, etc.
ApiKey string // upstream API key
Organization string // OpenAI org ID, etc.
ChannelBaseUrl string // override default endpoint
ParamOverride map[string]any // parameter overrides
HeadersOverride map[string]any // header overrides
UpstreamModelName string // model name after mapping
IsModelMapped bool
SupportStreamOptions bool
}
ChannelMeta holds channel/provider configuration for a relay request. This is a stripped version of LingRein's ChannelMeta — no ChannelId, ChannelIsMultiKey, or ChannelSetting (which were DB-coupled).
type HTTPClient ¶
HTTPClient returns the HTTP client to use for upstream calls. Can be overridden per-request.
type RelayInfo ¶
type RelayInfo struct {
IsStream bool
RelayMode int // relaymode.RelayMode*
OriginModelName string // original model name from client
RequestURLPath string // e.g. "/v1/chat/completions"
RequestHeaders map[string]string
ShouldIncludeUsage bool
RelayFormat types.RelayFormat // input format (openai/claude/gemini)
StartTime time.Time
// Channel/provider metadata.
*ChannelMeta
// The parsed request DTO (GeneralOpenAIRequest, ClaudeRequest, etc.)
Request dto.Request
// Request format conversion chain, e.g. ["openai", "claude"].
RequestConversionChain []types.RelayFormat
// Final request format sent to upstream.
FinalRequestRelayFormat types.RelayFormat
// API type for adaptor dispatch.
ApiType int
// Reasoning effort ("low", "medium", "high", "").
ReasoningEffort string
// Action is the task action type (e.g. constant.TaskActionGenerate).
// Used by async task providers.
Action string
// PublicTaskID is the client-facing task ID.
PublicTaskID string
// OriginTaskID is the original task ID for remix operations.
OriginTaskID string
// SendResponseCount / ReceivedResponseCount for streaming.
SendResponseCount int
ReceivedResponseCount int
// contains filtered or unexported fields
}
RelayInfo is the per-request context for a relay call. It is a stripped version of LingRein's RelayInfo — all user/token/billing/ subscription/quota/WebSocket fields have been removed.
func NewRelayInfo ¶
func NewRelayInfo() *RelayInfo
NewRelayInfo creates a RelayInfo with sensible defaults.
func (*RelayInfo) AppendRequestConversion ¶
func (info *RelayInfo) AppendRequestConversion(format types.RelayFormat)
func (*RelayInfo) ConvOptions ¶
convOptions is a cached conversion options snapshot. Zero value = every adaptation disabled, no defaults applied.
func (*RelayInfo) EnsureClaudeConvertInfo ¶
func (info *RelayInfo) EnsureClaudeConvertInfo() *convmeta.ClaudeConvertInfo
func (*RelayInfo) GetChannelID ¶
func (*RelayInfo) GetChannelType ¶
func (*RelayInfo) GetEstimatePromptTokens ¶
func (*RelayInfo) GetIsStream ¶
func (*RelayInfo) GetOriginModelName ¶
func (*RelayInfo) GetReasoningEffort ¶
func (*RelayInfo) GetSendResponseCount ¶
func (*RelayInfo) GetUpstreamModelName ¶
func (*RelayInfo) HasChannelMeta ¶
func (*RelayInfo) IncrSendResponseCount ¶
func (info *RelayInfo) IncrSendResponseCount()
func (*RelayInfo) SetReasoningEffort ¶
type TaskAdaptor ¶
type TaskAdaptor interface {
Init(info *RelayInfo)
// ValidateRequestAndSetAction validates the request and sets the action type.
ValidateRequestAndSetAction(ctx context.Context, info *RelayInfo) *TaskError
BuildRequestURL(info *RelayInfo) (string, error)
BuildRequestHeader(ctx context.Context, req *http.Request, info *RelayInfo) error
BuildRequestBody(ctx context.Context, info *RelayInfo) (io.Reader, error)
DoRequest(ctx context.Context, info *RelayInfo, requestBody io.Reader) (*http.Response, error)
DoResponse(ctx context.Context, resp *http.Response, info *RelayInfo) (taskID string, taskData []byte, err *TaskError)
GetModelList() []string
GetChannelName() string
FetchTask(baseURL, key string, body map[string]any, proxy string) (*http.Response, error)
ParseTaskResult(respBody []byte) (*TaskInfo, error)
}
TaskAdaptor is the interface for async task providers (video/music generation). It is a simplified version of LingRein's TaskAdaptor with billing methods removed.
The flow for an async task is:
- Init(info) — initialize adaptor
- ValidateRequestAndSetAction(ctx, info) — validate request and set action type
- BuildRequestURL(info) — build upstream URL
- BuildRequestHeader(ctx, req, info) — set headers
- BuildRequestBody(ctx, info) — build request body
- DoRequest(ctx, info, body) — submit task to upstream
- DoResponse(ctx, resp, info) — parse submit response, return taskID
- FetchTask(baseURL, key, body, proxy) — poll task status
- ParseTaskResult(respBody) — parse poll response into TaskInfo
func GetTaskAdaptor ¶
func GetTaskAdaptor(apiType int) TaskAdaptor
GetTaskAdaptor returns a task adaptor for the given API type.
type TaskAdaptorRegistry ¶
type TaskAdaptorRegistry struct {
// contains filtered or unexported fields
}
TaskAdaptorRegistry maps API types to task adaptor factories.
func NewTaskAdaptorRegistry ¶
func NewTaskAdaptorRegistry() *TaskAdaptorRegistry
NewTaskAdaptorRegistry creates an empty registry.
func (*TaskAdaptorRegistry) Get ¶
func (r *TaskAdaptorRegistry) Get(apiType int) TaskAdaptor
Get returns a new task adaptor instance for the given API type.
func (*TaskAdaptorRegistry) Register ¶
func (r *TaskAdaptorRegistry) Register(apiType int, factory func() TaskAdaptor)
Register associates an API type with a task adaptor factory.
type TaskError ¶
type TaskError struct {
Code string `json:"code"`
Message string `json:"message"`
Data any `json:"data"`
StatusCode int `json:"-"`
LocalError bool `json:"-"`
Err error `json:"-"`
}
TaskError represents an error from a task provider.
func ValidateBasicTaskRequest ¶
ValidateBasicTaskRequest is a stub for LingRein's request validation. In library mode it performs no validation.
func ValidateMultipartDirect ¶
ValidateMultipartDirect is a stub for LingRein's multipart validation.
type TaskInfo ¶
type TaskInfo struct {
Code int `json:"code"`
TaskID string `json:"task_id"`
Status string `json:"status"` // "PENDING", "RUNNING", "SUCCESS", "FAILURE"
Reason string `json:"reason,omitempty"`
Url string `json:"url,omitempty"`
RemoteUrl string `json:"remote_url,omitempty"`
Progress string `json:"progress,omitempty"`
CompletionTokens int `json:"completion_tokens,omitempty"`
TotalTokens int `json:"total_tokens,omitempty"`
}
TaskInfo holds the result of polling an async task.
func FailTaskInfo ¶
FailTaskInfo creates a failed TaskInfo.
type TaskSubmitReq ¶
type TaskSubmitReq struct {
Prompt string `json:"prompt"`
Images []string `json:"images,omitempty"`
Duration int `json:"duration,omitempty"`
Seconds string `json:"seconds,omitempty"`
Size string `json:"size,omitempty"`
Metadata map[string]any `json:"metadata,omitempty"`
Model string `json:"model,omitempty"`
Mode string `json:"mode,omitempty"`
Image string `json:"image,omitempty"`
InputReference string `json:"input_reference,omitempty"`
}
TaskSubmitReq is a stub for LingRein's task submission request type. In LingRein this is parsed from the HTTP request body by gin middleware. In library mode, callers populate this struct directly.
func GetTaskRequest ¶
func GetTaskRequest(ctx context.Context) (TaskSubmitReq, error)
GetTaskRequest is a stub that retrieves the task request from context. In library mode there is no gin context, so it returns a zero value.
func (*TaskSubmitReq) HasImage ¶
func (r *TaskSubmitReq) HasImage() bool
HasImage reports whether the request has an image input.
func (*TaskSubmitReq) UnmarshalMetadata ¶
func (r *TaskSubmitReq) UnmarshalMetadata(target any) error
UnmarshalMetadata marshals the request's Metadata map and unmarshals it into the target struct, allowing callers to decode metadata into typed structs in library mode.