Documentation
¶
Overview ¶
Package modelbridge converts between LLM API protocol formats (OpenAI Chat, OpenAI Responses, Claude, Gemini, Ollama).
Index ¶
- Constants
- Variables
- func Protocols() []string
- type CompatibilityWarning
- type ConvertedRequest
- type ConvertedResponse
- type ConvertedStream
- type Converter
- func (c *Converter) ConvertRequest(ctx context.Context, in RequestConversion) (*ConvertedRequest, error)
- func (c *Converter) ConvertResponse(ctx context.Context, in ResponseConversion) (*ConvertedResponse, error)
- func (c *Converter) ConvertStream(ctx context.Context, in StreamConversion, out io.Writer) (*ConvertedStream, error)
- func (c *Converter) DecodeRequestSummary(in RequestSummaryConversion) (*RequestSummary, error)
- type Options
- type RequestConversion
- type RequestSummary
- type RequestSummaryConversion
- type ResponseConversion
- type ResponseInfo
- type ResponseOptions
- type StreamConversion
- type StreamEventInfo
- type StreamEventType
- type ToolCallDeltaInfo
- type UsageInfo
Constants ¶
const ( ProtocolOpenAIChat = "openai-chat" ProtocolOpenAIResponses = "openai-responses" ProtocolClaude = "claude" ProtocolGemini = "gemini" ProtocolOllama = "ollama" )
GatewayMetricsField is the response field the bridge uses for its own telemetry extension. It is exported so library callers can read or strip the field without hard-coding the name.
Variables ¶
ErrEmptyUpstreamResponse reports a response with no protocol payload at all. Treating it as an empty assistant message would manufacture success from a broken or prematurely closed upstream connection.
var ErrIncompleteStream = errors.New("upstream stream ended without a semantic finish event")
var ErrMalformedUpstreamResponse = protocol.ErrMalformedUpstreamResponse
ErrMalformedUpstreamResponse reports a body that cannot be decoded as the configured target protocol. The converter never fabricates a successful assistant response from undecodable bytes.
var ErrUpstreamResponseFailed = errors.New("upstream response reported a failed terminal status")
ErrUpstreamResponseFailed reports a syntactically valid non-streaming response whose protocol-level terminal status is failed/cancelled. Such a response must never be re-encoded by a protocol that would turn it into a successful stop.
Functions ¶
Types ¶
type CompatibilityWarning ¶ added in v1.20.2
CompatibilityWarning reports a non-fatal conversion detail without exposing provider values. The converted wire payload remains available and the remote endpoint makes the final capability decision.
type ConvertedRequest ¶
type ConvertedRequest struct {
Method string
URL string
Path string
Header http.Header
Summary RequestSummary
Warnings []CompatibilityWarning
// Request is the fully constructed upstream HTTP request.
// Its Body has been read and reset; callers may read it multiple times.
Request *http.Request
}
ConvertedRequest is the target-protocol upstream request built from a source client request.
type ConvertedResponse ¶
type ConvertedResponse struct {
Status int
Header http.Header
Body []byte
Info ResponseInfo
Warnings []CompatibilityWarning
}
type ConvertedStream ¶
type ConvertedStream struct {
Status int
Header http.Header
Warnings []CompatibilityWarning
}
type Converter ¶
type Converter struct {
// contains filtered or unexported fields
}
func (*Converter) ConvertRequest ¶
func (c *Converter) ConvertRequest(ctx context.Context, in RequestConversion) (*ConvertedRequest, error)
func (*Converter) ConvertResponse ¶
func (c *Converter) ConvertResponse(ctx context.Context, in ResponseConversion) (*ConvertedResponse, error)
func (*Converter) ConvertStream ¶
func (c *Converter) ConvertStream(ctx context.Context, in StreamConversion, out io.Writer) (*ConvertedStream, error)
ConvertStream converts an upstream streaming response from TargetProtocol to SourceProtocol and writes the converted stream to out as events arrive. It closes the upstream body when conversion finishes or ctx is canceled.
func (*Converter) DecodeRequestSummary ¶
func (c *Converter) DecodeRequestSummary(in RequestSummaryConversion) (*RequestSummary, error)
type Options ¶
type Options struct {
// AnthropicVersion is the fallback anthropic-version header for Claude targets.
AnthropicVersion string
// DefaultMaxTokens is used for Claude targets when the inbound request omits max tokens.
DefaultMaxTokens int
// AnthropicSendAuthorization also sends Authorization: Bearer <token> to Claude targets.
AnthropicSendAuthorization bool
// StreamFrameLimit bounds one upstream SSE/NDJSON frame. Zero uses the safe
// default; a negative value disables the bound for a trusted endpoint.
StreamFrameLimit int
}
type RequestConversion ¶
type RequestConversion struct {
SourceProtocol string
TargetProtocol string
SourcePath string
// SourceHeader is the inbound client request header. Claude targets use it
// to pass through anthropic-version and anthropic-beta when present.
SourceHeader http.Header
Body []byte
TargetBaseURL string
TargetToken string
TargetModel string
}
RequestConversion converts an inbound client request from SourceProtocol into an upstream request for TargetProtocol.
type RequestSummary ¶
type RequestSummaryConversion ¶
type ResponseConversion ¶
type ResponseConversion struct {
SourceProtocol string
TargetProtocol string
Body []byte
Options ResponseOptions
}
ResponseConversion converts an upstream response body from TargetProtocol back into the inbound client SourceProtocol.
type ResponseInfo ¶
type ResponseInfo struct {
ID string
Model string
Text string
ReasoningText string
FinishReason string
Usage *UsageInfo
// GatewayMetrics carries this bridge's own timing/throughput telemetry
// for the converted response. It is a pointer so ResponseInfo stays
// comparable for callers that use it as a map key. It is nil when the
// caller did not supply telemetry (library conversions that are not part
// of a timed forward).
GatewayMetrics *metrics.Sample
}
type ResponseOptions ¶
type ResponseOptions struct {
// ExposeReasoning controls whether Claude-source responses expose
// reasoning/thinking content to the downstream Claude client. Integrity
// blocks accompanying a tool call are retained on same-Claude relays because
// some compatible upstreams require them on the next turn.
ExposeReasoning bool
}
type StreamConversion ¶
type StreamConversion struct {
SourceProtocol string
TargetProtocol string
Options ResponseOptions
// OnEvent observes normalized semantic events accepted by the destination
// writer after ResponseOptions display filtering. Provider Raw carrier frames
// are omitted; their standard semantic companion events are reported once.
// The callback is synchronous and should stay lightweight.
OnEvent func(StreamEventInfo)
// Response is the upstream streaming response. When set, Body, Header, and
// StatusCode are read from it.
Response *http.Response
// Body is the upstream streaming response body. It is used when Response is
// nil, and is closed after conversion when it also implements io.Closer.
Body io.Reader
Header http.Header
StatusCode int
}
StreamConversion converts an upstream streaming response from TargetProtocol back into the inbound client SourceProtocol.
type StreamEventInfo ¶
type StreamEventType ¶
type StreamEventType string
const ( StreamEventStart StreamEventType = "start" StreamEventTextDelta StreamEventType = "text_delta" StreamEventReasoningDelta StreamEventType = "reasoning_delta" StreamEventReasoningSignature StreamEventType = "reasoning_signature" StreamEventToolCallStart StreamEventType = "tool_call_start" StreamEventToolCallDelta StreamEventType = "tool_call_delta" StreamEventFinish StreamEventType = "finish" StreamEventUsage StreamEventType = "usage" StreamEventDone StreamEventType = "done" )
type UsageInfo ¶
type UsageInfo struct {
InputTokens int
OutputTokens int
TotalTokens int
CacheReadTokens int
CacheCreationTokens int
CacheMissTokens int
ReasoningTokens int
Extra map[string]any
// TokenSource reports whether the counters above came from the upstream
// or were estimated locally. It mirrors the value exposed in
// GatewayMetrics and is empty for library conversions that carry no
// provenance information.
TokenSource string
}
Source Files
¶
- converter.go
- events.go
- options.go
- stream.go
- summary.go