llmprotocol

package
v0.3.8 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 4, 2026 License: Apache-2.0 Imports: 11 Imported by: 0

Documentation

Overview

Package llmprotocol provides provider protocol conversion for LLM requests and responses.

Index

Constants

This section is empty.

Variables

View Source
var (
	// OpenAIChatCapabilities defines capabilities for the OpenAI Chat Completions API.
	OpenAIChatCapabilities = CapabilitySet{
		CapText:              true,
		CapToolCall:          true,
		CapReasoning:         true,
		CapImage:             true,
		CapParallelToolCalls: true,
	}

	// OpenAIResponsesCapabilities defines capabilities for the OpenAI Responses API.
	OpenAIResponsesCapabilities = CapabilitySet{
		CapText:              true,
		CapToolCall:          true,
		CapReasoning:         true,
		CapImage:             true,
		CapParallelToolCalls: true,
	}

	// AnthropicMessagesCapabilities defines capabilities for the Anthropic Messages API.
	AnthropicMessagesCapabilities = CapabilitySet{
		CapText:              true,
		CapToolCall:          true,
		CapReasoning:         true,
		CapImage:             true,
		CapFile:              true,
		CapParallelToolCalls: true,
	}

	// GeminiCapabilities defines capabilities for the Google Gemini API.
	GeminiCapabilities = CapabilitySet{
		CapText:              true,
		CapToolCall:          true,
		CapReasoning:         true,
		CapImage:             true,
		CapAudio:             true,
		CapFile:              true,
		CapParallelToolCalls: true,
	}
)

Predefined capability sets for supported protocols.

Functions

func CanonicalToolArguments added in v0.1.5

func CanonicalToolArguments(raw string, parsed map[string]interface{}) string

CanonicalToolArguments returns the JSON string used for wire-level tool call arguments. Raw JSON strings are preserved for protocols that already carry arguments as strings; structured argument objects are marshaled with stable map-key ordering for prompt-cache-friendly protocol conversion.

func NormalizeRequest

func NormalizeRequest(ir *IRRequest, targetCaps CapabilitySet) (*IRRequest, []Warning, error)

NormalizeRequest adapts an IR request to the constraints of a target protocol.

func RegisterAdapter

func RegisterAdapter(a ProtocolAdapter) error

RegisterAdapter registers a ProtocolAdapter for its protocol. It returns an error when the adapter is invalid or conflicts with an existing registration.

func TestGolden_AnthropicToChat_ResponseRoundTrip

func TestGolden_AnthropicToChat_ResponseRoundTrip(t *testing.T)

func TestGolden_AnthropicToChat_RoundTrip

func TestGolden_AnthropicToChat_RoundTrip(t *testing.T)

func TestGolden_ChatToAnthropic_RoundTrip

func TestGolden_ChatToAnthropic_RoundTrip(t *testing.T)

func TestGolden_ChatToGemini_RoundTrip

func TestGolden_ChatToGemini_RoundTrip(t *testing.T)

func TestGolden_ChatToResponses_ResponseRoundTrip

func TestGolden_ChatToResponses_ResponseRoundTrip(t *testing.T)

func TestGolden_ChatToResponses_RoundTrip

func TestGolden_ChatToResponses_RoundTrip(t *testing.T)

func TestGolden_GeminiToChat_ResponseRoundTrip

func TestGolden_GeminiToChat_ResponseRoundTrip(t *testing.T)

func TestGolden_GeminiToChat_RoundTrip

func TestGolden_GeminiToChat_RoundTrip(t *testing.T)

func TestGolden_ResponsesToChat_ResponseRoundTrip

func TestGolden_ResponsesToChat_ResponseRoundTrip(t *testing.T)

func TestGolden_ResponsesToChat_RoundTrip

func TestGolden_ResponsesToChat_RoundTrip(t *testing.T)

func TestIntegration_AnthropicToChat_NonStream

func TestIntegration_AnthropicToChat_NonStream(t *testing.T)

func TestIntegration_AnthropicToChat_StreamText

func TestIntegration_AnthropicToChat_StreamText(t *testing.T)

func TestIntegration_ChatToAnthropic_NonStream

func TestIntegration_ChatToAnthropic_NonStream(t *testing.T)

func TestIntegration_ChatToAnthropic_StreamText

func TestIntegration_ChatToAnthropic_StreamText(t *testing.T)

func TestIntegration_ChatToGemini_NonStream

func TestIntegration_ChatToGemini_NonStream(t *testing.T)

func TestIntegration_ChatToResponses_NonStream

func TestIntegration_ChatToResponses_NonStream(t *testing.T)

func TestIntegration_GeminiToChat_NonStream

func TestIntegration_GeminiToChat_NonStream(t *testing.T)

func TestIntegration_GeminiToChat_StreamText

func TestIntegration_GeminiToChat_StreamText(t *testing.T)

func TestIntegration_ResponsesToChat_NonStream

func TestIntegration_ResponsesToChat_NonStream(t *testing.T)

func TestIntegration_ResponsesToChat_StreamText

func TestIntegration_ResponsesToChat_StreamText(t *testing.T)

func UpstreamAPIPath

func UpstreamAPIPath(proto Protocol, hasV1 bool) string

UpstreamAPIPath returns the API path suffix for the given protocol.

Types

type Capability

type Capability string

Capability represents a single model capability.

const (
	CapText              Capability = "text"
	CapToolCall          Capability = "tool_call"
	CapReasoning         Capability = "reasoning"
	CapImage             Capability = "image"
	CapAudio             Capability = "audio"
	CapFile              Capability = "file"
	CapParallelToolCalls Capability = "parallel_tool_calls"
)

type CapabilitySet

type CapabilitySet map[Capability]bool

CapabilitySet is a set of capabilities supported by a target protocol.

func CapabilitiesForProtocol

func CapabilitiesForProtocol(proto Protocol) CapabilitySet

CapabilitiesForProtocol returns the capability set for a supported protocol.

func (CapabilitySet) Has

func (cs CapabilitySet) Has(cap Capability) bool

Has returns true if the capability is set.

type IRContentPart

type IRContentPart struct {
	ID           string                 `json:"id,omitempty"`
	Type         IRPartType             `json:"type"`
	Text         string                 `json:"text,omitempty"`
	ToolCall     *IRToolCallPart        `json:"tool_call,omitempty"`
	ToolResult   *IRToolResultPart      `json:"tool_result,omitempty"`
	Reasoning    *IRReasoningPart       `json:"reasoning,omitempty"`
	Refusal      *IRRefusalPart         `json:"refusal,omitempty"`
	Metadata     map[string]string      `json:"metadata,omitempty"`
	CacheControl map[string]interface{} `json:"cache_control,omitempty"`
}

IRContentPart is a single part within a message.

type IRMessage

type IRMessage struct {
	ID    string          `json:"id,omitempty"`
	Role  IRRole          `json:"role"`
	Name  string          `json:"name,omitempty"`
	Parts []IRContentPart `json:"parts,omitempty"`
}

IRMessage is a single message in a conversation.

func (IRMessage) GetTextContent

func (m IRMessage) GetTextContent() string

GetTextContent returns concatenated text from all text parts.

type IRPartType

type IRPartType string

IRPartType represents the type of a content part.

const (
	IRPartText       IRPartType = "text"
	IRPartToolCall   IRPartType = "tool_call"
	IRPartToolResult IRPartType = "tool_result"
	IRPartReasoning  IRPartType = "reasoning"
	IRPartRefusal    IRPartType = "refusal"
	IRPartImage      IRPartType = "image"
	IRPartAudio      IRPartType = "audio"
	IRPartFile       IRPartType = "file"
)

type IRReasoningPart

type IRReasoningPart struct {
	Content          string `json:"content"`
	Signature        string `json:"signature,omitempty"`
	Subtype          string `json:"subtype,omitempty"`
	EncryptedContent string `json:"encrypted_content,omitempty"`
}

IRReasoningPart holds model reasoning/thinking content.

type IRRefusalPart

type IRRefusalPart struct {
	Text string `json:"text"`
}

IRRefusalPart holds model refusal content.

type IRRequest

type IRRequest struct {
	ID                string                            `json:"id,omitempty"`
	Model             string                            `json:"model"`
	Stream            bool                              `json:"stream,omitempty"`
	User              string                            `json:"user,omitempty"`
	System            string                            `json:"system,omitempty"`
	SystemParts       []IRSystemPart                    `json:"system_parts,omitempty"`
	Instructions      string                            `json:"instructions,omitempty"`
	Messages          []IRMessage                       `json:"messages,omitempty"`
	Tools             []IRToolDecl                      `json:"tools,omitempty"`
	ToolChoice        *IRToolChoice                     `json:"tool_choice,omitempty"`
	ResponseFormat    *IRResponseFormat                 `json:"response_format,omitempty"`
	ReasoningEffort   string                            `json:"reasoning_effort,omitempty"`
	Thinking          *IRThinkingConfig                 `json:"thinking,omitempty"`
	MaxTokens         int                               `json:"max_tokens,omitempty"`
	Temperature       *float64                          `json:"temperature,omitempty"`
	TopP              *float64                          `json:"top_p,omitempty"`
	TopK              *int                              `json:"top_k,omitempty"`
	FrequencyPenalty  *float64                          `json:"frequency_penalty,omitempty"`
	PresencePenalty   *float64                          `json:"presence_penalty,omitempty"`
	Stop              []string                          `json:"stop,omitempty"`
	Seed              *int                              `json:"seed,omitempty"`
	ParallelToolCalls *bool                             `json:"parallel_tool_calls,omitempty"`
	StreamOptions     map[string]interface{}            `json:"stream_options,omitempty"`
	Metadata          map[string]interface{}            `json:"metadata,omitempty"`
	Store             *bool                             `json:"store,omitempty"`
	Include           []string                          `json:"include,omitempty"`
	CacheControl      map[string]interface{}            `json:"cache_control,omitempty"`
	Extensions        map[string]map[string]interface{} `json:"-"` // per-protocol, not serialized
}

IRRequest is the canonical request in the IR.

type IRResponse

type IRResponse struct {
	ID         string                            `json:"id"`
	Model      string                            `json:"model"`
	Created    int64                             `json:"created,omitempty"`
	Content    []IRContentPart                   `json:"content,omitempty"`
	Usage      *IRUsage                          `json:"usage,omitempty"`
	StopReason IRStopReason                      `json:"stop_reason,omitempty"`
	Extensions map[string]map[string]interface{} `json:"-"`
}

IRResponse is the canonical response in the IR.

type IRResponseFormat

type IRResponseFormat struct {
	Type       string                 `json:"type"`
	JSONSchema map[string]interface{} `json:"json_schema,omitempty"`
}

IRResponseFormat represents the response format configuration.

type IRRole

type IRRole string

IRRole represents a message role in the IR.

const (
	IRRoleUser      IRRole = "user"
	IRRoleAssistant IRRole = "assistant"
	IRRoleSystem    IRRole = "system"
	IRRoleTool      IRRole = "tool"
)

type IRStopReason

type IRStopReason string

IRStopReason represents why a response stopped.

const (
	IRStopEndTurn       IRStopReason = "end_turn"
	IRStopToolUse       IRStopReason = "tool_use"
	IRStopStopSequence  IRStopReason = "stop_sequence"
	IRStopMaxTokens     IRStopReason = "max_tokens"
	IRStopContentFilter IRStopReason = "content_filter"
	IRStopLength        IRStopReason = "length"
	IRStopError         IRStopReason = "error"
)

type IRStreamEvent

type IRStreamEvent struct {
	Type          IRStreamEventType `json:"type"`
	ResponseID    string            `json:"response_id,omitempty"`
	ResponseModel string            `json:"response_model,omitempty"`
	Index         int               `json:"index,omitempty"`
	Part          *IRContentPart    `json:"part,omitempty"`
	DeltaText     string            `json:"delta_text,omitempty"`
	DeltaJSON     string            `json:"delta_json,omitempty"`
	DeltaType     string            `json:"delta_type,omitempty"`
	StopReason    IRStopReason      `json:"stop_reason,omitempty"`
	Usage         *IRUsage          `json:"usage,omitempty"`
	ErrorMessage  string            `json:"error_message,omitempty"`
	ErrorType     string            `json:"error_type,omitempty"`
}

IRStreamEvent is a single streaming event in the IR.

type IRStreamEventType

type IRStreamEventType string

IRStreamEventType represents the type of a streaming event.

const (
	IRStreamMessageStart IRStreamEventType = "message_start"
	IRStreamContentStart IRStreamEventType = "content_part_start"
	IRStreamContentDelta IRStreamEventType = "content_part_delta"
	IRStreamContentStop  IRStreamEventType = "content_part_stop"
	IRStreamMessageDelta IRStreamEventType = "message_delta"
	IRStreamDone         IRStreamEventType = "done"
	IRStreamError        IRStreamEventType = "error"
)

type IRSystemPart added in v0.1.5

type IRSystemPart struct {
	Type         string                 `json:"type"`
	Text         string                 `json:"text,omitempty"`
	CacheControl map[string]interface{} `json:"cache_control,omitempty"`
}

IRSystemPart preserves block-level system prompt metadata for protocols such as Anthropic that allow cache breakpoints on system text blocks.

type IRThinkingConfig added in v0.1.5

type IRThinkingConfig struct {
	Type             string `json:"type,omitempty"`          // enabled, disabled, adaptive
	BudgetTokens     int    `json:"budget_tokens,omitempty"` // explicit token budget
	Effort           string `json:"effort,omitempty"`        // none, minimal, low, medium, high, xhigh, max
	Level            string `json:"level,omitempty"`         // Gemini thinkingLevel
	IncludeThoughts  *bool  `json:"include_thoughts,omitempty"`
	DynamicBudget    bool   `json:"dynamic_budget,omitempty"`
	Display          string `json:"display,omitempty"`
	EncryptedContent string `json:"encrypted_content,omitempty"`
	Subtype          string `json:"subtype,omitempty"`
}

IRThinkingConfig represents the thinking/extended-reasoning configuration for a request. It is the canonical IR form — protocol adapters map their native thinking controls to/from this struct.

type IRToolCallPart

type IRToolCallPart struct {
	ID            string                 `json:"id"`
	Name          string                 `json:"name"`
	ArgumentsRaw  string                 `json:"arguments_raw,omitempty"`
	ArgumentsJSON map[string]interface{} `json:"arguments_json,omitempty"`
	Status        string                 `json:"status,omitempty"` // in_progress, completed, failed
}

IRToolCallPart holds tool invocation data.

type IRToolChoice

type IRToolChoice struct {
	Type string `json:"type"`
	Name string `json:"name,omitempty"`
}

IRToolChoice represents how tool selection is handled.

type IRToolDecl

type IRToolDecl struct {
	Type         string                 `json:"type"`
	Name         string                 `json:"name"`
	Description  string                 `json:"description,omitempty"`
	Parameters   interface{}            `json:"parameters,omitempty"` // JSON Schema — acceptable here
	CacheControl map[string]interface{} `json:"cache_control,omitempty"`
}

IRToolDecl is a tool definition in the IR.

type IRToolResultPart

type IRToolResultPart struct {
	ToolCallID string          `json:"tool_call_id"`
	Content    []IRContentPart `json:"content,omitempty"`
	Status     string          `json:"status,omitempty"`
	Error      string          `json:"error,omitempty"`
}

IRToolResultPart holds the result of a tool execution.

type IRUsage

type IRUsage struct {
	InputTokens  int `json:"input_tokens"`
	OutputTokens int `json:"output_tokens"`
	TotalTokens  int `json:"total_tokens"`

	CacheReadInputTokens     int `json:"cache_read_input_tokens,omitempty"`
	CacheCreationInputTokens int `json:"cache_creation_input_tokens,omitempty"`

	ReasoningTokens int `json:"reasoning_tokens,omitempty"`

	ProviderRaw map[string]interface{} `json:"-"`
}

IRUsage represents token usage.

type ItemType

type ItemType int

ItemType identifies the kind of output item being streamed.

const (
	ItemUnknown ItemType = iota
	ItemText
	ItemToolCall
	ItemReasoning
)

type Protocol

type Protocol string

Protocol represents an LLM API protocol type.

const (
	// ProtocolOpenAIChat is the OpenAI Chat Completions protocol.
	ProtocolOpenAIChat Protocol = "openai_chat"
	// ProtocolOpenAIResponses is the OpenAI Responses API protocol.
	ProtocolOpenAIResponses Protocol = "openai_responses"
	// ProtocolAnthropicMessages is the Anthropic Messages API protocol.
	ProtocolAnthropicMessages Protocol = "anthropic_messages"
	// ProtocolGemini is the Google Gemini API protocol.
	ProtocolGemini Protocol = "gemini"
)

func ProtocolFromPath

func ProtocolFromPath(path string) (Protocol, error)

ProtocolFromPath determines the entry protocol from the request path.

type ProtocolAdapter

type ProtocolAdapter interface {
	// Protocol returns which protocol this adapter handles.
	Protocol() Protocol

	// DecodeRequest converts a raw protocol-specific request to canonical IR.
	DecodeRequest(raw map[string]interface{}) (*IRRequest, error)

	// EncodeRequest converts canonical IR to protocol-specific request format.
	EncodeRequest(ir *IRRequest) (map[string]interface{}, error)

	// DecodeResponse converts a raw protocol-specific response to canonical IR.
	DecodeResponse(raw map[string]interface{}) (*IRResponse, error)

	// EncodeResponse converts canonical IR to protocol-specific response format.
	EncodeResponse(ir *IRResponse) (map[string]interface{}, error)

	// NewStreamState creates a new opaque stream state for this adapter.
	// The returned value is owned entirely by this adapter.
	NewStreamState() interface{}

	// DecodeStreamEvent converts a raw SSE data payload to IR stream events.
	// May return multiple events from a single raw event (e.g., Gemini multi-part).
	DecodeStreamEvent(raw map[string]interface{}, state interface{}) ([]*IRStreamEvent, error)

	// EncodeStreamEvent converts an IR stream event to protocol-specific SSE payloads.
	// May return multiple payloads from a single IR event (e.g., Responses text.done+part.done+item.done).
	EncodeStreamEvent(ir *IRStreamEvent, state interface{}) ([]map[string]interface{}, error)
}

ProtocolAdapter defines the contract for protocol-specific encoding/decoding. Each adapter fully owns its stream state — the handler/converter never touches it.

func GetAdapter

func GetAdapter(proto Protocol) (ProtocolAdapter, error)

GetAdapter returns the registered ProtocolAdapter for the given protocol.

type StreamAggregator

type StreamAggregator struct {
	// contains filtered or unexported fields
}

StreamAggregator guarantees protocol lifecycle completeness.

Upstream adapters may emit IR events with missing lifecycle preamble (e.g. IRStreamContentDelta without preceding IRStreamContentStart). The aggregator intercepts IR events before they reach the entry adapter and ensures:

  1. response.created precedes all output events.
  2. output_item.added + content_part.added precede every text/reasoning delta.
  3. Every open item is closed (output_item.done) before IRStreamDone.
  4. IRStreamDone is emitted exactly once.

This eliminates "OutputTextDelta without active item" errors in Codex CLI.

func NewStreamAggregator

func NewStreamAggregator() *StreamAggregator

NewStreamAggregator creates a ready-to-use aggregator.

func (*StreamAggregator) Finalize

func (sa *StreamAggregator) Finalize() []*IRStreamEvent

Finalize triggers the target protocol's completion lifecycle.

Call once when the upstream stream ends — either on [DONE] or on reader EOF. Finalize closes every open output item, emits the aggregated stop_reason and usage, then emits IRStreamDone.

Idempotent. Repeated calls after the first are no-ops.

func (*StreamAggregator) GetUsage added in v0.3.0

func (sa *StreamAggregator) GetUsage() *IRUsage

GetUsage returns the accumulated token usage, or nil if no usage was reported.

func (*StreamAggregator) IsDone

func (sa *StreamAggregator) IsDone() bool

IsDone reports whether Finalize (or an IRStreamDone event) has already been processed.

func (*StreamAggregator) ProcessIREvent

func (sa *StreamAggregator) ProcessIREvent(evt *IRStreamEvent) []*IRStreamEvent

ProcessIREvent processes a single IR stream event and returns zero or more corrected IR events. Callers must feed the returned events to the entry adapter in order.

type StreamContext

type StreamContext struct {
	ResponseID string
	Model      string
	Usage      *IRUsage
	State      interface{} // adapter-owned opaque state
}

StreamContext holds shared context during stream conversion. State is opaque and owned by the adapter.

type Warning

type Warning struct {
	Field   string `json:"field"`
	Message string `json:"message"`
}

Warning represents a non-fatal degradation notice.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL