llmprotocol

package
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 7, 2026 License: Apache-2.0 Imports: 5 Imported by: 2

README

Scitrera LLM Protocol for Go

Provider-neutral LLM request, response, streaming, and protocol translation contracts for Go applications.

The package deliberately has no gateway, server, provider SDK, telemetry, or agent-runtime dependency. It is suitable for gateways, agents, evaluation tools, and direct provider clients.

Generation codecs cover OpenAI Chat Completions, OpenAI Responses, Anthropic Messages, Gemini GenerateContent, and Amazon Bedrock Converse. A separate embedding-codec registry covers OpenAI Embeddings, Gemini EmbedContent, and the Amazon Titan and Cohere Bedrock InvokeModel families. Gemini SSE and decoded Bedrock event-stream events use the same typed streaming contract as the other formats. Codecs reject known lossy conversions by default and return bounded, content-free diagnostics when permissive conversion is explicitly requested. Normalized input-token usage always includes cached-input and cache-creation components even when a provider reports those separately.

Embedding results use an explicit dense tensor: row-major Values plus a Shape whose rank is exactly one or two. Batch cardinality lives separately in EmbeddingResponse.Data, so a rank-2 embedding cannot be mistaken for two rank-1 results. Codecs validate shape products and fail closed when a target wire format can represent only rank 1. OpenAI float and base64 embeddings, Gemini shaped embeddings, and float Titan/Cohere responses are typed.

Structured outputs and strict tools are typed across OpenAI, Anthropic, and Bedrock. Gemini request controls include penalties, seed, candidate count, modalities, logprobs, and thinking configuration. OpenAI Chat and Gemini token log probabilities share buffered and streaming types.

The Bedrock codec does not own AWS credentials, SigV4, regional endpoint selection, or binary framing. Its streaming API consumes and produces decoded event type plus JSON payload pairs. The client package supplies a bounded, CRC-validating binary event-stream reader and standard-library SigV4 signer over a caller-owned credential provider; gateways may retain their own transport for policy and raw-byte capture.

Licensed under the Apache License, Version 2.0.

Run correctness tests and the allocation/throughput microbenchmarks with:

go test ./...
go test -run '^$' -bench . -benchmem ./codec

Documentation

Overview

Package llmprotocol defines provider-neutral LLM request, response, and streaming contracts. Wire-protocol codecs live in subpackages and depend only on these types.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Assembler 

type Assembler struct {
	// contains filtered or unexported fields
}

Assembler folds normalized stream events into a buffered Response. It keeps output and content ordering stable and is safe to use for Chat, Responses, or Anthropic streams. One assembler is used by one goroutine for one response.

func (*Assembler) Apply 

func (a *Assembler) Apply(event StreamEvent)

func (*Assembler) Done 

func (a *Assembler) Done() bool

func (*Assembler) Response 

func (a *Assembler) Response() Response

type ContentBlock 

type ContentBlock struct {
	Type       ContentType     `json:"type"`
	Text       string          `json:"text,omitempty"`
	Signature  string          `json:"signature,omitempty"`
	Source     *Source         `json:"source,omitempty"`
	ToolCall   *ToolCall       `json:"tool_call,omitempty"`
	Result     *ToolResult     `json:"tool_result,omitempty"`
	Provider   Format          `json:"provider,omitempty"`
	Raw        json.RawMessage `json:"raw,omitempty"`
	Extensions Extensions      `json:"extensions,omitempty"`
}

ContentBlock is an ordered tagged union. Exactly the fields appropriate for Type should be populated. Raw is retained only for ContentUnknown.

func Refusal 

func Refusal(text string) ContentBlock

func Text 

func Text(text string) ContentBlock

type ContentType 

type ContentType string
const (
	ContentText       ContentType = "text"
	ContentReasoning  ContentType = "reasoning"
	ContentImage      ContentType = "image"
	ContentAudio      ContentType = "audio"
	ContentVideo      ContentType = "video"
	ContentFile       ContentType = "file"
	ContentToolCall   ContentType = "tool_call"
	ContentToolResult ContentType = "tool_result"
	ContentRefusal    ContentType = "refusal"
	ContentUnknown    ContentType = "unknown"
)

type Diagnostic 

type Diagnostic struct {
	Kind   DiagnosticKind `json:"kind"`
	Path   string         `json:"path,omitempty"`
	Source Format         `json:"source,omitempty"`
	Target Format         `json:"target,omitempty"`
	Code   string         `json:"code"`
}

Diagnostic is deliberately content-free and safe for logs or metrics. Path identifies a bounded schema location, not a caller-provided value.

type DiagnosticKind 

type DiagnosticKind string
const (
	DiagnosticUnknownField DiagnosticKind = "unknown_field"
	DiagnosticLossy        DiagnosticKind = "lossy_conversion"
	DiagnosticGeneratedID  DiagnosticKind = "generated_id"
)

type EmbeddingInput 

type EmbeddingInput struct {
	Type   EmbeddingInputType `json:"type"`
	Text   string             `json:"text,omitempty"`
	Tokens []int64            `json:"tokens,omitempty"`
}

type EmbeddingInputType 

type EmbeddingInputType string

EmbeddingInputType identifies the portable form of one embedding input. Providers that accept only text must fail closed when asked to encode tokens. 

const (
	EmbeddingInputText   EmbeddingInputType = "text"
	EmbeddingInputTokens EmbeddingInputType = "tokens"
)

type EmbeddingOutput 

type EmbeddingOutput struct {
	Index      int             `json:"index"`
	Embedding  EmbeddingTensor `json:"embedding"`
	Extensions Extensions      `json:"extensions,omitempty"`
}

type EmbeddingRequest 

type EmbeddingRequest struct {
	Model          string           `json:"model,omitempty"`
	Inputs         []EmbeddingInput `json:"inputs"`
	EncodingFormat string           `json:"encoding_format,omitempty"`
	Dimensions     *int64           `json:"dimensions,omitempty"`
	User           string           `json:"user,omitempty"`
	TaskType       string           `json:"task_type,omitempty"`
	Title          string           `json:"title,omitempty"`
	InputType      string           `json:"input_type,omitempty"`
	Truncate       string           `json:"truncate,omitempty"`
	Normalize      *bool            `json:"normalize,omitempty"`
	AutoTruncate   *bool            `json:"auto_truncate,omitempty"`
	Extensions     Extensions       `json:"extensions,omitempty"`
	Preservation   Preservation     `json:"preservation,omitzero"`
}

func (*EmbeddingRequest) ClearPreservation 

func (r *EmbeddingRequest) ClearPreservation()

func (EmbeddingRequest) Validate 

func (r EmbeddingRequest) Validate() error

type EmbeddingResponse 

type EmbeddingResponse struct {
	Model          string            `json:"model,omitempty"`
	Data           []EmbeddingOutput `json:"data"`
	EncodingFormat string            `json:"encoding_format,omitempty"`
	Usage          Usage             `json:"usage,omitzero"`
	Extensions     Extensions        `json:"extensions,omitempty"`
	Preservation   Preservation      `json:"preservation,omitzero"`
}

func (*EmbeddingResponse) ClearPreservation 

func (r *EmbeddingResponse) ClearPreservation()

func (EmbeddingResponse) Validate 

func (r EmbeddingResponse) Validate() error

type EmbeddingTensor 

type EmbeddingTensor struct {
	Shape  []int64   `json:"shape"`
	Values []float64 `json:"values"`
}

EmbeddingTensor is an explicitly shaped, row-major dense tensor. Shape must have rank one or two. Keeping batch cardinality in EmbeddingResponse.Data means [d] is unambiguously a vector and [rows, columns] is unambiguously a matrix rather than a batch of vectors.

func NewEmbeddingMatrix 

func NewEmbeddingMatrix(rows [][]float64) (EmbeddingTensor, error)

func NewEmbeddingVector 

func NewEmbeddingVector(values []float64) EmbeddingTensor

func (EmbeddingTensor) Rank 

func (t EmbeddingTensor) Rank() int

func (EmbeddingTensor) Rows 

func (t EmbeddingTensor) Rows() ([][]float64, error)

Rows returns a defensive 2-D view. A rank-one tensor is returned as one row.

func (EmbeddingTensor) Validate 

func (t EmbeddingTensor) Validate() error

type Extensions 

type Extensions map[string]json.RawMessage

Extensions retains top-level provider fields that have no neutral meaning. Values are exact JSON fragments. Hosts should use an allowlist before sending extensions across trust boundaries.

type Format 

type Format string

Format identifies one provider wire contract rather than a vendor account. 

const (
	FormatOpenAIChat              Format = "openai_chat_completions"
	FormatOpenAIResponses         Format = "openai_responses"
	FormatAnthropic               Format = "anthropic_messages"
	FormatGemini                  Format = "gemini_generate_content"
	FormatBedrock                 Format = "bedrock_converse"
	FormatOpenAIEmbeddings        Format = "openai_embeddings"
	FormatGeminiEmbeddings        Format = "gemini_embed_content"
	FormatBedrockTitanEmbeddings  Format = "bedrock_titan_embeddings"
	FormatBedrockCohereEmbeddings Format = "bedrock_cohere_embeddings"
)

type Instruction 

type Instruction struct {
	Role       Role           `json:"role"`
	Content    []ContentBlock `json:"content"`
	Extensions Extensions     `json:"extensions,omitempty"`
}

type LossyPolicy 

type LossyPolicy string
const (
	LossyReject LossyPolicy = "reject"
	LossyAllow  LossyPolicy = "allow_with_diagnostics"
)

type Message 

type Message struct {
	ID         string         `json:"id,omitempty"`
	Role       Role           `json:"role"`
	Content    []ContentBlock `json:"content"`
	Extensions Extensions     `json:"extensions,omitempty"`
}

type Output 

type Output struct {
	MaxTokens      *int64          `json:"max_tokens,omitempty"`
	StopSequences  []string        `json:"stop_sequences,omitempty"`
	ResponseFormat json.RawMessage `json:"response_format,omitempty"`
	Choices        *int64          `json:"choices,omitempty"`
	Logprobs       *bool           `json:"logprobs,omitempty"`
	TopLogprobs    *int64          `json:"top_logprobs,omitempty"`
	Modalities     []string        `json:"modalities,omitempty"`
}

type Policy 

type Policy struct {
	UnknownFields UnknownFieldPolicy
	Lossy         LossyPolicy
	Preservation  PreservationPolicy
	Target        TargetCapabilities
}

func PermissivePolicy 

func PermissivePolicy() Policy

func StrictPolicy 

func StrictPolicy() Policy

func (Policy) Effective 

func (p Policy) Effective() Policy

type Preservation 

type Preservation struct {
	Requests  map[Format]json.RawMessage `json:"requests,omitempty"`
	Responses map[Format]json.RawMessage `json:"responses,omitempty"`
}

Preservation stores exact same-format bodies in memory. Code that mutates a decoded value must call ClearPreservation before encoding it.

type PreservationPolicy 

type PreservationPolicy string
const (
	PreserveInMemory PreservationPolicy = "in_memory"
	PreserveDisabled PreservationPolicy = "disabled"
)

type ProtocolError 

type ProtocolError struct {
	Type      string `json:"type,omitempty"`
	Code      string `json:"code,omitempty"`
	Message   string `json:"message"`
	Retryable bool   `json:"retryable,omitempty"`
}

type ProviderState 

type ProviderState struct {
	Store              *bool           `json:"store,omitempty"`
	Background         *bool           `json:"background,omitempty"`
	PreviousResponseID string          `json:"previous_response_id,omitempty"`
	Conversation       json.RawMessage `json:"conversation,omitempty"`
}

ProviderState represents request state controls shared by some, but not all, wire formats. A translator must never silently discard active state. False boolean values are intentionally retained so same-format decode/encode can distinguish an explicit opt-out from an omitted field.

type Reasoning 

type Reasoning struct {
	Effort       string          `json:"effort,omitempty"`
	BudgetTokens *int64          `json:"budget_tokens,omitempty"`
	Include      *bool           `json:"include,omitempty"`
	Level        string          `json:"level,omitempty"`
	Provider     Format          `json:"provider,omitempty"`
	Raw          json.RawMessage `json:"raw,omitempty"`
}

type Request 

type Request struct {
	Model             string           `json:"model,omitempty"`
	Instructions      []Instruction    `json:"instructions,omitempty"`
	Messages          []Message        `json:"messages,omitempty"`
	Tools             []ToolDefinition `json:"tools,omitempty"`
	ToolChoice        *ToolChoice      `json:"tool_choice,omitempty"`
	ParallelToolCalls *bool            `json:"parallel_tool_calls,omitempty"`
	Sampling          Sampling         `json:"sampling,omitzero"`
	Output            Output           `json:"output,omitzero"`
	Reasoning         Reasoning        `json:"reasoning,omitzero"`
	State             ProviderState    `json:"state,omitzero"`
	Stream            bool             `json:"stream,omitempty"`
	Extensions        Extensions       `json:"extensions,omitempty"`
	Preservation      Preservation     `json:"preservation,omitzero"`
}

func (*Request) ClearPreservation 

func (r *Request) ClearPreservation()

type Response 

type Response struct {
	ID           string           `json:"id,omitempty"`
	Model        string           `json:"model,omitempty"`
	Outputs      []ResponseOutput `json:"outputs,omitempty"`
	Usage        Usage            `json:"usage,omitzero"`
	Extensions   Extensions       `json:"extensions,omitempty"`
	Preservation Preservation     `json:"preservation,omitzero"`
}

func (*Response) ClearPreservation 

func (r *Response) ClearPreservation()

type ResponseOutput 

type ResponseOutput struct {
	ID         string         `json:"id,omitempty"`
	Index      *int           `json:"index,omitempty"`
	Role       Role           `json:"role"`
	Content    []ContentBlock `json:"content"`
	StopReason StopReason     `json:"stop_reason,omitempty"`
	Logprobs   []TokenLogprob `json:"logprobs,omitempty"`
	Extensions Extensions     `json:"extensions,omitempty"`
}

type Role 

type Role string
const (
	RoleSystem    Role = "system"
	RoleDeveloper Role = "developer"
	RoleUser      Role = "user"
	RoleAssistant Role = "assistant"
	RoleTool      Role = "tool"
)

type Sampling 

type Sampling struct {
	Temperature      *float64 `json:"temperature,omitempty"`
	TopP             *float64 `json:"top_p,omitempty"`
	TopK             *int64   `json:"top_k,omitempty"`
	FrequencyPenalty *float64 `json:"frequency_penalty,omitempty"`
	PresencePenalty  *float64 `json:"presence_penalty,omitempty"`
	Seed             *int64   `json:"seed,omitempty"`
}

type Source 

type Source struct {
	Kind      string          `json:"kind"`
	URL       string          `json:"url,omitempty"`
	Data      string          `json:"data,omitempty"`
	MediaType string          `json:"media_type,omitempty"`
	FileID    string          `json:"file_id,omitempty"`
	Filename  string          `json:"filename,omitempty"`
	Detail    string          `json:"detail,omitempty"`
	Raw       json.RawMessage `json:"raw,omitempty"`
}

Source carries one external, inline, or provider-owned media value. Data is base64 text when Kind is "base64"; provider_file identifies an opaque provider resource and must retain target affinity at the host layer.

type StopReason 

type StopReason string
const (
	StopEndTurn       StopReason = "end_turn"
	StopMaxTokens     StopReason = "max_tokens"
	StopToolUse       StopReason = "tool_use"
	StopContentFilter StopReason = "content_filter"
	StopError         StopReason = "error"
	StopUnknown       StopReason = "unknown"
)

type StreamEvent 

type StreamEvent struct {
	Type         StreamEventType `json:"type"`
	ResponseID   string          `json:"response_id,omitempty"`
	Model        string          `json:"model,omitempty"`
	OutputIndex  int             `json:"output_index,omitempty"`
	ContentIndex int             `json:"content_index,omitempty"`
	ItemID       string          `json:"item_id,omitempty"`
	Role         Role            `json:"role,omitempty"`
	Delta        string          `json:"delta,omitempty"`
	Signature    string          `json:"signature,omitempty"`
	ToolCallID   string          `json:"tool_call_id,omitempty"`
	ToolName     string          `json:"tool_name,omitempty"`
	ToolType     string          `json:"tool_type,omitempty"`
	Usage        *Usage          `json:"usage,omitempty"`
	StopReason   StopReason      `json:"stop_reason,omitempty"`
	Logprobs     []TokenLogprob  `json:"logprobs,omitempty"`
	Error        *ProtocolError  `json:"error,omitempty"`
	Provider     Format          `json:"provider,omitempty"`
	Raw          json.RawMessage `json:"raw,omitempty"`
}

StreamEvent is sufficiently granular to retain OpenAI Responses item/event ordering while still representing Chat and Anthropic deltas.

type StreamEventType 

type StreamEventType string
const (
	StreamResponseStart           StreamEventType = "response_start"
	StreamOutputStart             StreamEventType = "output_start"
	StreamTextDelta               StreamEventType = "text_delta"
	StreamReasoningDelta          StreamEventType = "reasoning_delta"
	StreamReasoningSignatureDelta StreamEventType = "reasoning_signature_delta"
	StreamRefusalDelta            StreamEventType = "refusal_delta"
	StreamToolCallStart           StreamEventType = "tool_call_start"
	StreamToolArgsDelta           StreamEventType = "tool_arguments_delta"
	StreamOutputDone              StreamEventType = "output_done"
	StreamUsage                   StreamEventType = "usage"
	StreamLogprobs                StreamEventType = "logprobs"
	StreamResponseDone            StreamEventType = "response_done"
	StreamError                   StreamEventType = "error"
	StreamUnknown                 StreamEventType = "unknown"
)

type TargetCapabilities 

type TargetCapabilities struct {
	Tools             *bool
	Images            *bool
	Audio             *bool
	Video             *bool
	Files             *bool
	Reasoning         *bool
	JSONSchema        *bool
	ParallelToolCalls *bool
}

type TokenLogprob 

type TokenLogprob struct {
	Chosen TokenProbability   `json:"chosen"`
	Top    []TokenProbability `json:"top,omitempty"`
}

type TokenProbability 

type TokenProbability struct {
	Token   string  `json:"token"`
	TokenID *int64  `json:"token_id,omitempty"`
	Logprob float64 `json:"logprob"`
	Bytes   []int64 `json:"bytes,omitempty"`
}

type ToolCall 

type ToolCall struct {
	ID   string `json:"id"`
	Name string `json:"name"`
	// Type retains provider tool-call families such as "custom" and
	// "apply_patch". Empty means an ordinary function call.
	Type string `json:"type,omitempty"`
	// Arguments is JSON. Custom tool input is wrapped as {"patch":"..."};
	// native apply_patch arguments are wrapped as {"operation":{...}}.
	Arguments json.RawMessage `json:"arguments"`
}

type ToolChoice 

type ToolChoice struct {
	Type ToolChoiceType  `json:"type"`
	Name string          `json:"name,omitempty"`
	Raw  json.RawMessage `json:"raw,omitempty"`
}

type ToolChoiceType 

type ToolChoiceType string
const (
	ToolChoiceAuto     ToolChoiceType = "auto"
	ToolChoiceRequired ToolChoiceType = "required"
	ToolChoiceNone     ToolChoiceType = "none"
	ToolChoiceTool     ToolChoiceType = "tool"
	ToolChoiceRaw      ToolChoiceType = "raw"
)

type ToolDefinition 

type ToolDefinition struct {
	Type        string          `json:"type,omitempty"`
	Name        string          `json:"name"`
	Description string          `json:"description,omitempty"`
	Parameters  json.RawMessage `json:"parameters"`
	Strict      *bool           `json:"strict,omitempty"`
	Extensions  Extensions      `json:"extensions,omitempty"`
	Raw         json.RawMessage `json:"raw,omitempty"`
}

type ToolResult 

type ToolResult struct {
	ToolCallID string `json:"tool_call_id"`
	// Type matches the originating ToolCall so a Responses codec can emit the
	// corresponding *_call_output item during stateless transcript replay.
	Type    string         `json:"type,omitempty"`
	Content []ContentBlock `json:"content"`
	IsError *bool          `json:"is_error,omitempty"`
}

type TranslationError 

type TranslationError struct {
	Format Format
	Path   string
	Code   string
	Detail string
}

func (*TranslationError) Error 

func (e *TranslationError) Error() string

type UnknownFieldPolicy 

type UnknownFieldPolicy string
const (
	UnknownPreserve UnknownFieldPolicy = "preserve"
	UnknownDrop     UnknownFieldPolicy = "drop"
	UnknownReject   UnknownFieldPolicy = "reject"
)

type Usage 

type Usage struct {
	// InputTokens is total input, including cached-input and cache-creation
	// components. Provider codecs normalize wire formats which report those
	// components separately.
	InputTokens              *int64           `json:"input_tokens,omitempty"`
	OutputTokens             *int64           `json:"output_tokens,omitempty"`
	TotalTokens              *int64           `json:"total_tokens,omitempty"`
	CachedInputTokens        *int64           `json:"cached_input_tokens,omitempty"`
	CacheCreationTokens      *int64           `json:"cache_creation_tokens,omitempty"`
	ReasoningTokens          *int64           `json:"reasoning_tokens,omitempty"`
	ToolUsePromptTokens      *int64           `json:"tool_use_prompt_tokens,omitempty"`
	AcceptedPredictionTokens *int64           `json:"accepted_prediction_tokens,omitempty"`
	RejectedPredictionTokens *int64           `json:"rejected_prediction_tokens,omitempty"`
	ProviderComponents       map[string]int64 `json:"provider_components,omitempty"`
}

type WireEvent 

type WireEvent struct {
	Event string
	Data  json.RawMessage
}

WireEvent is one decoded SSE event. Event is empty for protocols that use only data frames. Data excludes the SSE "data:" prefix.

Source Files

View all Source files

Directories

Path Synopsis
Package codec translates provider wire formats through llmprotocol's neutral representation.
 Package codec translates provider wire formats through llmprotocol's neutral representation.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.