llmcall

package
v0.0.0-...-a0319ed Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 2, 2026 License: AGPL-3.0 Imports: 15 Imported by: 0

Documentation

Overview

Package llmcall defines clrk's canonical intermediate representation (IR) for LLM API traffic and the provider registry that converts wire schemas (Anthropic, OpenAI, Google, ...) to and from it.

Hub and spoke

Translation between provider wire schemas is hub-and-spoke, never pairwise: each provider implements a Codec that decodes its wire format into the IR and encodes the IR back into its wire format. Translating a request captured in schema A to a backend speaking schema B is EncodeB(DecodeA(body)). Adding a provider therefore means adding one self-contained package under providers/ — no edits to other providers or to extproc core.

The IR's structure is based on the LLM-Rosetta paper (arXiv:2604.09360): a small set of typed content parts, a typed stream-event taxonomy, and explicit preserve/strip handling of provider fields the IR doesn't model. Its vocabulary — role names, part types, finish reasons, provider names — follows the OTel GenAI semantic conventions so the telemetry projection is a copy, not a mapping table.

Preserve vs strip

Every IR node carries a Wire with the bookkeeping needed for two encode modes:

  • ModePreserve re-emits unmodeled provider fields and original formatting; a decode/encode round trip of an unmutated body through the same provider's codec is byte-identical. The golden corpus in apoxy-cloud gates this property.
  • ModeStrip drops unmodeled fields (counting them) and emits the target provider's canonical form. Cross-schema translation always strips — extras are provider-specific by definition.

Registration

Providers self-register from package init via Register. The blank import list at providers/all is the only place that names them all; the parsers shim imports it, so any binary linking internal/extproc/parsers gets every provider. A program that imports llmcall directly without importing providers/all (or a specific provider) sees an empty registry — tests included.

Boundaries

The IR is internal-only. It is never serialized to the wire, never persisted, and never surfaces in the versioned clrk API; it may change shape freely between releases. Consequently there are no kubebuilder markers and no generated deepcopy here.

Index

Constants

View Source
const (
	BlockerNoCodec        = "no_codec"
	BlockerStreaming      = "streaming"
	BlockerOperation      = "operation"
	BlockerTools          = "tools"
	BlockerToolResults    = "tool_results"
	BlockerImageInput     = "image_input"
	BlockerAudioInput     = "audio_input"
	BlockerFileInput      = "file_input"
	BlockerReasoning      = "reasoning"
	BlockerSystem         = "system"
	BlockerUnknownContent = "unknown_content"
)

Translation blocker reasons returned by TranslationBlockers. Values are stable identifiers surfaced on telemetry (skipped-candidate accounting), not user-facing prose.

Variables

View Source
var ErrUnsupported = errors.New("llmcall: not supported by codec")

ErrUnsupported reports valid wire bytes outside the codec's modeled scope — an endpoint, operation, or response framing (event-stream, NDJSON) the codec doesn't translate yet. Callers react by skipping translation, not by failing the request.

Functions

func AppendSSE

func AppendSSE(dst []byte, name string, data []byte) []byte

AppendSSE appends one SSE frame to dst: an optional "event: name" line, then "data:" lines (data newlines become continuations), and the terminating blank line.

func AuthHeaderUnion

func AuthHeaderUnion() []string

AuthHeaderUnion returns the sorted union of every registered provider's AuthHeaders. Translated requests shed the whole union rather than just the source schema's headers: SDK defaults mean an agent may attach another provider's credential, and a stale credential reaching a foreign upstream is worse than over-removal.

func Canonical

func Canonical(name string) string

Canonical normalizes a CRD-supplied provider name to the canonical gen_ai.system value used everywhere downstream (route matching, span attributes). Unknown names pass through verbatim — `custom` has special semantics in the route matcher and a typo is better surfaced as a non-matching rule than silently rewritten. No case folding: callers lowercase CRD input before calling.

func CollectText

func CollectText(parts []Part) string

CollectText concatenates the text of all text parts. Codecs use it to re-flatten parts into the string-form content some wire shapes carry (and to verify string shadows against the current IR value).

func CompactJSON

func CompactJSON(raw []byte) ([]byte, error)

CompactJSON returns raw with insignificant whitespace removed. Key order, string escapes, and number literals are untouched — exactly the normalization the preserve-mode passthrough gate needs. Stays on encoding/json: Compact is a byte-level transform with no value parse, outside jsonx's scope.

func CompactRaw

func CompactRaw(value json.RawMessage) (json.RawMessage, error)

CompactRaw compacts a captured subtree for Wire storage, per the file invariant.

func DataURL

func DataURL(mime, dataB64 string) string

DataURL composes an RFC 2397 data URL from a MIME type and base64 payload — the inline-image form OpenAI's image_url accepts.

func DecodeBody

func DecodeBody(body []byte, contentEncoding string) ([]byte, bool)

DecodeBody returns body with its HTTP content-encoding undone, so a telemetry parser sees plaintext. It is the seam that lets usage parsing work against real provider traffic: agents (the Anthropic SDK / Claude Code, OpenAI SDK, etc.) send `accept-encoding: gzip, deflate, br, zstd` and clrk's MITM forwards it untouched, so providers gzip even SSE bodies. A compressed body has no `data:` lines for ScanSSEData to find, which silently zeroes every token count.

For an identity / empty encoding or an empty body it returns body unchanged with ok=true, so callers can invoke it unconditionally. ok is false only when a recognized encoding failed to inflate -- a corrupt or, far more often, a partially captured stream. The caller then keeps treating usage as absent, exactly as for any unparseable body, and must NOT use the returned slice.

Pass only a COMPLETE captured body. Streamed-response capture is keep-last-N, so a truncated body is a header-less tail that no codec can inflate; gate the call on !truncated.

func DecodeKnown

func DecodeKnown(raw json.RawMessage, w *Wire, known map[string]func(value json.RawMessage) error) error

DecodeKnown walks raw's object members in source order, recording KeyOrder on w, dispatching modeled keys to their handler, and capturing everything else as a compacted extra.

func DecodeNumber

func DecodeNumber(value json.RawMessage) (json.Number, error)

DecodeNumber unmarshals any JSON number as its literal. json.Number re-marshals byte-identically, so numbers stored this way never need shadows.

func DecodeObject

func DecodeObject(raw []byte, fn func(key string, value json.RawMessage) error) error

DecodeObject iterates the members of a JSON object in source order, handing each key and its verbatim value subtree to fn. Returns an error when raw is not a JSON object or fn fails. Duplicate keys are passed through as encountered; fidelity guarantees assume providers don't emit them. Stays on encoding/json: the order-preserving token walk has no std-compatible equivalent in jsonx's engine.

func DecodeShadowInt64

func DecodeShadowInt64(value json.RawMessage, w *Wire, key string) (int64, error)

DecodeShadowInt64 unmarshals an integer value, shadowing exotic literals ("12.0", "1e2") that wouldn't re-marshal identically.

func DecodeShadowString

func DecodeShadowString(value json.RawMessage, w *Wire, key string) (string, error)

DecodeShadowString unmarshals a JSON string value and records a shadow on w when the canonical re-marshal differs from the source bytes (\uXXXX escapes and friends).

func DecodeShadowStringSlice

func DecodeShadowStringSlice(value json.RawMessage, w *Wire, key string) ([]string, error)

DecodeShadowStringSlice unmarshals a JSON string array, recording a whole-value shadow keyed on the joined contents.

func DecodeTypedBlock

func DecodeTypedBlock(raw json.RawMessage, field string) (string, error)

DecodeTypedBlock peeks the discriminator field of a content-block object (e.g. "type" for Anthropic/OpenAI part shapes) without consuming the rest.

func EmitShadowInt64

func EmitShadowInt64(e *ObjectEncoder, w *Wire, key string, val int64)

EmitShadowInt64 is EmitShadowString for shadowed integers.

func EmitShadowString

func EmitShadowString(e *ObjectEncoder, w *Wire, key, val string)

EmitShadowString writes key with its shadowed source bytes when val is unmutated, canonical marshal otherwise.

func EncodeArray

func EncodeArray(elems []json.RawMessage) []byte

EncodeArray joins pre-encoded elements into a JSON array.

func EncodeOrderedObject

func EncodeOrderedObject(w *Wire, mode Mode, emit map[string]FieldEmitter, canonical []string, dropped *int) ([]byte, error)

EncodeOrderedObject assembles a node: in preserve mode it walks Wire.KeyOrder, emitting modeled keys via their emitter and extras verbatim, then appends modeled keys missing from KeyOrder (added after decode) in canonical order; in strip mode it walks canonical order only and counts the dropped extras into *dropped.

func EncodeRaw

func EncodeRaw(fresh []byte, raw json.RawMessage) (body []byte, exact bool)

EncodeRaw implements the preserve-mode passthrough gate shared by all codecs. fresh is the encoder's compact output built from the IR; raw is the original body captured at decode time (Wire.Raw). When fresh equals json.Compact(raw) — the encoder is byte-faithful modulo whitespace — the original raw bytes are returned, restoring formatting (providers pretty-print responses) with exact=true. Any difference, including post-decode mutation, falls back to fresh.

The equality gate is what keeps the golden corpus non-vacuous: a bug anywhere in the KeyOrder/Extras/Shadow machinery breaks the equality, the passthrough doesn't fire, and the corpus sees the byte diff.

func EnsureToolCallIDs

func EnsureToolCallIDs(req *Request)

EnsureToolCallIDs assigns deterministic IDs (call_1, call_2, ... in walk order) to tool calls decoded from schemas that don't carry one — Gemini's functionCall is name-keyed. anthropic/openai targets require an id on every assistant tool call. Tool results need no matching pass: name-keyed results only arise from Gemini decodes, and TranslationBlockers refuses tool results on any Gemini-side pair until the codec models them fully (Phase 3).

Mutates req in place — translation runs it on a DeepCopy.

func ErrorBodyFor

func ErrorBodyFor(system, msg string) []byte

ErrorBodyFor shapes msg in the error envelope of the schema the client speaks. Unknown and envelope-less schemas get the OpenAI envelope — the de-facto shape for compatible gateways.

func HasContentTypePrefix

func HasContentTypePrefix(headers map[string]string, prefix string) bool

HasContentTypePrefix reports whether the headers' content-type starts with prefix (case-insensitive). Header keys in Input are pre-lowered by ext_proc's headersToMap.

func JSONString

func JSONString(s string) string

JSONString quotes s as a JSON string literal — for hand-assembled wire fragments like provider error envelopes.

func KnownName

func KnownName(name string) bool

KnownName reports whether name (a lowercased CRD-supplied provider or schema spelling) is recognized: a registered canonical name, a registered alias, or a pending alias whose plugin hasn't landed yet. Pending aliases count so admission validation never tightens beyond the historical enum ("azure-openai"/"bedrock" were always legal). "custom" is not a registry concept — callers that accept it (route matching, admission validation) special-case it before asking.

func KnownNames

func KnownNames() []string

KnownNames returns every spelling KnownName accepts — registered canonical names plus (pending) aliases — sorted and deduplicated. Admission validation uses it for actionable NotSupported details.

func LastJSONLine

func LastJSONLine(b []byte) []byte

LastJSONLine returns the last newline-delimited byte slice in b that json.Unmarshal accepts into an empty interface. Returns nil if none decode. Used for application/x-ndjson streams where capture is keep-last-N: the final line is the one we want, but it may be missing or partial under tight caps.

func LiftSystemMessages

func LiftSystemMessages(req *Request)

LiftSystemMessages moves RoleSystem turns out of Messages into req.System, appending their parts in conversation order. OpenAI carries system content as positional messages while anthropic and Gemini carry a top-level field and reject a system role inside the message list, so translation lifts before encoding to either. OpenAI-decoded same-schema round trips never call this; the openai encoder re-emits a populated req.System as a leading system message.

Mutates req in place — translation runs it on a DeepCopy.

func MarshalCompact

func MarshalCompact(v any) ([]byte, error)

MarshalCompact is the single JSON marshal entrypoint for encoders. It differs from json.Marshal in not HTML-escaping <, >, & — Go's default escaping would silently break byte-identity against provider output, so codecs must not call bare json.Marshal.

func NormalizeContentEncoding

func NormalizeContentEncoding(contentEncoding string) string

NormalizeContentEncoding returns the non-identity content-encoding layers of a Content-Encoding header value, joined comma-separated in wire order (outermost last), or "" when the header is empty or names only identity. It is the label form of exactly the layers DecodeBody undoes, so a telemetry consumer can record which encoding a body arrived in without re-implementing the parse.

func Register

func Register(p Provider)

Register adds a provider to the registry. It must be called from package init only — lookups are unsynchronized reads after process start. It panics on identity conflicts (duplicate name, alias, or host) because a double registration is a link-time bug, not a runtime condition.

func ScanSSEData

func ScanSSEData(b []byte, fn func(payload []byte))

ScanSSEData iterates `data:` payloads in an SSE byte stream, oldest first, and hands each payload to fn. Lines that aren't `data:` (event type, comment, retry, id, etc.) are skipped. Multi-line `data:` continuations within one event are joined with '\n' per the SSE spec.

Capture is keep-last-N for streamed responses, so the first event the helper sees is often a partial fragment. fn is responsible for tolerating decode failures silently — the contract is "I'll show you every payload I find; you decide which ones decode."

func SplitDataURL

func SplitDataURL(url string) (mime, dataB64 string, ok bool)

SplitDataURL parses a base64 data URL back into its MIME type and payload. ok is false for any other URL shape (including non-base64 data URLs), in which case the caller should treat the value as a fetchable reference.

func StringSliceShadowVal

func StringSliceShadowVal(ss []string) string

StringSliceShadowVal is the comparable composite for string-slice shadows; encoders recompute it to detect mutation.

func StripRequestForTranslation

func StripRequestForTranslation(req *Request) int

StripRequestForTranslation removes the source schema's wire bookkeeping from a decoded request so a different provider's codec can encode it cleanly, and drops content the target cannot receive. Returns the number of source artifacts discarded (unmodeled extras across every node plus unmodeled provider blocks), which feeds the dropped-extras telemetry.

Stripping is what makes cross-codec encoding sound: Wire carries codec-PRIVATE conventions — hint keys collide across codecs ("content", "system"), KeyOrder/Extras hold source-schema members, and shadows hold source-schema bytes. A target encoder walking them would interleave another schema's fields into its output. Zeroed wires put every encoder on its canonical programmatic path.

Mutates req in place — translation runs it on a DeepCopy, after reading anything it needs from the source wire (header names, the original path).

func StripResponseForTranslation

func StripResponseForTranslation(resp *Response) int

StripResponseForTranslation is StripRequestForTranslation for the response direction (backend-schema decode re-encoded in the client's schema).

func TranslationBlockers

func TranslationBlockers(req *Request, src, tgt *Provider) []string

TranslationBlockers reports why a decoded request cannot be translated from src's wire schema to tgt's. An empty result means the pair is translatable for this request. Backend selection calls this per candidate to filter cross-schema backends; the blocker strings feed the skipped-candidate telemetry.

The checks compare the features the request actually uses against the target's Capabilities (which are honest to the codec, not the provider's API surface), plus three request-specific gates: streaming when either side cannot stream-translate (no stream codec, or a stream framing the codec doesn't cover); unmodeled provider blocks (PartTypeUnknown — their content exists only as source-schema wire bytes, so translation would silently delete it); and tool results through the Gemini schema, whose codec keys results by function name and stashes payloads in codec-private hints — lossy in either direction until they are modeled fully.

Types

type AssembledResponse

type AssembledResponse struct {
	Text      string
	ToolCalls []AssembledToolCall
}

AssembledResponse is the human-readable reconstruction of a streamed assistant message: the concatenated text and any tool calls, in the order their content blocks opened. Reasoning deltas are intentionally excluded — they are not the answer.

func AssembleStreamedResponse

func AssembleStreamedResponse(p *Provider, body []byte) (AssembledResponse, bool)

AssembleStreamedResponse reconstructs the assistant message from a captured streamed response body in provider p's wire framing. It drives p's StreamCodec, so every schema with a stream codec is covered by this one path rather than a per-provider reassembler.

Best-effort by construction: the captured body may be truncated (the keep-last-N ring drops the head of a long stream) or cut mid-frame, so decode errors are swallowed and whatever decoded so far is returned. ok is false when p has no stream codec, body is empty, or nothing decodable was produced. The decoder is read-only here — it is built with a nil request because only the encoder consults the request (for the model fallback), which assembly does not use.

func (AssembledResponse) Rendered

func (a AssembledResponse) Rendered() string

Rendered flattens the assembled response into one readable string for telemetry display: the text, then each tool call as `name(arguments)` on its own line. ASCII-only, no invented markup beyond the call form.

type AssembledToolCall

type AssembledToolCall struct {
	ID        string
	Name      string
	Arguments string
}

AssembledToolCall is one tool invocation reconstructed from a streamed response: the tool name and its fully-accumulated argument bytes (verbatim JSON, concatenated across the stream's argument deltas).

type AudioPart

type AudioPart struct {
	MIMEType string `json:"mimeType,omitempty" yaml:"mimeType,omitempty"`
	DataB64  string `json:"dataB64,omitempty" yaml:"dataB64,omitempty"`
	URL      string `json:"url,omitempty" yaml:"url,omitempty"`
	Wire     Wire   `json:"wire,omitempty" yaml:"wire,omitempty"`
}

AudioPart is audio content; field semantics mirror ImagePart.

+k8s:deepcopy-gen=true

func (*AudioPart) DeepCopy

func (in *AudioPart) DeepCopy() *AudioPart

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AudioPart.

func (*AudioPart) DeepCopyInto

func (in *AudioPart) DeepCopyInto(out *AudioPart)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type Capabilities

type Capabilities struct {
	// Operations are the gen_ai.operation.name values the schema
	// serves ("chat", "text_completion", "embeddings", ...).
	Operations []string `json:"operations,omitempty" yaml:"operations,omitempty"`

	Tools             bool `json:"tools,omitempty" yaml:"tools,omitempty"`
	ParallelToolCalls bool `json:"parallelToolCalls,omitempty" yaml:"parallelToolCalls,omitempty"`
	ImageInput        bool `json:"imageInput,omitempty" yaml:"imageInput,omitempty"`
	AudioInput        bool `json:"audioInput,omitempty" yaml:"audioInput,omitempty"`
	FileInput         bool `json:"fileInput,omitempty" yaml:"fileInput,omitempty"`
	Reasoning         bool `json:"reasoning,omitempty" yaml:"reasoning,omitempty"`
	StructuredOutput  bool `json:"structuredOutput,omitempty" yaml:"structuredOutput,omitempty"`
	Streaming         bool `json:"streaming,omitempty" yaml:"streaming,omitempty"`
	// SystemRole reports whether the schema carries system content
	// natively (top-level field or system role) rather than requiring
	// it to be folded into the first user message.
	SystemRole bool `json:"systemRole,omitempty" yaml:"systemRole,omitempty"`
}

Capabilities declares what a provider's wire schema can express. Backend selection (Phase 2, APO-742) filters cross-schema candidates by comparing a decoded request's feature use against the target provider's Capabilities, so values must be honest: claim only what the codec can actually encode.

+k8s:deepcopy-gen=true

func (*Capabilities) DeepCopy

func (in *Capabilities) DeepCopy() *Capabilities

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Capabilities.

func (*Capabilities) DeepCopyInto

func (in *Capabilities) DeepCopyInto(out *Capabilities)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type Choice

type Choice struct {
	Message      Message      `json:"message,omitempty" yaml:"message,omitempty"`
	FinishReason FinishReason `json:"finishReason,omitempty" yaml:"finishReason,omitempty"`
	// FinishReasonRaw is the provider's literal finish value when the
	// canonical mapping is lossy; empty when the mapping is bijective.
	FinishReasonRaw string `json:"finishReasonRaw,omitempty" yaml:"finishReasonRaw,omitempty"`
	Wire            Wire   `json:"wire,omitempty" yaml:"wire,omitempty"`
}

Choice is one generated completion.

+k8s:deepcopy-gen=true

func (*Choice) DeepCopy

func (in *Choice) DeepCopy() *Choice

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Choice.

func (*Choice) DeepCopyInto

func (in *Choice) DeepCopyInto(out *Choice)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type CitationPart

type CitationPart struct {
	Title   string `json:"title,omitempty" yaml:"title,omitempty"`
	URL     string `json:"url,omitempty" yaml:"url,omitempty"`
	Snippet string `json:"snippet,omitempty" yaml:"snippet,omitempty"`
	Wire    Wire   `json:"wire,omitempty" yaml:"wire,omitempty"`
}

CitationPart is a source citation attached to generated content.

+k8s:deepcopy-gen=true

func (*CitationPart) DeepCopy

func (in *CitationPart) DeepCopy() *CitationPart

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CitationPart.

func (*CitationPart) DeepCopyInto

func (in *CitationPart) DeepCopyInto(out *CitationPart)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type Codec

type Codec interface {
	DecodeRequest(in RequestInput) (*Request, error)
	EncodeRequest(req *Request, opts EncodeOptions) (*EncodedRequest, error)
	DecodeResponse(in ResponseInput, req *Request) (*Response, error)
	EncodeResponse(resp *Response, opts EncodeOptions) (*EncodedResponse, error)
}

Codec converts one provider's wire format to and from the IR. Implementations are stateless and safe for concurrent use.

DecodeRequest must record everything preserve-mode EncodeRequest needs to reproduce the original transaction: body bookkeeping in the IR's Wire fields, the original :path in Request.Wire.Path, and modeled headers in Request.Wire.Headers.

DecodeResponse takes the already-decoded request for context the response body alone doesn't carry (operation, model for schemas that don't echo it).

type EncodeOptions

type EncodeOptions struct {
	Mode Mode
}

EncodeOptions parameterizes Encode calls.

type EncodedRequest

type EncodedRequest struct {
	Body          []byte
	Path          string
	SetHeaders    map[string]string
	RemoveHeaders []string

	// DroppedExtras counts unmodeled fields and unknown parts dropped
	// in strip mode (0 in preserve mode); surfaced as the
	// clrk.translation.dropped_extras telemetry attribute.
	DroppedExtras int

	// Exact reports that Body is the verbatim decode-time capture,
	// emitted via the equality-gated passthrough (EncodeRaw).
	Exact bool
}

EncodedRequest is EncodeRequest's output: the body plus the header deltas the caller must apply. Path is the full :path including any query — schemas that carry the model in the URL (Gemini) synthesize it here.

type EncodedResponse

type EncodedResponse struct {
	Body          []byte
	SetHeaders    map[string]string
	DroppedExtras int
	Exact         bool
}

EncodedResponse is EncodeResponse's output.

type ExtraField

type ExtraField struct {
	Key   string          `json:"key,omitempty" yaml:"key,omitempty"`
	Value json.RawMessage `json:"value,omitempty" yaml:"value,omitempty"`
}

ExtraField is one unmodeled JSON object member captured at decode time. Value is the verbatim source subtree — nested key order, number literals, and string escapes inside it are preserved for free because it is never re-parsed. Extras are an ordered slice, never a map: preserve-mode encoding must re-emit members in source order.

+k8s:deepcopy-gen=true

func (*ExtraField) DeepCopy

func (in *ExtraField) DeepCopy() *ExtraField

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ExtraField.

func (*ExtraField) DeepCopyInto

func (in *ExtraField) DeepCopyInto(out *ExtraField)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type FieldEmitter

type FieldEmitter func(e *ObjectEncoder)

FieldEmitter writes one modeled key's member(s) to e. Emitters check presence themselves and may emit nothing.

type FilePart

type FilePart struct {
	MIMEType string `json:"mimeType,omitempty" yaml:"mimeType,omitempty"`
	DataB64  string `json:"dataB64,omitempty" yaml:"dataB64,omitempty"`
	URL      string `json:"url,omitempty" yaml:"url,omitempty"`
	Name     string `json:"name,omitempty" yaml:"name,omitempty"`
	Wire     Wire   `json:"wire,omitempty" yaml:"wire,omitempty"`
}

FilePart is document content (e.g. PDF); field semantics mirror ImagePart plus an optional display name.

+k8s:deepcopy-gen=true

func (*FilePart) DeepCopy

func (in *FilePart) DeepCopy() *FilePart

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new FilePart.

func (*FilePart) DeepCopyInto

func (in *FilePart) DeepCopyInto(out *FilePart)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type FinishReason

type FinishReason string

FinishReason is the canonical reason generation stopped. Values are the OTel GenAI semconv finish-reason spellings. Codecs that map several provider literals onto one canonical value keep the original in Choice.FinishReasonRaw so preserve-mode encoding round-trips.

const (
	FinishReasonStop          FinishReason = "stop"
	FinishReasonLength        FinishReason = "length"
	FinishReasonToolCalls     FinishReason = "tool_calls"
	FinishReasonContentFilter FinishReason = "content_filter"
	FinishReasonError         FinishReason = "error"
)

type GenerationConfig

type GenerationConfig struct {
	MaxTokens     json.Number `json:"maxTokens,omitempty" yaml:"maxTokens,omitempty"`
	Temperature   json.Number `json:"temperature,omitempty" yaml:"temperature,omitempty"`
	TopP          json.Number `json:"topP,omitempty" yaml:"topP,omitempty"`
	TopK          json.Number `json:"topK,omitempty" yaml:"topK,omitempty"`
	StopSequences []string    `json:"stopSequences,omitempty" yaml:"stopSequences,omitempty"`
	Wire          Wire        `json:"wire,omitempty" yaml:"wire,omitempty"`
}

GenerationConfig holds the cross-provider sampling and length parameters. Numeric fields are json.Number so the original wire literal ("1.0" vs "1", exponent forms) survives a preserve-mode round trip; the empty string means the field was absent.

+k8s:deepcopy-gen=true

func (*GenerationConfig) DeepCopy

func (in *GenerationConfig) DeepCopy() *GenerationConfig

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GenerationConfig.

func (*GenerationConfig) DeepCopyInto

func (in *GenerationConfig) DeepCopyInto(out *GenerationConfig)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type ImagePart

type ImagePart struct {
	MIMEType string `json:"mimeType,omitempty" yaml:"mimeType,omitempty"`
	DataB64  string `json:"dataB64,omitempty" yaml:"dataB64,omitempty"`
	URL      string `json:"url,omitempty" yaml:"url,omitempty"`
	Wire     Wire   `json:"wire,omitempty" yaml:"wire,omitempty"`
}

ImagePart is image content, carried inline (base64) or by reference. Exactly one of DataB64/URL is set; MIMEType qualifies inline data.

+k8s:deepcopy-gen=true

func (*ImagePart) DeepCopy

func (in *ImagePart) DeepCopy() *ImagePart

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ImagePart.

func (*ImagePart) DeepCopyInto

func (in *ImagePart) DeepCopyInto(out *ImagePart)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type Input

type Input struct {
	Method        string
	Path          string
	ReqHeaders    map[string]string
	RespHeaders   map[string]string
	ReqBody       []byte
	ReqTruncated  bool
	RespBody      []byte
	RespTruncated bool
}

Input is the narrow shape passed to a telemetry parser. Avoids importing extproc.Record so this package stays a leaf in the dep graph.

type MalformedError

type MalformedError struct {
	Provider string
	Detail   string
	Err      error
}

MalformedError reports bytes that are not the provider's wire format at all: truncation, non-JSON, HTML error pages. Callers must treat the capture as unparseable and never attempt translation.

func (*MalformedError) Error

func (e *MalformedError) Error() string

func (*MalformedError) Unwrap

func (e *MalformedError) Unwrap() error

type Message

type Message struct {
	Role  Role   `json:"role,omitempty" yaml:"role,omitempty"`
	Parts []Part `json:"parts,omitempty" yaml:"parts,omitempty"`
	Wire  Wire   `json:"wire,omitempty" yaml:"wire,omitempty"`
}

Message is one conversation turn.

+k8s:deepcopy-gen=true

func (*Message) DeepCopy

func (in *Message) DeepCopy() *Message

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Message.

func (*Message) DeepCopyInto

func (in *Message) DeepCopyInto(out *Message)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type Mode

type Mode int

Mode selects how an encoder treats wire bookkeeping. See the package documentation for the preserve/strip contract.

const (
	// ModePreserve re-emits extras and original formatting; an
	// unmutated same-provider round trip is byte-identical.
	ModePreserve Mode = iota
	// ModeStrip drops extras (counting them) and emits the provider's
	// canonical form. Cross-schema translation always strips.
	ModeStrip
)

type ObjectEncoder

type ObjectEncoder struct {
	// contains filtered or unexported fields
}

ObjectEncoder builds a compact JSON object member by member, letting codecs interleave canonically-marshaled fields with verbatim extras in Wire.KeyOrder sequence. Errors are sticky and surfaced by Bytes.

func NewObjectEncoder

func NewObjectEncoder() *ObjectEncoder

NewObjectEncoder returns an encoder with the object opened.

func (*ObjectEncoder) Bytes

func (e *ObjectEncoder) Bytes() ([]byte, error)

Bytes closes the object and returns it, or the first error.

func (*ObjectEncoder) Fail

func (e *ObjectEncoder) Fail(err error)

Fail records err on the encoder (sticky, surfaced by Bytes). Lets FieldEmitter closures propagate nested encode failures.

func (*ObjectEncoder) Field

func (e *ObjectEncoder) Field(key string, v any)

Field appends a member, marshaling v via MarshalCompact.

func (*ObjectEncoder) Raw

func (e *ObjectEncoder) Raw(key string, value json.RawMessage)

Raw appends a member with a verbatim (pre-encoded) value.

type Parser

type Parser interface {
	Parse(in Input) *ProviderInfo
}

Parser fills a ProviderInfo from one captured transaction.

type Part

type Part struct {
	Type PartType `json:"type,omitempty" yaml:"type,omitempty"`

	Text             *TextPart             `json:"text,omitempty" yaml:"text,omitempty"`
	Image            *ImagePart            `json:"image,omitempty" yaml:"image,omitempty"`
	Audio            *AudioPart            `json:"audio,omitempty" yaml:"audio,omitempty"`
	File             *FilePart             `json:"file,omitempty" yaml:"file,omitempty"`
	ToolCall         *ToolCallPart         `json:"toolCall,omitempty" yaml:"toolCall,omitempty"`
	ToolCallResponse *ToolCallResponsePart `json:"toolCallResponse,omitempty" yaml:"toolCallResponse,omitempty"`
	Reasoning        *ReasoningPart        `json:"reasoning,omitempty" yaml:"reasoning,omitempty"`
	Refusal          *RefusalPart          `json:"refusal,omitempty" yaml:"refusal,omitempty"`
	Citation         *CitationPart         `json:"citation,omitempty" yaml:"citation,omitempty"`

	Wire Wire `json:"wire,omitempty" yaml:"wire,omitempty"`
}

Part is one typed unit of message content — a tagged union in the same idiom as the CRD filter types: Type selects which variant pointer is set. PartTypeUnknown (the zero value) means no variant is set and Wire.Raw holds the unmodeled provider block verbatim.

The json/yaml tags on IR types serve debug dumps and golden tests; the wire bytes are owned by the codecs, never by these tags.

+k8s:deepcopy-gen=true

func UnknownPart

func UnknownPart(raw json.RawMessage) (Part, error)

UnknownPart captures an unmodeled content block verbatim (compacted) for preserve-mode re-emit / strip-mode drop+count.

func (*Part) DeepCopy

func (in *Part) DeepCopy() *Part

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Part.

func (*Part) DeepCopyInto

func (in *Part) DeepCopyInto(out *Part)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type PartType

type PartType string

PartType discriminates the Part union. Values are the OTel GenAI semconv message-part type spellings where semconv defines one (text, tool_call, tool_call_response, reasoning); the remaining kinds extend that vocabulary with the same naming convention.

const (
	PartTypeText             PartType = "text"
	PartTypeImage            PartType = "image"
	PartTypeAudio            PartType = "audio"
	PartTypeFile             PartType = "file"
	PartTypeToolCall         PartType = "tool_call"
	PartTypeToolCallResponse PartType = "tool_call_response"
	PartTypeReasoning        PartType = "reasoning"
	PartTypeRefusal          PartType = "refusal"
	PartTypeCitation         PartType = "citation"
	// PartTypeUnknown marks a provider content block the IR doesn't
	// model. The verbatim block rides in Wire.Raw: re-emitted in
	// preserve mode, dropped and counted in strip mode.
	PartTypeUnknown PartType = ""
)

type Provider

type Provider struct {
	// Name is the canonical gen_ai.system value ("anthropic", "openai",
	// "google_genai"). It is the key the route table, backend schema
	// resolution, and telemetry all share.
	Name string

	// Aliases are CRD short names that resolve to Name via Canonical
	// (e.g. AIProviderRouteMatch.Provider "google" -> "google_genai").
	Aliases []string

	// Hosts are the exact :authority hosts (lowercase, port already
	// stripped by the caller) this provider serves.
	Hosts []string

	// MatchHost, when set, replaces the exact Hosts match — for
	// wildcard surfaces like *.openai.azure.com. It receives a
	// lowercased host.
	MatchHost func(host string) bool

	// Telemetry extracts ProviderInfo from captured transactions.
	// Required for built-in providers.
	Telemetry Parser

	// Codec converts this provider's wire format to and from the IR.
	// May be nil: the provider is then registered (telemetry, routing)
	// but not translatable.
	Codec Codec

	// Capabilities declare what the schema can express; consumed by
	// cross-schema backend selection.
	Capabilities Capabilities

	// StreamCodec converts this provider's streamed response framing
	// to and from StreamEvents. Nil disables streaming translation for
	// any pair that includes this provider.
	StreamCodec StreamCodec

	// EnsureStreamUsage, when set, rewrites a raw request body in this
	// provider's wire schema so streamed responses always report
	// terminal usage (OpenAI's stream_options.include_usage). Returns
	// (newBody, true) only when a rewrite was needed. Nil for schemas
	// whose streams carry usage unconditionally. The hook decodes the
	// body — hot-path callers pre-probe with a cheap byte scan
	// (parsers.BodyAdvertisesStream).
	EnsureStreamUsage func(path string, body []byte) ([]byte, bool)

	// AuthHeaders are the credential-bearing request headers this
	// provider's clients send (lowercase). Translated requests shed
	// the union across all registered providers (AuthHeaderUnion) —
	// an agent-supplied credential belongs to the source schema and
	// must never reach a different upstream.
	AuthHeaders []string

	// ErrorBody shapes a proxy-generated error payload (translation
	// failures, fail-closed 502s) in this provider's error envelope —
	// the client SDK is mid-parse of this schema, and a foreign blob
	// would surface as a second, more confusing error at the call
	// site. Nil falls back to the OpenAI envelope (see ErrorBodyFor).
	ErrorBody func(msg string) []byte
}

Provider is one registered wire schema: identity, traffic matching, telemetry extraction, and (when implemented) IR conversion. Provider packages under providers/ construct one and call Register from init.

func ByHost

func ByHost(host string) *Provider

ByHost returns the provider serving host (the request's :authority host portion, port stripped). Returns nil when no provider matches.

func ByName

func ByName(system string) *Provider

ByName returns the provider registered under a canonical gen_ai.system value. It deliberately does NOT resolve aliases — callers hold canonical names (backend schema resolution runs Canonical first), and an alias-only system ("azure_openai" before its plugin exists) must stay unresolvable so downstream parsing is skipped, the same as for any unrecognized host.

func (*Provider) StreamsTranslatable

func (p *Provider) StreamsTranslatable() bool

StreamsTranslatable reports whether p can sit on either side of a streaming cross-schema translation: a stream codec exists and the capability bit is on. The bit matters independently of the codec — a future provider may stream in a framing its SSE-based codec does not cover (Bedrock's binary event stream).

type ProviderInfo

type ProviderInfo struct {
	// System is the canonical OTel gen_ai.system value
	// (e.g. "anthropic", "openai", "azure_openai", "google_genai",
	// "aws_bedrock"). Always non-empty when the parser ran.
	System string

	// Operation is the gen_ai.operation.name value
	// ("chat", "text_completion", "embeddings", ...). Empty when the
	// parser couldn't classify the path.
	Operation string

	// RequestModel is the model the caller asked for. From the request
	// body's "model" field. Empty when the request body wasn't decodable
	// (truncation, non-JSON).
	RequestModel string

	// ResponseModel is the model the provider actually served (often
	// equal to RequestModel but can include a version suffix). Some
	// providers don't echo it for non-streaming responses.
	ResponseModel string

	// InputTokens / OutputTokens are the provider-reported usage
	// counts. Zero when not present (truncation, error response with no
	// usage block, streaming response with no usage block).
	InputTokens  int64
	OutputTokens int64

	// StreamResponse is true when the response advertised
	// content-type: text/event-stream. Token accounting is intentionally
	// skipped on streams in MVP — providers report usage inconsistently
	// across SSE chunks.
	StreamResponse bool

	// UsageVisible indicates that the response body contained the usage
	// block we look for. False when truncation hid it; lets the OTLP
	// sink emit clrk.body.usage_visible=false to nudge operators to
	// raise CaptureBody.MaxBytes.
	UsageVisible bool

	// ResponseContent is the assistant message reconstructed from a
	// streamed response: concatenated text plus any tool calls. Unlike
	// the fields above it is NOT filled by the Parser — enrichRecord
	// populates it post-parse via AssembleStreamedResponse, which drives
	// the serving schema's StreamCodec so every provider is reassembled
	// by one path. Empty for non-streamed responses (whose body is
	// already a single readable JSON), when no stream codec exists, and
	// when nothing decoded. Best-effort: partial under a truncated
	// capture (the keep-last-N ring drops the head of long streams).
	ResponseContent string
}

ProviderInfo is the parsed view of one HTTP transaction against a known AI provider. All fields are best-effort — when the captured body is truncated or in an unrecognized shape, fields stay at their zero values and the caller should treat them as absent.

type ReasoningPart

type ReasoningPart struct {
	Text      string `json:"text,omitempty" yaml:"text,omitempty"`
	Signature string `json:"signature,omitempty" yaml:"signature,omitempty"`
	Wire      Wire   `json:"wire,omitempty" yaml:"wire,omitempty"`
}

ReasoningPart is extended-thinking / reasoning content. Signature carries provider verification material (e.g. Anthropic's thinking signature) required to round-trip the block.

+k8s:deepcopy-gen=true

func (*ReasoningPart) DeepCopy

func (in *ReasoningPart) DeepCopy() *ReasoningPart

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ReasoningPart.

func (*ReasoningPart) DeepCopyInto

func (in *ReasoningPart) DeepCopyInto(out *ReasoningPart)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type RefusalPart

type RefusalPart struct {
	Text string `json:"text,omitempty" yaml:"text,omitempty"`
	Wire Wire   `json:"wire,omitempty" yaml:"wire,omitempty"`
}

RefusalPart is an explicit model refusal (OpenAI refusal blocks).

+k8s:deepcopy-gen=true

func (*RefusalPart) DeepCopy

func (in *RefusalPart) DeepCopy() *RefusalPart

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RefusalPart.

func (*RefusalPart) DeepCopyInto

func (in *RefusalPart) DeepCopyInto(out *RefusalPart)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type Request

type Request struct {
	// Provider is the canonical gen_ai.system name of the schema the
	// request was decoded from, stamped by DecodeRequest.
	Provider string `json:"provider,omitempty" yaml:"provider,omitempty"`

	// Operation is the gen_ai.operation.name value ("chat", ...).
	Operation string `json:"operation,omitempty" yaml:"operation,omitempty"`

	// Model is the requested model ID. For schemas that carry the model
	// in the URL rather than the body (Gemini), codecs keep Wire.Path
	// authoritative and synthesize the path on encode.
	Model string `json:"model,omitempty" yaml:"model,omitempty"`

	// System is the system/instructions content. Kept separate from
	// Messages because providers disagree on whether it is a message
	// role or a top-level field.
	System []Part `json:"system,omitempty" yaml:"system,omitempty"`

	Messages   []Message        `json:"messages,omitempty" yaml:"messages,omitempty"`
	Tools      []ToolDefinition `json:"tools,omitempty" yaml:"tools,omitempty"`
	ToolChoice *ToolChoice      `json:"toolChoice,omitempty" yaml:"toolChoice,omitempty"`
	Generation GenerationConfig `json:"generation,omitempty" yaml:"generation,omitempty"`
	Stream     bool             `json:"stream,omitempty" yaml:"stream,omitempty"`

	// Wire holds root-level bookkeeping: the full original body in Raw,
	// the original :path, and modeled request headers.
	Wire Wire `json:"wire,omitempty" yaml:"wire,omitempty"`
}

Request is the canonical decoded form of one LLM API request.

+k8s:deepcopy-gen=true

func (*Request) DeepCopy

func (in *Request) DeepCopy() *Request

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Request.

func (*Request) DeepCopyInto

func (in *Request) DeepCopyInto(out *Request)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type RequestInput

type RequestInput struct {
	Method  string
	Path    string
	Headers map[string]string
	Body    []byte
}

RequestInput is the captured HTTP request handed to DecodeRequest. Header keys are lowercased, as ext_proc delivers them.

type Response

type Response struct {
	// Provider is the canonical gen_ai.system name of the schema the
	// response was decoded from.
	Provider string `json:"provider,omitempty" yaml:"provider,omitempty"`

	ID      string   `json:"id,omitempty" yaml:"id,omitempty"`
	Model   string   `json:"model,omitempty" yaml:"model,omitempty"`
	Choices []Choice `json:"choices,omitempty" yaml:"choices,omitempty"`
	Usage   Usage    `json:"usage,omitempty" yaml:"usage,omitempty"`

	Wire Wire `json:"wire,omitempty" yaml:"wire,omitempty"`
}

Response is the canonical decoded form of one LLM API response.

+k8s:deepcopy-gen=true

func (*Response) DeepCopy

func (in *Response) DeepCopy() *Response

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Response.

func (*Response) DeepCopyInto

func (in *Response) DeepCopyInto(out *Response)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type ResponseInput

type ResponseInput struct {
	Status  int
	Headers map[string]string
	Body    []byte
}

ResponseInput is the captured HTTP response handed to DecodeResponse.

type Role

type Role string

Role is a conversation participant. Values are the OTel GenAI semconv role spellings.

const (
	RoleSystem    Role = "system"
	RoleUser      Role = "user"
	RoleAssistant Role = "assistant"
	RoleTool      Role = "tool"
)

type SSEEvent

type SSEEvent struct {
	// Name is the "event:" field; empty for bare data frames.
	Name string
	// Data is the "data:" payload, multi-line continuations joined
	// with '\n' per the SSE spec.
	Data []byte
}

SSEEvent is one reassembled server-sent event.

type SSEScanner

type SSEScanner struct {
	// contains filtered or unexported fields
}

SSEScanner incrementally reassembles SSE events across arbitrary chunk boundaries — a chunk may end mid-line, mid-field, even mid-rune. Feed returns the events the new chunk completed; Flush drains the final unterminated event at end of stream. The zero value is ready to use.

Unlike ScanSSEData (whole-buffer, tolerant, telemetry-side), the scanner is strict about frame boundaries so stream codecs never decode a half-received payload.

func (*SSEScanner) Feed

func (s *SSEScanner) Feed(chunk []byte) []SSEEvent

Feed appends chunk and returns the events completed by it. Returned Data slices are owned by the caller (copied out of the buffer).

func (*SSEScanner) Flush

func (s *SSEScanner) Flush() (*SSEEvent, []byte)

Flush returns the final event when the stream ended without the terminating blank line (complete field lines only), plus any leftover partial line — non-empty leftover means the stream was truncated mid-frame.

type Shadow

type Shadow struct {
	Raw json.RawMessage `json:"raw,omitempty" yaml:"raw,omitempty"`
	Val any             `json:"val,omitempty" yaml:"val,omitempty"`
}

Shadow pairs the verbatim source bytes of a modeled scalar with the value decoded from them. Codecs record one when re-marshaling the decoded value would not reproduce the source bytes (\uXXXX escapes, exotic number literals, value-bearing formatting). At encode time the shadow's Raw is emitted only while Val still equals the field — a mutated field falls back to canonical marshaling.

func (*Shadow) DeepCopy

func (in *Shadow) DeepCopy() *Shadow

DeepCopy returns a deep copy of the Shadow.

func (*Shadow) DeepCopyInto

func (in *Shadow) DeepCopyInto(out *Shadow)

DeepCopyInto is hand-written because deepcopy-gen cannot generate for the interface-typed Val. Val holds only comparable scalars (string, bool, json.Number, int64 — see RecordShadow), so plain assignment is a correct deep copy.

type StreamCodec

type StreamCodec interface {
	// NewStreamDecoder returns a decoder for a streamed response to
	// req — the request as sent upstream, in this provider's schema.
	NewStreamDecoder(req *Request) StreamDecoder
	// NewStreamEncoder returns an encoder rendering a stream for the
	// client that sent req in this provider's schema. req supplies
	// fallbacks (model) and framing choices.
	NewStreamEncoder(req *Request) StreamEncoder
}

StreamCodec mints per-stream decoder/encoder state for one provider's streamed response framing. Implementations are stateless; per-stream state lives on the returned values.

type StreamContentEnd

type StreamContentEnd struct {
	Index int `json:"index,omitempty" yaml:"index,omitempty"`
}

StreamContentEnd closes the content block at Index.

+k8s:deepcopy-gen=true

func (*StreamContentEnd) DeepCopy

func (in *StreamContentEnd) DeepCopy() *StreamContentEnd

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new StreamContentEnd.

func (*StreamContentEnd) DeepCopyInto

func (in *StreamContentEnd) DeepCopyInto(out *StreamContentEnd)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type StreamContentStart

type StreamContentStart struct {
	Index int  `json:"index,omitempty" yaml:"index,omitempty"`
	Part  Part `json:"part,omitempty" yaml:"part,omitempty"`
}

StreamContentStart opens the content block at Index; Part carries the block's initial typed shape (empty text, tool-call ID + name).

+k8s:deepcopy-gen=true

func (*StreamContentStart) DeepCopy

func (in *StreamContentStart) DeepCopy() *StreamContentStart

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new StreamContentStart.

func (*StreamContentStart) DeepCopyInto

func (in *StreamContentStart) DeepCopyInto(out *StreamContentStart)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type StreamDecoder

type StreamDecoder interface {
	// Decode parses the events completed by chunk. eos marks the final
	// chunk: the decoder flushes its reassembly buffer, closes any
	// still-open blocks, synthesizes a terminal done event when the
	// provider's own terminator never arrived, and reports a dangling
	// partial frame via MalformedError.
	Decode(chunk []byte, eos bool) ([]StreamEvent, error)
}

StreamDecoder consumes one streamed response's raw upstream bytes — chunks may split anywhere, including mid-SSE-line — and yields canonical StreamEvents in arrival order. One decoder per stream; all reassembly and block-index state lives on the decoder. Not safe for concurrent use.

Decoders assign dense IR block indexes from 0 in open order and always emit content_start before a block's first delta, synthesizing one when the provider skipped its open frame. They set StreamFinish.FinishReasonRaw for telemetry; encoders ignore it.

type StreamEncoder

type StreamEncoder interface {
	// ContentType is the content-type the encoded stream rides on.
	ContentType() string
	// Encode appends the wire rendering of events. Empty output is
	// valid — events may buffer until a block completes, and a
	// mid-frame upstream chunk completes no events at all.
	Encode(events []StreamEvent) ([]byte, error)
}

StreamEncoder renders canonical StreamEvents in one provider's streamed response framing — SSE framing included. One encoder per stream. Not safe for concurrent use.

Encoders map only the canonical FinishReason, never FinishReasonRaw: streams are never preserve-mode re-encoded, so there is no fidelity contract, and the raw value is source-schema vocabulary that must not leak into the client's. Usage events are cumulative snapshots — last wins — folded into the encoder's terminal frame sequence.

type StreamError

type StreamError struct {
	Code    string `json:"code,omitempty" yaml:"code,omitempty"`
	Message string `json:"message,omitempty" yaml:"message,omitempty"`
}

StreamError is a mid-stream provider error event.

+k8s:deepcopy-gen=true

func (*StreamError) DeepCopy

func (in *StreamError) DeepCopy() *StreamError

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new StreamError.

func (*StreamError) DeepCopyInto

func (in *StreamError) DeepCopyInto(out *StreamError)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type StreamEvent

type StreamEvent struct {
	Type StreamEventType `json:"type,omitempty" yaml:"type,omitempty"`

	Start          *StreamStart          `json:"start,omitempty" yaml:"start,omitempty"`
	ContentStart   *StreamContentStart   `json:"contentStart,omitempty" yaml:"contentStart,omitempty"`
	TextDelta      *StreamTextDelta      `json:"textDelta,omitempty" yaml:"textDelta,omitempty"`
	ToolCallDelta  *StreamToolCallDelta  `json:"toolCallDelta,omitempty" yaml:"toolCallDelta,omitempty"`
	ReasoningDelta *StreamReasoningDelta `json:"reasoningDelta,omitempty" yaml:"reasoningDelta,omitempty"`
	ContentEnd     *StreamContentEnd     `json:"contentEnd,omitempty" yaml:"contentEnd,omitempty"`
	Usage          *Usage                `json:"usage,omitempty" yaml:"usage,omitempty"`
	Finish         *StreamFinish         `json:"finish,omitempty" yaml:"finish,omitempty"`
	Error          *StreamError          `json:"error,omitempty" yaml:"error,omitempty"`

	Wire Wire `json:"wire,omitempty" yaml:"wire,omitempty"`
}

StreamEvent is one canonical streamed-response event — a tagged union in the same idiom as Part. StreamEventDone has no payload.

+k8s:deepcopy-gen=true

func (*StreamEvent) DeepCopy

func (in *StreamEvent) DeepCopy() *StreamEvent

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new StreamEvent.

func (*StreamEvent) DeepCopyInto

func (in *StreamEvent) DeepCopyInto(out *StreamEvent)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type StreamEventType

type StreamEventType string

StreamEventType discriminates the StreamEvent union — the canonical taxonomy for streamed responses. Phase 1 defines the types only; streaming codecs are a later phase (APO-743).

const (
	// StreamEventStart opens a streamed response (response ID, model).
	StreamEventStart StreamEventType = "start"
	// StreamEventContentStart opens one content block at an index.
	StreamEventContentStart StreamEventType = "content_start"
	// StreamEventTextDelta appends text to an open block.
	StreamEventTextDelta StreamEventType = "text_delta"
	// StreamEventToolCallDelta appends tool-call argument bytes.
	StreamEventToolCallDelta StreamEventType = "tool_call_delta"
	// StreamEventReasoningDelta appends reasoning text.
	StreamEventReasoningDelta StreamEventType = "reasoning_delta"
	// StreamEventContentEnd closes the block at an index.
	StreamEventContentEnd StreamEventType = "content_end"
	// StreamEventUsage reports (cumulative) token usage.
	StreamEventUsage StreamEventType = "usage"
	// StreamEventFinish reports the finish reason.
	StreamEventFinish StreamEventType = "finish"
	// StreamEventError reports a mid-stream provider error.
	StreamEventError StreamEventType = "error"
	// StreamEventDone terminates the stream (OpenAI's [DONE]).
	StreamEventDone StreamEventType = "done"
)

type StreamFinish

type StreamFinish struct {
	FinishReason    FinishReason `json:"finishReason,omitempty" yaml:"finishReason,omitempty"`
	FinishReasonRaw string       `json:"finishReasonRaw,omitempty" yaml:"finishReasonRaw,omitempty"`
}

StreamFinish reports why generation stopped; FinishReasonRaw mirrors Choice.FinishReasonRaw.

+k8s:deepcopy-gen=true

func (*StreamFinish) DeepCopy

func (in *StreamFinish) DeepCopy() *StreamFinish

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new StreamFinish.

func (*StreamFinish) DeepCopyInto

func (in *StreamFinish) DeepCopyInto(out *StreamFinish)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type StreamReasoningDelta

type StreamReasoningDelta struct {
	Index int    `json:"index,omitempty" yaml:"index,omitempty"`
	Text  string `json:"text,omitempty" yaml:"text,omitempty"`
}

StreamReasoningDelta appends reasoning text to the block at Index.

+k8s:deepcopy-gen=true

func (*StreamReasoningDelta) DeepCopy

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new StreamReasoningDelta.

func (*StreamReasoningDelta) DeepCopyInto

func (in *StreamReasoningDelta) DeepCopyInto(out *StreamReasoningDelta)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type StreamStart

type StreamStart struct {
	ID    string `json:"id,omitempty" yaml:"id,omitempty"`
	Model string `json:"model,omitempty" yaml:"model,omitempty"`
}

StreamStart carries the stream-opening metadata.

+k8s:deepcopy-gen=true

func (*StreamStart) DeepCopy

func (in *StreamStart) DeepCopy() *StreamStart

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new StreamStart.

func (*StreamStart) DeepCopyInto

func (in *StreamStart) DeepCopyInto(out *StreamStart)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type StreamTextDelta

type StreamTextDelta struct {
	Index int    `json:"index,omitempty" yaml:"index,omitempty"`
	Text  string `json:"text,omitempty" yaml:"text,omitempty"`
}

StreamTextDelta appends text to the block at Index.

+k8s:deepcopy-gen=true

func (*StreamTextDelta) DeepCopy

func (in *StreamTextDelta) DeepCopy() *StreamTextDelta

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new StreamTextDelta.

func (*StreamTextDelta) DeepCopyInto

func (in *StreamTextDelta) DeepCopyInto(out *StreamTextDelta)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type StreamToolCallDelta

type StreamToolCallDelta struct {
	Index          int    `json:"index,omitempty" yaml:"index,omitempty"`
	ID             string `json:"id,omitempty" yaml:"id,omitempty"`
	Name           string `json:"name,omitempty" yaml:"name,omitempty"`
	ArgumentsDelta string `json:"argumentsDelta,omitempty" yaml:"argumentsDelta,omitempty"`
}

StreamToolCallDelta appends argument bytes to a tool call. ID and Name are set when the provider repeats them per chunk (OpenAI); empty otherwise.

+k8s:deepcopy-gen=true

func (*StreamToolCallDelta) DeepCopy

func (in *StreamToolCallDelta) DeepCopy() *StreamToolCallDelta

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new StreamToolCallDelta.

func (*StreamToolCallDelta) DeepCopyInto

func (in *StreamToolCallDelta) DeepCopyInto(out *StreamToolCallDelta)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type TextPart

type TextPart struct {
	Text string `json:"text,omitempty" yaml:"text,omitempty"`
	Wire Wire   `json:"wire,omitempty" yaml:"wire,omitempty"`
}

TextPart is plain text content.

+k8s:deepcopy-gen=true

func (*TextPart) DeepCopy

func (in *TextPart) DeepCopy() *TextPart

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new TextPart.

func (*TextPart) DeepCopyInto

func (in *TextPart) DeepCopyInto(out *TextPart)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type ToolCallPart

type ToolCallPart struct {
	ID        string          `json:"id,omitempty" yaml:"id,omitempty"`
	Name      string          `json:"name,omitempty" yaml:"name,omitempty"`
	Arguments json.RawMessage `json:"arguments,omitempty" yaml:"arguments,omitempty"`
	Wire      Wire            `json:"wire,omitempty" yaml:"wire,omitempty"`
}

ToolCallPart is the model requesting a tool invocation. Arguments is the verbatim JSON arguments object (providers disagree on whether it rides as an object or a JSON-encoded string; codecs normalize to the object form and shadow the original).

+k8s:deepcopy-gen=true

func (*ToolCallPart) DeepCopy

func (in *ToolCallPart) DeepCopy() *ToolCallPart

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ToolCallPart.

func (*ToolCallPart) DeepCopyInto

func (in *ToolCallPart) DeepCopyInto(out *ToolCallPart)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type ToolCallResponsePart

type ToolCallResponsePart struct {
	ToolCallID string `json:"toolCallID,omitempty" yaml:"toolCallID,omitempty"`
	Parts      []Part `json:"parts,omitempty" yaml:"parts,omitempty"`
	IsError    bool   `json:"isError,omitempty" yaml:"isError,omitempty"`
	Wire       Wire   `json:"wire,omitempty" yaml:"wire,omitempty"`
}

ToolCallResponsePart is a tool result returned to the model.

+k8s:deepcopy-gen=true

func (*ToolCallResponsePart) DeepCopy

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ToolCallResponsePart.

func (*ToolCallResponsePart) DeepCopyInto

func (in *ToolCallResponsePart) DeepCopyInto(out *ToolCallResponsePart)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type ToolChoice

type ToolChoice struct {
	Mode ToolChoiceMode `json:"mode,omitempty" yaml:"mode,omitempty"`
	Name string         `json:"name,omitempty" yaml:"name,omitempty"`
	Wire Wire           `json:"wire,omitempty" yaml:"wire,omitempty"`
}

ToolChoice steers tool selection for a request.

+k8s:deepcopy-gen=true

func (*ToolChoice) DeepCopy

func (in *ToolChoice) DeepCopy() *ToolChoice

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ToolChoice.

func (*ToolChoice) DeepCopyInto

func (in *ToolChoice) DeepCopyInto(out *ToolChoice)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type ToolChoiceMode

type ToolChoiceMode string

ToolChoiceMode is how the model is steered toward tool use.

const (
	ToolChoiceAuto     ToolChoiceMode = "auto"
	ToolChoiceNone     ToolChoiceMode = "none"
	ToolChoiceRequired ToolChoiceMode = "required"
	// ToolChoiceNamed forces one specific tool; Name carries it.
	ToolChoiceNamed ToolChoiceMode = "named"
)

type ToolDefinition

type ToolDefinition struct {
	Name        string          `json:"name,omitempty" yaml:"name,omitempty"`
	Description string          `json:"description,omitempty" yaml:"description,omitempty"`
	Parameters  json.RawMessage `json:"parameters,omitempty" yaml:"parameters,omitempty"`
	Wire        Wire            `json:"wire,omitempty" yaml:"wire,omitempty"`
}

ToolDefinition declares a tool the model may call. Parameters is the verbatim JSON Schema object — schema dialects ride through untouched.

+k8s:deepcopy-gen=true

func (*ToolDefinition) DeepCopy

func (in *ToolDefinition) DeepCopy() *ToolDefinition

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ToolDefinition.

func (*ToolDefinition) DeepCopyInto

func (in *ToolDefinition) DeepCopyInto(out *ToolDefinition)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type Usage

type Usage struct {
	InputTokens  int64 `json:"inputTokens,omitempty" yaml:"inputTokens,omitempty"`
	OutputTokens int64 `json:"outputTokens,omitempty" yaml:"outputTokens,omitempty"`
	Wire         Wire  `json:"wire,omitempty" yaml:"wire,omitempty"`
}

Usage is provider-reported token accounting.

+k8s:deepcopy-gen=true

func (*Usage) DeepCopy

func (in *Usage) DeepCopy() *Usage

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Usage.

func (*Usage) DeepCopyInto

func (in *Usage) DeepCopyInto(out *Usage)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type Wire

type Wire struct {
	// KeyOrder is the node's full original member sequence — modeled
	// and extra keys interleaved. The encoder walks it to reproduce
	// source order; modeled keys absent from it (added after decode)
	// are appended in the codec's canonical order.
	KeyOrder []string `json:"keyOrder,omitempty" yaml:"keyOrder,omitempty"`

	// Extras are the unmodeled members in encounter order.
	Extras []ExtraField `json:"extras,omitempty" yaml:"extras,omitempty"`

	// Shadow maps a modeled key to its source-byte fallback. See Shadow.
	Shadow map[string]Shadow `json:"shadow,omitempty" yaml:"shadow,omitempty"`

	// Hints record wire-shape variants the typed fields can't express,
	// e.g. {"content": "string"} when an OpenAI message carried string
	// content rather than a parts array. Keys and values are
	// codec-private conventions.
	Hints map[string]string `json:"hints,omitempty" yaml:"hints,omitempty"`

	// Raw is the verbatim source subtree. On root nodes (Request,
	// Response) it is the full original body, used by the
	// equality-gated passthrough (EncodeRaw). On unknown Parts it is
	// the whole unmodeled block.
	Raw json.RawMessage `json:"raw,omitempty" yaml:"raw,omitempty"`

	// Path is the original request :path. Root Request only.
	Path string `json:"path,omitempty" yaml:"path,omitempty"`

	// Headers are modeled request headers captured at decode time
	// (e.g. anthropic-version), lowercased keys. Root Request only.
	Headers map[string]string `json:"headers,omitempty" yaml:"headers,omitempty"`
}

Wire is per-node bookkeeping that makes preserve-mode encoding byte-faithful. The zero value means "programmatically built": no extras, canonical field order, canonical formatting.

+k8s:deepcopy-gen=true

func (*Wire) AddExtra

func (w *Wire) AddExtra(key string, value json.RawMessage)

AddExtra appends an unmodeled member, preserving encounter order.

func (*Wire) DeepCopy

func (in *Wire) DeepCopy() *Wire

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Wire.

func (*Wire) DeepCopyInto

func (in *Wire) DeepCopyInto(out *Wire)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

func (*Wire) Extra

func (w *Wire) Extra(key string) (json.RawMessage, bool)

Extra returns the value of the named extra member, if captured.

func (*Wire) HasKey

func (w *Wire) HasKey(key string) bool

HasKey reports whether key was present in the node's original member sequence. Emitters use it for presence semantics: a modeled key in KeyOrder re-emits even at its zero value (e.g. "stream": false), while a key absent from it emits only when meaningfully set.

func (*Wire) Hint

func (w *Wire) Hint(key string) string

Hint returns the named wire-shape hint ("" when unset).

func (*Wire) RecordShadow

func (w *Wire) RecordShadow(key string, raw json.RawMessage, val any)

RecordShadow stores source bytes for a modeled key whose canonical re-marshal differs from the source. Val must be a comparable scalar (string, bool, json.Number, int64) — ShadowRaw compares with ==.

func (*Wire) SetHint

func (w *Wire) SetHint(key, value string)

SetHint records a wire-shape hint.

func (*Wire) ShadowRaw

func (w *Wire) ShadowRaw(key string, val any) (json.RawMessage, bool)

ShadowRaw returns the recorded source bytes for key when the shadowed value still equals val — i.e. the field was not mutated since decode.

Directories

Path Synopsis
providers
all
Package all registers every built-in llmcall provider via blank imports.
Package all registers every built-in llmcall provider via blank imports.
anthropic
Package anthropic is the llmcall provider plugin for api.anthropic.com (the Anthropic Messages API).
Package anthropic is the llmcall provider plugin for api.anthropic.com (the Anthropic Messages API).
azureopenai
Package azureopenai is the llmcall provider plugin for Azure OpenAI (*.openai.azure.com).
Package azureopenai is the llmcall provider plugin for Azure OpenAI (*.openai.azure.com).
bedrock
Package bedrock is the llmcall provider plugin for AWS Bedrock's Converse API (bedrock-runtime.<region>.amazonaws.com).
Package bedrock is the llmcall provider plugin for AWS Bedrock's Converse API (bedrock-runtime.<region>.amazonaws.com).
google
Package google is the llmcall provider plugin for generativelanguage.googleapis.com (Google's consumer Gemini API; "google_genai" in OTel gen_ai semconv).
Package google is the llmcall provider plugin for generativelanguage.googleapis.com (Google's consumer Gemini API; "google_genai" in OTel gen_ai semconv).
openai
Package openai is the llmcall provider plugin for api.openai.com (the OpenAI Chat Completions API surface).
Package openai is the llmcall provider plugin for api.openai.com (the OpenAI Chat Completions API surface).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL