middleware

type AddToolInputExamplesMiddleware ¶

type AddToolInputExamplesMiddleware struct {
	BaseMiddleware

	// Prefix is prepended before the serialized examples list. Defaults to
	// "Input Examples:" when empty.
	Prefix string

	// Format renders a single example. Defaults to json.Marshal(ex.Input)
	// when nil.
	Format func(ex tool.ToolInputExample, index int) string

	// Remove clears the InputExamples field on the transformed tool after
	// folding them into the description. Defaults to true (matching ai-sdk).
	// Use a pointer? We stick with the zero-value semantics of ai-sdk: if
	// you want Remove=false you must opt in. ai-sdk's default is `true`, so
	// we invert via KeepExamples for Go-idiomatic zero-value defaults.
	KeepExamples bool
}

AddToolInputExamplesMiddleware serializes each function tool's InputExamples into its Description, for providers that don't natively accept an input_examples field. Mirrors ai-sdk's addToolInputExamplesMiddleware.

Provider-defined tools (Type == "provider") are skipped — examples on built-in tools aren't meaningful since the provider runs them itself.

func (m *AddToolInputExamplesMiddleware) TransformOptions(options *stream.CallOptions) *stream.CallOptions

type BaseEmbeddingMiddleware ¶

type BaseEmbeddingMiddleware struct{}

BaseEmbeddingMiddleware is a pass-through embedding middleware — embed it in your own type and override the hook(s) you care about.

func (BaseEmbeddingMiddleware) TransformOptions(options *model.EmbedCallOptions) *model.EmbedCallOptions

func (BaseEmbeddingMiddleware) WrapEmbed(ctx context.Context, options *model.EmbedCallOptions, doEmbed EmbedFunc) (*model.EmbedResult, error)

type BaseImageMiddleware ¶

type BaseImageMiddleware struct{}

BaseImageMiddleware is a pass-through image middleware.

func (BaseImageMiddleware) TransformOptions(options *model.ImageCallOptions) *model.ImageCallOptions

func (BaseImageMiddleware) WrapGenerate(ctx context.Context, options *model.ImageCallOptions, doGenerate GenerateImageFunc) (*model.ImageResult, error)

type BaseMiddleware ¶

type BaseMiddleware struct{}

BaseMiddleware provides a simple implementation that can be embedded. Override the methods you need.

func (b *BaseMiddleware) TransformOptions(options *stream.CallOptions) *stream.CallOptions

func (b *BaseMiddleware) WrapStream(ctx context.Context, options *stream.CallOptions, doStream StreamFunc) (<-chan stream.Event, error)

type DefaultEmbeddingSettingsMiddleware ¶

type DefaultEmbeddingSettingsMiddleware struct {
	BaseEmbeddingMiddleware

	DefaultHeaders         map[string]string
	DefaultProviderOptions map[string]any
}

DefaultEmbeddingSettingsMiddleware applies default Headers and ProviderOptions to every embedding call. Existing values on the incoming options take precedence.

func (d *DefaultEmbeddingSettingsMiddleware) TransformOptions(options *model.EmbedCallOptions) *model.EmbedCallOptions

type DefaultImageSettingsMiddleware ¶

type DefaultImageSettingsMiddleware struct {
	BaseImageMiddleware

	DefaultHeaders         map[string]string
	DefaultProviderOptions map[string]any
}

DefaultImageSettingsMiddleware applies default Headers and ProviderOptions to every image generation call. Existing values on the incoming options take precedence.

func (d *DefaultImageSettingsMiddleware) TransformOptions(options *model.ImageCallOptions) *model.ImageCallOptions

type DefaultSettingsMiddleware ¶

type DefaultSettingsMiddleware struct {
	BaseMiddleware

	// DefaultTemperature is applied if options.Temperature is nil.
	DefaultTemperature *float64

	// DefaultTopP is applied if options.TopP is nil.
	DefaultTopP *float64

	// DefaultMaxOutputTokens is applied if options.MaxOutputTokens is nil.
	DefaultMaxOutputTokens *int

	// DefaultProviderOptions are merged with options.ProviderOptions.
	DefaultProviderOptions map[string]any
}

DefaultSettingsMiddleware applies default settings to options.

func (d *DefaultSettingsMiddleware) TransformOptions(options *stream.CallOptions) *stream.CallOptions

type EmbedFunc ¶

type EmbedFunc func(ctx context.Context, options *model.EmbedCallOptions) (*model.EmbedResult, error)

EmbedFunc is the inner callable passed to WrapEmbed — invokes the wrapped model's Embed method with the (possibly transformed) options.

type EmbeddingMiddleware ¶

type EmbeddingMiddleware interface {
	TransformOptions(options *model.EmbedCallOptions) *model.EmbedCallOptions
	WrapEmbed(ctx context.Context, options *model.EmbedCallOptions, doEmbed EmbedFunc) (*model.EmbedResult, error)
}

EmbeddingMiddleware mirrors ai-sdk's EmbeddingModelMiddleware. TransformOptions rewrites the call options before they reach the model; WrapEmbed is a chance to run code before/after the actual Embed call. Either hook may be nil — nil TransformOptions passes options through unchanged, nil WrapEmbed passes the call straight to doEmbed.

type ExtractJsonMiddleware ¶

type ExtractJsonMiddleware struct {
	BaseMiddleware

	// Transform is an optional custom text transformer. When nil, the
	// default markdown-fence stripper is used.
	Transform func(string) string
}

ExtractJsonMiddleware strips markdown JSON code fences from streamed text so downstream structured-output parsers see raw JSON. Useful for models that wrap their JSON responses in ```json … ``` even when asked for strict JSON.

Mirrors ai-sdk's extractJsonMiddleware.

Two modes:

• Default (Transform == nil): streams content incrementally, stripping the leading ```json\n once the first newline arrives and trimming the trailing ``` at the end via a tail-buffer.
• Custom Transform: buffers the entire text block and applies Transform at text-end. Emits one text-delta with the transformed payload.

func (m *ExtractJsonMiddleware) WrapStream(ctx context.Context, options *stream.CallOptions, doStream StreamFunc) (<-chan stream.Event, error)

type ExtractReasoningMiddleware ¶

type ExtractReasoningMiddleware struct {
	BaseMiddleware

	// TagName is the XML tag enclosing reasoning content (e.g. "think").
	// Required.
	TagName string

	// Separator is inserted between consecutive reasoning or consecutive
	// text sections that span a tag boundary. Defaults to "\n".
	Separator string

	// StartWithReasoning treats the stream as beginning inside the
	// reasoning tag. Useful for models that omit the opening tag because
	// reasoning is guaranteed to come first.
	StartWithReasoning bool
}

ExtractReasoningMiddleware intercepts streamed text, splits <TagName>...</TagName> sections out as typed reasoning events, and emits the remaining text normally. Useful for models that embed reasoning inline as plain text with XML-style tags (DeepSeek-R1 via chat completions, some OSS reasoners, etc.) instead of native reasoning events.

Mirrors ai-sdk's extractReasoningMiddleware.

Implementation note: the buffer-scan pattern from ai-sdk is preserved — tags may arrive split across stream deltas, so we use getPotentialStartIndex to detect partial matches and wait for more bytes before deciding.

func (m *ExtractReasoningMiddleware) WrapStream(ctx context.Context, options *stream.CallOptions, doStream StreamFunc) (<-chan stream.Event, error)

type GenerateImageFunc ¶

type GenerateImageFunc func(ctx context.Context, options *model.ImageCallOptions) (*model.ImageResult, error)

GenerateImageFunc is the inner callable passed to WrapGenerate.

type ImageMiddleware ¶

type ImageMiddleware interface {
	TransformOptions(options *model.ImageCallOptions) *model.ImageCallOptions
	WrapGenerate(ctx context.Context, options *model.ImageCallOptions, doGenerate GenerateImageFunc) (*model.ImageResult, error)
}

ImageMiddleware mirrors ai-sdk's ImageModelMiddleware. Hooks run around Generate calls; nil implementations fall through.

type LoggingMiddleware ¶

type LoggingMiddleware struct {
	BaseMiddleware

	// OnCall is called when the model is invoked.
	OnCall func(modelID string, options *stream.CallOptions)

	// OnEvent is called for each stream event.
	OnEvent func(modelID string, event stream.Event)

	// OnError is called when an error occurs.
	OnError func(modelID string, err error)

	// ModelID stores the model ID for logging.
	ModelID string
}

LoggingMiddleware logs model calls for debugging.

func (l *LoggingMiddleware) WrapStream(ctx context.Context, options *stream.CallOptions, doStream StreamFunc) (<-chan stream.Event, error)

type Middleware ¶

type Middleware interface {
	// TransformOptions transforms the options before they're passed to the model.
	// Return nil to pass through unchanged.
	TransformOptions(options *stream.CallOptions) *stream.CallOptions

	// WrapStream wraps the stream operation.
	// If nil, the stream passes through unchanged.
	WrapStream(ctx context.Context, options *stream.CallOptions, doStream StreamFunc) (<-chan stream.Event, error)
}

Middleware defines the interface for language model middleware. Middleware can transform parameters and wrap stream operations.

type ProviderMiddlewares ¶

type ProviderMiddlewares struct {
	Language  []Middleware
	Embedding []EmbeddingMiddleware
	Image     []ImageMiddleware
}

ProviderMiddlewares bundles per-model-type middleware stacks that get applied to every model a wrapped provider hands out. Any field can be nil or empty; models without a corresponding middleware are returned unchanged.

Mirrors ai-sdk's wrapProvider({ provider, languageModelMiddleware, embeddingModelMiddleware, imageModelMiddleware }).

type RetryMiddleware ¶

type RetryMiddleware struct {
	BaseMiddleware

	// MaxRetries is the maximum number of retry attempts (default 3).
	MaxRetries int

	// ShouldRetry determines if an error should be retried.
	// Default: retry on any error.
	ShouldRetry func(error) bool
}

RetryMiddleware adds automatic retry on transient errors.

func (r *RetryMiddleware) WrapStream(ctx context.Context, options *stream.CallOptions, doStream StreamFunc) (<-chan stream.Event, error)

type SimulateStreamingMiddleware ¶

type SimulateStreamingMiddleware struct {
	BaseMiddleware
}

SimulateStreamingMiddleware consumes the inner stream in full, accumulates the contents, and re-emits them as a clean, coalesced stream — one TextDelta per text block, one ReasoningDelta per reasoning block, tool calls passed through in order.

Mirrors ai-sdk's simulateStreamingMiddleware. Its canonical use is wrapping providers that return all content at once; for providers that already stream natively, it's a no-op that still flattens per-token chunking into per-block chunks. Test fixtures use this to get deterministic event ordering.

func (s *SimulateStreamingMiddleware) WrapStream(ctx context.Context, options *stream.CallOptions, doStream StreamFunc) (<-chan stream.Event, error)

type StreamFunc ¶

type StreamFunc func(ctx context.Context, options *stream.CallOptions) (<-chan stream.Event, error)

StreamFunc is the function signature for stream operations.