Documentation
¶
Overview ¶
Package middleware provides middleware types for wrapping AI SDK providers. Each domain (chat, embed, image, speech, transcribe, video, rerank, object) has its own middleware type and Chain function for composing cross-cutting concerns like telemetry, retries, circuit breakers, and custom transformations.
This is the Go equivalent of the AI SDK's provider middleware system.
Index ¶
- Variables
- func Chain[T any, M ~func(T) T](middlewares ...M) M
- func ChainGeneric[T any, M ~func(T) T](middlewares ...M) M
- func DefaultRetryableError(err error) bool
- type BackoffStrategy
- type ChatMiddleware
- type ChatRequestHook
- type ChatResponseHook
- type CircuitBreakerConfig
- type CircuitState
- type EmbedMiddleware
- type ExponentialBackoff
- type HealthChecker
- type ImageMiddleware
- type ObjectMiddleware
- type RerankMiddleware
- type RetryConfig
- type RetryableError
- type SpeechMiddleware
- type TelemetryEmbedMiddleware
- type TelemetryImageMiddleware
- type TelemetryMiddleware
- type TelemetryObjectMiddleware
- type TelemetryRerankMiddleware
- type TelemetrySpeechMiddleware
- type TelemetryTranscribeMiddleware
- type TelemetryVideoMiddleware
- type TranscribeMiddleware
- type VideoMiddleware
Constants ¶
This section is empty.
Variables ¶
var ErrCircuitOpen = errors.New("circuit breaker is open")
ErrCircuitOpen is returned when the circuit breaker rejects a call because it is in the OPEN state and the open-timeout has not yet expired.
var ErrHealthCheckFailed = errors.New("health check failed")
ErrHealthCheckFailed is returned when a provider fails its health check.
Functions ¶
func Chain ¶
func Chain[T any, M ~func(T) T](middlewares ...M) M
Chain composes middleware functions left-to-right and is a convenience wrapper over ChainGeneric with the same semantics. Call sites in the repo use Chain[T](...) and rely on type inference for the middleware type M.
func ChainGeneric ¶
func ChainGeneric[T any, M ~func(T) T](middlewares ...M) M
ChainGeneric composes middleware functions left-to-right. The first middleware becomes the outermost wrapper. ChainGeneric is generic over the provider type T and accepts middleware defined as a named function type M whose underlying type is func(T) T (for example "type EmbedMiddleware func(embed.Provider) embed.Provider").
func DefaultRetryableError ¶
DefaultRetryableError retries on provider-unavailable, rate-limiting, and temporary network errors. It does NOT retry auth failures or invalid requests (which would always fail on retry).
The function matches against the shared errx sentinels and falls back to substring matching in the error message for common transient keywords when the error cannot be unwrapped to a known sentinel.
Types ¶
type BackoffStrategy ¶
BackoffStrategy calculates the delay before a retry attempt.
type ChatMiddleware ¶
ChatMiddleware wraps a chat.Provider to intercept and potentially modify requests and responses. Middleware can be stacked to compose behaviour.
Example (logging middleware):
func LoggingMiddleware(next chat.Provider) chat.Provider {
return &loggingProvider{next: next}
}
func ChainChat ¶
func ChainChat(ms ...ChatMiddleware) ChatMiddleware
ChainChat composes ChatMiddleware slices using the generic ChainGeneric.
func CircuitBreakerChat ¶
func CircuitBreakerChat(cfg CircuitBreakerConfig) ChatMiddleware
CircuitBreakerChat returns a ChatMiddleware that wraps the provider with a circuit breaker. Each invocation of the returned middleware creates an independent circuitBreaker instance so that different provider chains do not share failure/success counters.
func HealthCheckChat ¶
func HealthCheckChat() ChatMiddleware
HealthCheckChat wraps a chat.Provider with a pre-call health probe. If the provider implements HealthChecker, HealthCheck is called before each Chat/ChatStream call. If the health check fails, the call is short-circuited with ErrHealthCheckFailed.
func RetryChat ¶
func RetryChat(cfg RetryConfig, backoff BackoffStrategy, retryable RetryableError) ChatMiddleware
RetryChat returns a ChatMiddleware that retries failed Chat and ChatStream calls according to cfg, using backoff for delay and retryable to decide whether an error is transient.
ChatStream retries only on stream-creation failure; mid-stream errors are NOT retried.
type ChatRequestHook ¶
ChatRequestHook is called before a chat request is sent. It may modify the request or return an error to short-circuit.
type ChatResponseHook ¶
ChatResponseHook is called after a chat response is received. It may modify the response or record metrics.
type CircuitBreakerConfig ¶
type CircuitBreakerConfig struct {
// FailureThreshold is the number of consecutive failures in CLOSED state
// required to trip the breaker to OPEN.
FailureThreshold uint32
// SuccessThreshold is the number of consecutive successes in HALF_OPEN
// state required to transition back to CLOSED.
SuccessThreshold uint32
// OpenTimeout is the duration the breaker stays OPEN before
// automatically transitioning to HALF_OPEN to probe recovery.
OpenTimeout time.Duration
}
CircuitBreakerConfig controls the circuit breaker state machine behaviour. All fields are required; zero values will cause the breaker to never trip (FailureThreshold=0) or never recover (SuccessThreshold=0, OpenTimeout=0).
type CircuitState ¶
type CircuitState int
CircuitState represents the three possible states of the circuit breaker state machine: CLOSED (normal operation), OPEN (fail-fast with timeout), and HALF_OPEN (limited probing to test recovery).
const ( CircuitClosed CircuitState = iota CircuitHalfOpen CircuitOpen )
type EmbedMiddleware ¶
EmbedMiddleware wraps an embed.Provider to intercept and potentially modify calls. Middleware can be stacked to compose behaviour.
func ChainEmbed ¶
func ChainEmbed(ms ...EmbedMiddleware) EmbedMiddleware
ChainEmbed composes multiple EmbedMiddleware into a single middleware. It uses the generic Chain function from chain.go.
func CircuitBreakerEmbed ¶
func CircuitBreakerEmbed(cfg CircuitBreakerConfig) EmbedMiddleware
CircuitBreakerEmbed returns an EmbedMiddleware that wraps the provider with a circuit breaker. Each invocation of the returned middleware creates an independent circuitBreaker instance.
func RetryEmbed ¶
func RetryEmbed(cfg RetryConfig, backoff BackoffStrategy, retryable RetryableError) EmbedMiddleware
type ExponentialBackoff ¶
type ExponentialBackoff struct {
// BaseDelay is the initial delay for the first retry attempt (attempt 0).
// Defaults to 1s.
BaseDelay time.Duration
// MaxDelay caps the exponential backoff. Defaults to 30s.
MaxDelay time.Duration
// Multiplier is the exponential growth factor. Defaults to 2.0.
Multiplier float64
// Jitter is the full-jitter factor (0.0–1.0). Defaults to 0.5.
Jitter float64
}
ExponentialBackoff implements exponential backoff with full jitter.
Formula: min(base * multiplier^attempt, max) * (1 + jitter*(rand*2-1))
Zero-value fields are safe: a zero Backoff uses BaseDelay=1s, MaxDelay=30s, Multiplier=2, Jitter=0.5.
type HealthChecker ¶
HealthChecker is implemented by providers that support health probes.
type ImageMiddleware ¶
ImageMiddleware wraps an image.Provider to intercept and potentially modify calls. Middleware can be stacked to compose behaviour.
func ChainImage ¶
func ChainImage(ms ...ImageMiddleware) ImageMiddleware
ChainImage composes multiple ImageMiddleware into a single middleware. It uses the generic Chain function from chain.go.
func CircuitBreakerImage ¶
func CircuitBreakerImage(cfg CircuitBreakerConfig) ImageMiddleware
CircuitBreakerImage returns an ImageMiddleware that wraps the provider with a circuit breaker. Each invocation of the returned middleware creates an independent circuitBreaker instance.
func RetryImage ¶
func RetryImage(cfg RetryConfig, backoff BackoffStrategy, retryable RetryableError) ImageMiddleware
type ObjectMiddleware ¶
ObjectMiddleware wraps an object.Provider to intercept and potentially modify calls. Middleware can be stacked to compose behaviour.
func ChainObject ¶
func ChainObject(ms ...ObjectMiddleware) ObjectMiddleware
ChainObject composes multiple ObjectMiddleware into a single middleware. It uses the generic Chain function from chain.go.
func CircuitBreakerObject ¶
func CircuitBreakerObject(cfg CircuitBreakerConfig) ObjectMiddleware
CircuitBreakerObject returns an ObjectMiddleware that wraps the provider with a circuit breaker. Each invocation of the returned middleware creates an independent circuitBreaker instance.
func RetryObject ¶
func RetryObject(cfg RetryConfig, backoff BackoffStrategy, retryable RetryableError) ObjectMiddleware
type RerankMiddleware ¶
RerankMiddleware wraps a rerank.Provider to intercept and potentially modify calls. Middleware can be stacked to compose behaviour.
func ChainRerank ¶
func ChainRerank(ms ...RerankMiddleware) RerankMiddleware
ChainRerank composes multiple RerankMiddleware into a single middleware. It uses the generic Chain function from chain.go.
func CircuitBreakerRerank ¶
func CircuitBreakerRerank(cfg CircuitBreakerConfig) RerankMiddleware
CircuitBreakerRerank returns a RerankMiddleware that wraps the provider with a circuit breaker. Each invocation of the returned middleware creates an independent circuitBreaker instance.
func RetryRerank ¶
func RetryRerank(cfg RetryConfig, backoff BackoffStrategy, retryable RetryableError) RerankMiddleware
type RetryConfig ¶
type RetryConfig struct {
// MaxAttempts is the total number of attempts (1 = no retries).
MaxAttempts int
}
RetryConfig controls retry behaviour.
type RetryableError ¶
RetryableError returns true if the error should trigger a retry.
type SpeechMiddleware ¶
SpeechMiddleware wraps a speech.Provider to intercept and potentially modify calls. Middleware can be stacked to compose behaviour.
func ChainSpeech ¶
func ChainSpeech(ms ...SpeechMiddleware) SpeechMiddleware
ChainSpeech composes multiple SpeechMiddleware into a single middleware. It uses the generic Chain function from chain.go.
func CircuitBreakerSpeech ¶
func CircuitBreakerSpeech(cfg CircuitBreakerConfig) SpeechMiddleware
CircuitBreakerSpeech returns a SpeechMiddleware that wraps the provider with a circuit breaker. Each invocation of the returned middleware creates an independent circuitBreaker instance.
func RetrySpeech ¶
func RetrySpeech(cfg RetryConfig, backoff BackoffStrategy, retryable RetryableError) SpeechMiddleware
type TelemetryEmbedMiddleware ¶
type TelemetryEmbedMiddleware struct {
// contains filtered or unexported fields
}
TelemetryEmbedMiddleware wraps an embed.Provider with OpenTelemetry-compatible tracing. Each Embed call creates a span that records the provider name, model, and input count as attributes. Errors are recorded on the span before it ends.
func NewTelemetryEmbedMiddleware ¶
func NewTelemetryEmbedMiddleware(next embed.Provider, tracer telemetry.Tracer) *TelemetryEmbedMiddleware
NewTelemetryEmbedMiddleware creates a new telemetry middleware that wraps the given embed provider with tracing.
func (*TelemetryEmbedMiddleware) Embed ¶
func (t *TelemetryEmbedMiddleware) Embed(ctx context.Context, req embed.Request) (embed.Response, error)
Embed performs an embedding request wrapped in a span.
func (*TelemetryEmbedMiddleware) Name ¶
func (t *TelemetryEmbedMiddleware) Name() string
Name returns the name of the underlying provider.
type TelemetryImageMiddleware ¶
type TelemetryImageMiddleware struct {
// contains filtered or unexported fields
}
TelemetryImageMiddleware wraps an image.Provider with OpenTelemetry-compatible tracing. Each GenerateImage call creates a span that records the provider name, model, and image count as attributes. Errors are recorded on the span before it ends.
func NewTelemetryImageMiddleware ¶
func NewTelemetryImageMiddleware(next image.Provider, tracer telemetry.Tracer) *TelemetryImageMiddleware
NewTelemetryImageMiddleware creates a new telemetry middleware that wraps the given image provider with tracing.
func (*TelemetryImageMiddleware) GenerateImage ¶
func (t *TelemetryImageMiddleware) GenerateImage(ctx context.Context, req image.GenerateImageRequest) (image.GenerateImageResponse, error)
GenerateImage performs an image generation request wrapped in a span.
func (*TelemetryImageMiddleware) Name ¶
func (t *TelemetryImageMiddleware) Name() string
Name returns the name of the underlying provider.
type TelemetryMiddleware ¶
type TelemetryMiddleware struct {
// contains filtered or unexported fields
}
TelemetryMiddleware wraps a chat.Provider with OpenTelemetry-compatible tracing. Each Chat or ChatStream call creates a span that records the provider name, model, message count, and tool count as attributes. Errors are recorded on the span before it ends.
For ChatStream, the span is started at ChatStream time and ended when the returned stream is Closed; errors from Next() are recorded on the span but do not end it prematurely.
func NewTelemetryMiddleware ¶
func NewTelemetryMiddleware(next chat.Provider, tracer telemetry.Tracer) *TelemetryMiddleware
NewTelemetryMiddleware creates a new telemetry middleware that wraps the given provider with tracing.
func (*TelemetryMiddleware) ChatStream ¶
func (t *TelemetryMiddleware) ChatStream(ctx context.Context, req chat.Request) (chat.Stream, error)
ChatStream performs a streaming chat completion. The span is started at call time and ended when the returned stream is Closed.
func (*TelemetryMiddleware) Name ¶
func (t *TelemetryMiddleware) Name() string
Name returns the name of the underlying provider.
type TelemetryObjectMiddleware ¶
type TelemetryObjectMiddleware struct {
// contains filtered or unexported fields
}
TelemetryObjectMiddleware wraps an object.Provider with OpenTelemetry-compatible tracing. Each GenerateObject or StreamObject call creates a span that records the provider name, model, and max tokens as attributes. Errors are recorded on the span before it ends.
For StreamObject, the span is started at StreamObject time and ended when the returned stream is Closed; errors from Next() are recorded on the span but do not end it prematurely.
func NewTelemetryObjectMiddleware ¶
func NewTelemetryObjectMiddleware(next object.Provider, tracer telemetry.Tracer) *TelemetryObjectMiddleware
NewTelemetryObjectMiddleware creates a new telemetry middleware that wraps the given object provider with tracing.
func (*TelemetryObjectMiddleware) GenerateObject ¶
func (t *TelemetryObjectMiddleware) GenerateObject(ctx context.Context, req object.Request) (object.ObjectResult, error)
GenerateObject performs a non-streaming object generation wrapped in a span.
func (*TelemetryObjectMiddleware) Name ¶
func (t *TelemetryObjectMiddleware) Name() string
Name returns the name of the underlying provider.
func (*TelemetryObjectMiddleware) StreamObject ¶
func (t *TelemetryObjectMiddleware) StreamObject(ctx context.Context, req object.Request) (object.ObjectStream, error)
StreamObject performs a streaming object generation. The span is started at call time and ended when the returned stream is Closed.
type TelemetryRerankMiddleware ¶
type TelemetryRerankMiddleware struct {
// contains filtered or unexported fields
}
TelemetryRerankMiddleware wraps a rerank.Provider with OpenTelemetry-compatible tracing. Each Rerank call creates a span that records the provider name, model, document count, and top-n as attributes. Errors are recorded on the span before it ends.
func NewTelemetryRerankMiddleware ¶
func NewTelemetryRerankMiddleware(next rerank.Provider, tracer telemetry.Tracer) *TelemetryRerankMiddleware
NewTelemetryRerankMiddleware creates a new telemetry middleware that wraps the given rerank provider with tracing.
func (*TelemetryRerankMiddleware) Name ¶
func (t *TelemetryRerankMiddleware) Name() string
Name returns the name of the underlying provider.
type TelemetrySpeechMiddleware ¶
type TelemetrySpeechMiddleware struct {
// contains filtered or unexported fields
}
TelemetrySpeechMiddleware wraps a speech.Provider with OpenTelemetry-compatible tracing. Each GenerateSpeech call creates a span that records the provider name, model, and voice as attributes. Errors are recorded on the span before it ends.
func NewTelemetrySpeechMiddleware ¶
func NewTelemetrySpeechMiddleware(next speech.Provider, tracer telemetry.Tracer) *TelemetrySpeechMiddleware
NewTelemetrySpeechMiddleware creates a new telemetry middleware that wraps the given speech provider with tracing.
func (*TelemetrySpeechMiddleware) GenerateSpeech ¶
func (t *TelemetrySpeechMiddleware) GenerateSpeech(ctx context.Context, req speech.GenerateSpeechRequest) (speech.GenerateSpeechResponse, error)
GenerateSpeech performs a speech generation request wrapped in a span.
func (*TelemetrySpeechMiddleware) Name ¶
func (t *TelemetrySpeechMiddleware) Name() string
Name returns the name of the underlying provider.
type TelemetryTranscribeMiddleware ¶
type TelemetryTranscribeMiddleware struct {
// contains filtered or unexported fields
}
TelemetryTranscribeMiddleware wraps a transcribe.Provider with OpenTelemetry-compatible tracing. Each Transcribe call creates a span that records the provider name, model, and language as attributes. Errors are recorded on the span before it ends.
func NewTelemetryTranscribeMiddleware ¶
func NewTelemetryTranscribeMiddleware(next transcribe.Provider, tracer telemetry.Tracer) *TelemetryTranscribeMiddleware
NewTelemetryTranscribeMiddleware creates a new telemetry middleware that wraps the given transcribe provider with tracing.
func (*TelemetryTranscribeMiddleware) Name ¶
func (t *TelemetryTranscribeMiddleware) Name() string
Name returns the name of the underlying provider.
func (*TelemetryTranscribeMiddleware) Transcribe ¶
func (t *TelemetryTranscribeMiddleware) Transcribe(ctx context.Context, req transcribe.TranscribeRequest) (transcribe.TranscribeResponse, error)
Transcribe performs a transcription request wrapped in a span.
type TelemetryVideoMiddleware ¶
type TelemetryVideoMiddleware struct {
// contains filtered or unexported fields
}
TelemetryVideoMiddleware wraps a video.Provider with OpenTelemetry-compatible tracing. Each GenerateVideo call creates a span that records the provider name, model, resolution, and frame rate as attributes. Errors are recorded on the span before it ends.
func NewTelemetryVideoMiddleware ¶
func NewTelemetryVideoMiddleware(next video.Provider, tracer telemetry.Tracer) *TelemetryVideoMiddleware
NewTelemetryVideoMiddleware creates a new telemetry middleware that wraps the given video provider with tracing.
func (*TelemetryVideoMiddleware) GenerateVideo ¶
func (t *TelemetryVideoMiddleware) GenerateVideo(ctx context.Context, req video.GenerateVideoRequest) (video.GenerateVideoResponse, error)
GenerateVideo performs a video generation request wrapped in a span.
func (*TelemetryVideoMiddleware) Name ¶
func (t *TelemetryVideoMiddleware) Name() string
Name returns the name of the underlying provider.
type TranscribeMiddleware ¶
type TranscribeMiddleware func(transcribe.Provider) transcribe.Provider
TranscribeMiddleware wraps a transcribe.Provider to intercept and potentially modify calls. Middleware can be stacked to compose behaviour.
func ChainTranscribe ¶
func ChainTranscribe(ms ...TranscribeMiddleware) TranscribeMiddleware
ChainTranscribe composes multiple TranscribeMiddleware into a single middleware. It uses the generic Chain function from chain.go.
func CircuitBreakerTranscribe ¶
func CircuitBreakerTranscribe(cfg CircuitBreakerConfig) TranscribeMiddleware
CircuitBreakerTranscribe returns a TranscribeMiddleware that wraps the provider with a circuit breaker. Each invocation of the returned middleware creates an independent circuitBreaker instance.
func RetryTranscribe ¶
func RetryTranscribe(cfg RetryConfig, backoff BackoffStrategy, retryable RetryableError) TranscribeMiddleware
type VideoMiddleware ¶
VideoMiddleware wraps a video.Provider to intercept and potentially modify calls. Middleware can be stacked to compose behaviour.
func ChainVideo ¶
func ChainVideo(ms ...VideoMiddleware) VideoMiddleware
ChainVideo composes multiple VideoMiddleware into a single middleware. It uses the generic Chain function from chain.go.
func CircuitBreakerVideo ¶
func CircuitBreakerVideo(cfg CircuitBreakerConfig) VideoMiddleware
CircuitBreakerVideo returns a VideoMiddleware that wraps the provider with a circuit breaker. Each invocation of the returned middleware creates an independent circuitBreaker instance.
func RetryVideo ¶
func RetryVideo(cfg RetryConfig, backoff BackoffStrategy, retryable RetryableError) VideoMiddleware
Source Files
¶
- chain.go
- circuitbreaker.go
- circuitbreaker_chat.go
- circuitbreaker_embed.go
- circuitbreaker_image.go
- circuitbreaker_object.go
- circuitbreaker_rerank.go
- circuitbreaker_speech.go
- circuitbreaker_transcribe.go
- circuitbreaker_video.go
- doc.go
- embed.go
- health.go
- image.go
- middleware.go
- object.go
- rerank.go
- retry.go
- retry_chat.go
- retry_embed.go
- retry_image.go
- retry_object.go
- retry_rerank.go
- retry_speech.go
- retry_transcribe.go
- retry_video.go
- speech.go
- telemetry.go
- telemetry_embed.go
- telemetry_image.go
- telemetry_object.go
- telemetry_rerank.go
- telemetry_speech.go
- telemetry_transcribe.go
- telemetry_video.go
- transcribe.go
- video.go