llmresilience

package
v0.0.21 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 28, 2026 License: Apache-2.0 Imports: 16 Imported by: 0

Documentation

Overview

Package llmresilience provides a harness-level resilience decorator around any port.LLMProvider. It adds bounded retries with exponential backoff and jitter, a consecutive-failure circuit breaker, per-attempt timeouts, and pluggable error classification.

The load-bearing correctness rule is no replay after semantic visibility. Leading whitespace-only text, reasoning/replay metadata, phase, provider route, and tool calls are tentative and remain buffered in wire order. Usage is also buffered but is accounting, not semantic visibility: discarded attempts add it to the eventual success or terminal error without exposing their content. The first text delta that makes cumulative text non-whitespace flushes the semantic buffer and commits the attempt; after it escapes, a failure is terminal. A clean ChunkDone instead flushes the whole tentative turn, including pure-tool-call and whitespace-only turns. A retryable failure before either boundary discards the tentative attempt and may be replayed within the configured attempt limit.

Establishment ends on the first RAW chunk, independently of semantic progress. Subsequent raw chunk activity resets StreamIdleTimeout even while all chunks remain tentative, so active reasoning or tool assembly cannot trip either watchdog merely because it is not yet model-visible.

The breaker counts consecutive TRANSIENT establishment failures across calls (HTTP 429/408/5xx, network errors, per-attempt timeouts — see isTransientForBreaker). Permanent client errors (4xx other than 408/429, e.g. a policy-blocked or unavailable model returning 400/403/404) and caller cancellations are breaker-neutral: they neither open the breaker nor reset it. After BreakerThreshold transient failures it opens and Stream fails fast with *BreakerError for BreakerCooldown; it then half-opens to admit a single trial. Any success resets it. All breaker state is concurrency-safe.

The package depends only on the standard library and engine/port; the classifier reaches *openai.Error via errors.As to read its StatusCode, which is acceptable for an adapter.

Index

Constants

This section is empty.

Variables

View Source
var ErrCredentials = errors.New("credential unavailable")

ErrCredentials marks a failure to MINT OR LOAD the credential a request needs, as distinct from a failure of the provider that would have served it. A transport that resolves a token per request (the direct-mode bearer RoundTripper) wraps its token-source failures with it.

It exists because the two are otherwise indistinguishable here. net/http wraps ANY error a RoundTripper returns in *url.Error, and *url.Error carries Timeout()/Temporary() so it satisfies net.Error — meaning "I could not get a token" arrives looking exactly like "the network flaked". Both classifiers below would then call it retryable AND provider-unhealthy, so the one error the user can act on gets retried MaxAttempts times, counted toward the shared breaker, and finally replaced by the breaker's own error. Wrapping with this sentinel routes the failure to the permanent, breaker-neutral path instead, so the real cause reaches the caller on the first attempt.

Functions

func DefaultClassifier

func DefaultClassifier(err error) bool

DefaultClassifier is the legacy compatibility retry-policy projection of the provider-neutral tri-state classification. It returns true only for causally retryable failures. Config.Classifier is invoked after that classification as a tighten-only veto, so neither this function nor a custom classifier can upgrade unknown or permanent failures in Stream.

func DefaultDisposition

func DefaultDisposition(err error) port.RetryDisposition

DefaultDisposition classifies the causal provider failure without deciding whether policy permits another attempt. Unknown is conservative and is never promoted to permanent.

func Wrap

func Wrap(inner port.LLMProvider, cfg Config) port.LLMProvider

Wrap decorates inner with the resilience behaviour described by cfg and returns a port.LLMProvider. The returned provider is safe for concurrent use.

Types

type BreakerError

type BreakerError struct {
	// RetryAfter is how long until the breaker half-opens.
	RetryAfter time.Duration
}

BreakerError is returned by Stream while the circuit breaker is open. It carries the time at which the breaker is next eligible to half-open so callers can surface a meaningful terminal error.

func (*BreakerError) Error

func (e *BreakerError) Error() string

type Config

type Config struct {
	// MaxAttempts is the total number of attempts for establishing the stream
	// (the initial call plus retries). Values < 1 are treated as 1.
	MaxAttempts int
	// BaseBackoff is the backoff before the first retry; it grows exponentially.
	BaseBackoff time.Duration
	// MaxBackoff caps the per-attempt backoff. 0 means no cap.
	MaxBackoff time.Duration
	// PerAttemptTimeout bounds each attempt's establishment (connect + first raw
	// chunk). 0 disables it. It never overrides a shorter caller deadline.
	PerAttemptTimeout time.Duration
	// StreamIdleTimeout bounds the gap between consecutive raw chunks AFTER the
	// first chunk has been observed. 0 disables it. A longer stall is retryable
	// while semantic progress remains precommit and terminal after visible output.
	StreamIdleTimeout time.Duration
	// BreakerThreshold is the number of consecutive failed attempts that opens
	// the breaker. Values < 1 disable the breaker.
	BreakerThreshold int
	// BreakerCooldown is how long the breaker stays open before half-opening.
	BreakerCooldown time.Duration
	// Classifier is a legacy tighten-only retry-policy veto. Typed causal
	// classification runs first; Classifier is consulted only for errors already
	// classified retryable and can refuse another attempt. It cannot upgrade an
	// unknown or permanent error. nil selects DefaultClassifier.
	Classifier func(error) bool
	// Clock returns the current time; injectable for tests. nil selects
	// time.Now.
	Clock func() time.Time
	// Diagnostics is the optional operational-logging sink for stream-lifecycle
	// events (retries, per-attempt timeouts, idle stalls, breaker transitions,
	// exhaustion). nil selects port.NopDiagnostics. This is an ADAPTER seam — it is
	// NOT the loop's run-scoped sink and so is NOT subject to the loop's three-line
	// budget (see docs/adr/0020-diagnostics.md): the wrapper is per-provider and
	// logs provider-level lifecycle. It sees only port.LLMRequest + errors, never
	// prompt text, so every emitted record is metadata-only.
	Diagnostics port.Diagnostics
}

Config tunes the resilience decorator. The zero value is usable but inert (MaxAttempts <= 1 means a single attempt, no breaker); supply sensible values via Wrap.

type ExhaustedError

type ExhaustedError struct {
	// Attempts is how many attempts were made.
	Attempts int
	// Err is the final underlying error.
	Err error
	// PerAttempt is the per-attempt establishment budget that was in force, used to
	// name the timeout duration in the operator-facing message. 0 when unset.
	PerAttempt time.Duration
}

ExhaustedError is returned when every attempt to establish the stream failed. It wraps the last underlying error.

func (*ExhaustedError) Error

func (e *ExhaustedError) Error() string

Error renders an OPERATOR-FACING message (issue #82): it surfaces verbatim to the TUI footer / transcript, so it avoids the internal package prefix and the raw inner sentinel, says in plain language what went wrong, and names the next action. The first-chunk timeout (the model connected but never started responding within the per-attempt budget — errFirstChunkTimeout) gets a tailored message; any other exhausted cause falls back to a generic but still prefix-free, action-bearing line.

func (*ExhaustedError) Unwrap

func (e *ExhaustedError) Unwrap() error

Unwrap exposes the final underlying error to errors.Is/As.

type StreamIdleError

type StreamIdleError struct {
	// Idle is the configured idle budget that elapsed without a chunk.
	Idle time.Duration
}

StreamIdleError is synthesized when no raw chunk arrives within StreamIdleTimeout after activity began. The wrapper must synthesize it because providers may swallow the context error used to unblock their stream. It is a retryable transport failure while the semantic attempt is precommit, and terminal after visible output has escaped.

func (*StreamIdleError) Error

func (e *StreamIdleError) Error() string

func (*StreamIdleError) Unwrap

func (*StreamIdleError) Unwrap() error

Unwrap returns context.DeadlineExceeded so errors.Is(err, context.DeadlineExceeded) holds, classifying the stall as a deadline (not a caller cancel).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL