Documentation
¶
Overview ¶
Package execution owns attempt state, retry and fallback budgets, and the response-byte commitment boundary for inference execution.
Index ¶
- Variables
- func CanFallback(providerFailure *failure.Failure) bool
- func IsStreamTerminal(err error) bool
- func WithAttemptAction(err error, action AttemptAction) error
- type AttemptAction
- type AttemptEvidence
- type Availability
- type ChatAttempt
- type ChatResult
- type Clock
- type Config
- type EmbeddingAttempt
- type EmbeddingResult
- type Error
- type Executor
- func (e *Executor) ExecuteChat(ctx context.Context, plan *routing.Plan, attempt ChatAttempt) (*ChatResult, error)
- func (e *Executor) ExecuteEmbedding(ctx context.Context, plan *routing.Plan, attempt EmbeddingAttempt) (*EmbeddingResult, error)
- func (e *Executor) StartChatStream(ctx context.Context, plan *routing.Plan, attempt StreamAttempt) (ManagedStream, error)
- type ManagedStream
- type State
- type Stream
- type StreamAttempt
- type Transition
Constants ¶
This section is empty.
Variables ¶
var ( // ErrPlanRequired reports a missing route plan. ErrPlanRequired = errors.New("route plan is required") // ErrAttemptRequired reports a missing provider attempt function. ErrAttemptRequired = errors.New("provider attempt function is required") // ErrAttemptBudget reports exhaustion of the total logical-attempt budget. ErrAttemptBudget = errors.New("logical attempt budget exhausted") // ErrElapsedBudget reports exhaustion of the total elapsed-time budget. ErrElapsedBudget = errors.New("execution elapsed-time budget exhausted") // ErrAllAttemptsFailed reports that no planned route completed. ErrAllAttemptsFailed = errors.New("all planned attempts failed") )
Functions ¶
func CanFallback ¶
CanFallback reports whether policy can move to the next planned route.
func IsStreamTerminal ¶
IsStreamTerminal reports stream completion without treating it as failure.
func WithAttemptAction ¶ added in v1.0.2
func WithAttemptAction(err error, action AttemptAction) error
WithAttemptAction annotates a stream read failure with execution policy. The wrapped error remains available through errors.Is and errors.As.
Types ¶
type AttemptAction ¶ added in v1.0.2
type AttemptAction uint8
AttemptAction tells the executor how an attempt failure affects the current route. The executor still applies the one total attempt budget.
const ( // AttemptActionDefault applies normal retry and route-fallback policy. AttemptActionDefault AttemptAction = iota // AttemptActionContinueRoute consumes another attempt on the same route // without changing provider-health state or applying retry backoff. AttemptActionContinueRoute // AttemptActionFallbackRoute moves directly to the next planned route // without changing provider-health state or applying retry policy. AttemptActionFallbackRoute // AttemptActionStop ends execution without changing provider-health state. AttemptActionStop )
type AttemptEvidence ¶
type AttemptEvidence struct {
Number int
Route routing.Route
Retry int
State State
StartedAt time.Time
FinishedAt time.Time
Duration time.Duration
Failure *failure.Failure
Transitions []Transition
}
AttemptEvidence records one provider invocation or availability skip.
type Availability ¶
type Availability interface {
Acquire(routing.Route) bool
Release(routing.Route)
RecordSuccess(routing.Route, time.Duration)
RecordFailure(routing.Route, *failure.Failure, time.Duration)
}
Availability owns attempt admission and offering outcome transitions.
type ChatAttempt ¶
type ChatAttempt func(context.Context, routing.Attempt) (*inference.ChatResponse, *failure.Failure, AttemptAction)
ChatAttempt makes one non-streaming provider invocation.
type ChatResult ¶
type ChatResult struct {
Response inference.ChatResponse
Route routing.Route
Attempts []AttemptEvidence
StartedAt time.Time
FinishedAt time.Time
}
ChatResult is one canonical completed result with execution evidence.
type Config ¶
type Config struct {
MaxAttempts int
MaxRetriesPerRoute int
MaxElapsed time.Duration
RetryBackoff time.Duration
BackoffMultiplier float64
MaxBackoff time.Duration
}
Config defines the one total execution budget.
func DefaultConfig ¶
func DefaultConfig() Config
DefaultConfig returns bounded production defaults. A retry and a fallback consume the same MaxAttempts budget.
type EmbeddingAttempt ¶ added in v1.0.2
type EmbeddingAttempt func(context.Context, routing.Attempt) (*inference.EmbeddingResponse, *failure.Failure, AttemptAction)
EmbeddingAttempt makes one non-streaming provider invocation.
type EmbeddingResult ¶ added in v1.0.2
type EmbeddingResult struct {
Response inference.EmbeddingResponse
Route routing.Route
Attempts []AttemptEvidence
StartedAt time.Time
FinishedAt time.Time
}
EmbeddingResult is one canonical completed embedding result with execution evidence.
type Error ¶
type Error struct {
Reason error
Failure *failure.Failure
Attempts []AttemptEvidence
}
Error reports terminal execution evidence and preserves the normalized failure.
type Executor ¶
type Executor struct {
// contains filtered or unexported fields
}
Executor applies one total attempt and elapsed-time budget to one route plan.
func New ¶
func New(config Config, clock Clock, availability Availability) (*Executor, error)
New creates an executor with explicit total-budget policy.
func (*Executor) ExecuteChat ¶
func (e *Executor) ExecuteChat( ctx context.Context, plan *routing.Plan, attempt ChatAttempt, ) (*ChatResult, error)
ExecuteChat executes one immutable plan for a non-streaming request.
func (*Executor) ExecuteEmbedding ¶ added in v1.0.2
func (e *Executor) ExecuteEmbedding( ctx context.Context, plan *routing.Plan, attempt EmbeddingAttempt, ) (*EmbeddingResult, error)
ExecuteEmbedding executes one immutable plan for an embedding request.
func (*Executor) StartChatStream ¶
func (e *Executor) StartChatStream( ctx context.Context, plan *routing.Plan, attempt StreamAttempt, ) (ManagedStream, error)
StartChatStream starts an execution-owned stream. It can retry or fall back only before it returns the first canonical event to its caller.
type ManagedStream ¶
type ManagedStream interface {
Stream
Attempts() []AttemptEvidence
Committed() bool
ModelUsed() string
}
ManagedStream exposes execution evidence without changing the protocol stream contract.
type State ¶
type State string
State is one state in the logical-attempt state machine.
const ( // StateQueued identifies an attempt that has not started. StateQueued State = "queued" // StateRunning identifies an active provider attempt. StateRunning State = "running" // StateSucceeded identifies a completed provider attempt. StateSucceeded State = "succeeded" // StateFailed identifies a provider attempt that returned a failure. StateFailed State = "failed" // StateSkipped identifies a route that availability policy rejected. StateSkipped State = "skipped" // StateCanceled identifies an attempt stopped by context cancellation. StateCanceled State = "canceled" )
type Stream ¶
type Stream interface {
Read() (*inference.StreamEvent, error)
Close() error
}
Stream is a provider-neutral inference event stream.
type StreamAttempt ¶
StreamAttempt starts one provider stream. Read failures should be normalized as *failure.Failure values. Wrap a pre-commit read error with WithAttemptAction when it must continue the same route.
Source Files