Documentation
¶
Overview ¶
Package routeexample is the built-in cheapest_healthy routing strategy: it sends a virtual model's traffic to the cheapest (or fastest) target whose recent error rate is acceptable, learning target health from the attempt outcomes GoModel reports. It doubles as the reference implementation of a pluginapi.RouteStrategy plugin.
Index ¶
- Constants
- func New() pluginapi.Plugin
- type Plugin
- func (p *Plugin) Close(context.Context) error
- func (p *Plugin) Init(context.Context, json.RawMessage, pluginapi.Host) error
- func (p *Plugin) Manifest() pluginapi.Manifest
- func (p *Plugin) OnAttemptEnd(outcome pluginapi.RouteOutcome)
- func (p *Plugin) Select(_ context.Context, req pluginapi.RouteRequest) (pluginapi.RouteChoice, error)
Constants ¶
const ( // PreferCheapest picks the healthy candidate with the lowest input price. PreferCheapest = "cheapest" // PreferFastest picks the healthy candidate with the lowest median latency. PreferFastest = "fastest" // DefaultMaxErrorRate is the error-rate ceiling above which a target is // treated as unhealthy. DefaultMaxErrorRate = 0.2 // WindowSize is how many recent outcomes per target are remembered. WindowSize = 50 )
const Name = "cheapest_healthy"
Name is the manifest name of the plugin.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Plugin ¶
type Plugin struct {
// contains filtered or unexported fields
}
Plugin is one cheapest_healthy instance. Health is tracked per target across every virtual model that uses the instance.
func (*Plugin) OnAttemptEnd ¶
func (p *Plugin) OnAttemptEnd(outcome pluginapi.RouteOutcome)
OnAttemptEnd records the outcome in the target's sliding window.
func (*Plugin) Select ¶
func (p *Plugin) Select(_ context.Context, req pluginapi.RouteRequest) (pluginapi.RouteChoice, error)
Select keeps a healthy session target, otherwise picks among the healthy candidates by price or latency. When every candidate is unhealthy it chooses among all of them, so the strategy fails open rather than declining.
Source Files