README
¶
Scheduling Pickers
Scheduling Pickers represent the final phase of the scheduling cycle in the Gateway API Inference Extension. After candidate endpoints have been filtered and scored by preceding plugins, the Picker is responsible for selecting the final subset of endpoints (typically just one) to receive the request.
The framework provides three standard picker implementations:
All pickers share a common configuration structure and accept the maxNumOfEndpoints parameter.
[!NOTE] If
maxNumOfEndpointsis configured to be greater than1, the EPP will join all selected endpoints into a comma-separated string for the routing layer (e.g., assigned toTargetEndpoint). However, the framework's internal tracking for post-scheduling plugins (like response handlers) will only reference the first endpoint in the list.
Documentation
¶
Overview ¶
Package picker provides the standard implementations of the Picker interface for the scheduling framework. These plugins represent the final phase of the scheduling cycle, selecting target endpoints from the scored candidates.
For detailed descriptions of the available pickers and their configuration, see the package README.md.
Index ¶
Constants ¶
const ( // DefaultMaxNumOfEndpoints is the fallback maximum number of endpoints to pick if not specified // in the configuration. DefaultMaxNumOfEndpoints = 1 )
Variables ¶
var PickerRand = newLockedRand()
PickerRand is a thread-safe random number generator shared by all pickers to avoid seeding overhead and ensure controlled randomization.
Functions ¶
func ShuffleScoredEndpoints ¶
func ShuffleScoredEndpoints(scoredEndpoints []*fwksched.ScoredEndpoint)
ShuffleScoredEndpoints randomizes the order of the given scored candidates in-place.
Types ¶
type PickerParameters ¶
type PickerParameters struct {
MaxNumOfEndpoints int `json:"maxNumOfEndpoints"`
}
PickerParameters defines the common parameters for all pickers
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package maxscore implements a scheduling picker that selects the endpoint(s) with the highest score calculated during the scoring phase.
|
Package maxscore implements a scheduling picker that selects the endpoint(s) with the highest score calculated during the scoring phase. |
|
Package random implements a scheduling picker that selects endpoints uniformly at random, ignoring any scores calculated by scorer plugins.
|
Package random implements a scheduling picker that selects endpoints uniformly at random, ignoring any scores calculated by scorer plugins. |
|
Package weightedrandom implements a scheduling picker that selects endpoints randomly, where the probability of an endpoint being selected is proportional to its score.
|
Package weightedrandom implements a scheduling picker that selects endpoints randomly, where the probability of an endpoint being selected is proportional to its score. |