README
¶
Weighted Random Picker
Type: weighted-random-picker
Selects endpoint(s) randomly, where the probability of an endpoint being selected is proportional to its score.
What it does
- Receives a list of
ScoredEndpointcandidates. - If all candidates have a score of zero or less, it delegates to the
random-pickerfor uniform selection. - Uses the A-Res (Algorithm for Reservoir Sampling) algorithm to perform mathematically correct weighted random sampling.
- Generates a random key for each endpoint based on its score: $key_i = U_i^{(1/w_i)}$ where $U_i$ is a random number in $(0,1)$ and $w_i$ is the endpoint's score.
- Selects the candidates with the largest keys.
- Returns the top
maxNumOfEndpointscandidates.
Behavioral Intent
This picker resolves the trade-off between max-score-picker and random-picker. It prefers higher-scoring endpoints while maintaining exploration and avoiding extreme hot-spotting. It is ideal when paired with scorers that produce continuous scores (e.g., Token Load or Latency).
Inputs consumed
- Consumes the list of
ScoredEndpointresults and utilizes theScorevalue.
Configuration
The plugin config supports:
maxNumOfEndpoints(default 1)- The maximum number of endpoints to pick and return. Must be > 0. If more candidates are available than this limit, only the top subset is returned.
[!TIP] In most production scenarios,
maxNumOfEndpointsis left at its default value of1to select a single target endpoint for the request.
Documentation
¶
Overview ¶
Package weightedrandom implements a scheduling picker that selects endpoints randomly, where the probability of an endpoint being selected is proportional to its score.
For detailed behavioral intent and configuration, see the package README.
Index ¶
Constants ¶
const (
// WeightedRandomPickerType is the registered name of the weighted random picker plugin.
WeightedRandomPickerType = "weighted-random-picker"
)
Variables ¶
This section is empty.
Functions ¶
Types ¶
type WeightedRandomPicker ¶
type WeightedRandomPicker struct {
// contains filtered or unexported fields
}
WeightedRandomPicker picks endpoint(s) from the list of candidates based on weighted random sampling using A-Res algorithm. Reference: https://utopia.duth.gr/~pefraimi/research/data/2007EncOfAlg.pdf.
The picker at its core is picking endpoints randomly, where the probability of the endpoint to get picked is derived from its weighted score. Algorithm: - Uses A-Res (Algorithm for Reservoir Sampling): keyᵢ = Uᵢ^(1/wᵢ) - Selects k items with largest keys for mathematically correct weighted sampling - More efficient than traditional cumulative probability approach
Key characteristics: - Mathematically correct weighted random sampling - Single pass algorithm with O(n + k log k) complexity
func NewWeightedRandomPicker ¶
func NewWeightedRandomPicker(maxNumOfEndpoints int) *WeightedRandomPicker
NewWeightedRandomPicker initializes a new WeightedRandomPicker and returns its pointer.
func (*WeightedRandomPicker) Pick ¶
func (p *WeightedRandomPicker) Pick(ctx context.Context, scoredEndpoints []*fwksched.ScoredEndpoint) *fwksched.ProfileRunResult
Pick selects the endpoint(s) randomly from the list of candidates, where the probability of the endpoint to get picked is derived from its weighted score.
func (*WeightedRandomPicker) TypedName ¶
func (p *WeightedRandomPicker) TypedName() fwkplugin.TypedName
TypedName returns the type and name tuple of this plugin instance.
func (*WeightedRandomPicker) WithName ¶
func (p *WeightedRandomPicker) WithName(name string) *WeightedRandomPicker
WithName sets the name of the picker.
Source Files