README
¶
Max Score Picker
Type: max-score-picker
Selects the endpoint(s) with the highest score calculated during the scoring phase.
[!NOTE] This plugin is enabled by default if no other picker is specified. You do not need to explicitly declare it in your configuration.
What it does
- Receives a list of
ScoredEndpointcandidates. - Shuffles the list in-place to ensure random tie-breaking when multiple endpoints share the same maximum score.
- Sorts the candidates by score in descending order.
- Returns the top
maxNumOfEndpointscandidates.
Behavioral Intent
This picker maximizes the adherence to scoring objectives (e.g., cache affinity, lowest load). However, it is susceptible to hot-spotting if many concurrent requests produce identical scores for the same endpoint (e.g., identical prompts targeting a specific cache hit).
Inputs consumed
- Consumes the list of
ScoredEndpointresults from the scoring phase.
Configuration
The plugin config supports:
maxNumOfEndpoints(default 1)- The maximum number of endpoints to pick and return. Must be > 0. If more candidates are available than this limit, only the top subset is returned.
[!TIP] In most production scenarios,
maxNumOfEndpointsis left at its default value of1to select a single target endpoint for the request.
Documentation
¶
Overview ¶
Package maxscore implements a scheduling picker that selects the endpoint(s) with the highest score calculated during the scoring phase.
For detailed behavioral intent and configuration, see the package README.
Index ¶
Constants ¶
const (
// MaxScorePickerType is the registered name of the max score picker plugin.
MaxScorePickerType = "max-score-picker"
)
Variables ¶
This section is empty.
Functions ¶
Types ¶
type MaxScorePicker ¶
type MaxScorePicker struct {
// contains filtered or unexported fields
}
MaxScorePicker picks endpoint(s) with the highest score calculated during the scoring phase.
func NewMaxScorePicker ¶
func NewMaxScorePicker(maxNumOfEndpoints int) *MaxScorePicker
NewMaxScorePicker initializes a new MaxScorePicker and returns its pointer.
func (*MaxScorePicker) Pick ¶
func (p *MaxScorePicker) Pick(ctx context.Context, scoredEndpoints []*fwksched.ScoredEndpoint) *fwksched.ProfileRunResult
Pick selects the endpoint(s) with the highest score calculated during the scoring phase.
func (*MaxScorePicker) TypedName ¶
func (p *MaxScorePicker) TypedName() fwkplugin.TypedName
TypedName returns the type and name tuple of this plugin instance.
func (*MaxScorePicker) WithName ¶
func (p *MaxScorePicker) WithName(name string) *MaxScorePicker
WithName sets the picker's name
Source Files