scheduling

package

v0.9.2 Latest Latest Go to latest Published: Jun 23, 2026 License: Apache-2.0 Imports: 10 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/llm-d/llm-d-router

Links

Open Source Insights

Documentation ¶

Overview ¶

Package scheduling implements request scheduling algorithms.

Index ¶

type Scheduler
- func NewSchedulerWithConfig(config *SchedulerConfig) *Scheduler
- func (s *Scheduler) Schedule(ctx context.Context, request *fwksched.InferenceRequest, ...) (result *fwksched.SchedulingResult, err error)
type SchedulerConfig
- func NewSchedulerConfig(profileHandler fwksched.ProfileHandler, ...) *SchedulerConfig
- func (c *SchedulerConfig) String() string
type SchedulerProfile
- func NewSchedulerProfile() *SchedulerProfile
type WeightedScorer
- func NewWeightedScorer(scorer fwksched.Scorer, weight float64) *WeightedScorer
- func (s *WeightedScorer) Weight() float64

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

This section is empty.

Types ¶

type Scheduler ¶

type Scheduler struct {
	// contains filtered or unexported fields
}

func NewSchedulerWithConfig ¶

func NewSchedulerWithConfig(config *SchedulerConfig) *Scheduler

NewSchedulerWithConfig returns a new scheduler with the given scheduler plugins configuration.

func (*Scheduler) Schedule ¶

func (s *Scheduler) Schedule(ctx context.Context, request *fwksched.InferenceRequest, candidateEndpoints []fwksched.Endpoint) (result *fwksched.SchedulingResult, err error)

Schedule finds the target pod based on metrics and the requested lora adapter.

type SchedulerConfig ¶

type SchedulerConfig struct {
	// contains filtered or unexported fields
}

SchedulerConfig provides a configuration for the scheduler which influence routing decisions.

func NewSchedulerConfig ¶

func NewSchedulerConfig(profileHandler fwksched.ProfileHandler, profiles map[string]fwksched.SchedulerProfile) *SchedulerConfig

NewSchedulerConfig creates a new SchedulerConfig object and returns its pointer.

func (*SchedulerConfig) String ¶

func (c *SchedulerConfig) String() string

type SchedulerProfile ¶

type SchedulerProfile struct {
	// contains filtered or unexported fields
}

SchedulerProfile provides a profile configuration for the scheduler which influence routing decisions.

func NewSchedulerProfile ¶

func NewSchedulerProfile() *SchedulerProfile

NewSchedulerProfile creates a new SchedulerProfile object and returns its pointer.

func (*SchedulerProfile) AddPlugins ¶

func (p *SchedulerProfile) AddPlugins(pluginObjects ...plugin.Plugin) error

AddPlugins adds the given plugins to all scheduler plugins according to the interfaces each plugin implements. A plugin may implement more than one scheduler plugin interface. Special Case: In order to add a scorer, one must use the scorer.NewWeightedScorer function in order to provide a weight. if a scorer implements more than one interface, supplying a WeightedScorer is sufficient. The function will take the internal scorer object and register it to all interfaces it implements.

func (*SchedulerProfile) Run ¶

func (p *SchedulerProfile) Run(ctx context.Context, request *fwksched.InferenceRequest, candidateEndpoints []fwksched.Endpoint) (*fwksched.ProfileRunResult, error)

Run runs a SchedulerProfile. It invokes all the SchedulerProfile plugins for the given request in this order - Filters, Scorers, Picker. After completing all, it returns the result.

func (*SchedulerProfile) String ¶

func (p *SchedulerProfile) String() string

func (*SchedulerProfile) WithFilters ¶

func (p *SchedulerProfile) WithFilters(filters ...fwksched.Filter) *SchedulerProfile

WithFilters sets the given filter plugins as the Filter plugins. if the SchedulerProfile has Filter plugins, this call replaces the existing plugins with the given ones.

func (*SchedulerProfile) WithPicker ¶

func (p *SchedulerProfile) WithPicker(picker fwksched.Picker) *SchedulerProfile

WithPicker sets the given picker plugins as the Picker plugin. if the SchedulerProfile has Picker plugin, this call replaces the existing plugin with the given one.

func (*SchedulerProfile) WithScorers ¶

func (p *SchedulerProfile) WithScorers(scorers ...*WeightedScorer) *SchedulerProfile

WithScorers sets the given scorer plugins as the Scorer plugins. if the SchedulerProfile has Scorer plugins, this call replaces the existing plugins with the given ones.

type WeightedScorer ¶

type WeightedScorer struct {
	fwksched.Scorer
	// contains filtered or unexported fields
}

WeightedScorer is a struct that encapsulates a scorer with its weight.

func NewWeightedScorer ¶

func NewWeightedScorer(scorer fwksched.Scorer, weight float64) *WeightedScorer

NewWeightedScorer initializes a new WeightedScorer and returns its pointer.

func (*WeightedScorer) Weight ¶

func (s *WeightedScorer) Weight() float64

Weight returns the weight of the scorer.

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL