Documentation
¶
Overview ¶
Package throttle provides launch throttling for cost control and resource management
This package implements configurable launch rate limiting to prevent: - Accidental mass launches from script loops - Resource exhaustion and cost overruns - Unfair resource usage in multi-user environments
Key Features: - Global, per-user, and per-project throttling - Token bucket algorithm with configurable rates - Budget-aware throttling (more aggressive near limits) - Integration with policy and budget systems
Index ¶
- type Config
- type LaunchRequest
- type LaunchThrottler
- func (lt *LaunchThrottler) AllowLaunch(ctx context.Context, req LaunchRequest) error
- func (lt *LaunchThrottler) CheckLaunch(ctx context.Context, req LaunchRequest) error
- func (lt *LaunchThrottler) GetProjectOverride(projectID string) (Override, bool)
- func (lt *LaunchThrottler) GetStatus(scope string) Status
- func (lt *LaunchThrottler) ListProjectOverrides() []Override
- func (lt *LaunchThrottler) RemoveProjectOverride(projectID string)
- func (lt *LaunchThrottler) SetBudgetUsageFunc(fn func(projectID string) (used float64, limit float64, err error))
- func (lt *LaunchThrottler) SetProjectOverride(override Override)
- func (lt *LaunchThrottler) UpdateConfig(config Config)
- type Override
- type Status
- type ThrottleError
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Config ¶
type Config struct {
// Global throttling
Enabled bool `json:"enabled"`
MaxLaunches int `json:"max_launches"` // Max launches in time window
TimeWindow string `json:"time_window"` // e.g., "1h", "24h"
BurstSize int `json:"burst_size"` // Burst capacity (0 = same as max)
// Granular throttling
PerUser bool `json:"per_user"` // Separate buckets per user
PerProject bool `json:"per_project"` // Separate buckets per project
// Budget integration
BudgetAware bool `json:"budget_aware"` // Throttle more near budget limits
BudgetThreshold float64 `json:"budget_threshold"` // Throttle threshold (0.0-1.0, e.g., 0.8 = 80%)
BudgetMultiplier float64 `json:"budget_multiplier"` // Rate reduction factor (e.g., 0.5 = half rate)
// Behavior
QueueMode string `json:"queue_mode"` // "reject", "queue", "warn"
}
Config defines throttling configuration
func DefaultConfig ¶
func DefaultConfig() Config
DefaultConfig returns sensible default throttling configuration
type LaunchRequest ¶
type LaunchRequest struct {
UserID string // Research user or system user
ProjectID string // Project ID if applicable
TemplateName string // Template being launched
InstanceType string // EC2 instance type
IsGPU bool // Whether GPU instance
}
LaunchRequest contains information about a launch request for throttling check
type LaunchThrottler ¶
type LaunchThrottler struct {
// contains filtered or unexported fields
}
LaunchThrottler manages launch rate limiting across multiple scopes
func NewLaunchThrottler ¶
func NewLaunchThrottler(config Config) *LaunchThrottler
NewLaunchThrottler creates a new launch throttler
func (*LaunchThrottler) AllowLaunch ¶
func (lt *LaunchThrottler) AllowLaunch(ctx context.Context, req LaunchRequest) error
AllowLaunch checks if a launch is allowed under current throttling rules
This method checks all applicable throttling scopes (global, user, project) and returns an error if any scope rejects the launch.
func (*LaunchThrottler) CheckLaunch ¶ added in v0.37.0
func (lt *LaunchThrottler) CheckLaunch(ctx context.Context, req LaunchRequest) error
CheckLaunch reports whether AllowLaunch would admit req, without consuming any scope's capacity. A launch dry run uses it, so previewing a launch never throttles the real one.
func (*LaunchThrottler) GetProjectOverride ¶
func (lt *LaunchThrottler) GetProjectOverride(projectID string) (Override, bool)
GetProjectOverride returns the override for a project, if any
func (*LaunchThrottler) GetStatus ¶
func (lt *LaunchThrottler) GetStatus(scope string) Status
GetStatus returns current throttling status for a scope
func (*LaunchThrottler) ListProjectOverrides ¶
func (lt *LaunchThrottler) ListProjectOverrides() []Override
ListProjectOverrides returns all project overrides
func (*LaunchThrottler) RemoveProjectOverride ¶
func (lt *LaunchThrottler) RemoveProjectOverride(projectID string)
RemoveProjectOverride removes a project-specific override
func (*LaunchThrottler) SetBudgetUsageFunc ¶
func (lt *LaunchThrottler) SetBudgetUsageFunc(fn func(projectID string) (used float64, limit float64, err error))
SetBudgetUsageFunc sets the callback for budget usage checking
func (*LaunchThrottler) SetProjectOverride ¶
func (lt *LaunchThrottler) SetProjectOverride(override Override)
SetProjectOverride sets a project-specific throttling override
func (*LaunchThrottler) UpdateConfig ¶
func (lt *LaunchThrottler) UpdateConfig(config Config)
UpdateConfig updates the throttling configuration
type Override ¶
type Override struct {
ProjectID string `json:"project_id"`
MaxLaunches int `json:"max_launches"`
TimeWindow string `json:"time_window"`
CreatedBy string `json:"created_by"`
CreatedAt time.Time `json:"created_at"`
Reason string `json:"reason"` // Why override was created
}
Override represents a project-specific throttling override
type Status ¶
type Status struct {
Scope string `json:"scope"` // "global", "user:username", "project:projectid"
Enabled bool `json:"enabled"`
MaxLaunches int `json:"max_launches"`
TimeWindow string `json:"time_window"`
CurrentTokens float64 `json:"current_tokens"` // Available tokens
LaunchesInWindow int `json:"launches_in_window"` // Launches in current window
NextTokenRefill time.Time `json:"next_token_refill"` // When next token available
TimeUntilRefill time.Duration `json:"time_until_refill"` // Human-readable wait time
BudgetAdjusted bool `json:"budget_adjusted"` // Whether rate adjusted for budget
EffectiveRate float64 `json:"effective_rate"` // Current rate (may be reduced)
ConfiguredRate float64 `json:"configured_rate"` // Original configured rate
LastLaunchTime time.Time `json:"last_launch_time"` // Most recent launch
TotalThrottled int64 `json:"total_throttled"` // Total launches rejected
AllowedLaunches int64 `json:"allowed_launches"` // Total launches allowed
SuccessRate float64 `json:"success_rate"` // Percentage allowed (0.0-1.0)
}
Status represents current throttling status for a scope
type ThrottleError ¶
type ThrottleError struct {
Scope string `json:"scope"`
MaxLaunches int `json:"max_launches"`
LaunchCount int `json:"launch_count"`
TimeWindow string `json:"time_window"`
RetryAfter time.Duration `json:"retry_after"`
CurrentTokens float64 `json:"current_tokens"`
Message string `json:"message"`
}
ThrottleError indicates a launch was rejected due to throttling
func (*ThrottleError) Error ¶
func (e *ThrottleError) Error() string