scheduler

type ContainerQueue ¶

type ContainerQueue interface {
	Entries() (entries map[string]container.QueueEnt, updated time.Time)
	Lock(uuid string) error
	Unlock(uuid string) error
	Cancel(uuid string) error
	Forget(uuid string)
	Get(uuid string) (arvados.Container, bool)
	Subscribe() <-chan struct{}
	Unsubscribe(<-chan struct{})
	Update() error
}

A ContainerQueue is a set of containers that need to be started or stopped. Implemented by container.Queue and test stubs. See container.Queue method documentation for details.

type QueueEnt ¶

type QueueEnt struct {
	container.QueueEnt

	// Human-readable scheduling status as of the last scheduling
	// iteration.
	SchedulingStatus string `json:"scheduling_status"`
}

type Scheduler ¶

type Scheduler struct {
	// contains filtered or unexported fields
}

A Scheduler maps queued containers onto unallocated workers in priority order, creating new workers if needed. It locks containers that can be mapped onto existing/pending workers, and starts them if possible.

A Scheduler unlocks any containers that are locked but can't be mapped. (For example, this happens when the cloud provider reaches quota/capacity and a previously mappable container's priority is surpassed by a newer container.)

If it encounters errors while creating new workers, a Scheduler shuts down idle workers, in case they are consuming quota.

func New(ctx context.Context, client *arvados.Client, queue ContainerQueue, pool WorkerPool, reg *prometheus.Registry, staleLockTimeout, queueUpdateInterval time.Duration, minQuota, maxInstances int, supervisorFraction float64) *Scheduler

func (sch *Scheduler) Queue() []QueueEnt

func (sch *Scheduler) Start()

func (sch *Scheduler) Stop()

type WorkerPool ¶

type WorkerPool interface {
	Running() map[string]time.Time
	Unallocated() map[arvados.InstanceType]int
	CountWorkers() map[worker.State]int
	AtCapacity(arvados.InstanceType) bool
	AtQuota() bool
	Create(arvados.InstanceType) bool
	Shutdown(arvados.InstanceType) bool
	StartContainer(arvados.InstanceType, arvados.Container) bool
	KillContainer(uuid, reason string) bool
	ForgetContainer(uuid string)
	Subscribe() <-chan struct{}
	Unsubscribe(<-chan struct{})
}

A WorkerPool asynchronously starts and stops worker VMs, and starts and stops containers on them. Implemented by worker.Pool and test stubs. See worker.Pool method documentation for details.