The highest tagged major version is
README
¶
Worker Pool
A worker pool allows you to setup a pool of finite worker goroutines to perform jobs. This paradigm is especially useful where you need to limit work in progress. For example, if you need to run concurrent HTTP requests, you should limit the number of simultaneous requests because the OS may have limits, and the receieving server may also have limits.
Example
package main
import (
"fmt"
"time"
"github.com/app-nerds/kit/v3/workerpool"
)
type Job struct {
Index int
}
func (j *Job) Work(workerID int) {
fmt.Printf("Worker %d sleeping on index %d...\n", workerID, j.Index)
time.Sleep(2 * time.Second)
}
func main() {
var pool workerpool.IPool
pool = workerpool.NewPool(workerpool.PoolConfig{
MaxJobQueue: 100,
MaxWorkers: 10,
MaxWorkerWaitTime: 3 * time.Second,
})
pool.Start()
for index := 0; index < 30; index++ {
job := &Job{Index: index}
pool.QueueJob(job)
}
pool.Wait()
pool.Shutdown()
}
Documentation
¶
Overview ¶
Copyright 2018-2019 AppNinjas LLC. All rights reserved Use of this source code is governed by the MIT license.
Copyright 2018-2019 AppNinjas LLC. All rights reserved Use of this source code is governed by the MIT license.
Copyright 2018-2019 AppNinjas LLC. All rights reserved Use of this source code is governed by the MIT license.
Package workerpool implements a pool of workers (goroutines) who can perform jobs. Below is an example of a pool that has 10 workers, and if none are available will wait 2 seoncds for a worker to become available before returning a timeout error.
package main
import (
"fmt"
"time"
"github.com/app-nerds/kit/v3/workerpool"
)
type Job struct {
Index int
}
func (j *Job) Work(workerID int) {
fmt.Printf("Worker %d sleeping on index %d...\n", workerID, j.Index)
time.Sleep(2 * time.Second)
}
func main() {
var pool workerpool.IPool
pool = workerpool.NewPool(workerpool.PoolConfig{
MaxJobQueue: 100,
MaxWorkers: 10,
MaxWorkerWaitTime: 3 * time.Second,
})
pool.Start()
for index := 0; index < 30; index++ {
job := &Job{Index: index}
pool.QueueJob(job)
}
pool.Wait()
pool.Shutdown()
}
Copyright 2018-2019 AppNinjas LLC. All rights reserved Use of this source code is governed by the MIT license.
Copyright 2018-2019 AppNinjas LLC. All rights reserved Use of this source code is governed by the MIT license.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type ErrNoAvaialableWorkers ¶
type ErrNoAvaialableWorkers struct {
Job Job
}
ErrNoAvaialableWorkers is used to describe a situation where there are no workers available to work a job
func (ErrNoAvaialableWorkers) Error ¶
func (e ErrNoAvaialableWorkers) Error() string
func (ErrNoAvaialableWorkers) GetJob ¶
func (e ErrNoAvaialableWorkers) GetJob() Job
GetJob returns the job associated with this error
type IPool ¶
type IPool interface {
PutWorkerInTheQueue(worker IWorker)
Shutdown()
Start()
QueueJob(job Job)
Wait()
}
IPool describes an interface for managing a pool of workers who perform jobs
type Job ¶
type Job interface {
Work(workerID int)
}
A Job is an interface structs must implement which actually executes the work to be done by a worker in the pool. The workerID is the identifier of the worker performing the job
type Pool ¶
type Pool struct {
ErrorQueue chan JobError
// contains filtered or unexported fields
}
A Pool provides methods for managing a pool of workers who perform jobs. A pool can be configured to have a maximum number of available workers, and will wait up to a configurable amount of time for a worker to become available before returning an error
func (*Pool) PutWorkerInTheQueue ¶
PutWorkerInTheQueue puts a worker in the worker queue
type PoolConfig ¶
PoolConfig provides the ability to configure the worker pool. MaxWorkers specifies the maximum number of workers available. This essentially sets the channel size. MaxWorkerWaitTime is a duration that tells the pool how long it will wait before timing out when a client requests a worker.
type Worker ¶
A Worker is someone that performs a job. There are a finite number of workers in the pool
func (*Worker) DoJob ¶
DoJob executes the provided job. When the work is complete this worker will put itself back in the queue as available. This method execute a goroutine
func (*Worker) RejoinWorkerPool ¶
func (w *Worker) RejoinWorkerPool()
RejoinWorkerPool puts this worker back in the worker queue of the pool. A worker will rejoin the queue when she has finished the job
Source Files