Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
View Source
var ( ErrorInavlidMaxWorkers = errors.New("Inavlid max workers amount") ErrorNoTaskFound = errors.New("Did not find task with this name") )
View Source
var (
ErrorHTTPStatusCode = errors.New("HTTP status code greater than 203")
)
Functions ¶
func HandleArchived ¶
func HandleArchived(job Job)
func HandleDownloadImage ¶
func HandleDownloadImage(job Job)
HandleDownloadImage handles the download of images
func HandleDownloadMeta ¶
func HandleDownloadMeta(job Job)
HandleDownloadMeta handles the download of meta data
func HandleDownloadSources ¶
func HandleDownloadSources(job Job)
HandleDownloadSources handles the download of CSS and JS source files from archived entities
func HandleSave ¶
func HandleSave(job Job)
HandleSave handles the saving of entities to the database
func HandleSaveHtml ¶
func HandleSaveHtml(job Job)
Types ¶
type Job ¶
type Job struct {
Work string
Data string
Entity m.Entity
Result Result
Archive Archive
Scheduler *Scheduler
}
Job represents the work each worker consumes
type Scheduler ¶
type Scheduler struct {
Config *m.Config
DB *db.DB
Tasks map[string]func(Job)
Queue chan Job
Ready chan chan Job
Workers []*Worker
MaxWorkers int
HTTPClient http.Client
// contains filtered or unexported fields
}
Scheduler is the top level struct
func New ¶
New returns a new scheduler instance with n workers, provide n = -1 to use runtime.NumCPU - 1 workers.
func (*Scheduler) RegisterTasks ¶
Click to show internal directories.
Click to hide internal directories.