Documentation
¶
Overview ¶
Package tasks is the one place the server keeps track of its background work: library scans, metadata, module installs, transcodes, backups and anything else that takes longer than a request. The activity button, the WittyStats module and the native apps all read from it.
Using it from a job takes a few lines (docs/TASKS.md has the whole story):
t := reg.Start(tasks.Spec{Type: tasks.TypeBackup, Title: "Backing up", Cancel: cancel, Pausable: true})
defer func() { t.Finish(err) }()
for i, f := range files {
if err = t.Wait(ctx); err != nil { return } // holds here while paused; ctx's error once cancelled
t.Progress(int64(i), int64(len(files)))
t.Item(filepath.Base(f), f)
}
A nil *Registry and a nil *Handle are both fine to use and do nothing, so a job never has to ask whether anyone is keeping track.
It is built to cost next to nothing. Reporting takes a lock and sets a few fields. With nobody watching nothing else happens: there is no timer, no goroutine and no copying. With someone watching, one goroutine sends what changed at most twice a second, however often jobs report.
Index ¶
- Constants
- Variables
- func Explain(err error) (message, fix string)
- type By
- type DB
- type Handle
- func (h *Handle) Add(n int64)
- func (h *Handle) Begin()
- func (h *Handle) Fail(message, fix string, cause error)
- func (h *Handle) Finish(err error)
- func (h *Handle) ID() string
- func (h *Handle) Item(name, path string)
- func (h *Handle) Library(id int64, name string)
- func (h *Handle) PauseWanted() bool
- func (h *Handle) Phase(s string)
- func (h *Handle) Problem(item, message string)
- func (h *Handle) ProblemN(item, message string, n int)
- func (h *Handle) Progress(done, total int64)
- func (h *Handle) Result(s string)
- func (h *Handle) Steps(steps ...Step)
- func (h *Handle) Title(s string)
- func (h *Handle) Unit(s string)
- func (h *Handle) Wait(ctx context.Context) error
- type Problem
- type Query
- type Registry
- func (r *Registry) CanRetry(t Task) bool
- func (r *Registry) Cancel(id string) error
- func (r *Registry) Find(id string) (Task, bool)
- func (r *Registry) ForSource(moduleID string) *Reporter
- func (r *Registry) ForgetRecent(id string)
- func (r *Registry) Live() []Task
- func (r *Registry) OnRetry(taskType string, f RetryFunc)
- func (r *Registry) Pause(id string) error
- func (r *Registry) Queue(s Spec) *Handle
- func (r *Registry) Recent() []Task
- func (r *Registry) Resume(id string) error
- func (r *Registry) Retry(old Task, by By) error
- func (r *Registry) Settle()
- func (r *Registry) Start(s Spec) *Handle
- func (r *Registry) Subscribe() *Subscription
- func (r *Registry) Watchers() int
- type Report
- type Reporter
- type RetryFunc
- type SQLStore
- func (s *SQLStore) Begin(t Task)
- func (s *SQLStore) Clear(ctx context.Context, id string) error
- func (s *SQLStore) End(t Task)
- func (s *SQLStore) Get(ctx context.Context, id string) (Task, bool)
- func (s *SQLStore) List(ctx context.Context, q Query) ([]Task, error)
- func (s *SQLStore) MarkInterrupted(ctx context.Context)
- type Spec
- type State
- type Step
- type Store
- type Subscription
- type Summary
- type Task
- type Update
- type Viewer
Constants ¶
const ( TypeScan = "scan" TypeAnalysis = "analysis" TypeMetadata = "metadata" TypeThumbnail = "thumbnails" TypeIntro = "intros" TypeGames = "games" TypeModule = "module" TypeBackup = "backup" TypeTranscode = "transcode" TypeUpdate = "update" TypeGuide = "guide" TypeRecording = "recording" TypeDiagnose = "diagnose" TypeOther = "other" )
The kinds of work the server knows. A module or a new job may use any short name of its own: the app shows an unknown kind with a plain icon and its title.
Variables ¶
var ( ErrNotFound = errors.New("that task isn't known; it may have finished") ErrCant = errors.New("that can't be done to this task") )
Errors from the controls below.
Functions ¶
Types ¶
type DB ¶
type DB interface {
ExecContext(ctx context.Context, query string, args ...any) (sql.Result, error)
QueryContext(ctx context.Context, query string, args ...any) (*sql.Rows, error)
}
DB is what the history needs from the server's database.
type Handle ¶
type Handle struct {
// contains filtered or unexported fields
}
Handle is a job's hold on its entry in the registry. Every method may be called from any goroutine, and on a nil Handle.
func (*Handle) Finish ¶
Finish ends the task according to how the job ended: nil is done, a context's cancellation is cancelled, and anything else failed, explained with Explain. Calling it again changes nothing.
func (*Handle) Item ¶
Item says what it's working on now: a name, and optionally where that is. Only admins see these.
func (*Handle) PauseWanted ¶
PauseWanted reports whether someone asked for this to pause, for a job that wants to let go of something (its turn at the disk) before calling Wait.
func (*Handle) Phase ¶
Phase says which stage it's at: "Finding files", "Reading details". A phase named in Steps moves the task on to it, and counts its own Progress from nothing.
func (*Handle) Problem ¶
Problem notes something that went wrong with one thing, while the task carries on. The message is for a person, and should say what to do where that isn't obvious.
func (*Handle) Steps ¶
Steps names the phases a job goes through, in order, so the app shows one bar across them with each phase named, rather than a bar that starts again at each. A phase that isn't among them, such as waiting for a turn, leaves the bar where it was.
type Problem ¶
type Problem struct {
Item string `json:"item,omitempty"`
Message string `json:"message"`
Count int `json:"count,omitempty"` // this many times, if more than once
At int64 `json:"at"`
}
Problem is something that went wrong with one thing inside a task that carried on anyway: one file out of thousands.
type Query ¶
type Query struct {
Limit int
Before int64 // finished before this time
Type string
State string
Library int64
}
Query narrows a look through the history.
type Registry ¶
type Registry struct {
// contains filtered or unexported fields
}
Registry holds every task.
func (*Registry) ForSource ¶
ForSource makes a Reporter for a module process. On a nil registry it does nothing.
func (*Registry) ForgetRecent ¶
ForgetRecent drops what's held in memory about finished tasks: the history was cleared.
func (*Registry) OnRetry ¶
OnRetry says how to start a kind of task again. Only tasks with a RetryKey can be.
func (*Registry) Pause ¶
Pause asks a task to stop at its next step. It stays "running" until the job gets there, which is rarely long.
func (*Registry) Recent ¶
Recent lists what finished lately, newest first. The rest is in the store.
func (*Registry) Settle ¶
func (r *Registry) Settle()
Settle waits for what's on its way to the store. For tests, and an orderly shutdown.
func (*Registry) Subscribe ¶
func (r *Registry) Subscribe() *Subscription
Subscribe starts watching. Close it when done.
type Report ¶
type Report struct {
Key string // the module's own name for the task
Title string
State State // Running, Done or Failed
Done int64
Total int64
Item string
}
Report is what a module's process says about one of its own tasks: wittymodule's TaskReport, after its Validate has passed. The process may be someone's fork, so nothing here is believed further than that module's own tasks.
type Reporter ¶
type Reporter struct {
// contains filtered or unexported fields
}
Reporter takes the reports of one running module process. It never blocks for long, since it runs on the goroutine reading that process. Reports replace each other: when they come too quickly, the latest is applied a moment later and the rest are lost, which costs only steps along the way. A finish is never held back.
type SQLStore ¶
type SQLStore struct {
DB DB
// Keep says how many finished tasks to hold, and for how many days. Read when
// pruning, so a change in Settings applies from the next task.
Keep func() (count, days int)
// Failed is called when a write fails. Losing a history row isn't worth more than a log line.
Failed func(err error)
}
SQLStore keeps the history in the task_history table, with a limit on how much and for how long. Writes are small, and arrive from the registry's own writer.
func (*SQLStore) MarkInterrupted ¶
MarkInterrupted says what happened to tasks the last run of the server left going. Call it once, before anything starts.
type Spec ¶
type Spec struct {
Type string
Title string // no file paths: non-admins may read it
Phase string
LibraryID int64
LibraryName string
Unit string // what Done and Total count: "files", "titles", "bytes"
By By
Source string // a module's id
Key string // a module's task: "<module id>/<its key>"
// Cancel, when set, makes it cancellable. It should stop the job's context; the
// job then ends with Finish(ctx.Err()).
Cancel func()
// Pausable: the job calls Wait between pieces of work.
Pausable bool
// Private: only the person who started it, and admins, can see it.
Private bool
// AdminOnly: only admins can see it (a module's report, a server update).
AdminOnly bool
// NoHistory leaves it out of the history when it ends. For things that come and
// go constantly, such as someone's transcode.
NoHistory bool
// Quiet drops it from the history when it ends with nothing done and nothing
// wrong: a background service that woke up and found nothing to do.
Quiet bool
// RetryKey lets a handler given to OnRetry start this again, even after a restart.
RetryKey string
}
Spec describes a task when it starts.
type Step ¶
type Step struct {
Name string
}
Step is one of the phases a job names with Steps, and its rough part of the work, on any scale. 0 counts as 1.
type Store ¶
Store keeps tasks across restarts. Begin is called when one starts, so a task that a crash cut short can be told about afterwards, and End when it finishes.
type Subscription ¶
type Subscription struct {
// contains filtered or unexported fields
}
Subscription is someone watching. Next gives each change once, coalesced: a slow watcher gets the latest state rather than a backlog.
func (*Subscription) C ¶
func (s *Subscription) C() <-chan struct{}
C says something may be waiting for Take.
func (*Subscription) Close ¶
func (s *Subscription) Close()
Close stops watching. When the last one goes, the flusher stops too.
func (*Subscription) Take ¶
func (s *Subscription) Take() Update
Take returns what changed since the last Take.
type Summary ¶
type Summary struct {
Active int `json:"active"` // running or paused
Queued int `json:"queued"`
Paused int `json:"paused"`
Percent float64 `json:"percent"` // across what can be measured, or -1
}
Summary is the state of things in a few numbers, for the button.
type Task ¶
type Task struct {
ID string `json:"id"`
Type string `json:"type"`
Title string `json:"title"`
State State `json:"state"`
Phase string `json:"phase,omitempty"`
Steps []string `json:"steps,omitempty"` // the phases it goes through, in order
Step int `json:"step,omitempty"` // the one it's on, from 1 (0: none), and then Percent is across them all
LibraryID int64 `json:"libraryId,omitempty"`
LibraryName string `json:"libraryName,omitempty"`
Item string `json:"item,omitempty"` // the thing being worked on, for admins
Path string `json:"path,omitempty"` // where it is, for admins
Done int64 `json:"done"`
Total int64 `json:"total"` // 0: not known (yet)
Unit string `json:"unit,omitempty"`
Percent float64 `json:"percent"` // 0 to 100, or -1 when there's no telling
Rate float64 `json:"rate,omitempty"` // units per second, smoothed
ETASec int64 `json:"etaSec,omitempty"`
StartedAt int64 `json:"startedAt"`
UpdatedAt int64 `json:"updatedAt"`
FinishedAt int64 `json:"finishedAt,omitempty"`
ByID int64 `json:"byId,omitempty"`
By string `json:"by,omitempty"`
Source string `json:"source,omitempty"` // the module reporting it; empty for the server
Key string `json:"key,omitempty"` // a module's task: "<module id>/<its key>"
Result string `json:"result,omitempty"` // what came of it, in a sentence
Error string `json:"error,omitempty"` // what went wrong, for a person
Fix string `json:"fix,omitempty"` // what they can do about it
Detail string `json:"detail,omitempty"` // the error as it came, for admins and reports
ProblemCount int `json:"problemCount,omitempty"` // every problem, including those not kept
Problems []Problem `json:"problems,omitempty"`
CanPause bool `json:"canPause"`
CanResume bool `json:"canResume"`
CanCancel bool `json:"canCancel"`
CanRetry bool `json:"canRetry"`
// Private tasks are about one person's activity: only they and admins see them.
Private bool `json:"private,omitempty"`
// RetryKey is what a retry handler needs to start this again.
RetryKey string `json:"-"`
// AdminOnly: never shown to anyone else, whatever the settings say.
AdminOnly bool `json:"-"`
}
Task is a task as anyone outside this package sees it: a copy, safe to keep.
type Update ¶
Update is what changed. Tasks holds changed tasks whole, including finished ones; Gone lists tasks to forget.