tasks

package
v0.8.0-alpha Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 16, 2026 License: AGPL-3.0 Imports: 14 Imported by: 0

Documentation

Overview

Package tasks is the one place the server keeps track of its background work: library scans, metadata, module installs, transcodes, backups and anything else that takes longer than a request. The activity button, the WittyStats module and the native apps all read from it.

Using it from a job takes a few lines (docs/TASKS.md has the whole story):

t := reg.Start(tasks.Spec{Type: tasks.TypeBackup, Title: "Backing up", Cancel: cancel, Pausable: true})
defer func() { t.Finish(err) }()
for i, f := range files {
	if err = t.Wait(ctx); err != nil { return }  // holds here while paused; ctx's error once cancelled
	t.Progress(int64(i), int64(len(files)))
	t.Item(filepath.Base(f), f)
}

A nil *Registry and a nil *Handle are both fine to use and do nothing, so a job never has to ask whether anyone is keeping track.

It is built to cost next to nothing. Reporting takes a lock and sets a few fields. With nobody watching nothing else happens: there is no timer, no goroutine and no copying. With someone watching, one goroutine sends what changed at most twice a second, however often jobs report.

Index

Constants

View Source
const (
	TypeScan      = "scan"
	TypeAnalysis  = "analysis"
	TypeMetadata  = "metadata"
	TypeThumbnail = "thumbnails"
	TypeIntro     = "intros"
	TypeGames     = "games"
	TypeModule    = "module"
	TypeBackup    = "backup"
	TypeTranscode = "transcode"
	TypeUpdate    = "update"
	TypeGuide     = "guide"
	TypeRecording = "recording"
	TypeDiagnose  = "diagnose"
	TypeOther     = "other"
)

The kinds of work the server knows. A module or a new job may use any short name of its own: the app shows an unknown kind with a plain icon and its title.

Variables

View Source
var (
	ErrNotFound = errors.New("that task isn't known; it may have finished")
	ErrCant     = errors.New("that can't be done to this task")
)

Errors from the controls below.

Functions

func Explain

func Explain(err error) (message, fix string)

Explain turns an error into a sentence a person can understand, and what they can do about it. It's a fallback: a job that knows better should say so with Fail.

Types

type By

type By struct {
	ID   int64
	Name string
}

By says who asked for a task. The zero value is the server itself.

type DB

type DB interface {
	ExecContext(ctx context.Context, query string, args ...any) (sql.Result, error)
	QueryContext(ctx context.Context, query string, args ...any) (*sql.Rows, error)
}

DB is what the history needs from the server's database.

type Handle

type Handle struct {
	// contains filtered or unexported fields
}

Handle is a job's hold on its entry in the registry. Every method may be called from any goroutine, and on a nil Handle.

func (*Handle) Add

func (h *Handle) Add(n int64)

Add counts more done, for work done in parallel.

func (*Handle) Begin

func (h *Handle) Begin()

Begin moves a queued task to running.

func (*Handle) Fail

func (h *Handle) Fail(message, fix string, cause error)

Fail ends the task as failed, with words of the job's own.

func (*Handle) Finish

func (h *Handle) Finish(err error)

Finish ends the task according to how the job ended: nil is done, a context's cancellation is cancelled, and anything else failed, explained with Explain. Calling it again changes nothing.

func (*Handle) ID

func (h *Handle) ID() string

ID is the task's id, or "" on a nil handle.

func (*Handle) Item

func (h *Handle) Item(name, path string)

Item says what it's working on now: a name, and optionally where that is. Only admins see these.

func (*Handle) Library

func (h *Handle) Library(id int64, name string)

Library says which library this is about.

func (*Handle) PauseWanted

func (h *Handle) PauseWanted() bool

PauseWanted reports whether someone asked for this to pause, for a job that wants to let go of something (its turn at the disk) before calling Wait.

func (*Handle) Phase

func (h *Handle) Phase(s string)

Phase says which stage it's at: "Finding files", "Reading details". A phase named in Steps moves the task on to it, and counts its own Progress from nothing.

func (*Handle) Problem

func (h *Handle) Problem(item, message string)

Problem notes something that went wrong with one thing, while the task carries on. The message is for a person, and should say what to do where that isn't obvious.

func (*Handle) ProblemN

func (h *Handle) ProblemN(item, message string, n int)

ProblemN is Problem for something that happened n times.

func (*Handle) Progress

func (h *Handle) Progress(done, total int64)

Progress says how far along it is. total 0 means not known.

func (*Handle) Result

func (h *Handle) Result(s string)

Result says in a sentence what came of it, for the history.

func (*Handle) Steps

func (h *Handle) Steps(steps ...Step)

Steps names the phases a job goes through, in order, so the app shows one bar across them with each phase named, rather than a bar that starts again at each. A phase that isn't among them, such as waiting for a turn, leaves the bar where it was.

func (*Handle) Title

func (h *Handle) Title(s string)

Title changes what it's called. No file paths: non-admins may read it.

func (*Handle) Unit

func (h *Handle) Unit(s string)

Unit says what Progress counts.

func (*Handle) Wait

func (h *Handle) Wait(ctx context.Context) error

Wait holds a job here while its task is paused. It returns ctx's error once that is done, so it's also the check for having been cancelled.

type Problem

type Problem struct {
	Item    string `json:"item,omitempty"`
	Message string `json:"message"`
	Count   int    `json:"count,omitempty"` // this many times, if more than once
	At      int64  `json:"at"`
}

Problem is something that went wrong with one thing inside a task that carried on anyway: one file out of thousands.

type Query

type Query struct {
	Limit   int
	Before  int64 // finished before this time
	Type    string
	State   string
	Library int64
}

Query narrows a look through the history.

type Registry

type Registry struct {
	// contains filtered or unexported fields
}

Registry holds every task.

func New

func New(store Store) *Registry

New makes a registry. store may be nil.

func (*Registry) CanRetry

func (r *Registry) CanRetry(t Task) bool

CanRetry reports whether a finished task could be started again.

func (*Registry) Cancel

func (r *Registry) Cancel(id string) error

Cancel asks a task to stop. It ends when the job notices, moments later.

func (*Registry) Find

func (r *Registry) Find(id string) (Task, bool)

Find returns a task that's live or finished lately.

func (*Registry) ForSource

func (r *Registry) ForSource(moduleID string) *Reporter

ForSource makes a Reporter for a module process. On a nil registry it does nothing.

func (*Registry) ForgetRecent

func (r *Registry) ForgetRecent(id string)

ForgetRecent drops what's held in memory about finished tasks: the history was cleared.

func (*Registry) Live

func (r *Registry) Live() []Task

Live lists what's going on, oldest first.

func (*Registry) OnRetry

func (r *Registry) OnRetry(taskType string, f RetryFunc)

OnRetry says how to start a kind of task again. Only tasks with a RetryKey can be.

func (*Registry) Pause

func (r *Registry) Pause(id string) error

Pause asks a task to stop at its next step. It stays "running" until the job gets there, which is rarely long.

func (*Registry) Queue

func (r *Registry) Queue(s Spec) *Handle

Queue makes a task that is waiting its turn. Begin moves it to running.

func (*Registry) Recent

func (r *Registry) Recent() []Task

Recent lists what finished lately, newest first. The rest is in the store.

func (*Registry) Resume

func (r *Registry) Resume(id string) error

Resume lets a paused task carry on.

func (*Registry) Retry

func (r *Registry) Retry(old Task, by By) error

Retry starts a finished task again, by the handler for its type.

func (*Registry) Settle

func (r *Registry) Settle()

Settle waits for what's on its way to the store. For tests, and an orderly shutdown.

func (*Registry) Start

func (r *Registry) Start(s Spec) *Handle

Start makes a running task.

func (*Registry) Subscribe

func (r *Registry) Subscribe() *Subscription

Subscribe starts watching. Close it when done.

func (*Registry) Watchers

func (r *Registry) Watchers() int

Watchers says how many subscriptions are open.

type Report

type Report struct {
	Key   string // the module's own name for the task
	Title string
	State State // Running, Done or Failed
	Done  int64
	Total int64
	Item  string
}

Report is what a module's process says about one of its own tasks: wittymodule's TaskReport, after its Validate has passed. The process may be someone's fork, so nothing here is believed further than that module's own tasks.

type Reporter

type Reporter struct {
	// contains filtered or unexported fields
}

Reporter takes the reports of one running module process. It never blocks for long, since it runs on the goroutine reading that process. Reports replace each other: when they come too quickly, the latest is applied a moment later and the rest are lost, which costs only steps along the way. A finish is never held back.

func (*Reporter) CloseAll

func (m *Reporter) CloseAll(failed bool)

CloseAll ends whatever the process left going, since it has stopped: as failed if it failed, and otherwise as cancelled, because nobody can say that work finished.

func (*Reporter) Take

func (m *Reporter) Take(rep Report)

Take applies one report.

type RetryFunc

type RetryFunc func(old Task, by By) error

RetryFunc starts a task again from what's left of it.

type SQLStore

type SQLStore struct {
	DB DB
	// Keep says how many finished tasks to hold, and for how many days. Read when
	// pruning, so a change in Settings applies from the next task.
	Keep func() (count, days int)
	// Failed is called when a write fails. Losing a history row isn't worth more than a log line.
	Failed func(err error)
}

SQLStore keeps the history in the task_history table, with a limit on how much and for how long. Writes are small, and arrive from the registry's own writer.

func (*SQLStore) Begin

func (s *SQLStore) Begin(t Task)

Begin notes a task as it starts, so a crash can be told about afterwards.

func (*SQLStore) Clear

func (s *SQLStore) Clear(ctx context.Context, id string) error

Clear forgets the history, or one task of it. What's still going stays.

func (*SQLStore) End

func (s *SQLStore) End(t Task)

End saves a task as it ended, and trims the history.

func (*SQLStore) Get

func (s *SQLStore) Get(ctx context.Context, id string) (Task, bool)

Get reads one task of the history.

func (*SQLStore) List

func (s *SQLStore) List(ctx context.Context, q Query) ([]Task, error)

List reads the history, newest first.

func (*SQLStore) MarkInterrupted

func (s *SQLStore) MarkInterrupted(ctx context.Context)

MarkInterrupted says what happened to tasks the last run of the server left going. Call it once, before anything starts.

type Spec

type Spec struct {
	Type        string
	Title       string // no file paths: non-admins may read it
	Phase       string
	LibraryID   int64
	LibraryName string
	Unit        string // what Done and Total count: "files", "titles", "bytes"
	By          By
	Source      string // a module's id
	Key         string // a module's task: "<module id>/<its key>"

	// Cancel, when set, makes it cancellable. It should stop the job's context; the
	// job then ends with Finish(ctx.Err()).
	Cancel func()
	// Pausable: the job calls Wait between pieces of work.
	Pausable bool
	// Private: only the person who started it, and admins, can see it.
	Private bool
	// AdminOnly: only admins can see it (a module's report, a server update).
	AdminOnly bool
	// NoHistory leaves it out of the history when it ends. For things that come and
	// go constantly, such as someone's transcode.
	NoHistory bool
	// Quiet drops it from the history when it ends with nothing done and nothing
	// wrong: a background service that woke up and found nothing to do.
	Quiet bool
	// RetryKey lets a handler given to OnRetry start this again, even after a restart.
	RetryKey string
}

Spec describes a task when it starts.

type State

type State string

State is where a task is in its life.

const (
	Queued    State = "queued"
	Running   State = "running"
	Paused    State = "paused"
	Failed    State = "failed"
	Done      State = "done"
	Cancelled State = "cancelled"
)

func (State) Finished

func (s State) Finished() bool

Finished reports whether the task will not change again.

type Step

type Step struct {
	Name  string
	Share float64
}

Step is one of the phases a job names with Steps, and its rough part of the work, on any scale. 0 counts as 1.

type Store

type Store interface {
	Begin(t Task)
	End(t Task)
}

Store keeps tasks across restarts. Begin is called when one starts, so a task that a crash cut short can be told about afterwards, and End when it finishes.

type Subscription

type Subscription struct {
	// contains filtered or unexported fields
}

Subscription is someone watching. Next gives each change once, coalesced: a slow watcher gets the latest state rather than a backlog.

func (*Subscription) C

func (s *Subscription) C() <-chan struct{}

C says something may be waiting for Take.

func (*Subscription) Close

func (s *Subscription) Close()

Close stops watching. When the last one goes, the flusher stops too.

func (*Subscription) Take

func (s *Subscription) Take() Update

Take returns what changed since the last Take.

type Summary

type Summary struct {
	Active  int     `json:"active"` // running or paused
	Queued  int     `json:"queued"`
	Paused  int     `json:"paused"`
	Percent float64 `json:"percent"` // across what can be measured, or -1
}

Summary is the state of things in a few numbers, for the button.

func Summarize

func Summarize(list []Task) Summary

Summarize counts up a list of tasks. Each task weighs the same: a scan of ten files and one of ten thousand each move the ring by their share of the way.

type Task

type Task struct {
	ID           string    `json:"id"`
	Type         string    `json:"type"`
	Title        string    `json:"title"`
	State        State     `json:"state"`
	Phase        string    `json:"phase,omitempty"`
	Steps        []string  `json:"steps,omitempty"` // the phases it goes through, in order
	Step         int       `json:"step,omitempty"`  // the one it's on, from 1 (0: none), and then Percent is across them all
	LibraryID    int64     `json:"libraryId,omitempty"`
	LibraryName  string    `json:"libraryName,omitempty"`
	Item         string    `json:"item,omitempty"` // the thing being worked on, for admins
	Path         string    `json:"path,omitempty"` // where it is, for admins
	Done         int64     `json:"done"`
	Total        int64     `json:"total"` // 0: not known (yet)
	Unit         string    `json:"unit,omitempty"`
	Percent      float64   `json:"percent"`        // 0 to 100, or -1 when there's no telling
	Rate         float64   `json:"rate,omitempty"` // units per second, smoothed
	ETASec       int64     `json:"etaSec,omitempty"`
	StartedAt    int64     `json:"startedAt"`
	UpdatedAt    int64     `json:"updatedAt"`
	FinishedAt   int64     `json:"finishedAt,omitempty"`
	ByID         int64     `json:"byId,omitempty"`
	By           string    `json:"by,omitempty"`
	Source       string    `json:"source,omitempty"`       // the module reporting it; empty for the server
	Key          string    `json:"key,omitempty"`          // a module's task: "<module id>/<its key>"
	Result       string    `json:"result,omitempty"`       // what came of it, in a sentence
	Error        string    `json:"error,omitempty"`        // what went wrong, for a person
	Fix          string    `json:"fix,omitempty"`          // what they can do about it
	Detail       string    `json:"detail,omitempty"`       // the error as it came, for admins and reports
	ProblemCount int       `json:"problemCount,omitempty"` // every problem, including those not kept
	Problems     []Problem `json:"problems,omitempty"`

	CanPause  bool `json:"canPause"`
	CanResume bool `json:"canResume"`
	CanCancel bool `json:"canCancel"`
	CanRetry  bool `json:"canRetry"`

	// Private tasks are about one person's activity: only they and admins see them.
	Private bool `json:"private,omitempty"`
	// RetryKey is what a retry handler needs to start this again.
	RetryKey string `json:"-"`
	// AdminOnly: never shown to anyone else, whatever the settings say.
	AdminOnly bool `json:"-"`
}

Task is a task as anyone outside this package sees it: a copy, safe to keep.

type Update

type Update struct {
	Tasks []Task
	Gone  []string
}

Update is what changed. Tasks holds changed tasks whole, including finished ones; Gone lists tasks to forget.

type Viewer

type Viewer struct {
	Admin  bool
	UserID int64
	// Library says whether they may see a library. Nil means none.
	Library func(id int64) bool
}

Viewer is who's asking.

func (Viewer) For

func (v Viewer) For(t Task) (Task, bool)

For returns a task as this viewer may see it, and whether they may see it at all. A non-admin never gets a file name or path, anyone else's name, the words of an error (they often hold paths), or any control.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL