tasks

package
v0.7.35 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 9, 2026 License: BSD-3-Clause, BSD-3-Clause Imports: 14 Imported by: 0

Documentation

Overview

Package tasks provides a framework to asynchronously run cancellable tasks with simplified progress tracking.

The most important type of task is the pipeline, which allows the composition of individual tasks, whose inputs and outputs can be connected using edges.

This system is mostly designed for the GUI.

My(darwin) justification for this complicated approach is that the usual pipeline composition patterns make progress tracking, cancellation and error handling quite manual and error-prone. I think this approach strikes a fair middle-ground between ultra-manual code and drifting too deep into DSL/embedded langauge territory.

Index

Constants

This section is empty.

Variables

View Source
var Tasks = struct {
	// Params:
	//   dest (string): destination path (only present if successful)
	//   url (string): download URL
	//   perms (int, optional): unix perms for downloaded file
	//   progressStatus (bool, optional): write progress in mebibytes to status
	// Results: none
	Download TaskFunc
	// Params:
	//   path (string): path to archive
	//   dest (string): destination folder
	//   stripFirstDir (bool, optional): strip first directory from contained file paths
	// Results:
	//   files ([]string): extracted file names
	Unarchive TaskFunc
}{
	Download: func(ctx context.Context, params map[string]any, onProgress func(prog float64), onStatus func(string)) (_ map[string]any, err error) {
		dest := params["dest"].(string)
		url := params["url"].(string)
		perms, ok := params["perms"].(int)
		if !ok {
			perms = 0666
		}
		progressStatus := params["progressStatus"] == true

		onStatus("Fetching")
		req, err := http.NewRequestWithContext(ctx, "GET", url, nil)
		if err != nil {
			return nil, err
		}
		resp, err := http.DefaultClient.Do(req)
		if err != nil {
			return nil, err
		}
		defer resp.Body.Close()

		f, err := os.OpenFile(dest, os.O_RDWR|os.O_CREATE|os.O_TRUNC, os.FileMode(perms))
		if err != nil {
			return nil, err
		}
		defer f.Close()
		defer func() {
			if err != nil {
				os.Remove(dest)
			}
		}()

		if resp.StatusCode != http.StatusOK {
			return nil, fmt.Errorf("get %v: %v", url, resp.Status)
		}

		onStatus("Downloading")
		currentBytes := 0
		var buf [65536]byte
		for {
			if err := ctx.Err(); err != nil {
				return nil, err
			}

			n, err := resp.Body.Read(buf[:])
			if err != nil {
				if err == io.EOF {
					break
				}
				return nil, err
			}

			_, err = f.Write(buf[:n])
			if err != nil {
				return nil, err
			}

			if resp.ContentLength > 0 {
				if progressStatus {
					const mebi = 1 << 20
					onStatus(fmt.Sprintf("Downloading (%.1f/%.1f MiB)", float64(currentBytes)/mebi, float64(resp.ContentLength)/mebi))
				}
				onProgress(float64(currentBytes) / float64(resp.ContentLength))
			}
			currentBytes += n
		}
		if err := f.Sync(); err != nil {
			return nil, err
		}
		return nil, nil
	},
	Unarchive: func(ctx context.Context, params map[string]any, onProgress func(prog float64), onStatus func(string)) (_ map[string]any, err error) {
		arPath := params["path"].(string)
		dest := params["dest"].(string)
		stripFirstDir := params["stripFirstDir"] == true

		var arEx archives.Extractor
		{
			var ok bool
			arFormat, _, err := archives.Identify(ctx, arPath, nil)
			if err != nil {
				return nil, err
			}
			arEx, ok = arFormat.(archives.Extractor)
			if !ok {
				return nil, errors.New("unable to extract archive")
			}
		}

		limitedExponential := func(x, lambda float64) float64 {
			return 1 - math.Pow(2, -x/lambda)
		}

		var extractedFileNames []string
		arR, err := os.Open(arPath)
		if err != nil {
			return nil, err
		}
		defer arR.Close()
		if err := arEx.Extract(ctx, arR, func(ctx context.Context, fInfo archives.FileInfo) error {
			if err := ctx.Err(); err != nil {
				return err
			}

			if !fInfo.Mode().IsRegular() {
				return nil
			}
			f, err := fInfo.Open()
			if err != nil {
				return err
			}
			defer f.Close()
			nameInArchive := filepath.Clean(fInfo.NameInArchive)
			if stripFirstDir {
				i := strings.IndexAny(nameInArchive, "/\\")
				if i != -1 {
					nameInArchive = nameInArchive[i+1:]
				}
			}
			path := filepath.Clean(filepath.Join(dest, nameInArchive))
			if !strings.HasPrefix(path, dest) {
				return nil
			}
			if err := os.MkdirAll(filepath.Dir(path), os.ModePerm); err != nil {
				return err
			}
			out, err := os.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, fInfo.Mode())
			if err != nil {
				return err
			}
			defer out.Close()
			if _, err := io.Copy(out, f); err != nil {
				return err
			}
			extractedFileNames = append(extractedFileNames, nameInArchive)
			onProgress(limitedExponential(float64(len(extractedFileNames)), 5))
			return nil
		}); err != nil {
			return nil, err
		}
		return map[string]any{"files": extractedFileNames}, nil
	},
}

Collection of some useful predefined tasks.

Functions

This section is empty.

Types

type TaskExecution

type TaskExecution struct {
	// contains filtered or unexported fields
}

func NewErroredTaskExecution

func NewErroredTaskExecution(err error) *TaskExecution

Returns a new TaskExecution, which has just finished with the given error.

func (*TaskExecution) Cancel added in v0.7.35

func (ex *TaskExecution) Cancel()

Cancels the underlying cancellable context.

Does not need to be called manually after the task has finished.

Does nothing if ex is nil. May be called multiple times and from any goroutine.

func (*TaskExecution) CancelCause added in v0.7.35

func (ex *TaskExecution) CancelCause(cause error)

Like TaskExecution.Cancel, but with an explicit cause.

func (*TaskExecution) Poll added in v0.7.35

func (ex *TaskExecution) Poll() TaskState

Returns a snapshot of the current execution status.

If ex is nil, returns a TaskExecution with Done set to true and all other fields zeroed.

func (*TaskExecution) SoftPoll added in v0.7.35

func (ex *TaskExecution) SoftPoll() TaskState

Like TaskExecution.Poll, but doesn't reset JustFinished.

type TaskFunc

type TaskFunc func(ctx context.Context, params map[string]any, onProgress func(prog float64), onStatus func(string)) (result map[string]any, err error)

func Pipeline added in v0.7.35

func Pipeline(args ...any) TaskFunc

args is a list of arbitrary arguments, which are interpreted as a declarative language.

An arg of type TaskFunc (or the underlying func type) is added as a subtask, necessarily preceded by a "<name>##<id>##<weight>" string, where <name> is an optional string describing the task status prefix, <id> is a unique ID to identify the task and <weight> is an optional float (default 1), which describes how significant the progress on this task is to the overall progress (weight is automatically distributed between subtasks, so a weight sum of more than one is valid and encouraged).

A variable is an input or result value to the whole task (e.g. "url", "path"), or an input or result value of a subtask (e.g. "dl.url", "dl.numFiles", where "dl" is the task ID).

Other args specify edges from one variable to another: A value arg followed by "-> <var>" (where <var> is the variable) assigns a constant value to a variable. "<var> -> <var>" connects one variable to another (e.g. "downloader.output -> extractor.input"). A variable without a "." is considered an input/output to the resulting pipeline.

Quick reference: - "<name>##<id>##<weight>", TaskFunc: add subtask - val, "-> <var>": assign constant - "<var> -> <var>": connect variables

func (TaskFunc) Go

func (t TaskFunc) Go(ctx context.Context, params map[string]any) *TaskExecution

Runs the task asynchronously (with polling).

func (TaskFunc) GoCh added in v0.7.35

func (t TaskFunc) GoCh(outCh chan<- TaskState, ctx context.Context, params map[string]any) *TaskExecution

Runs the task asynchronously with the given channel. A snapshot of the state is sent on every update. The channel is closed after the task has completed.

func (TaskFunc) Run added in v0.7.35

func (t TaskFunc) Run(ctx context.Context, params map[string]any) TaskState

Runs the task synchronously.

type TaskState added in v0.7.35

type TaskState struct {
	// TaskFunc used to run the task.
	Fn TaskFunc
	// Status string.
	Status string
	// Progress in range [0,1].
	Prog float64
	// True when the task has finished (regardless of success).
	Done bool
	// Set to true for one call of [TaskExecution.Poll] just
	// after the task has finished, regardless of success
	// (use [TaskExecution.SoftPoll] to not affect this field
	// when polling).
	JustFinished bool
	// Task result and error (only set after task has finished).
	// Err is set to [context.Canceled] if the task was canceled
	// prematurely.
	Res map[string]any
	Err error
}

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL