dag

package
v0.21.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 15, 2020 License: MPL-2.0 Imports: 8 Imported by: 2

README 

= Directed Acyclic Graph Build System

Lightweight Directed Acyclic Graph (DAG) Build System.

It allows building a list of tasks and then running the tasks in different DAG trees.

The tree dependencies are calculated and tasks that have met their dependencies are run in parallel.
There is an option to run it serially for cases where user interaction is required.

== Usage

A detailed example can be found in link:../examples/dag/main.go[]

To see what the build system would do run:

----
$ go run main.go build --quiet --dot

digraph G {
        label = "build graph";
        rankdir = TB;
        "bt3";
        "bt1";
        "bt3" -> "bt1";
        "bt2";
        "bt3" -> "bt2";
}
----

----
$ go run main.go clean --quiet --dot

digraph G {
        label = "clean graph";
        rankdir = TB;
        "ct1";
        "ct3";
        "ct1" -> "ct3";
        "ct2";
        "ct2" -> "ct3";
}
----

----
$ go run main.go
SYNOPSIS:
    main [--dot] [--help|-?] [--quiet] <command> [<args>]

COMMANDS:
    build    build project artifacts
    clean    clean project artifacts
    help     Use 'main help <command>' for extra details.

OPTIONS:
    --dot        Generate graphviz dot diagram (default: false)

    --help|-?    (default: false)

    --quiet      (default: false)

Use 'main help <command>' for extra details.
exit status 1
----

== Motivation

I want something better than Bash scripts and Makefiles but not as limiting as Bazel, Buck and Please (never tried Pants but they are all inspired by Blaze).
Scons and Gradle are hard to work with.

Mage is awesome but the first thing I saw a developer do with it was run it in a debugger which is not possible with the current design.
Mage builds a separate go file that it then compiles and runs. It also relies on panic/recover for its control flow.

This build system leverages go-getoptions and a DAG tree to provide an alternative.

== Build System

In order to have a build system you need a couple of pieces:

* Define task dependencies (This package takes care of that).
It builds a tree for you that it then runs in parallel when possible.

* Define target and sources dependencies.
In other words, if my sources have changed I need to rebuild my targets.
(That is in the works in another place, in the meanwhile use mage's https://github.com/magefile/mage/blob/master/target/target.go[target package].

* Task idempotency.
This so you can run your build system tasks over and over without risk.
This one is on you!

== License

This file is part of go-getoptions.

Copyright (C) 2015-2020  David Gamba Rios

This Source Code Form is subject to the terms of the Mozilla Public
License, v. 2.0. If a copy of the MPL was not distributed with this
file, You can obtain one at http://mozilla.org/MPL/2.0/.

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var ErrorGraphHasCycle = fmt.Errorf("graph has a cycle")
View Source 
var ErrorSkipParents = fmt.Errorf("skip parents without failing")

ErrorSkipParents - Allows for conditional tasks that allow a task to Skip all parent tasks without failing the run

View Source 
var ErrorTaskDependencyDuplicate = fmt.Errorf("task dependency already defined")
View Source 
var ErrorTaskDuplicate = fmt.Errorf("task definition already exists")
View Source 
var ErrorTaskFn = fmt.Errorf("missing task function")
View Source 
var ErrorTaskID = fmt.Errorf("missing task ID")
View Source 
var ErrorTaskNil = fmt.Errorf("nil task given")
View Source 
var ErrorTaskNotFound = fmt.Errorf("task not found")
View Source 
var ErrorTaskSkipped = fmt.Errorf("skipped")
View Source 
var Logger = log.New(os.Stderr, "", log.LstdFlags)

Functions

This section is empty.

Types

type Errors 

type Errors struct {
	Msg    string
	Errors []error
}

func (*Errors) Error 

func (errs *Errors) Error() string

type Graph 

type Graph struct {
	Name           string
	TickerDuration time.Duration
	Vertices       map[ID]*Vertex
	// contains filtered or unexported fields
}

func NewGraph 

func NewGraph(name string) *Graph

NewGraph - Create a graph that allows running Tasks in parallel.

func (*Graph) AddTask 

func (g *Graph) AddTask(t *Task)

AddTask - Add Task to graph. Errors collected for Graph.Validate().

func (*Graph) DephFirstSort 

func (g *Graph) DephFirstSort() ([]*Vertex, error)

DephFirstSort - Returns a sorted list with the Vertices https://en.wikipedia.org/wiki/Topological_sorting#Depth-first_search It returns ErrorGraphHasCycle is there are cycles.

func (*Graph) Run 

func (g *Graph) Run(ctx context.Context, opt *getoptions.GetOpt, args []string) error

Run - Execute the graph tasks in parallel where possible. It checks for tasks updates every 1 Millisecond by default. Modify using the graph.TickerDuration

func (*Graph) SetSerial 

func (g *Graph) SetSerial() *Graph

SetSerial - The call to Run() will run the Tasks serially. Useful when the tasks require user input and the user needs to see logs in order to make a decision.

func (*Graph) String 

func (g *Graph) String() string

String - Returns a dot diagram of the graph.

func (*Graph) Task 

func (g *Graph) Task(id string) *Task

Task - Retrieve a Task from the graph by its ID.

func (*Graph) TaskDependensOn 

func (g *Graph) TaskDependensOn(t *Task, tDependencies ...*Task)

TaskDependensOn - Allows adding tasks to the graph and defining their edges.

func (*Graph) Validate 

func (g *Graph) Validate(tm *TaskMap) error

Validate - Verifies that there are no errors in the Graph. It also runs Validate() on the given TaskMap (pass nil if a TaskMap wasn't used).

type ID 

type ID string

type Task 

type Task struct {
	ID ID
	Fn getoptions.CommandFn
	// contains filtered or unexported fields
}

func NewTask 

func NewTask(id string, fn getoptions.CommandFn) *Task

NewTask - Allows creating a reusable Task that can be passed to multiple graphs.

func (*Task) Lock 

func (t *Task) Lock()

func (*Task) Unlock 

func (t *Task) Unlock()

type TaskMap 

type TaskMap struct {
	// contains filtered or unexported fields
}

func NewTaskMap 

func NewTaskMap() *TaskMap

NewTaskMap - Map helper to group multiple Tasks.

func (*TaskMap) Add 

func (tm *TaskMap) Add(id string, fn getoptions.CommandFn) *Task

Add - Adds a new task to the TaskMap. Errors collected for TaskMap.Validate().

func (*TaskMap) Get 

func (tm *TaskMap) Get(id string) *Task

Get - Gets the task from the TaskMap, if not found returns a new empty Task. Errors collected for TaskMap.Validate().

func (*TaskMap) Validate 

func (tm *TaskMap) Validate() error

Validate - Verifies that there are no errors in the TaskMap.

type Vertex 

type Vertex struct {
	ID       ID
	Task     *Task
	Children []*Vertex
	Parents  []*Vertex
	// contains filtered or unexported fields
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.