fab

package module
v0.20.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 5, 2022 License: MIT Imports: 30 Imported by: 0

README

Fab - software fabricator

Go Reference Go Report Card Tests Coverage Status

This is fab, a system for orchestrating software builds. It’s like Make, except that you express build rules and dependencies as Go code.

(But that doesn’t mean it’s for building Go programs only, any more than writing shell commands in a Makefile means Make builds only shell programs.)

Running fab on one or more targets ensures that the targets’ prerequisites, and the targets themselves, are up to date according to your build rules, while avoiding unnecessarily rebuilding any target that is already up to date.

Usage

You create a package of Go code in your project. By default fab looks for it in the fab.d subdir of your top-level directory. Every exported symbol in that package whose type satisfies the fab.Target interface is a target that fab can run.

For example, if you write this in fab.d/build.go:

package any_name_you_like

import (
  "os"

  "github.com/bobg/fab"
)

var (
  Build = &fab.Command{Shell: "go build ./..."}
  Vet   = &fab.Command{Shell: "go vet ./...", Stdout: os.Stdout}
  Test  = &fab.Command{Shell: "go test -race -cover ./...", Stdout: os.Stdout}
  Check = fab.All(Vet, Test)
)

then you can run fab Build, fab Check, etc. in the shell.

To express a dependency between targets, use the Deps construct:

// MyTarget ensures that pre1, pre2, etc. are built before post
// (each of which is some form of Target).
var MyTarget = fab.Deps(post, pre1, pre2, ...)

Alternatively, you can define your own type satisfying the Target interface, and express dependencies by calling the Run function in your type’s Run method:

type myTargetType struct {
  dependencies []fab.Target
  id           string
}

func (tt *myTargetType) Run(ctx, context.Context) error {
  if err := fab.Run(ctx, tt.dependencies...); err != nil {
    return err
  }
  // ...other myTargetType build logic...
}

// Each instance of any Target type must have a persistent, distinct ID.
// The fab.ID function can help with this.
func (tt *myTargetType) ID() string {
  if tt.id == "" {
    tt.id = fab.ID("MyTargetType”)
  }
  return tt.id
}

Fab ensures that no target runs more than once during a build, no matter how many times that target shows up in other targets’ dependencies or calls to Run, etc.

Details

By default, your build rules are found in the fab.d subdir. Running fab combines your rules with its own main function to produce a driver, which lives in $HOME/.fab by default. (These defaults can be overridden.)

When you run fab and the driver is already present and up to date (as determined by a hash of the code in the fab.d dir), then fab simply executes the driver without rebuilding it.

The directory $HOME/.fab also contains a hash database to tell when certain targets - those satisfying the HashTarget interface - are up to date and do not need rebuilding. When a HashTarget runs, it first computes a hash representing the complete state of the target - all inputs, outputs, and build rules. If that hash is in the database, the target is considered up to date and fab skips the build rule. (Otherwise, the build rule runs, and the hash is recomputed and added to the database.) This approach is preferable to using file modification times (like Make does, for example) to know when a target is up to date. Those aren’t always sufficient for this purpose, nor are they entirely reliable, considering the limited resolution of filesystem timestamps, the possibility of clock skew, etc.

Installation

Fab requires Go 1.19 or later. Download Go here.

Once a suitable version of Go is available (you can check by running go version), install Fab with:

go install github.com/bobg/fab/cmd/fab@latest

Why not Mage?

Fab was strongly inspired by the excellent Mage tool, which works similarly and has a similar feature set. But Fab has some features the author needed and did not find in Mage:

  • Errors from Target rules propagate out instead of causing an exit.
  • Targets are values, not functions, and are composable (e.g. with Seq, All, and Deps).
  • Rebuilding of up-to-date targets can be skipped based on file contents, not modtimes.

Documentation

Index

Constants

This section is empty.

Variables

View Source
var DefaultRunner = NewRunner()

DefaultRunner is a Runner used by default in Run.

Functions

func Compile added in v0.8.0

func Compile(ctx context.Context, pkgdir, binfile string) error

Compile compiles a "driver" from a directory of user code (combined with a main function supplied by fab) and places the executable result in a given file. The driver converts command-line target names into the necessary Fab rule invocations.

The package of user code should contain one or more exported identifiers whose types satisfy the Target interface. These become the build rules that the driver can invoke.

When Compile runs the "go" program must exist in the user's PATH. It must be Go version 1.19 or later.

How it works:

  • The user's code is loaded with packages.Load.
  • The set of exported top-level identifiers is filtered to find those implementing the fab.Target interface.
  • The user's code is then copied to a temp directory together with a main package (and main() function) that records the set of targets.
  • The go compiler is invoked to produce an executable, which is renamed into place as binfile.

func GetForce added in v0.16.0

func GetForce(ctx context.Context) bool

GetForce returns the value of the force boolean added to `ctx` with WithForce. The default, if WithForce was not used, is false.

func GetNames added in v0.6.0

func GetNames(ctx context.Context) map[uintptr]string

GetNames returns the map[uintptr]string added to `ctx` with WithNames. The default, if WithNames was not used, is nil.

func GetVerbose

func GetVerbose(ctx context.Context) bool

GetVerbose returns the value of the verbose boolean added to `ctx` with WithVerbose. The default, if WithVerbose was not used, is false.

func ID

func ID(prefix string) string

ID produces an ID string by appending a unique counter value to the given prefix. For example, ID("Foo") might produce "Foo-17".

func Indentf added in v0.9.0

func Indentf(ctx context.Context, format string, args ...any)

Indentf calls Runner.Indent with the given format and args if a Runner can be found in the given context. If one cannot, then the formatted string is simply printed (with a trailing newline added if needed).

func Name added in v0.6.0

func Name(ctx context.Context, target Target) string

Name returns a name for `target`. Normally this is just target.ID(). But if `ctx` has been decorated with a name map using WithNames and target's address is in it, then that name is used instead.

func Run

func Run(ctx context.Context, targets ...Target) error

Run runs the given targets with a Runner. If `ctx` contains a Runner (e.g., because this call is nested inside a pending call to Runner.Run and the context has been decorated using WithRunner) then it uses that Runner, otherwise it uses DefaultRunner.

func ToRelPath added in v0.20.0

func ToRelPath(pkgdir string) (string, error)

ToRelPath converts a Go package directory to a relative path beginning with ./ (suitable for use in a call to packages.Load, for example). It is an error for pkgdir to be outside the current working directory's tree.

func WithForce added in v0.16.0

func WithForce(ctx context.Context, force bool) context.Context

WithForce decorates a context with the value of a "force" boolean. Retrieve it with GetForce.

func WithHashDB

func WithHashDB(ctx context.Context, db HashDB) context.Context

WithHashDB decorates a context with a HashDB. Retrieve it with GetHashDB.

func WithNames added in v0.6.0

func WithNames(ctx context.Context, names map[uintptr]string) context.Context

WithNames decorates a context with a map[uintptr]string. The keys are the addresses of Target objects and the values are their "pretty names." When available, these are shown to the user at runtime in preference to the value of Target.ID.

func WithRunner

func WithRunner(ctx context.Context, r *Runner) context.Context

WithRunner decorates a context with a Runner. Retrieve it with GetRunner.

func WithVerbose

func WithVerbose(ctx context.Context, verbose bool) context.Context

WithVerbose decorates a context with the value of a "verbose" boolean. Retrieve it with GetVerbose.

Types

type Clean added in v0.18.0

type Clean struct {
	Files []string
	// contains filtered or unexported fields
}

Clean is a Target that deletes the files named in Files when it runs. Files that don't exist are silently ignored.

func (*Clean) ID added in v0.18.0

func (c *Clean) ID() string

ID implements Target.ID.

func (*Clean) Run added in v0.18.0

func (c *Clean) Run(_ context.Context) error

Run implements Target.Run.

type Command

type Command struct {
	// Shell is the command to run,
	// as a single string with command name and arguments together.
	// It is parsed as if by a Unix shell,
	// with quoting and so on,
	// in order to produce the command name
	// and a list of individual argument strings.
	//
	// To bypass this parsing behavior,
	// you may specify Cmd and Args directly.
	Shell string `json:"shell,omitempty"`

	// Cmd is the command to invoke,
	// either the path to a file,
	// or an executable file found in some directory
	// named in the PATH environment variable.
	//
	// Leave Cmd blank and specify Shell instead
	// to get shell-like parsing of a command and its arguments.
	Cmd string `json:"cmd,omitempty"`

	// Args is the list of command-line arguments
	// to pass to the command named in Cmd.
	Args []string `json:"args,omitempty"`

	// Stdout and Stderr tell where to send the command's output.
	// If either or both is nil,
	// that output is saved in case the subprocess encounters an error.
	// Then the returned error is a CommandErr containing that output.
	Stdout io.Writer `json:"-"`
	Stderr io.Writer `json:"-"`

	// Dir is the directory in which to run the command.
	// The default is the value of GetDir(ctx) when the Run method is called.
	Dir string `json:"dir,omitempty"`

	// Env is a list of VAR=VALUE strings to add to the environment when the command runs.
	Env []string `json:"env,omitempty"`
	// contains filtered or unexported fields
}

Command is a target whose Run function executes a command in a subprocess.

func (*Command) ID

func (c *Command) ID() string

ID implements Target.ID.

func (*Command) Run

func (c *Command) Run(ctx context.Context) error

Run implements Target.Run.

type CommandErr

type CommandErr struct {
	Err    error
	Output []byte
}

CommandErr is a type of error that may be returned from Command.Run. If the Command's Stdout or Stderr field was nil, then that output from the subprocess is in CommandErr.Output and the underlying error is in CommandErr.Err.

func (CommandErr) Error

func (e CommandErr) Error() string

Error implements error.Error.

func (CommandErr) Unwrap

func (e CommandErr) Unwrap() error

Unwrap produces the underlying error.

type FilesTarget added in v0.13.0

type FilesTarget struct {
	Target
	In  []string
	Out []string
}

FilesTarget is a HashTarget. It contains a list of input files, and a list of expected output files. It also contains an embedded Target whose Run method should produce the expected output files.

The FilesTarget's hash is computed from the target and all the input and output files. If none of those have changed since the last time the output files were built, then the output files are up to date and running of this FilesTarget can be skipped.

The Target must be of a type that can be JSON-marshaled.

The In list should mention every file where a change should cause a rebuild. Ideally this includes any files required by the Target's Run method, plus any transitive dependencies. See the deps package for helper functions that can compute dependency lists of various kinds.

func (FilesTarget) Hash added in v0.13.0

func (ft FilesTarget) Hash(ctx context.Context) ([]byte, error)

Hash implements HashTarget.Hash.

type HashDB

type HashDB interface {
	// Has tells whether the database contains the given entry.
	Has(context.Context, []byte) (bool, error)

	// Add adds an entry to the database.
	Add(context.Context, []byte) error
}

HashDB is the type of a database for storing hashes. It must permit concurrent operations safely. It may expire entries to save space.

func GetHashDB

func GetHashDB(ctx context.Context) HashDB

GetHashDB returns the value of the HashDB added to `ctx` with WithHashDB. The default, if WithHashDB was not used, is nil.

type HashTarget

type HashTarget interface {
	Target
	Hash(context.Context) ([]byte, error)
}

HashTarget is a Target that knows how to produce a hash (or "digest") representing the complete state of the target: the inputs, the outputs, and the rules for turning one into the other. Any change in any of those should produce a distinct hash value.

When a HashTarget is executed by Runner.Run, its Run method is skipped and it succeeds trivially if the Runner can determine that the outputs are up to date with respect to the inputs and build rules. It does this by consulting a HashDB that is populated with the hashes of HashTargets whose Run methods succeeded in the past.

Using such a hash to decide whether a target's outputs are up to date is preferable to using file modification times (like Make does, for example). Those aren't always sufficient for this purpose, nor are they entirely reliable, considering the limited resolution of filesystem timestamps, the possibility of clock skew, etc.

type Main added in v0.11.0

type Main struct {
	// Pkgdir is where to find the user's build-rules Go package, e.g. "fab.d".
	Pkgdir string

	// Fabdir is where to find the user's hash DB and compiled binaries, default $HOME/.fab.
	Fabdir string

	// Verbose tells whether to run the driver in verbose mode
	// (by supplying the -v command-line flag).
	Verbose bool

	// List tells whether to run the driver in list-targets mode
	// (by supplying the -list command-line flag).
	List bool

	// Force tells whether to force recompilation of the driver before running it.
	Force bool

	// Args contains the additional command-line arguments to pass to the driver, e.g. target names.
	Args []string
}

Main is the structure whose Run methods implements the main logic of the fab command.

func (Main) Run added in v0.11.0

func (m Main) Run(ctx context.Context) error

Run executes the main logic of the fab command. If a driver binary with the right dirhash does not exist in m.Fabdir, or if m.Force is true, it is created with Compile. It is then invoked with the command-line arguments indicated by the fields of m. Typically this will include one or more target names, in which case the driver will execute the associated rules as defined by the code in m.Pkgdir.

type Runner

type Runner struct {
	// contains filtered or unexported fields
}

Runner is an object that knows how to run Targets without ever running the same Target twice.

A zero runner is not usable. Use NewRunner to obtain one instead.

func GetRunner

func GetRunner(ctx context.Context) *Runner

GetRunner returns the value of the Runner added to `ctx` with WithRunner. The default, if WithRunner was not used, is nil.

func NewRunner

func NewRunner() *Runner

NewRunner produces a new Runner.

func (*Runner) Indentf added in v0.8.0

func (r *Runner) Indentf(format string, args ...any)

Indentf formats and prints its arguments with leading indentation based on the nesting depth of the Runner. The nesting depth increases with each call to Runner.Run and decreases at the end of the call.

A newline is added to the end of the string if one is not already there.

func (*Runner) Run

func (r *Runner) Run(ctx context.Context, targets ...Target) error

Run runs the given targets, skipping any that have already run.

A Runner remembers which targets it has already run (whether in this call or any previous call to Run), distinguishing them by their ID() values.

The targets are executed concurrently. A separate goroutine is created for each one passed to Run. If the Runner has never yet run the Target, it does so, and caches the result (error or no error). If the Target did already run, the cached error value is used. If another goroutine concurrently requests the same Target, it blocks until the first one completes, then uses the first one's result.

As a special case, if the Target is a HashTarget and there is a HashDB attached to the context, then the HashTarget's hash is computed and looked up in the HashDB. If it's found, the target's outputs are already up to date and its Run method can be skipped. Otherwise, Run is invoked and (if it succeeds) a new hash is computed for the target and added to the HashDB.

This function waits for all goroutines to complete. The return value may be an accumulation of multiple errors. These can be retrieved with multierr.Errors.

The runner is added to the context with WithRunner and can be retrieved with GetRunner. Calls to Run (the global function, not the Runner.Run method) will use it instead of DefaultRunner by finding it in the context.

type Target

type Target interface {
	// Run invokes the target's logic.
	//
	// Callers generally should not invoke a target's Run method.
	// Instead, pass the target to a Runner's Run method,
	// or to the global Run function.
	// That will handle concurrency properly
	// and make sure that the target is not rerun
	// when it doesn't need to be.
	Run(context.Context) error

	// ID is a unique ID for the target.
	// Each instance of each Target must have a persistent, unique ID.
	// The ID function can help with that.
	ID() string
}

Target is the interface that Fab targets must implement.

func All added in v0.3.0

func All(targets ...Target) Target

All produces a target that runs a collection of targets in parallel.

func Deps added in v0.3.0

func Deps(target Target, depTargets ...Target) Target

Deps wraps a target with a set of dependencies, making sure those run first.

It is equivalent to Seq(All(depTargets...), target).

func F

func F(f func(context.Context) error) Target

F produces a target whose Run function invokes the given function.

func Seq added in v0.4.0

func Seq(targets ...Target) Target

Seq produces a target that runs a collection of targets in sequence. Its Run method exits early when a target in the sequence fails.

Directories

Path Synopsis
cmd
fab command

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL