Documentation
¶
Overview ¶
Package progress reports what a command is doing while it runs.
The work is a tree of spans, one level for each kind of unit: the command, its steps, the batches of a staggered action, the targets a step works on, the calls made for a target, and the waits in between. A span travels in the context.Context the work is done under, so the span a call starts nests under the target it is made for without being handed down. Every change to a span is sent, as one Event of plain data, to the sinks of the Bus the context carries, in one total order. A counter, a live tree, an agent's progress notifications, an event log and a test are sinks; this package draws nothing itself, and writes only the event log, Log.
Without a Bus in the context, Start returns the context unchanged and a nil *Span, and every method of a nil *Span does nothing, so work that nobody watches pays only for the lookup.
The vocabulary is closed: an event carries the fields of Fields and nothing else, so an argument vector, a script, standard input or a header has nowhere to go. Text that came from elsewhere, an error or a remote line, is passed through Sanitize before any sink sees it.
The events map one to one onto a trace: the trace id is 16 bytes, a span id is 8, and each Bus draws the base of its span ids at random, so that runs sharing one trace do not share span ids. A Bus may continue a trace another program began, as ParseTraceContext reads it from a W3C traceparent.
Example ¶
A command attaches a Bus to its context and ends the spans it started; the library it calls reports its work under them.
package main
import (
"context"
"errors"
"fmt"
"github.com/GSI-HPC/go-clikit/progress"
)
// printer is a sink that prints each event on a line of its own.
type printer struct{}
func (printer) Handle(e progress.Event) {
fmt.Printf("%d %s %s %q %s", e.Seq, e.Type, e.Kind, e.Name, e.State)
if e.Type == progress.TypeEnd {
fmt.Printf(" %s", e.Status)
if e.Err != "" {
fmt.Printf(" (%s): %s", e.Class, e.Err)
}
}
fmt.Println()
}
// reset is library code: it reports its work as spans in ctx, whether a
// command watches or not.
func reset(ctx context.Context, nodes []string) error {
ctx, step := progress.Start(ctx, progress.KindStep, "reset the machines",
progress.WithFlags(progress.Fold), progress.Total(len(nodes)))
targets := make([]*progress.Span, len(nodes))
for i, node := range nodes {
_, targets[i] = progress.Start(ctx, progress.KindTarget, node, progress.Queued(), progress.Node(node))
}
var err error
for i, target := range targets {
target.Run()
if nodes[i] == "exe02" {
failed := errors.New("exe02: no answer")
target.End(failed)
err = failed
continue
}
target.End(nil)
}
step.End(err)
return err
}
func main() {
bus := progress.NewBus(progress.BusOptions{Sinks: []progress.Sink{printer{}}})
ctx, command := progress.Start(progress.WithBus(context.Background(), bus), progress.KindCommand, "power reset")
command.End(reset(ctx, []string{"exe01", "exe02"}))
bus.Close()
}
Output: 1 start command "power reset" running 2 start step "reset the machines" running 3 start target "exe01" queued 4 start target "exe02" queued 5 run target "exe01" running 6 end target "exe01" ended ok 7 run target "exe02" running 8 end target "exe02" ended failed (target): exe02: no answer 9 end step "reset the machines" ended failed (target): exe02: no answer 10 end command "power reset" ended failed (target): exe02: no answer
Index ¶
- Constants
- Variables
- func CountReader(ctx context.Context, r io.Reader) io.Reader
- func CountWriter(ctx context.Context, w io.Writer) io.Writer
- func Sanitize(s string, limit int) string
- func Skip(reason string) error
- func Suspend(ctx context.Context) (resume func())
- func Tee(ctx context.Context, w io.Writer, st Stream, parse func(Stream, string)) io.Writer
- func WithBus(ctx context.Context, b *Bus) context.Context
- type Bound
- type Bus
- type BusOptions
- type Class
- type Classifier
- type Count
- type Event
- type Fields
- type Flags
- type Kind
- type LineSink
- type Log
- type LogOptions
- type Meter
- type Option
- func Batch(i, n int) Option
- func Cache(s string) Option
- func Exit(code int) Option
- func HTTP(method, path string) Option
- func HTTPStatus(code int) Option
- func Host(s string) Option
- func Limit(n int) Option
- func Message(s string) Option
- func Node(s string) Option
- func Queued() Option
- func Role(s string) Option
- func Source(s string) Option
- func Timeout(d time.Duration) Option
- func Total(n int) Option
- func WithFlags(f Flags) Option
- func Work(u Unit, size int64) Option
- type Reading
- type Run
- type Sink
- type Span
- type SpanID
- type State
- type Status
- type Stream
- type Suspender
- type Tally
- type TraceContext
- type TraceID
- type TraceSink
- type Type
- type Unit
Examples ¶
Constants ¶
const ( // MaxText bounds the text of a line of output. MaxText = 512 // MaxErr bounds the error a span ended with. MaxErr = 1024 // MaxField bounds a span's name, its Message and every other text // field but Node, which is a node set a display folds and is never // cut. MaxField = 256 )
The bounds of the text an event carries, in bytes. Sanitize cuts to them.
const LogVersion = 1
LogVersion is the version of the event log's schema, the "v" of every line. It changes when what a key or a value means does, not when a key or a value is added: a reader leaves out the keys and the values it does not know.
Variables ¶
var ErrSkipped = errors.New("skipped")
ErrSkipped says that work was left out on purpose, such as a call a dry run only records, or the work for a node a command no longer tries once it found the node unreachable: not a failure. Span.End ends a span whose error is ErrSkipped, as errors.Is tells, skipped, and the pools of fanout do not count such an item among those that failed. Test for it with errors.Is, since the error Skip returns, and one that wraps it, say why instead.
errors.Is looks through every error an error joins, so one that joins a skip with a failure, by errors.Join or fmt.Errorf with two %w verbs, counts as a skip, and the failure goes uncounted. Work that gathers the errors of its parts returns a skip only when every part was skipped.
Functions ¶
func CountReader ¶ added in v0.3.0
CountReader returns a reader that reads from r and advances the innermost span in ctx by the bytes each read gave, a read that also returns an error included. It is meant for a span given Work(Bytes, ...), so that its amount is bytes; a span given no Work counts Items, as Advance does. The reader advances nothing once the span has ended, and a read returns what r returned. It returns r itself when ctx carries no span, which costs nothing.
func CountWriter ¶ added in v0.3.0
CountWriter returns a writer that writes to w and advances the innermost span in ctx by the bytes w took, from each call, a short write included. It is meant for a span given Work(Bytes, ...), so that its amount is bytes; a span given no Work counts Items, as Advance does. The writer advances nothing once the span has ended, and a write returns what w returned. It returns w itself when ctx carries no span, which costs nothing.
Example ¶
ExampleCountWriter copies an image through CountWriter under a call that gives its size. The Bus sends the amount when its place comes, and the call's End carries the newest.
package main
import (
"bytes"
"context"
"fmt"
"io"
"time"
"github.com/GSI-HPC/go-clikit/progress"
"github.com/GSI-HPC/go-clikit/progress/progresstest"
)
func main() {
now := time.Date(2026, 10, 10, 12, 0, 0, 0, time.UTC)
capture := &progresstest.Capture{}
bus := progress.NewBus(progress.BusOptions{
Sinks: []progress.Sink{capture},
Now: func() time.Time { return now },
})
ctx, step := progress.Start(progress.WithBus(context.Background(), bus), progress.KindStep, "copying the image")
ctx, call := progress.Start(ctx, progress.KindCall, "copy", progress.Work(progress.Bytes, 4<<20))
image := bytes.NewReader(make([]byte, 4<<20))
var dst bytes.Buffer
n, err := io.Copy(progress.CountWriter(ctx, &dst), image)
call.End(err)
step.End(err)
bus.Close()
for _, e := range capture.Events() {
if e.Kind == progress.KindCall {
fmt.Printf("%s %s %d/%d %s\n", e.Type, e.Name, e.Amount, e.Size, e.Unit)
}
}
fmt.Println("copied", n, "bytes")
}
Output: start copy 0/4194304 bytes advance copy 4194304/4194304 bytes end copy 4194304/4194304 bytes copied 4194304 bytes
func Sanitize ¶
Sanitize makes text from elsewhere, a remote line or an error, safe to show on one line of a terminal, in at most limit bytes; a limit of zero or less sets no bound.
A carriage return is first applied the way a terminal shows it: it goes back to the start of the line and what follows overwrites what was there, so "50%\r100%" reads "100%", "abcdef\r12" reads "12cdef", and a line that ends in "\r\n" loses nothing. What is left is escaped by termtext.Escape, the one escaper, which shows every other control character, an escape sequence's introducer, a bidirectional control, a newline and bytes that are not UTF-8 as a visible escape such as \x1b. The result is cut on a rune boundary.
The Bus applies it to every text of an event, each cut to its bound, MaxText, MaxErr or MaxField, and Node to none, so neither the work that reports a span nor a sink ever needs to call it. A program that shows text from elsewhere outside a Bus may call it to the same end.
Example ¶
Sanitize makes a line of output fit to show: a carriage return is applied as a terminal would, and what is left escaped and cut.
package main
import (
"fmt"
"github.com/GSI-HPC/go-clikit/progress"
)
func main() {
fmt.Println(progress.Sanitize("copying 50%\rcopying 100%\x1b[K", 0))
fmt.Println(progress.Sanitize("a line longer than it may be", 13))
}
Output: copying 100%\x1b[K a line longer
func Skip ¶
Skip returns an error that says work was left out on purpose, for the reason given: its text is reason, and errors.Is finds ErrSkipped in it. Span.End ends a span with it skipped, with reason as its Err, as Span.Skip does. An error that joins it with a failure counts as a skip too, as ErrSkipped says, so it is returned only when all the work was left out.
Example ¶
Work left out on purpose returns an error of Skip, which a span ends skipped rather than failed, whatever the program's rule for classes says, and which errors.Is tells from a failure.
package main
import (
"context"
"errors"
"fmt"
"github.com/GSI-HPC/go-clikit/progress"
)
// printer is a sink that prints each event on a line of its own.
type printer struct{}
func (printer) Handle(e progress.Event) {
fmt.Printf("%d %s %s %q %s", e.Seq, e.Type, e.Kind, e.Name, e.State)
if e.Type == progress.TypeEnd {
fmt.Printf(" %s", e.Status)
if e.Err != "" {
fmt.Printf(" (%s): %s", e.Class, e.Err)
}
}
fmt.Println()
}
func main() {
bus := progress.NewBus(progress.BusOptions{Sinks: []progress.Sink{printer{}}})
ctx := progress.WithBus(context.Background(), bus)
_, target := progress.Start(ctx, progress.KindTarget, "exe01")
err := fmt.Errorf("exe01: %w", progress.Skip("in maintenance"))
target.End(err)
bus.Close()
fmt.Println(errors.Is(err, progress.ErrSkipped))
}
Output: 1 start target "exe01" running 2 end target "exe01" ended skipped (none): exe01: in maintenance true
func Suspend ¶
Suspend takes every display off the terminal, for a question to be asked there, and returns once they are all off. Calling the function it returns puts them back, a display taken off the Bus in between for panicking too; calling that again does nothing. Suspensions may nest, and without a Bus in ctx nothing happens.
func Tee ¶
Tee returns a writer that writes to w, and cuts what it writes into lines on the way.
Each complete line goes to parse, when it is not nil, with its "\n" and a "\r" before it taken off, from the goroutine that writes; the tees of a command's two streams may call it at the same time. A parser sees only lines that ended: a line cut off by the end of the output, or longer than 64 KiB, is none it is given.
When the innermost span in ctx has ShowLines and a sink wants lines, they also go to the sinks as TypeLine events, sanitised and rate limited: a burst of 20, then one each 100ms. A line that finds no place waits for the next one, in place of the line that waited before it, which is counted as dropped; so the newest line is never lost, and is sent when its place comes, whether another line comes or not. A line that does not end is sent in pieces of 4 KiB, and what is left of it when the span ends is sent before the End. The End sends the line that waits, and what is left of the lines that did not end, though it finds no place for them: of those, the newest of each stream, so that the last line of stdout and that of stderr both reach the sinks. Until the End, the span keeps the line of a writer that has not ended, but neither the writer nor anything of one whose output ended with a line.
Tee returns w itself when nothing needs the lines. The lines never fail a write: what Write returns is what w returned.
Types ¶
type Bus ¶
type Bus struct {
// contains filtered or unexported fields
}
Bus hands the events of one command's spans to its sinks. It is safe for concurrent use: every event is numbered and delivered under one lock, so each sink sees the same order.
func (*Bus) Classify ¶
Classify tells why work failed from err as End does for the spans of the Bus that it does not end skipped: Classify with BusOptions.Classify as the fallback. A nil Bus has no fallback.
Example ¶
A library that runs work of its own learns from the Bus in its context what the program said of itself: its name, where panics go, and its rule for classes. Without a Bus the answers are the defaults.
package main
import (
"context"
"errors"
"fmt"
"github.com/GSI-HPC/go-clikit/progress"
)
// exitError is an error that asks a program to exit with a code.
type exitError struct{ code int }
func (e exitError) Error() string { return fmt.Sprintf("exit status %d", e.code) }
func main() {
byCode := func(err error) progress.Class {
if exit := (exitError{}); errors.As(err, &exit) && exit.code == 3 {
return progress.ClassTransport
}
return progress.ClassTarget
}
bus := progress.NewBus(progress.BusOptions{Program: "sind", Classify: byCode})
defer bus.Close()
for _, ctx := range []context.Context{
progress.WithBus(context.Background(), bus),
context.Background(),
} {
b := progress.BusFrom(ctx)
fmt.Printf("%q %s\n", b.Program(), b.Classify(exitError{3}))
}
}
Output: "sind" transport "" target
func (*Bus) Close ¶
func (b *Bus) Close()
Close ends every span still open as canceled, "not finished", the innermost first, and resumes a display still suspended, one taken off the Bus for panicking since too. Nothing is sent after Close; the sinks are not closed, which is for whoever made them to do once Close has returned. Closing a closed Bus does nothing.
func (*Bus) PanicLog ¶
PanicLog returns where the Bus writes the stack of a sink that panicked, BusOptions.PanicLog or the process's standard error, or nil for a nil Bus, so that a library that recovers its own goroutines can write their stacks to the same place.
func (*Bus) Program ¶
Program returns the program's name the Bus was given in BusOptions.Program, or "" for a nil Bus, so that a pool or a library that reports to the Bus of BusFrom names the program as the Bus does.
func (*Bus) TraceContext ¶
func (b *Bus) TraceContext() TraceContext
TraceContext returns the trace the Bus's spans belong to, and where it came from.
type BusOptions ¶
type BusOptions struct {
// Sinks receive every event, in the order given.
Sinks []Sink
// Now is the clock the events are stamped with; nil is time.Now. It
// must be safe for concurrent use: Advance and SetAmount read it on the
// goroutines that call them, without the Bus's lock, so a clock a test
// moves by hand guards its time with a mutex of its own.
Now func() time.Time
// Trace is the trace the spans belong to, as ParseTraceContext reads
// it from another program that handed it on. Its Parent, Flags and
// State say where the trace came from; they are recorded, for an
// event log, and change nothing else. A zero Trace.Trace draws a
// trace at random, and the rest of Trace is then ignored.
Trace TraceContext
// PanicLog receives the stack of a sink that panicked, the front
// end's diagnostics; nil is the process's standard error.
PanicLog io.Writer
// Program names the program in the line that says a sink panicked,
// "prog: …", so that it is not read as a line of the work's, and
// on the first line of an event log that names none of its own (see
// Run); empty leaves the name out.
Program string
// Classify is the fallback of the classes End gives the errors of
// spans, asked for an error that none of Classify's first three rules
// fits, such as a program's rule for its exit codes; nil, or an
// answer of ClassNone, is ClassTarget.
Classify func(error) Class
}
BusOptions configure a Bus.
type Class ¶
type Class uint8
Class says in a word why a span failed, for a display to group failures by. Classify tells it from the error.
The values defined here are the event log's vocabulary: a program maps its own failures onto them, through a Classifier or BusOptions.Classify. Values may be added in a minor release.
const ( // ClassNone is a span that did not fail. ClassNone Class = iota // ClassTarget is a target that answered and said no, such as a // command that exited non-zero. ClassTarget // ClassTransport is a peer that could not be reached, or whose // connection was lost. ClassTransport // ClassTimeout is work that ran out of time. ClassTimeout // ClassAuth is an account that was refused. ClassAuth // ClassPin is a peer whose identity does not match the one expected, // such as a pinned TLS certificate or a known ssh host key. ClassPin // ClassUsage is a request refused before anything was contacted. ClassUsage // ClassCanceled is work that was interrupted. ClassCanceled )
func Classify ¶
Classify tells why work failed from its error. The first rule that applies wins:
- the first error in the chain that is a Classifier, unless it answers ClassNone;
- context.Canceled is ClassCanceled;
- context.DeadlineExceeded, or an error that reports Timeout, such as a network timeout, is ClassTimeout;
- what fallback says of the error, the program's own rule, such as one that reads the exit code the error asks for, unless fallback is nil or answers ClassNone;
- ClassTarget.
A nil error is ClassNone, and fallback is not asked.
An error that sums up the failures of many targets is best a Classifier of its own, since the first of its targets' errors that says a class is no more the whole's than any other.
Example ¶
Classify tells why work failed; a program's own rule, here one for its exit codes, decides what the rules before it do not.
package main
import (
"context"
"errors"
"fmt"
"os"
"github.com/GSI-HPC/go-clikit/progress"
)
// exitError is an error that asks a program to exit with a code.
type exitError struct{ code int }
func (e exitError) Error() string { return fmt.Sprintf("exit status %d", e.code) }
func main() {
byCode := func(err error) progress.Class {
if exit := (exitError{}); errors.As(err, &exit) && exit.code == 3 {
return progress.ClassTransport
}
return progress.ClassTarget
}
for _, err := range []error{
nil,
context.Canceled,
fmt.Errorf("exe01: %w", context.DeadlineExceeded),
exitError{3},
exitError{1},
os.ErrPermission,
} {
fmt.Println(progress.Classify(err, byCode))
}
}
Output: none canceled timeout transport target target
type Classifier ¶
type Classifier interface {
// ProgressClass returns the error's class.
ProgressClass() Class
}
Classifier is an error that says its own Class, such as a certificate that does not match its pin or an account that was refused. ClassNone leaves the error to the rules of Classify.
type Count ¶
type Count struct {
// Span, Name and Flags are the step's or the batch's.
Span SpanID
Name string
Flags Flags
// Total is the span's Total, the targets it expects.
Total int
// Done counts the targets that ended below, however they ended, and
// the Total of every batch or Fold step below that was left out, each
// target once.
// Failed, Canceled and Skipped count those that ended so.
Done, Failed, Canceled, Skipped int
// Running counts the targets below that are running. Queued counts
// those that wait for their turn, and the Total of every batch below
// that waits for its own.
Running, Queued int
// Started counts the targets below that have started (TypeStart),
// whether queued, running or ended since; it only grows.
Started int
// Batch is the place, "i/n", of the batch below that ran last, or of
// the span itself when it is a batch.
Batch string
// Waits counts the waits open below, such as a pause between batches.
Waits int
}
Count is where the targets below a step or a batch that counts them stand. Fields may be added in a minor release.
type Event ¶
type Event struct {
// Seq numbers the events of a Bus from 1, with no gaps: the order in
// which every sink sees them.
Seq uint64
// Time is when the event happened, by the Bus's clock.
Time time.Time
// Type says what happened to the span.
Type Type
// Span is the span the event is about, and Parent the span it was
// started under; zero for the root. A TypeSuspend or TypeResume names
// the span it was asked for under, if any.
Span, Parent SpanID
// Kind is the level of the tree the span is on.
Kind Kind
// Name says what the span is, in few words: "ssh", "copy",
// "reset the machines", or a target's name.
Name string
// Flags are the span's flags, which say how displays show it.
Flags Flags
// State is the span's state once the event has happened.
State State
// Fields are the span's fields once the event has happened.
Fields
// Status, Class and Err say how a TypeEnd ended; Err is one line.
Status Status
Class Class
Err string
// Stream and Text are the line of a TypeLine, sanitised.
Stream Stream
Text string
// Dropped counts the lines the sinks were not sent: on a TypeLine,
// those left out since the span's previous line, and on a TypeEnd, all
// the span's.
Dropped int
}
Event is one change to a span, or a suspension of the displays. It is plain data: no errors, buffers or maps, so a sink may keep it as it is.
type Fields ¶
type Fields struct {
// Node is what a display folds finished targets by: the target as the
// program names it, such as a host, a container, a file or a port. It
// may be a node set, such as the nodes of a batch.
//
// For a target, a display folds by Name when Node is empty. Set Node
// all the same, so that the event log carries "node", and because
// Name is cut to MaxField while Node never is. A batch is folded by
// its Node alone.
Node string
// Host is the address the work goes to, and Role the target's role,
// as the program names it.
Host, Role string
// Total is how many targets to expect below a step or a batch. It
// only ever grows: a smaller Total given later is ignored.
Total int
// Limit is how many of those targets are worked on at once.
Limit int
// Batch is "i/n", the place of a batch among its step's.
Batch string
// Message is one short line for a display, such as the question a
// confirmation asks.
Message string
// Method and Path are the request of an HTTP call, and HTTPStatus
// its answer.
Method, Path string
HTTPStatus int
// Cache says where a lookup was answered from, such as "hit",
// "miss", "memory" or "disk".
Cache string
// Source is the kind of source a secret or a credential was read
// from, such as "age", "sops", "prompt" or "command"; never the value.
Source string
// Timeout is the bound of a call, or the length of a wait.
Timeout time.Duration
// Exit is the exit code of a command, nil when there was none. Exit,
// when set, points to a value every sink of the event shares; a sink
// never writes through it.
Exit *int
// Amount is how much of the span's own work is done, and Size how much
// there is, in Unit; a Size of 0 is one not known. Neither counts the
// work of the spans below. Every event of a span carries the Amount
// the Bus last sent in a TypeAdvance, and its TypeEnd the final one.
Amount, Size int64
Unit Unit
}
Fields is the whole vocabulary a span can say about itself. There is no bag of free-form attributes: a value without a field here cannot reach a sink, and so reaches no log or exporter either.
Fields is embedded in Event, and the event log writes both on one flat line, so a field added here takes neither the name of a member of Event nor a key of the log another part of an event has; a test holds both.
type Flags ¶
type Flags uint8
Flags tell a display how to show a span.
const ( // Hidden is plumbing, drawn only when it fails or takes long. A span // started under a Hidden one is Hidden too. Hidden Flags = 1 << iota // Fold marks a step whose finished targets a display folds into node // sets. A Fold step with no Fold step above it is where a counter // counts: its Total is what the counter expects. Fold // ShowLines lets the output of the calls below be shown as it // arrives, for a command whose output is the product, such as one that // runs a command on many hosts. A span started under one shows lines // too. ShowLines // DryRun marks the command of a dry run, and a call a dry run only // recorded. It is not passed down. DryRun )
type Kind ¶
type Kind uint8
Kind is the level of the tree a span is on.
const ( // KindCommand is the command being run, the root of the tree. KindCommand Kind = iota + 1 // KindStep is a named phase of a command, such as "reset the // machines". A step with Fold is where a pool's targets are counted. KindStep // KindBatch is one of the batches a staggered action runs in turn. KindBatch // KindTarget is one thing a step works on, such as a host, a // container, a file or a port. KindTarget // KindCall is one request made for a target, such as a remote // command, an HTTP request, a copy or a lookup. KindCall // KindWait is time spent waiting on purpose: the confirmation, the // pause between batches. KindWait )
type LineSink ¶
type LineSink interface {
// WantsLines reports whether the sink wants the lines of output.
WantsLines() bool
}
LineSink is a Sink that may ask for the lines of output, which are only produced when a sink asks. The Bus asks once, when it is made. A Sink that is not a LineSink asks for none, as Log does, though it is sent the lines another sink asked for.
type Log ¶
type Log struct {
// contains filtered or unexported fields
}
Log is a sink that writes the events of a Bus as JSON, one line each but for the advances it samples out (see LogOptions.AdvanceEvery): an event log, for a bug report, a test, or a converter to another tracing system.
The first line is the run, the trace its events belong to, where the trace came from when another program handed it on, and the program and its version that wrote it, as LogOptions name them:
{"v":1,"type":"trace","run":"5d0c…a1f3","trace":"4bf9…4736","parent":"00f067aa0ba902b7","traceFlags":"01","traceState":"rojo=00f067aa0ba902b7","version":"v0.4.0"}
and every line after it one event, with the run and the trace again, its Seq, Time and Type, the span's id and its parent's as 16 hexadecimal digits, its Kind, Name, Flags and State, the Fields it has, and how it ended:
{"v":1,"run":"5d0c…a1f3","trace":"4bf9…4736","seq":7,"time":"2026-09-26T12:00:03.000000000Z","type":"end","span":"9f3c…0007","parent":"9f3c…0002","kind":"target","name":"exe0002","state":"ended","node":"exe0002","host":"exe0002.hpc.example.org","status":"failed","class":"transport","err":"…"}
The run tells the lines of one Log from those of others appending to the same file, which may share the trace. A Timeout is in seconds. The Amount, Size and Unit of a span's work are on every line of the span from the first event that sets them, and an advance is written as "advance", at most once each LogOptions.AdvanceEvery a span: the others are left out, and counted in "leftOut" on the next advance written and on the span's end. A key whose value is zero or empty is left out. A line of output is written without its text, which never leaves the process: the event says only which stream it came from and how many lines before it were not sent; and since the log is no LineSink, and so asks for no lines, there are line events only while a display asks for them.
The lines are written by a goroutine of the log's own, never under the Bus's lock, in pieces of whole lines of at most 64 KiB, a longer line on its own, so that the lines of runs appending to one local file at once, each opened with os.O_APPEND, do not cut into each other. That holds only where a write is taken whole: a pipe takes one whole only up to PIPE_BUF bytes, 4 KiB on Linux, and a file on NFS not at all when several machines append to it, so runs that write there each need a file of their own, or one writer that takes their lines whole. The lines are written once 64 KiB wait, as a step, a batch or the command ends, and a second after a line came at the latest, so that a log followed as it grows, or one of a run killed, falls little behind. A write that fails stops the log, and so does a writer that falls 8 MiB behind, a pipe whose reader has stopped: the log never holds the work up. A log that falls behind still writes the lines it had taken, and leaves out those that come after. Close writes what is left, waiting five seconds at most, and says why the log stops short when it does; Done says when the writer is no longer written to.
func NewLog ¶
func NewLog(w io.Writer, o LogOptions) *Log
NewLog returns a Log that writes to w, and starts the goroutine that writes; Close stops it.
Example ¶
A Log writes the events as JSON lines. A clock, a trace and a run of the example's own keep them the same on every run, but for the span ids, which each Bus draws at random and the example writes as #1, #2 and so on.
package main
import (
"bytes"
"context"
"fmt"
"regexp"
"strings"
"time"
"github.com/GSI-HPC/go-clikit/progress"
)
func main() {
var buf bytes.Buffer
log := progress.NewLog(&buf, progress.LogOptions{Program: "prog", Version: "v1.2.3", Run: "0123456789abcdef"})
now := time.Date(2026, 9, 25, 12, 0, 0, 0, time.UTC)
bus := progress.NewBus(progress.BusOptions{
Sinks: []progress.Sink{log},
Now: func() time.Time { return now },
Trace: progress.TraceContext{Trace: progress.TraceID{0x4b, 0xf9, 0x2f, 0x35, 0x77, 0xb3, 0x4d, 0xa6, 0xa3, 0xce, 0x92, 0x9d, 0x0e, 0x0e, 0x47, 0x36}},
})
ctx, command := progress.Start(progress.WithBus(context.Background(), bus), progress.KindCommand, "uptime")
_, call := progress.Start(ctx, progress.KindCall, "ssh", progress.Host("exe01.example.org"), progress.Timeout(time.Minute))
call.End(nil, progress.Exit(0))
command.End(nil)
bus.Close()
if err := log.Close(); err != nil {
fmt.Println(err)
}
ids := map[string]string{}
fmt.Print(spanID.ReplaceAllStringFunc(buf.String(), func(field string) string {
key, id, _ := strings.Cut(field, ":")
if ids[id] == "" {
ids[id] = fmt.Sprintf(`"#%d"`, len(ids)+1)
}
return key + ":" + ids[id]
}))
}
// spanID matches the span ids of an event log.
var spanID = regexp.MustCompile(`"(span|parent)":"[0-9a-f]{16}"`)
Output: {"v":1,"type":"trace","run":"0123456789abcdef","trace":"4bf92f3577b34da6a3ce929d0e0e4736","program":"prog","version":"v1.2.3"} {"v":1,"run":"0123456789abcdef","trace":"4bf92f3577b34da6a3ce929d0e0e4736","seq":1,"time":"2026-09-25T12:00:00.000000000Z","type":"start","span":"#1","kind":"command","name":"uptime","state":"running"} {"v":1,"run":"0123456789abcdef","trace":"4bf92f3577b34da6a3ce929d0e0e4736","seq":2,"time":"2026-09-25T12:00:00.000000000Z","type":"start","span":"#2","parent":"#1","kind":"call","name":"ssh","state":"running","host":"exe01.example.org","timeout":60} {"v":1,"run":"0123456789abcdef","trace":"4bf92f3577b34da6a3ce929d0e0e4736","seq":3,"time":"2026-09-25T12:00:00.000000000Z","type":"end","span":"#2","parent":"#1","kind":"call","name":"ssh","state":"ended","host":"exe01.example.org","timeout":60,"exit":0,"status":"ok"} {"v":1,"run":"0123456789abcdef","trace":"4bf92f3577b34da6a3ce929d0e0e4736","seq":4,"time":"2026-09-25T12:00:00.000000000Z","type":"end","span":"#1","kind":"command","name":"uptime","state":"ended","status":"ok"}
func (*Log) Begin ¶
Begin writes the run the events belong to, the log's first line: its trace, and the program, LogOptions.Program or else r.Program.
func (*Log) Close ¶
Close writes what is left of the log, once the Bus is closed, waiting five seconds at most, and returns the first error the log met: nothing after it was written. Closing a closed Log only says so again.
The writer is not closed; that is for whoever opened it to do, once Close has returned nil. When the time runs out, no write is started after Close returns, but one already under way cannot be stopped: Close returns an error that says so, and Done is closed once that write has returned, which a writer that is not safe to close during a write, such as a bufio.Writer or a gzip.Writer, waits for.
type LogOptions ¶
type LogOptions struct {
// Program names the program that writes the log, which its first line
// names as "program"; empty takes it from the Bus, BusOptions.Program,
// and leaves the key out when that is empty too.
Program string
// Version is the version of the program that writes the log, which its
// first line names.
Version string
// Run names the run on every line; empty draws 16 hexadecimal digits
// at random.
Run string
// AdvanceEvery is the least time between two advances of a span that
// the log writes, by the events' times; the advances in between are
// left out, and counted in "leftOut". 0 is a second; a negative value
// writes every advance the Bus sends.
AdvanceEvery time.Duration
}
LogOptions configure a Log.
type Meter ¶ added in v0.3.0
type Meter struct {
// contains filtered or unexported fields
}
Meter keeps the work of a Bus's spans from its events, rolled up the tree, the way a display shows how far the work has got.
A span's rolled-up amount is its own and that of every span below it in the same unit, those that ended included; a span that reports no work of its own takes the unit of the first span below it that does. Its share done comes from the first of three rules that applies, which Bound names:
- BySize: a span with a size of its own is its amount over that size.
- ByTargets: a step with Fold, or a batch whose Total is known, once at least one target below it has been bounded, is as far as the targets that ended, however they ended, and the share done of each target running, over its Total. A running target that is unbounded adds nothing until it ends. The share stays short of done until Tally's count reaches its Total, so that a step whose running targets have all reached their sizes does not read as done.
- ByBelow: any other span is its amount over the sum of the sizes of the spans below it with work in its unit, those that ended included, as long as every one of them has a size. A span with a size of its own counts for everything below it: what is below it adds neither its size nor its want of one. Work in Percent counts with a size of 100, so a span over several such calls is as far as the mean of their shares.
Without any of them a span is Unbounded. A share is held to 0 to 1, so that an amount that passes its size reads as done.
A Meter is given every event of a Bus in order, as a sink is, and keeps a few seconds of each open span's work, for its rate and its time left. It forgets a span once it has ended, but its work still counts in the spans above. A Meter is not safe for concurrent use; a sink feeds it from Handle, under the Bus lock. The zero Meter is ready to use.
Example ¶
A Meter rolls the work of a Bus's spans up the tree. A step that counts its targets is as far as the targets that ended, and the share done of each one running: here two of four ended and one is half done.
package main
import (
"context"
"fmt"
"time"
"github.com/GSI-HPC/go-clikit/progress"
"github.com/GSI-HPC/go-clikit/progress/progresstest"
)
func main() {
now := time.Date(2026, 10, 10, 12, 0, 0, 0, time.UTC)
capture := &progresstest.Capture{}
bus := progress.NewBus(progress.BusOptions{
Sinks: []progress.Sink{capture},
Now: func() time.Time { return now },
})
ctx, _ := progress.Start(progress.WithBus(context.Background(), bus), progress.KindCommand, "deploy")
ctx, _ = progress.Start(ctx, progress.KindStep, "copying the image",
progress.WithFlags(progress.Fold), progress.Total(4))
targets := make([]*progress.Span, 4)
for i, node := range []string{"exe01", "exe02", "exe03", "exe04"} {
_, targets[i] = progress.Start(ctx, progress.KindTarget, node, progress.Queued(),
progress.Node(node), progress.Work(progress.Bytes, 1<<30))
}
for _, target := range targets[:2] {
target.Run()
target.Advance(1 << 30)
target.End(nil)
}
targets[2].Run()
now = now.Add(5 * time.Second)
targets[2].Advance(1 << 29)
var meter progress.Meter
var counted progress.SpanID
for _, e := range capture.Events() {
meter.Add(e)
if e.Kind == progress.KindStep && e.Type == progress.TypeStart {
counted = e.Span
}
}
r, _ := meter.Read(counted, now)
fmt.Println(r.Bound, r.Fraction, r.Amount, r.Size)
bus.Close()
}
Output: targets 0.625 2684354560 0
type Option ¶
type Option func(*options)
Option sets what a span says about itself when it starts, changes or ends.
Start takes every option. Update takes Total, Message and Work only, and ignores the others. End takes every option that sets a field, but not Queued or WithFlags, which only Start can give; it is where a field learned late is set, such as an address known only once the work has started, or the Exit of a command. A field an option does not set keeps the value it had.
func Queued ¶
func Queued() Option
Queued starts a span queued rather than running: announced now, worked on once Run is called. A pool starts every target this way before the first one runs, so that a display knows the whole of the work from the start.
func Work ¶ added in v0.3.0
Work says that a span reports work in u, out of size; a size of 0 or less is one not known, and a zero u is Items. Start and Update take it: Update to give the size once it is known, or to change it. A unit given before stays. The size of work in Percent is 100, whatever size is given. Advance and SetAmount move the amount.
type Reading ¶ added in v0.3.0
type Reading struct {
// Unit is the span's own, or that of the first span below it that
// reported work.
Unit Unit
// Amount is the span's own amount and that of the spans below in Unit.
Amount int64
// Size is the bound of Amount, under BySize and ByBelow; 0 otherwise.
Size int64
// Bound says where Fraction comes from; Unbounded has none.
Bound Bound
// Fraction is the share of the work done, from 0 to 1.
Fraction float64
// Rate is how much Amount grew a second over the last five seconds, or
// over the time the span has run when that is shorter. In the Reading
// Add returns as a span ends, it is the mean over the whole time the
// span ran.
Rate float64
// Left is the time the rest of the work should take, at the rate the
// share grew over the last five seconds, once the span has run for two
// seconds and while its share grows; 0 is not known.
Left time.Duration
// Stalled is how long Amount has not grown, once that is five seconds,
// while the span runs; 0 otherwise.
Stalled time.Duration
}
Reading is the work of a span, rolled up. Fields may be added in a minor release.
type Run ¶
type Run struct {
// Trace is the trace the run's spans belong to, and where it came
// from: BusOptions.Trace, or a trace drawn at random.
Trace TraceContext
// Program names the program, BusOptions.Program.
Program string
}
Run is what a Bus tells a TraceSink about the run its events belong to. Fields may be added in a minor release.
type Sink ¶
type Sink interface {
// Handle receives one event.
Handle(Event)
}
Sink receives the events of a Bus, one at a time, in Seq order, while the Bus is locked. It only updates memory, never blocks and never calls the Bus back; drawing happens on the sink's own time. A sink that panics is removed from the Bus, whatever its type, one that cannot be compared, such as a func type, included.
type Span ¶
type Span struct {
// contains filtered or unexported fields
}
Span is one unit of work. A nil *Span is one nobody watches, and every method of it does nothing. The methods are safe for concurrent use.
func Start ¶
Start begins a span under the innermost one ctx carries, and returns a context that carries the new span along with it. The span is running unless Queued is given. Without a Bus in ctx, once it is closed, or under a span that has ended, Start returns ctx itself and a nil span: what a worker starts after its target was ended for it, by the end of its step or of the command, belongs to nothing still open, and would outlive the span its ending ended.
Example (WithoutABus) ¶
Without a Bus, Start returns the context as it is and a nil span, whose methods do nothing: a library reports its work at no cost when no command is listening.
package main
import (
"context"
"fmt"
"github.com/GSI-HPC/go-clikit/progress"
)
func main() {
ctx := context.Background()
got, span := progress.Start(ctx, progress.KindStep, "reset the machines")
span.Run()
span.End(nil)
fmt.Println(got == ctx, span == nil)
}
Output: true true
func (*Span) Advance ¶ added in v0.3.0
Advance adds n to the amount of a span's work. A span that was given no Work counts Items. It does nothing to a span that is not running, and costs an atomic add and a read of BusOptions.Now, outside the Bus's lock, and the lock but once each 100 ms: the Bus sends the amount at most that often, the newest never lost, and with the span's End in any case.
Example ¶
A span reports its work: Work gives its unit and its size, here once the size is known, and Advance adds to the amount done. The Bus sends the amount at most once each 100 ms by its clock, and the End carries the newest, so the advance made 50 ms after the one before reaches the sinks with the End.
package main
import (
"context"
"fmt"
"time"
"github.com/GSI-HPC/go-clikit/progress"
)
// worker is a sink that prints the work each event of a span carries.
type worker struct{}
func (worker) Handle(e progress.Event) {
if e.Kind == progress.KindCall {
fmt.Printf("%s %q %d/%d %s\n", e.Type, e.Name, e.Amount, e.Size, e.Unit)
}
}
func main() {
now := time.Date(2026, 10, 10, 12, 0, 0, 0, time.UTC)
bus := progress.NewBus(progress.BusOptions{
Sinks: []progress.Sink{worker{}},
Now: func() time.Time { return now },
})
ctx := progress.WithBus(context.Background(), bus)
_, call := progress.Start(ctx, progress.KindCall, "download", progress.HTTP("GET", "/images/rocky-9.4.qcow2"))
call.Update(progress.Work(progress.Bytes, 2<<30))
for range 3 {
now = now.Add(100 * time.Millisecond)
call.Advance(8 << 20)
}
now = now.Add(50 * time.Millisecond)
call.Advance(8 << 20)
call.End(nil)
bus.Close()
}
Output: start "download" 0/0 unit(0) update "download" 0/2147483648 bytes advance "download" 8388608/2147483648 bytes advance "download" 16777216/2147483648 bytes advance "download" 25165824/2147483648 bytes end "download" 33554432/2147483648 bytes
func (*Span) End ¶
End ends a span with the outcome of its work: ok for a nil error; skipped, with no class, for an error that is ErrSkipped as errors.Is tells, such as one Skip returns, before Classify is asked, so that no rule of a program's turns work left out on purpose into a failure, even when err joins the skip with a failure, as ErrSkipped warns; and otherwise failed or canceled as Classify tells from err, with the Bus's BusOptions.Classify as its fallback. err's text becomes the span's one-line Err. The options set the fields that are known only at the end, such as Exit. Only the first End or Skip of a span counts. Spans started under it that are still open end first, as canceled, "not finished".
func (*Span) Run ¶
func (s *Span) Run()
Run marks a queued span as running, when its work takes its place in a pool. It does nothing to a span that is running or ended.
func (*Span) SetAmount ¶ added in v0.3.0
SetAmount sets the amount of a span's work, for work that says how far it has got rather than how much more it did, such as a remote task that reports a percentage, or a download that starts over. An amount below 0 is sent as 0. It does nothing to a span that is not running, and costs what Advance costs.
Example ¶
Work that says only how far it has got, such as a remote task, is in Percent, out of 100, and SetAmount sets how far it is.
package main
import (
"context"
"fmt"
"time"
"github.com/GSI-HPC/go-clikit/progress"
)
// worker is a sink that prints the work each event of a span carries.
type worker struct{}
func (worker) Handle(e progress.Event) {
if e.Kind == progress.KindCall {
fmt.Printf("%s %q %d/%d %s\n", e.Type, e.Name, e.Amount, e.Size, e.Unit)
}
}
func main() {
now := time.Date(2026, 10, 10, 12, 0, 0, 0, time.UTC)
bus := progress.NewBus(progress.BusOptions{
Sinks: []progress.Sink{worker{}},
Now: func() time.Time { return now },
})
ctx := progress.WithBus(context.Background(), bus)
_, call := progress.Start(ctx, progress.KindCall, "firmware update", progress.Work(progress.Percent, 0))
for _, done := range []int64{40, 75, 100} {
now = now.Add(time.Second)
call.SetAmount(done)
}
call.End(nil)
bus.Close()
}
Output: start "firmware update" 0/100 percent advance "firmware update" 40/100 percent advance "firmware update" 75/100 percent advance "firmware update" 100/100 percent end "firmware update" 100/100 percent
type SpanID ¶
type SpanID uint64
SpanID names a span. It is never zero, and unique within a trace; Seq, not the id, orders the events.
type Status ¶
type Status uint8
Status says how a span ended.
const ( // StatusOK is work that succeeded. StatusOK Status = iota + 1 // StatusFailed is work that failed. StatusFailed // StatusCanceled is work that was interrupted, or never started // because of an interrupt. StatusCanceled // StatusSkipped is work left out on purpose, such as a call a dry run // does not send or a batch after one that failed. StatusSkipped )
type Suspender ¶
type Suspender interface {
// Suspend takes the sink off the terminal, and returns once it is
// clear.
Suspend()
// Resume lets the sink draw again.
Resume()
}
Suspender is a Sink that draws on the terminal, and takes itself off it while something else asks a question there. Suspend and Resume are called outside the Bus lock, and Suspend returns once the terminal is clear: they are the only calls in which a sink may write to the terminal itself.
type Tally ¶
type Tally struct {
// contains filtered or unexported fields
}
Tally keeps count of the targets of a Bus's work, from its events, the way a display shows how far the work has got.
A step with Fold and a batch count the targets below them. A target that ends counts once in every such span above it, however it ended; a batch or a Fold step left out, skipped or canceled before any target of its own started, counts in the same spans as its Total, less what the batches and steps left out below it have counted there already. So a count reaches its Total once all the work it expects has ended, after an interrupt too, and never passes it: progresstest.Check holds every source to that.
The spans that count with none above them are the roots of the tally, what a counter shows: "power on" as a whole, not each of its batches.
A Tally is given every event of a Bus in order, as a sink is, and forgets a span once it has ended. It is not safe for concurrent use; a sink feeds it from Handle, under the Bus lock.
func (*Tally) Add ¶
Add counts e in. When e ends a span that counts, Add returns where its targets stood as it ended.
type TraceContext ¶
type TraceContext struct {
// Trace is the trace the spans belong to.
Trace TraceID
// Parent is the other program's span the work runs under; zero when
// the trace began with the Bus.
Parent SpanID
// Flags are the trace flags the other program gave, whose lowest bit
// says that it samples the trace.
Flags byte
// State is the tracestate the other program gave, the part of the
// trace context each tracing system keeps for itself, as it was given.
State string
}
TraceContext says which trace the spans of a Bus belong to and, when another program handed the trace on, where in it they belong: under a span of that program's, with the trace flags and the tracestate it gave. A Bus records where its trace came from, for an event log, and changes nothing else by it: a trace the other program does not sample is shown and logged all the same. A Bus takes its trace context as BusOptions.Trace.
func ParseTraceContext ¶
func ParseTraceContext(traceparent, tracestate string) (tc TraceContext, ok bool)
ParseTraceContext reads the trace context another program handed on, as a traceparent and a tracestate in the W3C Trace Context format, such as the TRACEPARENT and TRACESTATE a CI job runs a program with.
A traceparent is four fields of lowercase hexadecimal digits split by dashes: the version, the trace id, the parent's span id and the trace flags, as in 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01. Version 00 is exactly that; a later version may add fields after a further dash, which are left out. Version ff, a trace or span id of zeros, a letter in upper case and anything else that does not fit make the traceparent invalid, and an invalid one is no trace context at all: ok is false, and the tracestate is not looked at. The tracestate is kept as it was given when it is at most 512 bytes of printable ASCII and tabs, and left out otherwise.
Example ¶
A command continues the trace of the program that started it, such as a CI job, from the TRACEPARENT and TRACESTATE it was given.
package main
import (
"fmt"
"github.com/GSI-HPC/go-clikit/progress"
)
func main() {
tc, ok := progress.ParseTraceContext("00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01", "rojo=00f067aa0ba902b7")
fmt.Println(ok, tc.Trace, tc.Parent, tc.Flags, tc.State)
bus := progress.NewBus(progress.BusOptions{Trace: tc})
defer bus.Close()
_, ok = progress.ParseTraceContext("00-00000000000000000000000000000000-00f067aa0ba902b7-01", "")
fmt.Println(ok)
}
Output: true 4bf92f3577b34da6a3ce929d0e0e4736 00f067aa0ba902b7 1 rojo=00f067aa0ba902b7 false
type TraceSink ¶
type TraceSink interface {
// Begin receives the run the Bus's events belong to.
Begin(Run)
}
TraceSink is a Sink that records the run its events belong to, as an event log does. The Bus tells it once, when it is made, before any event.
type Type ¶
type Type uint8
Type is what an event reports.
const ( // TypeStart is a span that begins, queued or running. TypeStart Type = iota + 1 // TypeRun is a queued span whose work has started. TypeRun // TypeUpdate is a span whose Total, Message or Work (Unit and Size) // changed. TypeUpdate // TypeLine is a line of output from the work of a call. TypeLine // TypeEnd is a span that ended; Status says how. TypeEnd // TypeSuspend asks every display to take itself off the terminal, // since something is about to ask a question there. TypeSuspend // TypeResume ends a TypeSuspend. TypeResume // TypeAdvance is a span whose Amount changed. The Bus sends one at // most every 100 ms a span, by its clock, with the newest Amount. TypeAdvance )
type Unit ¶ added in v0.3.0
type Unit uint8
Unit is what the work of a span is counted in. Values may be added in a minor release.
const ( // Items counts things, such as files, rows or packages. Items Unit = iota + 1 // Bytes counts bytes, which a display draws in KiB, MiB, GiB and TiB. Bytes // Percent is how far work has got, from 0 to 100, for work that says // nothing else, such as a remote task. Its size is always 100, and a // display draws its share, never an amount or a rate. Percent )
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package cliprogress is the command line's side of progress: the words a flag such as --progress takes, the rule that picks a display from them and from what the program found out about its streams, and a Run, which makes the Bus, the display, the summary and the event log of one command and takes them down again, in order, once the command has ended.
|
Package cliprogress is the command line's side of progress: the words a flag such as --progress takes, the rule that picks a display from them and from what the program found out about its streams, and a Run, which makes the Bus, the display, the summary and the event log of one command and takes them down again, in order, once the command has ended. |
|
Package display shows the progress of a command on its standard error, from the events of its progress.Bus: the live tree, a few rows at the bottom of a terminal that show the work under way and what of it failed; the counter, one line there that says how far the work has got; plain lines, one for each thing worth a line, for a log as much as a terminal; and the summary a display leaves behind once the command has ended.
|
Package display shows the progress of a command on its standard error, from the events of its progress.Bus: the live tree, a few rows at the bottom of a terminal that show the work under way and what of it failed; the counter, one line there that says how far the work has got; plain lines, one for each thing worth a line, for a log as much as a terminal; and the summary a display leaves behind once the command has ended. |
|
Package progresstest captures the progress events of a test, draws them as a tree that does not depend on the order concurrent work happened in, and checks that they keep the promises the progress package makes to every sink.
|
Package progresstest captures the progress events of a test, draws them as a tree that does not depend on the order concurrent work happened in, and checks that they keep the promises the progress package makes to every sink. |