Documentation
¶
Overview ¶
Package trace provides the append-only, table-shaped run trace shared by the DAG schedulers.
Format ¶
A trace file is a tab-separated table that is never updated in place:
line 1 header: the schema's column names, in order line 2 one row per observed event line 3 ...
A run opens its trace by truncating the file and writing a fresh header, so a trace always describes exactly one attempt; everything after that is appended. One row is one event, which matches how a scheduler actually observes a node - the same node legitimately appears on several rows as it moves ready -> submitted -> running -> done - and it means a row never has to be found and rewritten.
Extending ¶
The column set is a Schema, so adding a field is a two-step change: declare the column (see NodeSchema for the columns the DAG schedulers share) and set it on the records that need it. Existing rows do not have to be migrated, because a column an event does not fill simply renders as an empty cell and the header stays a complete description of every row.
Safety ¶
Tracing is best effort and must never fail a run. A blank destination path yields a nil tracer, a nil *Tracer and a nil *Registry are safe to use, and every row is written with a single write so a crashed run still leaves a readable prefix of the table on disk.
Index ¶
Constants ¶
const ( // ColumnAnalysisNodeID identifies the analysis_node row the event is about. // It is empty on run-level rows. ColumnAnalysisNodeID = "analysis_node_id" // ColumnNodeID is the logical id of the compiled node. Unlike the record id // above it survives reruns, which makes it the right key when comparing two // traces of the same DAG. ColumnNodeID = "node_id" // ColumnNodeName is the display name of the node. ColumnNodeName = "node_name" // ColumnStatus is the status the event observed or produced. On a run-level // row it carries the analysis job status. ColumnStatus = "status" // ColumnSampleID is the scatter sample the node instance belongs to. ColumnSampleID = "sample_id" // ColumnScriptID is the script backing the node. ColumnScriptID = "script_id" // ColumnRerunReason explains why a reusable node was invalidated. ColumnRerunReason = "rerun_reason" )
Columns of the shared DAG run trace (see NodeSchema).
They are one flat namespace rather than per-scheduler fields, so that the legacy, dynamic and dataflow schedulers all emit rows a single reader can consume - adding a scheduler should not add a dialect.
const ( LevelDebug = "DEBUG" LevelInfo = "INFO" LevelWarn = "WARN" LevelError = "ERROR" )
Levels are plain labels: the tracer writes every record it is given and never filters by level, so a level is context for whoever reads the trace, not a switch that decides whether a row exists.
const ( // ColumnTimestamp is when the row was written (UTC, microsecond precision). ColumnTimestamp = "timestamp" // ColumnLevel is a plain label (see the Level* constants). The tracer never // filters on it: it exists so a reader can tell a routine transition from a // warning without parsing the event name. ColumnLevel = "level" // ColumnEvent is the dotted event name, for example "node.submit". ColumnEvent = "event" // ColumnError is the standard place for a failure message. It is shared by // every run trace so the reason for a row can be found without knowing which // scheduler wrote it. ColumnError = "error" )
Columns owned by the tracer itself. They are always the first columns of a trace table, so call sites never set them and every row carries them.
const ColumnSeparator = "\t"
ColumnSeparator is the field separator of every trace file.
Tabs keep the output in the same family as a Nextflow -with-trace table, so the usual tooling for those files (cut, awk, column -t) applies unchanged.
const TimestampFormat = "2006-01-02T15:04:05.000000Z07:00"
TimestampFormat is the layout of ColumnTimestamp. Microseconds are deliberate: several scheduler events for one node routinely land inside the same millisecond, and their relative order is the interesting part.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Record ¶
type Record struct {
// contains filtered or unexported fields
}
Record is one row of a trace: a level, an event name, and the column values the event wants to publish.
A record is only a payload. The tracer owns the destination, the timestamp and the ordering, so a call site describes what happened and nothing else, and the same record can be handed to a different sink unchanged.
func New ¶
New starts a record. The event name should be a stable dotted identifier such as "node.submit": that is what a reader greps, and what stays comparable across runs and across schedulers.
func (*Record) Set ¶
Set adds a column value; the last call for a column wins.
The column must be part of the tracer's schema, otherwise it is ignored when the row is rendered - see Schema.
type Registry ¶
type Registry[K comparable] struct { // contains filtered or unexported fields }
Registry keeps at most one Tracer per run key.
A scheduler stores its per-run tracer here instead of threading a *Tracer parameter through every method of a run: the key (the analysis id) already identifies the run, so instrumenting a new step stays a one-line change at the call site rather than a signature change through the call chain.
A nil *Registry is valid and every method is a no-op, so an orchestrator assembled without tracing keeps working.
func NewRegistry ¶
func NewRegistry[K comparable]() *Registry[K]
NewRegistry creates an empty, ready-to-use registry.
func (*Registry[K]) Close ¶
func (r *Registry[K]) Close(key K)
Close closes and forgets the tracer of key. It is safe to call for a key that was never opened, which is what lets a failure path clean up unconditionally.
type Schema ¶
type Schema struct {
// contains filtered or unexported fields
}
Schema is the ordered column contract of a trace file.
The schema is the single definition of the table: the tracer writes it as the first line of the file and lays every row out in exactly this order. That is what lets a reader parse a trace by position, and what stops a new field from silently shifting the columns that already exist.
func NewSchema ¶
NewSchema builds a schema from explicit column names, keeping the given order and dropping blanks and duplicates.
func NodeSchema ¶
func NodeSchema() *Schema
NodeSchema is the column contract of a DAG run trace.
It is a function rather than a shared variable on purpose: a caller cannot then mutate the schema the other schedulers render with, and rebuilding a handful of strings once per run costs nothing.
func SystemSchema ¶
SystemSchema builds a schema whose table starts with the tracer-owned columns followed by the given domain columns. Every run trace should be built this way so all trace files share the same leading columns.
type Tracer ¶
type Tracer struct {
// contains filtered or unexported fields
}
Tracer appends rows to one trace file.
A nil *Tracer is valid and drops every row, so best-effort tracing never needs a nil check at the call site. All methods are safe for concurrent use.
func Open ¶
Open creates path (truncating any existing file) and writes the schema header, so every run starts from an empty table that describes exactly that run.
A blank path is not an error: it returns (nil, nil) and the caller simply runs untraced, which is what keeps tracing optional.
Source Files