Documentation
¶
Overview ¶
Package workflow (application) translates the YAML file into the domain.
The separation exists because the rule is explicit: once published, the database is the source of truth, not the file. The YAML is publishing INPUT -- it comes in here, becomes domain, and the domain is what persists. Changing the file format must not touch the graph's invariants.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Parse ¶
Parse reads the YAML and returns the workflow already validated.
`caminho` serves two purposes: deriving the slug when the file carries no `name`, and naming the file in error messages -- a graph error without the file's name is useless when there are dozens of them.
func Resolve ¶ added in v0.8.0
Resolve expands every `uses:` across a set of workflows.
It works on the WHOLE set and not on one file, because a `uses:` names a sibling. That is also why the child must be in the same publish, and not merely published already: expansion that reads the database would make `brevis validate` -- which touches no database, on purpose -- answer a different question from `brevis publish`, and the file that passed CI would be the file that failed the deploy.
The child's steps take the parent step's place, prefixed with its id, and the parent node disappears. What arrives at the runner is one flat graph, which is the whole point: a step that TRIGGERED a child run and waited would hold a slot from the same per-process semaphore the child needs, and hang under load.
Types ¶
type Dependency ¶ added in v0.8.0
Dependency is one entry of `depends_on`. It accepts both shapes:
depends_on: [extract]
depends_on:
- {step: determine_load_type, label: changed existing data}
The bare form stays the normal one -- most dependencies have nothing to say and a label on every arrow is noise. The object exists for the branch, where two arrows leaving the same step with no labels is a diagram that requires opening the source to read.
func (*Dependency) UnmarshalYAML ¶ added in v0.8.0
func (d *Dependency) UnmarshalYAML(value *yaml.Node) error
UnmarshalYAML accepts a scalar or a mapping.
The scalar branch is what keeps every workflow ever written working: a list of strings decodes exactly as it did, and nothing in a published file has to change.
type OnErrorSpec ¶ added in v0.8.0
OnErrorSpec is a step's alert declaration as written in the file.
There is no `webhook`, `url` or `channel` field, and there will not be. The destination is a credential; the installation owns it. This says WHETHER and HOW.
type ParamSpec ¶
type ParamSpec struct {
Name string `yaml:"name"`
Type string `yaml:"type"`
Default string `yaml:"default"`
Description string `yaml:"description"`
Enum []string `yaml:"enum"`
Pattern string `yaml:"pattern"`
}
ParamSpec is a run parameter as written in the file.
params:
- name: load_full
type: boolean
default: "false"
- name: start_date
type: string
pattern: '^\d{4}-\d{2}-\d{2}$'
type ResourceSpec ¶
type ResourceSpec struct {
CPU string `yaml:"cpu"`
Memory string `yaml:"memory"`
Limits struct {
CPU string `yaml:"cpu"`
Memory string `yaml:"memory"`
} `yaml:"limits"`
}
ResourceSpec is the CPU and memory request in Kubernetes' format.
`limits` kept apart from `requests` because the difference between them is the difference between "how much I reserve" and "when I get killed": a dbt that blows the limit dies with OOMKilled, one that merely exceeds the request keeps running.
type Spec ¶
type Spec struct {
Name string `yaml:"name"`
Description string `yaml:"description"`
Schedule string `yaml:"schedule"`
Type string `yaml:"type"`
Tags []string `yaml:"tags"`
Image string `yaml:"image"`
Resources ResourceSpec `yaml:"resources"`
Params []ParamSpec `yaml:"params"`
// Env and Secrets apply to every step; a step overrides them name by name.
Env map[string]string `yaml:"env"`
Secrets map[string]string `yaml:"secrets"`
// Concurrency is Kestra's `concurrency.limit`, under the name most
// orchestrators use.
Concurrency int `yaml:"concurrency"`
Steps []StepSpec `yaml:"steps"`
}
Spec mirrors the YAML, and nothing more. Loose fields here, invariants in the domain.
type StepSpec ¶
type StepSpec struct {
ID string `yaml:"id"`
Run string `yaml:"run"`
Action string `yaml:"action"`
With map[string]any `yaml:"with"`
DependsOn []Dependency `yaml:"depends_on"`
// Image e Resources sobrescrevem os do workflow. Ausentes = herda.
Image string `yaml:"image"`
Host string `yaml:"host"`
Resources ResourceSpec `yaml:"resources"`
// Env are variables with a literal value in the file.
//
// env:
// BREVIS_LOG_LEVEL: info
Env map[string]string `yaml:"env"`
// Secrets are variables whose value is NOT in the file: the key is the
// variable's name, the value is where to find it.
//
// secrets:
// GABRIEL_SESSION_COOKIE: gabriel-session/cookie
Secrets map[string]string `yaml:"secrets"`
// Runtime and Tools say what this step runs in, when the engine cannot
// work it out from `run:` and `image:`.
//
// runtime: python
// tools: [dbt]
//
// Both optional, and the engine INFERS both when they are absent -- which
// is the normal case. Declaring one is for when the inference is wrong or
// blind: a wrapper script, a bare binary path, an image whose name says
// nothing.
//
// An id outside the vocabulary is refused at publish, naming what is
// valid. The alternative is a chip that renders blank on a screen three
// days later, with nothing to trace it to.
Runtime string `yaml:"runtime"`
Tools []string `yaml:"tools"`
// Shell: a pointer, to tell "did not declare" from "declared false". Without
// the pointer, every step without the key would become `shell: false` and
// images that do have a shell -- most of them -- would start receiving an
// argv, breaking any command with a pipe or a variable.
Shell *bool `yaml:"shell"`
// When is the trigger rule. Empty is `all_success`, which is what every
// workflow written before this existed means.
When string `yaml:"when"`
// Marker is a step that does nothing and exists to be a point in the
// graph -- a `start`, an `end`, a join. See dominio.Node.Marker.
Marker bool `yaml:"marker"`
// UnlessEmpty names a context key that decides whether this step runs.
// A key, not an expression. See dominio.Node.UnlessEmpty.
UnlessEmpty string `yaml:"unless_empty"`
// ForEach names a context key holding a list. The step runs once per
// element. See dominio.Node.ForEach.
ForEach string `yaml:"for_each"`
// Group draws this step inside a named box on the graph. Visual only.
// See dominio.Node.Group.
Group string `yaml:"group"`
// Uses names another workflow whose steps take this one's place, expanded
// at publish. See dominio.Node.Uses.
Uses string `yaml:"uses"`
// OnError announces this step's failures.
//
// on_error:
// type: SLACK
// when: attempt # optional; the default is give_up
//
// A pointer for the same reason Shell is: absent has to be different from
// declared-and-empty, and a zero OnError would look like a step asking to
// be announced to nowhere.
OnError *OnErrorSpec `yaml:"on_error"`
}
StepSpec is a step as written in the file.