Documentation
¶
Overview ¶
Package workflow (application) translates the YAML file into the domain.
The separation exists because §22 of the plan is explicit: once published, the database is the source of truth, not the file. The YAML is publishing INPUT -- it comes in here, becomes domain, and the domain is what persists. Changing the file format must not touch the graph's invariants.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type ParamSpec ¶
type ParamSpec struct {
Name string `yaml:"name"`
Type string `yaml:"type"`
Default string `yaml:"default"`
Description string `yaml:"description"`
Enum []string `yaml:"enum"`
Pattern string `yaml:"pattern"`
}
ParamSpec is a run parameter as written in the file.
params:
- name: load_full
type: boolean
default: "false"
- name: start_date
type: string
pattern: '^\d{4}-\d{2}-\d{2}$'
type ResourceSpec ¶
type ResourceSpec struct {
CPU string `yaml:"cpu"`
Memory string `yaml:"memory"`
Limits struct {
CPU string `yaml:"cpu"`
Memory string `yaml:"memory"`
} `yaml:"limits"`
}
ResourceSpec is the CPU and memory request in Kubernetes' format.
`limits` kept apart from `requests` because the difference between them is the difference between "how much I reserve" and "when I get killed": a dbt that blows the limit dies with OOMKilled, one that merely exceeds the request keeps running.
type Spec ¶
type Spec struct {
Name string `yaml:"name"`
Schedule string `yaml:"schedule"`
Type string `yaml:"type"`
Tags []string `yaml:"tags"`
Image string `yaml:"image"`
Resources ResourceSpec `yaml:"resources"`
Params []ParamSpec `yaml:"params"`
// Env and Secrets apply to every step; a step overrides them name by name.
Env map[string]string `yaml:"env"`
Secrets map[string]string `yaml:"secrets"`
// Concurrency is Kestra's `concurrency.limit`, under the name most
// orchestrators use.
Concurrency int `yaml:"concurrency"`
Steps []StepSpec `yaml:"steps"`
}
Spec mirrors the YAML, and nothing more. Loose fields here, invariants in the domain.
type StepSpec ¶
type StepSpec struct {
ID string `yaml:"id"`
Run string `yaml:"run"`
Action string `yaml:"action"`
With map[string]any `yaml:"with"`
DependsOn []string `yaml:"depends_on"`
// Image e Resources sobrescrevem os do workflow. Ausentes = herda.
Image string `yaml:"image"`
Resources ResourceSpec `yaml:"resources"`
// Env sao variaveis com valor literal no arquivo.
//
// env:
// BREVIS_LOG_LEVEL: info
Env map[string]string `yaml:"env"`
// Secrets are variables whose value is NOT in the file: the key is the
// variable's name, the value is where to find it.
//
// secrets:
// GABRIEL_SESSION_COOKIE: gabriel-session/cookie
Secrets map[string]string `yaml:"secrets"`
// Shell: a pointer, to tell "did not declare" from "declared false". Without
// the pointer, every step without the key would become `shell: false` and
// images that do have a shell -- most of them -- would start receiving an
// argv, breaking any command with a pipe or a variable.
Shell *bool `yaml:"shell"`
}
StepSpec is a step as written in the file.
Source Files