execution

package
v0.6.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 6, 2026 License: MIT Imports: 14 Imported by: 0

Documentation

Overview

Package execution (application) walks the graph and runs its nodes.

This is the LOCAL version: no queue, no persistence, no scheduler -- those pieces have phases of their own in the plan (§37, phases 2 and 4). What lives here is enough for `brevis run file.yaml` to run on the instance itself, which is what was asked for.

Index

Constants

View Source
const TetoDoLog = 128 << 10

TetoDoLog e quanto da saida de um passo vai para o banco.

128 KB cobre com folga uma run de dbt com 60 nos (~25 KB de texto) e ainda segura um backfill barulhento. O teto existe porque um `while true; do echo` num workflow qualquer nao pode encher o disco do Postgres.

Variables

This section is empty.

Functions

This section is empty.

Types

type ErroDePasso

type ErroDePasso struct {
	NodeID   string
	ExitCode int
	Mensagem string

	// Saida is the last few lines of stderr. Only the last ones, and not all of
	// them, because a chatty process would fill the database's error column --
	// and the cause is almost always at the end.
	Saida []string
}

ErroDePasso is a step's failure, with the context needed to understand it without opening a log: the exit code, what it means, and the last lines the process wrote to stderr.

Before, all that survived was "exited with code 127" -- technically correct and useless. The cause (`/bin/sh: python: not found`) went through the events as a log line and was dropped right there, so the screen showed the symptom without the explanation.

func (*ErroDePasso) Error

func (e *ErroDePasso) Error() string

type Etapa

type Etapa struct {
	Indice  int            `json:"indice"`
	Nome    string         `json:"nome"`
	Estado  string         `json:"estado"`
	Ms      *int64         `json:"ms,omitempty"`
	Em      string         `json:"em"`
	Numeros map[string]any `json:"numeros,omitempty"`
}

Etapa is one phase of an SDK step, as it stands now.

Indice is what identifies it, and not Nome: a pipeline with two Map stages announces `map` twice, and keying by name would make the second overwrite the first -- three declared stages collapsing into two boxes on the screen, with no warning.

EtapaGravada is the same type under the name it travels through the database with: it is what a test outside this package needs to check what was recorded.

type EtapaGravada

type EtapaGravada = Etapa

etapasConhecidas is a closed list on purpose: a phase this engine does not know is ignored, rather than becoming a meaningless box on the screen. EtapaGravada is the shape an Etapa takes in the JSONB column.

type Historico

type Historico interface {
	PassoJaTeveSucesso(ctx context.Context, workflowSlug, nodeID string, exceto uuid.UUID) (bool, error)
}

Persistidor records each step's state. Optional: the local `brevis run` has no database, and requiring one would make an ad-hoc execution depend on infrastructure. Historico answers whether a step has ever succeeded. It is what decides whether this is its FIRST run -- something the step does not know and only the engine holds.

A small interface, declared here in the consumer rather than in the package that implements it.

The question is per (workflow, step), not per workflow: a workflow with three fetchers writing to three tables would create only the first step's if the answer covered the whole workflow, and the other two would fail in silence.

type Persistidor

type Persistidor interface {
	IniciarTask(ctx context.Context, runID uuid.UUID, nodeID string, tentativa int) error
	TerminarTask(ctx context.Context, runID uuid.UUID, nodeID string, tentativa int,
		status run.Status, exit *int, erro string, log string) error

	// RegistrarEtapas records the phases of an SDK step while it runs. It is
	// what makes the screen advance before the step finishes.
	RegistrarEtapas(ctx context.Context, runID uuid.UUID, nodeID string, tentativa int,
		sdkVersao string, etapas json.RawMessage) error
}

type Reporter

type Reporter interface {
	Evento(execution.Event)
}

Reporter receives the execution's events. A small interface so the CLI, the tests and the persister can all watch the same stream.

type Runner

type Runner struct {
	Processo execution.Executor // atende `run:`; pode ser nil se so houver tasks Go
	Go       execution.Executor // atende `action:`; pode ser nil

	WorkDir string
	Env     map[string]string
	Report  Reporter

	// Timeout per node. Zero means no limit.
	Timeout time.Duration

	// MaxTentativas per node. Zero or 1 means a single attempt.
	MaxTentativas int
	BackoffBase   time.Duration

	// Persist and RunID are used together: without both, per-step state is not
	// recorded and the DAG in the UI shows up with no execution state.
	Persist Persistidor
	RunID   uuid.UUID

	// Params are this run's values. They reach the step's command through a
	// template (see execution.Renderizar) and the step's environment, so a
	// fetcher using the SDK sees them without being handed an argument.
	Params map[string]string

	// Trigger says why this Run exists: schedule, manual or backfill.
	Trigger string

	// LogicalDate is the slot this Run stands for. Nil on a manual trigger.
	LogicalDate *time.Time

	// Historico decides whether a step is running for the first time. Nil means
	// there is no way to know -- and then the step gets first=false, because
	// creating a table without being sure is worse than not creating it.
	Historico Historico

	// Vagas caps how many STEPS run at once -- in Kubernetes, how many pods
	// exist simultaneously. Nil means no limit.
	//
	// It has to be shared across every Runner in the process, which is why it
	// is injected rather than created here: the ceiling belongs to the CLUSTER,
	// not to one workflow. Without it, the dispatcher's concurrency limit
	// counted RUNS -- five runs with three parallel steps each gave fifteen
	// pods, not five.
	Vagas chan struct{}

	// TentativaDoRun is this RUN's attempt, counted by the dispatcher. It goes
	// into the pod name so a retry does not find the previous attempt's pod.
	TentativaDoRun int

	// Pods runs steps as pods in Kubernetes. When present it serves every step
	// that declares `image:` -- and the same DAG runs as a pod in the cluster
	// and as a process on a laptop, with no change to the YAML.
	Pods execution.Executor
}

Runner runs a whole workflow.

It holds TWO executors and picks per node: `run:` goes to the process one, `action:` resolves in the Go registry. The choice belongs to the runner and not to the executor, so each executor can go on ignoring that the other exists.

func (Runner) Run

func (r Runner) Run(ctx context.Context, w wf.Workflow) error

Run walks the graph by levels: everything inside a level runs in parallel, and the next level only starts once the previous one closes entirely.

It stops at the FIRST failure in a level, without starting the next. Carrying on after an error would produce a partial result that looks complete -- which is how a pipeline ran 28 days late without anyone seeing it, in the system this one replaces.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL