Help us shape the future of Go! Take the 2026 Go Developer Survey

engine

package
v0.1.0-alpha.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 25, 2026 License: Apache-2.0 Imports: 40 Imported by: 0

Documentation

Overview

Package engine executes requests: it resolves the version and settings, runs API hooks, renders the request from templates and secrets, sends it, checks the response, and records and saves the result. Progress is reported as a stream of typed events (design §7.6), which the CLI and GUI render.

Everything the engine records or emits is masked: values read from secrets never leave it in clear text.

Index

Constants

View Source
const (
	KindAsync      = "async"
	KindGRPC       = "grpc"
	OutcomeOK      = "ok"
	OutcomeTimeout = "timeout"
	OutcomeError   = "error"
	// OutcomeCancelled is a message-based run the user stopped; the messages that
	// had arrived are kept.
	OutcomeCancelled = "cancelled"
)

The kinds of record, and the outcomes of an async one.

Variables

View Source
var ErrExampleName = errors.New("example names use letters, digits, '.', '_' and '-', and do not start with '.'")

ErrExampleName is returned for a pin name that is not a plain file name.

Functions

func ExampleFile

func ExampleFile(requestFile, name string) string

ExampleFile is where the pinned example called name of a request file lives.

func GRPCCallOf

func GRPCCallOf(d Deps, in SendInput, includeSecrets bool) (grpcx.Call, diag.List)

GRPCCallOf renders the gRPC version of the request in in as a call without making it: no hook runs and nothing is sent. It is what a method picker needs (the server, metadata, TLS and .proto files). includeSecrets reads the vault (which may prompt); without it a secret stays as written.

func Observers

func Observers(fns ...func(Event)) func(Event)

Observers combines observers into one for Deps.Observe, calling each in order; nil entries are skipped. It returns nil when there is none, so a run with no observer pays nothing per event.

func PinExample

func PinExample(rec Record, requestFile, name string) (string, error)

PinExample writes rec as <request>.examples/<name>.json next to the request file (name defaults to the status code). rec must already be masked, as every record from a SendResult is.

func ValidExampleName

func ValidExampleName(name string) bool

ValidExampleName reports whether name can be a pinned example's file name: it must not reach outside the examples directory.

Types

type AttemptRecord

type AttemptRecord struct {
	Status   int           `json:"status,omitempty"`
	Error    string        `json:"error,omitempty"`
	Duration time.Duration `json:"duration"`
	Wait     time.Duration `json:"wait,omitempty"`
}

AttemptRecord is one attempt: a status, or the error that stopped it.

type Body

type Body struct {
	Text        string `json:"text,omitempty"`        // UTF-8 bodies
	Base64      string `json:"base64,omitempty"`      // other bodies
	JSON        any    `json:"json,omitempty"`        // parsed, when the body is JSON
	ParseErr    string `json:"parse_error,omitempty"` // the body claimed JSON but is not
	File        string `json:"file,omitempty"`        // a streamed download, or an uploaded file
	SHA256      string `json:"sha256,omitempty"`
	Size        int64  `json:"size"`
	RawSize     int64  `json:"raw_size,omitempty"`
	ContentType string `json:"content_type,omitempty"`
}

Body is a request or response body as recorded.

type Deps

type Deps struct {
	Lookup  Lookup
	Schemas SchemaChecker // nil: bodies are not checked or learned
	// Transports runs async requests, by protocol (nil → every protocol this
	// build speaks); tests pass fakes.
	Transports transport.Registry
	// GRPC makes gRPC calls (nil → the real client, grpcx.Run); tests pass fakes.
	GRPC    GRPCRunner
	Vault   *vault.Vault
	Environ []string // for the proxy fallback (HTTP_PROXY, …)
	DataDir func(collection model.ID) string
	Clock   func() time.Time
	NewID   func() string // ULIDs for runs and records
	// After waits like time.After (nil → time.After); tests pass a
	// controllable clock so delays and grace periods need no sleeping.
	After func(time.Duration) <-chan time.Time
	// Nodes returns a node type's runner (nil → nodes.Lookup); tests add
	// recording or blocking node types.
	Nodes func(model.NodeType) (nodes.Runner, bool)
	// Observe, when set, receives every event of every run as it is
	// emitted, in order, on the emitting goroutine under the run's emit
	// lock. It must return quickly and must not call back into the run.
	Observe func(Event)
	// UserAgent is the User-Agent requests send unless a setting or the
	// request names one ("" → "Flowi").
	UserAgent string
	// Traceparent, when set, returns the W3C traceparent header value for
	// the call made by node path ("" for a standalone send) of run runID.
	// It is added to outgoing requests that do not carry one already.
	Traceparent func(runID, path string) string
}

Deps are the engine's collaborators.

type Digest

type Digest struct {
	RunID, Path, Node, Format, Text string
}

Digest is a digest node's summary for the stdout sink, masked.

func (Digest) EventName

func (Digest) EventName() string

EventName returns "Digest".

type Event

type Event interface{ EventName() string }

Event is something that happened during a run. EventName is the CapitalCase name used in logs and NDJSON output.

type GRPCRunner

type GRPCRunner interface {
	Run(ctx context.Context, c grpcx.Call) (grpcx.Result, error)
}

GRPCRunner makes one gRPC call. The engine uses grpcx.Run; tests pass fakes.

type Header struct {
	Name  string `json:"name"`
	Value string `json:"value"`
}

Header is one header line, kept in order.

type HookChain

type HookChain struct {
	Pre, Post []HookRef
}

HookChain is every hook step for one request version, in run order: Pre outermost first (collection → folders → request → version), Post the exact reverse.

func ChainFor

ChainFor collects the hooks that apply to version v of request r in collection c. Folder hooks come from every _folder.yaml on the way down to the request's folder.

func (HookChain) Filter

func (c HookChain) Filter(ctl model.HookControl) HookChain

Filter applies a node's or send's hook control: all (default), none, or skip pre and/or post.

type HookFinished

type HookFinished struct {
	RunID, Path, Level, Phase, Step string
	Index, Depth                    int
	Status                          Status
	Duration                        time.Duration
	Diags                           diag.List
}

HookFinished reports a hook step's outcome.

func (HookFinished) EventName

func (HookFinished) EventName() string

EventName returns "HookFinished".

type HookRef

type HookRef struct {
	Level string // collection | folder | request | version
	Step  model.HookStep
}

HookRef is one hook step and where it was declared.

func (HookRef) Label

func (h HookRef) Label() string

Label names the step for events and logs: the hook flow's path, or "inline".

type HookStarted

type HookStarted struct {
	RunID, Path, Level, Phase, Step string
	Index, Depth                    int
}

HookStarted marks the start of one hook step. Level is collection, folder, request or version; Phase is pre or post; Step names the step (a flow path, or "inline"); Index is its position in its level. Path is the request node's path ("" for a standalone send).

func (HookStarted) EventName

func (HookStarted) EventName() string

EventName returns "HookStarted".

type Hop

type Hop struct {
	Status   int      `json:"status"`
	URL      string   `json:"url"`
	Location string   `json:"location"`
	Header   []Header `json:"headers,omitempty"`
	// Forwarded names the credential headers sent on to Location although
	// it is another origin, because http.redirect_credentials or
	// http.redirect_keep_headers allows it.
	Forwarded []string `json:"forwarded,omitempty"`
}

Hop is one redirect.

type Log

type Log struct {
	RunID, Path, Node, Message string
}

Log is a message from a log node, masked.

func (Log) EventName

func (Log) EventName() string

EventName returns "Log".

type Lookup

type Lookup interface {
	Collection(id model.ID) (model.Collection, bool)
	// Request and Flow resolve a reference made from collection from
	// (cross-collection references name their collection).
	Request(ref model.Ref, from model.ID) (model.Collection, model.Request, bool)
	Flow(ref model.Ref, from model.ID) (model.Collection, model.Flow, bool)
	// Settings resolves every layer for a request version in folder
	// (collection-relative, e.g. "requests/auth"), including the user,
	// workspace, environment-variable and flag layers.
	Settings(collection model.ID, folder string, version model.Settings) settings.Resolved
}

Lookup gives the engine the loaded collections. The facade implements it over the workspace.

type MessageReceived

type MessageReceived struct {
	RunID, Path, RequestID string
	Seq                    int
	Message                RecordMessage
	Depth                  int
}

MessageReceived reports a message an async request has received and kept, while it is still waiting for more. Seq counts from 1. The message is masked.

func (MessageReceived) EventName

func (MessageReceived) EventName() string

EventName returns "MessageReceived".

type NodeFinished

type NodeFinished struct {
	RunID, Path, Parent, Lane string
	Node, Type                string
	Status                    Status
	Handled                   bool
	Duration                  time.Duration
	Outputs                   map[string]any
	Diags                     diag.List
	Depth                     int
}

NodeFinished reports a node instance's outcome. Outputs are masked and leave out request/fetch bodies and headers (they are in the response record). Handled means on_error let the run continue past a failure.

func (NodeFinished) EventName

func (NodeFinished) EventName() string

EventName returns "NodeFinished".

type NodeStarted

type NodeStarted struct {
	RunID, Path, Parent, Lane string
	Node, Type, Name          string
	Depth                     int
}

NodeStarted marks the start of a node instance. Path is unique in the run (users[2]/create); Parent is the enclosing container or request node's path ("" at the top) and Lane the lane label within it ("[2]", "b1", "then", "pre:request:0"). Depth is the hook nesting depth.

func (NodeStarted) EventName

func (NodeStarted) EventName() string

EventName returns "NodeStarted".

type Preview

type Preview struct {
	Masked PreviewRequest
	Real   *PreviewRequest
	Diags  diag.List
}

Preview is a request rendered without sending: no hooks run. Masked shows values that read a secret as vault.Mask. Real holds the actual values and is set only when they were asked for.

func PreviewSend

func PreviewSend(d Deps, in SendInput, includeSecrets bool) Preview

PreviewSend renders the request in in as a send would after its pre-hooks, without sending anything and without running hooks. Names that cannot be resolved (an unknown variable, an unknown generator, a secret while the vault is locked) stay as written rather than failing. Diags are problems that make the request unsendable, such as a body file outside the collection. includeSecrets reads the vault (which may prompt) and returns the real values in Real.

type PreviewPart

type PreviewPart struct {
	Name, Value, File, ContentType string
}

PreviewPart is one multipart field of a previewed request.

type PreviewRequest

type PreviewRequest struct {
	Method   string
	URL      string
	Headers  []Header
	BodyKind string
	Body     string
	File     string
	Parts    []PreviewPart
}

PreviewRequest is a request rendered but not sent. BodyKind is "" (no body), "text" (JSON or text; Body is its text), "form" (Body is the encoded form), "multipart" (Parts) or "file" (File).

type Record

type Record struct {
	ID            string          `json:"id"`
	RunID         string          `json:"run_id"`
	CollectionID  model.ID        `json:"collection_id"`
	RequestID     model.ID        `json:"request_id"`
	RequestPath   string          `json:"request_path"`
	Version       string          `json:"version"`
	VersionSource string          `json:"version_source"` // pinned | preferred | default | highest
	At            time.Time       `json:"at"`
	Request       SentRequest     `json:"request"`
	Status        int             `json:"status"`
	Proto         string          `json:"proto,omitempty"`
	Header        []Header        `json:"headers,omitempty"`
	Body          Body            `json:"body"`
	Redirects     []Hop           `json:"redirects,omitempty"`
	Timings       Timings         `json:"timings"`
	Attempts      []AttemptRecord `json:"attempts"`
	Outputs       map[string]any  `json:"outputs,omitempty"`

	// Kind is "async" for an async request; "" is an HTTP one. An async
	// record has Status 0, Request.Method is the protocol and Outcome says
	// how it ended. Body is the first message received (else the one sent).
	Kind     string          `json:"kind,omitempty"`
	Outcome  string          `json:"outcome,omitempty"`
	Messages []RecordMessage `json:"messages,omitempty"`

	// A gRPC record ("grpc" kind) is also message based: Messages holds the
	// request as sent and the responses, Header the response metadata, and
	// these the final status (its name, e.g. NOT_FOUND, and message) and the
	// trailers. Status stays 0.
	GRPCCode    string   `json:"grpc_code,omitempty"`
	GRPCMessage string   `json:"grpc_message,omitempty"`
	Trailers    []Header `json:"trailers,omitempty"`
}

Record is a response record (design §6.4). Its JSON form is the saved response format.

func (*Record) HasMessages

func (r *Record) HasMessages() bool

HasMessages reports whether the record is message based (async or gRPC) and so has an Outcome and Messages instead of a status.

func (Record) Masked

func (r Record) Masked(m vault.Masker) Record

Masked returns a deep copy with every known secret masked in every string the record holds.

type RecordCaptured

type RecordCaptured struct {
	RunID, Path, Node string
	Depth             int
	Record            Record
}

RecordCaptured carries the record of a request node's call: the request as sent and the response (or messages), masked like every record the engine hands out. Path and Node are the request node's, as in NodeFinished; it comes before that node's NodeFinished. A node that sent nothing (it failed before sending, or was skipped) has none. Nothing else changes: the report and the saved responses are as before.

func (RecordCaptured) EventName

func (RecordCaptured) EventName() string

EventName returns "RecordCaptured".

type RecordMessage

type RecordMessage struct {
	Direction string    `json:"direction"` // sent | received
	At        time.Time `json:"at"`
	Headers   []Header  `json:"headers,omitempty"`
	Body      Body      `json:"body"`
	Key       string    `json:"key,omitempty"`
	Partition int32     `json:"partition,omitempty"`
	Offset    int64     `json:"offset,omitempty"`
	Sequence  uint64    `json:"sequence,omitempty"`
}

RecordMessage is one message of an async record, masked.

type RequestSent

type RequestSent struct {
	RunID, Path, RequestID, Version string
	RequestPath                     string // the saved request's path; "" for ad-hoc calls
	VersionSource                   string // pinned, preferred, default, highest
	Method, URL                     string
	Attempt                         int
	Depth                           int // 0 for the request being sent, >0 inside hook flows
}

RequestSent is emitted for every attempt; URL is masked. Path is the node making the call ("" for a standalone send).

func (RequestSent) EventName

func (RequestSent) EventName() string

EventName returns "RequestSent".

type ResponseReceived

type ResponseReceived struct {
	RunID, Path, RequestID string
	Status                 int
	Error                  string
	Duration               time.Duration
	Wait                   time.Duration // backoff before the next attempt, if any
	Attempt                int
	Depth                  int
	Outcome                string // async requests: ok, timeout or error (Status stays 0)
}

ResponseReceived reports an attempt's response (Status 0 when the attempt failed; Error then says why, masked).

func (ResponseReceived) EventName

func (ResponseReceived) EventName() string

EventName returns "ResponseReceived".

type Run

type Run struct {
	// contains filtered or unexported fields
}

Run is a flow run in progress.

Goroutine ownership: StartRun starts the run goroutine, which owns every node goroutine through the scheduler; all of them exit with the run's context. Events is closed after RunFinished, then Wait returns. Events must be drained: a full buffer blocks the run.

func StartRun

func StartRun(ctx context.Context, d Deps, in RunInput) *Run

StartRun starts running in.Flow. Cancelling ctx stops the run: in-flight requests are aborted and no new nodes start.

func (*Run) Events

func (r *Run) Events() <-chan Event

Events streams the run's events; it is closed when the run ends.

func (*Run) Wait

func (r *Run) Wait() RunResult

Wait blocks until the run has ended and returns its result.

type RunAborted

type RunAborted struct {
	RunID    string
	Reason   string // grace_period_exceeded
	Grace    time.Duration
	Inflight []string
}

RunAborted means a cancelled run did not stop within its grace period: Inflight are the node paths still running, in start order.

func (RunAborted) EventName

func (RunAborted) EventName() string

EventName returns "RunAborted".

type RunCancelled

type RunCancelled struct{ RunID string }

RunCancelled means the run's context was cancelled.

func (RunCancelled) EventName

func (RunCancelled) EventName() string

EventName returns "RunCancelled".

type RunFinished

type RunFinished struct {
	RunID    string
	Status   Status
	Duration time.Duration
}

RunFinished closes a run's event stream.

func (RunFinished) EventName

func (RunFinished) EventName() string

EventName returns "RunFinished".

type RunInput

type RunInput struct {
	Collection    model.Collection
	Flow          model.Flow
	Env           string            // environment name; "" → defaults.environment
	PreferVersion string            // run preference; beats the flow's prefer_version
	Inputs        map[string]string // templates for the flow's declared inputs
	Seed          uint64            // 0 → a random seed, reported in the result
	Grace         time.Duration     // 0 → the run.grace setting
	// Only, when non-nil, runs just these top-level nodes (in flow order);
	// the rest are left out of the run and its report. Prior holds the
	// outputs of the nodes left out, as an earlier run reported them
	// (masked, so a secret in them reads as the mask), for the chosen nodes'
	// templates to read as nodes.<id>. Prior is read by a plain flow only,
	// not by a flow-level run: loop.
	Only  []string
	Prior map[string]map[string]any
}

RunInput is one flow run.

type RunResult

type RunResult struct {
	RunID  string
	Seed   uint64
	Status Status
	// Diags are the unhandled failure's diagnostics and every warning.
	Diags diag.List
	Saved []string // files written (saved responses, values, digests)
	// Report is the run's report; ReportFile where it was saved
	// (runs/<flow-id>/<run-id>.json in the workspace data dir).
	Report     report.Run
	ReportFile string
	// Err is a system failure that did not stop the run (a file could not
	// be saved).
	Err error
}

RunResult is a finished flow run. Everything in it is masked.

type RunStarted

type RunStarted struct {
	RunID         string
	Kind          string
	FlowID        string
	FlowPath, Env string
	Seed          uint64
	At            time.Time
}

RunStarted opens a run's event stream. Kind is "send" or "flow"; the Flow fields are set for flow runs.

func (RunStarted) EventName

func (RunStarted) EventName() string

EventName returns "RunStarted".

type SchemaChecker

type SchemaChecker interface {
	Check(c model.Collection, r model.Request, version string, rec Record) diag.List
}

SchemaChecker validates a response against the schemas stored for its request and learns from it (the facade implements it). Its diagnostics are notes on the send: they never change its status.

type Send

type Send struct {
	// contains filtered or unexported fields
}

Send is a send in progress.

Goroutine ownership: StartSend starts one goroutine that runs the send. It closes Events after the last event (RunFinished or RunCancelled) and then completes Wait. Events must be read: a full buffer blocks the run.

func StartSend

func StartSend(ctx context.Context, d Deps, in SendInput) *Send

StartSend starts sending in.Request. Cancelling ctx aborts the request in flight; nothing is saved then.

func (*Send) Events

func (s *Send) Events() <-chan Event

Events streams the send's events; it is closed when the send ends.

func (*Send) Wait

func (s *Send) Wait() SendResult

Wait blocks until the send has ended and returns its result.

type SendInput

type SendInput struct {
	Collection    model.Collection
	Request       model.Request
	Env           string // environment name; "" → defaults.environment; still "" → no environment
	PinnedVersion string
	PreferVersion string
	With          map[string]string // templates, rendered once against env and generators
	Hooks         model.HookControl
	Seed          uint64 // 0 → a random seed, reported in the result
	Save          string // "" → the responses.save setting
}

SendInput is one standalone send.

type SendResult

type SendResult struct {
	RunID  string
	Seed   uint64
	Status Status
	Record *Record        // nil when nothing was sent
	Vars   map[string]any // variables set by hooks
	Diags  diag.List
	Saved  []string // files written (saved responses)
	// Err is a system failure that did not stop the send (saving a
	// response or the cookie jar failed).
	Err error
}

SendResult is a finished send. Everything in it is masked.

type SentRequest

type SentRequest struct {
	Method string   `json:"method"`
	URL    string   `json:"url"`
	Header []Header `json:"headers,omitempty"`
	Body   *Body    `json:"body,omitempty"`
}

SentRequest is the request as actually sent, masked.

type Status

type Status string

Status is the outcome of a run, hook or node.

const (
	StatusPassed    Status = "passed"
	StatusFailed    Status = "failed"
	StatusError     Status = "error"
	StatusCancelled Status = "cancelled"
	StatusSkipped   Status = "skipped"
	// StatusAborted is a cancelled run that did not stop within its grace
	// period.
	StatusAborted Status = "aborted"
)

Statuses.

type Timings

type Timings struct {
	DNS     time.Duration `json:"dns"`
	Connect time.Duration `json:"connect"`
	TLS     time.Duration `json:"tls"`
	TTFB    time.Duration `json:"ttfb"`
	Total   time.Duration `json:"total"`
}

Timings of the final attempt.

type Warning

type Warning struct {
	RunID string
	Diag  diag.Diagnostic
}

Warning carries a warning diagnostic (masked args).

func (Warning) EventName

func (Warning) EventName() string

EventName returns "Warning".

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL