Documentation
¶
Overview ¶
Package engine executes requests: it resolves the version and settings, runs API hooks, renders the request from templates and secrets, sends it, checks the response, and records and saves the result. Progress is reported as a stream of typed events (design §7.6), which the CLI and GUI render.
Everything the engine records or emits is masked: values read from secrets never leave it in clear text.
Index ¶
- Constants
- Variables
- func ExampleFile(requestFile, name string) string
- func GRPCCallOf(d Deps, in SendInput, includeSecrets bool) (grpcx.Call, diag.List)
- func Observers(fns ...func(Event)) func(Event)
- func PinExample(rec Record, requestFile, name string) (string, error)
- func ValidExampleName(name string) bool
- type AttemptRecord
- type Body
- type Deps
- type Digest
- type Event
- type GRPCRunner
- type Header
- type HookChain
- type HookFinished
- type HookRef
- type HookStarted
- type Hop
- type Log
- type Lookup
- type MessageReceived
- type NodeFinished
- type NodeStarted
- type Preview
- type PreviewPart
- type PreviewRequest
- type Record
- type RecordCaptured
- type RecordMessage
- type RequestSent
- type ResponseReceived
- type Run
- type RunAborted
- type RunCancelled
- type RunFinished
- type RunInput
- type RunResult
- type RunStarted
- type SchemaChecker
- type Send
- type SendInput
- type SendResult
- type SentRequest
- type Status
- type Timings
- type Warning
Constants ¶
const ( KindAsync = "async" KindGRPC = "grpc" OutcomeOK = "ok" OutcomeTimeout = "timeout" OutcomeError = "error" // OutcomeCancelled is a message-based run the user stopped; the messages that // had arrived are kept. OutcomeCancelled = "cancelled" )
The kinds of record, and the outcomes of an async one.
Variables ¶
var ErrExampleName = errors.New("example names use letters, digits, '.', '_' and '-', and do not start with '.'")
ErrExampleName is returned for a pin name that is not a plain file name.
Functions ¶
func ExampleFile ¶
ExampleFile is where the pinned example called name of a request file lives.
func GRPCCallOf ¶
GRPCCallOf renders the gRPC version of the request in in as a call without making it: no hook runs and nothing is sent. It is what a method picker needs (the server, metadata, TLS and .proto files). includeSecrets reads the vault (which may prompt); without it a secret stays as written.
func Observers ¶
Observers combines observers into one for Deps.Observe, calling each in order; nil entries are skipped. It returns nil when there is none, so a run with no observer pays nothing per event.
func PinExample ¶
PinExample writes rec as <request>.examples/<name>.json next to the request file (name defaults to the status code). rec must already be masked, as every record from a SendResult is.
func ValidExampleName ¶
ValidExampleName reports whether name can be a pinned example's file name: it must not reach outside the examples directory.
Types ¶
type AttemptRecord ¶
type AttemptRecord struct {
Status int `json:"status,omitempty"`
Error string `json:"error,omitempty"`
Duration time.Duration `json:"duration"`
Wait time.Duration `json:"wait,omitempty"`
}
AttemptRecord is one attempt: a status, or the error that stopped it.
type Body ¶
type Body struct {
Text string `json:"text,omitempty"` // UTF-8 bodies
Base64 string `json:"base64,omitempty"` // other bodies
JSON any `json:"json,omitempty"` // parsed, when the body is JSON
ParseErr string `json:"parse_error,omitempty"` // the body claimed JSON but is not
File string `json:"file,omitempty"` // a streamed download, or an uploaded file
SHA256 string `json:"sha256,omitempty"`
Size int64 `json:"size"`
RawSize int64 `json:"raw_size,omitempty"`
ContentType string `json:"content_type,omitempty"`
}
Body is a request or response body as recorded.
type Deps ¶
type Deps struct {
Lookup Lookup
Schemas SchemaChecker // nil: bodies are not checked or learned
// Transports runs async requests, by protocol (nil → every protocol this
// build speaks); tests pass fakes.
Transports transport.Registry
// GRPC makes gRPC calls (nil → the real client, grpcx.Run); tests pass fakes.
GRPC GRPCRunner
Vault *vault.Vault
Environ []string // for the proxy fallback (HTTP_PROXY, …)
DataDir func(collection model.ID) string
Clock func() time.Time
NewID func() string // ULIDs for runs and records
// After waits like time.After (nil → time.After); tests pass a
// controllable clock so delays and grace periods need no sleeping.
After func(time.Duration) <-chan time.Time
// Nodes returns a node type's runner (nil → nodes.Lookup); tests add
// recording or blocking node types.
Nodes func(model.NodeType) (nodes.Runner, bool)
// Observe, when set, receives every event of every run as it is
// emitted, in order, on the emitting goroutine under the run's emit
// lock. It must return quickly and must not call back into the run.
Observe func(Event)
// UserAgent is the User-Agent requests send unless a setting or the
// request names one ("" → "Flowi").
UserAgent string
// Traceparent, when set, returns the W3C traceparent header value for
// the call made by node path ("" for a standalone send) of run runID.
// It is added to outgoing requests that do not carry one already.
Traceparent func(runID, path string) string
}
Deps are the engine's collaborators.
type Digest ¶
type Digest struct {
RunID, Path, Node, Format, Text string
}
Digest is a digest node's summary for the stdout sink, masked.
type Event ¶
type Event interface{ EventName() string }
Event is something that happened during a run. EventName is the CapitalCase name used in logs and NDJSON output.
type GRPCRunner ¶
GRPCRunner makes one gRPC call. The engine uses grpcx.Run; tests pass fakes.
type HookChain ¶
type HookChain struct {
Pre, Post []HookRef
}
HookChain is every hook step for one request version, in run order: Pre outermost first (collection → folders → request → version), Post the exact reverse.
func ChainFor ¶
func ChainFor(c model.Collection, r model.Request, v model.RequestVersion) HookChain
ChainFor collects the hooks that apply to version v of request r in collection c. Folder hooks come from every _folder.yaml on the way down to the request's folder.
type HookFinished ¶
type HookFinished struct {
RunID, Path, Level, Phase, Step string
Index, Depth int
Status Status
Duration time.Duration
Diags diag.List
}
HookFinished reports a hook step's outcome.
func (HookFinished) EventName ¶
func (HookFinished) EventName() string
EventName returns "HookFinished".
type HookStarted ¶
HookStarted marks the start of one hook step. Level is collection, folder, request or version; Phase is pre or post; Step names the step (a flow path, or "inline"); Index is its position in its level. Path is the request node's path ("" for a standalone send).
func (HookStarted) EventName ¶
func (HookStarted) EventName() string
EventName returns "HookStarted".
type Hop ¶
type Hop struct {
Status int `json:"status"`
URL string `json:"url"`
Location string `json:"location"`
Header []Header `json:"headers,omitempty"`
// Forwarded names the credential headers sent on to Location although
// it is another origin, because http.redirect_credentials or
// http.redirect_keep_headers allows it.
Forwarded []string `json:"forwarded,omitempty"`
}
Hop is one redirect.
type Log ¶
type Log struct {
RunID, Path, Node, Message string
}
Log is a message from a log node, masked.
type Lookup ¶
type Lookup interface {
Collection(id model.ID) (model.Collection, bool)
// Request and Flow resolve a reference made from collection from
// (cross-collection references name their collection).
Request(ref model.Ref, from model.ID) (model.Collection, model.Request, bool)
Flow(ref model.Ref, from model.ID) (model.Collection, model.Flow, bool)
// Settings resolves every layer for a request version in folder
// (collection-relative, e.g. "requests/auth"), including the user,
// workspace, environment-variable and flag layers.
Settings(collection model.ID, folder string, version model.Settings) settings.Resolved
}
Lookup gives the engine the loaded collections. The facade implements it over the workspace.
type MessageReceived ¶
type MessageReceived struct {
RunID, Path, RequestID string
Seq int
Message RecordMessage
Depth int
}
MessageReceived reports a message an async request has received and kept, while it is still waiting for more. Seq counts from 1. The message is masked.
func (MessageReceived) EventName ¶
func (MessageReceived) EventName() string
EventName returns "MessageReceived".
type NodeFinished ¶
type NodeFinished struct {
RunID, Path, Parent, Lane string
Node, Type string
Status Status
Handled bool
Duration time.Duration
Outputs map[string]any
Diags diag.List
Depth int
}
NodeFinished reports a node instance's outcome. Outputs are masked and leave out request/fetch bodies and headers (they are in the response record). Handled means on_error let the run continue past a failure.
func (NodeFinished) EventName ¶
func (NodeFinished) EventName() string
EventName returns "NodeFinished".
type NodeStarted ¶
NodeStarted marks the start of a node instance. Path is unique in the run (users[2]/create); Parent is the enclosing container or request node's path ("" at the top) and Lane the lane label within it ("[2]", "b1", "then", "pre:request:0"). Depth is the hook nesting depth.
func (NodeStarted) EventName ¶
func (NodeStarted) EventName() string
EventName returns "NodeStarted".
type Preview ¶
type Preview struct {
Masked PreviewRequest
Real *PreviewRequest
Diags diag.List
}
Preview is a request rendered without sending: no hooks run. Masked shows values that read a secret as vault.Mask. Real holds the actual values and is set only when they were asked for.
func PreviewSend ¶
PreviewSend renders the request in in as a send would after its pre-hooks, without sending anything and without running hooks. Names that cannot be resolved (an unknown variable, an unknown generator, a secret while the vault is locked) stay as written rather than failing. Diags are problems that make the request unsendable, such as a body file outside the collection. includeSecrets reads the vault (which may prompt) and returns the real values in Real.
type PreviewPart ¶
type PreviewPart struct {
Name, Value, File, ContentType string
}
PreviewPart is one multipart field of a previewed request.
type PreviewRequest ¶
type PreviewRequest struct {
Method string
URL string
Headers []Header
BodyKind string
Body string
File string
Parts []PreviewPart
}
PreviewRequest is a request rendered but not sent. BodyKind is "" (no body), "text" (JSON or text; Body is its text), "form" (Body is the encoded form), "multipart" (Parts) or "file" (File).
type Record ¶
type Record struct {
ID string `json:"id"`
RunID string `json:"run_id"`
CollectionID model.ID `json:"collection_id"`
RequestID model.ID `json:"request_id"`
RequestPath string `json:"request_path"`
Version string `json:"version"`
VersionSource string `json:"version_source"` // pinned | preferred | default | highest
At time.Time `json:"at"`
Request SentRequest `json:"request"`
Status int `json:"status"`
Proto string `json:"proto,omitempty"`
Header []Header `json:"headers,omitempty"`
Body Body `json:"body"`
Redirects []Hop `json:"redirects,omitempty"`
Timings Timings `json:"timings"`
Attempts []AttemptRecord `json:"attempts"`
Outputs map[string]any `json:"outputs,omitempty"`
// Kind is "async" for an async request; "" is an HTTP one. An async
// record has Status 0, Request.Method is the protocol and Outcome says
// how it ended. Body is the first message received (else the one sent).
Kind string `json:"kind,omitempty"`
Outcome string `json:"outcome,omitempty"`
Messages []RecordMessage `json:"messages,omitempty"`
// A gRPC record ("grpc" kind) is also message based: Messages holds the
// request as sent and the responses, Header the response metadata, and
// these the final status (its name, e.g. NOT_FOUND, and message) and the
// trailers. Status stays 0.
GRPCCode string `json:"grpc_code,omitempty"`
GRPCMessage string `json:"grpc_message,omitempty"`
Trailers []Header `json:"trailers,omitempty"`
}
Record is a response record (design §6.4). Its JSON form is the saved response format.
func (*Record) HasMessages ¶
HasMessages reports whether the record is message based (async or gRPC) and so has an Outcome and Messages instead of a status.
type RecordCaptured ¶
RecordCaptured carries the record of a request node's call: the request as sent and the response (or messages), masked like every record the engine hands out. Path and Node are the request node's, as in NodeFinished; it comes before that node's NodeFinished. A node that sent nothing (it failed before sending, or was skipped) has none. Nothing else changes: the report and the saved responses are as before.
func (RecordCaptured) EventName ¶
func (RecordCaptured) EventName() string
EventName returns "RecordCaptured".
type RecordMessage ¶
type RecordMessage struct {
Direction string `json:"direction"` // sent | received
At time.Time `json:"at"`
Headers []Header `json:"headers,omitempty"`
Body Body `json:"body"`
Key string `json:"key,omitempty"`
Partition int32 `json:"partition,omitempty"`
Offset int64 `json:"offset,omitempty"`
Sequence uint64 `json:"sequence,omitempty"`
}
RecordMessage is one message of an async record, masked.
type RequestSent ¶
type RequestSent struct {
RunID, Path, RequestID, Version string
RequestPath string // the saved request's path; "" for ad-hoc calls
VersionSource string // pinned, preferred, default, highest
Method, URL string
Attempt int
Depth int // 0 for the request being sent, >0 inside hook flows
}
RequestSent is emitted for every attempt; URL is masked. Path is the node making the call ("" for a standalone send).
func (RequestSent) EventName ¶
func (RequestSent) EventName() string
EventName returns "RequestSent".
type ResponseReceived ¶
type ResponseReceived struct {
RunID, Path, RequestID string
Status int
Error string
Duration time.Duration
Wait time.Duration // backoff before the next attempt, if any
Attempt int
Depth int
Outcome string // async requests: ok, timeout or error (Status stays 0)
}
ResponseReceived reports an attempt's response (Status 0 when the attempt failed; Error then says why, masked).
func (ResponseReceived) EventName ¶
func (ResponseReceived) EventName() string
EventName returns "ResponseReceived".
type Run ¶
type Run struct {
// contains filtered or unexported fields
}
Run is a flow run in progress.
Goroutine ownership: StartRun starts the run goroutine, which owns every node goroutine through the scheduler; all of them exit with the run's context. Events is closed after RunFinished, then Wait returns. Events must be drained: a full buffer blocks the run.
func StartRun ¶
StartRun starts running in.Flow. Cancelling ctx stops the run: in-flight requests are aborted and no new nodes start.
type RunAborted ¶
type RunAborted struct {
RunID string
Reason string // grace_period_exceeded
Grace time.Duration
Inflight []string
}
RunAborted means a cancelled run did not stop within its grace period: Inflight are the node paths still running, in start order.
type RunCancelled ¶
type RunCancelled struct{ RunID string }
RunCancelled means the run's context was cancelled.
func (RunCancelled) EventName ¶
func (RunCancelled) EventName() string
EventName returns "RunCancelled".
type RunFinished ¶
RunFinished closes a run's event stream.
func (RunFinished) EventName ¶
func (RunFinished) EventName() string
EventName returns "RunFinished".
type RunInput ¶
type RunInput struct {
Collection model.Collection
Flow model.Flow
Env string // environment name; "" → defaults.environment
PreferVersion string // run preference; beats the flow's prefer_version
Inputs map[string]string // templates for the flow's declared inputs
Seed uint64 // 0 → a random seed, reported in the result
Grace time.Duration // 0 → the run.grace setting
// Only, when non-nil, runs just these top-level nodes (in flow order);
// the rest are left out of the run and its report. Prior holds the
// outputs of the nodes left out, as an earlier run reported them
// (masked, so a secret in them reads as the mask), for the chosen nodes'
// templates to read as nodes.<id>. Prior is read by a plain flow only,
// not by a flow-level run: loop.
Only []string
Prior map[string]map[string]any
}
RunInput is one flow run.
type RunResult ¶
type RunResult struct {
RunID string
Seed uint64
Status Status
// Diags are the unhandled failure's diagnostics and every warning.
Diags diag.List
Saved []string // files written (saved responses, values, digests)
// Report is the run's report; ReportFile where it was saved
// (runs/<flow-id>/<run-id>.json in the workspace data dir).
Report report.Run
ReportFile string
// Err is a system failure that did not stop the run (a file could not
// be saved).
Err error
}
RunResult is a finished flow run. Everything in it is masked.
type RunStarted ¶
type RunStarted struct {
RunID string
Kind string
FlowID string
FlowPath, Env string
Seed uint64
At time.Time
}
RunStarted opens a run's event stream. Kind is "send" or "flow"; the Flow fields are set for flow runs.
type SchemaChecker ¶
type SchemaChecker interface {
Check(c model.Collection, r model.Request, version string, rec Record) diag.List
}
SchemaChecker validates a response against the schemas stored for its request and learns from it (the facade implements it). Its diagnostics are notes on the send: they never change its status.
type Send ¶
type Send struct {
// contains filtered or unexported fields
}
Send is a send in progress.
Goroutine ownership: StartSend starts one goroutine that runs the send. It closes Events after the last event (RunFinished or RunCancelled) and then completes Wait. Events must be read: a full buffer blocks the run.
func StartSend ¶
StartSend starts sending in.Request. Cancelling ctx aborts the request in flight; nothing is saved then.
func (*Send) Wait ¶
func (s *Send) Wait() SendResult
Wait blocks until the send has ended and returns its result.
type SendInput ¶
type SendInput struct {
Collection model.Collection
Request model.Request
Env string // environment name; "" → defaults.environment; still "" → no environment
PinnedVersion string
PreferVersion string
With map[string]string // templates, rendered once against env and generators
Hooks model.HookControl
Seed uint64 // 0 → a random seed, reported in the result
Save string // "" → the responses.save setting
}
SendInput is one standalone send.
type SendResult ¶
type SendResult struct {
RunID string
Seed uint64
Status Status
Record *Record // nil when nothing was sent
Vars map[string]any // variables set by hooks
Diags diag.List
Saved []string // files written (saved responses)
// Err is a system failure that did not stop the send (saving a
// response or the cookie jar failed).
Err error
}
SendResult is a finished send. Everything in it is masked.
type SentRequest ¶
type SentRequest struct {
Method string `json:"method"`
URL string `json:"url"`
Header []Header `json:"headers,omitempty"`
Body *Body `json:"body,omitempty"`
}
SentRequest is the request as actually sent, masked.