Documentation
¶
Overview ¶
Package engine executes requests: it resolves the version and settings, runs API hooks, renders the request from templates and secrets, sends it, checks the response, and records and saves the result. Progress is reported as a stream of typed events (design §7.6), which the CLI and GUI render.
Everything the engine records or emits is masked: values read from secrets never leave it in clear text.
Index ¶
- Constants
- Variables
- func CredentialHeaders(rec Record) []string
- func ExampleFile(requestFile, name string) string
- func GRPCCallOf(d Deps, in SendInput, includeSecrets bool) (grpcx.Call, diag.List)
- func Observers(fns ...func(Event)) func(Event)
- func PinExample(rec Record, requestFile, name string) (string, error)
- func ResponseKey(version, overlay string) string
- func ValidExampleName(name string) bool
- func WriteHAR(w io.Writer, records []Record, version string) error
- type AttemptRecord
- type AuthPrompted
- type Body
- type DatasetRunner
- type Deps
- type Digest
- type Event
- type GRPCRunner
- type Header
- type HookChain
- type HookFinished
- type HookRef
- type HookStarted
- type Hop
- type Log
- type Lookup
- type MessageReceived
- type NodeFinished
- type NodeStarted
- type Overlay
- type Preview
- type PreviewPart
- type PreviewRequest
- type Record
- type RecordCaptured
- type RecordMessage
- type RequestSchemer
- type RequestSent
- type ResponseReceived
- type Run
- type RunAborted
- type RunCancelled
- type RunFinished
- type RunInput
- type RunResult
- type RunStarted
- type SchemaChecker
- type SchemaMatcher
- type SchemaViolation
- type Send
- type SendInput
- type SendResult
- type SentRequest
- type Status
- type Timings
- type Warning
Constants ¶
const ( KindAsync = "async" KindGRPC = "grpc" KindDataset = "dataset" OutcomeOK = "ok" OutcomeTimeout = "timeout" OutcomeError = "error" // OutcomeCancelled is a message-based run the user stopped; the messages that // had arrived are kept. OutcomeCancelled = "cancelled" // OutcomeStopped is a message-based run a message hook ended: the messages kept up to // the one that failed its check are the record. OutcomeStopped = "stopped" )
The kinds of record, and the outcomes of an async one.
Variables ¶
var ErrExampleName = errors.New("example names use letters, digits, '.', '_' and '-', and do not start with '.'")
ErrExampleName is returned for a pin name that is not a plain file name.
Functions ¶
func CredentialHeaders ¶
CredentialHeaders names the headers of rec's response that look like credentials: cookies being set, tokens, keys, sessions. The masker only knows secrets Flowi holds, so a session cookie a server hands out passes through it; this is what lets the window warn before such a response is pinned, since a pinned example is committed with the collection. Names are as the server sent them, once each, in order.
func ExampleFile ¶
ExampleFile is where the pinned example called name of a request file lives.
func GRPCCallOf ¶
GRPCCallOf renders the gRPC version of the request in in as a call without making it: no hook runs and nothing is sent. It is what a method picker needs (the server, metadata, TLS and .proto files). includeSecrets reads the vault (which may prompt); without it a secret stays as written.
func Observers ¶
Observers combines observers into one for Deps.Observe, calling each in order; nil entries are skipped. It returns nil when there is none, so a run with no observer pays nothing per event.
func PinExample ¶
PinExample writes rec as <request>.examples/<name>.json next to the request file (name defaults to the status code). rec must already be masked, as every record from a SendResult is.
func ResponseKey ¶
ResponseKey names the folder that keeps a version's saved responses. A send with a flowi overlay keeps its own, so a VIP response never overwrites the base one: "1" for the plain version, "1--vip" with the overlay vip. Version labels are numbers, so the double hyphen cannot be part of one.
func ValidExampleName ¶
ValidExampleName reports whether name can be a pinned example's file name: it must not reach outside the examples directory.
func WriteHAR ¶
WriteHAR writes records as a HAR 1.2 file, the format browsers' network tabs and most API tools read. Records are masked already, so the file holds no secret. Message based records (async and gRPC) have no HAR equivalent and are left out. version is Flowi's own, written as the creator.
Types ¶
type AttemptRecord ¶
type AttemptRecord struct {
Status int `json:"status,omitempty"`
Error string `json:"error,omitempty"`
Duration time.Duration `json:"duration"`
Wait time.Duration `json:"wait,omitempty"`
}
AttemptRecord is one attempt: a status, or the error that stopped it.
type AuthPrompted ¶
type AuthPrompted struct {
RunID, Path string // Path is the request node's path; "" for a standalone send
Kind string
URL, Code string
Expires time.Time
}
AuthPrompted says a request is waiting for the person to do something before it can be sent: Kind "open" asks them to open URL in a browser and approve; "device" asks them to open URL on any device and enter Code. It arrives while the request waits, so a window shows it and a terminal prints it. Expires is when the request stops being valid.
func (AuthPrompted) EventName ¶
func (AuthPrompted) EventName() string
EventName returns "AuthPrompted".
type Body ¶
type Body struct {
Text string `json:"text,omitempty"` // UTF-8 bodies
Base64 string `json:"base64,omitempty"` // other bodies
JSON any `json:"json,omitempty"` // parsed, when the body is JSON
ParseErr string `json:"parse_error,omitempty"` // the body claimed JSON but is not
File string `json:"file,omitempty"` // a streamed download, or an uploaded file
SHA256 string `json:"sha256,omitempty"`
Size int64 `json:"size"`
RawSize int64 `json:"raw_size,omitempty"`
ContentType string `json:"content_type,omitempty"`
}
Body is a request or response body as recorded.
type DatasetRunner ¶
DatasetRunner runs one dataset query. The engine uses dataset.Default; tests pass fakes.
type Deps ¶
type Deps struct {
Lookup Lookup
Schemas SchemaChecker // nil: bodies are not checked or learned
// Transports runs async requests, by protocol (nil → every protocol this
// build speaks); tests pass fakes.
Transports transport.Registry
// GRPC makes gRPC calls (nil → the real client, grpcx.Run); tests pass fakes.
GRPC GRPCRunner
// Dataset runs dataset queries (nil → the real stores, dataset.Default); tests pass fakes.
Dataset DatasetRunner
Vault *vault.Vault
Environ []string // for the proxy fallback (HTTP_PROXY, …)
DataDir func(collection model.ID) string
Clock func() time.Time
NewID func() string // ULIDs for runs and records
// After waits like time.After (nil → time.After); tests pass a
// controllable clock so delays and grace periods need no sleeping.
After func(time.Duration) <-chan time.Time
// Nodes returns a node type's runner (nil → nodes.Lookup); tests add
// recording or blocking node types.
Nodes func(model.NodeType) (nodes.Runner, bool)
// Observe, when set, receives every event of every run as it is
// emitted, in order, on the emitting goroutine under the run's emit
// lock. It must return quickly and must not call back into the run.
Observe func(Event)
// UserAgent is the User-Agent requests send unless a setting or the
// request names one ("" → "Flowi").
UserAgent string
// Traceparent, when set, returns the W3C traceparent header value for
// the call made by node path ("" for a standalone send) of run runID.
// It is added to outgoing requests that do not carry one already.
Traceparent func(runID, path string) string
// AuthTokens keeps OAuth 2.0 tokens between sends (nil: every send asks
// the token endpoint again).
AuthTokens *authx.TokenCache
// Transport, when set, sends every HTTP attempt instead of the network (nil: the real one). A run played
// back from a recording uses it.
Transport http.RoundTripper
}
Deps are the engine's collaborators.
type Digest ¶
type Digest struct {
RunID, Path, Node, Format, Text string
}
Digest is a digest node's summary for the stdout sink, masked.
type Event ¶
type Event interface{ EventName() string }
Event is something that happened during a run. EventName is the CapitalCase name used in logs and NDJSON output.
type GRPCRunner ¶
GRPCRunner makes one gRPC call. The engine uses grpcx.Run; tests pass fakes.
type HookChain ¶
type HookChain struct {
Pre, Post, Message []HookRef
}
HookChain is every hook step for one request version, in run order: Pre outermost first (collection → folders → request → version), Post the exact reverse, and Message (run for each message a message request keeps) outermost first like Pre.
func ChainFor ¶
func ChainFor(c model.Collection, r model.Request, v model.RequestVersion) HookChain
ChainFor collects the hooks that apply to version v of request r in collection c. Folder hooks come from every _folder.yaml on the way down to the request's folder.
type HookFinished ¶
type HookFinished struct {
RunID, Path, Level, Phase, Step string
Index, Depth int
Status Status
Duration time.Duration
Diags diag.List
}
HookFinished reports a hook step's outcome.
func (HookFinished) EventName ¶
func (HookFinished) EventName() string
EventName returns "HookFinished".
type HookStarted ¶
HookStarted marks the start of one hook step. Level is collection, folder, request or version; Phase is pre or post; Step names the step (a flow path, or "inline"); Index is its position in its level. Path is the request node's path ("" for a standalone send).
func (HookStarted) EventName ¶
func (HookStarted) EventName() string
EventName returns "HookStarted".
type Hop ¶
type Hop struct {
Status int `json:"status"`
URL string `json:"url"`
Location string `json:"location"`
Header []Header `json:"headers,omitempty"`
// Forwarded names the credential headers sent on to Location although
// it is another origin, because http.redirect_credentials or
// http.redirect_keep_headers allows it.
Forwarded []string `json:"forwarded,omitempty"`
}
Hop is one redirect.
type Log ¶
type Log struct {
RunID, Path, Node, Message string
}
Log is a message from a log node, masked.
type Lookup ¶
type Lookup interface {
Collection(id model.ID) (model.Collection, bool)
// Request and Flow resolve a reference made from collection from
// (cross-collection references name their collection).
Request(ref model.Ref, from model.ID) (model.Collection, model.Request, bool)
Flow(ref model.Ref, from model.ID) (model.Collection, model.Flow, bool)
// Settings resolves every layer for a request version in folder
// (collection-relative, e.g. "requests/auth"), including the user,
// workspace, environment-variable and flag layers.
Settings(collection model.ID, folder string, version model.Settings) settings.Resolved
}
Lookup gives the engine the loaded collections. The facade implements it over the workspace.
type MessageReceived ¶
type MessageReceived struct {
RunID, Path, RequestID string
Seq int
Message RecordMessage
Depth int
}
MessageReceived reports a message an async request has received and kept, while it is still waiting for more. Seq counts from 1. The message is masked.
func (MessageReceived) EventName ¶
func (MessageReceived) EventName() string
EventName returns "MessageReceived".
type NodeFinished ¶
type NodeFinished struct {
RunID, Path, Parent, Lane string
Node, Type string
Status Status
Handled bool
Duration time.Duration
Outputs map[string]any
Diags diag.List
Depth int
// Reason says why a node was skipped by the person ("skipped by you");
// "" otherwise, including a node skipped by its when.
Reason string
}
NodeFinished reports a node instance's outcome. Outputs are masked and leave out request/fetch bodies and headers (they are in the response record). Handled means on_error let the run continue past a failure.
func (NodeFinished) EventName ¶
func (NodeFinished) EventName() string
EventName returns "NodeFinished".
type NodeStarted ¶
NodeStarted marks the start of a node instance. Path is unique in the run (users[2]/create); Parent is the enclosing container or request node's path ("" at the top) and Lane the lane label within it ("[2]", "b1", "then", "pre:request:0"). Depth is the hook nesting depth.
func (NodeStarted) EventName ¶
func (NodeStarted) EventName() string
EventName returns "NodeStarted".
type Overlay ¶
Overlay is a flowi overlay chosen for a send: its name and the operations that apply to the request (shared part first).
type Preview ¶
type Preview struct {
Masked PreviewRequest
Real *PreviewRequest
Diags diag.List
}
Preview is a request rendered without sending: no hooks run. Masked shows values that read a secret as vault.Mask. Real holds the actual values and is set only when they were asked for.
func PreviewSend ¶
PreviewSend renders the request in in as a send would after its pre-hooks, without sending anything and without running hooks. Names that cannot be resolved (an unknown variable, an unknown generator, a secret while the vault is locked) stay as written rather than failing. Diags are problems that make the request unsendable, such as a body file outside the collection. includeSecrets reads the vault (which may prompt) and returns the real values in Real.
type PreviewPart ¶
type PreviewPart struct {
Name, Value, File, ContentType string
}
PreviewPart is one multipart field of a previewed request.
type PreviewRequest ¶
type PreviewRequest struct {
Method string
URL string
Headers []Header
BodyKind string
Body string
File string
Parts []PreviewPart
}
PreviewRequest is a request rendered but not sent. BodyKind is "" (no body), "text" (JSON or text; Body is its text), "form" (Body is the encoded form), "multipart" (Parts) or "file" (File).
type Record ¶
type Record struct {
ID string `json:"id"`
RunID string `json:"run_id"`
CollectionID model.ID `json:"collection_id"`
RequestID model.ID `json:"request_id"`
RequestPath string `json:"request_path"`
Version string `json:"version"`
VersionSource string `json:"version_source"` // pinned | preferred | default | highest
// Overlay names the flowi overlay the request was sent with, if any.
Overlay string `json:"overlay,omitempty"`
At time.Time `json:"at"`
Request SentRequest `json:"request"`
Status int `json:"status"`
Proto string `json:"proto,omitempty"`
Header []Header `json:"headers,omitempty"`
Body Body `json:"body"`
Redirects []Hop `json:"redirects,omitempty"`
Timings Timings `json:"timings"`
Attempts []AttemptRecord `json:"attempts"`
Outputs map[string]any `json:"outputs,omitempty"`
// Kind is "async" for an async request; "" is an HTTP one. An async
// record has Status 0, Request.Method is the protocol and Outcome says
// how it ended. Body is the first message received (else the one sent).
Kind string `json:"kind,omitempty"`
Outcome string `json:"outcome,omitempty"`
Messages []RecordMessage `json:"messages,omitempty"`
// A gRPC record ("grpc" kind) is also message based: Messages holds the
// request as sent and the responses, Header the response metadata, and
// these the final status (its name, e.g. NOT_FOUND, and message) and the
// trailers. Status stays 0.
GRPCCode string `json:"grpc_code,omitempty"`
GRPCMessage string `json:"grpc_message,omitempty"`
Trailers []Header `json:"trailers,omitempty"`
}
Record is a response record (design §6.4). Its JSON form is the saved response format.
func (*Record) HasMessages ¶
HasMessages reports whether the record is message based (async or gRPC) and so has an Outcome and Messages instead of a status.
type RecordCaptured ¶
RecordCaptured carries the record of a request node's call: the request as sent and the response (or messages), masked like every record the engine hands out. Path and Node are the request node's, as in NodeFinished; it comes before that node's NodeFinished. A node that sent nothing (it failed before sending, or was skipped) has none. Nothing else changes: the report and the saved responses are as before.
func (RecordCaptured) EventName ¶
func (RecordCaptured) EventName() string
EventName returns "RecordCaptured".
type RecordMessage ¶
type RecordMessage struct {
Direction string `json:"direction"` // sent | received
At time.Time `json:"at"`
Headers []Header `json:"headers,omitempty"`
Body Body `json:"body"`
Key string `json:"key,omitempty"`
Partition int32 `json:"partition,omitempty"`
Offset int64 `json:"offset,omitempty"`
Sequence uint64 `json:"sequence,omitempty"`
}
RecordMessage is one message of an async record, masked.
type RequestSchemer ¶
type RequestSchemer interface {
RequestSchema(c model.Collection, r model.Request, version string) (schema map[string]any, ok bool)
}
RequestSchemer is an optional part of a SchemaChecker: it gives the schema kept for the body a request sends, for a fuzz step to read the bounds from. ok is false when none is kept.
type RequestSent ¶
type RequestSent struct {
RunID, Path, RequestID, Version string
RequestPath string // the saved request's path; "" for ad-hoc calls
VersionSource string // pinned, preferred, default, highest
Method, URL string
Attempt int
Depth int // 0 for the request being sent, >0 inside hook flows
}
RequestSent is emitted for every attempt; URL is masked. Path is the node making the call ("" for a standalone send).
func (RequestSent) EventName ¶
func (RequestSent) EventName() string
EventName returns "RequestSent".
type ResponseReceived ¶
type ResponseReceived struct {
RunID, Path, RequestID string
Status int
Error string
Duration time.Duration
Wait time.Duration // backoff before the next attempt, if any
Attempt int
Depth int
Outcome string // async requests: ok, timeout or error (Status stays 0)
}
ResponseReceived reports an attempt's response (Status 0 when the attempt failed; Error then says why, masked).
func (ResponseReceived) EventName ¶
func (ResponseReceived) EventName() string
EventName returns "ResponseReceived".
type Run ¶
type Run struct {
// contains filtered or unexported fields
}
Run is a flow run in progress.
Goroutine ownership: StartRun starts the run goroutine, which owns every node goroutine through the scheduler; all of them exit with the run's context. Events is closed after RunFinished, then Wait returns. Events must be drained: a full buffer blocks the run.
func StartRun ¶
StartRun starts running in.Flow. Cancelling ctx stops the run: in-flight requests are aborted and no new nodes start.
type RunAborted ¶
type RunAborted struct {
RunID string
Reason string // grace_period_exceeded
Grace time.Duration
Inflight []string
}
RunAborted means a cancelled run did not stop within its grace period: Inflight are the node paths still running, in start order.
type RunCancelled ¶
type RunCancelled struct{ RunID string }
RunCancelled means the run's context was cancelled.
func (RunCancelled) EventName ¶
func (RunCancelled) EventName() string
EventName returns "RunCancelled".
type RunFinished ¶
RunFinished closes a run's event stream.
func (RunFinished) EventName ¶
func (RunFinished) EventName() string
EventName returns "RunFinished".
type RunInput ¶
type RunInput struct {
Collection model.Collection
Flow model.Flow
Env string // environment name; "" → defaults.environment
PreferVersion string // run preference; beats the flow's prefer_version
Inputs map[string]string // templates for the flow's declared inputs
Seed uint64 // 0 → a random seed, reported in the result
Grace time.Duration // 0 → the run.grace setting
// Only, when non-nil, runs just these top-level nodes (in flow order);
// the rest are left out of the run and its report. Prior holds the
// outputs of the nodes left out, as an earlier run reported them
// (masked, so a secret in them reads as the mask), for the chosen nodes'
// templates to read as nodes.<id>. Prior is read by a plain flow only,
// not by a flow-level run: loop.
Only []string
Prior map[string]map[string]any
// Skip lists node ids (or node paths) that do not run this time: they
// finish as skipped with the reason "skipped by you", wherever they are,
// in every iteration of a loop. A skipped node whose id is in Prior keeps
// those outputs ("hold"), so later steps read the old values; the
// flow's variables come from PriorVars as for a partial run.
Skip []string
// Overlay is a flowi overlay (or a stack, "vip+slow") laid over every request step that names no overlay of
// its own. A step's own overlay wins. "" lays none.
Overlay string
// PriorVars are the flow's variables as an earlier run ended (masked),
// for the chosen nodes' templates to read as vars.<name>: a node left out
// of a partial run may have set them.
PriorVars map[string]any
}
RunInput is one flow run.
type RunResult ¶
type RunResult struct {
RunID string
Seed uint64
Status Status
// Diags are the unhandled failure's diagnostics and every warning.
Diags diag.List
Saved []string // files written (saved responses, values, digests)
// Captured are the masked records of every call the run made, in the order they finished: what a HAR
// file is written from. A hook flow's calls are included.
Captured []Record
// Report is the run's report; ReportFile where it was saved
// (runs/<flow-id>/<run-id>.json in the workspace data dir).
Report report.Run
ReportFile string
// Err is a system failure that did not stop the run (a file could not
// be saved).
Err error
}
RunResult is a finished flow run. Everything in it is masked.
type RunStarted ¶
type RunStarted struct {
RunID string
Kind string
FlowID string
FlowPath, Env string
Overlay string // the run-level flowi overlay, if any
Seed uint64
At time.Time
}
RunStarted opens a run's event stream. Kind is "send" or "flow"; the Flow fields are set for flow runs.
type SchemaChecker ¶
type SchemaChecker interface {
Check(c model.Collection, r model.Request, version string, rec Record) diag.List
}
SchemaChecker validates a response against the schemas stored for its request and learns from it (the facade implements it). Its diagnostics are notes on the send: they never change its status.
type SchemaMatcher ¶
type SchemaMatcher interface {
Match(c model.Collection, r model.Request, version string, status int, body any) (violations []SchemaViolation, found bool)
}
SchemaMatcher is a SchemaChecker that can also check one body against the response schema kept for a request, for a flow's schema check. found is false when no schema is kept for that status.
type SchemaViolation ¶
SchemaViolation is one place a body does not match its schema.
type Send ¶
type Send struct {
// contains filtered or unexported fields
}
Send is a send in progress.
Goroutine ownership: StartSend starts one goroutine that runs the send. It closes Events after the last event (RunFinished or RunCancelled) and then completes Wait. Events must be read: a full buffer blocks the run.
func StartSend ¶
StartSend starts sending in.Request. Cancelling ctx aborts the request in flight; nothing is saved then.
func (*Send) Wait ¶
func (s *Send) Wait() SendResult
Wait blocks until the send has ended and returns its result.
type SendInput ¶
type SendInput struct {
Collection model.Collection
Request model.Request
Env string // environment name; "" → defaults.environment; still "" → no environment
PinnedVersion string
PreferVersion string
With map[string]string // templates, rendered once against env and generators
Hooks model.HookControl
Seed uint64 // 0 → a random seed, reported in the result
Save string // "" → the responses.save setting
// Overlay, when set, is laid over the resolved version before anything is
// rendered or sent. What does not fit is reported as warnings.
Overlay *Overlay
}
SendInput is one standalone send.
type SendResult ¶
type SendResult struct {
RunID string
Seed uint64
Status Status
Record *Record // nil when nothing was sent
Vars map[string]any // variables set by hooks
Diags diag.List
Saved []string // files written (saved responses)
// Err is a system failure that did not stop the send (saving a
// response or the cookie jar failed).
Err error
}
SendResult is a finished send. Everything in it is masked.
type SentRequest ¶
type SentRequest struct {
Method string `json:"method"`
URL string `json:"url"`
Header []Header `json:"headers,omitempty"`
Body *Body `json:"body,omitempty"`
}
SentRequest is the request as actually sent, masked.
Source Files
¶
- adhoc.go
- async.go
- auth.go
- build.go
- config.go
- connections.go
- credentials.go
- datafile.go
- dataset.go
- doc.go
- events.go
- execute.go
- family.go
- frame.go
- fuzzsubject.go
- grpc.go
- har.go
- hookrun.go
- hooks.go
- inflight.go
- inputs.go
- lanes.go
- limits.go
- noderun.go
- outcomes.go
- outputs.go
- overlay.go
- overrides.go
- preview.go
- record.go
- reportbuild.go
- run.go
- runner.go
- runtill.go
- runtime.go
- save.go
- schedule.go
- secrets.go
- send.go
- vars.go