Documentation
¶
Overview ¶
Package stream defines the VegaLoad run event stream (vegaload.io/stream/v1): the event format, the per-sink queues, and the file and push sinks.
The format is documented in docs/streaming.md. This package does not import the engine, the report package or any driver: the runner hands it plain values through the Emitter interface, so a run with no stream pays nothing and the core engine knows nothing about streaming (AGENTS.md, module boundaries).
Index ¶
- Constants
- func IsLoopbackHost(host string) bool
- func NewRunID() string
- func SanitizeTarget(target string) string
- func ValidRunID(id string) error
- func ValidatePushURL(raw string, insecure bool) (*url.URL, error)
- func Write(w io.Writer, events []Envelope) error
- type Aborter
- type Drainer
- type Emitter
- type Envelope
- type FileSink
- type Flusher
- type Point
- type PushOptions
- type PushSink
- func (s *PushSink) Abort()
- func (s *PushSink) Close() error
- func (s *PushSink) Deadline() (time.Time, bool)
- func (s *PushSink) Delivered() (delivered, lost int)
- func (s *PushSink) Drain()
- func (s *PushSink) Flush() error
- func (s *PushSink) Name() string
- func (s *PushSink) Warnings() []string
- func (s *PushSink) Write(line []byte) error
- type Recorder
- func (r *Recorder) Close(timeout time.Duration) []string
- func (r *Recorder) Failed(reason string, partial any)
- func (r *Recorder) Finished(result any, passed bool, exitCode int)
- func (r *Recorder) HasStarted() bool
- func (r *Recorder) Point(p Point)
- func (r *Recorder) RunID() string
- func (r *Recorder) Started(at time.Time, rs RunStarted)
- func (r *Recorder) Warning(msg string)
- func (r *Recorder) Warnings() []string
- type RecorderOptions
- type Recording
- type RunStarted
- type Sink
- type Stage
- type Target
- type ThresholdSpec
- type Warner
- type Warning
Constants ¶
const ( TypeRunStarted = "run.started" TypePoint = "point" TypeWarning = "warning" TypeRunFinished = "run.finished" TypeRunFailed = "run.failed" )
Event types.
const DefaultQueueDepth = 4096
DefaultQueueDepth bounds how many points a sink may have waiting.
const Schema = "vegaload.io/stream/v1"
Schema is the value of the schema field on every event.
Variables ¶
This section is empty.
Functions ¶
func IsLoopbackHost ¶
IsLoopbackHost reports whether host is localhost or a loopback IP.
func SanitizeTarget ¶
SanitizeTarget returns target without anything that can carry a credential: the userinfo, the query and the fragment of a URL, and a userinfo before an "@" in a plain host:port. Anything it cannot read as a URL it returns with every "@..." prefix and "?..." suffix cut off, so a secret is never passed on by accident.
func ValidRunID ¶
ValidRunID reports whether id can be used as a run id. The id ends up in file names and URLs, so the set is small.
func ValidatePushURL ¶
ValidatePushURL checks a push destination. It must be http or https, carry no credentials, and use https unless the host is loopback or insecure is set.
Types ¶
type Aborter ¶
type Aborter interface{ Abort() }
Aborter is implemented by sinks that can be stuck in a retry loop. Abort makes them give up, so Close can return.
type Drainer ¶
type Drainer interface{ Drain() }
Drainer is implemented by sinks that retry. Drain is called, from any goroutine, when the run is over and the recorder is about to wait for the sink for a bounded time. The sink should stop backing off and try for the rest of that time, because what is left includes the terminal event.
type Emitter ¶
type Emitter interface {
// Started begins the stream. at is the instant the run's clock started:
// offsetS on every event counts from it.
Started(at time.Time, rs RunStarted)
// Point reports one closed one-second bucket.
Point(p Point)
// Warning reports something the user should know.
Warning(msg string)
// Finished ends the stream with the run's result. result is marshalled
// as it is (the same value `-out` writes); passed and exitCode are added.
Finished(result any, passed bool, exitCode int)
// Failed ends the stream for a run that could not run or was interrupted.
// partial is the partial result, or nil.
Failed(reason string, partial any)
}
Emitter is what a runner needs from a stream. The runner holds one as a nil-able interface value: with no stream there is no Emitter and no cost. Every method is safe to call from any goroutine and none of them blocks on a sink or returns an error.
type Envelope ¶
type Envelope struct {
Schema string `json:"schema"`
RunID string `json:"runId"`
Seq int64 `json:"seq"`
TS time.Time `json:"ts"`
OffsetS float64 `json:"offsetS"`
Labels map[string]string `json:"labels,omitempty"`
Synthetic bool `json:"synthetic,omitempty"`
Type string `json:"type"`
Data json.RawMessage `json:"data"`
}
Envelope wraps every event. Data stays raw so that readers keep fields and event types they do not know.
type FileSink ¶
type FileSink struct {
// contains filtered or unexported fields
}
FileSink writes events as NDJSON, one line per event, unbuffered, so that `tail -f` works and another process can follow the run.
func NewFileSink ¶
NewFileSink creates (or truncates) the file at path, and the directories above it.
type Flusher ¶
type Flusher interface {
// Deadline reports when the pending batch must be sent; ok is false when
// nothing is pending.
Deadline() (t time.Time, ok bool)
Flush() error
}
Flusher is implemented by sinks that batch. The runner calls Flush when the deadline passes with nothing new to write, and once more when the run is over.
type Point ¶
type Point struct {
OffsetS float64 `json:"offsetS"`
RPS float64 `json:"rps"`
Requests int64 `json:"requests"`
Failed int64 `json:"failed"`
ErrorRate float64 `json:"errorRate"`
P50Ms float64 `json:"p50Ms"`
P95Ms float64 `json:"p95Ms"`
P99Ms float64 `json:"p99Ms"`
Total int64 `json:"total"`
TotalFailed int64 `json:"totalFailed"`
}
Point is the data of point: one closed one-second bucket of the run.
type PushOptions ¶
type PushOptions struct {
URL *url.URL
Token string
Version string
// Zero values take the documented defaults.
MaxBatch int // 100 events
MaxDelay time.Duration // 500 ms from the first event of a batch
Timeout time.Duration // 10 s per request
BackoffMin time.Duration // 500 ms
BackoffMax time.Duration // 10 s
Client *http.Client
}
PushOptions configures a PushSink.
type PushSink ¶
type PushSink struct {
// contains filtered or unexported fields
}
PushSink POSTs batches of NDJSON to an ingest endpoint (docs/streaming.md, "Push"). One batch is in flight at a time, so a receiver sees batches in order, and a retried batch repeats the same sequence range.
2xx acknowledges a batch. Network errors, 408, 429 and 5xx are retried with exponential backoff until the run is over and the deadline given to Recorder.Close passes. Any other 4xx drops the batch and adds a warning.
func NewPushSink ¶
func NewPushSink(opt PushOptions) *PushSink
NewPushSink creates a sink for opt.URL, which must already have passed ValidatePushURL.
func (*PushSink) Delivered ¶
Delivered returns how many events the receiver acknowledged and how many were given up on.
func (*PushSink) Flush ¶
Flush sends the pending batch, retrying where the rules say so. It returns an error only when the sink was aborted, which ends it.
type Recorder ¶
type Recorder struct {
// contains filtered or unexported fields
}
Recorder turns what the runner reports into events and hands each event to every sink. It implements Emitter. Sequence numbers and timestamps are assigned once, so every sink sees the same events.
A Recorder never blocks the caller on a sink and never returns an error to it: a failing sink only adds a warning.
func NewRecorder ¶
func NewRecorder(opt RecorderOptions, sinks ...Sink) *Recorder
NewRecorder creates a Recorder that writes to sinks. With no sinks it still numbers events, which is useful in tests.
func (*Recorder) Close ¶
Close stops accepting events, waits up to timeout for the sinks to write (and, for batching sinks, send) what is queued, and closes them. A sink that is still busy at the deadline is aborted. It returns the names of the sinks that did not finish.
func (*Recorder) HasStarted ¶
HasStarted reports whether run.started has been emitted.
type RecorderOptions ¶
type RecorderOptions struct {
RunID string // default: NewRunID()
Labels map[string]string // attached to every event
// QueueDepth overrides DefaultQueueDepth (tests).
QueueDepth int
}
RecorderOptions configures a Recorder.
type Recording ¶
type Recording struct {
Path string
Events []Envelope
// Raw holds each event's original line, so that serving a recording
// forwards fields this version does not know.
Raw [][]byte
}
Recording is one run's events as read from an NDJSON file.
func Read ¶
Read parses an NDJSON event stream and checks its structure: every line is a JSON object with the v1 schema, one run id, a type, data, and a seq that increases by exactly one starting at 1. A recording with no events, or with events after a terminal event, is rejected. Unknown fields are kept.
type RunStarted ¶
type RunStarted struct {
Name string `json:"name"`
Executor string `json:"executor"`
VUs int `json:"vus,omitempty"`
Rate float64 `json:"rate,omitempty"`
Duration string `json:"duration,omitempty"`
Stages []Stage `json:"stages,omitempty"`
Target *Target `json:"target,omitempty"`
Thresholds []ThresholdSpec `json:"thresholds,omitempty"`
AbortOnBreach bool `json:"abortOnBreach,omitempty"`
Version string `json:"version,omitempty"`
}
RunStarted is the data of run.started. It carries the shape of the run and nothing a user supplied as a credential: no header, body, option, secret, environment value or data cell (docs/streaming.md, "run.started").
type Sink ¶
type Sink interface {
// Name is used in warnings, for example "file run.ndjson".
Name() string
Write(line []byte) error
Close() error
}
Sink receives the serialized events of a run, one call per event, in order. Write is called from a single goroutine per sink and may block: the Recorder puts a bounded queue in front of every sink, so a slow sink cannot delay the run. A Write error marks the sink dead; the rest of the run's events for that sink are discarded.
type ThresholdSpec ¶
type ThresholdSpec struct {
Name string `json:"name"`
Metric string `json:"metric"`
Operator string `json:"operator"`
Value string `json:"value"`
Step string `json:"step,omitempty"`
Protocol string `json:"protocol,omitempty"`
Op string `json:"op,omitempty"`
}
ThresholdSpec is one pass/fail threshold as the run was given it.