Documentation
¶
Overview ¶
Package netapi gives VegaLoad's scripting runtimes (internal/scripting/js and /python) the real HTTP and WebSocket access FR-CLI-08 asks for: a scenario script can make an actual network call and read the response (status, headers, body) back into the script, unlike internal/protocol's drivers, which exist to repeat one fixed Target and discard the response body entirely -- see protocol.Result's doc comment, and http1.Driver.Do, which io.Copy's the body straight to io.Discard. Without the body coming back, a script has nothing to carry from one call into the next, which is the entire point of FR-CLI-08.
netapi knows nothing about JavaScript or Python: js.go and python.go are responsible for presenting these types in whatever shape their language needs (a goja.Object for JS; a proxied RPC call for Python, whose subprocess can't hold a Go value at all). It also knows nothing about FR-CLI-06's allowlist policy itself -- SafetyCheck is a hook the caller (cmd/vegaload) supplies, so the policy decision lives in one place (internal/safety) and this package only ever enforces whatever decision it's given.
Index ¶
- func ProtoFunctions() map[string][]string
- func RecordXferMeasures(rec CallRecorder, protocol, op string, ok bool, t XferTiming)
- func RunEnded(ctx context.Context) bool
- func ScriptHeaders(h http.Header) map[string]string
- type CallRecorder
- type CheckRecorder
- type Dataset
- type HTTPClient
- type HTTPResponse
- type Inputs
- type KafkaRecord
- type Options
- type ProtoCall
- type ProtoClient
- func (c *ProtoClient) Call(ctx context.Context, name string, call ProtoCall) (*ProtoReply, error)
- func (c *ProtoClient) Close()
- func (c *ProtoClient) FTP(ctx context.Context, mode string, call ProtoCall) (*ProtoReply, error)
- func (c *ProtoClient) GRPC(ctx context.Context, call ProtoCall) (*ProtoReply, error)
- func (c *ProtoClient) Kafka(ctx context.Context, mode string, call ProtoCall) (*ProtoReply, error)
- func (c *ProtoClient) MQTT(ctx context.Context, mode string, call ProtoCall) (*ProtoReply, error)
- func (c *ProtoClient) MySQL(ctx context.Context, call ProtoCall) (*ProtoReply, error)
- func (c *ProtoClient) Postgres(ctx context.Context, call ProtoCall) (*ProtoReply, error)
- func (c *ProtoClient) RabbitMQ(ctx context.Context, mode string, call ProtoCall) (*ProtoReply, error)
- func (c *ProtoClient) Redis(ctx context.Context, call ProtoCall) (*ProtoReply, error)
- func (c *ProtoClient) SFTP(ctx context.Context, mode string, call ProtoCall) (*ProtoReply, error)
- func (c *ProtoClient) SetCallRecorder(r CallRecorder)
- func (c *ProtoClient) TCP(ctx context.Context, call ProtoCall) (*ProtoReply, error)
- func (c *ProtoClient) UDP(ctx context.Context, call ProtoCall) (*ProtoReply, error)
- type ProtoMessage
- type ProtoReply
- type RabbitMessage
- type SafetyCheck
- type StepRecorder
- type WSConn
- type XferEntry
- type XferTiming
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ProtoFunctions ¶ added in v0.6.0
ProtoFunctions lists the calls a script can make, by namespace: tcp.send, mqtt.publish and so on. Each scripting language builds its globals from this list, so they cannot differ. It stays out of line so the compiler does not copy this body into every caller. That copy is what makes the binary larger.
func RecordXferMeasures ¶ added in v0.9.0
func RecordXferMeasures(rec CallRecorder, protocol, op string, ok bool, t XferTiming)
RecordXferMeasures records the phases an FTP or SFTP call ran. The times are whole milliseconds, so a 0 can mean "did not run" or "ran in under 1 ms". A call that worked ran every phase its mode has, so those are recorded even at 0. A call that failed is recorded only for a phase that reports a time above 0, because a phase it never reached also reports 0. connect_time and login_time are recorded only above 0: a reused connection reports 0, and a 0 would pull the percentiles down. A connect or login under 1 ms is therefore not counted.
func RunEnded ¶ added in v0.6.0
RunEnded reports whether the run is over: ctx is done, or its deadline has passed. The timer that cancels a ctx at its deadline can fire a few milliseconds late, and ctx.Err() stays nil until it does. Scripts use this to tell a call that was cut off by the end of the run from a real failure, in that gap too.
func ScriptHeaders ¶ added in v0.7.0
ScriptHeaders is the header map a scenario sees. Keys keep Go's canonical form (Set-Cookie, X-Request-Id). Every Set-Cookie value is kept, joined with newlines, because a comma join would split an Expires date. Every other header keeps its first value.
Types ¶
type CallRecorder ¶ added in v0.9.0
type CallRecorder interface {
RecordCall(protocol, op string, d time.Duration, failed bool, sent, received int64)
RecordMeasure(protocol, name string, d time.Duration)
}
CallRecorder receives every call a script makes to a protocol, and every call a protocol driver makes in -protocol mode. The report Collector implements it. A nil recorder is allowed everywhere it is accepted, and means the call still runs but is not counted.
type CheckRecorder ¶ added in v0.4.0
CheckRecorder receives the outcome of every check a scenario script makes (FR-CLI-12): the check's name and whether it passed. The report Collector implements it. A nil recorder is allowed everywhere it is accepted, and means checks are evaluated but not counted.
type Dataset ¶ added in v0.4.0
Dataset is one data file's rows. Next hands out the rows in order, one per call, shared by all virtual users, and starts over after the last. Random returns any row. Rows are read-only to the script.
type HTTPClient ¶
type HTTPClient struct {
// contains filtered or unexported fields
}
HTTPClient is one VU's HTTP client: one underlying *http.Client (and its connection pool) reused across every call for the life of the VU, the same keep-alive behaviour internal/protocol/http1.Driver gives a fixed target, just now available for a script's own ad hoc URLs.
func NewHTTPClient ¶
func NewHTTPClient(check SafetyCheck, timeout time.Duration) *HTTPClient
NewHTTPClient returns an HTTPClient that runs check (if non-nil) against every call's target host before connecting. timeout bounds each individual request the same way protocol/http1.New's timeout does; 0 means no per-request timeout beyond the context passed to Do.
func (*HTTPClient) Close ¶
func (c *HTTPClient) Close()
Close releases the client's idle connections. Safe to call even if no request was ever made.
func (*HTTPClient) Do ¶
func (c *HTTPClient) Do(ctx context.Context, method, rawURL string, body []byte, opts Options) (*HTTPResponse, error)
Do performs one HTTP request. method defaults to GET if empty; body may be nil. opts.InsecureSkipVerify, when set, uses a dedicated one-shot transport for this call rather than mutating the client's shared one, since a script may hit several hosts with different TLS needs across one iteration.
func (*HTTPClient) SetCallRecorder ¶ added in v0.9.0
func (c *HTTPClient) SetCallRecorder(r CallRecorder)
SetCallRecorder counts this client's HTTP calls. Nil means not counted.
type HTTPResponse ¶
HTTPResponse is what one HTTPClient.Do call hands back to a script -- enough to inspect a status, read a header, and -- the entire point of FR-CLI-08 -- parse a body to carry a value into the next call.
type Inputs ¶ added in v0.4.0
type Inputs interface {
// Env returns the value of an exposed environment variable.
Env(name string) (string, bool)
// EnvNames lists the exposed variable names.
EnvNames() []string
// Dataset returns the data file registered under name.
Dataset(name string) (Dataset, bool)
// DatasetNames lists the registered data file names.
DatasetNames() []string
}
Inputs is what a scenario script can read from outside itself (FR-CLI-17): environment variables the run was told to expose, and data files that feed rows to virtual users. The cmd layer builds it from the run's flags; a nil Inputs means the script can read nothing, so `env` is empty and `data` has no files.
type KafkaRecord ¶ added in v0.6.0
KafkaRecord is one Kafka record a call produced or read.
type Options ¶
type Options struct {
Headers map[string]string
InsecureSkipVerify bool
// Recorder counts a WebSocket connect. Scripts do not set it. HTTP
// uses the recorder on HTTPClient instead.
Recorder CallRecorder
}
Options are per-call overrides shared by HTTP and WebSocket calls, the scripting-layer equivalent of protocol.Target's own option fields.
type ProtoCall ¶ added in v0.6.0
type ProtoCall struct {
// URL is the target. tcp and udp accept a bare host:port too.
URL string
// Body is the data to send.
Body []byte
// Options are the driver's own options, the same keys as -opt.
Options map[string]string
// Insecure turns off TLS certificate checks.
Insecure bool
// Timeout bounds the whole call. Zero means the client's default.
Timeout time.Duration
// Password is the MQTT, Kafka, PostgreSQL, MySQL, Redis, RabbitMQ, FTP or SFTP password. nil means none. A script passes it
// here, from `env`, so it never has to be put in Options.
Password *string
// KeyPassphrase is the passphrase of an SFTP private key, from key_passphrase. nil means none.
KeyPassphrase *string
// Headers are the gRPC metadata of a grpc.call.
Headers map[string]string
}
ProtoCall is one call a script makes to a tcp, udp, mqtt, kafka, grpc, postgres, mysql, redis, rabbitmq or ftp target.
func ProtoCallFromArgs ¶ added in v0.6.0
ProtoCallFromArgs builds a call from a target and the options object a script passed. body, insecure, timeout and password are the script's own keys. For kafka, value is the same as body. Every other key becomes a driver option, and the driver rejects the ones it does not know. A trailing underscore is dropped from a key, so a language where a word such as "from" is reserved can write from_.
type ProtoClient ¶ added in v0.6.0
type ProtoClient struct {
// contains filtered or unexported fields
}
ProtoClient makes tcp, udp and mqtt calls for one VU. Each call opens its own connection, the same as the load-test drivers do, so the client holds no connection, except the Kafka clients that Close releases.
func NewProtoClient ¶ added in v0.6.0
func NewProtoClient(check SafetyCheck, timeout time.Duration) *ProtoClient
NewProtoClient returns a ProtoClient. check, when not nil, is run on the host of every call before it connects. timeout is the default for a call.
func (*ProtoClient) Call ¶ added in v0.6.0
func (c *ProtoClient) Call(ctx context.Context, name string, call ProtoCall) (*ProtoReply, error)
Call makes the call that name says, such as "tcp.send" or "kafka.admin". A finished call is recorded once here. The typed methods do not record, so a test that calls them directly is not counted.
func (*ProtoClient) Close ¶ added in v0.6.0
func (c *ProtoClient) Close()
Close closes the Kafka and gRPC clients, the SQL pools and the Redis clients. Safe to call more than once.
func (*ProtoClient) FTP ¶ added in v0.7.0
func (c *ProtoClient) FTP(ctx context.Context, mode string, call ProtoCall) (*ProtoReply, error)
FTP does one FTP job. mode is connect, download, upload, list, stat, delete or roundtrip. The session pool is kept for the next call with the same connection options.
func (*ProtoClient) GRPC ¶ added in v0.6.0
func (c *ProtoClient) GRPC(ctx context.Context, call ProtoCall) (*ProtoReply, error)
GRPC makes one unary gRPC call. Its connection is kept for the next call to the same target, as a real client keeps its channel.
func (*ProtoClient) Kafka ¶ added in v0.6.0
func (c *ProtoClient) Kafka(ctx context.Context, mode string, call ProtoCall) (*ProtoReply, error)
Kafka does one Kafka job. mode is produce, consume, roundtrip or admin. The client is kept for the next call with the same connection options.
func (*ProtoClient) MQTT ¶ added in v0.6.0
func (c *ProtoClient) MQTT(ctx context.Context, mode string, call ProtoCall) (*ProtoReply, error)
MQTT does one mqtt job. mode is publish, subscribe or roundtrip.
func (*ProtoClient) MySQL ¶ added in v0.7.0
func (c *ProtoClient) MySQL(ctx context.Context, call ProtoCall) (*ProtoReply, error)
MySQL runs one SQL text. The pool is kept for the next call with the same connection options, as an application keeps its pool. It has one connection, because a script is one user, unless pool says more.
func (*ProtoClient) Postgres ¶ added in v0.7.0
func (c *ProtoClient) Postgres(ctx context.Context, call ProtoCall) (*ProtoReply, error)
Postgres runs one SQL text. The pool is kept for the next call with the same connection options, as an application keeps its pool. It has one connection, because a script is one user, unless pool says more.
func (*ProtoClient) RabbitMQ ¶ added in v0.7.0
func (c *ProtoClient) RabbitMQ(ctx context.Context, mode string, call ProtoCall) (*ProtoReply, error)
RabbitMQ does one RabbitMQ job. mode is publish, consume, roundtrip or admin. The connection is kept for the next call with the same connection options.
func (*ProtoClient) Redis ¶ added in v0.7.0
func (c *ProtoClient) Redis(ctx context.Context, call ProtoCall) (*ProtoReply, error)
Redis runs one command, or several as a pipeline. The client is kept for the next call with the same connection options. It has one connection, because a script is one user, unless pool says more.
func (*ProtoClient) SFTP ¶ added in v0.9.0
func (c *ProtoClient) SFTP(ctx context.Context, mode string, call ProtoCall) (*ProtoReply, error)
SFTP does one SFTP job. mode is connect, download, upload, list, stat, delete or roundtrip. The connections are kept for the next call with the same connection options.
func (*ProtoClient) SetCallRecorder ¶ added in v0.9.0
func (c *ProtoClient) SetCallRecorder(r CallRecorder)
SetCallRecorder counts calls made through Call. Nil means not counted.
func (*ProtoClient) TCP ¶ added in v0.6.0
func (c *ProtoClient) TCP(ctx context.Context, call ProtoCall) (*ProtoReply, error)
TCP sends over one TCP connection and reads the reply the options ask for.
func (*ProtoClient) UDP ¶ added in v0.6.0
func (c *ProtoClient) UDP(ctx context.Context, call ProtoCall) (*ProtoReply, error)
UDP sends one datagram, and waits for a reply if the options ask for it.
type ProtoMessage ¶ added in v0.6.0
ProtoMessage is one MQTT message a call received.
type ProtoReply ¶ added in v0.6.0
type ProtoReply struct {
OK bool
Error string
BytesSent int64
BytesReceived int64
// Body is the reply read from a tcp or udp target. It is set even when
// the call failed after some bytes came in.
Body []byte
// Messages are the messages an mqtt subscribe or roundtrip received.
Messages []ProtoMessage
// IsKafka is true for a Kafka reply, the only one with Records and Text.
IsKafka bool
// Records are the records a kafka produce, consume or roundtrip
// wrote or read. For produce, Partition and Offset say where the broker
// stored each one.
Records []KafkaRecord
// Text is the answer of a kafka admin action.
Text string
// IsGRPC is true for a gRPC reply. Body is then the reply as JSON text
// (or base64), JSON is that text parsed, and GRPCStatus is the status
// code, 0 when the call worked.
IsGRPC bool
GRPCStatus int
GRPCStatusName string
JSON any
// IsSQL is true for a PostgreSQL or MySQL reply. SQLRows hold the first
// max_rows rows in the order of SQLColumns. SQLRowCount counts them all.
// commandTag is set only for PostgreSQL. lastInsertId is set only for MySQL.
IsSQL bool
SQLColumns []string
SQLRows [][]any
SQLRowCount int
SQLRowsAffected int64
SQLCommandTag string
SQLHasCommandTag bool
SQLLastInsertID int64
SQLHasLastInsert bool
// IsRedis is true for a Redis reply. Value is the last command's reply.
// Values has one entry per command. RowCount counts the last reply.
IsRedis bool
RedisValue any
RedisValues []any
RedisRowCount int
// IsRabbit is true for a RabbitMQ reply. RabbitMessages are the messages
// a consume or roundtrip read. RabbitText is an admin answer.
IsRabbit bool
RabbitMessages []RabbitMessage
RabbitText string
// Refused is true when the call was stopped before any bytes moved.
Refused bool
// IsXfer is true for an FTP or SFTP reply. XferEntries are the names a
// list or stat returned. XferText is the short answer. Times are UTC.
IsXfer bool
XferEntries []XferEntry
XferSize int64
XferTotal int
XferTiming XferTiming
XferText string
}
ProtoReply is what a call hands back. A call that fails on the network (refused, timed out, no reply) is a reply with OK false and Error set, not a Go error: a script is expected to look at it.
func (*ProtoReply) Fields ¶ added in v0.6.0
func (r *ProtoReply) Fields() map[string]any
Fields is the reply as plain data: the shape a script sees. body is the reply of tcp and udp (and of grpc, as JSON text, with json holding it parsed, and status and statusName for the gRPC code). messages is for mqtt, each {topic, body}. kafka adds records (each {topic, partition, offset, key, value}) and text. A RabbitMQ reply uses messages too, each {exchange, routingKey, body, messageId, redelivered}, and text for an admin answer. error is "" when the call worked.
type RabbitMessage ¶ added in v0.7.0
type RabbitMessage struct {
Exchange string
RoutingKey string
Body string
MessageID string
Redelivered bool
}
RabbitMessage is one AMQP message a consume or roundtrip read.
type SafetyCheck ¶
SafetyCheck is called with the bare host netapi is about to connect to, before every HTTP request and WebSocket dial. Returning a non-nil error refuses the call; the caller (cmd/vegaload) builds this from FR-CLI-06's allowlist and the run's -yes/-allow-target flags, enforced per call since a scripted target is only known once the script runs -- see run.go's doc comment for why that differs from protocol-direct mode's one pre-run check.
A nil SafetyCheck allows every host unchecked; HTTPClient and Dial treat it that way, so a test (or a future caller with no policy of its own) doesn't have to supply a no-op function.
type StepRecorder ¶ added in v0.6.0
StepRecorder receives every named step a scenario script runs (FR-CLI-18): the step's name, how long it took, and whether it failed. The report Collector implements it. A nil recorder is allowed everywhere it is accepted, and means steps still run but are not counted.
type WSConn ¶
type WSConn struct {
// contains filtered or unexported fields
}
WSConn is one open WebSocket connection, kept alive across multiple Send/Receive calls -- unlike internal/protocol/websocket.Driver, which opens, sends at most one message, reads at most one reply, and closes again on every single Do call. FR-CLI-08's own example (read frames until a flow completes or aborts) needs exactly this: a connection a script can read from repeatedly.
func Dial ¶
func Dial(ctx context.Context, check SafetyCheck, rawURL string, timeout time.Duration, opts Options) (*WSConn, error)
Dial opens a WebSocket connection to rawURL (ws:// or wss://), after running check against its host. timeout bounds the handshake only -- Receive has no deadline of its own unless the caller gives one, since a long-lived connection reading many frames shouldn't inherit a one-shot handshake budget.
func (*WSConn) Close ¶
Close closes the connection, sending a normal-closure frame on a best-effort basis first. Safe to call more than once.
func (*WSConn) Receive ¶
Receive waits for the next message and returns its payload. closed is true, with a nil error, on a normal close -- so a script's "read frames until it completes" loop can just check for that instead of treating the expected end of the stream as an error. A non-nil error means something other than a clean close: a read past ctx's deadline, or a protocol error.
Receive honors ctx's deadline if it has one; otherwise it blocks until a message arrives, the connection closes, or ctx is cancelled for some other reason (the run ending).
type XferTiming ¶ added in v0.9.0
type XferTiming struct {
ConnectMs int64
LoginMs int64
TransferMs int64
UploadMs int64
DownloadMs int64
DeleteMs int64
}
XferTiming is how long the parts of an FTP or SFTP call took, in milliseconds.
func XferTimingFrom ¶ added in v0.9.0
func XferTimingFrom(t xfercommon.Timing) XferTiming
XferTimingFrom copies phase times from a file-transfer driver.