kafka

package
v0.9.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 15, 2026 License: Apache-2.0 Imports: 24 Imported by: 0

Documentation

Overview

Package kafka implements the "kafka" protocol.Protocol driver, for load testing an Apache Kafka cluster (or any broker that speaks the Kafka protocol). It uses the franz-go client, which is pure Go.

The job of each call to Do is chosen with -opt mode=...:

produce    send count records and wait for the broker's ack. This is
           the default.
consume    read count records from the topic and check them.
roundtrip  produce count records, then read exactly those records
           back. This measures the way through the broker.
admin      do one admin action, chosen with -opt action=...

Options (set with -opt key=value):

mode           produce (default), consume, roundtrip, or admin.
topic          The topic. Required, except for the admin actions that
               list things. {id} is replaced by this call's unique id.
key            produce and roundtrip: the record key. {id} is replaced.
acks           produce: all (default), leader, or none. roundtrip
               needs the offset, so it does not accept none.
compression    produce: none (default), gzip, snappy, lz4, or zstd.
count          produce, consume, roundtrip: records per call (default 1).
expect         consume: every record value must contain this text.
               admin list actions: the listing must contain it.
from           consume: start (default) or end.
action         admin: list_topics, create_topic, delete_topic,
               topic_lifecycle (create and then delete), list_groups,
               or describe_cluster.
partitions     admin create: partitions of the new topic (default 1).
replication    admin create: replication factor (default 1).
sasl           plain, scram-sha-256, or scram-sha-512.
username       The SASL user name. Needs sasl.
password_env   The name of an environment variable that holds the SASL
               password. Needs username. The password is never put on
               the command line.
client_id      The client id sent to the brokers (default "vegaload").

The target is kafka://host:port (default port 9092) or kafkas://host:port (TLS, default port 9093). It is the first broker to ask. The client then connects to the broker addresses the cluster announces, so those must be reachable too.

produce, roundtrip and admin share one client between all users, as a real producer does. consume and the reading half of roundtrip open a new connection for each call, because each call reads its own part of the topic. That connect time is part of the measured time.

Index

Constants

This section is empty.

Variables

View Source
var Options = append([]string{
	"mode", "topic", "key", "acks", "compression", "count", "expect", "from",
	"action", "partitions", "replication", "sasl", "username", "password_env",
	"client_id",
}, tlsconf.Keys...)

Options are the -opt keys this driver accepts. The command's protocol table uses the same list, so the two cannot differ.

Functions

This section is empty.

Types

type Driver

type Driver struct {
	// contains filtered or unexported fields
}

Driver is a Kafka protocol.Protocol.

func New

func New(target protocol.Target, timeout time.Duration) (*Driver, error)

New returns a ready-to-use Driver for target. target.URL is kafka://host[:port] or kafkas://host[:port].

New returns an error only for a configuration problem, never for the cluster being unreachable, which Do reports per call.

func NewConn

func NewConn(target protocol.Target, timeout time.Duration, password *string) (*Driver, error)

NewConn is New for a scenario script. It makes only the connection: the client, its login and its producer settings (client_id, sasl, username, acks, compression). It needs no topic and no mode. The job of each call is then set with Call. password is the SASL password, which the script already holds, so password_env is not used with it.

func (*Driver) Call

func (d *Driver) Call(opts map[string]string, body []byte, timeout time.Duration) (*Driver, error)

Call returns a Driver for one call of a script: the same connection as d, with the job that opts and body describe. opts must hold the same connection options d was made with. The result shares d's client, so closing it is d's job, not the caller's.

func (*Driver) Close

func (d *Driver) Close() error

Close implements protocol.Protocol. It closes the shared client.

func (*Driver) Do

func (d *Driver) Do(parent context.Context) (protocol.Result, error)

Do implements protocol.Protocol.

func (*Driver) Name

func (d *Driver) Name() string

Name implements protocol.Protocol.

func (*Driver) Run

func (d *Driver) Run(parent context.Context) (protocol.Result, Reply)

Run is Do, and it also returns the records and text of the call. A load test does not need them. A scenario script does.

type Record

type Record struct {
	Topic     string
	Partition int32
	Offset    int64
	Key       []byte
	Value     []byte
}

Record is one Kafka record: one that was produced (its partition and offset are where the broker stored it) or one that was read.

type Reply

type Reply struct {
	Records []Record
	Text    string
}

Reply is what a call read or wrote, besides the result: the records of produce, consume and roundtrip, and the text answer of an admin action.

Directories

Path Synopsis
Package kafkatest is a small fake Kafka broker for tests, so the Kafka driver and the scripting layer can be tested without Docker.
Package kafkatest is a small fake Kafka broker for tests, so the Kafka driver and the scripting layer can be tested without Docker.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL