soak_clients

package

v0.3553.0-7.3.0-0-ce Latest Latest Go to latest Published: May 28, 2022 License: Apache-2.0 Imports: 15 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/apache/kafka

Links

Open Source Insights

README ¶

This program orchestrates the Trogdor agents that operate on the Soak Cluster. It has multiple responsibilities:

Create Trogdor scenarios with realistic workloads that run on the cluster for extended periods of time.
Run periodic jobs that inspect the health and status of said workloads

Usage

To run the soak clients in CPD:

cd ../
vi charts/cc-soak-clients/values.yaml # edit bootstrapServer and apiKey/Secret
vi charts/cc-soak-clients/templates/configMaps.yaml # optionally modify topics.json
helm install charts/cc-trogdor-service/  # spawn trogdor coordinator and agents
helm install charts/cc-soak-clients/  # spawn soak clients and cronjob for status reporting

Possible future improvements

Allow for interactive task start/pause/stoppage
Support more diverse scenarios

Documentation ¶

Index ¶

Constants
func Report(topicConfigPath string, trogdorCoordinatorHost string)
func Run(topicConfigPath string, trogdorCoordinatorHost string, trogdorAgentsCount int, ...)
type SoakTestConfig
type TopicConfiguration

Constants ¶

View Source

const (
	UNIFORM_WORKLOAD  = "uniform"
	GAUSSIAN_WORKLOAD = "gaussian"
)

Variables ¶

This section is empty.

Functions ¶

func Report ¶

func Report(topicConfigPath string, trogdorCoordinatorHost string)

func Run ¶

func Run(topicConfigPath string, trogdorCoordinatorHost string, trogdorAgentsCount int, bootstrapServers string)

Types ¶

type SoakTestConfig ¶

type SoakTestConfig struct {
	Topics                          []TopicConfiguration `json:"topics"`
	LongLivedTaskDurationMs         uint64               `json:"long_lived_task_duration_ms"`
	ShortLivedTaskDurationMs        uint64               `json:"short_lived_task_duration_ms"`
	ShortLivedTaskRescheduleDelayMs uint64               `json:"short_lived_task_reschedule_delay_ms"`
}

type TopicConfiguration ¶

type TopicConfiguration struct {
	Name                                  string                          `json:"name"`
	PartitionsCount                       int                             `json:"partitions_count"`
	ProduceMBsThroughput                  float32                         `json:"produce_mbs_throughput"`
	ConsumeMBsThroughput                  float32                         `json:"consume_mbs_throughput"`
	LongLivedProduceCount                 int                             `json:"long_lived_producer_count"`
	ShortLivedProduceCount                int                             `json:"short_lived_producer_count"`
	LongLivedConsumeCount                 int                             `json:"long_lived_consumer_count"`
	ShortLivedConsumeCount                int                             `json:"short_lived_consumer_count"`
	TransactionsEnabled                   bool                            `json:"transactions_enabled"`
	IdempotenceEnabled                    bool                            `json:"idempotence_enabled"`
	WorkloadType                          string                          `json:"workload_type"`
	ShortLivedRandomConsumerGroup         bool                            `json:"short_lived_random_consumer_group"`
	ShortLivedConsumerRecordBatchVerifier trogdor.RecordBatchVerifierSpec `json:"short_lived_consumer_record_batch_verifier"`
	LongLivedRandomConsumerGroup          bool                            `json:"long_lived_random_consumer_group"`
	LongLivedConsumerRecordBatchVerifier  trogdor.RecordBatchVerifierSpec `json:"long_lived_consumer_record_batch_verifier"`
}

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL