kafka

package
v1.2.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 22, 2026 License: Apache-2.0 Imports: 14 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type BatchWriter

type BatchWriter struct {
	// contains filtered or unexported fields
}

BatchWriter is a kafka writer that uses batches to send the data to the configured kafka topic.

func NewBatchWriter

func NewBatchWriter(ctx context.Context, config *Config, opts ...Option) (*BatchWriter, error)

func (*BatchWriter) Close

func (w *BatchWriter) Close() error

func (*BatchWriter) Name

func (w *BatchWriter) Name() string

func (*BatchWriter) ProcessWALEvent

func (w *BatchWriter) ProcessWALEvent(ctx context.Context, walEvent *wal.Event) (retErr error)

ProcessWALEvent is called on every new message from the wal. It can be called concurrently.

type Config

type Config struct {
	Kafka kafka.ConnConfig
	Batch batch.Config
	// PartitionKey determines the message key strategy used for DML events,
	// and therefore how they are distributed across the topic partitions. One
	// of "schema" (default), "table" or "primary_key".
	PartitionKey PartitionKey
}

type Option

type Option func(*BatchWriter)

func WithCheckpoint

func WithCheckpoint(c checkpointer.Checkpoint) Option

func WithInstrumentation

func WithInstrumentation(i *otel.Instrumentation) Option

func WithLogger

func WithLogger(l loglib.Logger) Option

type PartitionKey added in v1.2.2

type PartitionKey string

PartitionKey represents the message key strategy used for DML events.

const (
	// PartitionKeySchema keys messages by schema name. All events for a given
	// schema are routed to the same partition, which guarantees ordering per
	// schema. This is the default.
	PartitionKeySchema PartitionKey = "schema"
	// PartitionKeyTable keys messages by schema qualified table name. Events
	// for a given table are routed to the same partition, which guarantees
	// ordering per table, but not across tables in the same schema.
	PartitionKeyTable PartitionKey = "table"
	// PartitionKeyPrimaryKey keys messages by schema qualified table name and
	// the event primary key values. Events for a given row are routed to the
	// same partition, which guarantees ordering per row, but not across rows
	// or against DDL events. Events without an identifiable primary key
	// degrade to table keying.
	PartitionKeyPrimaryKey PartitionKey = "primary_key"
)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL