Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type BatchWriter ¶
type BatchWriter struct {
// contains filtered or unexported fields
}
BatchWriter is a kafka writer that uses batches to send the data to the configured kafka topic.
func NewBatchWriter ¶
func NewBatchWriter(config *Config, opts ...Option) (*BatchWriter, error)
func (*BatchWriter) Close ¶
func (w *BatchWriter) Close() error
func (*BatchWriter) Name ¶
func (w *BatchWriter) Name() string
func (*BatchWriter) ProcessWALEvent ¶
ProcessWalEvent is called on every new message from the wal. It can be called concurrently.
type Config ¶
type Config struct {
Kafka kafka.ConnConfig
// BatchTime is the max time interval at which the batch sending to kafka is
// triggered. Defaults to 1s
BatchTimeout time.Duration
// BatchBytes is the max size in bytes for a given batch. When this size is
// reached, the batch is sent to kafka. Defaults to 1572864 bytes.
BatchBytes int64
// BatchSize is the max number of messages to be sent per batch. When this
// size is reached, the batch is sent to kafka. Defaults to 100.
BatchSize int
// MaxQueueBytes is the max memory used by the batch writer for inflight
// batches. Defaults to 100MiB
MaxQueueBytes int64
}
type Option ¶
type Option func(*BatchWriter)
func WithCheckpoint ¶
func WithCheckpoint(c checkpointer.Checkpoint) Option
func WithInstrumentation ¶
func WithInstrumentation(i *otel.Instrumentation) Option
func WithLogger ¶
Click to show internal directories.
Click to hide internal directories.