Documentation
¶
Overview ¶
Package gatherer buffers incoming events in-process and flushes them to ClickHouse in batches (ADRs 0004, 0018). It implements model.Ingester: Ingest validates each row, quarantines the bad ones, buffers the good ones, and acks immediately (the API returns 202) — a single background flusher drains the buffer on max(flushSize rows, flushInterval). On graceful Close the remaining buffer is drained so nothing is lost on clean shutdown.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Gatherer ¶
type Gatherer struct {
// contains filtered or unexported fields
}
Gatherer is the in-process event buffer + flusher.
func New ¶
func New(ins model.CHInserter, reg model.DatasourceRegistry, opts ...Option) *Gatherer
New builds a Gatherer and starts its background flusher. Call Close to drain.
func (*Gatherer) Close ¶
Close stops the flusher and drains the buffer, returning when the final flush completes or ctx expires (graceful drain, ADR 0004). Safe to call once.
func (*Gatherer) Ingest ¶
func (g *Gatherer) Ingest(ctx context.Context, datasource string, rows []json.RawMessage) (successful, quarantined int, err error)
Ingest validates rows for a datasource and buffers them (ack-on-buffer, ADR 0004). Bad rows are quarantined rather than rejecting the batch (ADR 0018); an unknown datasource is the only per-call error (the API maps it to 4xx).
type Option ¶
type Option func(*Gatherer)
Option configures a Gatherer.
func WithFlushInterval ¶
WithFlushInterval sets the max time a row waits before flushing (default 5s).
func WithFlushSize ¶
WithFlushSize sets the row count that triggers an early flush (default 10000).
func WithLogger ¶
WithLogger sets the slog logger (default: slog.Default).
Source Files