streaming

package
v0.10.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 21, 2025 License: MIT Imports: 17 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type FlightDataReader

type FlightDataReader interface {
	Schema() *arrow.Schema
	Recv() (*flight.FlightData, error)
}

FlightDataReader is an interface that flight.NewRecordReader expects. flight.FlightService_DoPutServer implements this.

type MetricsCollector

type MetricsCollector interface {
	IncrementCounter(name string, labels ...string)
	StartTimer(name string) Timer
	RecordHistogram(name string, value float64, labels ...string)
}

MetricsCollector defines a subset of metrics collection capabilities needed by the service. This should align with the one in cmd/server/server/server.go or a shared package.

type StreamingRepository

type StreamingRepository interface {
	// IngestStream ingests Arrow RecordBatches into the database.
	// transactionID is the optional transaction identifier if the operation is part of a transaction.
	// target is a generic identifier for the data destination (e.g., table name).
	// schema is the schema of the incoming data.
	// reader provides the stream of RecordBatches.
	// Returns the number of records ingested and any error.
	IngestStream(ctx context.Context, transactionID string, target string, schema *arrow.Schema, reader flight.MessageReader) (int64, error)
}

StreamingRepository defines the interface for ingesting streaming data into the database.

func NewDuckDBStreamingRepository

func NewDuckDBStreamingRepository(p pool.ConnectionPool, alloc memory.Allocator, logger zerolog.Logger) StreamingRepository

NewDuckDBStreamingRepository creates a new DuckDBStreamingRepository.

type StreamingService

type StreamingService interface {
	// HandleDoPut streams data into a specified target.
	// The desc parameter provides context, including potentially the target identifier.
	// reader is the stream of Arrow RecordBatches.
	// writer is used to send back metadata/results (e.g., rows written or error messages).
	HandleDoPut(ctx context.Context, desc *flight.FlightDescriptor, schema *arrow.Schema, reader flight.MessageReader, writer flight.MetadataWriter) error
}

StreamingService defines the interface for handling streaming data operations.

func NewService

func NewService(repository StreamingRepository, logger zerolog.Logger, metrics MetricsCollector) StreamingService

NewService creates a new StreamingService.

type Timer

type Timer interface {
	Stop() float64
}

Timer defines a simple timer interface for metrics, compatible with server.Timer.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL