Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type ClickhouseSink ¶
type ClickhouseSink struct {
// contains filtered or unexported fields
}
ClickhouseSink inserts result batches into a ClickHouse table.
The Python sink hands the Arrow table to clickhouse_connect's insert_arrow, which maps Arrow columns to table columns by name. clickhouse-go has no Arrow entry point, so the batch is unpacked into rows against an explicit column list taken from the Arrow schema, which preserves that name-based mapping.
func NewClickhouseSink ¶
func NewClickhouseSink(conf config.ClickhouseSink) (*ClickhouseSink, error)
func (*ClickhouseSink) Batch ¶
func (s *ClickhouseSink) Batch() (arrow.Table, error)
Batch returns nothing. The Python ClickhouseSink alone among the sinks reports no batch: rows go straight to ClickHouse and are not held for a downstream reader.
func (*ClickhouseSink) Close ¶
func (s *ClickhouseSink) Close() error
func (*ClickhouseSink) Flush ¶
func (s *ClickhouseSink) Flush() error
func (*ClickhouseSink) WriteTable ¶
func (s *ClickhouseSink) WriteTable(batch arrow.Table) error
type ConsoleSink ¶
type ConsoleSink struct {
// contains filtered or unexported fields
}
ConsoleSink writes each result row to stdout as a JSON object.
func NewConsoleSink ¶
func NewConsoleSink() *ConsoleSink
func NewConsoleSinkTo ¶
func NewConsoleSinkTo(w io.Writer) *ConsoleSink
func (*ConsoleSink) Flush ¶
func (s *ConsoleSink) Flush() error
func (*ConsoleSink) WriteTable ¶
func (s *ConsoleSink) WriteTable(batch arrow.Table) error
type IcebergSink ¶
type IcebergSink struct {
// contains filtered or unexported fields
}
IcebergSink appends result batches to an Iceberg table.
func NewIcebergSink ¶
func NewIcebergSink(ctx context.Context, catalogName, tableName string) (*IcebergSink, error)
func (*IcebergSink) Flush ¶
func (s *IcebergSink) Flush() error
func (*IcebergSink) WriteTable ¶
func (s *IcebergSink) WriteTable(batch arrow.Table) error
type KafkaSink ¶
type KafkaSink struct {
// contains filtered or unexported fields
}
KafkaSink produces one message per result row, JSON encoded, matching the Python KafkaSink.
type SQLCommandSink ¶
type SQLCommandSink struct {
// contains filtered or unexported fields
}
SQLCommandSink materializes the result batch as a table and runs arbitrary DuckDB SQL against it. This is how the Python engine writes parquet, S3, Postgres, DuckLake and MotherDuck outputs.
func NewSQLCommandSink ¶
func NewSQLCommandSink(conn adbc.Connection, sql string, substitutions []config.SQLCommandSubstitution) (*SQLCommandSink, error)
func (*SQLCommandSink) Flush ¶
func (s *SQLCommandSink) Flush() error
func (*SQLCommandSink) WriteTable ¶
func (s *SQLCommandSink) WriteTable(batch arrow.Table) error
Source Files