columnifier

package

v0.0.0-...-8f686df Latest Latest Go to latest Published: May 31, 2025 License: Apache-2.0 Imports: 10 Imported by: 0

Details

This section is empty.

This section is empty.

func NewParquetColumnifier(st string, sf string, rt string, output string, config Config) (*parquetColumnifier, error)

NewParquetColumnifier creates a new parquetColumnifier.

type Columnifier interface {
	WriteFromReader(reader io.Reader) (int, error)
	WriteFromFiles(paths []string) (int, error)
	Close() error
}

Columnifier is the interface that converts input file to columnar format file.

func NewColumnifier(st string, sf string, rt string, o string, config Config) (Columnifier, error)

NewColumnifier creates a new Columnifier.

type Config struct {
	Parquet Parquet
}

func NewConfig(parquetPageSize, parquetRowGroupSize int64, parquetCompressionCodec string) (*Config, error)

type Parquet struct {
	PageSize         int64
	RowGroupSize     int64
	CompressionCodec parquet.CompressionCodec
}