storage

package

v0.2.0 Latest Latest Go to latest Published: Mar 28, 2026 License: MIT Imports: 6 Imported by: 0

Details

This section is empty.

This section is empty.

This section is empty.

type ColumnarDataset struct {
	// contains filtered or unexported fields
}

ColumnarDataset represents a columnar dataset stored in the Arrow IPC file format.

func NewColumnarDataset(rw io.ReadWriteSeeker, schema *arrow.Schema) (*ColumnarDataset, error)

NewColumnarDataset creates a new ColumnarDataset for writing.

func NewColumnarDatasetReader(rs ipc.ReadAtSeeker) (*ColumnarDataset, error)

NewColumnarDatasetReader creates a new ColumnarDataset for reading.

func NewColumnarDatasetReaderFromReader(r io.Reader) (*ColumnarDataset, error)

NewColumnarDatasetReaderFromReader creates a new ColumnarDataset for reading from any io.Reader.

func OpenForReading(r ipc.ReadAtSeeker) (*ColumnarDataset, error)

OpenForReading initializes the dataset for reading from an Arrow IPC file.

func OpenForReadingFromPipe(r io.Reader) (*ColumnarDataset, error)

OpenForReadingFromPipe initializes the dataset for reading from an Arrow IPC stream.

func (cd *ColumnarDataset) Chunk(i int) (arrow.Record, error)

Chunk returns a specific chunk by its index.

func (cd *ColumnarDataset) Close() error

func (cd *ColumnarDataset) NumChunks() int

NumChunks returns the number of chunks in the dataset.

func (cd *ColumnarDataset) Read() (arrow.Record, error)

func (cd *ColumnarDataset) Schema() *arrow.Schema

Schema returns the schema of the dataset.

func (cd *ColumnarDataset) Write(rec arrow.Record) error

type Writer interface {
	Write(arrow.Record) error
	Close() error
}

Writer is an interface for writing Arrow record batches.

func NewWriter(w io.Writer, schema *arrow.Schema) (Writer, error)

NewWriter creates a new Arrow writer.