Documentation
¶
Index ¶
- type Field
- type FileStats
- type ImportRequest
- type ImportResult
- type ParquetImporter
- func (p *ParquetImporter) Close() error
- func (p *ParquetImporter) GetTableLocation(tableIdent table.Identifier) string
- func (p *ParquetImporter) ImportTable(ctx context.Context, req ImportRequest) (*ImportResult, error)
- func (p *ParquetImporter) InferSchema(parquetFile string) (*Schema, *FileStats, error)
- type Schema
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Field ¶
type Field struct {
Name string `json:"name"`
Type string `json:"type"`
Nullable bool `json:"nullable"`
}
Field represents a single column in a schema
type FileStats ¶
type FileStats struct {
RecordCount int64 `json:"record_count"`
FileSize int64 `json:"file_size"`
ColumnCount int `json:"column_count"`
}
FileStats contains statistics about a Parquet file
type ImportRequest ¶
type ImportRequest struct {
ParquetFile string
TableIdent table.Identifier
NamespaceIdent table.Identifier
Schema *Schema
Overwrite bool
PartitionBy []string
}
ImportRequest contains all parameters for importing a table
type ImportResult ¶
type ImportResult struct {
TableIdent table.Identifier
RecordCount int64
DataSize int64
TableLocation string
}
ImportResult contains the results of a table import
type ParquetImporter ¶
type ParquetImporter struct {
// contains filtered or unexported fields
}
ParquetImporter handles importing Parquet files into Iceberg tables
func NewParquetImporter ¶
func NewParquetImporter(cfg *config.Config) (*ParquetImporter, error)
NewParquetImporter creates a new Parquet importer
func (*ParquetImporter) Close ¶
func (p *ParquetImporter) Close() error
Close closes the importer and releases resources
func (*ParquetImporter) GetTableLocation ¶
func (p *ParquetImporter) GetTableLocation(tableIdent table.Identifier) string
GetTableLocation returns the location where table data would be stored
func (*ParquetImporter) ImportTable ¶
func (p *ParquetImporter) ImportTable(ctx context.Context, req ImportRequest) (*ImportResult, error)
ImportTable imports a Parquet file into an Iceberg table
func (*ParquetImporter) InferSchema ¶
func (p *ParquetImporter) InferSchema(parquetFile string) (*Schema, *FileStats, error)
InferSchema reads a Parquet file and infers the schema
Click to show internal directories.
Click to hide internal directories.