Documentation
¶
Overview ¶
Package files reads several files of one format as one scan, in order, each file by a source of its own, with columns every row of a file shares after the files' own: a hive dataset's partition keys, and the file's path (step 360).
Index ¶
- type File
- type Shared
- type Source
- func (s *Source) Caps() plan.Caps
- func (s *Source) ClassifyPredicates(preds []expr.Node) []plan.Pushdown
- func (s *Source) Describe() string
- func (s *Source) Name() string
- func (s *Source) Open(ctx context.Context, spec source.ScanSpec) (source.BatchSource, error)
- func (s *Source) Schema(ctx context.Context) (*dtype.Schema, error)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type File ¶
File is one file of the scan: its source, and its path, which names it in an error and is its value of the path column.
type Shared ¶
type Shared struct {
}
Shared are the columns every row of a file shares: a hive dataset's partition keys, of the types Types names or their values infer, Polars' hive_schema and try_parse_hive_dates, and the column of each file's path, Polars' include_file_paths.
type Source ¶
type Source struct {
// contains filtered or unexported fields
}
Source is files read as one, of the first file's columns, every other file's read by name and refused where its columns differ, as Polars reads several IPC files; then the shared columns.
func New ¶
New is a source of files, op naming it in errors, name and desc in Explain, with the shared columns with asks for.
func (*Source) ClassifyPredicates ¶
ClassifyPredicates reports a conjunct of the shared columns alone as Inexact: it rules out whole files, judged on their values before any is opened (keep), and the Filter above stays to judge the rows, as Parquet's partition keys are pruned.
Source Files