Documentation
¶
Overview ¶
Package tableparquet reads and writes the Parquet files a registered table is declared over (#1833). It reads a file's columns from its footer alone -- the last bytes of the file -- and refuses one a table over it could not read back exactly; it writes query results and script rows as Parquet a table can be registered over. It knows nothing about registrations: tableregister asks it for columns, and the export paths hand it rows.
The type mapping in both directions is written against the Hive connector the scratch catalogs use, with hive.timestamp-precision=MICROSECONDS. A Parquet type outside it is refused by name rather than declared as something that reads back differently.
Index ¶
Constants ¶
const ContentType = "application/vnd.apache.parquet"
ContentType is the media type of a Parquet file.
const Extension = ".parquet"
Extension is the file extension of a Parquet file.
const Magic = "PAR1"
Magic is the four bytes a Parquet file begins and ends with.
Variables ¶
var ErrNotParquet = errors.New("the file is not a Parquet file")
ErrNotParquet means the bytes are not a Parquet file: they do not begin and end with the magic, or the footer between does not parse.
Functions ¶
func CheckNames ¶
CheckNames refuses column and field names a table over the written file could not declare, before a byte of it is written.
Parquet itself keeps any name, and Columns does not: a Hive column name may not be empty, edged with whitespace, hold a comma or leave ASCII, a ROW field name is narrower still, and two names one apart by case are one name to the reader. Without this an export writes a file and stores it, and the registration the export was made for then refuses it (#1833).
func Columns ¶
Columns reads a Parquet file's footer through r, which holds size bytes, and returns the columns a table over it declares, in file order, each lowercased and typed by the mapping. Only the footer is read: the first four bytes, the last eight, and the footer they point at, so a file of any size costs a few kilobytes.
func Write ¶
Write renders rows as a Parquet file with one column per entry of columns, in that order, compressed with ZSTD. Every column, element, map value and row field is optional, so a null anywhere is kept a null.
A row's values are positional, matching columns. The values accepted for a type are the ones the two writers hand in: a query's (the Trino driver's own values, and its nested values as the JSON protocol carries them) and a script's (the values DecodeJSON produces). See convert.go.
Every file written here registers with manage_table: the types it writes are the ones Columns reads back, which is why a type Parquet could store more precisely -- a UUID, a TIME -- is written as a string, and the names are held to the rules Columns holds them to before anything is written.
Types ¶
This section is empty.
Source Files