Documentation
¶
Overview ¶
Package parquetcheck defines the parquet export schema invariant shared by every Forma parquet consumer: the three system columns each generation carries regardless of attribute evolution, with the exact DuckDB types both exporters emit (delta flush and init/compaction base). The federated read path and the compaction merge both enforce it before trusting a file (#189): their scans run union_by_name, which would otherwise NULL-pad a malformed file's system columns and silently misfold its rows instead of failing loudly (#187).
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Check ¶
Check verifies one probed parquet schema (column name → DuckDB type) against the invariant. The returned error names the object, the offending column, and the expected state; callers wrap it with their own sentinel or operational context.
func DescribeColumns ¶
DescribeColumns reads one parquet object's footer schema through the given DuckDB session and returns column name → DuckDB type. It is the shared byte-truth probe behind manifest schema stamping (#256) and compaction's pre-merge validation; the federated read path issues its own query over its executor abstraction and shares the row scan via ScanDescribeColumns. The URI is escaped as a SQL literal, so callers may pass untrusted paths.
func ScanDescribeColumns ¶
func ScanDescribeColumns(rows DescribeRowScanner, uri string) (map[string]string, error)
ScanDescribeColumns folds one DESCRIBE result into column name → DuckDB type. It is the single row-scan surface behind both footer probes: this package's *sql.DB probe (manifest schema stamping #256, compaction's pre-merge validation) and the federated read path's probe over its executor abstraction. The two used to be verbatim twins; the DESCRIBE row shape (6 columns: name, type, null, key, default, extra) is now pinned in one place.
uri is carried purely for error context — every returned error names the object being described, per the read-path consistency contract.
Types ¶
type DescribeRowScanner ¶
DescribeRowScanner is the row surface of a DESCRIBE result — satisfied by *sql.Rows and by the federated engine's duckDBRowsIterator. It deliberately omits Close: lifetime stays with the caller that issued the query, so this package never closes a handle it did not open.
type SystemColumn ¶
SystemColumn is one invariant column with its required DuckDB type.
func SystemColumns ¶
func SystemColumns() []SystemColumn
SystemColumns lists the invariant, ordered so violation errors are deterministic — row_id first, since everything folds on it.
Source Files