Documentation
¶
Overview ¶
Package tabletype is the one model of a column type the platform declares a table with or writes a typed file from (#1833): the Trino types a registered table's columns carry, the parse of the type text Trino reports for a query's columns, and the inference of a column's type from JSON values.
It knows nothing about registrations, files or connectors. A JSON-lines registration and a script's Parquet export infer their types here, so the two cannot disagree about what a column of numbers is, and a Parquet file's columns and a trino_export's columns are rendered through the same Type.
Index ¶
Constants ¶
const DeclaredTimestampPrecision = 6
DeclaredTimestampPrecision is the precision every TIMESTAMP column is declared with; see Timestamp.
Variables ¶
var ErrTooDeep = fmt.Errorf("a value is nested more than %d levels deep", maxDepth)
ErrTooDeep is returned for a JSON value nested beyond what a table's type can be declared with.
Functions ¶
func CheckFieldName ¶
CheckFieldName refuses a ROW field name the Hive metastore cannot store.
A table's nested types are kept in the metastore as Hive type text ("struct<a:bigint>"), whose parser admits a field name made of letters, digits, '_', '.', '$' and ' ' and nothing else. Trino creates a table whose ROW declares any other name, and every later statement on it -- a SELECT, a DROP -- fails with "Could not read table schema" (observed on Trino 453), so a registration refused here is a table nobody is left unable to remove. A top-level column is not held to this: its name is stored on its own.
func CheckName ¶
CheckName refuses a name a Hive column or ROW field cannot carry, or one the JSON reader would leave empty on every row. It is the rule for a JSON-lines key at any depth and for a Parquet column or field.
func DecodeJSON ¶
DecodeJSON decodes one JSON value into the values the inference reads: nil, bool, json.Number, string, []any and *Object. A second value after the first is an error, and so is one nested deeper than maxDepth.
func IsTrailing ¶
IsTrailing reports whether DecodeJSON refused a second value after the first.
Types ¶
type Inferrer ¶
type Inferrer struct {
// contains filtered or unexported fields
}
Inferrer reads the type of every column across a sequence of records, every record rather than a sample: one value on the last line that does not fit what the first thousand said is a query that fails on the last line.
The rules, per key, ignoring null and an absent key:
- every value a JSON boolean: BOOLEAN;
- every value an integer within int64: BIGINT;
- any number with a fraction or an exponent, alone or among integers: DOUBLE;
- anything else a scalar can be -- strings, a mix of kinds, an integer too large for BIGINT, nothing but nulls: VARCHAR, which is the text of the value and loses nothing;
- objects: a ROW over the union of their keys, each field inferred the same way; lists: an ARRAY of what their elements infer to.
A key whose values cannot be one type is refused, naming the key, because no declaration reads both: an object, a list and a scalar are three shapes, and a key holding any two of them in different records is a conflict. Two scalars are not -- they meet at VARCHAR.
func (*Inferrer) Columns ¶
Columns returns the columns in first-seen order with the type each infers to.
type Kind ¶
type Kind string
Kind names a type. The declarable kinds are the ones a Hive table over a JSON-lines or Parquet file can be declared with; the source-only kinds are what a query can return that no registration declares, and a writer stores them as text.
const ( Boolean Kind = "BOOLEAN" Tinyint Kind = "TINYINT" Smallint Kind = "SMALLINT" Integer Kind = "INTEGER" Bigint Kind = "BIGINT" Real Kind = "REAL" Double Kind = "DOUBLE" Decimal Kind = "DECIMAL" Varchar Kind = "VARCHAR" Varbinary Kind = "VARBINARY" Date Kind = "DATE" // Timestamp is declared at microseconds, TIMESTAMP(6), whatever the // precision its source had: a Hive table's timestamps are read at the // catalog's hive.timestamp-precision, the connector refuses a column // declared at any other ("Incorrect timestamp precision for timestamp(3); // the configured precision is MICROSECONDS"), and MICROSECONDS is the // setting a catalog reads a Parquet file's timestamps exactly at. Timestamp Kind = "TIMESTAMP" Array Kind = "ARRAY" Map Kind = "MAP" Row Kind = "ROW" )
The declarable kinds.
const ( // TimestampTZ is a timestamp with a time zone. A Parquet file keeps the // instant and not the zone. TimestampTZ Kind = "TIMESTAMP WITH TIME ZONE" // Text is any other type a query returns as text -- CHAR, TIME, JSON, // UUID, IPADDRESS, an interval -- which a file stores as a string. Text Kind = "TEXT" )
The source-only kinds.
type Object ¶
Object is a JSON object with its keys in the order they were written. Decoding into a Go map loses that order, and the order is what a table's columns and a ROW's fields are declared in.
type Type ¶
type Type struct {
Kind Kind
Precision int
Scale int
Elem *Type
Key *Type
Fields []Field
Source string
}
Type is one column type. Precision and Scale describe a DECIMAL, and Precision a source TIMESTAMP's fractional digits; Elem is an ARRAY's element or a MAP's value, Key a MAP's key, and Fields a ROW's fields in order. Source is the type text a query reported, kept for a Text type so a writer can say what it stored as text.
func Parse ¶
Parse reads the type text Trino reports for a query column -- "bigint", "decimal(12,2)", "timestamp(6) with time zone", "array(row(a bigint, "b c" varchar))" -- into a Type. A type with no Parquet form of its own (TIME, JSON, UUID, an interval, anything this parse does not know) is Text, which a writer stores as a string, so a query never fails to export over one column.
The text is case-insensitive and a ROW field name may be quoted. Precision and scale a caller learned elsewhere -- a top-level DECIMAL reported as the bare "DECIMAL" with its numbers beside it -- are applied with WithPrecision.
func (Type) SQL ¶
SQL renders the type as a CREATE TABLE declares it. A ROW's field names are quoted, because they come from a file somebody wrote.
func (Type) WithPrecision ¶
WithPrecision applies a precision and scale reported beside a bare type name: a DECIMAL's two numbers, or a TIMESTAMP's fractional digits. A type that already carries its own is returned unchanged.
Source Files