Help us shape the future of Go! Take the 2026 Go Developer Survey

dtype

package
v0.6.8 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 26, 2026 License: MIT Imports: 10 Imported by: 0

Documentation

Overview

Package dtype is ursus's type system: DataType, Field and Schema.

It is a public package and the definition site for these types. The root package re-exports everything here by type alias, so ursus.Int64 and ursus.Schema are the spellings you should normally write. Import dtype directly only when implementing a custom scan source.

(It is public rather than internal for a mundane but real reason: an alias to an internal type renders on pkg.go.dev as a bare line with no methods documented, and DataType and Schema are the two most-read types in the library.)

DataType is comparable

DataType is a small comparable struct, so `a == b` and `map[DataType]T` both work — the kernel registry and the type-promotion table depend on it. Nested and parameterised types keep their payload behind an interned pointer, so pointer equality is semantic equality. See intern.go.

Its fields are unexported for exactly that reason: an exported Inner or Fields would let a caller construct a DataType that bypasses interning, at which point `==` silently starts returning false for equal types. Use the constructors.

Index

Constants

View Source 
const MaxDecimalPrecision = 38

MaxDecimalPrecision is the most digits a Decimal holds. 10^38 - 1 is the largest all-nines number below 2^127, so it is the widest precision 128 bits can promise.

Variables

View Source 
var (
	Null    = DataType{/* contains filtered or unexported fields */}
	Bool    = DataType{/* contains filtered or unexported fields */}
	Int8    = DataType{/* contains filtered or unexported fields */}
	Int16   = DataType{/* contains filtered or unexported fields */}
	Int32   = DataType{/* contains filtered or unexported fields */}
	Int64   = DataType{/* contains filtered or unexported fields */}
	Uint8   = DataType{/* contains filtered or unexported fields */}
	Uint16  = DataType{/* contains filtered or unexported fields */}
	Uint32  = DataType{/* contains filtered or unexported fields */}
	Uint64  = DataType{/* contains filtered or unexported fields */}
	Float32 = DataType{/* contains filtered or unexported fields */}
	Float64 = DataType{/* contains filtered or unexported fields */}
	Int128  = DataType{/* contains filtered or unexported fields */}
	String  = DataType{/* contains filtered or unexported fields */}
	Binary  = DataType{/* contains filtered or unexported fields */}
	Date    = DataType{/* contains filtered or unexported fields */}

	// Categorical is text with a type of its own, Polars' Categorical (step 351):
	// ordered as its text, as Polars' has been since 1.32, and stored as a String
	// is, so every kernel that reads a String's storage reads it. Its codes, the
	// order its values first appear in, are ToPhysical's; a String's functions are
	// refused it, as Polars refuses them, and Cat() has its own.
	Categorical = DataType{/* contains filtered or unexported fields */}
)

--- Simple types ------------------------------------------------------------

Declared as vars rather than consts because DataType is a struct. They are never mutated; treat them as constants.

View Source 
var EmptySchema = &Schema{index: map[string]int{}}

EmptySchema is the schema with no columns.

Functions

func CanCast 

func CanCast(from, to DataType) bool

CanCast reports whether an explicit Cast from `from` to `to` is defined.

This is broader than Promote: a cast may lose information (Int64 → Int8, Float64 → Int32), because the user asked for it. What it may not do is be meaningless — there is no cast from Struct to Int64.

Non-strict casts turn unrepresentable values into nulls rather than erroring; that is a kernel-level concern, not a type-level one.

func DecimalDigits added in v0.3.0 

func DecimalDigits(t DataType) (p, s int, ok bool)

DecimalDigits is t as a Decimal's precision and scale: a Decimal's own, and an integer's as Decimal(d, 0), d being the digits its width holds. Int128 needs 39, which no Decimal has, so its precision is reported as 39 and every rule that asks refuses it.

func ExactMismatch added in v0.3.0 

func ExactMismatch(a, b DataType) bool

ExactMismatch reports whether a and b promote, but only to a type that rounds one of them: an integer wider than 32 bits with a float.

func FormatDecimal 

func FormatDecimal(unscaled string, scale uint8) string

FormatDecimal places the decimal point in an unscaled integer's digits. 

FormatDecimal("1234", 2)  == "12.34"
FormatDecimal("5", 3)     == "0.005"
FormatDecimal("-5", 2)    == "-0.05"

It takes the digits as a string rather than an Int128 so that dtype does not depend on the i128 package for what is purely presentation. Callers hold the scale (from DataType.Scale) and the digits (from Int128.String).

Without this a Decimal renders as its storage — 12.34 printed as "1234" — which is not a rounding error but a factor of 10^scale, and looks entirely plausible.

func FormatTemporal 

func FormatTemporal(d DataType, ticks int64) string

FormatTemporal renders one stored tick count of type d.

The formats are ISO 8601, which is what every other tool in the ecosystem reads and what the CSV writer emits — so a frame printed to a terminal and a frame written to a file agree, and neither shows the raw integer that this function exists to stop appearing.

func IsLeapYear added in v0.5.0 

func IsLeapYear(y int) bool

IsLeapYear reports whether year y of the proleptic Gregorian calendar has 366 days, asking the standard library, which owns the calendar.

func LooksTemporal 

func LooksTemporal(d DataType, s string) bool

LooksTemporal reports whether s could be one of the layouts ParseTemporal accepts for d. Used by CSV schema inference, which must be conservative: a column of bare integers like 20240101 must NOT become dates.

func MeetHint added in v0.3.0 

func MeetHint(a, b DataType) string

MeetHint is the advice for two types that do not meet exactly — a join key, a Concat column, an Unpivot value — naming a cast that would make them meet. Every cast it names is one CanCast permits.

func MonthEnd added in v0.5.0 

func MonthEnd(t time.Time) time.Time

MonthEnd is t on the last day of its month, as MonthStart is on the first.

func MonthStart added in v0.5.0 

func MonthStart(t time.Time) time.Time

MonthStart is t on the first day of its month, at the same time of day on the wall clock of t's own zone, as Polars' dt.month_start keeps it. The wall clock is read back as AddTo reads it.

func ParseFloat added in v0.3.0 

func ParseFloat(s string, bits int) (float64, error)

ParseFloat parses s as a decimal floating-point number, rounded once to bits bits. It is the one float grammar: the CSV reader's inference and its parser, and Cast(String) to a float, all ask it, so text one of them reads as a number the others do too.

The grammar is the decimal one every dataframe and database reads: 

[+-] (digits [. [digits]] | . digits) [(e|E) [+-] digits]
[+-] (inf | infinity | nan)            case-insensitive

It is strconv.ParseFloat's, less what is Go's own: an underscore between digits, and a hexadecimal float. Those made "1_000" read as 1000 and "0x1p3" as 8, where Polars, pandas and DuckDB read both as text. strconv still does the rounding, but for a plain decimal exactFloat parses in one pass, to strconv's answer.

func ParseTemporal 

func ParseTemporal(d DataType, s string) (int64, bool)

ParseTemporal reads a textual instant into a tick count of type d.

It accepts the layouts FormatTemporal produces, plus the common relaxations: a space instead of the T separator, and a missing timezone (read as UTC). That asymmetry is deliberate — a writer should be strict and a reader forgiving, because the reader is the one handed files it did not create.

func ParseTemporalText added in v0.6.7 

func ParseTemporalText(d DataType, s string) (int64, bool)

ParseTemporalText reads s into a tick count of d, a Date, a Datetime or a Time, by Polars' CSV patterns, and else as ParseTemporal does. A Date reads a date, a Time a time of day, and a Datetime either a date, its midnight, or a date and a time; an offset makes it the instant, and none takes the wall clock as UTC, as ParseTemporal does. A fraction past the unit is cut, as Polars cuts one.

func QuoteName added in v0.6.0 

func QuoteName(s string) string

QuoteName renders a field name or an Enum category as it is, unless that would make a type's rendering ambiguous: empty, edged with a space, or holding a separator, a bracket, a quote or anything unprintable. Then it is quoted, as Go quotes a string. Every ordinary name stays bare.

func TicksPerDay added in v0.3.0 

func TicksPerDay(d DataType) (int64, bool)

TicksPerDay is 24 hours expressed in d's own unit.

It is the modulus that makes a Time a time of day, and it lives here so there is one definition of a day: the arithmetic kernel that wraps, the cast that normalises, and data's boundary check all ask this rather than each deriving it. Two copies of a modulus is two chances to disagree about whether midnight belongs to today or tomorrow.

Reports false for a type with no tick length. Date has one — 24h exactly — so this answers 1 for it, which is true and useless; callers gate on TypeTime.

func ValidDecimal added in v0.3.0 

func ValidDecimal(precision, scale uint8) bool

ValidDecimal reports whether a precision and scale describe a Decimal that can exist: 1 to MaxDecimalPrecision digits, of which at most all follow the point.

Types

type DataType 

type DataType struct {
	// contains filtered or unexported fields
}

DataType is a logical column type.

It is a comparable value type: `a == b` is exact type equality, and DataType works as a map key. Parameterised and nested types keep their payload in an interned *typeExt, so two independently-constructed List(Int64) values share one pointer and compare equal.

The zero value is Null.

func Array 

func Array(inner DataType, size int) DataType

Array is a fixed-length list of exactly size elements of inner.

func Datetime 

func Datetime(u TimeUnit, tz string) DataType

Datetime is an instant at the given resolution. An empty tz means naive (wall-clock, no zone); otherwise tz is an IANA name such as "UTC".

The distinction matters: converting a zone changes the wall clock and keeps the instant, while replacing a zone keeps the wall clock and changes the instant.

func Decimal 

func Decimal(precision, scale uint8) DataType

Decimal is a fixed-point decimal with the given precision (total digits) and scale (digits after the point), stored in 128 bits.

It does not refuse a precision and scale that no Decimal can have — the type is a value, and constructors here do not return errors — so ValidDecimal is what a consumer checks. Every consumer that builds a Decimal from outside does.

func DecimalMeet added in v0.3.0 

func DecimalMeet(a, b DataType) (DataType, bool)

DecimalMeet is the Decimal holding every value of a and b exactly, each a Decimal or an integer, or ok false when it would need more than 38 digits.

func Duration 

func Duration(u TimeUnit) DataType

Duration is an elapsed time at the given resolution.

func Enum 

func Enum(cats ...string) DataType

Enum is a string type whose category set is frozen and part of the type. Physically it is a Uint32 index into cats.

Enum is internable precisely because its categories never change. Categorical's grow as data arrives, so it carries none in its type, and its values are stored as text (step 351).

func List 

func List(inner DataType) DataType

List is a variable-length list of inner.

func Promote 

func Promote(a, b DataType) (DataType, bool)

Promote returns the common type two operands are cast to before a binary operation, or ok=false if there is no lossless-enough common type.

The rule is: never silently lose information that the user is likely to care about, and never silently produce a wrong answer. 

Null + T                 → T          (a null literal adopts the other side)
same type                → that type
intN + intM  (same sign) → the wider
intN + uintM             → the smallest SIGNED type holding both
signed + uint64          → Int128     (see below)
Int128 + any integer     → Int128
float + float            → the wider
int + float              → a float wide enough for the int's mantissa
anything else            → not ok

Uint64 combined with a signed type has no common type at 64 bits: the choices would be to wrap (silently wrong for large values) or to route through Float64 (silently lossy above 2^53), and Polars wraps. ursus used to reject the combination outright and make the user cast.

Int128 removes the dilemma — every uint64 and every int64 fits — so the combination now promotes losslessly. That was a free side effect of widening integer `sum` to 128 bits, and it is the better answer: an explicit cast that exists only to work around a missing type is a tax, not a safety feature.

Bool is not numeric here: `true + true` is rejected. Comparisons of two Bools are handled by the caller before reaching Promote.

func PromoteExact added in v0.3.0 

func PromoteExact(a, b DataType) (DataType, bool)

PromoteExact is Promote restricted to a type that holds every value of BOTH operands exactly. It is the question a join key, a Concat column and an Unpivot value ask, where a value that rounds is a wrong match or a wrong value — not the expected inexactness of arithmetic, which keeps Promote.

It differs from Promote in one place: an integer wider than 32 bits — Int64, Uint64, Int128 — with a float. Promote answers Float64, which holds an integer exactly only up to 2^53, so an Int64 key of 2^53+1 matched a Float64 key of 2^53 and a Concat turned it into 2^53. No type here holds both, so PromoteExact answers none, as Polars and PyArrow refuse the same join. ExactMismatch says why.

Temporal types are Promote's: Datetime(ms) and Datetime(ns) meet at Datetime(ns), and a value the finer unit cannot hold is a range question, refused by the strict cast at execution, not a type question.

func Struct 

func Struct(fields ...Field) DataType

Struct is a composite of named fields, in order.

func Supertype added in v0.6.7 

func Supertype(a, b DataType) (DataType, bool)

Supertype is the type a relaxed concat meets two columns at, Polars' supertype (step 307). It is Promote's, which may round, an Int64 beside a Float64 a Float64; and past Promote, text beside a number, a Boolean or an instant is text, a Boolean beside a number the number, and a Date beside a Datetime the Datetime. A List or a Struct meets only its own type.

func Time 

func Time(u TimeUnit) DataType

Time is a time of day at the given resolution.

func (DataType) BitWidth 

func (d DataType) BitWidth() int

BitWidth returns the storage width in bits, or 0 if the type is not fixed width.

func (DataType) Categories 

func (d DataType) Categories() []string

Categories returns an Enum's category set. The result aliases interned storage and must not be mutated.

func (DataType) CategoryIndex 

func (d DataType) CategoryIndex(s string) (uint32, bool)

CategoryIndex returns the physical index of an Enum category.

func (DataType) Equal 

func (d DataType) Equal(o DataType) bool

Equal reports exact type equality. It is identical to ==, and exists because `a.Equal(b)` reads better in conditionals and because a future non-comparable type would keep the call sites working.

func (DataType) Fields 

func (d DataType) Fields() []Field

Fields returns a Struct's fields. The result aliases interned storage and must not be mutated.

func (DataType) FromTime 

func (d DataType) FromTime(t time.Time) (int64, bool)

FromTime converts an instant into a tick count of type d, flooring toward negative infinity so the mapping stays monotone across the epoch.

It reports false for an instant d cannot hold, and callers must honour that. The fine-unit branch used to call t.UnixNano(), whose result Go documents as UNDEFINED before 1678 or after 2262, and then returned true regardless — so every caller that dutifully checked the flag was told a wrapped value was fine, and 2300-06-01 became 1715-11-11T00:25:26.290448384Z through Values, through Lit and through a cast.

Deriving the ticks from Unix SECONDS rather than from a nanosecond count also widens what actually fits: a millisecond instant in the year 3000 is an ordinary tick count and only the intermediate was ever the problem.

func (DataType) FromTimeBound added in v0.3.0 

func (d DataType) FromTimeBound(t time.Time, up bool) int64

FromTimeBound converts an instant into a tick count the way FromTime does, but SATURATES at d's extremes instead of reporting failure. up picks which extreme an unrepresentable instant becomes.

A value that does not fit is a lie; a bound that does not fit is the type's own extreme. That is the whole distinction, and it is why this is a second function rather than a flag on the first: a caller storing a DATUM must be told it cannot (FromTime, and ToDuration's doc argues the same for the other direction), while a caller computing the EDGE of a window wants the widest edge the type can express. Clamping is not an approximation there — every value it will be compared against is itself a representable tick, so a clamped bound admits exactly the set an unbounded one would.

The as-of join's tolerance is the caller: a bound one month past 2262 is beyond every nanosecond instant that exists, so the window is open on that side.

What it guarantees is exactly what FromTime reports, and FromTime is exact for any VALID time.Time. It is not a defence against an invalid one: Go's calendar arithmetic wraps silently past year 292277026596, and such an instant still prints a plausible date while its Unix() has gone negative — so a bound computed from it saturates to nothing at all. See step 63's as-built; the as-of join reaches this only with a Datetime(s) key within a month of that year.

func (DataType) HasStringStorage 

func (d DataType) HasStringStorage() bool

HasStringStorage reports whether values are stored as offsets + character data, i.e. whether Column.Strings() is the right accessor. This is the storage question, and it deliberately excludes Enum.

func (DataType) ID 

func (d DataType) ID() TypeID

ID returns the type's shape tag.

func (DataType) Inner 

func (d DataType) Inner() DataType

Inner returns the element type of a List or Array, and Null otherwise.

func (DataType) IsBool 

func (d DataType) IsBool() bool

func (DataType) IsFixedWidth 

func (d DataType) IsFixedWidth() bool

IsFixedWidth reports whether values occupy a constant number of bits, i.e. whether the column has a flat values buffer and no offsets.

func (DataType) IsFloat 

func (d DataType) IsFloat() bool

func (DataType) IsHashable 

func (d DataType) IsHashable() bool

IsHashable reports whether values can be group-by keys or deduplicated.

IsOrdered's types, and since step 344 the nested ones of them, field by field and element by element, which have no order: the two questions were kept apart for this, "a Struct could be hashable field-by-field long before anyone defines a total order over it".

func (DataType) IsInteger 

func (d DataType) IsInteger() bool

func (DataType) IsListLike added in v0.6.7 

func (d DataType) IsListLike() bool

IsListLike reports whether values are stored as a List's are, offsets over a child column: a List, or an Array, each of whose valid rows has its Size of elements and each null row none (step 341). This is the storage question; a List's kernels read an Array so, and its type says where it is fixed.

func (DataType) IsNested 

func (d DataType) IsNested() bool

func (DataType) IsNull 

func (d DataType) IsNull() bool

func (DataType) IsNumeric 

func (d DataType) IsNumeric() bool

IsNumeric reports whether arithmetic is defined on the type.

Bool is deliberately excluded: `true + true` is a bug in every query it appears in, and rejecting it at type-resolution time is more useful than promoting to integer the way C does. Decimal is included; its kernels arrive later.

func (DataType) IsOrdered 

func (d DataType) IsOrdered() bool

IsOrdered reports whether ursus can compute a total order over this type.

It is the single source of truth for Sort, Min/Max, ordering comparisons and GroupBy, all of which previously carried their own copies of this list — five of them, already disagreeing about Binary, Enum and Decimal. Divergence there is worse than it sounds: the plan layer's copies decide what is REJECTED AT PLAN TIME, and the kernel layer's copies decide what actually works, so a mismatch either rejects a working feature or lets a query through to fail at runtime.

Enum is ordered by its CATEGORY INDEX, not lexically. For a frozen category set the index order is the order the user declared — Enum("low","med","high") sorts low < med < high, which is almost always the intent and is not expressible lexically. Categorical, whose mapping grows at runtime, would make the same choice non-deterministic, which is one reason it stays unconstructible.

func (DataType) IsSignedInteger 

func (d DataType) IsSignedInteger() bool

func (DataType) IsString 

func (d DataType) IsString() bool

IsString reports whether the type's logical values are strings, including Enum.

Careful: an Enum is logically a string but is stored as a Uint32 index, so a caller that uses IsString to decide how to READ a column will reach for the string accessor and find nothing there. Use it for semantic questions ("can this be compared to a string literal?") and Physical() for storage questions.

func (DataType) IsTemporal 

func (d DataType) IsTemporal() bool

func (DataType) IsUnsignedInteger 

func (d DataType) IsUnsignedInteger() bool

func (DataType) NanosPerTick 

func (d DataType) NanosPerTick() (int64, bool)

NanosPerTick returns the length of one stored tick in nanoseconds, and whether d is a temporal type with a defined tick length.

This is the conversion factor every temporal cast, parse and format needs, in one place. Date is included at whole days: it stores days since the epoch, so its tick is 86400 seconds even though it carries no TimeUnit of its own.

The widest value is Date's 8.64e13, and multiplying a tick count by it is how a caller loses. That product leaves int64 at 106_751 ticks — 292 years for a Date, and the same 292 years for every Datetime unit, because the product is a nanosecond count either way. The as-of join scaled its keys up like that and matched two dates three centuries apart against a one-hour tolerance; the fix was to scale the TOLERANCE down into ticks instead, where nothing can overflow. This comment used to claim the product was safe within ~10^5 years, which is the false premise that made it look fine.

func (DataType) Physical 

func (d DataType) Physical() DataType

Physical returns the storage type used to represent d.

Kernels dispatch on the physical type: Date and Int32 share integer kernels, Datetime and Duration share Int64 kernels, Enum indexes are Uint32. Only operations that need calendar or category semantics look at the logical type.

func (DataType) Precision 

func (d DataType) Precision() uint8

Precision and Scale return a Decimal's parameters.

func (DataType) Scale 

func (d DataType) Scale() uint8

func (DataType) Size 

func (d DataType) Size() int

Size returns an Array's fixed length, and 0 otherwise.

func (DataType) String 

func (d DataType) String() string

String renders the type with its parameters.

This output is load-bearing: it appears in golden Explain files, in ursustest failure messages, and in every type error. It must stay stable.

The rendering is unambiguous, so two types render alike only if they are one type: an Enum's category or a Struct's field name is quoted where it holds a character the rendering itself uses (QuoteName). Deduplication keys aggregates, window functions and literals on renderings, and Enum("a, b") and Enum("a", "b") both rendered Enum(a, b): two casts of one column, one to each, became one aggregate, or one window function's answer, until step 179 (audit row O13).

func (DataType) TimeUnit 

func (d DataType) TimeUnit() TimeUnit

TimeUnit returns the resolution of a Time, Datetime or Duration; Second otherwise.

func (DataType) TimeZone 

func (d DataType) TimeZone() string

TimeZone returns a Datetime's IANA zone name, or "" if naive or not a Datetime.

func (DataType) ToDuration 

func (d DataType) ToDuration(ticks int64) (time.Duration, bool)

ToDuration converts a stored tick count of a Duration type into a time.Duration. Reports false for any other type, and for a span time.Duration cannot hold.

The unrepresentable case is not hypothetical

`ticks * npt` was an unchecked int64 multiply five lines below ToTime, which documents and implements exactly this defence. A time.Duration IS int64 nanoseconds, so it spans about 292 years — and a Duration(Second) column reaches that at 9.2e9 ticks, which `Datetime - Datetime` produces from two instants three centuries apart. The product wrapped and the span rendered NEGATIVE.

Reporting false rather than saturating, because FormatTemporal's fallback is the raw tick count: an integer a reader can see is unusual is a better answer than a duration that looks ordinary and has the wrong sign.

func (DataType) ToTime 

func (d DataType) ToTime(ticks int64) (time.Time, bool)

ToTime converts a stored tick count of type d into a time.Time in UTC.

Date is days since the epoch; Datetime and Time are tick counts at d's unit. Duration is NOT an instant and is rejected — use ToDuration.

type Field 

type Field struct {
	Name     string
	Type     DataType
	Nullable bool
}

Field is a named, typed column slot.

Nullable is a static property of the schema, not a count of nulls actually present: a non-nullable field is one where the engine may skip validity handling entirely. Operations propagate it conservatively — an arithmetic result is nullable if either operand is, a non-strict cast is always nullable, IsNull is never nullable.

Field is comparable, because DataType is.

func NotNull 

func NotNull(name string, t DataType) Field

NotNull builds a non-nullable field.

func Of 

func Of(name string, t DataType) Field

Of builds a nullable field. Nullable is the safe default: claiming a column has no nulls when it does causes silent wrong answers, while the reverse only costs a little speed.

func (Field) AsNullable 

func (f Field) AsNullable() Field

AsNullable returns a copy of f that is nullable.

func (Field) Equal 

func (f Field) Equal(o Field) bool

Equal reports exact field equality, including nullability.

func (Field) Rename 

func (f Field) Rename(name string) Field

Rename returns a copy of f with a new name. Used on every naming path — Alias, Name().Prefix, join suffixing.

func (Field) String 

func (f Field) String() string

String renders "name: Type", or "name: Type!" when the field is non-nullable. Stable: golden files depend on it.

func (Field) WithType 

func (f Field) WithType(t DataType) Field

WithType returns a copy of f with a new type, preserving name and nullability.

type Interval 

type Interval struct {
	// contains filtered or unexported fields
}

Interval is a calendar-aware span of time.

Why time.Duration is not enough

A Duration is a fixed number of nanoseconds. "1 month" is not: it is 28, 29, 30 or 31 days depending on where you start. Neither is "1 day", in any zone with daylight saving — it is 23, 24 or 25 hours depending on the date. A type that stores only nanoseconds cannot express either, so it cannot express the windows a temporal group-by is asked for.

Three fields, and they are not interchangeable

months, days and nanos are stored separately and applied in that order, because each obeys a different rule: 

months  calendar arithmetic with END-OF-MONTH CLAMPING
days    calendar days — 23, 24 or 25 hours across a DST boundary
nanos   absolute elapsed time

So Every("1d") is NOT FromDuration(24*time.Hour), and the difference is visible on exactly two days a year in any DST zone. Storing 1 day as 86,400,000,000,000 nanos would collapse the distinction at construction time, where it can never be recovered. This is Arrow's month-day-nano interval layout, chosen for the same reason.

Errors are carried, not returned

Every("banana") has to do something, and returning (Interval, error) would break the one call shape the API is for — an option struct field. So a parse failure is carried in the value and surfaced when the query is built, which is the deferred error rule the rest of the library already follows.

func Every 

func Every(s string) Interval

Every parses an interval string: "1y", "3mo", "1w", "2d", "6h", "15m", "30s", "1ns". Components may be concatenated — "1h30m" — and a leading "-" negates the whole thing.

Units: ns, us, ms, s, m, h, d, w, mo, q, y. Note "m" is MINUTES and "mo" is months, which is the one ambiguity in the notation and the reason the unit table is ordered longest-first. "i" counts an integer index, as Polars' "3i" does, and mixes with no other unit (step 312).

func FromDuration 

func FromDuration(d time.Duration) Interval

FromDuration builds an absolute interval. The result is never calendar-aware — 24*time.Hour is twenty-four hours, not one day.

The single most negative Duration is refused, and it is the only one. MinInt64 has no negation, so an interval holding it would be its own Neg() — see the invariant on Neg. Refusing the one value keeps that invariant true of every interval that exists, which is worth more than a Duration nobody means: it is about -292 years, and every consumer refuses it as negative anyway.

func GivenZero added in v0.6.7 

func GivenZero() Interval

GivenZero is an interval of nothing that an option reads as given rather than unset: see Given.

func IntervalOf 

func IntervalOf(months, days int32, nanos int64) Interval

IntervalOf rebuilds an interval from its three components.

It exists for the kernel. An expression argument travels as a *expr.Lit and litreflect only builds those from primitive kinds, so an Interval crosses that boundary as three plain integers and is reassembled here — rather than teaching the literal path about a struct type, which would be a second construction route for a value whose whole point is that its three fields are not interchangeable.

func (Interval) AddTo 

func (i Interval) AddTo(t time.Time) time.Time

AddTo advances t by the interval, in t's own location.

The three components are applied in order — months, then days, then nanos — and the order is part of the contract, not an implementation detail. Adding one month and one day to January 31st gives February 29th then March 1st in a leap year; the other order gives February 1st then March 1st. Both are defensible; only one can be the answer, and this is the one Arrow, Polars and PostgreSQL all give.

Months and days move the wall clock; nanos move the instant

The calendar parts keep the WALL CLOCK, so crossing a daylight-saving boundary moves the instant by 23 or 25 hours and midnight stays midnight — which is what a calendar day means. The date is advanced in UTC, where every day is a day, and read back in t's zone once, by wallTime: where that wall clock is read twice, at t's own offset if it holds there, and where it is never read, at the instant the gap ends.

It used AddDate, whose time.Date picks a side of a gap as it likes, and picked the one before: in Santiago, 2024-09-08 began at 01:00, and one day after 2024-09-07 00:00 was 2024-09-07 23:00, the same date (step 145).

func (Interval) AsWeekly added in v0.6.0 

func (i Interval) AsWeekly() Interval

AsWeekly is i, written in weeks: the kernel's way back to a weekly interval, which crosses into it as three numbers. It is i unchanged unless i is whole weeks alone.

func (Interval) Days 

func (i Interval) Days() int32

func (Interval) Err 

func (i Interval) Err() error

Err reports a parse failure from Every. Nil for every Interval built any other way.

func (Interval) Given added in v0.6.7 

func (i Interval) Given() bool

Given reports whether the interval was set: any interval that spans something, and GivenZero's. A rolling group's offset of none is GivenZero, where its zero value is Polars' default, minus the period.

func (Interval) Index added in v0.6.7 

func (i Interval) Index() bool

Index reports whether the interval counts an integer index, Polars' "3i": its Nanos are the count.

func (Interval) IsCalendar 

func (i Interval) IsCalendar() bool

IsCalendar reports whether the interval's length depends on WHERE IT STARTS.

True for months and for days, and the days half is the part that is easy to get wrong: a day is a calendar day, so in a DST zone it is not a fixed span either. Anything built from a Duration is never calendar.

func (Interval) IsZero 

func (i Interval) IsZero() bool

IsZero reports whether the interval spans nothing.

func (Interval) Months 

func (i Interval) Months() int32

Months, Days and Nanos are the three components. Exported as accessors rather than fields so the zero value stays meaningful and nobody constructs a half-valid Interval that skipped Every's validation.

func (Interval) Nanos 

func (i Interval) Nanos() int64

func (Interval) Neg 

func (i Interval) Neg() Interval

Neg reverses the interval. Used to walk a window grid backwards and to turn a window's width into the offset from its end to its start.

Total, with no check, and that rests on an invariant rather than on luck: no interval holds a component at its type's minimum. Every bounds its accumulators by each type's MAXIMUM, so a negated total lands in [-max, 0]; IntervalOf and FromDuration refuse the minimum outright. A guard here could not fire, and this codebase treats an unreachable guard as protection that is not there.

func (Interval) Negative 

func (i Interval) Negative() bool

Negative reports whether the interval runs backwards.

func (Interval) RoundTo added in v0.5.0 

func (i Interval) RoundTo(t time.Time) time.Time

RoundTo rounds t to the nearer end of the window that holds it, on TruncateTo's grid, and a t exactly halfway to the later end, as Polars' dt.round does. The interval must be positive.

Nearer is in elapsed time

A sub-day grid rounds t plus half the interval down, so the later end is the next boundary t's clock reaches: across a fall-back fold, where the wall clock goes back, 01:30 EDT rounds by the hour to 01:00 EST, half an hour later, as Polars rounds it. A boundary taken from the wall clock would have been 02:00 EST, two hours after the window's start.

A day or a month has no fixed half, so its window's two ends are found on the calendar and t compared with each: on a 23-hour day the middle is 11:30 after the start, and 2024-02-15T12:00 is past February's middle and rounds to March.

func (Interval) String 

func (i Interval) String() string

String renders the interval in Every's own notation, so String and Every round-trip over every interval that can be built.

It hoists ONE leading '-' and then negates all three components, which is correct because no interval mixes signs — IntervalOf refuses that pair, and Every cannot produce it since the parser gives the sign to the whole interval rather than to a component. And the negation cannot be a fixed point, by the invariant on Neg. Both halves used to be false: IntervalOf(1, -1, 0) rendered "--1mo1d", and a count that wrapped to MinInt32 months rendered "--178956970y-8mo".

func (Interval) TruncateTo 

func (i Interval) TruncateTo(t time.Time) time.Time

TruncateTo floors t to the start of the window containing it, on a grid of this interval anchored at the Unix epoch, in t's own location.

Local, not UTC — sub-day included

The grid is built from wall-clock components, so truncating by "1d" in America/New_York gives local midnight rather than 00:00 UTC, and by "1h" in Asia/Kolkata, at +05:30, the local hour: 10:47 floors to 10:00, where the UTC grid said 10:30. Dt().Truncate given a time.Duration floors the absolute instant instead, because a Duration IS absolute, and never reaches here.

The floored wall clock is read back by wallTime, at every size: at t's own offset when that offset still holds there, so in a fall-back fold the second 01:30 floors to the second 01:00; otherwise at the offset in force; and when no offset makes it a real wall clock — it fell into a spring-forward gap — at the instant the gap ends. Days and months were read back by time.Date until step 145, and a day that began at 01:00 began, by it, at 23:00 the day before.

Mixed intervals are refused by the caller

An interval with two non-zero components has no well-defined grid — "every 1 month and 3 days" does not tile the line — so a window grid must be built from exactly one component. GroupByDynamic rejects the rest; this function uses the coarsest component present so it is total rather than panicking on input it cannot get.

func (Interval) Weekly added in v0.6.0 

func (i Interval) Weekly() bool

Weekly reports whether the interval was written in weeks alone, so that a grid of it starts on a Monday.

type Schema 

type Schema struct {
	// contains filtered or unexported fields
}

Schema is an ordered, name-unique sequence of Fields.

A *Schema is IMMUTABLE once constructed and is shared freely between plan nodes, batches and goroutines. Nothing in ursus mutates one. Accessors that hand back a slice therefore return a fresh copy — see Names and FieldSlice.

Column order is meaningful (it is the output column order) but there is no row index: ursus follows Polars in treating row order as a property of the data rather than an addressable label space.

func MustSchema 

func MustSchema(fields ...Field) *Schema

MustSchema is NewSchema for tests and package-level vars, panicking on duplicates.

func NewSchema 

func NewSchema(fields ...Field) (*Schema, error)

NewSchema builds a schema, rejecting duplicate column names.

Duplicates are an error rather than a silent last-wins or auto-suffix because every downstream lookup is by name: tolerating them would make Col("x") ambiguous and the ambiguity would surface far from its cause.

func (*Schema) All 

func (s *Schema) All() iter.Seq2[int, Field]

All iterates the schema in column order.

func (*Schema) ByName 

func (s *Schema) ByName(name string) (Field, bool)

ByName returns the named field.

func (*Schema) Drop 

func (s *Schema) Drop(names ...string) *Schema

Drop returns a schema without the named columns. Unknown names are ignored, because Drop is used to express "remove these if present" during rewrites.

func (*Schema) Equal 

func (s *Schema) Equal(o *Schema) bool

Equal reports whether both schemas have the same fields in the same order.

func (*Schema) Field 

func (s *Schema) Field(i int) Field

Field returns the i'th field. It panics if i is out of range, matching slice indexing: an out-of-range column ordinal is always an ursus bug, never user error.

func (*Schema) FieldSlice 

func (s *Schema) FieldSlice() []Field

FieldSlice returns the fields in order, as a FRESH slice the caller may mutate.

func (*Schema) Has 

func (s *Schema) Has(name string) bool

Has reports whether the schema contains name.

func (*Schema) IndexOf 

func (s *Schema) IndexOf(name string) int

IndexOf returns the position of name, or -1.

func (*Schema) Len 

func (s *Schema) Len() int

Len returns the number of columns.

func (*Schema) Names 

func (s *Schema) Names() []string

Names returns the column names in order, as a FRESH slice the caller may mutate. Projection pushdown sorts and filters the result in place.

func (*Schema) Select 

func (s *Schema) Select(names []string) (*Schema, error)

Select returns a schema containing only the named columns, in the order given. Unknown names produce an UnknownColumn error listing what is available.

func (*Schema) String 

func (s *Schema) String() string

String renders "{a: Int64, b: String}". Stable: golden Explain files and every schema mismatch message depend on it.

type TemporalText added in v0.6.7 

type TemporalText struct {
	Year, Month, Day, Hour, Minute, Second, Nanos, Offset int
	HasDate, HasTime, HasZone                             bool
}

TemporalText is a date, a time of day or both, and an offset, as Polars' patterns read them: Year, Month and Day where HasDate; Hour, Minute, Second and Nanos where HasTime; and Offset, seconds east of UTC, where HasZone.

func ScanTemporalText added in v0.6.7 

func ScanTemporalText(s string) (TemporalText, bool)

ScanTemporalText reads s by Polars' patterns, as str.to_date, str.to_datetime and str.to_time read a value without a format (step 356), or ok false.

type TemporalTextKind added in v0.6.7 

type TemporalTextKind uint8

TemporalTextKind is what a text reads as: a date, an instant of no zone or of one, or a time of day. 

const (
	NoTemporalText TemporalTextKind = iota
	DateText
	DatetimeText
	ZonedDatetimeText
	TimeText
)

func ClassifyTemporalText added in v0.6.7 

func ClassifyTemporalText(s string) TemporalTextKind

ClassifyTemporalText is what s reads as by Polars' CSV patterns, or NoTemporalText.

type TimeUnit 

type TimeUnit uint8

TimeUnit is the resolution of a Time, Datetime or Duration. 

const (
	Second TimeUnit = iota
	Milli
	Micro
	Nano
)

func (TimeUnit) Duration 

func (u TimeUnit) Duration() time.Duration

Duration returns the length of one tick.

func (TimeUnit) Finer 

func (u TimeUnit) Finer(v TimeUnit) bool

Finer reports whether u has strictly higher resolution than v. Used by type promotion: combining two temporal columns keeps the finer unit, so no precision is silently discarded.

func (TimeUnit) String 

func (u TimeUnit) String() string

type TypeID 

type TypeID uint8

TypeID identifies a type's shape. It is the switch tag for every dtype-erased dispatch in the engine. 

const (
	TypeNull TypeID = iota
	TypeBool

	TypeInt8
	TypeInt16
	TypeInt32
	TypeInt64

	TypeUint8
	TypeUint16
	TypeUint32
	TypeUint64

	TypeFloat32
	TypeFloat64

	TypeDecimal

	TypeString
	TypeBinary

	TypeDate     // days since the Unix epoch, int32
	TypeTime     // time of day since midnight, int64 in the type's unit
	TypeDatetime // instant since the Unix epoch, int64 in the type's unit
	TypeDuration // elapsed time, int64 in the type's unit

	TypeList   // variable length, one child type
	TypeArray  // fixed length, one child type
	TypeStruct // named fields

	TypeEnum // frozen category set, part of the type

	// TypeInt128 is the accumulator and output type of integer Sum. Constructible.
	//
	// Categorical and Uint128 remain reserved: Categorical cannot be interned the
	// way Enum can, because its category mapping grows at runtime, and Uint128 has
	// no use while Int128 covers every unsigned value with room to spare.
	TypeCategorical
	TypeInt128
	TypeUint128

	// TypeIDCount is one past the last TypeID. Tables indexed by TypeID are
	// declared [TypeIDCount]T.
	TypeIDCount
)

func (TypeID) String 

func (id TypeID) String() string

String returns the type's base name, without parameters.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.