Documentation
¶
Overview ¶
Package chartspec is the pure visual-algebra compiler of the analytics engine.
A types.ChartSpec places cube fields on shelves as table-algebra expressions (concat +, cross ×, nest /). Parse decodes and validates the JSON spec against a cube's listquery allow-list (field roles and permitted aggregates); Normalize turns a shelf into its tuple table; Compile derives the one AggregateQuery the store runs, the pane table and a mark per pane; Drill swaps a dimension for its child in the cube's hierarchy; and the reducer (Partial, Merge, Finalize) combines additive partials and mergeable DDSketch quantile sketches.
Encode splits a spec into content-addressed Nodes, each addressed by the SHA-256 of its RFC 8785 canonical JSON, so equal subtrees share one node and a leaf change re-addresses only the nodes on its path to the root. Decode rebuilds a spec from a root hash, verifying every node against its address, and Diff walks two roots and reports the changed paths, descending only where hashes differ. The package does no I/O: callers supply stored nodes through a NodeGetter.
Index ¶
- Constants
- func Decode(root Hash, get NodeGetter) (types.ChartSpec, error)
- func Drill(spec types.ChartSpec, cube listquery.Schema, field string) (types.ChartSpec, error)
- func EncodePartial(p Partial) []byte
- func Finalize(p Partial, agg types.Aggregate) float64
- func Next(start time.Time, g types.Grain) time.Time
- func Parse(b []byte, m *listquery.Map) (types.ChartSpec, error)
- func Truncate(t time.Time, g types.Grain) time.Time
- func ValidGrain(g types.Grain) bool
- func Validate(spec types.ChartSpec, m *listquery.Map) (types.ChartSpec, error)
- type Change
- type Hash
- type Node
- type NodeGetter
- type PaneKey
- type Partial
- type Plan
- type Tree
- type Tuple
Constants ¶
const MaxShelfDepth = 8
MaxShelfDepth bounds a shelf expression's nesting (a leaf counts as one level), so a crafted deep spec is a clean CodeInvalidInput, not unbounded recursion.
const SketchRelativeAccuracy = 0.01
SketchRelativeAccuracy is the relative accuracy of every quantile sketch: a p50/p95/p99 is within 1% of the exact value at its rank.
Variables ¶
This section is empty.
Functions ¶
func Decode ¶ added in v0.0.2
func Decode(root Hash, get NodeGetter) (types.ChartSpec, error)
Decode rebuilds the ChartSpec rooted at root. Every fetched node is re-hashed: a missing node is CodeNotFound, a node whose content does not hash to its address is CodeInternal, and an unknown kind, a kind out of place, a malformed field or nesting past MaxShelfDepth or types.MaxFilterDepth is CodeInvalidInput. The result carries FilterTree but no compiled Filter; the time range comes back in UTC.
func Drill ¶
Drill returns a copy of spec with the dimension field replaced, wherever it is used as a dimension (shelves, detail, encodings, sort keys), by its child in the cube's hierarchy. The input spec is not modified. A cube without Fields, a field in no hierarchy, at a hierarchy's last level, or whose child the cube does not declare is CodeInvalidInput.
func EncodePartial ¶
EncodePartial serializes p (version byte, sum, count, min, max as big-endian float64, then the DDSketch with its index mapping) for storage in a row.
func Finalize ¶
Finalize computes agg from p: sum and count (0 when empty), avg, min, max, and p50/p95/p99 from the sketch. An empty summary yields NaN for the value-dependent aggregates; count_distinct, which additive partials cannot answer, yields NaN.
func Next ¶
Next returns the start of the grain period after the one starting at start; an unknown grain returns start unchanged (callers validate with ValidGrain).
func Parse ¶
Parse decodes a JSON ChartSpec and validates it against the cube's allow-list m: every referenced field must be allow-listed; a measure must carry one of its permitted aggregates and a dimension or time field none; detail fields must be dimensions; shelves nest at most MaxShelfDepth levels; the grain, mark, time range must be valid; a sort key must name a dimension or aggregated measure the spec places (shelves, detail, encodings); a nil m is rejected; and the filter tree is compiled with listquery.ParseFilter. Every rejection is CodeInvalidInput.
func Truncate ¶
Truncate returns t in UTC truncated to the start of its grain (minute, hour, UTC day or UTC month); an unknown grain returns t in UTC unchanged.
func ValidGrain ¶
ValidGrain reports whether g is one of the grains Truncate and Next know.
func Validate ¶ added in v0.0.6
Validate checks an already-decoded spec exactly as Parse checks a JSON one, against the cube's allow-list m, and returns it with its filter tree compiled. A caller that holds a typed spec (mapped from a proto, say) validates it here rather than encoding it to JSON for Parse. Every rejection is CodeInvalidInput.
Types ¶
type Change ¶ added in v0.0.2
type Change struct {
// Path locates the change from the root, dotted, with list positions in brackets:
// "columns.args[0].agg" or "filter.members[1]". A root field has no prefix.
Path string
// Before is the node hash on the old side: for a changed field, the node that owns
// it; for an added, removed or re-kinded child, the child itself (zero if absent).
Before Hash
// After is the node hash on the new side, with the same meaning as Before.
After Hash
}
Change is one difference between two versions of a spec.
func Diff ¶ added in v0.0.2
func Diff(a, b Hash, get NodeGetter) ([]Change, error)
Diff reports how the spec rooted at b differs from the one rooted at a, descending only where hashes differ: equal roots return no changes without a fetch. Fields, children and lists are walked in name order and list elements by position, so a reordered list reports each moved position. A zero side (no previous or no next version) is one root Change carrying both hashes, also without a fetch. Fetch failures carry Decode's codes.
type Hash ¶ added in v0.0.2
type Hash [32]byte
Hash is the SHA-256 content address of a Node.
func ParseHash ¶ added in v0.0.2
ParseHash reads a hash from its lowercase or uppercase hex form; anything else is CodeInvalidInput.
type Node ¶ added in v0.0.2
type Node struct {
// Fields are the node's scalar members: strings, bools, numbers (json.Number or
// float64) and, for an in/not_in operand, a []any of those. A nil value, typed nils
// such as a nil slice included, is absent; Encode leaves out a struct member at its
// zero value.
Fields map[string]any
// Children references each singular child node by hash.
Children map[string]Hash
// Lists references each repeated child, in order, by hash.
Lists map[string][]Hash
// Kind names the struct the node encodes (for example chart_spec or filter_clause).
Kind string
}
Node is one struct of a ChartSpec tree, addressed by the hash of its canonical encoding. Scalars sit in Fields; a singular child struct is referenced from Children and a repeated one from Lists, in order, so a change to one subtree changes only the hashes on its path to the root.
func (Node) Canonical ¶ added in v0.0.2
Canonical returns the node's RFC 8785 (JCS) encoding: its JSON shape with keys sorted, numbers in their shortest ECMAScript form and no insignificant whitespace. Equal nodes always encode to the same bytes.
func (Node) Hash ¶ added in v0.0.2
Hash returns the node's content address: SHA-256 over the domain prefix "chartspec/v1\x00" followed by Canonical.
func (Node) MarshalJSON ¶ added in v0.0.2
MarshalJSON writes the node as {kind, fields, children, lists} with hashes as hex, omitting nil fields (typed nils included), zero child hashes and empty lists (and a section left empty). Canonical is the hashed form of the same object.
func (*Node) UnmarshalJSON ¶ added in v0.0.2
UnmarshalJSON reads a node from its JSON shape, keeping numbers as json.Number. A missing kind or a malformed hash is CodeInvalidInput.
type NodeGetter ¶ added in v0.0.2
NodeGetter fetches a stored node by hash, reporting false when the store lacks it.
type PaneKey ¶
type PaneKey struct {
// Row is the row tuple's field keys, comma-joined.
Row string
// Column is the column tuple's field keys, comma-joined.
Column string
}
PaneKey identifies one pane: the Key-joined row tuple and column tuple ("team,model" / "sum(tokens_in)"); an empty shelf contributes "".
type Partial ¶
type Partial struct {
// Sketch is the quantile sketch (nil while empty).
Sketch *ddsketch.DDSketch
// Sum is the sum of the observations.
Sum float64
// Count is the number of observations.
Count float64
// Min is the smallest observation (meaningless while Count is 0).
Min float64
// Max is the largest observation (meaningless while Count is 0).
Max float64
}
Partial is the mergeable summary of a set of observations of one measure: the additive sum, count, min and max, and a DDSketch for quantiles. The zero value is the empty summary (Count 0, no sketch). Merging is associative and commutative, so buckets can be reduced in any order.
func DecodePartial ¶
DecodePartial parses an EncodePartial blob; a truncated or unknown-version blob is CodeInvalidInput.
func NewPartial ¶
NewPartial returns the summary of one observation. A NaN or infinite value is CodeInvalidInput.
type Plan ¶
type Plan struct {
// Marks is the mark drawn in each pane.
Marks map[PaneKey]types.Mark
// Panes are the pane keys, rows outer, columns inner.
Panes []PaneKey
// Query is the store query answering every pane.
Query types.AggregateQuery
}
Plan is a compiled spec: the one AggregateQuery the store runs, the panes in row-major order, and the mark of each pane. Query.OrgID is left empty; the caller sets the verified tenant.
func Compile ¶
Compile turns a parsed spec into its Plan: the pane table is rows × columns of the normalized shelves; the query groups by every shelf dimension (and the time field) plus Detail and encoded dimensions, computes every shelf and encoded measure, and carries the filter, range, grain and sort. Each pane's mark is the spec's Mark or the inference O×O→text, O×Q→bar, T×Q→line, Q×Q→point. A cube mismatch, a cube without Fields, an undeclared grain, a count_distinct measure (which additive partials cannot answer) or a sort key the query neither groups by nor computes is CodeInvalidInput. The cube is the listquery.Schema that list endpoints use too.
type Tree ¶ added in v0.0.2
type Tree struct {
// Nodes holds every node of the tree by hash.
Nodes map[Hash]Node
// Root is the hash of the chart_spec node.
Root Hash
}
Tree is a ChartSpec as content-addressed nodes: every distinct node keyed by its hash, and the root's hash. Equal subtrees share one node.
func Encode ¶ added in v0.0.2
Encode splits spec into content-addressed nodes. A struct member at its zero value is left out of its node, except a filter clause's value, which only nil omits. The compiled Filter is not encoded: FilterTree is the stored filter, which Decode returns for the caller to Parse again. A nil shelf operand or a filter node with more than one arm set is CodeInvalidInput.
type Tuple ¶
type Tuple struct {
// Fields are the field references of the header path, outermost first.
Fields []types.FieldRef
// Nested marks a tuple produced (at any level) by the nest operator.
Nested bool
}
Tuple is one row (or column) of a shelf's normalized table: the field references along that header path, outermost first. Nested marks a tuple built by the nest operator: the reducer keeps only the combinations observed in the data.
func Normalize ¶
Normalize returns the tuple table of a shelf expression: a leaf is one single-field tuple, concat is the union (operands' tuples in order), cross the Cartesian product, and nest the Cartesian product flagged Nested. An empty (nil) shelf normalizes to one empty tuple, the identity of the product.