types

package
v0.25.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 10, 2026 License: MIT Imports: 8 Imported by: 0

Documentation

Overview

Package types provides shared type definitions for pulse.

Index

Constants

View Source
const (
	PairwiseNSourceCellNUnweighted = "cell_n_unweighted"
	PairwiseNSourceCellValueWeight = "cell_value_weighted"
	PairwiseNSourceRowMarginN      = "row_margin_n"
	PairwiseNSourceColumnMarginN   = "column_margin_n"
	PairwiseNSourceNWithin         = "n_within"
	PairwiseNSourceCellWeightSum   = "cell_weight_sum"
)

Pairwise sample-size source modes (PairwiseOverlayParams.NSource).

View Source
const (
	PairwisePSourceCellValuePct = "cell_value_pct"
	PairwisePSourceCellValue    = "cell_value"
)

Pairwise proportion source modes (PairwiseOverlayParams.PSource).

Variables

View Source
var OverlayStreamability = map[OverlayKind]bool{

	OverlayKindChiSqCol: false,

	OverlayKindChiSqMatrix: false,

	OverlayKindChiSqRow: false,

	OverlayKindChiSqVsPop: false,

	OverlayKindChiSqVsRef: false,

	OverlayKindDeltaVsBaseline: false,
	OverlayKindDeltaVsMargin:   false,

	OverlayKindDeltaVsRef: true,

	OverlayKindDeltaVsSibling: false,

	OverlayKindDeltaVsStage: false,

	OverlayKindFisherExactCell: false,

	OverlayKindFormula: false,

	OverlayKindIndexVsBaseline: false,
	OverlayKindIndexVsMargin:   false,

	OverlayKindIndexVsRef: true,

	OverlayKindIndexVsPop: true,

	OverlayKindIndexVsPrior: true,

	OverlayKindIndexVsRollingMean: false,

	OverlayKindIndexVsSibling: false,

	OverlayKindIndexVsStage: false,

	OverlayKindIndexVsTotal: true,

	OverlayKindKSVsPop: false,

	OverlayKindPanelIndexVsRef: true,

	OverlayKindPairwiseProbitT:   false,
	OverlayKindPairwisePropZ:     false,
	OverlayKindPairwiseTwoMeansZ: false,
	OverlayKindPairwiseWelchT:    false,

	OverlayKindPropZCell: false,

	OverlayKindPropZPanel: false,

	OverlayKindRank:       false,
	OverlayKindShareOfCol: false,
	OverlayKindShareOfRow: false,

	OverlayKindShareOfTotal: true,

	OverlayKindTCell: false,

	OverlayKindTVsRef: false,

	OverlayKindYoY:            false,
	OverlayKindZScoreVsMargin: false,

	OverlayKindZScoreVsPop: true,

	OverlayKindZScoreVsRolling: false,

	OverlayKindZScoreVsTotal: true,

	OverlayKindZCell: false,

	OverlayKindZVsRef: false,
}

OverlayStreamability declares whether each overlay kind can be computed inside the streaming Process path. Keyed by the on-wire SCREAMING_SNAKE OverlayKind value.

Per kind-catalog-v1.md "Streaming-capable subset", INDEX_VS_MARGIN is NOT in the streamable set today — its host crosstab is always buffered.

Functions

func AxisFieldNames added in v0.12.0

func AxisFieldNames(axis []*Group) []string

AxisFieldNames returns the field names making up an axis in axis order. Helper for predict warnings and reshape lookups.

func AxisTypes added in v0.12.0

func AxisTypes(axis []*Group) []string

AxisTypes returns the GroupType strings making up an axis in order.

func CanonicalHash added in v0.11.0

func CanonicalHash(tag string, v any) string

CanonicalHash returns a deterministic 32-character hex hash of v's canonical JSON form. The hash is stable across processes and Pulse versions where the request's semantic meaning is unchanged.

The algorithm: encoding/json.Marshal(v) → walk the resulting JSON tree to drop unset entries (omitempty already handled by the struct tags, so this step is mostly a no-op for the request types) and normalize numeric edge cases (negative zero collapses to zero) → re-emit with sorted map keys and no whitespace → SHA-256 of the canonical bytes prefixed with a domain tag → first 16 bytes hex.

tag namespaces hashes so a Request and a ComposedRequest with the same wire bytes do not collide. Callers can use "" for an anonymous hash but the concrete Request.Hash / ComposedRequest.Hash / etc. methods always pass a non-empty tag.

func FormulaNamespace added in v0.19.0

func FormulaNamespace(shape OverlayShape) []string

FormulaNamespace returns the canonical variable identifier set the OVERLAY_FORMULA evaluator exposes for a given host shape. It is the single source of truth both the runtime FORMULA env builders (`processing/overlay_formula.go` `buildFormulaPrototypeEnv*`) and the predict-time identifier validator (`descriptor/overlay_formula.go` `validateFormulaOverlay`) consult so the two surfaces stay in lock-step — adding a new variable to the MATRIX namespace is a one-line change here, and both the runtime prototype env + the predict-time allowed set widen automatically.

Living at this leaf (types/) lets descriptor/ read the namespace without importing processing/ — preserving the TestPredictNoExecutionImports gate (CLAUDE.md "What NOT to Do").

Per-shape contents (research note `.planning/result-overlay-system/research/formula-namespace.md` § 2):

  • OverlayShapeMatrix: cell, margin_row, margin_col, margin_grand, sd_row, sd_col, sd_grand. ref_cell + slot.<label>.cell live on the Compose-specific overlay walk and are added by the Compose validator on top of this base, not here.
  • OverlayShapeSeries: value, total, prior. baseline is opt-in via `OverlaySpec.Params["baseline_position"]` and added by the validator when the param is set, not by this base set. ref_value lives on the Compose walk.
  • OverlayShapeScalar: value. ref lives on the Compose walk.

Returned slices are freshly allocated per call (callers may mutate the returned slice without disturbing future calls). Unknown shapes return an empty slice.

func IsPairwiseOverlayKind added in v0.24.0

func IsPairwiseOverlayKind(kind OverlayKind) bool

IsPairwiseOverlayKind reports whether kind is a member of the OVERLAY_PAIRWISE_* family.

func IsValidNormalize added in v0.12.0

func IsValidNormalize(mode CrosstabNormalize) bool

IsValidNormalize reports whether mode is one of the known normalize strings. Used by validators that accept the zero value as "none".

func IsValidShape added in v0.12.0

func IsValidShape(s CrosstabShape) bool

IsValidShape reports whether s is one of the known shape strings. Used by validators that accept the zero value as "matrix".

func OverlayStreamable added in v0.19.0

func OverlayStreamable(kind OverlayKind) (streamable, known bool)

OverlayStreamable reports whether the given overlay kind streams and whether it is a known entry in OverlayStreamability. An unknown kind returns (false, false) — callers that want to treat "unknown" as "buffered" can ignore the known bit; callers wiring the validator or the manifest capability block can branch on `known` to surface a PULSE_OVERLAY_UNKNOWN-style diagnostic.

Source of truth for predict.Streamable on overlay-bearing requests; cross-checked at test time by TestStreamability_OverlaysKnown which enumerates AllOverlayKinds() and demands every kind has a row here.

func PairwiseKindUsesWelford added in v0.24.0

func PairwiseKindUsesWelford(kind OverlayKind) bool

PairwiseKindUsesWelford reports whether kind reads the Welford triple {mean, variance, n} (welch / two-means) rather than a proportion + n.

func ValidPairwiseNSource added in v0.24.0

func ValidPairwiseNSource(s string) bool

ValidPairwiseNSource reports whether s names a supported NSource mode. Empty counts as valid (defaults to cell_n_unweighted).

func ValidPairwisePSource added in v0.24.0

func ValidPairwisePSource(s string) bool

ValidPairwisePSource reports whether s names a supported PSource mode. Empty counts as valid (defaults to cell_value_pct).

Types

type Aggregation

type Aggregation struct {
	// Type is the aggregation operation to perform.
	Type AggregationType `json:"type"`

	// Field is the name of the data field to aggregate.
	Field string `json:"field"`

	// Label is an optional output label for the aggregation result.
	Label string `json:"label,omitempty"`

	// Params holds type-specific configuration as raw JSON.
	// Used by aggregation types that require additional parameters (e.g., AGG_PERCENTILE).
	Params json.RawMessage `json:"params,omitempty"`
}

Aggregation defines a single aggregation operation to apply to a field.

type AggregationComponents added in v0.20.0

type AggregationComponents struct {
	// Label mirrors the originating Aggregation.Label so callers can
	// join the components entry back to its request slot.
	Label string `json:"label,omitempty"`

	// N is the number of input records observed by the aggregator
	// (post-filter, pre-null-skip). Universal floor — emitted by every
	// aggregator.
	N int `json:"n"`

	// NNull is the number of records the aggregator skipped because
	// the source field was null. Universal floor — emitted by every
	// aggregator.
	NNull int `json:"n_null"`

	// Operator carries the per-aggregator schema-declared keys. Key
	// set is governed by the operator's ComponentSchema declaration in
	// descriptor/capabilities_aggregators.go. Values are JSON-compatible
	// scalars or nested maps.
	Operator map[string]any `json:"operator,omitempty"`
}

AggregationComponents carries per-aggregator constituent-parts metadata. The universal floor — N (records observed) and NNull (records skipped due to null input) — is a typed field on every entry; per-operator keys (mean / variance / mode_count / range_min, etc.) ride inside Operator. Slot identity rides on Label, which mirrors the originating Aggregation.Label so consumers can join the components entry back to the request slot without index arithmetic.

type AggregationType

type AggregationType string

AggregationType identifies a specific aggregation operation.

const (
	AGG_COUNT          AggregationType = "AGG_COUNT"
	AGG_SUM            AggregationType = "AGG_SUM"
	AGG_AVERAGE        AggregationType = "AGG_AVERAGE"
	AGG_MIN            AggregationType = "AGG_MIN"
	AGG_MAX            AggregationType = "AGG_MAX"
	AGG_STDDEV         AggregationType = "AGG_STDDEV"
	AGG_RANGE          AggregationType = "AGG_RANGE"
	AGG_FREQUENCY      AggregationType = "AGG_FREQUENCY"
	AGG_ZSCORE         AggregationType = "AGG_ZSCORE"
	AGG_MEDIAN         AggregationType = "AGG_MEDIAN"
	AGG_VARIANCE       AggregationType = "AGG_VARIANCE"
	AGG_MODE           AggregationType = "AGG_MODE"
	AGG_SKEWNESS       AggregationType = "AGG_SKEWNESS"
	AGG_KURTOSIS       AggregationType = "AGG_KURTOSIS"
	AGG_DISTINCT_COUNT AggregationType = "AGG_DISTINCT_COUNT"
	AGG_PERCENTILE     AggregationType = "AGG_PERCENTILE"

	// AGG_NULL_COUNT counts records where the field is null. Inverse of
	// AGG_COUNT, which counts records where the field is non-null. Stays
	// streamable (O(1) state per row, no buffering).
	AGG_NULL_COUNT AggregationType = "AGG_NULL_COUNT"

	// AGG_WEIGHTED_MEAN computes a weighted arithmetic mean of the
	// target field using a separately-named weight field. Params:
	// `weight_field` (string, required). Emits sum(field * w) / sum(w).
	// Streamable + mergeable via weighted Chan-Welford.
	AGG_WEIGHTED_MEAN AggregationType = "AGG_WEIGHTED_MEAN"

	// AGG_RATIO emits sum(numerator_field) / sum(denominator_field) at
	// finalize. Params: `numerator_field` + `denominator_field`. The
	// Aggregation's own Field is ignored. Streamable + mergeable (two
	// independent sums).
	AGG_RATIO AggregationType = "AGG_RATIO"

	// AGG_CI_LOWER and AGG_CI_UPPER emit the lower / upper bound of a
	// confidence interval for the mean of the named numeric field.
	// Params: `confidence` (float, default 0.95), `method` (string,
	// default "normal"). The "normal" method uses Welford mean + sample
	// variance with z = inv_normal((1+confidence)/2); streamable +
	// mergeable. The "bootstrap" method is reserved for a buffered
	// follow-up — returns PROCESSING_CONFIG today.
	AGG_CI_LOWER AggregationType = "AGG_CI_LOWER"
	AGG_CI_UPPER AggregationType = "AGG_CI_UPPER"

	// AGG_WELFORD emits the streaming Welford-Pébaÿ triple — running
	// mean, running sample variance (n-1 denominator), and observed
	// count — over a numeric field. Returns the running mean through
	// the legacy float64 contract and the typed WelfordTriple via the
	// RichAggregator interface so overlay handlers (OVERLAY_T_CELL /
	// OVERLAY_Z_CELL) can type-switch on the moment payload
	// without re-deriving variance from raw rows. Streamable; mergeable
	// via Chan-Welford. Margin reducibility is recompute (variance does
	// not pool by addition). Field type restricted to the strict scalar
	// numeric family (u8/u16/u32/u64/f32/f64); decimal128 deferred.
	AGG_WELFORD AggregationType = "AGG_WELFORD"

	// AGG_SET_UNION reduces by bitwise OR across rows. The result is a
	// uint64 mask representing the union of every selection observed.
	// Mergeable + streamable; margin-summable for crosstab.
	AGG_SET_UNION AggregationType = "AGG_SET_UNION"

	// AGG_SET_INTERSECTION reduces by bitwise AND across rows. The
	// result mask carries a bit only when every contributing row had
	// that bit set. Mergeable + streamable, but NOT margin-reducible
	// (AND across cells ≠ AND across all rows in general) — crosstab
	// margins recompute from raw rows.
	AGG_SET_INTERSECTION AggregationType = "AGG_SET_INTERSECTION"

	// AGG_SET_FREQUENCY emits a per-element histogram. The result is
	// map[label]int — one entry per dictionary value, counting the rows
	// whose mask had that bit set. Mergeable (bin-by-bin sum) + summable
	// margin. This is the survey-friendly default ("respondents per
	// issuer"). Used as a Crosstab Cell aggregator the result is a
	// map-valued cell payload.
	AGG_SET_FREQUENCY AggregationType = "AGG_SET_FREQUENCY"

	// AGG_SET_CARDINALITY_SUM sums popcount(mask) across rows. Mergeable
	// + streamable + summable margin.
	AGG_SET_CARDINALITY_SUM AggregationType = "AGG_SET_CARDINALITY_SUM"

	// AGG_SET_CARDINALITY_AVG returns mean popcount(mask) per row.
	// Mergeable + streamable; margin = mean-reducible (needs per-cell n).
	AGG_SET_CARDINALITY_AVG AggregationType = "AGG_SET_CARDINALITY_AVG"

	// AGG_SET_DISTINCT_VALUES counts the distinct exact masks observed
	// (treats each combination as atomic). Mergeable via the union of
	// seen-mask sets.
	AGG_SET_DISTINCT_VALUES AggregationType = "AGG_SET_DISTINCT_VALUES"
)

func AllAggregationTypes

func AllAggregationTypes() []AggregationType

AllAggregationTypes returns all defined aggregation types.

func (AggregationType) MapValued added in v0.15.0

func (t AggregationType) MapValued() bool

MapValued reports whether the aggregator emits a non-scalar rich payload via the RichAggregator interface — a per-bit / per-key map (AGG_SET_FREQUENCY) or a structured moment triple (AGG_WELFORD). Used by Crosstab to gate normalize compatibility (dividing one rich payload by another is undefined) and by predict to surface the constraint up front. Aggregators that emit []string rich payloads (AGG_SET_UNION, AGG_SET_INTERSECTION) still expose a meaningful scalar fallback (popcount) so they remain scalar-compatible and are NOT classified as map-valued.

func (AggregationType) MarginReducibility added in v0.12.0

func (t AggregationType) MarginReducibility() MarginReducibility

MarginReducibility classifies how a crosstab margin for this aggregator can be computed. Three classes:

  • MarginSummable — margin = sum of cells (e.g. AGG_COUNT, AGG_SUM, AGG_NULL_COUNT). The reshape pass can derive the margin cheaply from the long-form cell table without re-aggregating.
  • MarginMeanReducible — margin derivable only when each cell also carries its observation count (e.g. AGG_AVERAGE = Σ(cellMean·cellN) / ΣcellN). Pulse does not yet emit per-cell counts in long form, so v1 routes these through the recompute path; the classification is preserved for future optimization.
  • MarginRecompute — margin cannot be derived from cells and must be recomputed over the raw rows (every order- or distribution- dependent aggregator: AGG_MEDIAN, AGG_PERCENTILE, AGG_STDDEV, AGG_VARIANCE, AGG_MODE, etc.).

The default branch returns MarginRecompute so any newly-added aggregator that forgets to opt in computes the correct margin (slower but right) rather than silently producing the wrong margin.

service/crosstab.go consults this method to decide whether the margin can be derived from cell sums (when MarginSummable + no normalization round-off concern) or must be recomputed via a sibling Compose request. In v1 every margin is recomputed (see skills/crosstab-guide.md "Margin computation"); the classification drives the manifest capability block and future fast-path work.

func (AggregationType) Mergeable added in v0.8.0

func (t AggregationType) Mergeable() bool

Mergeable reports whether this aggregation type's running state can be combined across partitions of the input via an associative+ commutative merge (count/sum/min/max/null_count), a parallel-friendly recurrence (Welford-mean / variance / stddev), or a union of per- value count maps (frequency / mode / distinct_count). The per-shard parallel reducer in service/shard_reduce.go consults this method (mirrored by processing.CanMergeRequest) to decide whether to fan out shard processing across a bounded worker pool.

Mergeable implies Streamable — a buffered-only aggregator cannot expose mergeable state. The default branch returns false so newly- added aggregator types must opt in explicitly. AGG_MEDIAN / AGG_PERCENTILE / AGG_ZSCORE require a sorted view of every value and stay non-mergeable; AGG_SKEWNESS / AGG_KURTOSIS rely on M3/M4 recurrences whose parallel-merge formula is non-trivial and is deferred to a follow-up — they fall through to the serial path.

func (AggregationType) Streamable added in v0.2.0

func (t AggregationType) Streamable() bool

Streamable reports whether this aggregation type supports the single-pass streaming execution path. Streamable aggregators implement processing.OnlineAggregator (UpdateRow + Finalize) and produce a result with O(1) or O(unique) state per row.

Source of truth for predict.Streamable; cross-checked at test time against the processing registry by TestRegistryStreamabilityMatchesTypes.

Default branch returns false so newly-added aggregator types must opt in explicitly.

type Attribute

type Attribute struct {
	// Type is the attribute computation to perform.
	Type AttributeType `json:"type"`

	// Field is the name of the source data field. Optional for the
	// regression attribute family (ATTR_REG_FITTED / RESIDUAL / LEVERAGE)
	// which read Target + Predictors instead; when empty the output Label
	// defaults to "<TYPE>_<Target>".
	Field string `json:"field"`

	// Label is the output name for the derived attribute.
	Label string `json:"label,omitempty"`

	// Expression is a runtime expression for ATTR_FORMULA type.
	Expression string `json:"expression,omitempty"`

	// Params holds type-specific configuration as raw JSON.
	// Each attribute type defines its own params schema.
	Params json.RawMessage `json:"params,omitempty"`

	// Target is the dependent variable for ATTR_REG_FITTED / RESIDUAL /
	// LEVERAGE. Required for those types.
	Target string `json:"target,omitempty"`

	// Predictors lists the independent variables for the regression-
	// attribute family. At least one predictor is required.
	Predictors []string `json:"predictors,omitempty"`

	// Penalty selects the OLS regularization scheme for ATTR_REG_FITTED /
	// ATTR_REG_RESIDUAL. One of "" (unpenalized), "l1" (lasso), "l2"
	// (ridge), or "elasticnet". ATTR_REG_LEVERAGE rejects any non-empty
	// Penalty (penalized leverage and GLM leverage are deferred to a
	// later phase).
	Penalty string `json:"penalty,omitempty"`

	// Alpha is the regularization strength for the regression-attribute
	// family when Penalty is non-empty. Must be > 0 in that case.
	Alpha float64 `json:"alpha,omitempty"`

	// L1Ratio is the elastic-net mixing parameter for the regression-
	// attribute family when Penalty == "elasticnet" (0 < L1Ratio < 1).
	L1Ratio float64 `json:"l1_ratio,omitempty"`
}

Attribute defines a derived attribute computation.

type AttributeType

type AttributeType string

AttributeType identifies a specific derived-attribute computation.

const (
	ATTR_ZSCORE     AttributeType = "ATTR_ZSCORE"
	ATTR_TSCORE     AttributeType = "ATTR_TSCORE"
	ATTR_NORMALIZED AttributeType = "ATTR_NORMALIZED"
	ATTR_FORMULA    AttributeType = "ATTR_FORMULA"
	ATTR_PERCENTILE AttributeType = "ATTR_PERCENTILE"
	ATTR_DATE_PART  AttributeType = "ATTR_DATE_PART"

	// ATTR_REG_FITTED emits the per-row fitted value ŷᵢ = Xᵢ · β + β₀ for
	// each filter-passing record, using a regression fit computed in a
	// streaming prepass over the same record set. Two-pass: prepass folds
	// the OLS sufficient statistics, finalize freezes the coefficient
	// vector, pass 2 emits ŷᵢ. Carries its own RegressionSpec-shaped
	// fields (Target, Predictors, Penalty, Alpha, L1Ratio); refits
	// internally per attribute (Option A). Accepts any OLS penalty
	// (unpenalized, ridge, lasso, elasticnet); GLM and Bayesian fits are
	// deferred.
	ATTR_REG_FITTED AttributeType = "ATTR_REG_FITTED"

	// ATTR_REG_RESIDUAL emits the per-row residual yᵢ − ŷᵢ for each
	// filter-passing record, using the same OLS prepass machinery as
	// ATTR_REG_FITTED. Sums to ≈ 0 when the fit includes an intercept
	// (always true for unpenalized OLS) and the response is observed for
	// every contributing row.
	ATTR_REG_RESIDUAL AttributeType = "ATTR_REG_RESIDUAL"

	// ATTR_REG_LEVERAGE emits the per-row hat-matrix diagonal
	// hᵢᵢ = xᵢ · (XᵀX)⁻¹ · xᵢᵀ for each filter-passing record. Restricted
	// to unpenalized OLS — penalized leverage uses a different formula
	// (involving the regularized resolvent) and GLM leverage requires the
	// IRLS weight matrix; both deferred to Phase 9. Specs with any
	// Penalty set are rejected at factory time with PROCESSING_CONFIG.
	ATTR_REG_LEVERAGE AttributeType = "ATTR_REG_LEVERAGE"

	// ATTR_SET_POPCOUNT emits popcount(mask) per row as a numeric column.
	// Field must reference a set_* field. Streamable.
	ATTR_SET_POPCOUNT AttributeType = "ATTR_SET_POPCOUNT"

	// ATTR_SET_HAS emits a packed_bool per row indicating whether the
	// set field has a specific dictionary label set. Params.value names
	// the label whose bit is tested. Streamable.
	ATTR_SET_HAS AttributeType = "ATTR_SET_HAS"
)

func AllAttributeTypes

func AllAttributeTypes() []AttributeType

AllAttributeTypes returns all defined attribute types.

func (AttributeType) Streamable added in v0.2.0

func (t AttributeType) Streamable() bool

Streamable reports whether this attribute type can be computed in a streaming path. Three tiers exist at runtime:

  • Row-local: FORMULA, DATE_PART implement processing.RowLocalAttribute and execute inline with no PrePass.
  • Two-pass: ZSCORE, TSCORE, NORMALIZED implement processing.TwoPassAttribute and need a PrePass over filter-passing records, Finalize, then per-row Row() in pass 2 (iter.Reset()).
  • Buffered-only: PERCENTILE needs a sorted view of every value; no streaming algorithm preserves exact rank semantics.

Streamable() returns true for the first two tiers since both routes avoid materializing the full record set in memory.

type AxisHeader added in v0.12.0

type AxisHeader struct {
	// Fields lists the grouper Field names in axis order.
	Fields []string `json:"fields"`
	// Types lists the GroupType strings in axis order (e.g.
	// "GROUP_CATEGORY", "GROUP_RANGE"). Carried so downstream consumers
	// can render bin labels correctly without re-deriving from a
	// schema.
	Types []string `json:"types"`
}

AxisHeader names the fields making up one axis tuple. Field i in the header corresponds to position i in every AxisKey on that axis.

type AxisKey added in v0.12.0

type AxisKey []any

AxisKey is the per-axis tuple of dictionary keys identifying one row or column of the materialized matrix. Each entry is the value emitted by the corresponding grouper in the configured Rows / Columns slice (in declaration order). Encoded as []any so callers can compare against the long-form result rows directly — numeric bins stay numeric, categorical keys stay strings, date keys stay strings (whatever the grouper emitted).

type ChainOverlaySpec added in v0.19.0

type ChainOverlaySpec struct {
	// Name is the renderer-facing label for this overlay. When
	// empty, the processing layer synthesises a deterministic
	// default.
	Name string `json:"name,omitempty"`

	// Kind selects the whole-chain overlay catalog entry to
	// execute.
	Kind OverlayKind `json:"kind"`

	// Ref names the baseline stage to compare against. Exactly one
	// of Ref.Index / Ref.Name must be populated.
	Ref StageRef `json:"ref"`

	// Target names the stage whose result the comparison surface
	// decorates. Exactly one of Target.Index / Target.Name must be
	// populated.
	Target StageRef `json:"target"`

	// Scope declares where the overlay lands relative to the
	// target stage's result. Reuses the universal OverlayScope
	// enum.
	Scope OverlayScope `json:"scope"`

	// Params holds kind-specific configuration as a free-form map.
	// The per-kind schema is documented alongside the kind's
	// processor. Omitted on the wire when nil.
	Params map[string]any `json:"params,omitempty"`
}

ChainOverlaySpec is the request-side definition of one whole-chain overlay layer. Each spec produces one OverlayLayer in ChainResponse.Overlays in matching index order. The shape mirrors OverlaySpec (Name / Kind / Scope / Params) but swaps the per-result OverlayRef discriminated union for the StageRef pair (Ref baseline + Target) so the whole-chain kinds can address two distinct stage results inside the same ChainRequest without re-opening OverlayRef.

Validation rules (enforced in descriptor + processing layers, not this file — land in descriptor/chain_overlay.go + processing/overlay_chain_dispatch.go):

  • Kind is required and must be a known OverlayKind whose whole-chain catalog entry exists (OVERLAY_INDEX_VS_STAGE, OVERLAY_DELTA_VS_STAGE).
  • Scope is required.
  • Ref and Target each populate exactly one of (Index, Name) (XOR validated by the per-kind validator).
  • Name (when set) becomes the renderer-facing label; when empty the processing layer synthesises a deterministic default keyed by Kind + Ref + Target.
  • Params carries kind-specific configuration; the per-kind schema lives alongside the kind's processor.

type ChainRequest added in v0.10.0

type ChainRequest struct {
	// Cohort identifies the source .pulse file (single file or shard
	// archive) for stage 0.
	Cohort *Cohort `json:"cohort"`

	// Stages is the ordered list of pipeline stages. Length must be
	// >= 1. Each stage's Request is run as a normal Process call
	// against the synthesized iterator from the prior stage (or
	// against Cohort for stage 0).
	Stages []*ChainStage `json:"stages"`

	// Overlays is the optional list of whole-chain overlay layer
	// specifications. Whole-chain overlays operate AFTER every stage
	// in Stages finishes (NOT between stages — per-stage overlays
	// ride the individual ChainStage.Request.Overlays slot) and emit
	// one ChainResponse.Overlays entry per spec in matching index
	// order. The dual-slot design (per-stage on Stages[i].Request,
	// whole-chain here) means callers can decorate any individual
	// stage's result OR the post-finalize chain result with the same
	// universal OverlayKind catalog; the StageRef discriminated
	// reference (see below) is what the whole-chain kinds
	// (OVERLAY_INDEX_VS_STAGE, OVERLAY_DELTA_VS_STAGE) consume to
	// identify which stage's result is the comparison baseline and
	// which stage's result is the comparison target.
	//
	// Forward-compat: a ChainRequest without whole-chain overlays
	// produces byte-identical JSON to the same request with
	// `Overlays: nil` because the slot is `omitempty` — nil / empty
	// slices marshal to no key at all. The canonical-hash routine
	// inherits that contract via the data-driven JSON walk in
	// types/hash.go (see TestChainCanonicalHash_OverlayFreeByteIdentity).
	Overlays []*ChainOverlaySpec `json:"overlays,omitempty"`
}

ChainRequest bundles a source-rooted linear chain of Requests. Each stage after the first feeds off the previous stage's output rows rather than re-opening the cohort file. Mergeable-only at v1: see processing.CanChainRequest for the gate; non-mergeable stages surface PULSE_CHAIN_NOT_MERGEABLE with the offending index so the caller can fall back to per-stage Process calls.

Cohort identifies the source for the FIRST stage. Stages after the first ignore their inner Request.Cohort; the chain executor wires the prior stage's output as the next stage's input automatically.

func (*ChainRequest) Hash added in v0.11.0

func (r *ChainRequest) Hash() string

Hash returns the canonical content hash of the ChainRequest.

Slot coverage is data-driven: every JSON-tagged field on ChainRequest participates via the json.Marshal → canonicalize → sha256 pipeline. The whole-chain Overlays slot is covered automatically — each ChainOverlaySpec contributes its Name / Kind / Ref{Index,Name} / Target{Index,Name} / Scope / Params to the canonical bytes in declared spec order (slices preserve order), and the StageRef discriminated reference hashes only the populated arm because every field carries `json:",omitempty"`. An overlay-free ChainRequest (nil or empty Overlays slice) omits the `overlays` key entirely via the slot's own `omitempty` tag, so its hash is byte-identical to the pre-Overlays canonical form — see TestChainCanonicalHash_OverlayFreeByteIdentity.

type ChainResponse added in v0.10.0

type ChainResponse struct {
	// Stages holds per-stage responses in input order. The last
	// entry matches Final.
	Stages []*Response `json:"stages"`

	// Final aliases the last entry in Stages for ergonomic access.
	Final *Response `json:"final"`

	// NormalizedRequest is the chain request the engine actually
	// executed — each stage's Request reflects smart-defaults
	// resolution against its per-stage schema (the original cohort
	// for stage 0, the synthesised stage-output schemas for stages
	// 1+). Populated only when pulse.Options.EchoRequest is true at
	// service-construction time; nil otherwise so the wire size is
	// unchanged for callers that do not need it. Serialised under
	// the omitempty rule.
	NormalizedRequest *ChainRequest `json:"normalized_request,omitempty"`

	// Overlays carries the executed whole-chain overlay layers, one
	// entry per ChainRequest.Overlays spec in matching index order.
	// Reuses the OverlayLayer wrapper from types/overlay.go so
	// downstream renderers handle a per-stage layer and a
	// whole-chain layer with the same payload / summary machinery.
	//
	// Empty (or nil) when the request carried no whole-chain
	// overlays — the slot is `omitempty` so the wire form stays
	// byte-identical to the overlay-free ChainResponse for overlay-free
	// chains.
	Overlays []*OverlayLayer `json:"overlays,omitempty"`
}

ChainResponse is the result of a ChainRequest. Final is the last stage's response; Stages carries per-stage responses for callers that want to inspect intermediate outputs (matches Compose's per-request response shape).

type ChainStage added in v0.10.0

type ChainStage struct {
	// Name is an optional diagnostic label (e.g. "filter_active",
	// "group_by_region"). Echoed in chain envelope.
	Name string `json:"name,omitempty"`

	// Request is the per-stage processing request. Cohort field is
	// ignored for stages > 0. Per-stage overlays ride
	// Request.Overlays — whole-chain overlays ride the parent
	// ChainRequest.Overlays slot. The dual-slot design is
	// intentional: per-stage overlays decorate an intermediate
	// result, whole-chain overlays decorate the post-finalize chain
	// result.
	Request *Request `json:"request"`
}

ChainStage wraps a single Request with optional diagnostic name. The name is surfaced verbatim in ChainResponse.Stages and in PULSE_CHAIN_NOT_MERGEABLE error details when a stage rejects the chain gate.

type Cohort

type Cohort struct {
	// Filename is the name of the .pulse file.
	Filename string `json:"filename"`

	// DataDir is the directory containing the cohort file.
	DataDir string `json:"data_dir,omitempty"`
}

Cohort identifies a .pulse data file for processing.

type ComponentsMergeability added in v0.20.0

type ComponentsMergeability string

ComponentsMergeability classifies how an operator's components map folds across streaming chunks. The orchestrator uses this flag to decide whether to emit per-chunk component deltas, stage a terminal merge, or suppress streaming-chunk components entirely.

String constants are wire-stable and surface in the manifest JSON; the values are snake_case to match the rest of the --json envelope (Output Format Contract in CLAUDE.md).

types/ is the single source of truth — descriptor/ aliases this type (see descriptor/components.go) so the enum stays accessible from every package (types is the leaf; descriptor and processing both import it without cycle risk).

const (
	// ComponentsMergeable signals that the components map folds across
	// chunks via the same associative/commutative path as the scalar
	// value. Welford-family (mean / variance / sum_squares / m2),
	// sums, counts, set masks, and weighted accumulators are
	// mergeable. Per-chunk components are safe to emit and merge
	// online.
	ComponentsMergeable ComponentsMergeability = "mergeable"

	// ComponentsPartial signals that the components map merges across
	// chunks but at non-trivial allocation cost — map / set unions
	// where the fold is associative but not constant-space.
	// AGG_FREQUENCY, AGG_MODE, AGG_DISTINCT_COUNT, and
	// AGG_SET_FREQUENCY are partial. The orchestrator may stage the
	// merge at terminal flush.
	ComponentsPartial ComponentsMergeability = "partial"

	// ComponentsNone signals that the components map cannot be
	// computed from a per-chunk partial — the operator needs a
	// sorted view (or equivalent) of the full input. AGG_MEDIAN /
	// AGG_PERCENTILE are the canonical Nones. Streaming chunks omit
	// components for these operators; predict declares the slot as
	// buffered-components-only.
	ComponentsNone ComponentsMergeability = "none"
)

type ComposeOverlaySpec added in v0.19.0

type ComposeOverlaySpec struct {
	// Name is the renderer-facing label for this overlay. When empty,
	// the processing layer synthesises a deterministic default keyed
	// by Kind + Reference + Targets.
	Name string `json:"name,omitempty"`

	// Kind selects the Compose-only overlay catalog entry to execute.
	// Compose-only kinds consume Reference / Targets instead of the
	// in-Request OverlayRef discriminated union; the universal
	// OverlayKind enum still namespaces every kind.
	Kind OverlayKind `json:"kind"`

	// Scope declares where the overlay lands relative to the target
	// slot's result. Reuses the universal OverlayScope enum so
	// downstream renderers handle Compose layers and per-Request
	// layers with the same scope machinery.
	Scope OverlayScope `json:"scope,omitempty"`

	// Reference names the baseline slot to compare against. Resolves
	// against the parent ComposedRequest's per-Request Label field
	// (after the auto-default has filled empty Labels with
	// `request_<index+1>`). Non-empty is the structural requirement;
	// per-kind validation rejects references that do not resolve to a
	// known slot label with PULSE_OVERLAY_REF_UNKNOWN.
	Reference string `json:"reference"`

	// Targets names the slot label(s) whose result(s) the comparison
	// surface decorates. Length >= 1 for every kind today; multi-ref
	// kinds consume the full slice, single-target kinds
	// consume Targets[0]. Order is significant — the canonical-hash
	// walker preserves slice order via the data-driven JSON walk.
	Targets []string `json:"targets,omitempty"`

	// Level truncates the same axis the overlay scopes to a
	// parent-grouper prefix at the configured depth (mirrors the
	// per-Request OverlaySpec.Level semantics; see types/overlay.go
	// for the buffered-crosstab NormalizeLevel mapping). Default zero
	// is `omitempty` so the wire shape stays byte-identical to the
	// pre-Level handler output. Honoured only by matrix-shape Compose
	// kinds; non-matrix Compose kinds ignore the slot.
	Level int `json:"level,omitempty"`

	// Within fixes a prefix of the OPPOSITE axis at the configured
	// depth (mirrors the per-Request OverlaySpec.Within semantics).
	// Default zero is `omitempty`. Honoured only by matrix-shape
	// Compose kinds; non-matrix Compose kinds ignore the slot.
	Within int `json:"within,omitempty"`

	// Params holds kind-specific configuration as a free-form map.
	// The per-kind schema lands alongside each kind's processor; v1
	// uses: Params["population"] for OVERLAY_RANK,
	// Params["scale"] for share-vs-index variants, etc. Omitted on
	// the wire when nil.
	Params map[string]any `json:"params,omitempty"`

	// Options carries per-spec optimization knobs that switch the
	// runtime into a faster but caller-attested-correct execution
	// mode. Default nil (and field-level omitempty inside
	// OverlayOptions) keeps the canonical-hash byte-identical to a
	// pre-Options ComposeOverlaySpec — every knob defaults to the SAFE
	// path. The available knobs (DictPrefixFast, MaxPanelTargets) live
	// on OverlayOptions in types/overlay.go.
	Options *OverlayOptions `json:"options,omitempty"`
}

ComposeOverlaySpec is the request-side definition of one Compose-only overlay layer attached to a ComposedRequest. Each spec produces one OverlayLayer in ComposedResponse.Overlays in matching index order. The shape mirrors OverlaySpec (Name / Kind / Scope / Level / Within / Params) but swaps the per-result OverlayRef discriminated union for a pair of caller-supplied slot-label strings — Reference (the baseline slot to compare against) and Targets (one or more slot labels whose results the comparison surface decorates). The slot labels resolve against the parent ComposedRequest's per-Request Label field; the auto-default rule means an empty caller-supplied Label resolves to `request_<index+1>` (1-based) before reference lookup, so every slot has a stable name to address.

Field-order matters for the canonical hash — keep Name, Kind, Scope, Reference, Targets, Level, Within, Params in that order across the Go struct and the JSON envelope. The canonical-hash walker is data-driven over json.Marshal output so re-ordering would not break the hash, but the field order is the documented contract for downstream parsers + visual diff stability.

type ComposedRequest

type ComposedRequest struct {
	// Requests is the list of individual requests to execute.
	Requests []*Request `json:"requests"`

	// Overlays is the optional list of Compose-only overlay layer
	// specifications. Each spec is a cross-Request overlay that
	// decorates one slot's result with a comparison against another
	// slot's result inside the same ComposedRequest — the slots are
	// addressed by their per-Request Label field (with empty Labels
	// auto-defaulted to `request_<index+1>` before reference lookup,
	// before reference lookup). Produces one OverlayLayer in
	// ComposedResponse.Overlays in matching index order.
	//
	// Forward-compat: a ComposedRequest without overlays produces
	// byte-identical JSON to the same request with `Overlays: nil`
	// because the slot is `omitempty` — nil / empty slices marshal to
	// no key at all. The canonical-hash routine inherits that contract
	// via the data-driven JSON walk in types/hash.go (see
	// TestComposedRequest_OverlayFreeByteIdentity in
	// types/hash_test.go).
	Overlays []ComposeOverlaySpec `json:"overlays,omitempty"`
}

ComposedRequest bundles multiple requests for batch execution.

func (*ComposedRequest) Hash added in v0.11.0

func (r *ComposedRequest) Hash() string

Hash returns the canonical content hash of the ComposedRequest.

Slot coverage is data-driven: every JSON-tagged field on ComposedRequest participates in the hash via the json.Marshal → canonicalize → sha256 pipeline. The per-Request Label slot folds in through the json walk of each *Request — empty Labels are `omitempty` and stay byte-identical to the pre-Label form. Before hashing, the helper applies the auto-default rule that service/compose_label.go uses at validate time: every slot whose Label arrives empty is rewritten to `request_<index+1>` (1-based) against a shallow clone, so two ComposedRequests differing only in empty-vs-auto-defaulted Labels hash identically. Callers that already filled the Label explicitly are unaffected — the auto- default only fires when Label is empty. This mirrors the validate- time normalizer in service/compose_label.go so hash equality lines up with execution-time slot identity.

The Compose-level Overlays slot is covered automatically — each ComposeOverlaySpec contributes its Name / Kind / Scope / Reference / Targets / Level / Within / Params to the canonical bytes in declared spec order (slices preserve order). An overlay- free ComposedRequest (nil or empty Overlays slice) omits the `overlays` key entirely via the slot's own `omitempty` tag, so its hash is byte-identical to the pre-Overlays canonical form — see TestComposedRequest_OverlayFreeByteIdentity.

type ComposedResponse added in v0.19.0

type ComposedResponse struct {
	// Responses is the per-slot list of Response objects, one entry per
	// ComposedRequest.Requests slot in matching order.
	Responses []*Response `json:"responses"`

	// Overlays carries the executed Compose-level overlay layers, one
	// entry per ComposedRequest.Overlays spec in matching order. Each
	// layer holds its derived payload (scalar / series / matrix) plus
	// optional renderer-friendly summary metadata — reuses the
	// OverlayLayer shape so renderers handle Compose layers and
	// per-Request layers with the same machinery. Omitted entirely when
	// the originating ComposedRequest had no Overlays.
	Overlays []OverlayLayer `json:"overlays,omitempty"`
}

ComposedResponse is the structured response shape for ComposedRequest execution. It carries the per-slot Response objects emitted by service.Compose / service.ComposeParallel alongside the Compose-level Overlays slice — one OverlayLayer per ComposedRequest.Overlays spec in matching index order.

Scope note: the type lives here so the request-side ComposeOverlaySpec catalog has a typed response sibling to write against. The pulse.Pulse.Compose facade now returns *ComposedResponse and the runtime populates the Overlays slot here at the post-slot barrier.

Forward-compat: every ComposedResponse marshalled before any overlay landed produces byte-identical JSON to the same shape with `Overlays: nil` because the slot is `omitempty` — nil / empty slices marshal to no key at all. See TestComposedResponse_OverlayFreeByteIdentical in types/types_test.go for the lock.

type CrosstabComponents added in v0.20.0

type CrosstabComponents struct {
	// CellCounts is the per-cell record count matrix. CellCounts[r][c]
	// mirrors MatrixPayload.Cells[r][c] coordinate-for-coordinate.
	CellCounts [][]int `json:"cell_counts,omitempty"`

	// CellComponents is the per-cell aggregator components matrix.
	// CellComponents[r][c] carries the keys declared by the cell
	// aggregator's ComponentSchema.
	CellComponents [][]map[string]any `json:"cell_components,omitempty"`

	// RowMarginCounts is the per-row-margin record count vector,
	// indexed in MatrixPayload.RowKeys order.
	RowMarginCounts []int `json:"row_margin_counts,omitempty"`

	// RowMarginComponents is the per-row-margin aggregator components
	// vector, indexed in MatrixPayload.RowKeys order.
	RowMarginComponents []map[string]any `json:"row_margin_components,omitempty"`

	// ColumnMarginCounts is the per-column-margin record count vector,
	// indexed in MatrixPayload.ColumnKeys order.
	ColumnMarginCounts []int `json:"column_margin_counts,omitempty"`

	// ColumnMarginComponents is the per-column-margin aggregator
	// components vector, indexed in MatrixPayload.ColumnKeys order.
	ColumnMarginComponents []map[string]any `json:"column_margin_components,omitempty"`

	// GrandTotalCount is the grand-total record count (mirrors the
	// count behind MatrixPayload.GrandTotal).
	GrandTotalCount int `json:"grand_total_count,omitempty"`

	// GrandTotalComponents is the grand-total aggregator components
	// payload (mirrors the components behind MatrixPayload.GrandTotal).
	GrandTotalComponents map[string]any `json:"grand_total_components,omitempty"`

	// RowKeyComponents carries one grouper-components payload per row,
	// indexed in MatrixPayload.RowKeys order. The key set on each
	// element matches GrouperComponents.Operator for the row-axis
	// grouper.
	RowKeyComponents []map[string]any `json:"row_key_components,omitempty"`

	// ColumnKeyComponents carries one grouper-components payload per
	// column, indexed in MatrixPayload.ColumnKeys order. The key set on
	// each element matches GrouperComponents.Operator for the
	// column-axis grouper.
	ColumnKeyComponents []map[string]any `json:"column_key_components,omitempty"`

	// IncludedRecords is the number of records that contributed to at
	// least one cell (sanity counter — independent of the cell count
	// matrix sum, which double-counts records under multi-key groupers).
	IncludedRecords int `json:"included_records,omitempty"`

	// ExcludedRecords is the number of records dropped at the crosstab
	// stage (null axis key, missing dict entry, etc.). IncludedRecords
	// + ExcludedRecords equals the post-filter input record count.
	ExcludedRecords int `json:"excluded_records,omitempty"`
}

CrosstabComponents carries the constituent-parts metadata for a crosstab response. Mirrors MatrixPayload coordinate-for-coordinate so consumers can index components by the same (row, column) tuple they already use to read MatrixPayload.Cells / RowMargins / ColumnMargins / GrandTotal. Every field is `omitempty` so an unpopulated CrosstabComponents marshals to byte-identical wire output against the pre-Components baseline; an empty struct produces no JSON keys at all.

Layout (each axis indexed in the same order as MatrixPayload.RowKeys / MatrixPayload.ColumnKeys):

  • CellCounts[r][c] — per-cell record count (mirrors MatrixPayload.Cells[r][c]).
  • CellComponents[r][c] — per-cell aggregator components; keys are governed by the cell aggregator's ComponentSchema declaration in descriptor/capabilities_aggregators.go.
  • RowMarginCounts[r] / RowMarginComponents[r] — row-margin counts
  • components, indexed by row (mirrors MatrixPayload.RowMargins).
  • ColumnMarginCounts[c] / ColumnMarginComponents[c] — column-margin counts + components, indexed by column (mirrors MatrixPayload.ColumnMargins).
  • GrandTotalCount / GrandTotalComponents — grand-total counterparts (mirror MatrixPayload.GrandTotal).
  • RowKeyComponents[r] / ColumnKeyComponents[c] — per-axis grouper components carried alongside each row / column tuple (bucket edges, dict mappings, etc. — same shape as GrouperComponents.Operator). Indexed by row / column position so consumers can join axis-key metadata to the corresponding MatrixPayload.RowKeys[r] / ColumnKeys[c] entry.
  • IncludedRecords / ExcludedRecords — sanity counters. IncludedRecords is the number of records that contributed to at least one cell; ExcludedRecords is the number that were filtered out at the crosstab stage (null axis key, etc.). Their sum equals the post-filter input record count.

Service-side population lives in service/crosstab.go.

type CrosstabMargins added in v0.12.0

type CrosstabMargins struct {
	Rows    bool `json:"rows,omitempty"`
	Columns bool `json:"columns,omitempty"`
	Grand   bool `json:"grand,omitempty"`
}

CrosstabMargins selects which margins are emitted on the response. Display flags are independent; a normalize direction may still trigger the internal computation of a margin whose display flag is false.

type CrosstabNormalize added in v0.12.0

type CrosstabNormalize string

CrosstabNormalize selects how cell values are normalized prior to emission.

const (
	// CrosstabNormalizeNone leaves cell values as the raw cell aggregation.
	CrosstabNormalizeNone CrosstabNormalize = "none"
	// CrosstabNormalizeRow divides each cell by its row margin (cells in a row sum to 1).
	CrosstabNormalizeRow CrosstabNormalize = "row"
	// CrosstabNormalizeColumn divides each cell by its column margin (cells in a column sum to 1).
	CrosstabNormalizeColumn CrosstabNormalize = "column"
	// CrosstabNormalizeTotal divides each cell by the grand-total margin (whole table sums to 1).
	CrosstabNormalizeTotal CrosstabNormalize = "total"
)

func NormalizedString added in v0.12.0

func NormalizedString(mode string) CrosstabNormalize

NormalizedString returns the trimmed, lowercased form of a normalize mode string so callers can accept user-supplied variants like "Row".

type CrosstabResult added in v0.12.0

type CrosstabResult struct {
	// Shape echoes the shape that was emitted (after default resolution).
	Shape CrosstabShape `json:"shape"`

	// Matrix is the dense matrix payload. Nil when shape=long.
	Matrix *MatrixPayload `json:"matrix,omitempty"`
}

CrosstabResult is the top-level result of a crosstab request. Carries the matrix payload when shape=matrix; long-form rows land on Response.Data and Result.Long indicates the shape.

type CrosstabShape added in v0.12.0

type CrosstabShape string

CrosstabShape selects the response payload layout.

const (
	// CrosstabShapeMatrix returns Response.Crosstab populated with a
	// MatrixPayload (row/col axis tuples + dense cell matrix + margins).
	// Inherently buffered.
	CrosstabShapeMatrix CrosstabShape = "matrix"
	// CrosstabShapeLong returns Response.Data with one tuple per
	// (row-key, column-key) cell — the existing grouped-tuple shape any
	// consumer of grouped output already handles. Margin rows are
	// flagged via the `_margin` field when emitted.
	CrosstabShapeLong CrosstabShape = "long"
)

type CrosstabSpec added in v0.12.0

type CrosstabSpec struct {
	Rows    []*Group     `json:"rows"`
	Columns []*Group     `json:"columns"`
	Cell    *Aggregation `json:"cell"`

	Margins   CrosstabMargins   `json:"margins,omitzero"`
	Normalize CrosstabNormalize `json:"normalize,omitempty"`
	Shape     CrosstabShape     `json:"shape,omitempty"`

	// NormalizeLevel selects the depth in the nested axis whose value
	// constitutes the 100% denominator for normalization. Zero-indexed
	// from the top of the axis (0 = first grouper). Applies only when
	// Normalize=row or Normalize=column. Absent (nil) defaults to the
	// leaf — len(axis)-1 — which is the original per-leaf-tuple
	// normalization behavior. Rejected when set with normalize=total
	// (no axis to descend) or normalize=none.
	NormalizeLevel *int `json:"normalize_level,omitempty"`

	// NormalizeWithin selects a prefix depth on the OTHER axis whose
	// value, combined with the full normalize-axis key (or its
	// NormalizeLevel-truncated form), constitutes the 100% denominator.
	// Zero-indexed from the top of the other axis (0 = first grouper).
	// Applies only when Normalize=row or Normalize=column. Absent (nil)
	// leaves the denominator scope unchanged — the existing row /
	// column marginal. Rejected when set with normalize=none or
	// normalize=total (no other axis to partition).
	//
	// Example: Rows=[brand], Columns=[wavedate, xxx], Normalize=row,
	// NormalizeWithin=0 ⇒ each cell is divided by Σ over xxx within
	// (brand, wavedate); the (brand, wavedate) slab of xxx cells sums
	// to 1. Combines independently with NormalizeLevel — that field
	// truncates the normalize axis; this one fixes a prefix of the
	// OTHER axis.
	NormalizeWithin *int `json:"normalize_within,omitempty"`
}

CrosstabSpec is the request-side definition of a cross-tabulation. It composes the existing grouper + aggregator machinery for cell computation and adds a reshape + margins + normalization layer.

Validation rules (enforced in both predict and execution):

  • Rows must contain at least one Group.
  • Columns must contain at least one Group.
  • Cell is required.
  • A crosstab section cannot coexist with top-level Aggregations / Groups on the same Request (PULSE_CROSSTAB_CONFLICTS_WITH_GROUPS).
  • Normalize=row requires row margins internally; normalize=column requires column margins; normalize=total requires the grand total. The margins are computed even when the corresponding display flag is false; only emission depends on the display flag.

func (*CrosstabSpec) CellLabel added in v0.12.0

func (s *CrosstabSpec) CellLabel() string

CellLabel returns the output label the cell aggregation would emit in a long-form result row. Mirrors processing.AggregationLabel without importing processing/.

func (*CrosstabSpec) IsBuffered added in v0.12.0

func (s *CrosstabSpec) IsBuffered() bool

IsBuffered reports whether the section forces the buffered execution path. Matrix shape, any margin, or any normalization buffers; only shape=long with no margins and normalize=none can stream the cell aggregation through the existing grouped streaming path.

func (*CrosstabSpec) LowerColumnsOnly added in v0.12.0

func (s *CrosstabSpec) LowerColumnsOnly(src *Request) *Request

LowerColumnsOnly builds the columns-only request used to recompute column margins.

func (*CrosstabSpec) LowerGrandOnly added in v0.12.0

func (s *CrosstabSpec) LowerGrandOnly(src *Request) *Request

LowerGrandOnly builds the no-grouper request used to recompute the grand-total margin. Equivalent to a plain aggregation over the filter-passing record set.

func (*CrosstabSpec) LowerRowsOnly added in v0.12.0

func (s *CrosstabSpec) LowerRowsOnly(src *Request) *Request

LowerRowsOnly builds the rows-only request used to recompute row margins. Cohort, Filterers, and Labels carry through; the cell aggregation rides verbatim. Cell may be nil only when the caller has already validated the spec — production callers always set it.

func (*CrosstabSpec) LowerToGroupedRequest added in v0.12.0

func (s *CrosstabSpec) LowerToGroupedRequest(src *Request) *Request

LowerToGroupedRequest builds the equivalent grouped Process request that produces the long-form cell values for this crosstab. The Cohort, Filterers, Labels, and feature slots on src are carried through verbatim; the crosstab section itself is cleared on the result so downstream processors do not loop. The returned Request shares no pointer state with src.Crosstab — modifying the returned slice is safe even when src is concurrently in use.

func (*CrosstabSpec) NeedsColumnMargin added in v0.12.0

func (s *CrosstabSpec) NeedsColumnMargin() bool

NeedsColumnMargin reports whether the column-margin vector is required.

func (*CrosstabSpec) NeedsGrandMargin added in v0.12.0

func (s *CrosstabSpec) NeedsGrandMargin() bool

NeedsGrandMargin reports whether the grand-total margin is required.

func (*CrosstabSpec) NeedsRowMargin added in v0.12.0

func (s *CrosstabSpec) NeedsRowMargin() bool

NeedsRowMargin reports whether the row-margin vector is required either for display or to support the configured normalization mode.

func (*CrosstabSpec) NormalizeLevelOrLeaf added in v0.14.0

func (s *CrosstabSpec) NormalizeLevelOrLeaf(axisLen int) int

NormalizeLevelOrLeaf returns the configured normalize depth clamped to a valid axis position. axisLen is the count of groupers on the relevant axis (rows when normalize=row, columns when normalize=column). When NormalizeLevel is nil, negative, or >= axisLen, the leaf depth (axisLen-1) is returned. An axisLen of 0 yields 0; callers must guard against the empty-axis case independently (the validator does).

func (*CrosstabSpec) NormalizeOrDefault added in v0.12.0

func (s *CrosstabSpec) NormalizeOrDefault() CrosstabNormalize

NormalizeOrDefault returns the configured normalization mode, defaulting to "none" on the zero value.

func (*CrosstabSpec) ShapeOrDefault added in v0.12.0

func (s *CrosstabSpec) ShapeOrDefault() CrosstabShape

ShapeOrDefault returns the configured shape, defaulting to "matrix" on the zero value (matches the canonical handoff §2 default).

type FacetDiscrete added in v0.7.0

type FacetDiscrete struct {
	// Values is the per-value count list, sorted descending by count
	// (ties broken ascending by value-string for determinism).
	Values []FacetValueCount `json:"values"`

	// DistinctCount is the total distinct non-null values seen. May
	// exceed len(Values) when DiscreteTopK was set.
	DistinctCount int64 `json:"distinct_count"`

	// TruncatedAt is the count of values dropped by DiscreteTopK
	// truncation. Zero means no cap was applied.
	TruncatedAt int `json:"truncated_at,omitempty"`
}

FacetDiscrete summarises a categorical/boolean field.

type FacetField added in v0.7.0

type FacetField struct {
	// Kind is "discrete" | "numeric".
	Kind string `json:"kind"`

	// TypeName is the field's Pulse type as a string ("u8", "f64",
	// "categorical_u16", ...).
	TypeName string `json:"type_name"`

	// Description is the field's schema description.
	Description string `json:"description,omitempty"`

	// NullCount is the number of filtered records where this field was
	// null.
	NullCount int64 `json:"null_count"`

	// Discrete is non-nil when Kind == "discrete".
	Discrete *FacetDiscrete `json:"discrete,omitempty"`

	// Numeric is non-nil when Kind == "numeric".
	Numeric *FacetNumeric `json:"numeric,omitempty"`
}

FacetField wraps either a discrete or numeric summary.

type FacetHistogram added in v0.7.0

type FacetHistogram struct {
	Min  float64 `json:"min"`
	Max  float64 `json:"max"`
	Bins []int64 `json:"bins"`
}

FacetHistogram is a fixed-width binning of a numeric field. Bins is a slice of length HistogramBins; bin i covers [Min + i*step, Min + (i+1)*step) where step = (Max-Min)/HistogramBins. Values that fall on Max land in the last bin.

type FacetNumeric added in v0.7.0

type FacetNumeric struct {
	Min         float64            `json:"min"`
	Max         float64            `json:"max"`
	Mean        float64            `json:"mean"`
	StdDev      float64            `json:"stddev"`
	Count       int64              `json:"count"`
	Sum         float64            `json:"sum"`
	Percentiles map[string]float64 `json:"percentiles,omitempty"`
	Histogram   *FacetHistogram    `json:"histogram,omitempty"`
}

FacetNumeric summarises a numeric field. Min, Max, Mean, StdDev, Sum, and Count are always populated; Percentiles and Histogram are populated when the request asked for them.

type FacetRequest added in v0.7.0

type FacetRequest struct {
	// Cohort selects the .pulse file. Same shape as Request.Cohort.
	Cohort *Cohort `json:"cohort,omitempty"`

	// Fields is the explicit list of fields to summarise. Empty is an
	// error.
	Fields []string `json:"fields"`

	// Filterers optionally narrow the record set before accumulation.
	// Same semantics as Request.Filterers.
	Filterers []*Filterer `json:"filterers,omitempty"`

	// AdditiveFields lists fields for which the engine computes
	// "contribution" counts — accumulated independently of the matching
	// base filter, with the named field's own predicates stripped so all
	// values are counted equally. Empty = no additive analysis.
	AdditiveFields []string `json:"additive_fields,omitempty"`

	// DiscreteTopK caps the number of distinct values returned per
	// discrete field. Zero = no cap (return all). Capped fields carry a
	// TruncatedAt count in the response so callers know what was dropped.
	DiscreteTopK int `json:"discrete_top_k,omitempty"`

	// NumericPercentiles lists percentile points (strictly in (0, 1)) to
	// compute for each numeric field. Nil/empty = none. Adding any
	// percentile forces the buffered execution path.
	NumericPercentiles []float64 `json:"numeric_percentiles,omitempty"`

	// IncludeHistogram, when true, computes a fixed-width histogram for
	// each numeric field with HistogramBins buckets. Defaults to 20 bins
	// when bins unset and IncludeHistogram=true. Streaming-compatible
	// only when HistogramRange is supplied.
	IncludeHistogram bool `json:"include_histogram,omitempty"`

	// HistogramBins is the bucket count when IncludeHistogram is true.
	// Defaults to 20 when zero. Capped at 256.
	HistogramBins int `json:"histogram_bins,omitempty"`

	// HistogramRange supplies the [min, max] bounds for fixed-width
	// histogram binning. Required when IncludeHistogram is true. The two
	// values must satisfy min < max.
	HistogramRange [2]float64 `json:"histogram_range,omitempty"`

	// Labels rewrites or augments per-value keys in the response
	// using embedder-registered label tables. Each binding's Field
	// must appear in Fields and reference a categorical column;
	// numeric fields ignore labels. See types.LabelBinding for
	// semantics.
	Labels []*LabelBinding `json:"labels,omitempty"`

	// Overlays is the list of FACET-host overlay-layer specifications
	// that decorate the primary FacetResult with derived projections —
	// per-value population-comparison indices (OVERLAY_INDEX_VS_POP),
	// z-scores (OVERLAY_ZSCORE_VS_POP), goodness-of-fit χ² statistics
	// (OVERLAY_CHISQ_VS_POP), and Kolmogorov-Smirnov distances
	// (OVERLAY_KS_VS_POP). Each spec produces one OverlayLayer in
	// FacetResult.Overlays in matching order. Per-kind semantics,
	// required Ref fields, and host-field selection live in the
	// overlay catalog (see types/overlay.go) and skills/overlay-system.md.
	// The four FACET-host kinds ship today; the host-field the overlay
	// decorates is read from OverlaySpec.Params["field"] (which MUST
	// reference one of FacetRequest.Fields) — when the FacetRequest
	// declares exactly one Field the slot may be omitted.
	Overlays []OverlaySpec `json:"overlays,omitempty"`
}

FacetRequest specifies a multi-field rich facet computation. It is the input shape consumed by pulse.FacetSchema. The simpler single-field pulse.Facet entry point accepts only (path, field) and returns distinct values; FacetSchema returns per-value counts, null tallies, numeric statistics, optional percentiles, optional histograms, and optional "additive" contribution counts.

func (*FacetRequest) Hash added in v0.11.0

func (r *FacetRequest) Hash() string

Hash returns the canonical content hash of the FacetRequest.

type FacetResult added in v0.7.0

type FacetResult struct {
	// Fields maps each requested field name to its summary.
	Fields map[string]*FacetField `json:"fields"`

	// Additive maps each AdditiveFields entry to its independent
	// contribution counts. Same FacetField shape; values reflect counts
	// with the field's own filter predicates stripped.
	Additive map[string]*FacetField `json:"additive,omitempty"`

	// TotalRecords is the cohort's header record count (unfiltered).
	TotalRecords int64 `json:"total_records"`

	// FilteredRecords is the count of records passing Filterers.
	FilteredRecords int64 `json:"filtered_records"`

	// Warnings carry per-field diagnostics (top-K truncation, etc.).
	Warnings []string `json:"warnings,omitempty"`

	// Overlays carries the executed FACET-host overlay layers, one
	// entry per FacetRequest.Overlays spec in matching order. Each
	// layer holds its derived payload (scalar / series) plus optional
	// renderer-friendly summary metadata. Omitted entirely when the
	// originating FacetRequest had no overlays — the no-overlay shape
	// is byte-identical to the overlay-free FacetResult JSON output.
	Overlays []OverlayLayer `json:"overlays,omitempty"`
}

FacetResult is the response shape returned by pulse.FacetSchema.

type FacetValueCount added in v0.7.0

type FacetValueCount struct {
	Value string `json:"value"`
	Count int64  `json:"count"`
}

FacetValueCount is one value/count tuple inside FacetDiscrete.Values.

type Feature added in v0.2.0

type Feature struct {
	// Type is the feature operator to perform.
	Type FeatureType `json:"type"`

	// Field is the source field name. Required by every operator except
	// FEAT_TRAIN_TEST_SPLIT (which reads no field by default — params may
	// optionally name a stratify field).
	Field string `json:"field,omitempty"`

	// Label is an output column name (single-output operators) or output
	// column prefix (multi-output operators). When empty, the operator
	// derives a default — typically "<TYPE>_<field>".
	Label string `json:"label,omitempty"`

	// Params holds operator-specific parameters as raw JSON. See the
	// feature-engineering skill for the per-operator schema.
	Params json.RawMessage `json:"params,omitempty"`
}

Feature defines a feature engineering operation. Features run pre-filter (before any FILTER_* predicate) and may produce one or more derived columns. Global-pass features (TARGET_ENCODE, FREQUENCY_ENCODE) require a stats sweep before per-row write; per-row features compute one row at a time.

type FeatureType added in v0.2.0

type FeatureType string

FeatureType identifies a specific ML feature engineering operator. Features run pre-filter and may emit one or more output columns.

const (
	FEAT_LOG              FeatureType = "FEAT_LOG"
	FEAT_SQRT             FeatureType = "FEAT_SQRT"
	FEAT_BUCKETIZE        FeatureType = "FEAT_BUCKETIZE"
	FEAT_ONE_HOT          FeatureType = "FEAT_ONE_HOT"
	FEAT_DATE_FEATURES    FeatureType = "FEAT_DATE_FEATURES"
	FEAT_FREQUENCY_ENCODE FeatureType = "FEAT_FREQUENCY_ENCODE"
	FEAT_TARGET_ENCODE    FeatureType = "FEAT_TARGET_ENCODE"
	FEAT_TRAIN_TEST_SPLIT FeatureType = "FEAT_TRAIN_TEST_SPLIT"
	FEAT_POLY             FeatureType = "FEAT_POLY"
)

func AllFeatureTypes added in v0.2.0

func AllFeatureTypes() []FeatureType

AllFeatureTypes returns every defined feature type in alphabetical order.

func (FeatureType) Streamable added in v0.2.0

func (t FeatureType) Streamable() bool

Streamable reports whether this feature type can run in the pre-pass+finalize+emit streaming pipeline (feature.StreamingComputer).

Source of truth is feature.IsStreamable(req.Features, schema) at runtime; this method mirrors the per-type capability used by predict.

type FileRequest

type FileRequest struct {
	// Filename is the name of the file.
	Filename string `json:"filename"`

	// DataDir is the directory containing the file.
	DataDir string `json:"data_dir,omitempty"`
}

FileRequest identifies a file for operations like inspect.

type FileResponse

type FileResponse struct {
	// Filename is the name of the file.
	Filename string `json:"filename"`

	// RecordCount is the number of records in the file.
	RecordCount int64 `json:"record_count"`

	// Fields is the list of field names in the file.
	Fields []string `json:"fields,omitempty"`
}

FileResponse describes a file's metadata.

type Filterer

type Filterer struct {
	// Type is the filter operation to perform.
	Type FiltererType `json:"type"`

	// Field is the name of the data field to filter on.
	// Not required for FILTER_EXPRESSION.
	Field string `json:"field,omitempty"`

	// Values is a list of values for include/exclude/range filters.
	Values []string `json:"values,omitempty"`

	// Expression is a runtime expression for FILTER_EXPRESSION type.
	Expression string `json:"expression,omitempty"`
}

Filterer defines a filter to apply to records before processing.

type FiltererComponents added in v0.20.0

type FiltererComponents struct {
	// Label mirrors the originating Filterer.Label so callers can join
	// the components entry back to its request slot.
	Label string `json:"label,omitempty"`

	// NIn is the number of records the filterer observed (post-prior-
	// stage, pre-this-filter).
	NIn int `json:"n_in"`

	// NOut is the number of records the filterer admitted to the next
	// stage (post-this-filter).
	NOut int `json:"n_out"`

	// NNullInput is the number of records the filterer dropped because
	// the source field was null.
	NNullInput int `json:"n_null_input"`
}

FiltererComponents carries per-filterer constituent-parts metadata — records in (post-prior-stage), records out (post-this-filter), and records dropped due to null input on the filter field. Slot identity rides on Label, which mirrors the originating Filterer.Label so consumers can join the components entry back to its request slot.

type FiltererType

type FiltererType string

FiltererType identifies a specific filter operation.

const (
	FILTER_INCLUDE    FiltererType = "FILTER_INCLUDE"
	FILTER_EXCLUDE    FiltererType = "FILTER_EXCLUDE"
	FILTER_RANGE      FiltererType = "FILTER_RANGE"
	FILTER_EXPRESSION FiltererType = "FILTER_EXPRESSION"

	// FILTER_NULL keeps or drops records based on null state of a field.
	// Mode is "is_null" (keep null-valued records) or "is_not_null" (keep
	// non-null records). Row-local; streamable.
	FILTER_NULL FiltererType = "FILTER_NULL"

	// FILTER_TRUE keeps records where Field is logically true. Default
	// (strict) mode requires Field to be packed_bool and keeps records
	// whose value is 1. Opt-in JavaScript-style truthiness coercion is
	// enabled with Values=["truthy"]; in that mode any field type is
	// accepted and the same rules JS uses for `Boolean(value)` apply
	// (0, NaN, empty string, null → falsy; everything else → truthy).
	// Row-local; streamable.
	FILTER_TRUE FiltererType = "FILTER_TRUE"

	// FILTER_FALSE keeps records where Field is logically false. Default
	// (strict) mode requires Field to be packed_bool and keeps records
	// whose value is 0. Opt-in JavaScript-style falsiness coercion is
	// enabled with Values=["truthy"]; in that mode any field type is
	// accepted and records whose value would coerce to falsy under
	// `Boolean(value)` are kept (0, NaN, empty string, null → kept).
	// Row-local; streamable.
	FILTER_FALSE FiltererType = "FILTER_FALSE"

	// FILTER_SET_CONTAINS_ANY keeps records whose set mask shares at
	// least one bit with the resolved query mask (row & q != 0).
	FILTER_SET_CONTAINS_ANY FiltererType = "FILTER_SET_CONTAINS_ANY"

	// FILTER_SET_CONTAINS_ALL keeps records whose set mask has every bit
	// in the resolved query mask set (row & q == q).
	FILTER_SET_CONTAINS_ALL FiltererType = "FILTER_SET_CONTAINS_ALL"

	// FILTER_SET_CONTAINS_NONE keeps records whose set mask shares no
	// bits with the resolved query mask (row & q == 0).
	FILTER_SET_CONTAINS_NONE FiltererType = "FILTER_SET_CONTAINS_NONE"

	// FILTER_SET_EQUALS keeps records whose set mask exactly equals the
	// resolved query mask.
	FILTER_SET_EQUALS FiltererType = "FILTER_SET_EQUALS"
)

func AllFiltererTypes

func AllFiltererTypes() []FiltererType

AllFiltererTypes returns all defined filterer types.

func (FiltererType) Streamable added in v0.2.0

func (t FiltererType) Streamable() bool

Streamable reports whether this filterer type evaluates per-row without looking at other rows. All registered filterers are row-local today.

type FrameSpec added in v0.2.0

type FrameSpec struct {
	Mode      string `json:"mode"`
	Preceding *int   `json:"preceding,omitempty"`
	Following *int   `json:"following,omitempty"`
}

FrameSpec specifies the window frame bounds. Mode is "rows" — only frame mode supported in v1. Preceding nil means UNBOUNDED PRECEDING; Following nil means UNBOUNDED FOLLOWING. Following==0 with Preceding==0 selects the current row only.

type Group

type Group struct {
	// Type is the grouping operation to perform.
	Type GroupType `json:"type"`

	// Field is the name of the data field to group by.
	Field string `json:"field"`

	// Interval is used by GROUP_ROUNDED and GROUP_RANGE to define the bucket width.
	Interval float64 `json:"interval,omitempty"`

	// Params holds type-specific configuration as raw JSON.
	// Used by group types that require additional parameters (e.g., GROUP_DATE).
	Params json.RawMessage `json:"params,omitempty"`

	// Include optionally restricts the grouper to a fixed set of bucket
	// keys. Rows whose computed key (or, for per-element groupers, each
	// individual key produced by a single row) is not present in this
	// list are skipped — identical to the null-key handling path, no
	// bucket is created for the rejected key. Supported by:
	//
	//   - GROUP_CATEGORY        — matches the categorical label string.
	//   - GROUP_SET_VALUE       — matches the sorted, pipe-joined
	//                             composite bucket key (e.g. "MC|VISA").
	//   - GROUP_SET_PER_ELEMENT — matches each fan-out label
	//                             independently; the row contributes
	//                             only to surviving labels.
	//
	// A non-empty Include is order-significant: output buckets emit in
	// the order values are listed here — across both the plain grouped
	// payload (Response.Data rows and Response.Components buckets) and
	// each crosstab axis independently (an axis without Include keeps
	// its prior alphabetical / dict-index order). Include values that
	// match zero records are still dropped, never emitted as empty
	// buckets.
	//
	// Empty / nil Include means "no inclusion filter" and preserves the
	// prior alphabetical bucket order (dict-index order for
	// GROUP_SET_PER_ELEMENT) — byte-identical to the pre-Include output.
	// Other grouper types ignore Include — predict / runtime gates flag
	// misuse.
	Include []string `json:"include,omitempty"`
}

Group defines a grouping operation to partition results.

type GroupType

type GroupType string

GroupType identifies a specific grouping operation.

const (
	GROUP_CATEGORY GroupType = "GROUP_CATEGORY"
	GROUP_ROUNDED  GroupType = "GROUP_ROUNDED"
	GROUP_RANGE    GroupType = "GROUP_RANGE"
	GROUP_QUANTILE GroupType = "GROUP_QUANTILE"
	GROUP_DATE     GroupType = "GROUP_DATE"

	// GROUP_SET_VALUE treats the set mask as an atomic value. Each
	// distinct mask is its own bucket; the bucket key is the sorted
	// pipe-delimited label list (e.g. "AMEX|VISA"). One row → one
	// bucket. Streamable + mergeable.
	GROUP_SET_VALUE GroupType = "GROUP_SET_VALUE"

	// GROUP_SET_PER_ELEMENT explodes the set: each row fans into one
	// bucket per set bit, keyed by the resolved dictionary label. One
	// row → N buckets (where N = popcount(mask)). Cardinality multiplies.
	// Streamable via the multi-key streaming hook (KeysForRow on
	// StreamingGrouper). This is the survey-friendly default
	// ("respondents per option").
	GROUP_SET_PER_ELEMENT GroupType = "GROUP_SET_PER_ELEMENT"
)

func AllGroupTypes

func AllGroupTypes() []GroupType

AllGroupTypes returns all defined group types.

func (GroupType) Mergeable added in v0.8.0

func (t GroupType) Mergeable() bool

Mergeable reports whether this group type's per-key state can be combined across partitions of the input. CATEGORY and RANGE (online) derive their key purely from the row's value so per-shard buckets merge by key-union; QUANTILE/DATE depend on the full set or a finalize-time bucketization that the parallel reducer cannot replicate piecewise. ROUNDED could be mergeable in principle but is deferred — the parallel orchestrator only opts in on combinations we exercise in goldens today.

func (GroupType) Streamable added in v0.2.0

func (t GroupType) Streamable() bool

Streamable reports whether this group type can emit groups before the input is exhausted. CATEGORY/ROUNDED/RANGE bucket per row; QUANTILE/DATE require finalize-time work over the full set.

The streaming Process path does not currently emit grouped output even for streamable groupers — Request.Streamable returns false whenever groups are present. The method is wired through so a future grouped streaming iterator can flip the gate without re-deriving the rule.

type GrouperComponents added in v0.20.0

type GrouperComponents struct {
	// Field mirrors the originating Group.Field so callers can join
	// the components entry back to its request slot.
	Field string `json:"field,omitempty"`

	// TotalN is the number of records partitioned by the grouper
	// (post-filter). Universal floor — emitted by every grouper.
	// Zero-valued ints stay absent from JSON via omitempty so a
	// grouper entry with only operator keys does not leak a noisy
	// `total_n: 0` onto the wire.
	TotalN int `json:"total_n,omitempty"`

	// NNull is the number of records that landed in a null / skip
	// path (null field value, empty set, missing key). Universal
	// floor — emitted by every grouper. Zero-valued ints stay absent
	// from JSON via omitempty.
	NNull int `json:"n_null,omitempty"`

	// Operator carries the per-grouper schema-declared keys (bucket
	// edges, dict mappings, range_min / range_max, etc.). Key set is
	// governed by the operator's ComponentSchema declaration in
	// descriptor/capabilities_groupers.go.
	Operator map[string]any `json:"operator,omitempty"`

	// Label mirrors the originating Group.Label so callers can join
	// the components entry back to its request slot when multiple
	// groupers share the same Field.
	Label string `json:"label,omitempty"`
}

GrouperComponents carries per-grouper constituent-parts metadata — bucket inventory, edge values, dictionary mappings. The universal floor — TotalN (records partitioned, post-filter) and NNull (records that landed in a null / skip path) — is a typed field on every entry; per-operator keys (bucket arrays, edges, dict size, granularity, etc.) ride inside Operator. Slot identity rides on Field, which mirrors the originating Group.Field, plus Label, which mirrors the originating Group.Label when present so consumers can join the components entry back to its request slot without index arithmetic.

For single-key groupers (CATEGORY / RANGE / ROUNDED / QUANTILE / DATE / SET_VALUE), TotalN equals the sum of bucket counts. For multi-key streaming groupers (GROUP_SET_PER_ELEMENT), a single record may contribute to multiple buckets, so the sum of bucket counts will exceed TotalN — TotalN remains row count, not emission count.

type JoinSpec added in v0.10.0

type JoinSpec struct {
	// Right is the cohort path for the right side. Single-file `.pulse`,
	// shard archive, or `archive#shard.pulse` anchor — same dispatch
	// rules as the primary cohort path.
	Right string `json:"right"`

	// Kind selects the join semantic. "inner" (the only kind in v1)
	// emits one joined record per matched (left, right) pair. The
	// reserved kinds "left", "outer", "anti" will land once the null
	// bitmap correctness path is fully wired.
	Kind string `json:"kind,omitempty"`

	// On lists the equi-join key pairs. At least one entry is required.
	// Multiple entries are AND-ed (composite-key join). Field types
	// must match between left and right; mismatches surface
	// PULSE_JOIN_TYPE_MISMATCH.
	On []OnPair `json:"on"`

	// As is an optional prefix prepended to every right-side field's
	// name in the joined schema. Lets callers disambiguate columns
	// when both sides carry the same field name. Empty == no prefix.
	As string `json:"as,omitempty"`
}

JoinSpec describes a single hash-join leg attached to a Request. Each spec joins the request's primary cohort against a named right- side cohort path. The first cut implements inner join only; "left"/"outer"/"anti" kinds are reserved and return PULSE_JOIN_KIND_NOT_IMPLEMENTED.

Cardinality: at most one JoinSpec per Request in v1. Multi-join chains will land when the orchestrator gains a per-join intermediate state machine.

type LabelBinding added in v0.10.1

type LabelBinding struct {
	Field string    `json:"field"`
	Table string    `json:"table"`
	Mode  LabelMode `json:"mode,omitempty"`
}

LabelBinding requests that an output surface render the named categorical field through the named label table.

Field must reference a categorical_u8 / categorical_u16 / categorical_u32 schema field. Table must reference a label table registered on pulse.Options.Extensions.LabelTables. Mode picks between "replace" (rewrite the field's value) and "augment" (emit a sibling "<field>_label" field).

Duplicate bindings for the same field within a single request produce PULSE_LABEL_DUPLICATE_BINDING.

func (*LabelBinding) AugmentFieldName added in v0.10.1

func (b *LabelBinding) AugmentFieldName() string

AugmentFieldName is the fixed sibling-column name used when Mode is LabelModeAugment. The suffix is not overridable; collisions surface as PULSE_LABEL_FIELD_COLLISION at validation time.

func (*LabelBinding) LabelModeOrDefault added in v0.10.1

func (b *LabelBinding) LabelModeOrDefault() LabelMode

LabelModeOrDefault returns the binding's mode falling back to LabelModeReplace when empty. Callers may use this when an empty JSON value should mean "the simplest behaviour."

type LabelMode added in v0.10.1

type LabelMode string

LabelMode selects how a LabelBinding affects output rendering.

Replace: the binding's field is rewritten to the label string in every output surface (row values, aggregation map keys, facet counts, exported cells). When two source values map to the same label the output disambiguates with the source value appended in parentheses (e.g. "United States (US)") and emits a PULSE_LABEL_COLLISION warning.

Augment: the binding's field is left untouched; a sibling string field named "<field>_label" is added to the output schema and populated with the label. Collision with an existing schema field name surfaces PULSE_LABEL_FIELD_COLLISION at validation time.

Labels are output-only: filter expressions, formula attributes, sort keys, and group keys continue to operate on the raw categorical value. Labels never influence what records pass through the pipeline.

const (
	LabelModeReplace LabelMode = "replace"
	LabelModeAugment LabelMode = "augment"
)

type MarginAxis added in v0.19.0

type MarginAxis string

MarginAxis names which margin family an OverlayMarginRef targets. Mirrors the AxisKey conventions used by CrosstabSpec.

const (
	// MarginAxisRow targets the row-margin vector (Σ over columns per
	// row key).
	MarginAxisRow MarginAxis = "row"

	// MarginAxisColumn targets the column-margin vector (Σ over rows
	// per column key).
	MarginAxisColumn MarginAxis = "column"

	// MarginAxisGrand targets the grand-total margin (Σ over every
	// filter-passing row).
	MarginAxisGrand MarginAxis = "grand"
)

type MarginReducibility added in v0.12.0

type MarginReducibility string

MarginReducibility classifies how a crosstab margin can be derived from per-cell aggregations. See AggregationType.MarginReducibility.

const (
	// MarginSummable means the margin equals the sum of cell values
	// (count, sum, null_count, distinct_count, frequency).
	MarginSummable MarginReducibility = "summable"
	// MarginMeanReducible means the margin is derivable only when each
	// cell also carries its observation count (average, ratio).
	MarginMeanReducible MarginReducibility = "mean_reducible"
	// MarginRecompute means the margin cannot be derived from cells and
	// must be recomputed over the raw filter-passing rows (median,
	// stddev, percentile, mode, ...).
	MarginRecompute MarginReducibility = "recompute"
)

type MatrixCell added in v0.12.0

type MatrixCell struct {
	Value   any  `json:"value,omitempty"`
	Present bool `json:"present"`
}

MatrixCell is one entry in the dense cell matrix. Present=false marks a structurally missing cell (no underlying record matched the row × column tuple), which downstream consumers must render as null / empty rather than zero (the AGG_AVERAGE of an empty set is undefined, not zero).

Value is a scalar/rich union. Scalar aggregators populate a float64 (existing JSON shape preserved byte-for-byte through encoding/json). RichAggregator implementations populate the structured payload directly: AGG_SET_FREQUENCY emits map[string]int (per-label row counts), AGG_SET_UNION / AGG_SET_INTERSECTION emit []string (sorted dictionary labels). AGG_WELFORD is the named carve-out — its rich WelfordTriple payload does NOT ride MatrixCell.Value (the carrier is Response.Components.Crosstab.CellComponents[r][c]'s `{mean, variance, n}` triple instead, populated by the orchestrator's MetaAggregator pass); the cell value falls back to the scalar mean (matching welfordAggregator.Aggregate / Finalize) so downstream renderers see a plain float64. Normalize modes (row/column/total) are only defined for scalar cells; the Crosstab validator rejects them paired with map-valued aggregators (see types.AggregationType.MapValued).

func (MatrixCell) Scalar added in v0.15.0

func (c MatrixCell) Scalar() float64

Scalar returns the cell's scalar form for callers that expect a numeric value. Returns 0 when the cell is absent OR when Value is non-scalar (rich payload). Use Value directly with a type switch to distinguish scalar / map / slice payloads.

type MatrixPayload added in v0.12.0

type MatrixPayload struct {
	// RowHeader names the row-axis grouper fields and types in order.
	RowHeader AxisHeader `json:"row_header"`
	// ColumnHeader names the column-axis grouper fields and types in order.
	ColumnHeader AxisHeader `json:"column_header"`

	// RowKeys is the deterministic, sorted list of row tuples.
	RowKeys []AxisKey `json:"row_keys"`
	// ColumnKeys is the deterministic, sorted list of column tuples.
	ColumnKeys []AxisKey `json:"column_keys"`

	// Cells is a 2-D array indexed [row_index][column_index]. Missing
	// cells set MatrixCell.Present=false; present cells set Value to the
	// (possibly normalized) cell aggregation.
	Cells [][]MatrixCell `json:"cells"`

	// RowMargins carries the per-row margin in RowKeys order. Empty when
	// row margins were not requested and not required by normalization.
	RowMargins []MatrixCell `json:"row_margins,omitempty"`
	// ColumnMargins carries the per-column margin in ColumnKeys order.
	ColumnMargins []MatrixCell `json:"column_margins,omitempty"`
	// GrandTotal is the grand-total margin. Present=false when neither
	// requested nor required by normalization.
	GrandTotal MatrixCell `json:"grand_total"`

	// CellLabel echoes the underlying aggregation's effective label so
	// downstream consumers can colorize/sort by it without inspecting
	// the originating Request.
	CellLabel string `json:"cell_label"`

	// NormalizeApplied echoes the normalization mode that was actually
	// applied. Mirrors CrosstabSpec.Normalize after defaulting.
	NormalizeApplied CrosstabNormalize `json:"normalize_applied"`
}

MatrixPayload is the structured matrix-shape response carried on Response.Crosstab. Includes enough information for a downstream renderer (Prism heatmap, terminal grid, etc.) to lay out the table without re-deriving axis structure from the long-form result.

type OnPair added in v0.10.0

type OnPair struct {
	LeftField  string `json:"left_field"`
	RightField string `json:"right_field"`
}

OnPair describes one equi-join key relationship: left.LeftField equals right.RightField.

type OrderKey added in v0.2.0

type OrderKey struct {
	Field string `json:"field"`
	Desc  bool   `json:"desc,omitempty"`
}

OrderKey specifies an ordering key for a window's ORDER BY clause.

type Output

type Output struct {
	// Format is the output format (e.g., "json", "csv").
	Format string `json:"format"`

	// Filename is an optional output file path.
	Filename string `json:"filename,omitempty"`

	// Pretty enables indented/formatted output.
	Pretty bool `json:"pretty,omitempty"`

	// IncludeNil controls whether nil/null values appear in output.
	IncludeNil bool `json:"include_nil,omitempty"`
}

Output configures how processing results are formatted.

type OverlayBaselineIndexRef added in v0.19.0

type OverlayBaselineIndexRef struct {
	// Row names the baseline row-axis tuple as a sorted, axis-ordered
	// list of dictionary keys for the MATRIX-host arm. Empty list
	// means "use the grand total". Reserved for the future "vs
	// designated cell" Crosstab overlay family — not consumed by any
	// shipping handler today.
	Row []string `json:"row,omitempty"`

	// Column names the baseline column-axis tuple for the MATRIX-host
	// arm. Empty list means "use the grand total". Reserved.
	Column []string `json:"column,omitempty"`

	// Position pins a 0-based ordinal on the host's ordered axis for
	// the SERIES-host arm — typically a `GROUP_DATE`-keyed grouped
	// Process result whose key order is the chronological ordering the
	// orchestrator baked in at finalize. Resolved at runtime via
	// `processing.ResolveBaselineIndex(host, ref)`; negative values
	// and values `>= len(series keys)` fail at predict time
	// (`descriptor.ValidateOverlays`) and runtime with
	// `PULSE_OVERLAY_REF_UNKNOWN` carrying `{baseline_index,
	// series_length}` Details. Consumed by INDEX_VS_BASELINE,
	// DELTA_VS_BASELINE, INDEX_VS_ROLLING_MEAN, and YOY.
	Position int `json:"position,omitempty"`
}

OverlayBaselineIndexRef discriminates two overlapping baseline- reference shapes:

  • MATRIX-host crosstab-cell coordinate (Row + Column). Each slot names a baseline coordinate as a sorted, axis-ordered list of dictionary keys. Empty lists mean "use the grand total". Not populated by any shipping kind today and consumed only by the future "vs designated cell" Crosstab overlay family.

  • SERIES-host ordered-axis positional baseline (Position, the windowed-Process arm). Position pins a 0-based ordinal against the host's ordered series — typically a GROUP_DATE-keyed grouped Process result whose key order is the chronological ordering the orchestrator baked in at finalize time. The `OVERLAY_INDEX_VS_BASELINE`, `OVERLAY_DELTA_VS_BASELINE`, `OVERLAY_INDEX_VS_ROLLING_MEAN`, and `OVERLAY_YOY` handlers resolve a single host value at this ordinal via `processing.ResolveBaselineIndex` and compare every other ordered-axis point against it. Negative or out-of-range values are rejected at predict time (`descriptor.ValidateOverlays`) and at runtime (`processing.ResolveBaselineIndex`) with `PULSE_OVERLAY_REF_UNKNOWN` plus a `{baseline_index, series_length}` Details map.

Exactly one of (Row + Column) or Position is meaningfully populated per spec — the union is host-shape-disambiguated rather than pointer-discriminated. The two arms never collide in practice (Row + Column resolve on a MATRIX host; Position resolves on a SERIES host) so a single struct cleanly carries both reservations until the future MATRIX baseline-cell family ships.

type OverlayKind added in v0.19.0

type OverlayKind string

OverlayKind identifies one entry in the overlay catalog. On the wire every kind is SCREAMING_SNAKE and prefixed `OVERLAY_`; the exported Go identifier uses mixed case.

const (
	// OverlayKindChiSqMatrix emits a whole-matrix χ² independence test
	// across the row × column contingency table built from the host
	// crosstab cells. MATRIX scope over a MATRIX (crosstab) host with
	// SCALAR payload — the layer carries a single chi-square statistic
	// plus its degrees of freedom and the corresponding p-value, all
	// surfaced through OverlaySummary (Statistic / PValue / Parameters
	// {"df"}). First inferential overlay kind and first SCALAR-shape
	// Crosstab overlay; establishes the SCALAR payload plumbing pattern
	// the remaining inferential kinds and the post-test family reuse.
	//
	// Math:
	//
	//	expected[r,c] = row_margin[r] * col_margin[c] / grand_total
	//	chisq         = Σ_{r,c} (observed[r,c] - expected[r,c])² / expected[r,c]
	//	df            = (rows - 1) * (cols - 1)
	//	p_value       = 1 - chi2_cdf(chisq, df)
	//
	// Implementation reuses the χ² survival helper that backs TEST_CHISQ
	// (processing/test_stat.go chiSquareSurvival) so the overlay and the
	// row-test surface produce identical p-values for the same
	// contingency.
	//
	// Absent-cell policy: a structurally absent host cell (Present=false)
	// is treated as an observed count of 0 — the matrix shape stays
	// rectangular, the row / column / grand margins continue to drive
	// the expected-count recurrence, and an absent observation does not
	// invent a count. The handler documents the policy alongside the
	// runtime dispatch.
	//
	// Low-expected-count warning: when any expected[r,c] < 5 the handler
	// emits a single PULSE_OVERLAY_EXPECTED_LOW warning (canonical χ²
	// low-count heuristic; mirrors PULSE_TEST_EXPECTED_COUNT_TOO_LOW on
	// the TEST_CHISQ surface). The canonical
	// errors.PULSE_OVERLAY_EXPECTED_LOW constant is the source of truth.
	//
	// Scope is MATRIX (not CELL) because the test is whole-table; the
	// validator rejects any other scope. The Ref union is left empty —
	// the test is implicit-margin (uses the host's row / column / grand
	// margins inline), so callers supplying a Ref.Margin (or any other
	// ref-family pointer) get PULSE_OVERLAY_REF_INCOMPATIBLE_WITH_SHAPE.
	//
	// Inherently buffered because the host crosstab path is always
	// buffered (margins recomputed from raw rows).
	OverlayKindChiSqMatrix OverlayKind = "OVERLAY_CHISQ_MATRIX"

	// OverlayKindChiSqRow emits a per-row χ² goodness-of-fit test
	// across the host crosstab's row × column contingency table. ROW
	// scope over a MATRIX (crosstab) host with SERIES payload — one
	// per-row entry carrying the row's χ² statistic, degrees of
	// freedom (cols - 1), and p-value via OverlaySummary
	// {Statistic, PValue, Parameters{"df"}}. First SERIES-shape
	// Crosstab overlay; establishes the SeriesPayload entries
	// plumbing pattern that the remaining series families reuse.
	//
	// Math (per row r):
	//
	//	observed[c] = host.Cell(r, c)
	//	expected[c] = row_margin[r] * col_margin[c] / grand_total
	//	chisq_r    = Σ_c (observed[c] - expected[c])² / expected[c]
	//	df          = cols - 1
	//	p_value     = chi2_survival(chisq_r, df)   // = 1 - chi2_cdf
	//
	// Tests whether each row's observed column distribution differs
	// from the expected distribution derived from column margins under
	// independence. The χ² survival helper (chiSquareSurvival) is the
	// same helper that backs TEST_CHISQ and the CHISQ_MATRIX overlay —
	// overlay surfaces produce identical p-values for the same
	// contingency.
	//
	// Absent-cell policy: a structurally absent host cell (Present=
	// false) is treated as an observed count of 0 (matches the
	// CHISQ_MATRIX policy). The per-row recurrence still consumes
	// every column slot; an absent observation does not collapse the
	// column count.
	//
	// Low-expected-count warning: when any expected[c] < 5 in row r
	// the handler emits ONE PULSE_OVERLAY_EXPECTED_LOW warning per
	// offending row (not per cell — the row is the diagnostic unit
	// for goodness-of-fit). Canonical errors.PULSE_OVERLAY_EXPECTED_LOW
	// constant.
	//
	// Scope is ROW (not CELL or MATRIX) because each row's test is
	// independent — the validator rejects any other scope. The Ref
	// union is left empty — the test is implicit-margin (uses the
	// host's row / column / grand margins inline), so callers
	// supplying a Ref.Margin (or any other ref-family pointer) get
	// PULSE_OVERLAY_REF_INCOMPATIBLE_WITH_SHAPE.
	//
	// Inherently buffered because the host crosstab path is always
	// buffered (margins recomputed from raw rows).
	OverlayKindChiSqRow OverlayKind = "OVERLAY_CHISQ_ROW"

	// OverlayKindChiSqCol emits a per-column χ² goodness-of-fit test
	// across the host crosstab's row × column contingency table. COLUMN
	// scope over a MATRIX (crosstab) host with SERIES payload — one
	// per-column entry carrying the column's χ² statistic, degrees of
	// freedom (rows - 1), and p-value via OverlaySummary
	// {Statistic, PValue, Parameters{"df"}}. Mechanical column-axis twin
	// of OVERLAY_CHISQ_ROW; mirrors the SeriesPayload entries plumbing
	// pattern (Entries[i].Key == host ColumnKeys[i] element-for-element).
	//
	// Math (per column c):
	//
	//	observed[r] = host.Cell(r, c)
	//	expected[r] = row_margin[r] * col_margin[c] / grand_total
	//	chisq_c    = Σ_r (observed[r] - expected[r])² / expected[r]
	//	df          = rows - 1
	//	p_value     = chi2_survival(chisq_c, df)   // = 1 - chi2_cdf
	//
	// Tests whether each column's observed row distribution differs
	// from the expected distribution derived from row margins under
	// independence. The χ² survival helper (chiSquareSurvival) is the
	// same helper that backs TEST_CHISQ and the CHISQ_MATRIX / CHISQ_ROW
	// overlays — overlay surfaces produce identical p-values for the
	// same contingency.
	//
	// Absent-cell policy: a structurally absent host cell (Present=
	// false) is treated as an observed count of 0 (matches the
	// CHISQ_MATRIX / CHISQ_ROW policy). The per-column recurrence still
	// consumes every row slot; an absent observation does not collapse
	// the row count.
	//
	// Low-expected-count warning: when any expected[r] in column c is
	// below 5 the handler emits ONE PULSE_OVERLAY_EXPECTED_LOW warning
	// per offending column (not per cell — the column is the diagnostic
	// unit for goodness-of-fit). Canonical
	// errors.PULSE_OVERLAY_EXPECTED_LOW constant.
	//
	// Scope is COLUMN (not CELL, ROW, or MATRIX) because each column's
	// test is independent — the validator rejects any other scope. The
	// Ref union is left empty — the test is implicit-margin (uses the
	// host's row / column / grand margins inline), so callers supplying
	// a Ref.Margin (or any other ref-family pointer) get
	// PULSE_OVERLAY_REF_INCOMPATIBLE_WITH_SHAPE.
	//
	// Inherently buffered because the host crosstab path is always
	// buffered (margins recomputed from raw rows).
	OverlayKindChiSqCol OverlayKind = "OVERLAY_CHISQ_COL"

	// OverlayKindChiSqVsPop emits a single scalar χ² goodness-of-fit
	// statistic + p-value comparing the host Facet subset distribution
	// against the resolved population distribution. GROUP scope over a
	// FACET (FacetSchema) host with SCALAR payload — the layer carries
	// the chi-square statistic plus degrees of freedom and the
	// corresponding p-value via OverlaySummary{Statistic, PValue,
	// Parameters{"df"}}. A FACET-host overlay (siblings:
	// OVERLAY_INDEX_VS_POP, OVERLAY_ZSCORE_VS_POP) and the first
	// inferential FACET-host kind. Pairs with the
	// MATRIX-host CHISQ family (CHISQ_MATRIX / CHISQ_ROW / CHISQ_COL) as
	// the canonical χ² family — the viz developer renders the SCALAR
	// statistic as a goodness-of-fit badge near the facet header.
	//
	// Math:
	//
	//	subset_N      = sum(host counts)              // discrete arm only
	//	pop_freq[v]   = FacetPopulationView.DiscreteFrequency(v)
	//	expected[v]   = pop_freq[v] * subset_N        // expected scaled to subset N
	//	observed[v]   = host.Discrete.Values[v].Count
	//	chisq         = Σ_v (observed[v] - expected[v])² / expected[v]
	//	df            = len(observed) - 1
	//	p_value       = chiSquareSurvival(chisq, df)  // = 1 - chi2_cdf
	//
	// Discrete arm only: χ² goodness-of-fit requires categorical buckets
	// to form the observed × expected contingency. A numeric host (no
	// discrete payload) emits zero entries and surfaces a coded
	// PROCESSING_INTERNAL error carrying
	// PULSE_OVERLAY_REF_INCOMPATIBLE_WITH_SHAPE — the predict-time
	// validator rejects numeric hosts before runtime, so this arm is
	// defense in depth.
	//
	// Reuses the χ² survival helper backing TEST_CHISQ and the
	// MATRIX-host CHISQ family (chiSquareSurvival in
	// processing/test_stat.go) so the overlay and the row-test surface
	// produce identical p-values for the same contingency.
	//
	// Streaming finalize hook: the handler runs at the FacetSchema
	// post-host-finalize entry — once the per-value `(value, count)` map
	// is folded into the FacetField.Discrete.Values slice the overlay
	// reads the already-finalized host distribution + the resolver's
	// population view to emit the SCALAR statistic. The handler depends
	// only on POST-FINALIZE state so running it against a streaming
	// Facet host vs a buffered one produces byte-identical output.
	//
	// Inferential — ALWAYS BUFFERED (per PRD §2 Non-Goals "Streaming
	// overlay path for inferential kinds"). The streamability row is
	// `false` regardless of host streamability — inferential overlays as
	// a family stay buffered until a streamable-test path is plumbed.
	// Distinct from the streamable INDEX_VS_POP / ZSCORE_VS_POP siblings
	// which emit per-value descriptive statistics.
	//
	// Absent-population path: when a host value is missing from the
	// population's discrete payload, pop_freq = 0 and expected = 0.
	// Goodness-of-fit math drops the term (mirrors TEST_CHISQ's "expected
	// > 0" guard) but the low-expected warning still fires.
	//
	// Low-expected-cell warning: when any expected count is below 5 the
	// handler emits ONE PULSE_OVERLAY_EXPECTED_LOW warning carrying the
	// count of low-expected categories and the offending minimum.
	// Canonical errors.PULSE_OVERLAY_EXPECTED_LOW constant — the same
	// warning shape the MATRIX-host CHISQ family + FISHER_EXACT_CELL
	// use (PRD FR-J1 — the warning code is shared with the crosstab
	// Fisher exact path).
	//
	// Degenerate inputs:
	//   - Empty host distribution (no values): the layer emits a SCALAR
	//     payload with NaN statistic + NaN p-value and a layer-level
	//     warning carrying PULSE_OVERLAY_REF_ZERO. No χ² test can be
	//     computed without observed counts.
	//   - subset_N == 0 (every observed count is zero): same as above —
	//     PULSE_OVERLAY_REF_ZERO + NaN statistic.
	//   - pop_freq[v] == 0 for every host value (population is empty):
	//     every expected count is zero, every term drops, statistic
	//     stays at 0 and df = 0. The handler emits NaN statistic + NaN
	//     p-value with PULSE_OVERLAY_REF_ZERO.
	//   - Single category (df = 0): chi-square is undefined; the handler
	//     emits NaN p-value alongside the statistic — descriptive only.
	//
	// Ref handling: `Ref.Population` MUST be populated (the cohort name
	// lives on `Ref.Population.Cohort`); any other ref-family pointer
	// (Margin / Sibling / BaselineIndex / Prior / RollingMean / YoY /
	// Stage / Slot) fires PULSE_OVERLAY_REF_INCOMPATIBLE_WITH_SHAPE at
	// predict time (per-kind validator lives in descriptor/overlay_facet.go). Mirrors
	// the INDEX_VS_POP / ZSCORE_VS_POP sibling-kind ref contract.
	//
	// Scope must be GROUP. `Level` / `Within` MUST be zero (population
	// comparison is a single-value lookup, not an axis prefix); non-zero
	// values fire PULSE_OVERLAY_LEVEL_OUT_OF_RANGE mirroring the
	// implicit-ref family.
	//
	// Renderer-facing shape: the viz developer reads `OverlayPayload.Scalar`
	// for the χ² statistic and `OverlaySummary` for the renderer
	// metadata. The recommended rendering is a goodness-of-fit badge
	// near the facet header carrying the (statistic, df, p-value) triple
	// — small p-value flags "the subset distribution diverges from the
	// population". The layer-level Baseline stays unset (inferential
	// overlays do not surface a ratio centerpoint).
	OverlayKindChiSqVsPop OverlayKind = "OVERLAY_CHISQ_VS_POP"

	// OverlayKindDeltaVsMargin emits a per-cell additive delta against
	// the matching axis margin: cell - margin. CELL-scoped over a MATRIX
	// (crosstab) host. Unlike INDEX_VS_MARGIN (a ratio) and the SHARE_OF_*
	// triad (each a ratio scaled to 1.0), DELTA_VS_MARGIN preserves the
	// host cell's units — a $-valued AGG_SUM cell minus a $-valued row
	// margin yields a $-valued deviation in the same currency. There is
	// no division and no Welford recurrence, so the handler never
	// surfaces PULSE_OVERLAY_REF_ZERO. Supports all three axes (row /
	// column / grand) — callers pick the axis explicitly via
	// Ref.Margin.Axis and the handler dispatches the matching margin
	// slot. Inherently buffered because the host crosstab path is always
	// buffered (margins recomputed from raw rows).
	OverlayKindDeltaVsMargin OverlayKind = "OVERLAY_DELTA_VS_MARGIN"

	// OverlayKindFisherExactCell emits a per-cell Fisher's exact two-
	// sided p-value computed against a 2×2 contingency table formed from
	// the host crosstab cell, its row margin, its column margin, and the
	// grand total. CELL-scoped over a MATRIX (crosstab) host. Closes the
	// inferential overlay catalog as the canonical low-count χ²
	// backstop — when any expected count in the 2×2 falls below 5 the
	// χ² approximation becomes unreliable and Fisher's exact is the
	// correct surface to compute the p-value.
	//
	// Per-cell 2×2 contingency (for cell at (rowIdx, colIdx)):
	//
	//	            col=c              col≠c
	//	  row=r     cell               row_margin - cell
	//	  row≠r    col_margin - cell   grand - row_margin - col_margin + cell
	//
	// All four cells of the 2×2 are non-negative because every margin is
	// recomputed by the buffered crosstab orchestrator from the same
	// filter-passing row set the cells were built from — the row total
	// dominates each individual row's cell, the column total dominates
	// each individual column's cell, and the grand total equals the sum
	// of row totals (= sum of column totals).
	//
	// Math: Fisher's exact two-sided p-value sums hypergeometric
	// probabilities P(X = x | row_margin, col_margin, grand_total) for
	// every feasible x in the marginal-constrained range whose log-
	// probability is at most logPObs (the observed table's log-prob).
	// Reuses logHypergeometric (processing/test_fisher.go) so the overlay
	// surface produces identical p-values to TEST_FISHER_EXACT for the
	// same 2×2.
	//
	// Output shape: MATRIX payload mirroring the host's RowKeys /
	// ColumnKeys / headers so renderers can lay the overlay on top of
	// the base matrix with the same header machinery as INDEX_VS_MARGIN.
	// Each present host cell becomes a MatrixCell whose Value is the
	// two-sided p-value as a float64. Missing host cells stay absent on
	// the overlay; cells with a missing row or column margin become
	// absent overlay cells (defense in depth — the buffered crosstab
	// orchestrator already populates margins before the overlay fold,
	// so this branch should not fire in practice).
	//
	// Absent-cell policy: a structurally absent host cell (Present=
	// false) stays absent on the overlay (mirrors the SHARE_OF_* triad
	// and INDEX_VS_MARGIN policy — an absent observation does not
	// invent a 2×2).
	//
	// Ref handling: implicit-margin (empty Ref accepted). Row + column
	// margins are resolved from the buffered crosstab host view's
	// MarginFor(Row/Col, ...) resolver. Explicit Ref-family pointers
	// (Margin / Sibling / BaselineIndex / Population / Stage / Slot)
	// fire PULSE_OVERLAY_REF_INCOMPATIBLE_WITH_SHAPE at predict time.
	// Mirrors the CHISQ_MATRIX / CHISQ_ROW / CHISQ_COL ref policy.
	//
	// Low expected-cell warning (Cochran rule): when ANY of the four
	// 2×2 expected counts {row_margin × col_margin / grand,
	// row_margin × (grand - col_margin) / grand,
	// (grand - row_margin) × col_margin / grand,
	// (grand - row_margin) × (grand - col_margin) / grand} is below 1,
	// OR when at least 20% of the four expected counts are below 5,
	// the handler emits one PULSE_OVERLAY_EXPECTED_LOW warning per
	// offending cell. Canonical errors.PULSE_OVERLAY_EXPECTED_LOW
	// constant. The warning is
	// advisory — Fisher's exact stays exact in the low-count regime and
	// the p-value is still emitted alongside.
	// The threshold runs on the OVERLAY itself (not the underlying χ²
	// surface) because Fisher's exact is the SOLUTION to the low-
	// expected-count problem — the warning's intent here is to flag the
	// CELLS where Fisher's exact (rather than the cheaper χ²
	// approximation) is structurally required.
	//
	// Degenerate inputs:
	//   - grand_total <= 0: layer emits absent cells everywhere
	//     (no 2×2 can be formed); single PULSE_OVERLAY_REF_ZERO warning
	//     surfaces the degenerate host.
	//   - row_margin <= 0 OR col_margin <= 0 OR row_margin >= grand
	//     OR col_margin >= grand for some cell: the 2×2 collapses to a
	//     degenerate shape (one of the four cells must be zero); the
	//     handler still emits a p-value of 1.0 (no rejection possible
	//     under the fully-degenerate hypergeometric) without a warning.
	//
	// Inherently buffered because the host crosstab path is always
	// buffered (margins recomputed from raw rows). PRD § 4.C FR-C2
	// calls out OVERLAY_FISHER_EXACT_CELL as the canonical low-count
	// contingency overlay closing the inferential family.
	OverlayKindFisherExactCell OverlayKind = "OVERLAY_FISHER_EXACT_CELL"

	// OverlayKindFormula emits a per-cell / per-entry / per-scalar
	// expression-driven projection computed from a caller-supplied
	// formula string evaluated against a per-host-shape variable
	// namespace. The expression syntax + evaluator is `expr-lang/expr`
	// — the same evaluator that backs `ATTR_FORMULA` and
	// `FILTER_EXPRESSION` so embedder `ExprFunctions` registered at
	// `pulse.New` time are reachable from FORMULA expressions with no
	// additional registration (the function surface widens; the
	// variable surface stays fixed per-host-shape).
	//
	// Authoring shape:
	//
	//	{
	//	  "kind":  "OVERLAY_FORMULA",
	//	  "scope": "cell",                                   // matrix host
	//	  "params": { "formula": "(cell - margin_grand) / sd_grand" }
	//	}
	//
	// Per-host-shape variable namespace (resolved in research note
	// `.planning/result-overlay-system/research/formula-namespace.md`
	// § 2 and bound by the per-shape binder):
	//
	//   - MATRIX host (Crosstab): `cell`, `margin_row`, `margin_col`,
	//     `margin_grand`, `sd_row`, `sd_col`, `sd_grand` (+ `ref_cell`
	//     when COMPOSE).
	//   - SERIES host (grouped Process / windowed Process / Facet
	//     discrete): `value`, `total`, `prior` (+ `baseline` opt-in via
	//     `Params["baseline_position"]`, + `ref_value` when COMPOSE).
	//   - SCALAR host (whole-matrix p-values, whole-chain SCALAR
	//     overlays, Compose-scalar): `value` (+ `ref` when COMPOSE).
	//   - Compose hosts add the dotted `slot.<label>.{cell|value}`
	//     namespace tied to `Request.Label`.
	//
	// Evaluator: compile-once / run-many — the handler compiles the
	// formula via `expr.Compile` ONCE per overlay spec (at handler
	// entry) against a prototype env carrying zeroed slots for the
	// per-shape variable namespace, then runs the program via
	// `expr.Run` per cell / entry / scalar with the per-iteration env
	// overwriting the prototype's value slots. Mirrors the
	// `processing.formulaAttribute.Row` compile / run pattern (single
	// allocation per overlay layer; O(1) writes per iteration).
	//
	// Predict-time identifier validation: predict
	// parses the formula via `expr-lang/expr/parser`, walks the AST
	// via `expr-lang/expr/ast` (the same packages
	// `processing.NeededFields` walks for `ATTR_FORMULA`), and rejects
	// every identifier not in the per-host-shape variable table or the
	// embedder `ExprFunctions` function set. Unknown identifiers fire
	// `PULSE_OVERLAY_FORMULA_INVALID_IDENT` carrying the offending
	// identifier + the per-shape variable enumerate-set.
	//
	// Failure modes:
	//
	//   - `PULSE_OVERLAY_FORMULA_PARSE_ERROR` — `expr.Compile` returned
	//     a parse error (syntax error in the formula string). Carries
	//     `{formula, parse_error}` Details. Surfaced at both predict
	//     and runtime — the runtime path defends in case the predict
	//     step was bypassed.
	//   - `PULSE_OVERLAY_FORMULA_TYPE_MISMATCH` — `expr.Run` returned a
	//     value whose type cannot be coerced to float64. The coercion
	//     accepts `float64` / `float32` / `int` / `int64` natively,
	//     widens `bool` to `0.0 / 1.0`, and rejects everything else.
	//     Carries `{returned_type, formula}` Details.
	//   - `PULSE_OVERLAY_FORMULA_INVALID_IDENT` — predict-time AST
	//     walk found an identifier not in the per-host-shape variable
	//     table or the function set. Carries `{ident, host_shape,
	//     available_vars}` Details.
	//
	// Buffered-only at v1 across every host shape — the variable
	// namespace depends on post-fold state (margins / totals / SDs are
	// not available mid-stream). The streamability row in
	// `types/overlay_streamability.go` stays `false` regardless of
	// host streamability. Future work may carve out a streamable
	// variant under a distinct kind constant without re-opening this
	// kind's contract; see research note § 6 for the rationale.
	//
	// Scope / Ref handling: the resolved host shape determines which
	// scope values are accepted (MATRIX hosts accept `cell`; SERIES
	// hosts accept `group`; SCALAR hosts accept `total`). Ref family
	// pointers are NOT required — the `ref_cell` / `ref_value` / `ref`
	// COMPOSE variables source from the slot's reference shape (the
	// per-shape binder resolves them). v1 ships in-Request
	// (`Request.Overlays`) and Compose (`ComposedRequest.Overlays`)
	// FORMULA surfaces only; whole-chain FORMULA is deferred
	// (the `stage.<id>.<var>` namespace is documented in the research
	// note for forward-compat).
	//
	// Embedder extensibility: `pulse.Options.Extensions.ExprFunctions`
	// merges into the FORMULA env. Functions widen the function
	// surface only — they do NOT widen the variable identifier surface.
	// Embedders that need new variables MUST register a custom kind
	// via `pulse.Options.Extensions.OverlayKinds` per the existing
	// extension-points policy (CLAUDE.md "Extension Points → Naming
	// policy"); FORMULA cannot widen its variable namespace from
	// outside.
	//
	// LookupTables are OUT of scope at v1 — `lookup(...)` is NOT
	// registered into the FORMULA env. Embedders who need a
	// lookup-driven overlay register the lookup as a custom
	// `ExprFunctions` entry instead (full control over the function
	// signature and predict treatment).
	OverlayKindFormula OverlayKind = "OVERLAY_FORMULA"

	// OverlayKindDeltaVsBaseline emits a per-point additive delta against a
	// single fixed positional baseline of an ordered SERIES (grouped
	// Process) host: `delta_i = point_value_i - baseline_value` where
	// `baseline_value = host.ValueAt(Ref.BaselineIndex.Position)`. GROUP
	// scope over a SERIES host with SERIES payload — one `SeriesEntry` per
	// host group key in host order, each carrying the delta on
	// `Summary.Statistic`. Absolute-difference sibling of
	// `OVERLAY_INDEX_VS_BASELINE`. Like its sibling it consumes the
	// `Ref.BaselineIndex.Position` arm of the OverlayBaselineIndexRef
	// discriminated union.
	//
	// Math (per host group ordinal `i`):
	//
	//	baseline_value = host.ValueAt(Ref.BaselineIndex.Position)
	//	if present[i]  ⇒ point - baseline
	//	if !present[i] ⇒ absent passthrough (no Statistic)
	//
	// Baseline resolution: the handler resolves the baseline value ONCE
	// up front via `processing.ResolveBaselineIndex(host,
	// Ref.BaselineIndex)`. Negative or out-of-range `Position` values
	// fail at predict time
	// (`descriptor.validateOverlayBaselineIndexPredict`) and at runtime
	// (`processing.ResolveBaselineIndex`) with
	// `PULSE_OVERLAY_REF_UNKNOWN` carrying `{baseline_index,
	// series_length}` Details. The handler propagates the resolver's
	// CodedError verbatim — the runtime resolver's coded shape IS the
	// kind's runtime range-check surface (same contract as the
	// `OVERLAY_INDEX_VS_BASELINE` twin).
	//
	// Baseline-at-self semantics: when the host's first present point IS
	// at `Position` (typical "anchor against first point" authoring), that
	// ordinal's delta is exactly `0.0` (self-vs-self under additive
	// subtraction). Renderers centre diverging colour ramps on
	// `baseline = 0` (mirrors `OVERLAY_DELTA_VS_MARGIN` /
	// `OVERLAY_DELTA_VS_SIBLING` / `OVERLAY_ZSCORE_VS_*`).
	//
	// Zero-baseline semantics: unlike the `OVERLAY_INDEX_VS_BASELINE` twin
	// (which divides by the baseline and rejects zero with
	// `PULSE_OVERLAY_REF_ZERO`), DELTA_VS_BASELINE performs subtraction
	// and is mathematically defined for every finite baseline value
	// including zero. The handler does NOT emit
	// `PULSE_OVERLAY_REF_ZERO`; a zero baseline simply yields
	// `delta_i = point_value_i - 0 = point_value_i` (the raw host value
	// passes through). Mirrors the existing `OVERLAY_DELTA_VS_SIBLING`
	// rule against zero sibling values.
	//
	// Absent-point policy: a host that did not produce a value for group
	// `i` (the resolver returns `(0, false)`) surfaces a `SeriesEntry`
	// whose `Summary` leaves `Statistic` unset — the canonical "present
	// slot, empty summary" shape from the SERIES dispatch contract.
	// Absent groups do NOT participate in the delta computation. The
	// baseline ordinal itself MAY be absent — `ResolveBaselineIndex` calls
	// `host.ValueAt` and the host's own resolver decides whether the
	// requested ordinal surfaces a present value or reports absent. An
	// absent baseline (`present=false` at the baseline ordinal) yields a
	// baseline value of `0.0` from the host; subsequent points subtract
	// that zero and the layer carries the raw host values verbatim (no
	// warning — distinct from the `OVERLAY_INDEX_VS_BASELINE` zero-
	// baseline `PULSE_OVERLAY_REF_ZERO` arm).
	//
	// Ref handling: `Ref.BaselineIndex` (with `Position >= 0`) is
	// REQUIRED. Empty `Ref` is rejected at predict time
	// (`PULSE_OVERLAY_REF_INCOMPATIBLE_WITH_SHAPE`). Any other ref-family
	// pointer (`Margin` / `Sibling` / `Prior` / `Population` / `Stage` /
	// `Slot`) is rejected with the same code.
	//
	// Scope must be GROUP. `Level` / `Within` MUST be zero (the baseline
	// is a single fixed positional anchor, not an axis prefix; non-zero
	// values fire `PULSE_OVERLAY_LEVEL_OUT_OF_RANGE` mirroring the
	// `INDEX_VS_BASELINE` / `INDEX_VS_TOTAL` / `INDEX_VS_PRIOR`
	// implicit-margin / windowed family rule).
	//
	// Buffered. Per `types/overlay_streamability.go`, the streamability
	// row is `false` — resolving a single positional baseline requires
	// the materialised host series (`host.ValueAt(Position)` reads after
	// finalize). The handler runs at the buffered post-host-finalize exit
	// via `ApplyOverlaysSeries`. Forward-compat: future work may lift
	// the baseline resolver into a streaming-aware shape; when that lands
	// the streamability flag flips to true.
	OverlayKindDeltaVsBaseline OverlayKind = "OVERLAY_DELTA_VS_BASELINE"

	// OverlayKindDeltaVsSibling emits a per-group additive delta against a
	// sibling group named in `Ref.Sibling`. GROUP scope over a SERIES
	// (grouped Process) host with SERIES payload — one SeriesEntry per
	// host group key in host order, each carrying `group_val - sibling_val`
	// on `Summary.Statistic`. The sibling is identified by `(Field, Value)`
	// — `Field` names a grouper Field present on the SERIES host, `Value`
	// names the specific axis-key value to compare against. The sibling
	// reference resolves to a single host group via the sibling resolver
	// (`processing/overlay_sibling_resolver.go`); every present group
	// emits a delta against that fixed reference point. The sibling
	// group itself emits `0` (self-vs-self under additive subtraction).
	//
	// Buffered (per kind-catalog-v1 "Streaming-capable subset"): sibling
	// resolution requires the full materialised SeriesPayload — the
	// streaming Process pass cannot resolve a `(Field, Value)` lookup
	// against the per-group accumulators until they are finalised, so the
	// handler runs at the buffered post-host-finalize exit. The
	// `OverlayStreamability[OverlayKindDeltaVsSibling]` row is `false`
	// (mirrors `OverlayKindIndexVsSibling`).
	//
	// Math (per host group i):
	//
	//	sibling_val = host[Ref.Sibling.Field, Ref.Sibling.Value]   // resolver lookup
	//	delta_i     = group_val[i] - sibling_val
	//
	// Absent-group policy: a host that did not produce a value for group
	// i (resolver returns `(0, false)`) surfaces a SeriesEntry whose
	// Summary leaves Statistic unset — canonical "present slot, empty
	// summary" shape from the SERIES dispatch contract. Absent groups
	// do NOT participate in the delta computation (and DO NOT surface
	// zero — they carry no Statistic).
	//
	// Unknown sibling path: when `Ref.Sibling.Field` is not a grouper
	// field on the host OR `Ref.Sibling.Value` does not match any
	// observed axis-key value, the handler emits ONE
	// PULSE_OVERLAY_REF_UNKNOWN warning carrying the offending
	// `(field, value)` pair and surfaces NaN statistics across every
	// present entry. Mirrors the INDEX_VS_TOTAL / SHARE_OF_TOTAL SERIES
	// PULSE_OVERLAY_REF_ZERO emission shape (one warning per layer, not
	// per cell). DELTA is mathematically defined even when sibling is
	// resolved AND sibling_val is zero — the delta is simply
	// `group_val[i] - 0 = group_val[i]`, so the zero-sibling-value case
	// does NOT raise PULSE_OVERLAY_REF_ZERO on this kind (unlike the
	// INDEX_VS_SIBLING twin which divides by sibling and rejects zero).
	//
	// Ref handling: `Ref.Sibling` is REQUIRED — both `Field` and `Value`
	// must be non-empty strings. Any other ref-family pointer (Margin /
	// BaselineIndex / Population / Stage / Slot) fails
	// PULSE_OVERLAY_REF_INCOMPATIBLE_WITH_SHAPE at predict time.
	//
	// Scope must be GROUP. The validator rejects any other scope.
	OverlayKindDeltaVsSibling OverlayKind = "OVERLAY_DELTA_VS_SIBLING"

	// OverlayKindIndexVsBaseline emits a per-point ratio index against a
	// single fixed positional baseline of an ordered SERIES (grouped
	// Process) host: `index_i = point_value_i / baseline_value * 100`
	// where `baseline_value` is the host's metric at the ordinal pinned
	// by `Ref.BaselineIndex.Position`. GROUP scope over a SERIES host with
	// SERIES payload — one `SeriesEntry` per host group key in host order,
	// each carrying the index on `Summary.Statistic`. Second windowed-
	// Process overlay in the catalog (first kind to consume the
	// `Ref.BaselineIndex.Position` arm of the OverlayBaselineIndexRef
	// discriminated union; sibling to `OVERLAY_INDEX_VS_PRIOR`).
	//
	// Math (per host group ordinal `i`):
	//
	//	baseline_value = host.ValueAt(Ref.BaselineIndex.Position)
	//	if baseline_value == 0 ⇒ NaN + warning  (PULSE_OVERLAY_REF_ZERO)
	//	if present[i]          ⇒ point / baseline * 100
	//	if !present[i]         ⇒ absent passthrough (no Statistic)
	//
	// Baseline resolution: the handler resolves the baseline value ONCE
	// up front via `processing.ResolveBaselineIndex(host,
	// Ref.BaselineIndex)`. Negative or out-of-range `Position` values
	// fail at predict time
	// (`descriptor.validateOverlayBaselineIndexPredict`) and at runtime
	// (`processing.ResolveBaselineIndex`) with
	// `PULSE_OVERLAY_REF_UNKNOWN` carrying `{baseline_index,
	// series_length}` Details. The handler propagates the resolver's
	// CodedError verbatim — the runtime resolver's coded shape IS the
	// kind's runtime range-check surface.
	//
	// Baseline-at-self semantics: when the host's first present point is
	// at `Position` (typical "anchor against first point" authoring), that
	// ordinal's index is exactly `100.0` (self-vs-self under the ratio
	// scaling). Renderers centre diverging colour ramps on `baseline = 100`
	// (mirrors `OVERLAY_INDEX_VS_MARGIN` / `OVERLAY_INDEX_VS_TOTAL` /
	// `OVERLAY_INDEX_VS_SIBLING` / `OVERLAY_INDEX_VS_PRIOR`).
	//
	// Zero-baseline path: when the resolved baseline value is `0` the
	// handler emits ONE `PULSE_OVERLAY_REF_ZERO` warning carrying the
	// overlay kind, the host group count, and the baseline ordinal; every
	// emitted entry's `Summary.Statistic` is NaN. Division by zero is
	// mathematically undefined and the same `PULSE_OVERLAY_REF_ZERO`
	// contract used by the share / index / zscore family applies (one
	// warning per layer, not per cell). Distinct from the
	// `OVERLAY_DELTA_VS_BASELINE` twin which performs subtraction and
	// does NOT raise on zero baseline.
	//
	// Absent-point policy: a host that did not produce a value for group
	// `i` (the resolver returns `(0, false)`) surfaces a `SeriesEntry`
	// whose `Summary` leaves `Statistic` unset — the canonical "present
	// slot, empty summary" shape from the SERIES dispatch contract.
	// Absent groups do NOT participate in the index computation. The
	// baseline ordinal itself MAY be absent — `ResolveBaselineIndex` calls
	// `host.ValueAt` and the host's own resolver decides whether the
	// requested ordinal surfaces a present value or reports absent. An
	// absent baseline (`present=false` at the baseline ordinal) yields a
	// baseline value of `0.0` from the host, which then routes through
	// the zero-baseline `PULSE_OVERLAY_REF_ZERO` arm above.
	//
	// Ref handling: `Ref.BaselineIndex` (with `Position >= 0`) is REQUIRED.
	// Empty `Ref` is rejected at predict time
	// (`PULSE_OVERLAY_REF_INCOMPATIBLE_WITH_SHAPE`). Any other ref-family
	// pointer (`Margin` / `Sibling` / `Prior` / `Population` / `Stage` /
	// `Slot`) is rejected with the same code.
	//
	// Scope must be GROUP. `Level` / `Within` MUST be zero (the baseline
	// is a single fixed positional anchor, not an axis prefix; non-zero
	// values fire `PULSE_OVERLAY_LEVEL_OUT_OF_RANGE` mirroring the
	// `INDEX_VS_TOTAL` / `INDEX_VS_PRIOR` implicit-margin / windowed
	// family rule).
	//
	// Buffered. Per `types/overlay_streamability.go`, the streamability
	// row is `false` — resolving a single positional baseline requires
	// the materialised host series (`host.ValueAt(Position)` reads after
	// finalize). The handler runs at the buffered post-host-finalize exit
	// via `ApplyOverlaysSeries`. Forward-compat: future work may lift
	// the baseline resolver into a streaming-aware shape (carrying the
	// resolved baseline inline as a kind-specific accumulator advanced
	// during the streaming pass once the orchestrator hits the baseline
	// ordinal); when that lands the streamability flag flips to true.
	OverlayKindIndexVsBaseline OverlayKind = "OVERLAY_INDEX_VS_BASELINE"

	// OverlayKindIndexVsMargin produces an index score per cell (or per
	// row/column, depending on Scope) by comparing the cell value against
	// the matching axis margin: 100 * cell / margin. Scope=CELL emits one
	// scalar per cell; Scope=ROW or COLUMN emits one scalar per axis key
	// when the comparison degenerates (e.g. row-share index vs grand
	// total). The default reference (Ref.Margin) names the axis whose
	// margin is the denominator. Inherently buffered because margins are
	// always recomputed from raw rows in the crosstab path.
	OverlayKindIndexVsMargin OverlayKind = "OVERLAY_INDEX_VS_MARGIN"

	// OverlayKindIndexVsPop emits a per-value population-comparison index
	// against a Facet host. For each value of the host Facet field the layer
	// surfaces `subset_freq / pop_freq * 100` where `subset_freq` is the
	// host FacetResult's per-value frequency (one filtered cohort) and
	// `pop_freq` is the resolver-supplied per-value frequency of the
	// reference population (typically an unfiltered or alternately-filtered
	// cohort). GROUP scope over a FACET host with SERIES payload — one
	// `SeriesEntry` per host value in payload order, each carrying the
	// index on `Summary.Statistic`. First FACET-host overlay in the
	// catalog and the first kind to consume the
	// `Ref.Population` arm of the OverlayRef discriminated union (the
	// resolver shape lives at `processing.FacetPopulationView`).
	//
	// Math (per host value `v`):
	//
	//	subset_freq = host.Discrete.Values[v].Count / host.FilteredRecords
	//	pop_freq    = population.DiscreteFrequency(v)
	//	if pop_freq == 0 ⇒ NaN entry + PULSE_OVERLAY_REF_ZERO (skip)
	//	otherwise        ⇒ subset_freq / pop_freq * 100
	//
	// Streaming finalize hook: the handler runs at
	// the FacetSchema streaming finalize point — once the per-value
	// `(value, count)` map is folded into the `FacetField.Discrete.Values`
	// slice the overlay reads the already-finalized host distribution and
	// the resolver's population view to emit the per-value index. No
	// second pass over records. The streaming-vs-buffered byte-identity
	// guarantee for the host Facet path holds because the handler depends
	// ONLY on POST-FINALIZE state.
	//
	// Categorical host fast path (v1 scope): the handler walks the
	// host's `FacetDiscrete.Values` slice in payload order (descending by
	// count, ties ascending by value-string) and the resolver looks each
	// value up in the population's discrete payload via
	// `FacetPopulationView.DiscreteFrequency(value)`. One division per
	// host value.
	//
	// Numeric host path (v1 scope): the handler walks the host's
	// histogram bins (when `IncludeHistogram` was true on the host
	// FacetRequest) and emits one index per bin via the resolver's
	// `NumericHistogram()` surface — same `subset_bin_freq /
	// pop_bin_freq * 100` math. When the host did not request a
	// histogram the kind emits zero entries (the host shape carries
	// only Welford summary stats, not per-value tallies — without a
	// histogram there are no per-value buckets to index). The handler
	// does NOT walk percentiles — percentile-bucket indexing is reserved
	// for `OVERLAY_KS_VS_POP`.
	//
	// Zero pop_freq path: when `pop_freq == 0` for some value v (the
	// value never appeared in the population, or the population has zero
	// records altogether), the handler emits ONE
	// `PULSE_OVERLAY_REF_ZERO` warning per affected entry carrying the
	// kind + value and SKIPS the index entry (Statistic stays unset on
	// that entry). The acceptance criterion is explicit: "on
	// `pop_freq == 0` emit warning code `PULSE_OVERLAY_REF_ZERO` and
	// skip the index entry". Subsequent entries continue to emit indices
	// against the same population view.
	//
	// Absent-population path: when the resolver could not find any
	// population entry for `value` (the value never appeared in the
	// population dictionary), the handler treats it as the zero-pop_freq
	// case and emits the same warning + skip behavior. Distinct from
	// `PULSE_OVERLAY_REF_UNKNOWN` which fires at the predict / runtime
	// boundary when the named population FIELD is unknown (the resolver
	// itself returns that code from `ResolveFacetPopulation`).
	//
	// Ref handling: `Ref.Population` MUST be populated. The Population
	// arm carries the comparison-population cohort name (resolver
	// matches against a FacetResult derived from that cohort). Any other
	// ref-family pointer (`Margin` / `Sibling` / `BaselineIndex` /
	// `Prior` / `RollingMean` / `YoY` / `Stage` / `Slot`) fires
	// `PULSE_OVERLAY_REF_INCOMPATIBLE_WITH_SHAPE` at predict time (the
	// per-kind validator lives in descriptor/overlay_facet.go).
	//
	// Scope must be GROUP. `Level` / `Within` MUST be zero (population
	// comparison is a single-value lookup, not an axis prefix); non-zero
	// values fire `PULSE_OVERLAY_LEVEL_OUT_OF_RANGE` mirroring the other
	// implicit-ref kinds.
	//
	// Streamable. Per `types/overlay_streamability.go`, the streamability
	// row is `true` — the handler runs as a single post-finalize fold
	// over the host's already-materialized per-value distribution and
	// the resolver's already-materialized population view. The
	// host-side streaming Facet pass keeps its existing online
	// accumulators (Welford / per-value count map); the overlay does
	// NOT widen the streaming carrier — it consumes finalized state.
	// Streamable matches the kind-catalog-v1 "Streaming-capable subset"
	// — `OVERLAY_INDEX_VS_POP` is explicitly listed as YES.
	//
	// Renderers centre diverging colour ramps on `baseline = 100`
	// (mirrors `OVERLAY_INDEX_VS_MARGIN` / `OVERLAY_INDEX_VS_TOTAL` /
	// `OVERLAY_INDEX_VS_SIBLING` / `OVERLAY_INDEX_VS_PRIOR` /
	// `OVERLAY_INDEX_VS_BASELINE`). Per-entry Statistic carries the
	// renderer-facing index value; layer-level Summary carries the
	// baseline + Min/Max/Count summary over present entries.
	OverlayKindIndexVsPop OverlayKind = "OVERLAY_INDEX_VS_POP"

	// OverlayKindIndexVsPrior emits a per-point windowed index against the
	// immediately preceding point of an ordered SERIES (grouped Process)
	// host: `index_i = point_value_i / prior_value_{i-1} * 100`. GROUP
	// scope over a SERIES host with SERIES payload — one `SeriesEntry`
	// per host group key in host order, each carrying the index on
	// `Summary.Statistic`. First **streamable** windowed-Process overlay
	// in the catalog and the first kind to use the windowed
	// `Ref.Prior` arm of the `OverlayRef` discriminated union.
	//
	// Math (per host group ordinal `i`):
	//
	//	if i == 0           ⇒ NaN              (no prior available)
	//	if prior_value == 0 ⇒ NaN + warning    (PULSE_OVERLAY_REF_ZERO)
	//	otherwise           ⇒ point / prior * 100
	//
	// Carrier shape (single-state lag): the handler walks the ordered
	// host series once. A single `float64` lag carrier remembers the most
	// recently seen PRESENT value; each subsequent present point divides
	// by the carrier and then advances the carrier to its own value.
	// Absent host points (resolver reports `(0, false)`) emit NaN for
	// that ordinal and DO NOT advance the carrier — the "prior" for the
	// next present point remains the last present value. This single-
	// state lag is what makes the kind streamable: the streaming-Process
	// orchestrator carries one f64 lag value alongside the per-group
	// accumulators inside the streaming fold, and the post-host finalize
	// is the divide step.
	//
	// First-point semantics: the first ordinal has no prior by
	// construction. The handler emits NaN on the first present entry and
	// does NOT raise `PULSE_OVERLAY_REF_ZERO` — this is "no comparison
	// available" rather than "denominator was zero". Renderers should
	// surface the first entry as "no comparison" (typically a blank
	// cell) rather than a degenerate signal.
	//
	// Zero-prior path: when the lag carrier is `0` at the divide step
	// (the previous present point had a value of zero), the handler emits
	// one `PULSE_OVERLAY_REF_ZERO` warning and surfaces NaN on the
	// affected entries. Subsequent points continue to use the same lag
	// carrier (since absent points do not advance the carrier, but a
	// PRESENT zero DOES advance the carrier — and the next point will
	// then again hit the zero-prior path). Mirrors the existing
	// `PULSE_OVERLAY_REF_ZERO` contract used by the share / index /
	// zscore family.
	//
	// Forward-compat lag knob: the `Ref.Prior.Lag` slot is reserved for
	// future window-N priors (e.g. lag-3 for "compare against three
	// points ago"). v1 ships lag-1 only — non-zero `Lag` is not
	// exercised by this kind today; later stories will widen the carrier
	// to a small ring buffer.
	//
	// Ref handling: the windowed `Ref.Prior` arm is the implicit default
	// for this kind. The validator accepts both a populated
	// `Ref.Prior` (with `Lag` zero or unset for v1) AND an entirely empty
	// `Ref` (the omitempty-friendly authoring shape) — both spell "lag-
	// 1 prior". Any other ref-family pointer (`Margin` / `Sibling` /
	// `BaselineIndex` / `Population` / `Stage` / `Slot`) fires
	// `PULSE_OVERLAY_REF_INCOMPATIBLE_WITH_SHAPE` at predict time.
	//
	// Scope must be GROUP. `Level` / `Within` MUST be zero (windowed
	// kind — the lag carrier folds across the ordered axis without a
	// prefix-bucket denominator; non-zero values fire
	// `PULSE_OVERLAY_LEVEL_OUT_OF_RANGE` mirroring the
	// `INDEX_VS_TOTAL` / χ² / Fisher implicit-margin family).
	//
	// Streamable. Per `types/overlay_streamability.go`, the streamability
	// row is `true` — the single-state lag carrier is one f64 carried
	// alongside the per-group accumulators inside the streaming Process
	// fold, and the divide step happens at host finalize. Renderers
	// centre diverging colour ramps on `baseline = 100` (mirrors
	// `OVERLAY_INDEX_VS_MARGIN` / `OVERLAY_INDEX_VS_TOTAL`).
	OverlayKindIndexVsPrior OverlayKind = "OVERLAY_INDEX_VS_PRIOR"

	// OverlayKindIndexVsRollingMean emits a per-point windowed index against
	// the arithmetic mean of the W immediately preceding points of an ordered
	// SERIES (grouped Process) host: `index_i = point_value_i / mean(point_{i-W} .. point_{i-1}) * 100`.
	// GROUP scope over a SERIES host with SERIES payload — one `SeriesEntry`
	// per host group key in host order, each carrying the index on
	// `Summary.Statistic`. Fourth windowed-Process overlay in the catalog
	// (siblings: `OVERLAY_INDEX_VS_PRIOR`, `OVERLAY_INDEX_VS_BASELINE`,
	// `OVERLAY_DELTA_VS_BASELINE`) and the first kind to consume the
	// `Ref.RollingMean` arm of the OverlayRef discriminated union.
	//
	// Window-via-Params convention: the rolling window width is supplied via
	// `OverlaySpec.Params["window"]` as a positive integer (mirrors the
	// `WIN_*` operator convention; see `skills/window-design.md` for the
	// pattern). The `Ref.RollingMean` arm is an empty marker struct
	// (`OverlayRollingMeanRef{}`) tagging the ref family — the v1 window
	// value lives entirely on `Params`. Forward-compat: `OverlayRollingMeanRef`
	// may grow non-Window knobs (e.g. weighting modes) without re-opening
	// the parent `OverlayRef`.
	//
	// Math (per host group ordinal `i`):
	//
	//	W = Params["window"] (positive int)
	//	if fewer than W present prior points have been seen ⇒ NaN (no warning)
	//	if mean(prior W points) == 0                       ⇒ NaN + PULSE_OVERLAY_REF_ZERO
	//	otherwise                                          ⇒ point / mean * 100
	//
	// Carrier shape (rolling window, buffered): the handler maintains a ring
	// buffer of the W most recently observed PRESENT values. The stored
	// shape is the (count, mean, M2) Welford triple — only the mean
	// is read by INDEX_VS_ROLLING_MEAN, but the M2 slot is reserved so the
	// `OVERLAY_ZSCORE_VS_ROLLING` handler can lift `sqrt(M2 / (count-1))`
	// from the same carrier (the per-group accumulator cost is +1 f64 per
	// layer per group — trivial relative to the overlay grand-budget; the
	// reuse keeps the windowed family's accumulator footprint flat as more
	// kinds land).
	//
	// Absent-point policy (ring buffer does not advance): an absent host
	// ordinal (resolver reports `(0, false)`) emits a present `SeriesEntry`
	// whose Summary leaves Statistic unset and DOES NOT advance the ring
	// buffer. Mirrors the `OVERLAY_INDEX_VS_PRIOR` absent-point lag carrier
	// rule — the next present ordinal will compare against the most recent
	// W PRESENT values, not absent slots.
	//
	// Window-fill semantics: the first W present ordinals all emit NaN
	// without warning — "window not yet filled" is structurally distinct
	// from "denominator was zero" (mirrors the `OVERLAY_INDEX_VS_PRIOR`
	// first-point NaN rule). When the host series is shorter than W (no
	// ordinal ever sees a full window) every entry's Statistic is NaN with
	// no warning.
	//
	// Zero-rolling-mean path: when the window mean is exactly `0` at the
	// divide step (e.g. every prior point in the window was zero) the
	// handler emits ONE `PULSE_OVERLAY_REF_ZERO` warning and surfaces NaN
	// on the affected entry. Subsequent points may again hit the zero-mean
	// path as the ring rolls; one warning per layer per zero-mean event
	// occurrence. Mirrors the share / index / zscore family's
	// zero-denominator contract.
	//
	// Window param validation: `Params["window"]` MUST be present and
	// MUST be a positive integer (JSON numbers decoded as float64 are
	// accepted when integral). Missing param fires
	// `PULSE_OVERLAY_PARAM_MISSING` carrying `{kind, param}` Details at
	// both predict (`descriptor.validateOverlayIndexVsRollingMean`) and
	// runtime (`processing.applyIndexVsRollingMean`). Window `<= 0` fires
	// `PULSE_OVERLAY_LEVEL_OUT_OF_RANGE` carrying `{window}` Details — the
	// LEVEL_OUT_OF_RANGE code is reused for "value out of valid range"
	// semantics so the new code surface stays narrow (mirrors the existing
	// usage on Level / Within slots).
	//
	// Ref handling: `Ref.RollingMean` MUST be populated (empty struct is
	// fine; the marker tags the ref family). Any other ref-family pointer
	// (`Margin` / `Sibling` / `BaselineIndex` / `Prior` / `Population` /
	// `Stage` / `Slot`) fires `PULSE_OVERLAY_REF_INCOMPATIBLE_WITH_SHAPE`
	// at predict time.
	//
	// Scope must be GROUP. `Level` / `Within` MUST be zero (windowed kind —
	// the rolling-mean carrier folds across the ordered axis without a
	// prefix-bucket denominator; non-zero values fire
	// `PULSE_OVERLAY_LEVEL_OUT_OF_RANGE` mirroring the
	// `INDEX_VS_PRIOR` / `INDEX_VS_BASELINE` / `INDEX_VS_TOTAL`
	// implicit-margin / windowed family).
	//
	// Buffered. Per `types/overlay_streamability.go`, the streamability row
	// is `false` — the ring buffer carries the full window of present values
	// per group, which the streaming Process pass cannot maintain inline
	// with the per-record fold today (the carrier widens beyond a single
	// f64 — `OVERLAY_INDEX_VS_PRIOR`'s streamable lag carrier is one f64;
	// rolling-mean's ring is `W` f64s and grows the streaming-fold state
	// past v1's single-state lag accumulator). Forward-compat: a future
	// story may lift the rolling-mean ring into a streaming-aware shape
	// (the streaming orchestrator's per-group accumulator carries the ring
	// inline alongside the per-group online aggregator); when that lands
	// the streamability flag flips to true.
	OverlayKindIndexVsRollingMean OverlayKind = "OVERLAY_INDEX_VS_ROLLING_MEAN"

	// OverlayKindIndexVsStage is the whole-chain ratio overlay against
	// another ProcessChain stage's result. CHAIN scope over a CHAIN
	// (ProcessChain) host with a shape inherited from the target stage
	// (scalar / series / matrix depending on the target stage's host
	// result shape). The first whole-chain overlay in the catalog
	// (whole-chain kind; runtime handler in processing/) and the
	// first kind to consume the StageRef discriminated reference family
	// (Ref + Target slots on `ChainOverlaySpec` — both populate exactly
	// one of Index / Name per StageRef value).
	//
	// Math (per host coordinate `k`):
	//
	//	target_val_k = target_stage.ValueAt(k)
	//	ref_val_k    = ref_stage.ValueAt(k)
	//	if ref_val_k == 0 ⇒ NaN + warning (PULSE_OVERLAY_REF_ZERO)
	//	otherwise         ⇒ target_val_k / ref_val_k * 100
	//
	// Whole-chain barrier semantics: the handler runs at the
	// post-stage-loop barrier inside `service.ProcessChain` (see
	// `service/chain.go`'s `applyChainOverlays` hook). Per-stage overlays
	// (`Stages[i].Overlays` on the request half of the dual-slot design)
	// land on each stage's individual `Response.Overlays` as a side
	// effect of the per-stage Process call; whole-chain overlays land on
	// `ChainResponse.Overlays` AFTER every stage has finalised. The
	// whole-chain fold operates exclusively on already-materialised
	// `*Response` objects — no record re-traversal.
	//
	// Default Target: when both `Target.Index` is nil AND `Target.Name`
	// is empty, the resolver defaults to the latest stage
	// (`len(Stages) - 1`). The default makes the typical "anchor against
	// the chain's final result" authoring shape concise. Setting either
	// slot explicitly overrides the default. `Ref` has no default —
	// every spec MUST name a baseline stage explicitly.
	//
	// Shape-divergence rule (PRD §6 FR-F2): when the target stage's host
	// shape (matrix vs series vs scalar) differs from the reference
	// stage's host shape the handler emits
	// `PULSE_OVERLAY_CHAIN_STAGE_SHAPE_DIVERGENT` and surfaces
	// NaN values across every coordinate. The shape-divergence check
	// runs at the whole-chain barrier — predict cannot catch it because
	// the stage shapes are not known until the chain has executed.
	//
	// Unknown stage path: when `Target` or `Ref` names a stage that does
	// not exist (Index out of range OR Name does not match any
	// `ChainStage.Name`), the resolver fires
	// `PULSE_OVERLAY_TARGET_UNKNOWN` / `PULSE_OVERLAY_REFERENCE_UNKNOWN`
	// respectively (catalog reservations — until those codes land, the
	// dispatcher chassis falls back to `PULSE_OVERLAY_REF_UNKNOWN` for
	// both arms).
	//
	// Scope must be CHAIN. `Level` / `Within` MUST be zero (the
	// reference is a single named stage, not an axis prefix); non-zero
	// values fire `PULSE_OVERLAY_LEVEL_OUT_OF_RANGE` mirroring the
	// implicit-ref family.
	//
	// Buffered. Per `types/overlay_streamability.go`, the streamability
	// row is `false` — the whole-chain barrier runs after every stage
	// has produced a `*Response`, so there is no streamable arm for the
	// whole-chain kind family by construction. Renderers centre
	// diverging colour ramps on `baseline = 100` (mirrors the
	// INDEX_VS_* family on other host shapes).
	OverlayKindIndexVsStage OverlayKind = "OVERLAY_INDEX_VS_STAGE"

	// OverlayKindDeltaVsStage is the whole-chain additive-delta sibling
	// of `OVERLAY_INDEX_VS_STAGE`. CHAIN scope over a CHAIN (ProcessChain)
	// host with a shape inherited from the target stage (scalar / series
	// / matrix depending on the target stage's host result shape).
	// Second whole-chain overlay in the catalog (runtime handler in
	// processing/).
	//
	// Math (per host coordinate `k`):
	//
	//	target_val_k = target_stage.ValueAt(k)
	//	ref_val_k    = ref_stage.ValueAt(k)
	//	delta_k      = target_val_k - ref_val_k
	//
	// Unlike the `OVERLAY_INDEX_VS_STAGE` twin (which divides by the
	// reference value and emits `PULSE_OVERLAY_REF_ZERO` against a zero
	// denominator), `OVERLAY_DELTA_VS_STAGE` performs subtraction and is
	// mathematically defined for every finite reference value including
	// zero — the handler does NOT emit `PULSE_OVERLAY_REF_ZERO`. Mirrors
	// the `OVERLAY_DELTA_VS_BASELINE` / `OVERLAY_DELTA_VS_MARGIN` /
	// `OVERLAY_DELTA_VS_SIBLING` family rule.
	//
	// Output preserves the target stage's units — a $-valued aggregator
	// in the target stage minus a $-valued aggregator in the reference
	// stage yields a $-valued deviation in the same currency. Renderers
	// centre diverging colour ramps on `baseline = 0` (mirrors the rest
	// of the DELTA family on other host shapes).
	//
	// Shape-divergence rule (PRD §6 FR-F2): same as
	// `OVERLAY_INDEX_VS_STAGE` — when the target stage's host shape
	// differs from the reference stage's host shape the handler emits
	// `PULSE_OVERLAY_CHAIN_STAGE_SHAPE_DIVERGENT` and surfaces
	// NaN values across every coordinate.
	//
	// Default Target / Unknown-stage / Scope / Level / Within / Buffered
	// rules: identical to `OVERLAY_INDEX_VS_STAGE`.
	OverlayKindDeltaVsStage OverlayKind = "OVERLAY_DELTA_VS_STAGE"

	// OverlayKindIndexVsSibling emits a per-group index score against a
	// sibling group named in `Ref.Sibling`: `(group_val / sibling_val) *
	// 100.0` per host group key. GROUP scope over a SERIES (grouped
	// Process) host with SERIES payload — one SeriesEntry per host group
	// key in host order, each carrying the index on `Summary.Statistic`.
	// The sibling is identified by `(Field, Value)` — `Field` names a
	// grouper Field present on the SERIES host, `Value` names the
	// specific axis-key value to compare against. The sibling reference
	// resolves to a single host group via the sibling resolver
	// (`processing/overlay_sibling_resolver.go`); every present group
	// emits an index against that fixed reference point. The sibling
	// group itself emits `100.0` (self-vs-self under the ratio scaling).
	//
	// Buffered (per kind-catalog-v1 "Streaming-capable subset"): sibling
	// resolution requires the full materialised SeriesPayload — the
	// streaming Process pass cannot resolve a `(Field, Value)` lookup
	// against the per-group accumulators until they are finalised, so the
	// handler runs at the buffered post-host-finalize exit. The
	// `OverlayStreamability[OverlayKindIndexVsSibling]` row is `false`
	// (mirrors `OverlayKindDeltaVsSibling`).
	//
	// Math (per host group i):
	//
	//	sibling_val = host[Ref.Sibling.Field, Ref.Sibling.Value]   // resolver lookup
	//	index_i     = (group_val[i] / sibling_val) * 100.0
	//
	// Absent-group policy: a host that did not produce a value for group
	// i (resolver returns `(0, false)`) surfaces a SeriesEntry whose
	// Summary leaves Statistic unset — canonical "present slot, empty
	// summary" shape from the SERIES dispatch contract. Absent groups
	// do NOT participate in the index computation.
	//
	// Unknown sibling path: when `Ref.Sibling.Field` is not a grouper
	// field on the host OR `Ref.Sibling.Value` does not match any
	// observed axis-key value, the handler emits ONE
	// PULSE_OVERLAY_REF_UNKNOWN warning carrying the offending
	// `(field, value)` pair and surfaces NaN statistics across every
	// present entry. Mirrors DELTA_VS_SIBLING's unknown-sibling
	// emission shape.
	//
	// Zero-sibling path: when the sibling resolves but its value is zero
	// (legitimate group with a zero post-filter sum), the handler emits
	// ONE PULSE_OVERLAY_REF_ZERO warning and surfaces NaN across every
	// present entry — division by zero is mathematically undefined and
	// the same PULSE_OVERLAY_REF_ZERO contract the SERIES INDEX_VS_TOTAL
	// / SHARE_OF_TOTAL kinds use applies here. DELTA_VS_SIBLING does NOT
	// emit this warning (subtraction by zero is well-defined).
	//
	// Ref handling: `Ref.Sibling` is REQUIRED — both `Field` and `Value`
	// must be non-empty strings. Any other ref-family pointer (Margin /
	// BaselineIndex / Population / Stage / Slot) fails
	// PULSE_OVERLAY_REF_INCOMPATIBLE_WITH_SHAPE at predict time.
	//
	// Scope must be GROUP. The validator rejects any other scope.
	OverlayKindIndexVsSibling OverlayKind = "OVERLAY_INDEX_VS_SIBLING"

	// OverlayKindIndexVsTotal emits a per-group index score against the
	// grand total of a SERIES host (grouped Process result): for each
	// host group key the layer surfaces `(group_val / grand_total) *
	// 100.0`. GROUP scope over a SERIES (grouped Process) host with
	// SERIES payload — one SeriesEntry per host group key in host order,
	// each carrying the index score on `Summary.Statistic`. First
	// streamable overlay in the catalog (per kind-catalog-v1 §
	// "Streaming-capable subset"): the handler folds at the end of the
	// existing streaming Process pass via a running grand-total
	// accumulator that runs alongside the per-group accumulators — no
	// second pass over records, the post-host finalize divides each
	// group value by the running grand total.
	//
	// Math (per host group i):
	//
	//	grand_total = Σ_j group_val[j]   (post-filter rows, AGG_SUM
	//	                                   semantics — never reads pre-
	//	                                   filter row count)
	//	index_i     = (group_val[i] / grand_total) * 100.0
	//
	// Zero-grand-total path: when grand_total == 0 (every group's
	// post-filter value sums to zero, including the degenerate "no
	// groups survived the filter" case) the handler emits ONE
	// PULSE_OVERLAY_REF_ZERO warning and populates every entry's
	// Summary.Statistic with NaN. Mirrors the existing
	// PULSE_OVERLAY_REF_ZERO contract used by the share / index family
	// against a missing axis margin on the MATRIX host.
	//
	// Absent-group policy: a host that did not produce a record for
	// group i (the resolver returns (0, false)) surfaces a SeriesEntry
	// whose Summary leaves Statistic unset — the canonical "present
	// slot, empty summary" shape established by the SERIES
	// dispatch contract. Absent groups do NOT contribute to the grand
	// total (the resolver gates the accumulator the same way it gates
	// per-group emission).
	//
	// Ref handling: implicit-grand-total. The Ref union is left EMPTY
	// — the kind's denominator is the host series' own grand total, so
	// callers supplying any Ref family pointer (Margin / Sibling /
	// BaselineIndex / Population / Stage / Slot) fail
	// PULSE_OVERLAY_REF_INCOMPATIBLE_WITH_SHAPE at predict time
	// (mirrors the CHISQ_* implicit-margin contract).
	//
	// Scope is GROUP (not CELL or ROW): the kind decorates each grouper
	// level the host emits. The validator rejects any other scope.
	//
	// Streamable. Per types/overlay_streamability.go, the streamability
	// row is `true` — the grand-total accumulator is one f64 carried
	// alongside the per-group accumulators inside the streaming Process
	// fold, and the division step happens at host finalize. The
	// streaming-Process orchestrator wires the inner per-record hook;
	// this kind ships with a SERIES-host post-finalize entry
	// (ApplyOverlaysSeries route) so the streaming-vs-buffered byte-
	// identity test holds at the entry-point level today.
	OverlayKindIndexVsTotal OverlayKind = "OVERLAY_INDEX_VS_TOTAL"

	// OverlayKindKSVsPop emits a single scalar Kolmogorov-Smirnov distance
	// (`D = sup_x |F_subset(x) - F_pop(x)|`) plus an asymptotic p-value
	// comparing the host Facet subset NUMERIC distribution against the
	// resolved population NUMERIC distribution. GROUP scope over a FACET
	// (FacetSchema) host with SCALAR payload — the layer carries the KS
	// statistic plus its asymptotic p-value via OverlaySummary{Statistic,
	// PValue, Parameters{"n_subset", "n_pop"}}. Fourth and final FACET-host
	// overlay in the catalog (siblings: OVERLAY_INDEX_VS_POP
	// descriptive, OVERLAY_ZSCORE_VS_POP descriptive,
	// OVERLAY_CHISQ_VS_POP inferential discrete-arm only). KS is
	// the canonical NUMERIC-arm distributional-shift indicator — it pairs
	// with CHISQ_VS_POP as the two inferential FACET-host kinds: CHISQ
	// quantifies divergence on categorical distributions, KS quantifies
	// divergence on numeric distributions. The viz developer renders the
	// SCALAR statistic as a distributional-shift indicator near the facet
	// header.
	//
	// Math:
	//
	//	D       = sup_x |F_subset(x) - F_pop(x)|      // KS distance
	//	n_subset = host.Numeric.Count
	//	n_pop    = pop.Numeric.Count
	//	en      = sqrt(n_subset * n_pop / (n_subset + n_pop))
	//	p_value = kolmogorovSurvival((en + 0.12 + 0.11/en) * D)
	//
	// Reuses the same Kolmogorov-survival helper backing TEST_KS
	// (kolmogorovSurvival in processing/test_stat.go) so the overlay and
	// the row-test surface produce identical p-values for the same D + N.
	// The empirical-CDF distance reuses the same `ksTwoSampleD` helper
	// pattern (processing/test_ks.go) — for KS_VS_POP the handler hand-
	// rolls a histogram-edge or percentile-anchor sup-CDF walk because the
	// population view does not retain raw values (see below).
	//
	// Numeric arm only: KS is undefined on categorical distributions
	// (there is no continuous CDF to compare). A categorical host (no
	// numeric payload) fires PULSE_OVERLAY_SCOPE_UNSUPPORTED at runtime
	// (the per-kind validator lives in descriptor/overlay_facet.go; the
	// runtime check is belt-and-braces). Distinct from
	// CHISQ_VS_POP which is DISCRETE-arm only and rejects numeric hosts
	// with PULSE_OVERLAY_REF_INCOMPATIBLE_WITH_SHAPE — the two kinds
	// partition the FACET-host inferential surface cleanly by host arm.
	//
	// Population-sample availability (KEY DESIGN NOTE — see commit message
	// + skill catalog row): the finalized `FacetPopulationView` exposes
	// ONLY summary state for numeric fields — Welford mean/stdev/count, an
	// optional percentile map, and an optional fixed-width histogram. Raw
	// population values were folded into Welford state at FacetSchema fold
	// time and DISCARDED. KS requires two empirical CDFs, so the handler
	// picks the highest-precision reconstruction available:
	//
	//	  1. Histogram path (PREFERRED — best precision when both arms
	//	     carry a histogram): when host AND population each surface a
	//	     histogram via FacetNumeric.Histogram, the handler aligns bucket
	//	     edges, builds the cumulative bucket-fraction CDF on both sides,
	//	     and computes sup |F_subset(edge_i) - F_pop(edge_i)|. KS
	//	     precision is bounded by bucket width — finer histograms yield
	//	     finer KS distance.
	//	  2. Percentile path (FALLBACK — coarser but always reconstructable
	//	     when both arms requested percentiles): when histograms are
	//	     absent but BOTH arms surface a percentile map via
	//	     FacetNumeric.Percentiles, the handler turns each percentile
	//	     entry into a CDF sample point (`p25` -> CDF=0.25 at the
	//	     percentile's value) and computes sup-CDF-difference over the
	//	     common percentile labels. Lower precision than histograms (a
	//	     handful of CDF samples per side vs hundreds of bucket edges)
	//	     but still well-defined.
	//	  3. Welford-only path (DEGENERATE): when neither histograms nor
	//	     percentiles are available the handler cannot reconstruct an
	//	     empirical CDF — it emits NaN statistic + NaN p-value and one
	//	     layer-level PULSE_OVERLAY_REF_ZERO warning. The caller's
	//	     FacetRequest can lift this case by setting either
	//	     IncludeHistogram=true on both arms or by passing
	//	     NumericPercentiles=["p10","p25","p50","p75","p90"] on both
	//	     arms.
	//
	// FORWARD-COMPAT PRECISION UPLIFT: future work may extend the
	// resolver to retain raw sorted population values when the host
	// FacetRequest names a KS overlay — the handler would then call
	// ksTwoSampleD directly on the raw arms (path 0) and the histogram /
	// percentile fallback would become an opt-out branch. The current
	// implementation is path-agnostic at the dispatch level so adding the
	// raw-value path is a strictly additive change.
	//
	// Inherently buffered (per PRD §2 Non-Goals "Streaming overlay path
	// for inferential kinds"). The streamable subset stays empty for KS
	// because the empirical-CDF construction requires sorted values on
	// both sides; an online folded shape would have to widen the streaming
	// carrier to a heap or a histogram per group. The streamability row in
	// types/overlay_streamability.go is `false` regardless of host
	// streamability.
	//
	// Degenerate inputs:
	//
	//   - Empty host distribution (n_subset == 0) OR empty population
	//     (n_pop == 0): NaN statistic + NaN p-value with one
	//     PULSE_OVERLAY_REF_ZERO warning. No KS distance can be computed
	//     without samples on both sides.
	//   - Host has neither histogram NOR percentiles (the
	//     reconstruction-impossible case above): NaN + NaN with
	//     PULSE_OVERLAY_REF_ZERO. The warning Details carry which knobs
	//     were missing so the caller can audit which to flip on the
	//     FacetRequest.
	//   - Histogram arms with mismatched edges (different Min / Max /
	//     bucket count): handler falls through to the percentile path if
	//     available, else emits PULSE_OVERLAY_REF_ZERO carrying the
	//     mismatching shape Details.
	//
	// Ref handling: `Ref.Population` MUST be populated (the cohort name
	// lives on `Ref.Population.Cohort`); any other ref-family pointer
	// (Margin / Sibling / BaselineIndex / Prior / RollingMean / YoY /
	// Stage / Slot) fires PULSE_OVERLAY_REF_INCOMPATIBLE_WITH_SHAPE at
	// predict time (per-kind validator lives in
	// descriptor/overlay_facet.go). Mirrors the INDEX_VS_POP /
	// ZSCORE_VS_POP / CHISQ_VS_POP ref contract.
	//
	// Scope must be GROUP. `Level` / `Within` MUST be zero (population
	// comparison is a single-statistic lookup, not an axis prefix); non-
	// zero values fire PULSE_OVERLAY_LEVEL_OUT_OF_RANGE mirroring the
	// implicit-ref family.
	//
	// Renderer-facing shape: the viz developer reads `OverlayPayload.Scalar`
	// for the KS distance D and `OverlaySummary` for the renderer
	// metadata. The recommended rendering is a distributional-shift badge
	// near the facet header carrying the (D, p_value) pair — large D and
	// small p_value flag "the subset distribution shape diverges from the
	// population". The layer-level Baseline stays unset (inferential
	// overlays do not surface a ratio centerpoint; mirrors CHISQ_VS_POP).
	OverlayKindKSVsPop OverlayKind = "OVERLAY_KS_VS_POP"

	// OverlayKindShareOfCol emits a per-cell share-of-column ratio: cell
	// / col_margin. CELL-scoped over a MATRIX (crosstab) host. Cells
	// along a single column sum to 1.0 in the absence of missing cells;
	// renderers can present the layer as a 100%-stacked vertical
	// projection. Structural twin of OVERLAY_SHARE_OF_ROW with the
	// column-axis margin slot as the denominator. Inherently buffered for
	// the same reason as INDEX_VS_MARGIN — the host crosstab
	// orchestrator recomputes margins from raw rows before ApplyOverlays
	// runs.
	OverlayKindShareOfCol OverlayKind = "OVERLAY_SHARE_OF_COL"

	// OverlayKindShareOfRow emits a per-cell share-of-row ratio: cell /
	// row_margin. CELL-scoped over a MATRIX (crosstab) host. Cells along
	// a single row sum to 1.0 in the absence of missing cells; renderers
	// can present the layer as a 100%-stacked horizontal projection.
	// Inherently buffered for the same reason as INDEX_VS_MARGIN — the
	// host crosstab orchestrator recomputes margins from raw rows before
	// ApplyOverlays runs.
	OverlayKindShareOfRow OverlayKind = "OVERLAY_SHARE_OF_ROW"

	// OverlayKindShareOfTotal emits a share-of-grand-total ratio.
	// Dual-shape overload — the dispatch selects the host shape and the
	// runtime handler differs by host:
	//
	//   - MATRIX dispatch: per-cell ratio `cell / grand_total`.
	//     CELL-scoped over a MATRIX (crosstab) host. The entire matrix
	//     sums to 1.0 in the absence of missing cells; renderers can
	//     present the layer as a single-population share projection
	//     where each cell's contribution to the whole table is visible
	//     at a glance. Completes the matrix share triad (row / col /
	//     total). Structural twin of OVERLAY_SHARE_OF_ROW and
	//     OVERLAY_SHARE_OF_COL with the grand-axis margin slot as the
	//     denominator. Buffered (the host crosstab orchestrator
	//     recomputes margins from raw rows before ApplyOverlays runs).
	//     `Ref.Margin` is required (grand-axis-locked even though the
	//     handler ignores the axis value).
	//
	//   - SERIES dispatch: per-group ratio
	//     `group_val / grand_total`, scale 1.0 (no ×100 — emits the raw
	//     share so cells over a complete partition sum to 1.0 within
	//     ULP). GROUP-scoped over a SERIES (grouped Process) host with
	//     SERIES payload — one `SeriesEntry` per host group key in host
	//     order, each carrying the share on `Summary.Statistic`.
	//     Streamable — sibling kind to OVERLAY_INDEX_VS_TOTAL, same
	//     grand-total accumulator (`computeSeriesGrandTotal` in
	//     processing/overlay_series.go) carried alongside the per-group
	//     accumulators in the streaming Process fold (the streaming
	//     orchestrator wires it inline; this kind ships the
	//     post-finalize entry today). `Ref` MUST be empty
	//     (implicit-grand-total); any Ref-family pointer fires
	//     `PULSE_OVERLAY_REF_INCOMPATIBLE_WITH_SHAPE` at predict time.
	//     Zero `grand_total` emits one `PULSE_OVERLAY_REF_ZERO` warning
	//     and populates every entry's Statistic with NaN. Absent host
	//     groups (resolver reports `(0, false)`) surface a present
	//     SeriesEntry whose Summary leaves Statistic unset and do NOT
	//     contribute to the grand total.
	//
	// The kind-catalog-v1 interview's resolved-decision rule kept the
	// SHARE and INDEX kind names distinct ("readable kind names beat
	// scale-param overloading at JSON-authoring time"), so the SERIES
	// SHARE_OF_TOTAL dispatch is intentionally a separate kind from
	// INDEX_VS_TOTAL even though the math overlaps — callers cannot
	// confuse `share` with `index / 100` at the spec authoring surface.
	OverlayKindShareOfTotal OverlayKind = "OVERLAY_SHARE_OF_TOTAL"

	// OverlayKindZScoreVsRolling emits a per-point windowed z-score against
	// the rolling-window mean + sample SD of the W immediately preceding
	// points of an ordered SERIES (grouped Process) host:
	// `zscore_i = (point_value_i - rolling_mean(W)) / rolling_sd(W)`
	// where `rolling_sd = sqrt(M2 / (count - 1))` (SAMPLE SD, n-1
	// denominator). GROUP scope over a SERIES host with SERIES payload —
	// one `SeriesEntry` per host group key in host order, each carrying
	// the z-score on `Summary.Statistic`. Fifth windowed-Process overlay
	// in the catalog (siblings: `OVERLAY_INDEX_VS_PRIOR`,
	// `OVERLAY_INDEX_VS_BASELINE`, `OVERLAY_DELTA_VS_BASELINE`,
	// `OVERLAY_INDEX_VS_ROLLING_MEAN`) and the second
	// consumer of the `Ref.RollingMean` arm of the OverlayRef
	// discriminated union (sibling windowed-rolling family — both kinds
	// carry the window width on `Params["window"]`).
	//
	// Window-via-Params convention: the rolling window width is supplied
	// via `OverlaySpec.Params["window"]` as a positive integer (mirrors
	// the `WIN_*` operator convention; see `skills/window-design.md`).
	// The `Ref.RollingMean` arm is an empty marker struct
	// (`OverlayRollingMeanRef{}`) tagging the ref family — identical to
	// the `OVERLAY_INDEX_VS_ROLLING_MEAN` shape so the validator's
	// "exactly one ref arm populated per kind" contract stays uniform.
	//
	// Variance choice (SAMPLE, not population) — KEY CONTRAST with
	// `OVERLAY_ZSCORE_VS_TOTAL`: the rolling z-score uses SAMPLE SD
	// (divide by `count - 1`, NOT N). The rationale is structural: a
	// rolling window of W observations IS a sample of the wider time
	// series, so unbiased sample variance is the correct convention. By
	// contrast `OVERLAY_ZSCORE_VS_TOTAL` uses POPULATION SD (`sqrt(M2 /
	// N)`) because the per-group aggregation set IS the whole population
	// being standardised against. The two surfaces are intentionally
	// orthogonal: rolling = local sample; total = global population. The
	// `OVERLAY_INDEX_VS_ROLLING_MEAN` carrier stores the Welford
	// `(count, mean, M2)` trio precisely so this kind can lift the
	// rolling SD via `sqrt(M2 / (count - 1))` without re-folding.
	//
	// Math (per host group ordinal `i`):
	//
	//	W = Params["window"] (positive int)
	//	if fewer than 2 PRESENT prior points in the window
	//	                                                  ⇒ NaN (no warning)
	//	if rolling_sd(W) == 0                             ⇒ NaN + PULSE_OVERLAY_REF_ZERO
	//	otherwise                                         ⇒ (point - mean) / sd
	//
	// Carrier shape (shared with `OVERLAY_INDEX_VS_ROLLING_MEAN`): the
	// handler maintains the same per-group ring buffer of the W most
	// recently observed PRESENT values plus the Welford-Pébaÿ (count,
	// mean, M2) trio updated alongside each push/evict. INDEX_VS_ROLLING_MEAN
	// reads only `mean`; ZSCORE_VS_ROLLING reads BOTH `mean` and `M2`. The
	// shared carrier keeps the per-group accumulator footprint flat as
	// more rolling-family kinds land.
	//
	// Window-fill semantics: when the carrier `count < 2` (the Welford
	// recurrence requires at least two observations to define a sample
	// variance), the handler emits NaN without warning — "window not yet
	// filled with at least 2 priors" is structurally distinct from
	// "denominator was zero" (mirrors the `OVERLAY_INDEX_VS_ROLLING_MEAN`
	// window-not-yet-filled rule). When the host series is shorter than W
	// (no ordinal ever sees a full window) every entry's Statistic is NaN
	// with no warning.
	//
	// Absent-point policy (ring buffer does not advance): an absent host
	// ordinal (resolver reports `(0, false)`) emits a present
	// `SeriesEntry` whose Summary leaves Statistic unset and DOES NOT
	// advance the ring buffer. Mirrors the
	// `OVERLAY_INDEX_VS_ROLLING_MEAN` / `OVERLAY_INDEX_VS_PRIOR` absent-
	// point carrier rule — the next present ordinal will compare against
	// the most recent W PRESENT values, not absent slots.
	//
	// Zero rolling-SD path: when the rolling SD is exactly `0` at the
	// divide step (e.g. every prior point in the window has the same
	// value — constant series), the handler emits ONE
	// `PULSE_OVERLAY_REF_ZERO` warning per layer and surfaces NaN on the
	// affected entry. Subsequent points may again hit the zero-SD path
	// as the ring rolls; one warning per layer per zero-SD event
	// occurrence. Mirrors the share / index / zscore family's zero-
	// denominator contract.
	//
	// Window param validation: `Params["window"]` MUST be present and
	// MUST be a positive integer (JSON numbers decoded as float64 are
	// accepted when integral). Missing param fires
	// `PULSE_OVERLAY_PARAM_MISSING` carrying `{kind, param}` Details at
	// both predict (`descriptor.validateOverlayZScoreVsRolling`) and
	// runtime (`processing.applyZScoreVsRolling`). Window `<= 0` fires
	// `PULSE_OVERLAY_LEVEL_OUT_OF_RANGE` carrying `{window}` Details —
	// the LEVEL_OUT_OF_RANGE code is reused for "value out of valid
	// range" semantics so the new code surface stays narrow (mirrors the
	// existing usage on Level / Within slots).
	//
	// Ref handling: `Ref.RollingMean` MUST be populated (empty struct is
	// fine; the marker tags the ref family). Any other ref-family
	// pointer (`Margin` / `Sibling` / `BaselineIndex` / `Prior` /
	// `Population` / `Stage` / `Slot`) fires
	// `PULSE_OVERLAY_REF_INCOMPATIBLE_WITH_SHAPE` at predict time.
	//
	// Scope must be GROUP. `Level` / `Within` MUST be zero (windowed
	// kind — the rolling-window carrier folds across the ordered axis
	// without a prefix-bucket denominator; non-zero values fire
	// `PULSE_OVERLAY_LEVEL_OUT_OF_RANGE` mirroring the
	// `INDEX_VS_ROLLING_MEAN` / `INDEX_VS_PRIOR` / `INDEX_VS_BASELINE` /
	// `INDEX_VS_TOTAL` implicit-margin / windowed family).
	//
	// Buffered. Per `types/overlay_streamability.go`, the streamability
	// row is `false` — the ring buffer carries the full window of present
	// values plus the Welford trio per group, which the streaming Process
	// pass cannot maintain inline with the per-record fold today (matches
	// the `OVERLAY_INDEX_VS_ROLLING_MEAN` buffered rationale verbatim).
	// Forward-compat: future work may lift the rolling carrier into a
	// streaming-aware shape; when that lands the streamability flag flips
	// to true.
	//
	// Chan-Welford merge documentation (per CLAUDE.md § Execution modes
	// → Parallel buffered Process): the Welford triple combines under
	// Chan-Welford with the standard `delta = mean_B - mean_A;
	// mean = mean_A + delta * n_B/n;
	// M2 = M2_A + M2_B + delta² * n_A*n_B/n` recurrence. v1 ships
	// serial-per-group execution so the merge path is not exercised
	// today, but the carrier shape stays parallel-safe so future work
	// that lifts the rolling fold into the parallel buffered Process
	// pipeline can reuse the existing merge plumbing.
	//
	// Renderers centre diverging colour ramps on `baseline = 0` (mirrors
	// `OVERLAY_ZSCORE_VS_TOTAL` / `OVERLAY_ZSCORE_VS_MARGIN` — every
	// z-score family produces a centred distribution, not a ratio).
	OverlayKindZScoreVsRolling OverlayKind = "OVERLAY_ZSCORE_VS_ROLLING"

	// OverlayKindZScoreVsPop emits a per-value population-comparison
	// z-score against a FACET host: `z = (subset_freq - pop_freq) / sd_pop`
	// for each value of the host Facet field. GROUP scope over a FACET
	// host with SERIES payload — one `SeriesEntry` per host value in
	// payload order, each carrying the z-score on `Summary.Statistic`.
	// Sibling streamable FACET-host kind to `OVERLAY_INDEX_VS_POP`
	// (OVERLAY_INDEX_VS_POP); pairs with that kind as the two streamable Facet overlay
	// kinds — a viz developer requesting both together gets two parallel
	// series layers from a single Facet pass.
	//
	// Math:
	//
	//   Discrete host: subset_freq = host.Discrete.Values[v].Count /
	//     sum(host.Discrete.Values[*].Count); pop_freq =
	//     FacetPopulationView.DiscreteFrequency(v); sd_pop =
	//     FacetPopulationView.DiscreteFrequencyStdev() (population SD
	//     across the population's per-category frequencies — Welford-
	//     Pébaÿ accumulator already used by FacetSchema, surfaced by the
	//     resolver via DiscreteCounts() walk).
	//
	//   Numeric host: subset_val and pop_mean / pop_sd come from the
	//     respective FacetNumeric.Mean / StdDev slots. The per-bin path
	//     would require a histogram on both host and pop and matches the
	//     INDEX_VS_POP numeric shape — but unlike INDEX_VS_POP which
	//     reads bin frequencies, ZSCORE_VS_POP operates on the values
	//     themselves so the numeric path reads the Welford summary
	//     directly. Each bin's center value standardised against the
	//     population's Welford (mean, sd) producing the per-bin z-score.
	//
	// On `sd_pop == 0`: emit ONE warning code `PULSE_OVERLAY_REF_ZERO`
	// per affected entry and SKIP the z-score entry (Statistic stays
	// unset on that entry — "present slot, empty summary" shape).
	// Mirrors the INDEX_VS_POP zero-pop_freq contract.
	//
	// Absent-population path: when the resolver could not find any
	// population entry for `value` (the value never appeared in the
	// population dictionary), the handler treats it as the zero-pop_freq
	// case and emits the same warning + skip behavior (mirrors
	// INDEX_VS_POP).
	//
	// Ref handling: `Ref.Population` MUST be populated (mirrors
	// INDEX_VS_POP). Any other ref-family pointer (`Margin` / `Sibling` /
	// `BaselineIndex` / `Prior` / `RollingMean` / `YoY` / `Stage` /
	// `Slot`) fires `PULSE_OVERLAY_REF_INCOMPATIBLE_WITH_SHAPE` at
	// predict time (per-kind validator lives in descriptor/overlay_facet.go).
	//
	// Scope must be GROUP. `Level` / `Within` MUST be zero (population
	// comparison is a single-value lookup, not an axis prefix); non-zero
	// values fire `PULSE_OVERLAY_LEVEL_OUT_OF_RANGE` mirroring the
	// implicit-ref family.
	//
	// Streamable. Per `types/overlay_streamability.go`, the streamability
	// row is `true` — the handler runs as a post-finalize fold over the
	// host's already-materialized per-value distribution + the resolver's
	// already-materialized population view (sd_pop derives from the
	// Welford accumulator FacetSchema already folded for the population
	// cohort). The host-side streaming Facet pass keeps its existing
	// online accumulators (Welford trio / per-value count map); the
	// overlay does NOT widen the streaming carrier — it consumes
	// finalized state only. Streamable matches the kind-catalog-v1
	// "Streaming-capable subset" — pairs with `OVERLAY_INDEX_VS_POP` as
	// the two streamable Facet kinds.
	//
	// Renderers centre diverging colour ramps on `baseline = 0` (mirrors
	// `OVERLAY_ZSCORE_VS_MARGIN` / `OVERLAY_ZSCORE_VS_TOTAL` /
	// `OVERLAY_ZSCORE_VS_ROLLING` — every z-score family produces a
	// centred distribution, not a ratio).
	OverlayKindZScoreVsPop OverlayKind = "OVERLAY_ZSCORE_VS_POP"

	// OverlayKindZScoreVsMargin emits a per-cell standardized-margin
	// z-score: (cell - margin) / sd where margin is the matching axis
	// margin slot and sd is the population standard deviation of the
	// cell values within the same margin slice. CELL-scoped over a
	// MATRIX (crosstab) host. Unlike the SHARE_OF_* triad (each of
	// which is structurally axis-locked), ZSCORE_VS_MARGIN supports
	// every axis a Margin reference can target (row / column / grand);
	// callers pick the axis explicitly via Ref.Margin.Axis and the
	// handler dispatches the matching slice. First non-ratio overlay
	// in the catalog — output is unitless deviation, not a ratio or
	// percentage. Inherently buffered for the same reason as
	// INDEX_VS_MARGIN — both the host crosstab path and the per-slice
	// Welford recurrence rely on a fully-materialised matrix.
	OverlayKindZScoreVsMargin OverlayKind = "OVERLAY_ZSCORE_VS_MARGIN"

	// OverlayKindYoY emits a per-point year-over-year ratio against the
	// same period one year prior in an ordered SERIES (grouped Process)
	// host whose grouper is `GROUP_DATE`. GROUP scope over a SERIES host
	// with SERIES payload — one `SeriesEntry` per host group key in host
	// order, each carrying the YoY ratio on `Summary.Statistic`. Sixth
	// windowed-Process overlay in the catalog (siblings:
	// `OVERLAY_INDEX_VS_PRIOR`, `OVERLAY_INDEX_VS_BASELINE`,
	// `OVERLAY_DELTA_VS_BASELINE`, `OVERLAY_INDEX_VS_ROLLING_MEAN`,
	// `OVERLAY_ZSCORE_VS_ROLLING`) and the first consumer of the empty
	// `Ref.YoY` marker arm of the OverlayRef discriminated union.
	//
	// Math (per host group ordinal `i`):
	//
	//	frequency = spec.Params["frequency"] OR req.Groups[0].Params["frequency"]
	//	prior_i = lookup for the host series point at "i - <stride>" based on
	//	          frequency:
	//	            annual    ⇒ i - 1
	//	            quarterly ⇒ i - 4
	//	            monthly   ⇒ i - 12
	//	            weekly    ⇒ i - 52
	//	            daily     ⇒ exact-key lookup against host key index at
	//	                        host.Key(i).AddDate(-1, 0, 0); Feb 29 in a
	//	                        non-leap prior year emits NaN (no exact-key
	//	                        match — explicit non-goal: no day-of-week or
	//	                        leap-year realignment in v1).
	//	            hourly    ⇒ exact-key lookup against host key index at
	//	                        host.Key(i).Add(-365*24*time.Hour) with the
	//	                        same exact-key rule.
	//	if !present_at(prior_i)  ⇒ NaN (first year of data is legitimate —
	//	                            "no comparison available" rather than
	//	                            "denominator was zero"; no warning)
	//	if prior_value == 0      ⇒ NaN + PULSE_OVERLAY_REF_ZERO warning
	//	otherwise                ⇒ point_value / prior_value * 100
	//
	// Required GROUP_DATE host grouper: the kind only operates against a
	// SERIES host whose single grouper is `GROUP_DATE`. Other grouper
	// kinds (CATEGORY / RANGE / ROUNDED / QUANTILE / SET_VALUE) cannot
	// resolve the "same period one year prior" semantics — predict
	// (`descriptor.validateOverlayYoY`) and runtime (the handler's host
	// introspection arm via `SeriesHostView.GrouperKind`) both reject
	// non-DATE hosts with `PULSE_OVERLAY_REF_INCOMPATIBLE_WITH_SHAPE`.
	//
	// Required `frequency` Param: the kind cannot infer the correct prior-
	// period stride from the GROUP_DATE `component` slot alone because
	// the per-component stride for "one year prior" varies by component
	// (annual ⇒ 1, quarterly ⇒ 4, monthly ⇒ 12, weekly ⇒ 52, daily ⇒
	// 365-day calendar arithmetic, hourly ⇒ 365×24-hour arithmetic). The
	// handler reads the explicit `frequency` value from
	// `spec.Params["frequency"]` first (the YoY's own override) and falls
	// back to `req.Groups[0].Params["frequency"]` (the canonical GROUP_DATE
	// authoring slot). Missing both fires
	// `PULSE_OVERLAY_YOY_FREQUENCY_MISSING` with `{kind, host_grouper}`
	// Details at both predict and runtime. Outside the supported set
	// (`annual` | `quarterly` | `monthly` | `weekly` | `daily` | `hourly`)
	// fires `PULSE_OVERLAY_YOY_INCOMPATIBLE_FREQUENCY` with
	// `{frequency, supported}` Details.
	//
	// Calendar-week / day-of-week realignment is an explicit non-goal in
	// v1: weekly frequency uses calendar-week-aligned `i - 52` arithmetic
	// (no day-of-week realignment); daily frequency uses exact-key lookup
	// against the host key index after subtracting one year via
	// `time.Time.AddDate(-1, 0, 0)` (Feb 29 in a non-leap prior year
	// emits NaN — no exact-key match). The runtime documents both rules
	// explicitly so callers do not silently get realigned results.
	//
	// Absent-point policy: a host that did not produce a value for group
	// `i` (the resolver returns `(0, false)`) surfaces a `SeriesEntry`
	// whose `Summary` leaves `Statistic` unset — the canonical "present
	// slot, empty summary" shape from the SERIES dispatch contract.
	// Absent groups do NOT participate in the YoY computation. The first
	// year of data (every ordinal whose prior-year index lands at < 0 OR
	// whose daily/hourly prior-year date does not match an exact host
	// key) emits NaN without warning — "no comparison available" is
	// structurally distinct from "denominator was zero" (mirrors the
	// `OVERLAY_INDEX_VS_PRIOR` first-point NaN rule).
	//
	// Zero-prior path: when the resolved prior value is `0` at the divide
	// step (legitimate prior point with a zero post-filter sum) the
	// handler emits ONE `PULSE_OVERLAY_REF_ZERO` warning per layer and
	// surfaces NaN on the affected entry. Mirrors the share / index /
	// zscore family's zero-denominator contract (one warning per layer,
	// not per cell).
	//
	// Ref handling: `Ref.YoY` MUST be populated (empty marker is fine —
	// the v1 frequency value lives on Params per the WIN_* operator
	// convention). Any other ref-family pointer (`Margin` / `Sibling` /
	// `BaselineIndex` / `Prior` / `RollingMean` / `Population` / `Stage` /
	// `Slot`) fires `PULSE_OVERLAY_REF_INCOMPATIBLE_WITH_SHAPE`.
	//
	// Scope must be GROUP. `Level` / `Within` MUST be zero (windowed
	// kind — the prior-period lookup folds across the ordered axis
	// without a prefix-bucket denominator); non-zero values fire
	// `PULSE_OVERLAY_LEVEL_OUT_OF_RANGE` mirroring the
	// `INDEX_VS_PRIOR` / `INDEX_VS_BASELINE` / `INDEX_VS_ROLLING_MEAN`
	// family.
	//
	// Buffered. Per `types/overlay_streamability.go`, the streamability
	// row is `false` — the prior-period lookup requires the materialised
	// host series (daily / hourly arms walk an exact-key index over the
	// full host key list; coarser frequencies index into an arbitrary
	// prior ordinal). The handler runs at the buffered post-host-
	// finalize exit via `ApplyOverlaysSeries`. Forward-compat: a future
	// story may lift the per-frequency lookup into a streaming-aware
	// shape when the streaming-Process orchestrator carries an exact-key
	// host index inline.
	//
	// Renderers centre diverging colour ramps on `baseline = 100`
	// (mirrors `OVERLAY_INDEX_VS_MARGIN` / `OVERLAY_INDEX_VS_TOTAL` /
	// `OVERLAY_INDEX_VS_SIBLING` / `OVERLAY_INDEX_VS_PRIOR` /
	// `OVERLAY_INDEX_VS_BASELINE` / `OVERLAY_INDEX_VS_ROLLING_MEAN`).
	OverlayKindYoY OverlayKind = "OVERLAY_YOY"

	// OverlayKindZScoreVsTotal emits a per-group standardized z-score
	// against the host series' grand-total distribution: for each host
	// group key the layer surfaces `(group_val - mean) / sd` where
	// `mean = Σ_j group_val[j] / N` and `sd = sqrt(M2 / N)` (population
	// variance) folded across the N present per-group aggregated values.
	// GROUP scope over a SERIES (grouped Process) host with SERIES
	// payload — one SeriesEntry per host group key in host order, each
	// carrying the z-score on `Summary.Statistic`. Third and final
	// streamable overlay in the grouped-Process subset (sibling to
	// OVERLAY_INDEX_VS_TOTAL and the SERIES dispatch of
	// OVERLAY_SHARE_OF_TOTAL): the Welford accumulator is one (count,
	// mean, M2) triple carried alongside the per-group accumulators
	// inside the streaming Process fold — no second pass over records,
	// the post-host finalize emits `(group_val - mean) / sd` per group.
	//
	// Math (per host group i):
	//
	//	count       = number of present per-group values
	//	mean        = Σ_j group_val[j] / count       (Welford recurrence,
	//	                                              single-pass)
	//	M2          = Σ_j (group_val[j] - mean)²    (Welford accumulator)
	//	sd          = sqrt(M2 / count)               (population SD)
	//	zscore_i    = (group_val[i] - mean) / sd
	//
	// Variance choice (population, not sample): the kind name says
	// `_VS_TOTAL` which implies the host's per-group aggregation set IS
	// the whole population we are standardising against — we are not
	// inferring a wider population from a sample of groups, we are
	// standardising every present group against the entire observed
	// distribution. Population variance (divide by N, not N-1) is the
	// correct convention; the sibling buffered `ATTR_ZSCORE` attribute
	// reuses the same convention against raw records. Reuses the same
	// numerical convention (single-pass Welford-Pébaÿ) as the parallel
	// buffered Process path and the crosstab ZSCORE_VS_MARGIN handler so
	// cross-mode equivalence tests stay byte-equal within ULP.
	//
	// Zero-variance path: when `sd == 0` (every present group value is
	// equal, including the every-group-zero degenerate case and the
	// single-present-group case) the handler emits ONE
	// PULSE_OVERLAY_REF_ZERO warning and populates every entry's
	// `Summary.Statistic` with NaN. Mirrors the existing
	// PULSE_OVERLAY_REF_ZERO contract used by the share / index family
	// against a missing axis margin on the MATRIX host AND the sibling
	// SERIES kinds against a zero grand total.
	//
	// Absent-group policy: a host that did not produce a record for
	// group i (the resolver returns (0, false)) surfaces a SeriesEntry
	// whose Summary leaves Statistic unset — the canonical "present
	// slot, empty summary" shape established by the SERIES
	// dispatch contract. Absent groups do NOT contribute to the
	// Welford accumulator (the resolver gates the fold the same way it
	// gates per-group emission, identical to INDEX_VS_TOTAL).
	//
	// Ref handling: implicit-grand-total. The Ref union is left EMPTY —
	// the kind's centerpoint is the host series' own grand-total
	// distribution (mean), so callers supplying any Ref family pointer
	// (Margin / Sibling / BaselineIndex / Population / Stage / Slot)
	// fail PULSE_OVERLAY_REF_INCOMPATIBLE_WITH_SHAPE at predict time
	// (mirrors INDEX_VS_TOTAL / SHARE_OF_TOTAL SERIES contract).
	//
	// Scope is GROUP (not CELL or ROW): the kind decorates each grouper
	// level the host emits. The validator rejects any other scope.
	//
	// Streamable. Per types/overlay_streamability.go, the streamability
	// row is `true` — the Welford accumulator (count + mean + M2) is
	// three f64s carried alongside the per-group accumulators inside
	// the streaming Process fold, and the standardisation step happens
	// at host finalize. The streaming-Process orchestrator wires the
	// inner per-record hook; this kind ships with a SERIES-host
	// post-finalize entry (ApplyOverlaysSeries route) so the streaming-
	// vs-buffered byte-identity test holds at the entry-point level
	// today. Gotcha: the streaming pass
	// folds Welford over GROUPS, not raw records — variance is across
	// N groups, not N records, distinct from `ATTR_ZSCORE`'s record-
	// level semantics.
	OverlayKindZScoreVsTotal OverlayKind = "OVERLAY_ZSCORE_VS_TOTAL"

	// OverlayKindIndexVsRef is the COMPOSE-host dual-shape ratio index
	// of each target slot's value against the matching reference slot
	// value: `(target / ref) * scale` where `scale` defaults to `100`
	// (set `Params["scale"] = 1` for a raw ratio). Dual-shape catalog
	// row — the host-shape disambiguator (matrix vs series) routes the
	// per-coordinate / per-group dispatch:
	//
	//   - MATRIX host: CELL scope over a MATRIX (crosstab) host
	//     with MATRIX payload. Per-cell ratio at `(rowKey, colKey)`.
	//   - SERIES host: GROUP scope over a SERIES (grouped Process)
	//     host with SERIES payload. Per-group ratio at the host's
	//     group key.
	//
	// First COMPOSE-only crosstab-shape overlay kind in the catalog
	// and the first kind to consume the
	// `ComposeOverlaySpec.Reference` / `Targets` slot-label pair across
	// both MATRIX and SERIES slots. Sibling kind to
	// `OVERLAY_DELTA_VS_REF` (subtractive twin sharing the SERIES arm).
	//
	// Math (per host coordinate `(i, j)` for MATRIX, `i` for SERIES):
	//
	//	target_val = target_slot.<MatrixCell|SeriesValue>(i, j)
	//	ref_val    = ref_slot.<MatrixCell|SeriesValue>(i, j)
	//	scale      = Params["scale"] (default 100, accepts 1 for raw ratio)
	//	if ref_val == 0 ⇒ NaN + PULSE_OVERLAY_REF_ZERO warning
	//	otherwise       ⇒ (target / ref) * scale
	//
	// Streamable. Per `types/overlay_streamability.go` the row is
	// `true` — the SERIES dispatch is fold-only (single accumulator per
	// group; no peer-cell lookup) and matches the kind-catalog-v1
	// "Streaming-capable subset". The MATRIX dispatch remains forced
	// buffered by the slot barrier in `service.Compose` /
	// `service.ComposeParallel`; the flag describes the kind's
	// INTRINSIC streaming capability (mirrors the
	// `OVERLAY_SHARE_OF_TOTAL` dual-shape convention — MATRIX-arm
	// routes through `canFuseCrosstab`'s overlays-force-buffered arm,
	// SERIES-arm rides the streaming Process pass). Renderers centre
	// diverging colour ramps on `baseline = scale` (default `100`
	// mirrors the per-Request `INDEX_VS_*` family).
	OverlayKindIndexVsRef OverlayKind = "OVERLAY_INDEX_VS_REF"

	// OverlayKindDeltaVsRef is the COMPOSE-host dual-shape additive
	// delta of each target slot's value against the matching reference
	// slot value: `target - ref`. Dual-shape catalog row — the
	// host-shape disambiguator (matrix vs series) routes the
	// per-coordinate / per-group dispatch:
	//
	//   - MATRIX host: CELL scope over a MATRIX (crosstab) host
	//     with MATRIX payload. Per-cell delta at `(rowKey, colKey)`.
	//   - SERIES host: GROUP scope over a SERIES (grouped Process)
	//     host with SERIES payload. Per-group delta at the host's
	//     group key.
	//
	// Subtractive twin of `OVERLAY_INDEX_VS_REF`; shares the same
	// reference / target resolution pipeline and the same
	// key-alignment / schema-match / dict-prefix gates.
	//
	// Math (per host coordinate `(i, j)` for MATRIX, `i` for SERIES):
	//
	//	target_val = target_slot.<MatrixCell|SeriesValue>(i, j)
	//	ref_val    = ref_slot.<MatrixCell|SeriesValue>(i, j)
	//	delta      = target_val - ref_val
	//
	// Unlike the `OVERLAY_INDEX_VS_REF` sibling (which divides by the
	// reference value and emits `PULSE_OVERLAY_REF_ZERO` against a zero
	// denominator), `OVERLAY_DELTA_VS_REF` performs subtraction and is
	// mathematically defined for every finite reference value including
	// zero — the handler does NOT emit `PULSE_OVERLAY_REF_ZERO` on zero
	// references. Mirrors the per-Request DELTA family rule
	// (`OVERLAY_DELTA_VS_MARGIN` / `OVERLAY_DELTA_VS_BASELINE` /
	// `OVERLAY_DELTA_VS_SIBLING`). Missing reference values (target
	// carries a key the reference did not surface) still emit a
	// `PULSE_OVERLAY_REF_ZERO` with a `ref_missing=true` Detail flag —
	// the same missing-key warning shape the MATRIX arm uses.
	//
	// Output preserves the target slot's units — a $-valued AGG_SUM
	// target cell minus a $-valued reference cell yields a $-valued
	// deviation in the same currency. Renderers centre diverging colour
	// ramps on `baseline = 0` (mirrors the rest of the DELTA family on
	// other host shapes).
	//
	// Streamable. Per `types/overlay_streamability.go` the row is
	// `true` — the SERIES dispatch is fold-only (single accumulator
	// per group; no peer-cell lookup) and matches the kind-catalog-v1
	// "Streaming-capable subset". The MATRIX dispatch remains forced
	// buffered by the slot barrier in `service.Compose` /
	// `service.ComposeParallel`; the flag describes the kind's
	// INTRINSIC streaming capability (mirrors the
	// `OVERLAY_INDEX_VS_REF` dual-shape convention).
	OverlayKindDeltaVsRef OverlayKind = "OVERLAY_DELTA_VS_REF"

	// OverlayKindPropZCell is the COMPOSE-host per-cell two-proportion
	// z-test against the reference slot's matching cell. CELL scope over
	// a MATRIX (crosstab) host with MATRIX payload — each cell's value
	// is the two-sided p-value as a float64. Cells are treated as
	// success counts; the reference cell's matching row margin is
	// treated as the sample size on the reference side, and the target
	// cell's matching row margin is treated as the sample size on the
	// target side. The two-proportion z-test reuses the same pooled-
	// standard-error formula that backs `TEST_PROP_Z` (see
	// `processing/test_propz.go`):
	//
	//	p_target = target_cell / target_row_margin
	//	p_ref    = ref_cell    / ref_row_margin
	//	pooled   = (target_cell + ref_cell) / (target_row_margin + ref_row_margin)
	//	se       = sqrt(pooled * (1 - pooled) * (1/n_target + 1/n_ref))
	//	z        = (p_target - p_ref) / se
	//	p_value  = 2 * (1 - Φ(|z|))
	//
	// Reuses the `standardNormalCDF` helper backing `TEST_PROP_Z` so the
	// overlay and the row-test surface produce identical p-values for
	// the same (success, n) pair.
	//
	// Degenerate inputs (pooled = 0 or 1, missing row margin, etc.)
	// produce NaN p-values with a `PULSE_OVERLAY_REF_ZERO` warning per
	// affected cell carrying the offending diagnostic. Mirrors the
	// `OVERLAY_FISHER_EXACT_CELL` cell-level warning emission shape.
	//
	// Buffered. Inferential overlays as a family stay buffered until a
	// streamable-test path is plumbed (PRD §2 Non-Goals "Streaming
	// overlay path for inferential kinds").
	OverlayKindPropZCell OverlayKind = "OVERLAY_PROP_Z_CELL"

	// OverlayKindPropZPanel is the COMPOSE-host multi-reference per-cell
	// pairwise two-proportion z-test across N + 1 slots (the reference
	// slot plus every target slot). CELL scope over a MATRIX (crosstab)
	// host with MATRIX payload — each cell's `Value` is NOT a scalar
	// p-value but the flattened upper-triangular slice of pairwise
	// two-sided p-values across the panel's N + 1 slots. First multi-ref
	// COMPOSE-only kind in the catalog and the first kind to
	// emit a per-cell vector payload via `MatrixCell.Value` carrying a
	// `[]float64` (the scalar `MatrixCell.Value any` slot accepts the
	// vector shape verbatim — same precedent the RichAggregator family
	// uses for map / slice cell values).
	//
	// Slot ordering: the panel ordering is `{reference, targets[0],
	// targets[1], ..., targets[N-1]}` — index 0 is the reference, every
	// subsequent index is one target slot in `ComposeOverlaySpec.Targets`
	// order. The handler reads each slot's matching cell at `(rowKey,
	// colKey)` and folds N + 1 (count, total) pairs into the pairwise
	// matrix.
	//
	// Output shape — flattened upper-triangular slice (row-major, NO
	// diagonal): for an `M = N + 1` slot panel, the per-cell `[]float64`
	// carries the pairs in canonical `(i, j)` order with `i < j`:
	//
	//	[(0,1), (0,2), ..., (0,M-1), (1,2), ..., (1,M-1), ..., (M-2,M-1)]
	//
	// The slice length is `M * (M - 1) / 2`. Index helper:
	//
	//	pairIndex(i, j, M) = i * (2*M - i - 1) / 2 + (j - i - 1)
	//
	// The diagonal is implicit (every diagonal entry is `1.0` —
	// self-vs-self) and the lower triangular is implicit (`p[j, i] ==
	// p[i, j]` — symmetric matrix). Renderers reconstruct the dense
	// matrix from the flattened slice via the pair index helper.
	// Documented in `skills/overlay-system.md`.
	//
	// Math (per pair `(i, j)`): the same pooled-SE two-proportion z-test
	// formula `OVERLAY_PROP_Z_CELL` uses via the shared
	// `twoProportionZ` helper. Each slot's matching row margin is its
	// sample size; the helper accepts `(success_i, n_i, success_j, n_j)`
	// and returns the two-sided p-value. Reuses the same
	// `standardNormalCDF` helper backing `TEST_PROP_Z` so every pairwise
	// p-value matches the corresponding `OVERLAY_PROP_Z_CELL` output
	// byte-for-byte.
	//
	// Cap enforcement — `OverlayOptions.MaxPanelTargets`: panel
	// combinatorics scale O(N²) per cell, so the handler enforces a
	// caller-attested cap on the number of target slots. The cap
	// defaults to `16` when `ComposeOverlaySpec.Options` is nil OR
	// `Options.MaxPanelTargets == 0`. `len(spec.Targets) >
	// MaxPanelTargets` fires `PULSE_OVERLAY_PANEL_TARGETS_OVER_CAP` at
	// handler entry with `{observed, cap}` Details. Per the interview
	// risk paragraph "Multi-reference combinatorics", bumping the
	// default 16 requires explicit interview update — `Options` is the
	// per-request override surface for callers who need a larger panel
	// today.
	//
	// Degenerate inputs: missing row margins fall back to the cell value
	// as the sample size (mirrors `OVERLAY_PROP_Z_CELL`); pairwise
	// `(pooled ∈ {0, 1}, se == 0)` produces NaN at the affected pair
	// position with one `PULSE_OVERLAY_REF_ZERO` warning per (cell,
	// pair) tuple. Cells where the reference value is absent on any
	// slot surface an empty (nil) per-cell vector with one
	// `PULSE_OVERLAY_REF_ZERO` warning carrying `ref_missing: true`.
	//
	// Buffered. Inferential overlays as a family stay buffered until a
	// streamable-test path is plumbed (PRD §2 Non-Goals).
	OverlayKindPropZPanel OverlayKind = "OVERLAY_PROP_Z_PANEL"

	// OverlayKindPairwiseProbitT is the intra-matrix axis-pairwise probit
	// t-test (syndicated meaningfulness / uniqueness family). Per opposite-
	// axis position it transforms each leg's proportion through Φ⁻¹ and
	// tests the difference: t = (Φ⁻¹(p_i) - Φ⁻¹(p_j)) / sqrt(1/n_i + 1/n_j)
	// against Student-t with df = n_i + n_j - 2, two-sided. Reuses the
	// Student-t survival helper backing TEST_T. Reads proportion + n per
	// leg (p_source / n_source). See the OVERLAY_PAIRWISE_* family note.
	OverlayKindPairwiseProbitT OverlayKind = "OVERLAY_PAIRWISE_PROBIT_T"

	// OverlayKindPairwisePropZ is the intra-matrix axis-pairwise pooled-SE
	// two-proportion z-test (custom / A&U / syndicated top2box family). Per
	// opposite-axis position it tests leg i vs leg j via the same
	// twoProportionZ helper backing OVERLAY_PROP_Z_CELL, so p-values match
	// byte-for-byte. Reads proportion + n per leg (p_source / n_source).
	// See the OVERLAY_PAIRWISE_* family note.
	OverlayKindPairwisePropZ OverlayKind = "OVERLAY_PAIRWISE_PROP_Z"

	// OverlayKindPairwiseTwoMeansZ is the intra-matrix axis-pairwise
	// two-means z-test on AGG_WELFORD cells (AU Q137/Q140/Q141 family). Per
	// opposite-axis position it tests mean_i vs mean_j using the Welford
	// triple {mean, variance, n} on CellComponents with a normal-CDF tail
	// (no df adjustment): z = (m_i - m_j) / sqrt(v_i/n_i + v_j/n_j). Reuses
	// standardNormalCDF. A non-Welford host fails fast with
	// PULSE_OVERLAY_SHAPE_MISMATCH. See the OVERLAY_PAIRWISE_* family note.
	OverlayKindPairwiseTwoMeansZ OverlayKind = "OVERLAY_PAIRWISE_TWO_MEANS_Z"

	// OverlayKindPairwiseWelchT is the intra-matrix axis-pairwise
	// Welch–Satterthwaite t-test on AGG_WELFORD cells (AU Q99 family). Per
	// opposite-axis position it tests mean_i vs mean_j using the Welford
	// triple {mean, variance, n} with the Welch SE and Satterthwaite df,
	// two-sided via the Student-t survival helper backing TEST_T /
	// TEST_WELCH. A non-Welford host fails fast with
	// PULSE_OVERLAY_SHAPE_MISMATCH. See the OVERLAY_PAIRWISE_* family note.
	OverlayKindPairwiseWelchT OverlayKind = "OVERLAY_PAIRWISE_WELCH_T"

	// OverlayKindPanelIndexVsRef is the COMPOSE-host multi-reference
	// dual-shape ratio index — indexes EVERY target slot against the
	// SHARED reference slot and emits ONE OverlayLayer per target. The
	// chassis treats the spec as a panel container: every emitted layer
	// shares the reference slot's row/col coord space (matrix host) or
	// group key set (series host) so renderers can lay the panel on a
	// single co-ordinate canvas. Second multi-reference COMPOSE-only kind
	// in the catalog (sibling: OVERLAY_PROP_Z_PANEL)
	// and the first kind to emit MULTIPLE OverlayLayer entries from a
	// single ComposeOverlaySpec — the chassis dispatch widens to
	// `[]types.OverlayLayer` per spec via the multi-layer handler
	// surface added alongside this kind.
	//
	// Output layout (one layer per target):
	//
	//	layers[i].Name  = "<spec.Name>__<spec.Targets[i]>"
	//	layers[i].Kind  = OVERLAY_PANEL_INDEX_VS_REF
	//	layers[i].Scope = spec.Scope
	//	layers[i].Payload mirrors the reference slot's host shape
	//	  - MATRIX host: RowKeys / ColumnKeys cloned from the reference
	//	    matrix (the schema-match + key-alignment gates guarantee every
	//	    target shares the same coord space, so the reference is the
	//	    canonical anchor — mirrors the single-target INDEX_VS_REF /
	//	    PROP_Z_PANEL precedent).
	//	  - SERIES host: SeriesPayload whose Entries match the target's
	//	    Response.Data row order element-for-element (modulo missing-
	//	    ref drops).
	//
	// Spec.Name fallback: when `spec.Name == ""` the layer name template
	// degenerates to `"OVERLAY_PANEL_INDEX_VS_REF__<target_label>"` so
	// renderers can address every panel target by a stable handle even
	// against an unnamed authoring shape (mirrors the
	// `composeOverlayLayerName` fallback rule on the rest of the
	// COMPOSE catalog).
	//
	// Math: each emitted layer's per-coordinate value is mathematically
	// equivalent to OVERLAY_INDEX_VS_REF for the same (reference,
	// target[i]) pair — the handler reuses the single-target
	// `applyIndexVsRef` math arm verbatim, then renames the resulting
	// layer per the template above. This guarantees per-target byte-
	// identity with the single-pair OVERLAY_INDEX_VS_REF output for the
	// same slots.
	//
	// Cap enforcement — `OverlayOptions.MaxPanelTargets`: panel
	// combinatorics scale O(N) per cell (one independent ratio per
	// target) so the cap is informational rather than asymptotic — but
	// the catalog standardises on the same cap surface
	// OVERLAY_PROP_Z_PANEL uses so renderers can budget panel-layout
	// state against a single knob. Default 16 when `Options` is nil or
	// `Options.MaxPanelTargets == 0`. `len(spec.Targets) >
	// MaxPanelTargets` fires `PULSE_OVERLAY_PANEL_TARGETS_OVER_CAP` at
	// handler entry with `{kind, observed, cap}` Details.
	//
	// Layer slice order: spec order then target order — layers[i]
	// corresponds to spec.Targets[i], so renderers can address the
	// emitted panel by target offset without any auxiliary index map.
	// Stable across re-runs of the same spec.
	//
	// Shared coord space: the schema-match and key-alignment
	// gates already enforce coord-space identity across every
	// (reference, target) pair; the handler relies on that invariant
	// rather than re-validating per emission. A divergence at any pair
	// fails the chassis-side gate BEFORE the handler dispatches so the
	// emitted panel always shares one coord space.
	//
	// Streamable. Per `types/overlay_streamability.go` the row is `true`
	// — each emitted layer reuses the INDEX_VS_REF arithmetic (a
	// fold-only handler) so the per-target SERIES emission is fold-only
	// and the MATRIX emission rides through the slot barrier identically
	// to its single-target sibling. Mirrors the
	// `OVERLAY_INDEX_VS_REF` dual-shape streamability convention.
	OverlayKindPanelIndexVsRef OverlayKind = "OVERLAY_PANEL_INDEX_VS_REF"

	// OverlayKindTCell is the COMPOSE-host per-cell Welch t-test
	// against the reference slot's matching cell. CELL scope over a
	// MATRIX (crosstab) host with MATRIX payload — each cell's value is
	// the two-sided p-value as a float64. Each cell is treated as a
	// sample mean; the variance defaults to `1.0` and the sample size
	// defaults to `Params["sample_size"]` (or `2` when not supplied).
	// The Welch t-test reuses the same Welch-Satterthwaite degrees-of-
	// freedom recurrence that backs `TEST_T` two-sample (see
	// `processing/test_t.go`):
	//
	//	se      = sqrt(var_target/n_target + var_ref/n_ref)
	//	t       = (target_cell - ref_cell) / se
	//	df      = (var_target/n_target + var_ref/n_ref)^2 /
	//	          ( (var_target/n_target)^2 / (n_target-1)
	//	          + (var_ref/n_ref)^2     / (n_ref-1)     )
	//	p_value = studentTTwoSidedP(t, df)
	//
	// Reuses the `studentTTwoSidedP` helper backing `TEST_T` so the
	// overlay and the row-test surface produce identical p-values for
	// the same (mean, variance, n) triple.
	//
	// Default-variance and default-sample-size policy: v1 ships with
	// `var = 1.0` and `n = 2` defaults so the per-cell handler stays
	// well-defined against a minimal Compose authoring surface. Callers
	// who want a real Welch test should supply
	// `Params["variance_target"]`, `Params["variance_ref"]`,
	// `Params["sample_size_target"]`, `Params["sample_size_ref"]`.
	// Forward-compat: future work may lift the per-cell `(mean,
	// variance, n)` triple into a richer Compose authoring surface (the
	// crosstab cell carrier could grow to carry sample statistics
	// alongside the scalar mean); when that lands the defaults become
	// opt-in fallbacks rather than universal defaults.
	//
	// Buffered. Same rationale as `OVERLAY_PROP_Z_CELL`.
	OverlayKindTCell OverlayKind = "OVERLAY_T_CELL"

	// OverlayKindZCell is the COMPOSE-host per-cell two-sample z-test on
	// the means against the reference slot's matching cell. CELL scope
	// over a MATRIX (crosstab) host with MATRIX payload — each cell's
	// value is the two-sided p-value as a float64. Each cell is treated
	// as a sample mean; the variance defaults to `1.0` and the sample
	// size defaults to `Params["sample_size"]` (or `2` when not supplied).
	// The z-test reuses the same Welch-style standard-error recurrence
	// that backs `OVERLAY_T_CELL` and finalises against the standard
	// normal survival helper that backs `TEST_Z_TWO_SAMPLE` (see
	// `processing/test_z.go`):
	//
	//	se      = sqrt(var_target/n_target + var_ref/n_ref)
	//	z       = (target_cell - ref_cell) / se
	//	p_value = 2 * (1 - Φ(|z|))
	//
	// Reuses the `standardNormalCDF` helper backing `TEST_Z_TWO_SAMPLE` so
	// the overlay and the row-test surface produce identical p-values for
	// the same (mean, variance, n) triple.
	//
	// Default-variance and default-sample-size policy: v1 ships with
	// `var = 1.0` and `n = 2` defaults so the per-cell handler stays
	// well-defined against a minimal Compose authoring surface. Callers
	// who want a real z-test should supply
	// `Params["variance_target"]`, `Params["variance_ref"]`,
	// `Params["sample_size_target"]`, `Params["sample_size_ref"]`.
	// Forward-compat: future work may lift the per-cell `(mean,
	// variance, n)` triple into a richer Compose authoring surface (the
	// crosstab cell carrier could grow to carry sample statistics
	// alongside the scalar mean); when that lands the defaults become
	// opt-in fallbacks rather than universal defaults.
	//
	// Buffered. Same rationale as `OVERLAY_PROP_Z_CELL`.
	OverlayKindZCell OverlayKind = "OVERLAY_Z_CELL"

	// OverlayKindChiSqVsRef is the COMPOSE-host whole-matrix χ² test
	// against the reference slot's matrix. The two matrices are treated
	// as two independent contingency tables and the test answers "do
	// the target and reference distributions differ?" via the standard
	// two-sample χ² approach: target cells are observed; reference
	// cells (scaled to target N) are expected. SCALAR payload — the
	// layer carries a single p-value on `OverlayPayload.Scalar` plus an
	// `OverlaySummary{Statistic, PValue, Parameters{"df"}}` summary.
	//
	// Math:
	//
	//	target_N  = sum(target_cells)
	//	ref_N     = sum(ref_cells)
	//	expected  = ref_cell * (target_N / ref_N)  // ref distribution scaled to target N
	//	chisq     = Σ (target_cell - expected)² / expected
	//	df        = (target cells with expected > 0) - 1
	//	p_value   = chiSquareSurvival(chisq, df)
	//
	// Reuses the `chiSquareSurvival` helper backing `TEST_CHISQ` and the
	// MATRIX-host CHISQ family so the overlay and the row-test surface
	// produce identical p-values for the same contingency.
	//
	// Degenerate inputs (`target_N == 0`, `ref_N == 0`, every expected
	// = 0) emit a NaN statistic + NaN p-value with one
	// `PULSE_OVERLAY_REF_ZERO` warning. Low-expected-cell warning
	// follows the canonical χ² rule the rest of the CHISQ family uses
	// (any `expected < 5` → one `PULSE_OVERLAY_EXPECTED_LOW` warning).
	//
	// Buffered. Inferential overlays as a family stay buffered until a
	// streamable-test path is plumbed.
	OverlayKindChiSqVsRef OverlayKind = "OVERLAY_CHISQ_VS_REF"

	// OverlayKindTVsRef is the COMPOSE-host per-group Welch t-test
	// against the reference slot's matching group, against a SERIES host
	// (grouped Process result). GROUP scope over a SERIES host with
	// SERIES payload — one `SeriesEntry` per host group key in host
	// order, each carrying the p-value on `Summary.Statistic`. Series-
	// shape sibling of `OVERLAY_T_CELL` and twin to the SERIES
	// arm of `OVERLAY_INDEX_VS_REF` / `OVERLAY_DELTA_VS_REF`. Distinct
	// kind from `OVERLAY_T_CELL` because the host shape disambiguator
	// (series vs matrix) and the per-group dispatch differ — the
	// renderer surfaces this kind as a per-group inferential strip,
	// not a per-cell badge.
	//
	// Math (per host group `i`):
	//
	//	target_val = target_slot.SeriesValue(i)
	//	ref_val    = ref_slot.SeriesValue(i)
	//	var_target = Params["variance_target"] (default 1.0)
	//	var_ref    = Params["variance_ref"]    (default 1.0)
	//	n_target   = Params["sample_size_target"] (default 2)
	//	n_ref      = Params["sample_size_ref"]    (default 2)
	//	se         = sqrt(var_target/n_target + var_ref/n_ref)
	//	t          = (target_val - ref_val) / se
	//	df         = (var_target/n_target + var_ref/n_ref)^2 /
	//	             ((var_target/n_target)^2/(n_target-1) +
	//	              (var_ref/n_ref)^2/(n_ref-1))
	//	p_value    = studentTTwoSidedP(t, df)
	//
	// Reuses the `studentTTwoSidedP` helper backing `TEST_T` so the
	// overlay and the row-test surface produce identical p-values for
	// the same (mean, variance, n) triple. Mirrors `OVERLAY_T_CELL`'s
	// default-variance and default-sample-size policy verbatim — v1
	// ships well-defined defaults so the per-group handler stays usable
	// against minimal Compose authoring surfaces.
	//
	// Missing reference rows (target row key not present in the
	// reference series) emit `PULSE_OVERLAY_REF_ZERO` with a
	// `ref_missing=true` Detail flag; the affected entry's Statistic is
	// NaN. Degenerate inputs (`se == 0`, n < 2) produce NaN p-values
	// with the same warning.
	//
	// Buffered. Inferential overlays as a family stay buffered until a
	// streamable-test path is plumbed (PRD §2 Non-Goals "Streaming
	// overlay path for inferential kinds"). Even though the SERIES
	// host carrying the Welch t arms may individually stream, the
	// inferential family is buffered as a matter of policy.
	OverlayKindTVsRef OverlayKind = "OVERLAY_T_VS_REF"

	// OverlayKindZVsRef is the COMPOSE-host per-group two-sample z-test on
	// the means against the reference slot's matching group, against a
	// SERIES host (grouped Process result). GROUP scope over a SERIES host
	// with SERIES payload — one `SeriesEntry` per host group key in host
	// order, each carrying the p-value on `Summary.Statistic`. Series-
	// shape sibling of `OVERLAY_Z_CELL` and twin to the SERIES
	// arm of `OVERLAY_T_VS_REF` — same fold-and-finalise shape, different
	// distribution (standard normal instead of Student's t). Distinct
	// kind from `OVERLAY_Z_CELL` because the host shape disambiguator
	// (series vs matrix) and the per-group dispatch differ — the
	// renderer surfaces this kind as a per-group inferential strip,
	// not a per-cell badge.
	//
	// Math (per host group `i`):
	//
	//	target_val = target_slot.SeriesValue(i)
	//	ref_val    = ref_slot.SeriesValue(i)
	//	var_target = Params["variance_target"] (default 1.0)
	//	var_ref    = Params["variance_ref"]    (default 1.0)
	//	n_target   = Params["sample_size_target"] (default 2)
	//	n_ref      = Params["sample_size_ref"]    (default 2)
	//	se         = sqrt(var_target/n_target + var_ref/n_ref)
	//	z          = (target_val - ref_val) / se
	//	p_value    = 2 * (1 - Φ(|z|))
	//
	// Reuses the `standardNormalCDF` helper backing `TEST_Z_TWO_SAMPLE` so
	// the overlay and the row-test surface produce identical p-values for
	// the same (mean, variance, n) triple. Mirrors `OVERLAY_Z_CELL`'s
	// default-variance and default-sample-size policy verbatim — v1
	// ships well-defined defaults (`var = 1.0`, `n = 2`) so the per-group
	// handler stays usable against minimal Compose authoring surfaces.
	// Forward-compat: future work may lift the per-group `(mean,
	// variance, n)` triple into a richer Compose authoring surface (the
	// SERIES entry carrier could grow to carry sample statistics
	// alongside the scalar mean); when that lands the defaults become
	// opt-in fallbacks rather than universal defaults.
	//
	// Missing reference rows (target row key not present in the
	// reference series) emit `PULSE_OVERLAY_REF_ZERO` with a
	// `ref_missing=true` Detail flag; the affected entry's Statistic is
	// NaN. Degenerate inputs (`se == 0`, n < 2) produce NaN p-values
	// with the same warning.
	//
	// Buffered. Inferential overlays as a family stay buffered until a
	// streamable-test path is plumbed (PRD §2 Non-Goals "Streaming
	// overlay path for inferential kinds"). Even though the SERIES
	// host carrying the z arms may individually stream, the
	// inferential family is buffered as a matter of policy.
	OverlayKindZVsRef OverlayKind = "OVERLAY_Z_VS_REF"

	// OverlayKindRank is the COMPOSE-host per-cell rank of each target
	// cell within a configurable population per `Params["population"]`
	// (`"row" | "column" | "matrix"`; defaults to `"matrix"`). CELL
	// scope over a MATRIX (crosstab) host with MATRIX payload — each
	// cell's value is the 1-based rank (1 = largest) computed against
	// the population the spec selected.
	//
	// Population modes:
	//
	//   - `"row"`: rank within the cell's own row across all columns.
	//     Ties take the average rank.
	//   - `"column"`: rank within the cell's own column across all rows.
	//   - `"matrix"` (default): rank across every present cell of the
	//     target matrix.
	//
	// The reference slot is required to anchor the resolution + key-set
	// gates but the rank computation itself reads only
	// the target cells — the reference matrix's structural schema is
	// the alignment constraint; its values are not consumed by the rank
	// math. This intentional asymmetry keeps RANK orthogonal to the
	// ref-vs-target comparison family (INDEX / DELTA / PROP_Z / T /
	// CHISQ) while still reusing the same resolution + alignment
	// pipeline.
	//
	// Ties: equal cell values receive the average of their would-be
	// ranks (canonical "average" tie-breaking; matches the convention
	// scipy.stats.rankdata uses by default).
	//
	// Absent cells: a structurally absent target cell stays absent on
	// the overlay; it does NOT participate in the population's
	// denominator (mirrors the absent-cell policy of the rest of the
	// MATRIX-host overlay family). Ranks are computed over PRESENT
	// cells only.
	//
	// Buffered. Same rationale as the rest of the COMPOSE-host kinds.
	OverlayKindRank OverlayKind = "OVERLAY_RANK"
)

func AllOverlayKinds added in v0.19.0

func AllOverlayKinds() []OverlayKind

AllOverlayKinds returns every defined overlay kind in alphabetical order. Mirrors AllAggregationTypes / AllRegressionTypes — the streamability table and per-kind validator iterate this surface so a new kind only needs to be appended here, declared in the constant block above, and have its streamability row added in types/overlay_streamability.go (the TestStreamability_OverlaysKnown gate enforces table completeness).

type OverlayLayer added in v0.19.0

type OverlayLayer struct {
	// Name echoes the renderer-facing label — either the request
	// Name or the synthesised default.
	Name string `json:"name"`

	// Kind echoes the overlay catalog entry that produced this layer.
	Kind OverlayKind `json:"kind"`

	// Scope echoes the spec's scope.
	Scope OverlayScope `json:"scope"`

	// Ref echoes the spec's discriminated reference.
	Ref OverlayRef `json:"ref"`

	// Payload carries the derived numbers.
	Payload OverlayPayload `json:"payload"`

	// Summary carries optional renderer-friendly metadata. Omitted
	// when the layer reported nothing useful.
	Summary *OverlaySummary `json:"summary,omitempty"`

	// Warnings carries per-layer diagnostics emitted by the overlay
	// handler. Additive slot — empty in the per-Request overlay path
	// (existing handlers do not produce warnings via this carrier
	// today). Populated by the Compose-host fold and the
	// Chain-host barrier when the previously-discarded warning
	// slices get wired through. `omitempty` keeps overlay-free
	// marshals byte-identical to the legacy overlay-free shape.
	Warnings []OverlayWarning `json:"warnings,omitempty"`
}

OverlayLayer is the response-side wrapper for one executed overlay spec. Response.Overlays carries one OverlayLayer per Request.Overlays entry in matching order.

type OverlayMarginRef added in v0.19.0

type OverlayMarginRef struct {
	// Axis selects which margin slot is the denominator. Required.
	Axis MarginAxis `json:"axis"`
}

OverlayMarginRef references one of the base result's margin slots. One pointer family of the OverlayRef union; siblings cover sibling cells, baseline indices, population comparisons, multi-stage chain references, and slot lookups.

type OverlayOptions added in v0.19.0

type OverlayOptions struct {
	// DictPrefixFast opts into the byte-equal-prefix fast path for
	// cross-slot categorical dictionary comparison. Default false (the
	// SAFE by-label join). When true, processing.ApplyComposeOverlays
	// runs a per-invocation prefix-equality probe over the reference
	// and every target dictionary; mismatched prefixes fire
	// PULSE_OVERLAY_DICT_PREFIX_DRIFT and the spec is rejected.
	DictPrefixFast bool `json:"dict_prefix_fast,omitempty"`

	// MaxPanelTargets caps the number of target slots a multi-reference
	// COMPOSE-host overlay (today: OVERLAY_PROP_Z_PANEL) will fold into
	// a pairwise output. Default 16 when the slot is zero — per the
	// interview risk paragraph "Multi-reference combinatorics" the
	// default 16 is the explicit upper bound on per-cell pairwise
	// p-value computation; bumping the default requires an interview
	// update. Setting a non-zero value here is the per-request override
	// surface for callers who need a larger panel (or a stricter cap)
	// today. `len(spec.Targets) > MaxPanelTargets` fires
	// PULSE_OVERLAY_PANEL_TARGETS_OVER_CAP at handler entry (and the
	// equivalent predict-time check in descriptor.ValidateComposedRequest
	// via descriptor.ValidateComposedRequest). Non-panel kinds ignore
	// this knob.
	MaxPanelTargets int `json:"max_panel_targets,omitempty"`
}

OverlayOptions is the per-spec optimization knob bag for Compose-only overlays. Carried on ComposeOverlaySpec.Options (omitempty). Default behaviour is the SAFE path on every knob; non-default values opt the runtime into a faster but caller-attested-correct execution mode.

Knobs:

  • DictPrefixFast — opt-in fast path for cross-slot categorical dictionary comparison. Default false (SAFE by-label join: every cell / group key is decoded via the slot's dictionary before comparison and tolerates arbitrary dict reordering). Setting true opts the runtime into direct-index comparison across slots; processing.ApplyComposeOverlays runs a per-invocation prefix- equality probe (checkDictPrefixEquality) over the reference + every target dictionary BEFORE per-kind dispatch; any divergence fires PULSE_OVERLAY_DICT_PREFIX_DRIFT. Probe-validation at `pulse.New` time is a non-goal — cohort pairing is a per-request decision rather than a registration-time one, so the probe runs per ApplyComposeOverlays invocation instead. The fast path produces identical output to the safe path when the probe passes; the probe is the correctness barrier.

Forward-compat: omitempty on every field keeps the canonical hash byte-identical to a pre-OverlayOptions ComposeOverlaySpec when the caller doesn't supply Options at all. New knobs land here under the same omitempty rule.

type OverlayPayload added in v0.19.0

type OverlayPayload struct {
	// Shape declares which of Scalar / Series / Matrix is populated.
	Shape OverlayShape `json:"shape"`

	// Scalar is the single-value payload. Populated when Shape =
	// OverlayShapeScalar.
	Scalar *float64 `json:"scalar,omitempty"`

	// Series is the keys-and-values strip payload. Populated when
	// Shape = OverlayShapeSeries.
	Series *SeriesPayload `json:"series,omitempty"`

	// Matrix is the dense row × column payload. Populated when Shape =
	// OverlayShapeMatrix. Reuses crosstab.MatrixPayload so renderers
	// handle the overlay grid with the same header machinery as the
	// base layer.
	Matrix *MatrixPayload `json:"matrix,omitempty"`
}

OverlayPayload is the discriminated union carrying the actual derived numbers an overlay produced. Exactly one of Scalar / Series / Matrix is meaningfully populated; Shape echoes which one.

Renderers branch on Shape and read the matching field. Matrix reuses crosstab.MatrixPayload so a CELL-scoped overlay layered on top of a matrix base shares the same row/column header conventions as the base.

type OverlayPopulationRef added in v0.19.0

type OverlayPopulationRef struct {
	// Cohort names the .pulse cohort whose unfiltered (or alternately
	// filtered) statistics constitute the comparison population.
	Cohort string `json:"cohort,omitempty"`
}

OverlayPopulationRef is reserved for "vs population" comparisons that compare a filtered cohort against an unfiltered (or differently- filtered) population.

type OverlayPriorRef added in v0.19.0

type OverlayPriorRef struct {
	// Lag pins the window width back along the ordered axis. Zero (the
	// omitempty default) means lag-1 — the immediately preceding point.
	// Reserved: v1 windowed kinds ship lag-1 only and reject non-zero
	// values at predict time; later stories widen the carrier.
	Lag int `json:"lag,omitempty"`
}

OverlayPriorRef is the ref-arm that carries the "previous point in the ordered axis" semantics for windowed-SERIES kinds (the windowed catalog — see kind-catalog-v1 PRD §4 windowed family). It tags a spec as "compare against the lag-N point" against a SERIES host whose group-key order is the chronological ordering the orchestrator baked in at finalize time (typically a `GROUP_DATE`-keyed grouped Process result).

`OVERLAY_INDEX_VS_PRIOR` is the first kind to consume this arm and ships with lag-1 only. The `Lag` slot is reserved for future window-N priors (lag-3 for "compare against three points ago", etc.); non-zero `Lag` is NOT exercised by any shipping kind today. The authoring shape stays forward-compatible — a v1 caller can leave `Ref` entirely empty (the implicit default for the kind) OR populate `Ref.Prior` with `Lag = 0` (or unset), and both shapes spell "lag-1 prior". When later windowed-N stories land they will accept positive `Lag` values and the runtime carrier widens from a single f64 to a small ring buffer.

type OverlayRef added in v0.19.0

type OverlayRef struct {
	// Margin selects an axis-margin slot of the base result.
	Margin *OverlayMarginRef `json:"margin,omitempty"`

	// Sibling selects another cell on the same axis. Reserved.
	Sibling *OverlaySiblingRef `json:"sibling,omitempty"`

	// BaselineIndex selects a fixed baseline coordinate. Reserved.
	BaselineIndex *OverlayBaselineIndexRef `json:"baseline_index,omitempty"`

	// Prior selects the lag-N point along the host's ordered axis
	// (windowed-SERIES family). `OVERLAY_INDEX_VS_PRIOR` is
	// the first consumer and ships with lag-1 only.
	Prior *OverlayPriorRef `json:"prior,omitempty"`

	// RollingMean tags the spec as a windowed rolling-mean kind. The
	// window width lives on `OverlaySpec.Params["window"]` per the
	// `WIN_*` operator convention; the marker struct is intentionally
	// empty. `OVERLAY_INDEX_VS_ROLLING_MEAN` is the first
	// consumer; `OVERLAY_ZSCORE_VS_ROLLING` reuses it.
	RollingMean *OverlayRollingMeanRef `json:"rolling_mean,omitempty"`

	// YoY tags the spec as a windowed year-over-year kind. The frequency
	// (`annual` | `quarterly` | `monthly` | `weekly` | `daily` | `hourly`)
	// lives on `OverlaySpec.Params["frequency"]` (the YoY's own override)
	// or falls back to the host's `GROUP_DATE` config at
	// `req.Groups[0].Params["frequency"]`; the marker struct is
	// intentionally empty. `OVERLAY_YOY` is the first consumer.
	YoY *OverlayYoYRef `json:"yoy,omitempty"`

	// Population selects an alternate cohort / population. Reserved.
	Population *OverlayPopulationRef `json:"population,omitempty"`

	// Stage selects an earlier ProcessChain stage's result. The
	// pointee is the same StageRef discriminated reference declared
	// in types/chain.go and consumed by the ChainOverlaySpec
	// Ref/Target slots — there is exactly one StageRef type in the
	// codebase. The per-stage overlay surface
	// (Request.Overlays-style) can therefore name a chain stage
	// using the same shape the whole-chain overlay surface uses.
	// Wired into the per-kind validators starting with the whole-chain
	// overlay family.
	Stage *StageRef `json:"stage,omitempty"`

	// Slot selects a named slot on the base result. Reserved.
	Slot *OverlaySlotRef `json:"slot,omitempty"`
}

OverlayRef is the discriminated union identifying what an overlay compares against. Each pointer field corresponds to one comparison family; exactly one is meaningfully populated per OverlaySpec. The validator (descriptor.ValidateOverlays) rejects an OverlaySpec that populates the wrong pointer for its Kind.

Placeholder pointers cover overlay families that have not yet landed runtime handlers — they exist so adding a new family does not migrate embedder-side JSON.

type OverlayRollingMeanRef added in v0.19.0

type OverlayRollingMeanRef struct{}

OverlayRollingMeanRef is the ref-arm that tags a spec as a windowed rolling-mean kind (windowed catalog — see kind-catalog-v1 PRD §4 windowed family) against a SERIES host whose group-key order is the chronological ordering the orchestrator baked in at finalize time (typically a `GROUP_DATE`-keyed grouped Process result).

`OVERLAY_INDEX_VS_ROLLING_MEAN` is the first kind to consume this arm and the empty marker is intentional — the v1 window width lives on `OverlaySpec.Params["window"]` per the `WIN_*` operator convention (see `skills/window-design.md`). The empty struct tags the ref family so the validator's "exactly one ref arm populated per kind" contract stays uniform with the rest of the catalog (every kind picks exactly one ref family, even when its parameters live on `Params`).

Forward-compat: future stories may extend the struct with non-Window knobs (e.g. weighting modes for exponentially-weighted means, edge- fill policies for the warmup window) without re-opening the parent `OverlayRef`. The `OVERLAY_ZSCORE_VS_ROLLING` handler REUSES this same ref-arm (sibling windowed-rolling family — both kinds carry the window width on `Params["window"]`) so a single ref family tag suffices for the rolling-window catalog.

type OverlayScope added in v0.19.0

type OverlayScope string

OverlayScope declares where an overlay's computation lands in the base result. It is independent of OverlayShape — a CELL-scoped overlay typically produces a matrix payload, a ROW-scoped overlay typically produces a series, but the choice is per-overlay.

const (
	// OverlayScopeCell decorates every cell of the base result. For a
	// crosstab base this is one value per (row_key, column_key) pair.
	OverlayScopeCell OverlayScope = "cell"

	// OverlayScopeRow decorates every row tuple of the base result —
	// one value per row key, independent of columns.
	OverlayScopeRow OverlayScope = "row"

	// OverlayScopeColumn decorates every column tuple of the base
	// result — one value per column key, independent of rows.
	OverlayScopeColumn OverlayScope = "column"

	// OverlayScopeMatrix decorates the matrix as a whole; the payload
	// typically carries a derived matrix that mirrors the base shape
	// (e.g. a column-normalized re-projection of the cell values).
	OverlayScopeMatrix OverlayScope = "matrix"

	// OverlayScopeGroup decorates one grouper level. Reserved for future
	// nested-axis families; v1 emits OVERLAY_NOT_IMPLEMENTED if used
	// against OVERLAY_INDEX_VS_MARGIN.
	OverlayScopeGroup OverlayScope = "group"

	// OverlayScopeTotal decorates the grand-total margin slot — a single
	// scalar covering the whole result.
	OverlayScopeTotal OverlayScope = "total"
)

type OverlayShape added in v0.19.0

type OverlayShape string

OverlayShape declares the structural footprint of an overlay's rendered payload. Downstream renderers branch on this to lay the overlay grid on top of the base result.

const (
	// OverlayShapeScalar carries a single float64 — a Total-scoped index,
	// a single sibling-vs-baseline delta, etc.
	OverlayShapeScalar OverlayShape = "scalar"

	// OverlayShapeSeries carries one float64 per axis key — a row-wise
	// index strip, a per-column deviation strip. SeriesPayload below
	// carries the keys + values in matching order.
	OverlayShapeSeries OverlayShape = "series"

	// OverlayShapeMatrix carries a full row × column grid of float64
	// cells — most commonly produced by Scope=CELL overlays where every
	// cell of the base matrix receives a derived score. MatrixPayload
	// (from crosstab.go) is reused so renderers handle both layers with
	// one shape.
	OverlayShapeMatrix OverlayShape = "matrix"
)

type OverlaySiblingRef added in v0.19.0

type OverlaySiblingRef struct {
	// Field names the axis dimension whose sibling is referenced
	// (typically a grouper Field name on the row or column axis).
	Field string `json:"field,omitempty"`

	// Value names the specific axis-key value to compare against.
	Value string `json:"value,omitempty"`
}

OverlaySiblingRef is reserved for sibling-cell comparison overlays (e.g. compare each cell against the cell at the same row but a different column key). Included as a placeholder slot so future kinds can drop in without re-opening this file.

type OverlaySlotRef added in v0.19.0

type OverlaySlotRef struct {
	// Name identifies the slot to reference.
	Name string `json:"name,omitempty"`
}

OverlaySlotRef is reserved for slot-aware overlays that reference a named slot of the base result (e.g. a labelled regression coefficient, a named percentile bucket). Placeholder slot.

type OverlaySpec added in v0.19.0

type OverlaySpec struct {
	// Name is the renderer-facing label for this overlay. When empty,
	// the processing layer synthesises a deterministic default.
	Name string `json:"name,omitempty"`

	// Kind selects the overlay catalog entry to execute.
	Kind OverlayKind `json:"kind"`

	// Scope declares where the overlay lands relative to the base
	// result.
	Scope OverlayScope `json:"scope"`

	// Ref names what the overlay compares against. Family pointer
	// selection depends on Kind.
	Ref OverlayRef `json:"ref"`

	// Params holds operator-specific configuration as raw JSON. Per-
	// kind schema documented alongside the kind's processor.
	Params json.RawMessage `json:"params,omitempty"`

	// Level truncates the same axis the overlay scopes (when paired
	// with a CELL-scope handler that honours Level/Within) to a
	// parent-grouper prefix at the configured depth. Default zero
	// (omitted on the wire) preserves the leaf-axis denominator, which
	// is byte-identical to the pre-Level handler output. Non-zero
	// values mirror the buffered crosstab's NormalizeLevel semantics:
	// the row/column-margin denominator is truncated to the configured
	// depth of the SAME axis the overlay axis-locks to. Honoured by the
	// share/index/delta/zscore family; the χ²/Fisher inferential family
	// rejects non-zero values with PULSE_OVERLAY_LEVEL_OUT_OF_RANGE
	// because those kinds compute their own contingency from the host
	// row + column margins (Level/Within would alter the implicit-
	// margin contract). See processing/crosstab_normalize.go for the
	// shared key-prefix helpers and skills/overlay-system.md for the
	// per-kind matrix.
	Level int `json:"level,omitempty"`

	// Within fixes a prefix of the OPPOSITE axis at the configured
	// depth (when paired with a CELL-scope handler that honours
	// Level/Within), producing a cross-axis partitioned denominator
	// rather than a same-axis truncated one. Default zero (omitted on
	// the wire) preserves the leaf-axis denominator, which is byte-
	// identical to the pre-Within handler output. Non-zero values
	// mirror the buffered crosstab's NormalizeWithin semantics: a
	// SHARE_OF_ROW overlay with Within=0 produces row shares that sum
	// to 1.0 within each fixed level-0 column prefix (instead of across
	// the full row). Composes with Level the same way the crosstab
	// path composes NormalizeLevel + NormalizeWithin. Honoured by the
	// share/index/delta/zscore family; the χ²/Fisher inferential family
	// rejects non-zero values with PULSE_OVERLAY_LEVEL_OUT_OF_RANGE
	// (same rationale as Level above). See
	// processing/crosstab_normalize.go and
	// skills/overlay-system.md for the per-kind matrix.
	Within int `json:"within,omitempty"`
}

OverlaySpec is the request-side definition of one overlay layer. Multiple specs may ride the same Request.Overlays slice; each produces one OverlayLayer in Response.Overlays in matching order.

Validation rules (enforced in descriptor + processing layers, not in this file — see descriptor.ValidateOverlays):

  • Kind is required and must be a known OverlayKind.
  • Scope is required and must be a known OverlayScope.
  • Ref must populate exactly one family pointer matching Kind's contract (OVERLAY_INDEX_VS_MARGIN ⇒ Ref.Margin must be set).
  • Name, when set, becomes the renderer-facing label; when empty the processing layer synthesises a deterministic default keyed by Kind + Scope + Ref.
  • Params carries operator-specific configuration; the per-kind schema lives alongside the kind's processor.

type OverlaySummary added in v0.19.0

type OverlaySummary struct {
	// Min is the minimum derived value across the layer's payload.
	Min *float64 `json:"min,omitempty"`

	// Max is the maximum derived value across the layer's payload.
	Max *float64 `json:"max,omitempty"`

	// Count is the number of present (non-null, non-missing) entries
	// the layer produced. Zero is a valid value (e.g. an empty
	// matrix); the pointer distinguishes "0 known" from "not
	// reported".
	Count *int `json:"count,omitempty"`

	// Baseline is the comparison anchor — 100 for index-vs-margin
	// (anything < 100 underperforms, > 100 overperforms), 0 for delta
	// overlays, 1 for ratio overlays. Renderers use it to centre
	// diverging colour ramps.
	Baseline *float64 `json:"baseline,omitempty"`

	// Statistic is the headline test statistic for inferential overlays
	// (chi-square statistic for OVERLAY_CHISQ_MATRIX; KS D statistic,
	// Fisher odds ratio, etc.). Pointer + omitempty so
	// descriptive overlays leave the field absent — Statistic is only
	// meaningful when the kind produces a single distinguished scalar
	// the renderer should highlight (typically alongside PValue).
	Statistic *float64 `json:"statistic,omitempty"`

	// PValue is the inferential overlay's p-value (probability under
	// the null hypothesis). Pointer + omitempty so descriptive overlays
	// leave the field absent — non-nil only when the kind produces a
	// hypothesis-test result the renderer should surface alongside
	// Statistic.
	PValue *float64 `json:"p_value,omitempty"`

	// Parameters carries kind-specific test parameters in a flexible
	// shape so the inferential overlay catalog (CHISQ_MATRIX, CHISQ_ROW,
	// CHISQ_COL, FISHER_EXACT_CELL, CHISQ_VS_POP) can
	// expose distribution / model parameters without per-kind struct
	// churn. Examples:
	//   OVERLAY_CHISQ_MATRIX  → {"df": 4.0}
	//   OVERLAY_FISHER_MATRIX → {"odds_ratio": 1.42}
	//   OVERLAY_KS_*          → {"d": 0.18}
	// Keys are SCREAMING_SNAKE-free lowercase strings; values are
	// float64. Empty / nil when the kind has no extra parameters to
	// surface. The map shape is forward-compatible — new keys land
	// additively without breaking existing renderer code.
	Parameters map[string]float64 `json:"parameters,omitempty"`
}

OverlaySummary carries optional renderer-friendly metadata for one overlay layer — min/max for colour-ramp scaling, count of present cells for sparsity hints, an optional baseline reference value. Every field is omitempty so a producer can populate just the summary slots that make sense for the kind in question (e.g. INDEX_VS_MARGIN reports min/max but not baseline; a future Z-score overlay reports baseline=0 and the populated standard deviation).

type OverlayWarning added in v0.22.0

type OverlayWarning struct {
	// Code is the canonical overlay error code (today: errors.PULSE_OVERLAY_REF_ZERO).
	Code string
	// Message is the human-readable summary.
	Message string
	// Details carries structured context for the warning — overlay
	// index, kind, axis, the offending row / column key indices, etc.
	Details map[string]any
}

OverlayWarning is the in-process diagnostic emitted by an overlay handler when it could not produce a meaningful value for some subset of cells (zero denominator, missing margin slot). Carries the canonical error code (today: errors.PULSE_OVERLAY_REF_ZERO), a renderer-friendly message, and structured details so the orchestrator can promote each entry into a types.ResponseWarning on the envelope.

Relocated from processing.OverlayWarning so the OverlayLayer slot can carry it directly without types/ importing processing/. Moving the value-only diagnostic up the dependency graph keeps the types package behavioural-import-free while letting both processing/ and service/ produce warnings against the same shape.

type OverlayYoYRef added in v0.19.0

type OverlayYoYRef struct{}

OverlayYoYRef is the ref-arm that tags a spec as a year-over-year windowed kind (windowed catalog — see kind-catalog-v1 PRD §4 windowed family) against a SERIES host whose grouper is `GROUP_DATE`.

`OVERLAY_YOY` is the first kind to consume this arm and the empty marker is intentional — the v1 frequency value lives on `OverlaySpec.Params["frequency"]` (the YoY's own override) or falls back to the host's `GROUP_DATE` config at `req.Groups[0].Params["frequency"]`. The supported frequencies are `annual` | `quarterly` | `monthly` | `weekly` | `daily` | `hourly`; the handler picks the matching stride (annual ⇒ -1 ordinal, quarterly ⇒ -4 ordinals, monthly ⇒ -12 ordinals, weekly ⇒ -52 ordinals, daily ⇒ -365-day exact-key lookup, hourly ⇒ -365×24-hour exact-key lookup).

Calendar-week / day-of-week realignment is an explicit non-goal in v1: weekly frequency uses calendar-week-aligned `i - 52` arithmetic (no day-of-week realignment); daily frequency uses exact-key lookup against the host key index — Feb 29 in a non-leap prior year emits NaN because no exact-key match exists, not an off-by-one realignment to Feb 28 or Mar 1.

The empty struct tags the ref family so the validator's "exactly one ref arm populated per kind" contract stays uniform with the rest of the catalog (every kind picks exactly one ref family, even when its parameters live on `Params`). Forward-compat: future stories may extend the struct with non-frequency knobs (e.g. fiscal-year alignment, leap-year fill policies) without re-opening the parent `OverlayRef`.

type PairwiseOverlayParams added in v0.24.0

type PairwiseOverlayParams struct {
	// PairAlongDim, when non-nil, restricts pair generation to pair-axis
	// indexes whose key tuples agree on every dim position EXCEPT this
	// one ("all dims agree except this one" buckets). nil = every pair.
	// Must be >= 0 and < the pair-axis dim count.
	PairAlongDim *int `json:"pair_along_dim,omitempty"`

	// NSource selects where the sample-size leg is read. One of the
	// PairwiseNSource* constants. Empty = cell_n_unweighted. Ignored by
	// the Welford-input kinds (welch / two-means read n from the triple).
	NSource string `json:"n_source,omitempty"`

	// NWithinDepth, with NSource=n_within, fixes the first NWithinDepth+1
	// pair-axis dim positions in the denominator (mirrors
	// CrosstabSpec.NormalizeWithin). Must be >= 0.
	NWithinDepth int `json:"n_within_depth,omitempty"`

	// PSource selects how the proportion leg is derived for the
	// proportion-input kinds (prop-Z, probit-t). One of the
	// PairwisePSource* constants. Empty = cell_value_pct. Ignored by the
	// Welford-input kinds.
	PSource string `json:"p_source,omitempty"`
}

PairwiseOverlayParams is the decoded OverlaySpec.Params shape for the OVERLAY_PAIRWISE_* family. Every field is optional; the zero value (no params) means "every pair-axis index vs every other, cell n unweighted, cell value as a 0..100 percentage". Axis (row vs column pairing) rides on OverlaySpec.Scope, not here.

func DecodePairwiseParams added in v0.24.0

func DecodePairwiseParams(raw json.RawMessage) (PairwiseOverlayParams, error)

DecodePairwiseParams decodes a raw OverlaySpec.Params blob into a PairwiseOverlayParams. A nil / empty blob yields the zero value (all defaults). Malformed JSON returns an error so callers surface a clean predict-time / runtime diagnostic rather than panicking.

type RegressionResult added in v0.6.0

type RegressionResult struct {
	// Name echoes RegressionSpec.Name or the synthesized default.
	Name string `json:"name,omitempty"`

	// Type is the regression operator that produced this result.
	Type RegressionType `json:"type"`

	// Family echoes RegressionSpec.Family (GLM only).
	Family string `json:"family,omitempty"`

	// Link echoes RegressionSpec.Link (GLM only).
	Link string `json:"link,omitempty"`

	// Penalty echoes RegressionSpec.Penalty (REG_OLS only).
	Penalty string `json:"penalty,omitempty"`

	// Alpha echoes RegressionSpec.Alpha (regularized OLS only).
	Alpha float64 `json:"alpha,omitempty"`

	// L1Ratio echoes RegressionSpec.L1Ratio (elastic-net OLS only).
	L1Ratio float64 `json:"l1_ratio,omitempty"`

	// Prior echoes RegressionSpec.Prior (Bayes only).
	Prior string `json:"prior,omitempty"`

	// Resample echoes RegressionSpec.Resample when set.
	Resample string `json:"resample,omitempty"`

	// Selection echoes RegressionSpec.Selection when set.
	Selection string `json:"selection,omitempty"`

	// Criterion echoes RegressionSpec.Criterion when Selection is set.
	Criterion string `json:"criterion,omitempty"`

	// Coefficients maps predictor name (including the synthesized
	// "(intercept)" key) to its fitted scalar value.
	Coefficients map[string]float64 `json:"coefficients,omitempty"`

	// StdErrors maps predictor name to its asymptotic standard error.
	StdErrors map[string]float64 `json:"std_errors,omitempty"`

	// PValues maps predictor name to its two-sided p-value under the
	// null β = 0.
	PValues map[string]float64 `json:"p_values,omitempty"`

	// R2 is the coefficient of determination (REG_OLS / REG_BAYES_LINEAR).
	R2 float64 `json:"r2,omitempty"`

	// AdjR2 is the adjusted R² (REG_OLS / REG_BAYES_LINEAR).
	AdjR2 float64 `json:"adj_r2,omitempty"`

	// Deviance is the model deviance (REG_GLM).
	Deviance float64 `json:"deviance,omitempty"`

	// NullDeviance is the deviance of the intercept-only model
	// (REG_GLM).
	NullDeviance float64 `json:"null_deviance,omitempty"`

	// PseudoR2 is a McFadden-style 1 − Deviance/NullDeviance (REG_GLM).
	PseudoR2 float64 `json:"pseudo_r2,omitempty"`

	// NObs is the number of observations used to fit the model.
	NObs int `json:"n_obs,omitempty"`

	// ResidualStdErr is the residual standard error
	// (REG_OLS / REG_BAYES_LINEAR).
	ResidualStdErr float64 `json:"residual_std_err,omitempty"`

	// ConvergedIters is the IRLS / coordinate-descent iteration count
	// that produced the final estimate (iterative fits only). Zero
	// when the fit is closed-form (REG_OLS without penalty,
	// REG_BAYES_LINEAR).
	ConvergedIters int `json:"converged_iters,omitempty"`

	// SelectedFeatures lists the predictors retained by Selection (in
	// fit order); empty when Selection is unset.
	SelectedFeatures []string `json:"selected_features,omitempty"`

	// CredibleIntervals maps predictor name to its
	// [lower, upper] posterior credible interval (REG_BAYES_LINEAR).
	CredibleIntervals map[string][2]float64 `json:"credible_intervals,omitempty"`
}

RegressionResult is the per-spec outcome embedded in Response.Regressions. Fields irrelevant to a given operator carry their zero value; engines never partially populate a result on failure.

Locked at Phase 0: every field of every operator family is declared now. Later phases populate the engine-specific branches but never change the struct shape.

type RegressionSpec added in v0.6.0

type RegressionSpec struct {
	// Type names the regression operator (REG_OLS, REG_GLM,
	// REG_BAYES_LINEAR).
	Type RegressionType `json:"type"`

	// Name is an optional alias for the result entry; defaults to a
	// synthesized name derived from Type and Target.
	Name string `json:"name,omitempty"`

	// Target is the response variable's field name. Required.
	Target string `json:"target"`

	// Predictors lists the predictor (independent variable) field
	// names. At least one is required. A single predictor produces
	// simple / linear regression; multiple predictors produce multiple
	// linear regression.
	Predictors []string `json:"predictors,omitempty"`

	// Penalty selects the regularization scheme for REG_OLS. One of
	// "" (no penalty), "l1" (lasso), "l2" (ridge), or "elasticnet".
	// Ignored by REG_GLM and REG_BAYES_LINEAR.
	Penalty string `json:"penalty,omitempty"`

	// Alpha is the regularization strength for l1 / l2 / elasticnet
	// penalties. Must be > 0 when Penalty is non-empty.
	Alpha float64 `json:"alpha,omitempty"`

	// L1Ratio is the elastic-net mixing parameter in [0, 1]; 0 is pure
	// l2, 1 is pure l1. Only read when Penalty == "elasticnet".
	L1Ratio float64 `json:"l1_ratio,omitempty"`

	// Family is the GLM error family. One of "binomial" (logistic),
	// "poisson", or "gamma". Required for REG_GLM.
	Family string `json:"family,omitempty"`

	// Link is the GLM link function. Defaults are family-specific
	// ("logit" for binomial, "log" for poisson and gamma).
	Link string `json:"link,omitempty"`

	// MaxIters caps the IRLS / coordinate-descent iteration count for
	// iterative fits. Defaults to a registry-specific value when zero.
	MaxIters int `json:"max_iters,omitempty"`

	// Tol is the relative convergence tolerance for iterative fits.
	// Defaults to a registry-specific value when zero.
	Tol float64 `json:"tol,omitempty"`

	// Prior names the prior family for REG_BAYES_LINEAR. Only "nig"
	// (Normal-Inverse-Gamma, conjugate) is supported in v1.
	Prior string `json:"prior,omitempty"`

	// PriorMu is the prior mean vector for the coefficients
	// (REG_BAYES_LINEAR). Length must match the predictor count; zero
	// vector when nil.
	PriorMu []float64 `json:"prior_mu,omitempty"`

	// PriorPrecision is the prior precision scalar (REG_BAYES_LINEAR).
	PriorPrecision float64 `json:"prior_precision,omitempty"`

	// PriorShape and PriorRate parameterize the inverse-gamma prior on
	// the residual variance (REG_BAYES_LINEAR).
	PriorShape float64 `json:"prior_shape,omitempty"`
	PriorRate  float64 `json:"prior_rate,omitempty"`

	// CredibleLevel is the credible-interval mass for the posterior
	// summaries (REG_BAYES_LINEAR). Defaults to 0.95 when zero.
	CredibleLevel float64 `json:"credible_level,omitempty"`

	// Resample selects an orthogonal resampling layer applied to any
	// regression family. One of "" (none), "jackknife" (leave-one-out),
	// or "bootstrap". Non-empty forces the buffered path.
	Resample string `json:"resample,omitempty"`

	// BootstrapIters is the bootstrap replicate count when
	// Resample == "bootstrap". Ignored otherwise.
	BootstrapIters int `json:"bootstrap_iters,omitempty"`

	// RNGSeed seeds the bootstrap RNG for reproducibility. Defaults to
	// 0 (a deterministic seed derived from the request).
	RNGSeed int64 `json:"rng_seed,omitempty"`

	// Selection requests an automated subset-selection wrapper around
	// any regression family. One of "" (none), "forward", "backward",
	// or "stepwise". Non-empty forces the buffered path.
	Selection string `json:"selection,omitempty"`

	// Criterion is the information criterion driving Selection. One of
	// "aic" or "bic". Required when Selection is non-empty.
	Criterion string `json:"criterion,omitempty"`
}

RegressionSpec describes a single regression operator entry in a Request. Every field of every variant is declared up front; engines populate or ignore branches based on Type and modifiers.

The wire shape is shared across the three operator families so a request parser only needs to dispatch on Type. Unused branches stay at their zero values.

func (RegressionSpec) Streamable added in v0.6.0

func (s RegressionSpec) Streamable() bool

Streamable reports whether this concrete RegressionSpec can run via the streaming execution path today. The check folds the type-level Streamable() value with one layer:

  • modifier downgrade: any non-empty Resample or Selection forces the buffered path.

REG_OLS streams both unpenalized (Phase 1) and penalized (l1 / l2 / elasticnet, Phase 2) — the streaming Gram is identical; the iterative solver runs at finalize over a p×p matrix, not over rows. REG_BAYES_LINEAR (Phase 4) streams the same Welford sufficient stats and applies the conjugate Normal-Inverse-Gamma posterior at finalize.

type RegressionType added in v0.6.0

type RegressionType string

RegressionType identifies a specific regression-modeling operator.

Pulse exposes three top-level regression operators that cover the thirteen textbook regression variants through a small, composable core:

  • REG_OLS — ordinary least squares; optional l1/l2/elasticnet penalty, simple/multiple predictors, polynomial features supplied upstream by FEAT_POLY for polynomial regression.
  • REG_GLM — generalized linear model; Family ∈ {binomial, poisson, gamma} with matching Link. Covers logistic and Poisson regression.
  • REG_BAYES_LINEAR — Bayesian linear regression via conjugate Normal-Inverse-Gamma prior.

Two spec-level modifiers compose with any of the three: `Resample` (jackknife / bootstrap) and `Selection` (forward / backward / stepwise) plus the matching information criterion (AIC / BIC).

Per-operator semantics, modifier behavior, and the 13-name → spec mapping live in skills/regression-modeling.md.

const (
	// REG_OLS fits ordinary least squares with optional regularization.
	// Streams over sufficient statistics (n, Σx, Σy, XᵀX, Xᵀy, Σy²) when
	// Penalty is empty; the same Gram matrix plus a regularized
	// finalize-solve handles the l1/l2/elasticnet variants.
	REG_OLS RegressionType = "REG_OLS"

	// REG_GLM fits a generalized linear model via IRLS. Always buffered:
	// Newton-Raphson needs multiple passes over the data.
	REG_GLM RegressionType = "REG_GLM"

	// REG_BAYES_LINEAR fits a Bayesian linear model under a conjugate
	// Normal-Inverse-Gamma prior. Streams the same sufficient statistics
	// as REG_OLS, then applies the closed-form posterior update.
	REG_BAYES_LINEAR RegressionType = "REG_BAYES_LINEAR"
)

func AllRegressionTypes added in v0.6.0

func AllRegressionTypes() []RegressionType

AllRegressionTypes returns every defined regression operator in alphabetical order.

func (RegressionType) Streamable added in v0.6.0

func (t RegressionType) Streamable() bool

Streamable reports whether a regression type can run via the streaming execution path in isolation, ignoring spec-level modifiers like Resample or Selection. REG_OLS and REG_BAYES_LINEAR stream over sufficient statistics; REG_GLM always needs IRLS and therefore the buffered path.

Use RegressionSpec.Streamable() instead of this type-level method when checking a concrete request: the spec-level helper downgrades streamability whenever Resample or Selection is set, mirroring the runtime gate.

Default branch returns false so newly-added regression types must opt in.

type Request

type Request struct {
	// Label is an optional caller-supplied alias used by Compose-only
	// overlay kinds to resolve sibling references across slots in a
	// ComposedRequest (e.g. ComposeOverlaySpec.Reference / Targets).
	// Empty is the zero value and the additive contract holds — the
	// `omitempty` tag keeps the JSON wire shape byte-identical to
	// pre-Label callers, including the CanonicalHash output (see
	// TestCanonicalHash_RequestLabelEmptyByteIdentical).
	//
	// Inside a Compose batch the engine synthesizes `request_<index+1>`
	// (1-based) for every slot that arrives with Label empty, before
	// dispatching the slot — the synthesis happens against an in-memory
	// clone so the caller's *Request pointer is not mutated. Two slots
	// resolving to the same final Label are rejected with
	// PULSE_COMPOSE_LABEL_COLLISION at the same validate hook.
	//
	// Outside Compose (a standalone pulse.Process call) Label is
	// retained verbatim but has no behavioural effect today; future
	// stories may light it up for downstream reference resolution.
	Label string `json:"label,omitempty"`

	// Cohort identifies the .pulse file to process.
	Cohort *Cohort `json:"cohort,omitempty"`

	// Filterers is the list of filters to apply before processing.
	Filterers []*Filterer `json:"filterers,omitempty"`

	// Aggregations is the list of aggregation operations.
	Aggregations []*Aggregation `json:"aggregations,omitempty"`

	// Attributes is the list of derived attribute computations.
	Attributes []*Attribute `json:"attributes,omitempty"`

	// Groups is the list of grouping operations.
	Groups []*Group `json:"groups,omitempty"`

	// Outputs configures result formatting.
	Outputs []*Output `json:"outputs,omitempty"`

	// Windows is the list of window operations evaluated after aggregation.
	Windows []*Window `json:"windows,omitempty"`

	// Features is the list of pre-filter feature engineering operators.
	// Each operator may add one or more derived columns to the working
	// schema before filters and downstream stages run.
	Features []*Feature `json:"features,omitempty"`

	// Sort orders response rows by the listed keys. Applied last in the
	// pipeline (after windows). Each key field must reference a schema
	// field, an aggregation/attribute/group/window output label, or any
	// column produced by upstream stages. Stable sort; nulls last
	// regardless of direction.
	Sort []OrderKey `json:"sort,omitempty"`

	// Tests is the list of tier-1 statistical tests evaluated alongside
	// aggregators against the raw row stream. Online-moments tests
	// (TEST_T, TEST_WELCH, TEST_CHISQ, TEST_ANOVA_F) execute in the
	// streaming Process path; sort-required tests (TEST_KS) force the
	// buffered path. Results land in Response.Tests in the same order.
	Tests []*Test `json:"tests,omitempty"`

	// PostTests is the list of tier-2 statistical tests evaluated after
	// the window stage on the materialized result row set. Typical uses:
	// ANOVA across grouper buckets, Tukey HSD post-hoc on per-group
	// means, trend tests over windowed series. Always buffered (the
	// result row set must be materialized before tier-2 runs). Results
	// land in Response.PostTests in the same order.
	PostTests []*Test `json:"post_tests,omitempty"`

	// Regressions is the list of regression-modeling operators (REG_OLS,
	// REG_GLM, REG_BAYES_LINEAR) evaluated against the filtered record
	// set. Each spec produces one RegressionResult in Response.Regressions
	// in matching order. Streamability follows RegressionSpec.Streamable
	// — closed-form OLS/Bayes stream over sufficient statistics; GLM,
	// Resample, and Selection variants force the buffered path.
	Regressions []*RegressionSpec `json:"regressions,omitempty"`

	// Joins describes pushdown hash-join legs attached to the primary
	// cohort. v1 supports exactly one inner join per Request; multi-
	// join chains and the left/outer/anti kinds land in a follow-up.
	// When set, the orchestrator opens each Right cohort, builds a
	// hash table over the smaller side's join keys, and streams the
	// larger side as the probe — joined records flow through the
	// standard filter / attribute / group / aggregator pipeline.
	Joins []*JoinSpec `json:"joins,omitempty"`

	// Labels rewrites or augments categorical output values using
	// embedder-registered label tables. Each binding pairs a
	// categorical field with a label-table name and a mode (replace
	// or augment). Labels are display-only: filters, formulas, sort
	// keys, and group keys still see raw values. Tables are sourced
	// from pulse.Options.Extensions.LabelTables.
	Labels []*LabelBinding `json:"labels,omitempty"`

	// Crosstab is an optional cross-tabulation directive. When set,
	// the engine composes the cell aggregation across the row × column
	// grouper grid, computes the requested margins via sibling Compose
	// requests, applies the configured normalization, and returns the
	// result either as a structured matrix on Response.Crosstab
	// (shape=matrix, default) or as long-form rows on Response.Data
	// (shape=long). Mutually exclusive with top-level Groups /
	// Aggregations — see PULSE_CROSSTAB_CONFLICTS_WITH_GROUPS.
	Crosstab *CrosstabSpec `json:"crosstab,omitempty"`

	// Overlays is the list of overlay-layer specifications that
	// decorate the primary result with derived projections — index-
	// vs-margin scores, sibling comparisons, baseline lifts, etc.
	// Each spec produces one OverlayLayer in Response.Overlays in
	// matching order. Per-kind semantics, required Ref fields, and
	// scope compatibility live in the overlay catalog (see
	// types/overlay.go).
	Overlays []OverlaySpec `json:"overlays,omitempty"`

	// DisableComponents overrides the engine-level
	// pulse.Options.DisableComponents setting for this single request.
	// nil (the default) inherits the engine setting. Explicit `true`
	// suppresses Response.Components regardless of the engine default;
	// explicit `false` forces components on regardless of the engine
	// default. Use a pointer so the unset state is distinguishable from
	// `false` — the standard tri-state override pattern.
	//
	// When the resolved decision is "disabled", Response.Components stays
	// nil and the wire form is byte-identical to the pre-Components
	// baseline; format_version is not bumped. The gate sits at the
	// processor's attach helpers, so the MetaAggregator.Components and
	// MetaGrouper.Components construction work is skipped — not
	// built-then-discarded.
	DisableComponents *bool `json:"disable_components,omitempty"`
}

Request is the primary processing request type. It specifies a cohort, filters, aggregations, attributes, groups, and output config.

func (*Request) Hash added in v0.11.0

func (r *Request) Hash() string

Hash returns the canonical content hash of the Request. Same logical request → same hash, across processes and Pulse versions where the request's semantic meaning is unchanged. See CanonicalHash for the algorithm.

Slot coverage is data-driven: every JSON-tagged field on Request participates in the hash via the json.Marshal → canonicalize → sha256 pipeline. The Overlays slot is covered automatically — each OverlaySpec contributes its Name/Kind/Scope/Ref to the canonical bytes in declared spec order (slices preserve order), and the discriminated OverlayRef union hashes only the populated arm because every family pointer carries `json:",omitempty"`. An overlay-free Request (nil or empty Overlays slice) omits the `overlays` key entirely via the slot's own `omitempty` tag, so its hash is byte-identical to the pre-Overlays canonical form — see TestCanonicalHash_OverlayFreeByteIdentity.

type Response

type Response struct {
	// Data holds the result rows as key-value maps.
	Data []map[string]any `json:"data,omitempty"`

	// Metadata holds information about the processing run.
	Metadata *ResponseMetadata `json:"metadata,omitempty"`

	// Tests holds tier-1 statistical test results, one per entry in
	// Request.Tests and in the same order.
	Tests []*TestResult `json:"tests,omitempty"`

	// PostTests holds tier-2 statistical test results, one per entry in
	// Request.PostTests and in the same order.
	PostTests []*TestResult `json:"post_tests,omitempty"`

	// Regressions holds the per-spec regression fits, one entry per
	// Request.Regressions in the same order. Engines never partially
	// populate a result on failure; a failed fit surfaces as a
	// PROCESSING_REGRESSION_* error on the envelope instead.
	Regressions []*RegressionResult `json:"regressions,omitempty"`

	// Warnings carries cross-cutting diagnostics surfaced after the
	// processor finishes. Today this slot is populated by the label-
	// display resolver (PULSE_LABEL_COLLISION, PULSE_LABEL_LOOKUP_MISS);
	// future runtime overlays attach here too. CLI / MCP envelope
	// wrappers promote each entry into the envelope's Warnings array.
	Warnings []*ResponseWarning `json:"warnings,omitempty"`

	// Crosstab carries the structured matrix payload when the originating
	// Request set Crosstab and Shape=matrix (the default). Nil otherwise.
	// For Shape=long the cell rows land on Data verbatim — the existing
	// grouped-tuple format — so any consumer of long-form rows keeps
	// working unchanged.
	Crosstab *CrosstabResult `json:"crosstab,omitempty"`

	// Overlays carries the executed overlay layers, one entry per
	// Request.Overlays spec in matching order. Each layer holds its
	// derived payload (scalar / series / matrix) plus optional
	// renderer-friendly summary metadata. Omitted entirely when the
	// originating Request had no overlays.
	Overlays []OverlayLayer `json:"overlays,omitempty"`

	// Components carries the constituent-parts metadata emitted by each
	// operator during a processing run — per-aggregation N / NNull,
	// per-grouper bucket inventory, per-filterer N-in / N-out, crosstab
	// cell components, and run-wide totals.
	//
	// Naming-collision note: Response.Metadata (above) holds runtime
	// facts about the orchestrator pass — total / filtered row counts,
	// cohort filename — that are produced by service-layer wiring and
	// are independent of any specific operator. Response.Components
	// holds the computational components emitted BY the operators
	// themselves (the AGG_*, GROUP_*, FILTER_*, crosstab pipeline) and
	// keyed back to their slot identity. The two are deliberately
	// distinct slots: Metadata is "what the run did", Components is
	// "what each operator contributed". Consumers wanting both pull
	// from both slots; renderer-friendly summaries (cell components for
	// the parity overlays, etc.) live on Components.
	//
	// Additive omitempty contract: the slot is `*ResponseComponents` so
	// an unpopulated run marshals to no `components` key at all —
	// byte-identical to the pre-Components wire form. Every nested
	// slice / pointer / map is `omitempty` for the same reason; an
	// AGG-only run with no groupers omits `groupers`, a no-filter run
	// omits `filterers`, etc. format_version stays at "1.0" because
	// the slot is additive.
	Components *ResponseComponents `json:"components,omitempty"`
}

Response is the processing result type.

type ResponseComponents added in v0.20.0

type ResponseComponents struct {
	// Aggregations carries one AggregationComponents entry per
	// Request.Aggregations slot in matching declared order.
	Aggregations []AggregationComponents `json:"aggregations,omitempty"`

	// Groupers carries one GrouperComponents entry per Request.Groups
	// slot in matching declared order.
	Groupers []GrouperComponents `json:"groupers,omitempty"`

	// Crosstab carries the crosstab-level components (cell matrix +
	// margins). Populated only when the originating Request set
	// Crosstab; nil otherwise.
	Crosstab *CrosstabComponents `json:"crosstab,omitempty"`

	// Filterers carries one FiltererComponents entry per
	// Request.Filterers slot in matching declared order.
	Filterers []FiltererComponents `json:"filterers,omitempty"`

	// Run carries run-wide totals — cohort record count, filtered
	// count, null count, shard count, partial-cohort reason. Distinct
	// from Response.Metadata which holds orchestrator-pass facts about
	// the request execution (file name, etc.).
	Run *RunComponents `json:"run,omitempty"`
}

ResponseComponents carries the constituent-parts metadata emitted by each operator slot during a processing run. Every nested field is `omitempty` so partially populated runs (an AGG-only request, a filter-free request, etc.) marshal to byte-identical wire output against the pre-Components baseline. See Response.Components for the runtime-facts vs computational-components naming-collision note.

Field-level semantics:

  • Aggregations: one entry per Request.Aggregations spec in matching declared order. Slot identity rides on Label (mirrors Aggregation.Label). The universal floor (N, NNull) is a typed field on every entry; per-operator keys ride inside Operator.
  • Groupers: one entry per Request.Groups spec in matching declared order. Slot identity rides on Field. Per-grouper schema (bucket edges, dict mappings, etc.) rides inside Operator.
  • Crosstab: populated when the originating Request set Crosstab.
  • Filterers: one entry per Request.Filterers spec in matching declared order. Slot identity rides on Label.
  • Run: optional run-wide totals (cohort record count, filtered count, null count, shard count, partial-cohort reason). Distinct from Response.Metadata which holds orchestrator-pass facts.

type ResponseMetadata

type ResponseMetadata struct {
	// TotalRows is the total number of records in the cohort.
	TotalRows int64 `json:"total_rows"`

	// FilteredRows is the number of records after filtering.
	FilteredRows int64 `json:"filtered_rows"`

	// CohortFile is the filename of the processed cohort.
	CohortFile string `json:"cohort_file,omitempty"`
}

ResponseMetadata holds metadata about a processing result.

type ResponseWarning added in v0.10.1

type ResponseWarning struct {
	Code    string         `json:"code"`
	Message string         `json:"message"`
	Details map[string]any `json:"details,omitempty"`
}

ResponseWarning is the envelope-ready projection of a cross-cutting diagnostic emitted during a processing run (today: label-display resolver warnings). Codes match the errors.Code namespace so a CLI / MCP envelope wrapper can promote them verbatim.

type RunComponents added in v0.20.0

type RunComponents struct {
	// TotalRecords is the total number of records in the underlying
	// cohort (pre-filter).
	TotalRecords int64 `json:"total_records"`

	// FilteredRecords is the number of records that survived the
	// filter chain.
	FilteredRecords int64 `json:"filtered_records"`

	// NullRecords is the number of records dropped due to null input
	// somewhere in the pipeline.
	NullRecords int64 `json:"null_records"`

	// ShardCount is the number of shards processed when the cohort
	// resolved to a shard archive. Zero for single-file cohorts.
	ShardCount int `json:"shard_count,omitempty"`

	// PartialCohortReason carries a free-form diagnostic explaining
	// why the run consumed less than the full cohort (e.g. a shard
	// failed to open). Empty when the run consumed the full cohort.
	PartialCohortReason string `json:"partial_cohort_reason,omitempty"`
}

RunComponents carries run-wide constituent-parts metadata — cohort totals, filtered counts, null counts, shard count, partial-cohort reason. Distinct from Response.Metadata which holds orchestrator- pass facts (cohort filename, etc.); RunComponents is the computational-component view of the same run.

Metadata-vs-Run coexistence (see CLAUDE.md Output Format Contract): Response.Metadata.TotalRows and Components.Run.TotalRecords carry the same value by construction at every orchestrator exit. Metadata retains the non-numerical run facts (cohort filename); Run carries the typed counters consumers compute against (joins, ratios, partial-cohort diagnostics). Both slots are populated on every successful Process call so consumers can rely on either side.

type SampleRequest added in v0.10.1

type SampleRequest struct {
	// Cohort selects the .pulse file. Same shape as Request.Cohort.
	Cohort *Cohort `json:"cohort,omitempty"`
	// N is the maximum row count returned. Zero or negative returns
	// no rows.
	N int `json:"n"`
	// Labels rewrites or augments categorical row values using
	// embedder-registered label tables. See LabelBinding.
	Labels []*LabelBinding `json:"labels,omitempty"`
}

SampleRequest is the input shape for pulse.SampleWithRequest. It carries the cohort path, the row cap, and the per-request label bindings that the no-struct Sample entry point cannot accept. The shape is intentionally tiny — the legacy Sample(ctx, path, n) signature stays for back-compat and is implemented as a thin wrapper that builds a SampleRequest with no labels.

type SeriesEntry added in v0.19.0

type SeriesEntry struct {
	// Key is the composite axis-tuple key identifying this entry.
	// Matches the host axis-key list element-for-element (RowKeys[i]
	// for CHISQ_ROW; ColumnKeys[i] for CHISQ_COL).
	Key AxisKey `json:"key"`

	// Summary carries the per-entry statistic / p-value / parameter
	// map. Mirrors the SCALAR-payload OverlaySummary shape so
	// inferential SERIES overlays surface the same renderer-facing
	// fields per entry (e.g. CHISQ_ROW: Statistic = χ²_r,
	// PValue = p_r, Parameters = {"df": cols - 1}).
	Summary OverlaySummary `json:"summary"`
}

SeriesEntry is one per-axis-key entry in a series-shaped overlay payload. Key is the composite axis tuple (matching the host's AxisKey shape, e.g. MatrixPayload.RowKeys[i] for a CHISQ_ROW overlay); Summary carries the entry's statistic / p-value / parameters in the same shape the SCALAR-payload OverlaySummary uses on inferential overlays.

type SeriesPayload added in v0.19.0

type SeriesPayload struct {
	// Entries is the ordered list of per-axis-key entries. Order
	// matches the host axis-key list element-for-element.
	Entries []SeriesEntry `json:"entries"`
}

SeriesPayload is the canonical strip used by series-shaped overlay payloads. Each Entry pairs a composite axis-tuple key (matching the host layer's AxisKey shape element-for-element) with an OverlaySummary carrying the per-entry statistic / p-value / parameter map. The shape generalises across host families: for a Crosstab host the entry order matches MatrixPayload.RowKeys (CHISQ_ROW) or ColumnKeys (CHISQ_COL); for a future process-grouped host the entry order matches the host's grouper-key vector.

Per-entry key contract (OVERLAY_CHISQ_ROW): the Series entries are a parallel slice to the host axis-key list — entry i carries the same composite key as RowKeys[i] (CHISQ_ROW) or ColumnKeys[i] (CHISQ_COL). This contract is the renderer-facing guarantee that overlay layers can be laid on top of the host axis without re-keying.

Why the per-entry shape (Entries + Summary) over the earlier flat-strip placeholder (Keys + Values): the earlier shape carried a single float64 per key, which suits descriptive overlays but cannot surface inferential metadata (statistic, p-value, parameter map) without inventing a parallel summary slice. The canonical shape lets SCALAR + SERIES inferential overlays reuse the same OverlaySummary surface — CHISQ_ROW's per-row {Statistic, PValue, Parameters{"df"}} reuses CHISQ_MATRIX's summary shape verbatim, and the per-row renderer reads each entry exactly the way it reads a SCALAR layer. Future descriptive series families (per-row deviation strips) populate only Summary.Min / Max / Count and leave Statistic / PValue unset.

AxisKey is []any so numeric / categorical / date axis values keep their native types (matching MatrixPayload.RowKeys); renderers join or display tuple elements the same way they handle host row keys.

type StageRef added in v0.19.0

type StageRef struct {
	// Index is the zero-based stage index into ChainRequest.Stages.
	// Pointer so `0` is meaningful — the canonical "stage 0" call
	// site sets Index = &zero, NOT Index unset.
	Index *int `json:"index,omitempty"`

	// Name matches against ChainStage.Name. Empty when Index is set.
	Name string `json:"name,omitempty"`
}

StageRef is the discriminated reference one ChainOverlaySpec uses to identify a specific stage in the parent ChainRequest.Stages slice. Exactly one of Index / Name is populated per StageRef value — the XOR is validated downstream by the chain-overlay validator and the per-kind runtime handler:

  • Index is a pointer so the zero value (`Index: ptr(0)`, meaningfully "stage 0") is distinguishable from "no index supplied" (`Index: nil`). A literal `int` field would collapse those two cases onto the same wire shape.
  • Name matches against ChainStage.Name verbatim. Empty Name + nil Index is rejected as malformed; both set is also rejected (XOR is "exactly one").

The StageRef family is the canonical entry point for whole-chain overlay reference resolution: the whole-chain kinds (OVERLAY_INDEX_VS_STAGE, OVERLAY_DELTA_VS_STAGE) populate both Ref (the baseline stage to compare against) and Target (the stage whose result the comparison surface decorates) with StageRef instances. OverlayRef.Stage in types/overlay.go references the same StageRef struct — there is exactly one StageRef declaration in the codebase.

type Test added in v0.3.0

type Test struct {
	// Type is the statistical test to perform.
	Type TestType `json:"type"`

	// Field is the primary numeric field under test. Required for TEST_T,
	// TEST_WELCH, TEST_ANOVA_F, TEST_KS, and TEST_TREND. Omitted for
	// TEST_CHISQ (uses Rows × Cols instead).
	Field string `json:"field,omitempty"`

	// Field2 is the secondary numeric field reference used by paired or
	// bivariate tests (TEST_PEARSON_R, TEST_PAIRED_T). For TEST_PAIRED_T
	// the difference d = Field − Field2 is tested. For TEST_PEARSON_R
	// the correlation between Field and Field2 is reported. Empty for
	// every other test.
	Field2 string `json:"field2,omitempty"`

	// SplitBy is a categorical field whose distinct values partition Field
	// into groups. Required for two-sample TEST_T / TEST_WELCH and for
	// TEST_ANOVA_F. Two-sample tests expect exactly 2 groups; ANOVA expects
	// ≥ 2.
	SplitBy string `json:"split_by,omitempty"`

	// Rows is the row-axis categorical field for TEST_CHISQ contingency.
	Rows string `json:"rows,omitempty"`

	// Cols is the column-axis categorical field for TEST_CHISQ contingency.
	Cols string `json:"cols,omitempty"`

	// SubjectField identifies the within-subject grouping for
	// repeated-measures designs (TEST_ANOVA_RM). Each distinct value
	// represents one subject contributing one observation per condition
	// (SplitBy). Empty for every other test.
	SubjectField string `json:"subject_field,omitempty"`

	// Alpha is the significance level. Defaults to 0.05 when zero.
	// Must lie in the open interval (0, 1).
	Alpha float64 `json:"alpha,omitempty"`

	// Label is the output alias for the test result. When empty, defaults
	// to "<TYPE>" or "<TYPE>_<field>" depending on the operator.
	Label string `json:"label,omitempty"`

	// OrderBy supplies ordering keys for tests that need a sorted series
	// (TEST_TREND, TEST_KS when run as a series test). Tier-2 trend tests
	// typically reference an output column produced by an upstream window
	// or grouper.
	OrderBy []OrderKey `json:"order_by,omitempty"`

	// Params holds operator-specific configuration as raw JSON.
	//   TEST_T (one-sample): {"mu": <hypothesized mean>}
	//   TEST_T / TEST_WELCH (two-sample): {"variant": "welch"} (default)
	//   TEST_KS: {"alternative": "two-sided"|"less"|"greater"}
	//   TEST_TUKEY_HSD: {"ms_within": <f64>, "df_within": <f64>}
	//   TEST_TREND: {"variant": "mann_kendall"} (default)
	Params json.RawMessage `json:"params,omitempty"`
}

Test defines a single statistical test to evaluate during a Process run. Tests appear in two request slots: Request.Tests (tier 1, row-level) and Request.PostTests (tier 2, result-level). The shape is shared because most fields are common across tiers; per-test validation rules live in the registry and predict layer.

type TestResult added in v0.3.0

type TestResult struct {
	// Label echoes the request Label or the operator's synthesized default.
	Label string `json:"label,omitempty"`

	// Type is the test that produced this result.
	Type TestType `json:"type"`

	// Variant names the specific algorithm when a test supports more than
	// one (e.g. "welch_two_sample", "mann_kendall", "independence").
	Variant string `json:"variant,omitempty"`

	// Statistic is the test statistic (t, F, χ², KS D, Mann-Kendall S, etc.).
	Statistic float64 `json:"statistic"`

	// DF is the degrees of freedom. Tests with multiple DF values (e.g.
	// ANOVA's between/within) report the numerator DF here and surface the
	// remainder in Details.
	DF float64 `json:"df,omitempty"`

	// PValue is the two-sided p-value under the null hypothesis.
	PValue float64 `json:"p_value"`

	// Alpha is the significance threshold applied to RejectNull.
	Alpha float64 `json:"alpha"`

	// RejectNull is true when PValue < Alpha.
	RejectNull bool `json:"reject_null"`

	// Details holds operator-specific payload (per-group n/mean/variance,
	// contingency table, pairwise comparisons, confidence intervals,
	// effect-size measures). Marshals naturally as a nested JSON object.
	Details map[string]any `json:"details,omitempty"`

	// Warnings carries per-test diagnostics (low N, near-zero variance,
	// expected-count thresholds, etc.) so the envelope-level Warnings list
	// is not the only signal.
	Warnings []string `json:"warnings,omitempty"`
}

TestResult is the per-test outcome embedded in Response.Tests and Response.PostTests. Common statistics live as top-level fields; operator- specific payload (contingency tables, pairwise comparisons, effect sizes, per-group moments) lives in Details so the result shape stays predictable across the catalog.

type TestType added in v0.3.0

type TestType string

TestType identifies a specific statistical test operation.

Pulse splits statistical testing into two tiers, both expressed via this enum and both executed inside a single Process pipeline:

  • Tier 1 (row tests): listed in Request.Tests, evaluated against the raw row stream alongside aggregators. Online-moments tests reuse the running mean/variance/n that aggregators already compute, so they add near-zero cost when their inputs overlap with active aggregations.
  • Tier 2 (post tests): listed in Request.PostTests, evaluated after the window stage on the materialized result row set. Useful for ANOVA across grouper buckets, post-hoc pairwise comparisons, and trend tests over windowed series.

Per-type semantics, required fields, and streamability are documented in skills/statistical-testing.md.

const (
	// TEST_T is a one-sample or two-sample Welch t-test on a numeric field.
	// When SplitBy is set, the test partitions Field by SplitBy and compares
	// the two resulting groups; otherwise it tests Field's mean against the
	// hypothesized value supplied in Params.
	TEST_T TestType = "TEST_T"

	// TEST_WELCH is an explicit two-sample Welch t-test alias used when the
	// caller wants to be unambiguous about the variant. Behaves identically
	// to TEST_T with SplitBy set; provided so requests document intent.
	TEST_WELCH TestType = "TEST_WELCH"

	// TEST_CHISQ is a chi-square independence test on a 2D contingency
	// table built from two categorical fields (Rows × Cols).
	TEST_CHISQ TestType = "TEST_CHISQ"

	// TEST_ANOVA_F is a one-way analysis of variance F-test comparing the
	// means of a numeric Field across k groups defined by a categorical
	// SplitBy field. k must be ≥ 2.
	TEST_ANOVA_F TestType = "TEST_ANOVA_F"

	// TEST_KS is a Kolmogorov-Smirnov two-sample distribution test. Not
	// streamable — requires sorted ECDFs.
	TEST_KS TestType = "TEST_KS"

	// TEST_TUKEY_HSD is a post-hoc pairwise comparison of group means using
	// Tukey's Honestly Significant Difference. Tier-2 only: consumes the
	// per-group means and counts produced by upstream aggregators.
	TEST_TUKEY_HSD TestType = "TEST_TUKEY_HSD"

	// TEST_TREND is a Mann-Kendall trend test over an ordered numeric
	// series. Tier-2 typical: runs over windowed result rows; tier-1
	// possible when the raw field is naturally ordered by an OrderBy key.
	TEST_TREND TestType = "TEST_TREND"

	// TEST_PEARSON_R is the parametric correlation test between two
	// numeric fields. Streams via an extended Welford recurrence that
	// also tracks the running cross-product. p-value via the t-statistic
	// r·√((n−2)/(1−r²)) with df = n − 2.
	TEST_PEARSON_R TestType = "TEST_PEARSON_R"

	// TEST_PAIRED_T is the paired-sample t-test on the per-row difference
	// d = Field − Field2. Streams via the same Welford machinery as the
	// one-sample TEST_T, driven by two-field reads.
	TEST_PAIRED_T TestType = "TEST_PAIRED_T"

	// TEST_PROP_Z is the two-proportion z-test on a categorical SplitBy
	// field where each row counts as a success or failure based on
	// Params.success (the dictionary value treated as a "success" on
	// Field). Streams via per-group success / total counts.
	TEST_PROP_Z TestType = "TEST_PROP_Z"

	// TEST_MANN_WHITNEY_U is the nonparametric two-sample alternative to
	// TEST_T. Buffered: ranks the combined value set under tie correction,
	// sums the ranks in group A → R_A, then computes
	//   U_A = R_A − n_A(n_A+1)/2
	// Two-sided p-value via the normal approximation with tie correction.
	TEST_MANN_WHITNEY_U TestType = "TEST_MANN_WHITNEY_U"

	// TEST_WILCOXON_SR is the Wilcoxon signed-rank test on the per-row
	// difference d = Field − Field2. Buffered: drops zero-diff pairs,
	// ranks |d| under tie correction, sums positive-sign ranks → W.
	// Two-sided p-value via the normal approximation with tie correction.
	TEST_WILCOXON_SR TestType = "TEST_WILCOXON_SR"

	// TEST_KRUSKAL_WALLIS is the nonparametric k-group alternative to
	// TEST_ANOVA_F. Buffered: ranks the combined value set under tie
	// correction, sums ranks per group, then
	//   H = (12/(N(N+1))) · Σ (R_i²/n_i) − 3(N+1)
	// p-value via chi-square survival with k−1 df.
	TEST_KRUSKAL_WALLIS TestType = "TEST_KRUSKAL_WALLIS"

	// TEST_SPEARMAN_R is the rank-based correlation between Field and
	// Field2 (monotonic association). Buffered: mid-ranks each column
	// under tie correction, then runs Pearson on the ranks. p-value via
	//   t = ρ · √((n−2)/(1−ρ²))   df = n − 2
	TEST_SPEARMAN_R TestType = "TEST_SPEARMAN_R"

	// TEST_KENDALL_TAU is concordance-based correlation between Field
	// and Field2. Buffered O(n²) pair count under tie correction. Two-
	// sided p-value via normal approximation with the standard variance
	// adjustment for ties in either column.
	TEST_KENDALL_TAU TestType = "TEST_KENDALL_TAU"

	// TEST_ANOVA_WELCH is the heteroscedasticity-robust one-way ANOVA.
	// Streams via the same per-group Welford buckets as TEST_ANOVA_F.
	// Statistic uses per-group weights w_i = n_i / s²_i, with the
	// Welch-Satterthwaite denominator correction for unequal variances.
	TEST_ANOVA_WELCH TestType = "TEST_ANOVA_WELCH"

	// TEST_ANOVA_RM is the repeated-measures one-way ANOVA. Buffered:
	// requires SubjectField to pivot rows into the wide subject ×
	// condition table. Decomposes SS into between-subjects, treatment,
	// and error components; F = MS_treatment / MS_error.
	TEST_ANOVA_RM TestType = "TEST_ANOVA_RM"

	// TEST_BROWN_FORSYTHE is a homogeneity-of-variance test. Replaces
	// each value with its absolute deviation from the group median, then
	// runs one-way ANOVA on the deviations. Buffered: per-group medians
	// require a sort.
	TEST_BROWN_FORSYTHE TestType = "TEST_BROWN_FORSYTHE"

	// TEST_FISHER_EXACT is the exact two-sided p-value for a 2×2
	// contingency table. The canonical small-sample alternative to
	// TEST_CHISQ when any expected cell count is below 5. Buffered:
	// requires the full contingency table.
	TEST_FISHER_EXACT TestType = "TEST_FISHER_EXACT"

	// TEST_SHAPIRO_WILK is the Shapiro-Wilk normality test on Field.
	// When SplitBy is set the test runs per-group; otherwise it runs
	// over the full filtered set. Buffered: requires the ordered values.
	// Implementation supports n ≤ 5000; n above the bound surfaces a
	// PULSE_TEST_SHAPIRO_N_BOUND warning.
	TEST_SHAPIRO_WILK TestType = "TEST_SHAPIRO_WILK"

	// TEST_Z_TWO_SAMPLE is the two-sample z-test on the means of a
	// numeric Field across two groups defined by a categorical SplitBy.
	// Mathematically identical to TEST_WELCH for the test statistic and
	// standard error, but the p-value is computed from the standard
	// normal CDF Φ rather than the Student-t CDF T_df. The variant
	// matches large-sample survey conventions (e.g. weighted-proportion
	// tests already computed as means with an n denominator) where the
	// Student-t correction is not desired. For small n the divergence
	// from TEST_WELCH is non-trivial; predict surfaces no warning, so
	// callers choose intentionally. Streams via the same per-group
	// Welford buckets as TEST_T / TEST_WELCH.
	TEST_Z_TWO_SAMPLE TestType = "TEST_Z_TWO_SAMPLE"
)

func AllTestTypes added in v0.3.0

func AllTestTypes() []TestType

AllTestTypes returns every defined statistical test type in alphabetical order.

func (TestType) Streamable added in v0.3.0

func (t TestType) Streamable() bool

Streamable reports whether this test type can be evaluated in the streaming Process path as a tier-1 row test. Two tiers exist at runtime:

  • Online-moments tests (TEST_T, TEST_WELCH, TEST_CHISQ, TEST_ANOVA_F) reuse the running (mean, variance, n) and per-bucket counts already produced by the streaming aggregator path. They consume zero extra passes when their inputs overlap with an active aggregator.
  • Buffered-only tests (TEST_KS, TEST_TUKEY_HSD, TEST_TREND) require a sorted view, a finalized post-hoc matrix, or an ordered series and cannot stream. Predict flags requests containing these tests as non-streamable.

Tier-2 PostTests are always buffered regardless of TestType: they execute over the materialized result row set, after windows. Streamable here reports tier-1 capability only.

Default branch returns false so newly-added test types must opt in.

type VersionResponse

type VersionResponse struct {
	// Version is the semantic version string.
	Version string `json:"version"`

	// BuildDate is the build timestamp.
	BuildDate string `json:"build_date,omitempty"`

	// GoVersion is the Go compiler version used.
	GoVersion string `json:"go_version,omitempty"`
}

VersionResponse provides build and version information.

type Window added in v0.2.0

type Window struct {
	// Type is the window operation to perform.
	Type WindowType `json:"type"`

	// Field is the source field name. Required for all operators except
	// ROW_NUMBER, RANK, and DENSE_RANK.
	Field string `json:"field,omitempty"`

	// Label is the output column name. When empty, defaults to "<TYPE>_<field>"
	// or "<TYPE>" for ROW_NUMBER/RANK/DENSE_RANK.
	Label string `json:"label,omitempty"`

	// PartitionBy is the list of fields whose distinct values define
	// independent partitions for window evaluation. Empty means a single
	// partition over all rows.
	PartitionBy []string `json:"partition_by,omitempty"`

	// OrderBy is the list of order keys. Required (≥1) for every window operator.
	OrderBy []OrderKey `json:"order_by"`

	// Frame is the window frame specification. Required for RUNNING_*, MOVING_AVG,
	// and EWMA. Rejected for LAG, LEAD, ROW_NUMBER, RANK, DENSE_RANK, PCT_CHANGE.
	Frame *FrameSpec `json:"frame,omitempty"`

	// Params holds operator-specific parameters as raw JSON.
	// LAG/LEAD: {"offset": 1, "default": null}
	// EWMA: {"alpha": 0.5}
	// PCT_CHANGE: {"periods": 1}
	Params json.RawMessage `json:"params,omitempty"`
}

Window defines a window operation.

type WindowType added in v0.2.0

type WindowType string

WindowType identifies a specific window operation.

const (
	WIN_LAG         WindowType = "WIN_LAG"
	WIN_LEAD        WindowType = "WIN_LEAD"
	WIN_ROW_NUMBER  WindowType = "WIN_ROW_NUMBER"
	WIN_RANK        WindowType = "WIN_RANK"
	WIN_DENSE_RANK  WindowType = "WIN_DENSE_RANK"
	WIN_RUNNING_SUM WindowType = "WIN_RUNNING_SUM"
	WIN_RUNNING_AVG WindowType = "WIN_RUNNING_AVG"
	WIN_MOVING_AVG  WindowType = "WIN_MOVING_AVG"
	WIN_EWMA        WindowType = "WIN_EWMA"
	WIN_PCT_CHANGE  WindowType = "WIN_PCT_CHANGE"
)

func AllWindowTypes added in v0.2.0

func AllWindowTypes() []WindowType

AllWindowTypes returns all defined window types in alphabetical order.

func (WindowType) Streamable added in v0.2.0

func (t WindowType) Streamable() bool

Streamable reports whether this window type can be computed without buffering. All window operators run over the post-aggregate row set in a final pass; none stream today.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL