descriptor

package
v1.0.0-alpha.6 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 9, 2026 License: MIT Imports: 1 Imported by: 0

Documentation

Overview

Package descriptor holds the result and envelope types of Pulse's no-execute self-description surface: the --json Envelope, the manifest (Manifest and its capability blocks), InspectResult and PredictResult, and the per-operator ComponentSchema contract. The builders that produce them (manifest, payload schema, predict, inspect) are internal; reach them through the root pulse facade.

Index

Constants

View Source
const (
	// Mergeable forwards types.ComponentsMergeable. The components map
	// folds across chunks via the same associative/commutative path
	// as the scalar value (Welford-family, sums, counts, set masks,
	// weighted accumulators).
	Mergeable = types.ComponentsMergeable

	// Partial forwards types.ComponentsPartial. The components map
	// merges across chunks but at non-trivial allocation cost — map /
	// set unions where the fold is associative but not constant-
	// space (AGG_MODE_COUNT, AGG_MODE, AGG_DISTINCT_COUNT,
	// AGG_SET_FREQUENCY). The orchestrator may stage the merge at
	// terminal flush.
	Partial = types.ComponentsPartial

	// None forwards types.ComponentsNone. The components map cannot
	// be computed from a per-chunk partial — the operator needs a
	// sorted view (or equivalent) of the full input (AGG_MEDIAN,
	// AGG_PERCENTILE, GROUP_QUANTILE). Streaming chunks omit
	// components for these operators; predict declares the slot as
	// buffered-components-only.
	None = types.ComponentsNone
)
View Source
const (
	// PValueBasisExact: every contribution is fixed by the request
	// shape (a test, a whole-table overlay).
	PValueBasisExact = "exact"
	// PValueBasisDictionary: at least one overlay's count was read off
	// a crosstab axis's categorical dictionary, assuming every
	// dictionary entry is a populated bucket (an unobserved entry
	// lowers the real count, a null bucket raises it).
	PValueBasisDictionary = "dictionary"
	// PValueBasisLowerBound: at least one overlay's extent could not be
	// derived from the schema (a numeric or multi-field axis); it
	// counted as one p-value, so Total and Uncorrected are floors.
	PValueBasisLowerBound = "lower_bound"
)

How a PValueCount was derived, weakest contribution wins.

View Source
const (
	// ZoneSourceSlot: the slot's own `tz`.
	ZoneSourceSlot = "slot"
	// ZoneSourceRequest: the request root's `time_zone`.
	ZoneSourceRequest = "request"
	// ZoneSourceOptions: pulse.Options.DefaultTimeZone.
	ZoneSourceOptions = "options"
	// ZoneSourceDefault: nothing named a zone; UTC.
	ZoneSourceDefault = "default"
)

Zone-resolution sources reported on ResolvedZone.Source.

View Source
const (
	// WeightSourceSlot: the slot's own `weight` (a `null` included).
	WeightSourceSlot = "slot"
	// WeightSourceRequest: the request root's `weight`.
	WeightSourceRequest = "request"
	// WeightSourceOptions: pulse.Options.DefaultWeight.
	WeightSourceOptions = "options"
	// WeightSourceNone: nothing named a weight.
	WeightSourceNone = "none"
)

Weight-resolution sources reported on ResolvedWeight.Source.

View Source
const (
	// WeightStatusApplied: the slot's operator consumes the resolved
	// weight.
	WeightStatusApplied = "applied"
	// WeightStatusSkippedNotWeightAware: a weight resolved, but the
	// slot's operator does not consume one; the slot runs unweighted.
	WeightStatusSkippedNotWeightAware = "skipped_not_weight_aware"
	// WeightStatusOptedOut: the slot's own `weight` is `null`.
	WeightStatusOptedOut = "opted_out"
	// WeightStatusNone: no weight resolves for the slot.
	WeightStatusNone = "none"
)

Weight-resolution outcomes reported on ResolvedWeight.Status.

View Source
const MultiplicityTriggerThreshold = 10

MultiplicityTriggerThreshold is the uncorrected p-value count at or above which correction is worth suggesting: with ten independent tests at alpha 0.05 the chance of at least one false positive is already about 40%. Echoed on PValueCount.Threshold.

View Source
const RoleUnbounded = -1

RoleUnbounded is the Role.Max value meaning "no upper bound on how many fields may fill this role".

View Source
const WeightSuggestionSourceSPSSSidecar = "spss_sidecar"

WeightSuggestionSourceSPSSSidecar is SuggestedWeight.Source for a weight read from the cohort's SPSS metadata sidecar (cohort.pulse.spss.json).

Variables

This section is empty.

Functions

This section is empty.

Types

type AggregationPredict added in v0.20.0

type AggregationPredict struct {
	// Type echoes the aggregation type from the request slot, in the
	// canonical SCREAMING_SNAKE form (e.g. "AGG_MEDIAN").
	Type types.AggregationType `json:"type"`

	// Field echoes the field referenced by the request slot. Empty
	// when the slot was authored without a field (defaults may have
	// filled it; the resolved post-defaults clone is reflected here
	// when applicable).
	Field string `json:"field,omitempty"`

	// Label echoes the response-side label the caller assigned to the
	// slot (or the engine-default label rule fallback when empty).
	// Predict echoes the raw label only — label fallback synthesis is
	// the orchestrator's responsibility.
	Label string `json:"label,omitempty"`

	// ComponentSchema is the static schema for the operator the engine
	// will use at runtime, projected from the capabilities table
	// (capabilities_aggregators.go for built-ins; the embedder snapshot
	// for extensions). Empty Keys + empty Mergeability means the slot
	// references an extension operator that has not (yet) declared a
	// ComponentSchema — the universal-floor populator still attaches
	// {"n", "n_null"} at runtime.
	ComponentSchema ComponentSchema `json:"component_schema"`

	// BufferedComponents is true iff ComponentSchema.Mergeability is
	// descriptor.None. The flag advertises that the slot's Components
	// block will arrive only on the terminal buffered flush in a
	// streaming run; it does NOT flip the overall Streamable axis (the
	// data slice still streams). AGG_MEDIAN and AGG_PERCENTILE are the
	// canonical Nones today.
	BufferedComponents bool `json:"buffered_components,omitempty"`
}

AggregationPredict is the per-slot predict surface for one entry of req.Aggregations. Carries the operator identity echoed back to the caller plus the static ComponentSchema declared in the capabilities table — so callers can budget the runtime Components block shape before the engine runs and so streaming consumers can branch on the BufferedComponents hint without consulting the manifest.

Predict stays no-execute: every field on this struct is sourced from the descriptor capability projection (or the embedder snapshot for extension operators) — never from a constructed operator. Custom operators registered via pulse.Options.Extensions that do not (yet) supply a ComponentSchema produce an empty ComponentSchema and BufferedComponents=false; the universal-floor populator at orchestrator emission time still attaches {"n", "n_null"} at runtime.

type Alternative

type Alternative struct {
	// When describes the situation in plain words.
	When string `json:"when"`
	// Use names the better choice: a bare registered operator name
	// (e.g. "TEST_WELCH") or a "<kind>:<name>" feature spelling with
	// kind one of capability, operator, io_format or mcp_extra.
	Use string `json:"use"`
}

Alternative is one "not for" entry in a Purpose: when the operator is the wrong choice, and what to use instead.

type Band

type Band struct {
	// Min is the inclusive lower bound; nil means unbounded below.
	Min *float64 `json:"min,omitempty"`
	// Max is the exclusive upper bound; nil means unbounded above.
	Max *float64 `json:"max,omitempty"`
	// Label names the band (e.g. "small", "moderate").
	Label string `json:"label"`
}

Band is one labelled interval of an output value, half-open [Min, Max). A nil Min or Max is an open end.

type CohortFieldType

type CohortFieldType struct {
	Name                  string   `json:"name"`
	Categorical           bool     `json:"categorical"`
	ShardedCapable        bool     `json:"sharded_capable"`
	CompatibleAggregators []string `json:"compatible_aggregators,omitempty"`
	CompatibleAttributes  []string `json:"compatible_attributes,omitempty"`
	CompatibleFilterers   []string `json:"compatible_filterers,omitempty"`
	CompatibleGroupers    []string `json:"compatible_groupers,omitempty"`
	CompatibleWindows     []string `json:"compatible_windows,omitempty"`
	CompatibleFeatures    []string `json:"compatible_features,omitempty"`
}

CohortFieldType describes a field type available in .pulse files and the operator catalog that accepts it. The Compatible* slices are derived from the per-operator AcceptsTypes declarations and let an LLM look up "what can I do with a date field" in one place.

ShardedCapable reports whether the type participates in a shard archive without restriction. Every built-in field type is sharded- capable today; the flag exists for forward compatibility with future types that might not work across the union of shards (e.g. types whose semantics depend on per-shard locality). Embedders should treat the flag as advisory.

type Command

type Command struct {
	Name        string `json:"name"`
	Description string `json:"description"`

	// Annotations carries the three per-command capability hints
	// (streamable / deterministic / expensive). Embedders use these to
	// decide whether to wrap a command in caching or invoke it directly.
	// Always populated for built-in commands.
	Annotations CommandAnnotations `json:"annotations"`
}

Command describes a CLI leaf command in the manifest.

type CommandAnnotations added in v0.11.0

type CommandAnnotations struct {
	Streamable    bool `json:"streamable"`
	Deterministic bool `json:"deterministic"`
	Expensive     bool `json:"expensive"`
}

CommandAnnotations carries three capability flags per command.

  • Streamable: the command has a streaming variant. Callers can invoke the streaming form for incremental output.
  • Deterministic: the command produces the same output given the same inputs (including the source file's content hash). Callers can safely cache results keyed by the request hash.
  • Expensive: the command is worth caching. Cheap operations may not be worth the cache machinery; expensive ones (regression, filter-to-file, profile) typically are. Hint to consumers, not a hard constraint.

type ComponentKey added in v0.20.0

type ComponentKey struct {
	// Name is the snake_case key the operator emits into the runtime
	// map[string]any returned by MetaAggregator.Components.
	Name string `json:"name"`
	// Type is the human-readable kind name of the emitted value.
	Type string `json:"type"`
	// Description is a single-sentence prose description suitable for
	// manifest / predict / MCP surfaces.
	Description string `json:"description"`
	// Optional marks a key that is emitted only under a condition the
	// Description states — the weighted floor keys (sum_weights, n_eff,
	// n_weight_invalid) appear only on a weighted slot. Omitted (false)
	// for a key every run emits.
	Optional bool `json:"optional,omitempty"`
}

ComponentKey describes one named entry in an operator's components map. The triple (Name, Type, Description) is the manifest projection embedders consume to learn what shape of value to expect at runtime.

Type is a human-readable kind string (e.g. "int", "float64", "map[string]int", "[]string", "WelfordTriple") rather than a reflect.Kind so the manifest can serialize it directly into JSON without an extra mapping table. Implementations may use anything that round-trips through manifest consumers — keep it short and unambiguous.

Name is the wire-stable snake_case key the operator emits into the runtime components map (e.g. "mean", "variance", "mode_count", "range_min"). Description is a single sentence in plain prose used by the manifest, predict, and MCP surfaces. Both are required.

type ComponentSchema added in v0.20.0

type ComponentSchema struct {
	// Keys enumerates the named operator-specific components, in
	// emission order. May be empty when only the universal floor
	// applies.
	Keys []ComponentKey `json:"keys,omitempty"`
	// Mergeability classifies how the components map folds across
	// streaming chunks. Required for every schema (no zero-value
	// default — embedders must declare intent explicitly).
	Mergeability ComponentsMergeability `json:"mergeability"`
}

ComponentSchema is the per-operator declaration carried by every descriptor capability entry. The Keys slice enumerates the operator- specific components emitted at runtime (the universal floor of {"n", "n_null"} is filled unconditionally by the orchestrator and is NOT listed here); Mergeability classifies how those components fold across streaming chunks.

An empty Keys slice paired with Mergeability == Mergeable is a valid schema — it declares an operator whose only contribution is the universal floor (AGG_COUNT is the canonical example). Consumers receiving an empty Keys slice still see {"n", "n_null"} in the runtime response under the orchestrator's universal-floor pass.

Manifest serialization preserves Keys order — declare keys in a stable order matching the operator's emission for golden-test readability.

type Components

type Components struct {
	Aggregators []Operator `json:"aggregators"`
	Attributes  []Operator `json:"attributes"`
	Filterers   []Operator `json:"filterers"`
	Groupers    []Operator `json:"groupers"`
	Windows     []Operator `json:"windows"`
	Features    []Operator `json:"features"`
}

Components lists every registered processing component grouped by category. Each slice carries one Operator entry per component so LLM-side authoring has access to per-operator params, accepted field types, emit type, and streamability without further discovery round-trips.

type ComponentsMergeability added in v0.20.0

type ComponentsMergeability = types.ComponentsMergeability

ComponentsMergeability is re-exported from types/ where the canonical enum lives (see types/streamability.go). Descriptor aliases the type and forwards the constants so existing capability-file declarations (descriptor.Mergeable / descriptor.Partial / descriptor.None) and predict-side switches keep compiling unchanged while the source of truth is the leaf-package types declaration. Wire JSON values are "mergeable" / "partial" / "none".

type ComponentsSchemasBlock added in v0.20.0

type ComponentsSchemasBlock struct {
	Aggregators map[string]ComponentSchema `json:"aggregators,omitempty"`
	Groupers    map[string]ComponentSchema `json:"groupers,omitempty"`
	Filterers   map[string]ComponentSchema `json:"filterers,omitempty"`
	Matrices    map[string]ComponentSchema `json:"matrices,omitempty"`
}

ComponentsSchemasBlock surfaces the per-operator components contract (ResponseComponents.Components[] payload shape) at manifest level so LLM clients can reason about meta-fields without crawling per- category Operator entries. Each map is keyed by operator name and sorted deterministically at serialization time.

An entry's ComponentSchema mirrors the same value carried on the per-Operator entry under Components.Aggregators[i].ComponentSchema. The two surfaces are deliberately redundant — Operator-level keeps the per-operator entry self-contained for category-by-category drilldown, while this top-level block lets a client materialize the whole meta-fields catalog in one O(N) scan.

type CrosstabCapability added in v0.12.0

type CrosstabCapability struct {
	// Name is the canonical entry identifier ("crosstab").
	Name string `json:"name"`

	// NormalizeModes lists every valid CrosstabSpec.Normalize value,
	// sorted alphabetically.
	NormalizeModes []string `json:"normalize_modes"`

	// Shapes lists every valid CrosstabSpec.Shape value, sorted
	// alphabetically.
	Shapes []string `json:"shapes"`

	// SummableAggregators is the alphabetized list of aggregator names
	// whose margin equals the sum of cells. v1 recomputes every margin
	// from raw rows for correctness; the classification is exposed so
	// callers can reason about future fast-path optimizations.
	SummableAggregators []string `json:"summable_aggregators"`
	// MeanReducibleAggregators lists aggregators whose margin is
	// derivable when each cell also carries its count.
	MeanReducibleAggregators []string `json:"mean_reducible_aggregators"`
	// IndependentAggregators lists aggregators that maintain their own
	// margin accumulator — the margin is neither a sum of cells nor a
	// re-scan (distinct_count, distinct_sum). Both crosstab paths feed
	// every record into independent row / column / grand accumulators,
	// so the margin is exact after one pass.
	IndependentAggregators []string `json:"independent_aggregators"`
	// RecomputeAggregators lists aggregators whose margin cannot be
	// derived from cells and must be recomputed over the raw filter-
	// passing rows.
	RecomputeAggregators []string `json:"recompute_aggregators"`

	// StreamingExceptions documents the only request shape that may
	// stream through the standard grouped path: shape=long, no
	// margins, no normalization. Intended for LLM-side reasoning.
	StreamingExceptions []string `json:"streaming_exceptions"`

	// RejectionRules names the structural failures the validator
	// emits. Intended for LLM-side autocomplete; not a strict schema.
	RejectionRules []string `json:"rejection_rules"`

	// SupportsNormalizeLevel reports whether the engine honors
	// CrosstabSpec.NormalizeLevel — the depth-index selector that
	// picks a parent grouper as the 100% denominator on nested row /
	// column axes. Always true in v1; carried so clients can detect
	// the feature without inferring it from CLI version.
	SupportsNormalizeLevel bool `json:"supports_normalize_level"`

	// NormalizeLevelRules names the rejection rules specific to the
	// NormalizeLevel slot. Intended for LLM-side autocomplete.
	NormalizeLevelRules []string `json:"normalize_level_rules"`

	// SupportsNormalizeWithin reports whether the engine honors
	// CrosstabSpec.NormalizeWithin — the cross-axis partition
	// selector that fixes a prefix of the OPPOSITE axis (columns
	// when normalize=row, rows when normalize=column) inside the
	// 100% denominator. Composes independently with NormalizeLevel.
	SupportsNormalizeWithin bool `json:"supports_normalize_within"`

	// NormalizeWithinRules names the rejection rules specific to the
	// NormalizeWithin slot.
	NormalizeWithinRules []string `json:"normalize_within_rules"`

	// SupportsMarginAggregations reports whether the engine honors
	// CrosstabSpec.MarginAggregations — the auxiliary aggregations
	// evaluated into the row / column / grand margin accumulators only
	// and never into a cell. The canonical use is a respondent base
	// riding alongside a weighted metric without a second scan.
	SupportsMarginAggregations bool `json:"supports_margin_aggregations"`

	// MarginAggregationRules names the rejection rules specific to the
	// MarginAggregations slot, plus the advisory that fires when the
	// section emits no margin to carry the auxiliary figures.
	MarginAggregationRules []string `json:"margin_aggregation_rules"`

	// MapValuedCellAggregators is the alphabetized list of aggregator
	// names whose Crosstab Cell output is map-valued (rich payload
	// per cell, e.g. AGG_SET_FREQUENCY's per-label row counts). These
	// aggregators emit map[string]int into MatrixCell.Value and the
	// long-shape cell rows; normalize modes (row / column / total) are
	// incompatible because dividing one map by another is undefined.
	// Derived from types.AggregationType.MapValued() at build time.
	MapValuedCellAggregators []string `json:"map_valued_cell_aggregators"`
}

CrosstabCapability describes the cross-tabulation endpoint surfaced via Request.Crosstab. The manifest carries one CrosstabCapability entry under Manifest.Crosstab so LLM clients can detect the section, learn the supported normalize / shape values, and route between the matrix and long shapes appropriately.

type DefaultApplied added in v0.5.0

type DefaultApplied struct {
	Path     []string `json:"path"`
	Field    string   `json:"field"`
	Type     string   `json:"type"`
	Category string   `json:"category"`
	Reason   string   `json:"reason"`
}

DefaultApplied describes a single defaulted operator slot. Returned by ResolveDefaults so callers (predict, service, MCP transcripts) can echo exactly which slots had their Type inferred from the schema. The shape is JSON-serializable and lands on PredictResult.DefaultsApplied.

Path uses JSON-style segments e.g. ["Aggregations", "0", "Type"] so the caller can address the slot in the original request. Field names the schema column that drove inference; Type is the operator string that was filled in; Category is "aggregation" or "grouper"; Reason carries a short human-readable rule trace, e.g. "f64 → AGG_SUM (rule: numeric default)".

type DictionaryInfo

type DictionaryInfo struct {
	TotalEntries int      `json:"total_entries"`
	Truncated    bool     `json:"truncated"`
	Values       []string `json:"values"`
}

DictionaryInfo describes the categorical dictionary for a field.

type DistributionMeta added in v0.5.0

type DistributionMeta struct {
	// Name is the distribution kind identifier (e.g. "lognormal").
	Name string `json:"name"`

	// Description is a one-sentence prose summary.
	Description string `json:"description"`

	// AppliesTo lists the FieldSpec types the distribution can drive.
	// Values are coarse families: "numeric", "categorical", "date",
	// "bool", "any".
	AppliesTo []string `json:"applies_to"`

	// Params lists the distribution-specific parameters.
	Params []Param `json:"params"`

	// Intents lists the intent-taxonomy IDs (Manifest.Intents) the
	// distribution serves, sorted. Omitted when it declares no purpose.
	Intents []string `json:"intents,omitempty"`
}

DistributionMeta describes a synth distribution entry. One entry per internal/synth.AllDistributions() value.

type Domain

type Domain string

Domain is an audience domain a Purpose's use cases are written for. The set is closed: DomainSurvey, DomainOps, DomainScience and DomainHarness.

const (
	// DomainSurvey is survey and market-research analysis: ratings,
	// segments, waves, weighted respondents.
	DomainSurvey Domain = "survey"
	// DomainOps is operational and business analytics: orders,
	// channels, funnels, service metrics.
	DomainOps Domain = "ops"
	// DomainScience is scientific and experimental analysis:
	// measurements, treatments, trials.
	DomainScience Domain = "science"
	// DomainHarness is automated callers — agent harnesses and
	// pipelines that author requests programmatically.
	DomainHarness Domain = "harness"
)

The audience domains a Purpose.UseCases map may be keyed by.

type Envelope

type Envelope struct {
	FormatVersion string           `json:"format_version"`
	Data          any              `json:"data"`
	Request       any              `json:"request,omitempty"`
	Errors        []*EnvelopeEntry `json:"errors"`
	Warnings      []*EnvelopeEntry `json:"warnings"`
}

Envelope is the standard JSON output wrapper for all descriptor operations. All --json output follows this shape.

Request, when non-nil, carries the *normalized* request the operation executed — smart defaults resolved, label bindings expanded, per-stage requests captured for chains. Opt-in via pulse.Options.EchoRequest / CLI --echo-request; absent (and omitted from JSON) by default so the envelope shape is unchanged for callers that do not need it. The shape varies by operation: *types.Request for process/predict, *types.ComposedRequest for compose, *types.ChainRequest for process-chain, *types.FacetRequest for facet, *types.SampleRequest for sample.

func NewEnvelope

func NewEnvelope(data any) *Envelope

NewEnvelope creates an envelope with the given data and no errors/warnings.

func NewEnvelopeWithRequest added in v0.13.0

func NewEnvelopeWithRequest(data, request any) *Envelope

NewEnvelopeWithRequest creates an envelope with both data and the normalized request that produced it. Use when EchoRequest is enabled at the operation boundary. Passing a nil request degrades to NewEnvelope semantics (the field is omitted from JSON via omitempty).

func (*Envelope) AddError

func (e *Envelope) AddError(code, message string, details map[string]any)

AddError appends a structured error to the envelope.

func (*Envelope) AddWarning

func (e *Envelope) AddWarning(code, message string, details map[string]any)

AddWarning appends a structured warning to the envelope.

func (*Envelope) MarshalJSON

func (e *Envelope) MarshalJSON() ([]byte, error)

MarshalJSON produces deterministic JSON output. It encodes through types.MarshalFinite, so an undefined figure (a non-finite float) anywhere in Data, Request or an entry's Details is written as JSON null rather than failing the whole envelope; output holding no non-finite float is byte-identical to json.Marshal.

func (*Envelope) WithRequest added in v0.13.0

func (e *Envelope) WithRequest(request any) *Envelope

WithRequest attaches a normalized request to an existing envelope. Returns the receiver for fluent use. Nil-safe.

type EnvelopeEntry

type EnvelopeEntry struct {
	Code    string         `json:"code"`
	Message string         `json:"message"`
	Details map[string]any `json:"details,omitempty"`
}

EnvelopeEntry represents a single error or warning in the envelope.

type ExportCapability added in v0.19.0

type ExportCapability struct {
	// Formats enumerates every format the export dispatcher supports,
	// sorted alphabetically by Name for deterministic golden output.
	// One entry per format. Empty slice never appears — Pulse always
	// ships at least the canonical seven adapters.
	Formats []ExportFormatCapability `json:"formats"`
}

ExportCapability is the cross-format export envelope. Carries the alphabetised per-format slice so LLM planners can pick a target format aware of its overlay-embedding shape without inspecting the io/ packages.

type ExportFormatCapability added in v0.19.0

type ExportFormatCapability struct {
	// Name is the canonical format identifier matching the
	// io.Format constants (csv / tsv / ndjson / jsonarray /
	// parquet / arrow / excel). Stable across format-version 1.0;
	// new formats land additively.
	Name string `json:"name"`

	// OverlaySupport names the per-format overlay-embedding strategy
	// the export dispatcher follows when ExportJob.IncludeOverlays
	// resolves to true. See ExportFormatCapability godoc for the
	// canonical label vocabulary.
	OverlaySupport string `json:"overlay_support"`

	// TimeZone is what the format does with ExportJob.TimeZone (CLI
	// --tz) naming a non-UTC zone:
	//
	//   - "local_offset" — datetime cells render as RFC 3339 literals on
	//     the zone's wall clock with its numeric offset
	//     (2026-03-29T08:00:00+05:30), which re-import to the same
	//     instants. Arrow and Parquet carry datetimes as UTF8 strings,
	//     so they follow the text formats.
	//   - "refused" — the export stops with a coded error (spss: a .sav
	//     DATETIME is a naive wall-clock count with no offset slot).
	//
	// An absent or UTC-equivalent zone is byte-identical UTC output on
	// every format.
	TimeZone string `json:"time_zone"`
}

ExportFormatCapability describes the per-format export envelope surfaced on the manifest. One entry per format the engine's export dispatch supports, carrying the format identifier plus the per-format overlay-embedding strategy LLM planners need to decide whether a Response.Overlays slice will round-trip through an ExportJob to the chosen target.

OverlaySupport is the canonical embedding-shape label declared by research/export-embedding-shape.md:

  • "sidecar" — the format carries a top-level LIST<STRUCT> column-family appended to the host stream (Arrow / Parquet). The host record stream is byte-identical to the overlay-free shape; readers that ignore the column family see the unchanged host.
  • "sheets" — one workbook sheet per overlay layer named "__overlay_<layer_name>" (Excel). The host workbook sheet is unchanged from the overlay- free shape.
  • "trailing_block" — a single trailing line `{"_overlays": [...]}` after the last host record (NDJSON). The host stream is byte-identical to the overlay- free shape until the trailer.
  • "warn_and_skip" — the format cannot embed overlays at all (CSV / TSV / SPSS). The dispatcher emits one PULSE_OVERLAY_EXPORT_CSV_UNSUPPORTED warning and writes the host body verbatim; no overlay output lands. Setting IncludeOverlays=false suppresses the warning while keeping the same body.

type ExprFunctionMeta added in v0.7.0

type ExprFunctionMeta struct {
	Name        string `json:"name"`
	Description string `json:"description,omitempty"`
	Signature   string `json:"signature,omitempty"`
	Pure        bool   `json:"pure,omitempty"`
}

ExprFunctionMeta is the manifest projection of an embedder- registered expression function.

type ExtensionsManifest added in v0.7.0

type ExtensionsManifest struct {
	Aggregators        []OperatorMeta     `json:"aggregators"`
	Attributes         []OperatorMeta     `json:"attributes"`
	Filterers          []OperatorMeta     `json:"filterers"`
	Groupers           []OperatorMeta     `json:"groupers"`
	Windows            []OperatorMeta     `json:"windows"`
	Features           []OperatorMeta     `json:"features"`
	Tests              []OperatorMeta     `json:"tests"`
	SynthDistributions []OperatorMeta     `json:"synth_distributions"`
	ExprFunctions      []ExprFunctionMeta `json:"expr_functions"`
	LookupTables       []LookupTableMeta  `json:"lookup_tables"`
	LabelTables        []LabelTableMeta   `json:"label_tables"`
	RangeTables        []RangeTableMeta   `json:"range_tables"`
}

ExtensionsManifest is the manifest-side projection of embedder- registered operators and expression-side state. Built-in operators continue to live in Manifest.Components; this block lists everything the host process adds on top, so a reviewer can see at a glance which subset is Pulse-shipped vs registered by the embedder.

All slices are sorted by Name for deterministic output. An empty Extensions block emits as nested empty slices (not null) for JSON-stability across releases.

type FacetCapability added in v0.7.0

type FacetCapability struct {
	// Name is the canonical entry identifier ("facet_schema").
	Name string `json:"name"`

	// SupportsDiscrete reports whether per-value count summaries are
	// available for categorical / boolean / geo fields.
	SupportsDiscrete bool `json:"supports_discrete"`

	// SupportsNumeric reports whether streaming statistics
	// (count, sum, min, max, mean, stddev) are produced for numeric
	// fields.
	SupportsNumeric bool `json:"supports_numeric"`

	// SupportsPercentiles reports whether NumericPercentiles are
	// computed via a buffered second-stage sort.
	SupportsPercentiles bool `json:"supports_percentiles"`

	// SupportsHistogram reports whether IncludeHistogram +
	// HistogramRange + HistogramBins are honoured.
	SupportsHistogram bool `json:"supports_histogram"`

	// SupportsAdditive reports whether AdditiveFields contribution
	// counts are computed.
	SupportsAdditive bool `json:"supports_additive"`

	// SupportsOverlays reports whether FacetRequest.Overlays is
	// honoured. When true, the SupportedOverlayKinds slice enumerates
	// the FACET-host overlay catalog kinds the endpoint dispatches.
	// Four FACET-host kinds wire into FacetSchema's buffered exit:
	// OVERLAY_INDEX_VS_POP / OVERLAY_ZSCORE_VS_POP / OVERLAY_CHISQ_VS_POP
	// / OVERLAY_KS_VS_POP.
	SupportsOverlays bool `json:"supports_overlays,omitempty"`

	// SupportedOverlayKinds enumerates the FACET-host overlay kinds the
	// endpoint dispatches. Populated only when SupportsOverlays is true;
	// each entry is a canonical OverlayKind constant string. Stable
	// alphabetical order so LLM clients see a deterministic enum.
	SupportedOverlayKinds []string `json:"supported_overlay_kinds,omitempty"`

	// StreamableConditions lists the human-readable rules that govern
	// which requests run in a single pass vs. force the buffered
	// secondary sort. The order is stable and intended for LLM-side
	// reasoning, not strict parsing.
	StreamableConditions []string `json:"streamable_conditions"`
}

FacetCapability describes the rich-facet endpoint surfaced by pulse.FacetSchema / pulse_facet_schema. The manifest carries one FacetCapability entry under Manifest.Facet so LLM clients can detect the endpoint's feature set without inspecting the source.

type FieldKind

type FieldKind string

FieldKind is the coarse family of a cohort field type, the vocabulary intent shapes are written in. Every registered field type maps to exactly one kind.

const (
	// FieldKindNumeric covers the unsigned integer, float and
	// decimal128 field types.
	FieldKindNumeric FieldKind = "numeric"
	// FieldKindCategorical covers the categorical_* field types.
	FieldKindCategorical FieldKind = "categorical"
	// FieldKindDate covers the date and datetime field types.
	FieldKindDate FieldKind = "date"
	// FieldKindBool covers the packed_bool field type.
	FieldKindBool FieldKind = "bool"
	// FieldKindSet covers the set_* multi-select field types.
	FieldKindSet FieldKind = "set"
)

The coarse field kinds.

type FiltererPredict added in v0.20.0

type FiltererPredict struct {
	// Type echoes the filterer type from the request slot, in the
	// canonical SCREAMING_SNAKE form (e.g. "FILTER_RANGE").
	Type types.FiltererType `json:"type"`

	// Field echoes the field referenced by the request slot. Empty
	// when the slot was authored without a field (FILTER_EXPRESSION
	// reads any record field through its expression body).
	Field string `json:"field,omitempty"`

	// ComponentSchema is the static schema for the filterer the engine
	// will use at runtime, projected from the capabilities table
	// (capabilities_filterers.go). Empty Keys + empty Mergeability
	// means the slot references an extension filterer that has not
	// (yet) declared a ComponentSchema — the universal-floor populator
	// still attaches {"n_in", "n_out", "n_null_input"} at runtime.
	ComponentSchema ComponentSchema `json:"component_schema"`

	// BufferedComponents is true iff ComponentSchema.Mergeability is
	// descriptor.None. The flag advertises that the slot's Components
	// block will arrive only on the terminal buffered flush in a
	// streaming run; it does NOT flip the overall Streamable axis. In
	// v1 every built-in filterer is Mergeable so this flag is always
	// false; the surface exists for future per-filter specifics that
	// may downgrade individual entries.
	BufferedComponents bool `json:"buffered_components,omitempty"`
}

FiltererPredict is the per-slot predict surface for one entry of req.Filterers. Carries the operator identity echoed back to the caller plus the static ComponentSchema declared in the capabilities table — so callers can budget the runtime Components.Filterers block shape before the engine runs and so streaming consumers can branch on the BufferedComponents hint without consulting the manifest.

Predict stays no-execute: every field on this struct is sourced from the descriptor capability projection — never from a constructed filterer. Custom filterers registered via pulse.Options.Extensions that do not (yet) supply a ComponentSchema produce an empty ComponentSchema and BufferedComponents=false; the universal-floor populator at orchestrator emission time still attaches {"n_in", "n_out", "n_null_input"} at runtime.

Components are uniform across every built-in filterer in v1: the universal floor {n_in, n_out, n_null_input} is the entire schema and Mergeability is Mergeable (counters fold trivially via integer addition). The struct mirrors AggregationPredict / GroupPredict so future per-filter specifics (e.g. n_below / n_above for FILTER_RANGE) can downgrade the slot to Partial / None without changing the predict shape.

type GroupPredict added in v0.20.0

type GroupPredict struct {
	// Type echoes the grouper type from the request slot, in the
	// canonical SCREAMING_SNAKE form (e.g. "GROUP_QUANTILE").
	Type types.GroupType `json:"type"`

	// Field echoes the field referenced by the request slot. Empty
	// when the slot was authored without a field (defaults may have
	// filled it; the resolved post-defaults clone is reflected here
	// when applicable).
	Field string `json:"field,omitempty"`

	// ComponentSchema is the static schema for the grouper the engine
	// will use at runtime, projected from the capabilities table
	// (capabilities_groupers.go). Empty Keys + empty Mergeability
	// means the slot references an extension grouper that has not
	// (yet) declared a ComponentSchema — the universal-floor
	// populator still attaches {"total_n", "n_null"} at runtime.
	ComponentSchema ComponentSchema `json:"component_schema"`

	// BufferedComponents is true iff ComponentSchema.Mergeability is
	// descriptor.None. The flag advertises that the slot's Components
	// block will arrive only on the terminal buffered flush in a
	// streaming run; it does NOT flip the overall Streamable axis (the
	// data slice still streams). GROUP_QUANTILE is the canonical
	// None today.
	BufferedComponents bool `json:"buffered_components,omitempty"`
}

GroupPredict is the per-slot predict surface for one entry of req.Groups. Carries the grouper identity echoed back to the caller plus the static ComponentSchema declared in the capabilities table — so callers can budget the runtime Components.Groupers block shape before the engine runs and so streaming consumers can branch on the BufferedComponents hint without consulting the manifest.

Predict stays no-execute: every field on this struct is sourced from the descriptor capability projection — never from a constructed grouper. Custom groupers registered via pulse.Options.Extensions that do not (yet) supply a ComponentSchema produce an empty ComponentSchema and BufferedComponents=false; the universal-floor populator at orchestrator emission time still attaches {"total_n", "n_null"} at runtime.

type ImportCapability added in v0.30.0

type ImportCapability struct {
	// Formats enumerates every format the import dispatch supports,
	// sorted alphabetically by Name for deterministic golden output.
	// The native "pulse" format is excluded: it needs no tabular
	// reader and passes through untouched.
	Formats []ImportFormatCapability `json:"formats"`
}

ImportCapability is the cross-format import envelope — the read-side peer of ExportCapability. Carries the alphabetised per-format slice so LLM planners can route a source file to `pulse import` (or pulse_import) knowing both that the format is readable and whether the cohort's types will be the source's own.

type ImportFormatCapability added in v0.30.0

type ImportFormatCapability struct {
	// Name is the canonical format identifier matching the io.Format
	// package constants (csv / tsv / ndjson / jsonarray / parquet /
	// arrow / excel / spss).
	Name string `json:"name"`

	// Extensions lists the lowercase file extensions (leading dot
	// included) that io.FormatFromPath resolves to this format, in the
	// dispatch's own order. Present so a planner can answer "what will
	// this path be detected as" without a round trip.
	Extensions []string `json:"extensions"`

	// SchemaSource is "authoritative" when the adapter implements
	// io.SchemaAwareReader and the source's own dictionary becomes the
	// .pulse schema, "inferred" when the shared sample-and-vote pass
	// decides the types. See ImportFormatCapability godoc.
	SchemaSource string `json:"schema_source"`

	// Export reports whether Pulse can also WRITE this format today.
	// False for import-only formats; cross-reference
	// ExportCapability.Formats for the writable set's overlay shapes.
	Export bool `json:"export"`
}

ImportFormatCapability describes one tabular source format the import dispatch (the io.NewReader factory) accepts, surfaced on the manifest so an LLM planner can answer "can Pulse read this file, and will the resulting cohort's types be the source's or a guess?" without crawling io/.

SchemaSource is the load-bearing slot and carries exactly two values:

  • "inferred" — the adapter yields rows of text and the shared inference pass (internal/io/infer.go) samples them and votes on a type per column. Correct in the common case, but it is a guess: a categorical column's dictionary is built in first-seen order, and a type can change with the sample window.
  • "authoritative" — the adapter implements io.SchemaAwareReader and hands over a schema its own source dictionary DECLARES. Inference is skipped entirely, so types, nullability and categorical dictionary ORDER come from the file rather than from its cell text.

Export reports whether the SAME format can also be written today. It is deliberately not inferable from this block's presence: a format being readable says nothing about whether a writer exists. Every format Pulse reads it can also write as of E5-S6, which mounted the `.sav` writer and flipped the last false here to true — the slot stays because the two halves are independent and a future read-only format would need it again. Cross-reference ExportCapability.Formats for the per-format overlay-embedding shape.

type InspectField

type InspectField struct {
	Name              string          `json:"name"`
	Type              string          `json:"type"`
	ByteOffset        int             `json:"byte_offset"`
	BitPosition       int             `json:"bit_position"`
	Description       string          `json:"description"`
	DescriptionSource string          `json:"description_source"`
	Categorical       bool            `json:"categorical"`
	Dictionary        *DictionaryInfo `json:"dictionary,omitempty"`
	// Precision is the decimal128 precision (1-38). Present only for
	// decimal128 / nullable_decimal128 fields.
	Precision *uint8 `json:"precision,omitempty"`
	// Scale is the decimal128 scale (0-precision). Present only for
	// decimal128 / nullable_decimal128 fields.
	Scale *uint8 `json:"scale,omitempty"`
	// Group is present only when the field is a parent-group member
	// (0x02): its value lives in that group's dictionary, not the row.
	// In a grouped cohort every field's ByteOffset is its LOGICAL-row
	// offset (the schema keeps the ungrouped descriptors), never a
	// physical one — Layout.PhysicalRecordStride is the on-disk row.
	Group *InspectFieldGroup `json:"group,omitempty"`
}

InspectField describes a single field in the inspect output.

type InspectFieldGroup added in v0.39.0

type InspectFieldGroup struct {
	Group int    `json:"group"`
	Key   bool   `json:"key"`
	Kind  string `json:"kind"`
}

InspectFieldGroup marks a field stored in a parent group's dictionary instead of the row. Kind "constant" means the field is constant-elided: one value for the whole cohort and no per-record bytes at all.

type InspectGroup added in v0.39.0

type InspectGroup struct {
	Group           int      `json:"group"`
	Label           string   `json:"label"`
	Kind            string   `json:"kind"`
	Key             []string `json:"key,omitempty"`
	Members         []string `json:"members"`
	Fields          []string `json:"fields"`
	EntryCount      int      `json:"entry_count"`
	EntryWidth      int      `json:"entry_width"`
	DictionaryBytes int64    `json:"dictionary_bytes"`
	MemberRowBytes  int      `json:"member_row_bytes"`
	IndexWidth      int      `json:"index_width"`
	Ratio           float64  `json:"ratio"`
	BreakEvenRatio  float64  `json:"break_even_ratio"`
	RatioFloor      float64  `json:"ratio_floor"`
	ByteDelta       int64    `json:"byte_delta"`
	GrowsFile       bool     `json:"grows_file"`
	Verdict         string   `json:"verdict"`
}

InspectGroup is one parent group's realized dedup figures.

Every number comes from internal/encoding.AssessGroup over the schema and the file-length record count, so inspect, import and retro-dedup report identical arithmetic. Ratio is record_count ÷ entry_count; DictionaryBytes is the group's dictionary, held RESIDENT in memory for as long as the cohort is open. Verdict applies the import-time ratio floor (internal/encoding.DefaultDedupRatioFloor) through the same GroupViability.Finding rule the import gate uses: "low_ratio" means the group would draw PULSE_DEDUP_LOW_RATIO at import today. Inspect reports it as a figure, never as an envelope warning — a deliberately deduped low-ratio group is a legitimate file.

Key / Members / Fields mirror the import report's GroupReport, so a group reads back as the `--group KEY:MEMBER` declaration that formed it: Key is the declared key (omitted when every member is key, i.e. a plain tuple group), Members the remaining members, Fields every member in logical (schema) order.

Label numbers the group by its position in the file; the format has no group-name slot and no declaration ordinal (see internal/encoding.GroupSpecOf(schema, g)), and it lists a keyless group's members in schema order rather than declaration order.

type InspectLayout added in v0.39.0

type InspectLayout struct {
	PulseFormatVersion   int `json:"pulse_format_version"`
	PhysicalRecordStride int `json:"physical_record_stride"`
	LogicalRecordStride  int `json:"logical_record_stride"`
}

InspectLayout is the on-wire row shape of a cohort whose format version is newer than 0x01. PhysicalRecordStride is what each record occupies on disk (group indices + row fields + the narrowed null bitmap); LogicalRecordStride is the row an ungrouped twin would carry — the width operators and the field listing reason in. Their difference is the per-record saving the groups buy, before the resident dictionaries are paid for.

type InspectOptions

type InspectOptions struct {
	// FullDict disables dictionary truncation when true.
	FullDict bool
	// DictionaryLimit overrides the default truncation limit.
	// Zero means use DefaultDictionaryLimit.
	DictionaryLimit int
}

InspectOptions controls inspect behavior.

type InspectResult

type InspectResult struct {
	FieldCount  int             `json:"field_count"`
	Fields      []*InspectField `json:"fields"`
	Shards      []ShardInfo     `json:"shards"`
	RecordCount int64           `json:"record_count"`
	// Layout describes the physical row of a 0x02 cohort. Omitted for 0x01.
	Layout *InspectLayout `json:"layout,omitempty"`
	// Groups reports each parent group in file order. Omitted when the
	// schema declares none.
	Groups []*InspectGroup `json:"groups,omitempty"`
	// SuggestedWeight names the row weight the cohort's own metadata
	// proposes — today the SPSS weighting variable recorded in the
	// cohort's SPSS metadata sidecar. A suggestion only: nothing ever
	// applies it. Omitted when there is no sidecar, the sidecar names
	// no weight, the variable is not a weight-capable column of this
	// schema, or the instance hides row weighting.
	SuggestedWeight *SuggestedWeight `json:"suggested_weight,omitempty"`
}

InspectResult holds the schema inspection output.

Shards is populated when the inspected file is a Pulse shard archive (first four bytes match the zip magic PK\x03\x04). Entries are listed in zip central-directory order, which equals shard insertion order. Single-file cohorts leave Shards as an empty slice (never nil) so the JSON envelope emits "shards": [] rather than null.

For archive-backed cohorts, FieldCount and Fields reflect the canonical schema carried in the reserved _schema.pulse entry, and RecordCount (when present in metadata) plus aggregate counting via per-shard headers populate the cumulative total — see ShardInfo.

RecordCount is populated on BOTH paths: the cumulative per-shard sum for an archive, and (payload_bytes / Schema.RecordByteSize) for a single-file cohort — derived from the file length, never by reading a record. A real 0 means an empty cohort. When the payload length is not a whole multiple of the record stride (a truncated tail) the count is the floor and the envelope carries an ENCODING_INVALID warning naming the leftover bytes.

Layout and Groups are present only for a cohort written in a format newer than 0x01 (parent groups, constant elision). A 0x01 cohort omits both keys, so its output is byte-identical to the pre-0x02 shape. Every figure in them is derived from the header, the schema block and the file length — the same header-only facts RecordCount uses — never from a record.

type Intent

type Intent struct {
	// ID is the stable identifier operator purposes and examples cite
	// (e.g. "compare_groups").
	ID string `json:"id"`
	// Label is a short human-readable name.
	Label string `json:"label"`
	// Analytic is true for a question answered by operators; false for
	// an intent that routes to tooling instead (prepare, simulate,
	// lookup).
	Analytic bool `json:"analytic"`
	// Sounds lists example phrasings of the question.
	Sounds []string `json:"sounds"`
	// Shapes lists the alternative data shapes the intent applies to.
	Shapes []Shape `json:"shapes"`
}

Intent is one entry in the closed intent taxonomy: a kind of question a non-statistician recognises in their own words.

type Interpretation

type Interpretation struct {
	// Field is the normalised output path (e.g. "statistic",
	// "p_value", "details.effect_size.cohens_d").
	Field string `json:"field"`
	// Means says what the value measures, in plain words.
	Means string `json:"means,omitempty"`
	// Bands labels value ranges; Convention is required when set.
	Bands []Band `json:"bands,omitempty"`
	// Abs is true when Bands apply to the absolute value.
	Abs bool `json:"abs,omitempty"`
	// Convention attributes the Bands (e.g. "Cohen (1988)") — bands are
	// a labelled convention, never authoritative.
	Convention string `json:"convention,omitempty"`
	// Sign maps "+" / "-" to what each direction means.
	Sign map[string]string `json:"sign,omitempty"`
	// Caveats lists plain-language warnings about the value.
	Caveats []string `json:"caveats,omitempty"`
	// Shared names a shared rule set this field follows instead of
	// declaring its own (e.g. "p-value").
	Shared string `json:"shared,omitempty"`
}

Interpretation says how to read one output field of an operator's result.

type JoinCapability added in v0.10.0

type JoinCapability struct {
	// Name is the canonical identifier ("hash_join").
	Name string `json:"name"`

	// MaxJoinsPerRequest is the upper bound on Request.Joins length.
	// 1 today; lifts when multi-join chains land.
	MaxJoinsPerRequest int `json:"max_joins_per_request"`

	// Kinds lists supported JoinSpec.Kind values. "inner" today;
	// "left", "outer", "anti" land once the null-bitmap correctness
	// path is fully wired.
	Kinds []string `json:"kinds"`

	// SpillBytes reports the in-memory threshold beyond which the
	// build side would spill. Zero today (no spill — the build side
	// is always materialised in RAM); set when the spill path lands.
	SpillBytes int64 `json:"spill_bytes"`

	// SpillEnv names the environment variable that overrides the
	// spill threshold. Set when the spill path lands.
	SpillEnv string `json:"spill_env,omitempty"`

	// Limitations lists the v1 envelope notes for LLM clients.
	Limitations []string `json:"limitations"`
}

JoinCapability describes the pushdown hash-join endpoint surfaced by Request.Joins. The manifest carries one JoinCapability entry under Manifest.Join so LLM clients can detect the v1 envelope (inner-only, single-join) without inspecting the source.

type LabelTableMeta added in v0.10.1

type LabelTableMeta struct {
	Name        string `json:"name"`
	Description string `json:"description,omitempty"`
	HasRowsData bool   `json:"has_rows_data"`
}

LabelTableMeta is the manifest projection of a registered string- valued label table. HasRowsData distinguishes the static Rows-backed table from the function-driven Lookup table without exposing the embedder's data.

type Level

type Level string

Level grades how much statistical background an operator assumes, so guidance can prefer simpler tools first.

const (
	// LevelBasic needs no statistics background (totals, counts, means).
	LevelBasic Level = "basic"
	// LevelIntermediate assumes familiarity with common tests and
	// their assumptions.
	LevelIntermediate Level = "intermediate"
	// LevelAdvanced assumes modelling background (regressions,
	// multivariate methods).
	LevelAdvanced Level = "advanced"
)

The levels a Purpose may declare.

type LimitFinding

type LimitFinding struct {
	Limit      string     `json:"limit"`
	Configured int64      `json:"configured"`
	Estimated  int64      `json:"estimated"`
	Grade      LimitGrade `json:"grade"`
}

LimitFinding is one instance resource limit a request's predicted figure exceeds. Limit is the snake_case limit name (the `limit` detail of PULSE_LIMIT_EXCEEDED), Configured the instance's effective value and Estimated predict's figure in the same unit.

type LimitGrade

type LimitGrade string

LimitGrade is how sure a LimitFinding is that the run would breach the limit.

const (
	// LimitGradeCertain: the estimate is the exact figure the runtime
	// sees; the request is refused (Valid false, PULSE_LIMIT_EXCEEDED).
	LimitGradeCertain LimitGrade = "certain"
	// LimitGradePossible: the estimate is an upper bound; the request
	// may still run.
	LimitGradePossible LimitGrade = "possible"
)

type LimitMeta

type LimitMeta struct {
	// Name is the snake_case limit key — the spelling of the feature
	// profile `limits` section, `pulse mcp --limit` and every
	// PULSE_LIMIT_* error's `limit` detail.
	Name string `json:"name"`
	// Value is the effective limit in Unit; -1 means no limit.
	Value int64 `json:"value"`
	// Default is the built-in default in Unit; -1 means no limit.
	Default int64 `json:"default"`
	// Unit is "nanoseconds" (request_timeout), "bytes"
	// (max_estimated_memory) or "count".
	Unit string `json:"unit"`
}

LimitMeta is one effective resource limit as the manifest lists it.

type LookupTableMeta added in v0.7.0

type LookupTableMeta struct {
	Name        string `json:"name"`
	Description string `json:"description,omitempty"`
	HasRowsData bool   `json:"has_rows_data"`
}

LookupTableMeta is the manifest projection of a registered lookup table. HasRowsData distinguishes the static Rows-backed table from the function-driven Lookup table without exposing the embedder's data.

type MCPTool added in v0.5.0

type MCPTool struct {
	// Name is the tool identifier (e.g. "pulse_predict").
	Name string `json:"name"`

	// Description mirrors the description string registered with the
	// MCP server.
	Description string `json:"description"`
}

MCPTool describes a single registered MCP tool. One entry per gosdk.RegisteredTools() value.

type Manifest

type Manifest struct {
	FormatVersion string `json:"format_version"`
	// PulseVersion is the build version of the binary or library that
	// produced the manifest (buildinfo.Version: ldflags, then module
	// version, then "devel[+rev]"). It identifies the build, not the
	// payload contract — that is the envelope format_version.
	PulseVersion string `json:"pulse_version"`
	// TZDataVersion is the IANA tz database release embedded in the
	// build (e.g. "2026c"). Every zone name Pulse accepts and every
	// offset it applies comes from that copy alone — never the host's
	// zoneinfo — so two builds with the same value resolve every zone
	// identically. A build fact like PulseVersion: not part of
	// feature_set_digest.
	TZDataVersion string `json:"tzdata_version"`
	// FeatureSetDigest identifies the feature set the manifest
	// describes: "fs1:" + sha256hex over the instance's sorted enabled
	// feature names and its effective behaviour switches. Two manifests
	// with the same pulse_version and digest describe the same surface,
	// so, with LimitsDigest, it keys caches of self-description output.
	// Every manifest carries one; a profile-free instance (and the CLI)
	// carries the full-registry digest.
	FeatureSetDigest string `json:"feature_set_digest"`
	// LimitsDigest identifies the instance's effective resource limits
	// (Limits): "lim1:" + sha256hex over every limit's name and
	// effective value. It moves iff an effective limit changes and is
	// independent of FeatureSetDigest, so a manifest cache keys on
	// (pulse_version, feature_set_digest, limits_digest).
	LimitsDigest string    `json:"limits_digest"`
	Commands     []Command `json:"commands"`

	// Operations enumerates library-only entry points that do not back a
	// CLI leaf (today: filter_to_file, watch, process_stream). Each entry
	// carries the same CommandAnnotations as a CLI leaf so consumers can
	// reason uniformly about caching / streaming. The slice is sorted by
	// Name for determinism.
	Operations  []Command        `json:"operations"`
	Components  Components       `json:"components"`
	Tests       []TestMeta       `json:"tests"`
	PostTests   []TestMeta       `json:"post_tests"`
	Regressions []RegressionMeta `json:"regressions"`
	// Matrices lists the registered MAT_* matrix operators (empty when
	// the instance hides capability:matrices).
	Matrices           []MatrixMeta       `json:"matrices"`
	SynthDistributions []DistributionMeta `json:"synth_distributions"`
	// ErrorCodesCount is the total number of registered error codes.
	ErrorCodesCount int `json:"error_codes_count"`
	// ErrorDomains is the alphabetized list of distinct domain prefixes
	// (e.g. "CLI", "DATA", "ENCODING", "PROCESSING", "PULSE",
	// "SERVICE"). One entry per domain, six entries in v1.
	ErrorDomains []string `json:"error_domains"`
	// ErrorCodes is the alphabetized list of code identifiers.
	// Per-code Message + Fixup prose lives behind the
	// `pulse_errors_lookup` MCP tool / `pulse errors lookup CODE` CLI
	// leaf — depth-on-demand, not common-path.
	ErrorCodes        []string          `json:"error_codes"`
	MCPTools          []MCPTool         `json:"mcp_tools"`
	CohortTypes       []CohortFieldType `json:"cohort_types"`
	Skills            []SkillMeta       `json:"skills"`
	ExamplesCount     int               `json:"examples_count"`
	ExampleCategories []string          `json:"example_categories"`
	ExampleTags       []string          `json:"example_tags"`
	// Extensions enumerates embedder-registered operators + expression
	// state. Built-in operators continue to live in Components; this
	// block is the additive layer registered via
	// pulse.Options.Extensions. Empty slices on every field for a
	// host with no extensions.
	Extensions ExtensionsManifest `json:"extensions"`

	// Facet is the rich-facet endpoint capability descriptor. One
	// entry today (facet_schema); future variants land under a slice
	// when added.
	// Omitted (nil) when the instance does not offer capability:facet.
	Facet *FacetCapability `json:"facet,omitempty"`

	// ProcessChain is the source-rooted linear chain endpoint
	// capability descriptor (one entry today: process_chain).
	// Carries the mergeable-operator allowlist and rejection rules
	// so LLM clients can route between chain and per-stage fallback.
	// Omitted (nil) when the instance does not offer capability:process_chain.
	ProcessChain *ProcessChainCapability `json:"process_chain,omitempty"`

	// Join is the pushdown hash-join capability descriptor (one
	// entry today: hash_join). Carries the kind allowlist, spill
	// envelope, and v1 limitations.
	// Omitted (nil) when the instance does not offer capability:joins.
	Join *JoinCapability `json:"join,omitempty"`

	// Crosstab is the cross-tabulation endpoint capability
	// descriptor (Request.Crosstab). Carries the normalize / shape
	// allowlists plus the per-aggregator margin-reducibility
	// classification so LLM clients can decide which cell aggregator
	// will recompute its margin and which is summable.
	// Omitted (nil) when the instance does not offer capability:crosstab.
	Crosstab *CrosstabCapability `json:"crosstab,omitempty"`

	// Export is the cross-format export envelope. Carries one
	// ExportFormatCapability entry per format the export dispatcher
	// supports, declaring the per-format overlay-embedding shape
	// (sidecar / sheets / trailing_block / warn_and_skip) so LLM
	// planners can route Response.Overlays through ExportJob without
	// inspecting the io/ packages.
	// Omitted (nil) when the instance does not offer capability:export. Formats lists only the I/O formats the instance offers.
	Export *ExportCapability `json:"export,omitempty"`

	// Import is the cross-format import envelope — the read-side peer
	// of Export. Carries one ImportFormatCapability per format
	// io.NewReader factory accepts, declaring the file extensions that
	// resolve to it, whether its .pulse schema comes from the source's
	// own dictionary ("authoritative") or from the shared inference
	// pass ("inferred"), and whether the same format can also be
	// written. Read and write surfaces are NOT symmetric — SPSS is
	// import-only — so a planner must consult both blocks rather than
	// assuming one implies the other.
	// Omitted (nil) when the instance does not offer capability:import. Formats lists only the I/O formats the instance offers.
	Import *ImportCapability `json:"import,omitempty"`

	// Matrix is the matrix-slot capability descriptor
	// (Request.Vectors + Request.Matrices): encodings, matrix kinds,
	// missing modes, the merge block size and the v1 limitations.
	// Omitted (nil) when the instance does not offer
	// capability:matrices.
	Matrix *MatrixCapability `json:"matrix,omitempty"`

	// Overlays enumerates the registered overlay catalog — one
	// OverlayCapability per types.AllOverlayKinds() entry. Each entry
	// declares the supported OverlayShape × OverlayScope × OverlayRef
	// kinds the kind accepts plus its Buffered flag (the inverse of
	// types.OverlayStreamable). LLM clients route between overlay
	// catalog lookup and per-spec validation without crawling the
	// type system. Sorted alphabetically by Kind via
	// OverlayCapabilities() so the golden manifest stays stable.
	Overlays []OverlayCapability `json:"overlays"`

	// ComponentsSchemas projects the per-operator components contract
	// across categories as a name-keyed map. Embedders rely on this
	// block to plan ResponseComponents.Components[] consumption without
	// iterating Components.* per category.
	ComponentsSchemas ComponentsSchemasBlock `json:"components_schemas"`

	// Intents lists the intent-taxonomy IDs, sorted — the closed set of
	// question kinds every entry's intents list draws from. IDs only:
	// the full Intent records (labels, phrasings, data shapes) are
	// fetched on demand. Static: the same list under every feature
	// profile.
	Intents []string `json:"intents"`

	// ReturnPresets lists the named `return` presets (full / standard /
	// minimal, in that order), each with the response paths it selects
	// EXPANDED against this instance: a path a hidden feature owns is
	// absent. Always present: `return` is a plain request slot, not a
	// feature.
	ReturnPresets []ReturnPresetMeta `json:"return_presets"`

	// Limits lists the instance's EFFECTIVE resource limits, one entry
	// per limit in declaration order (request_timeout, max_groups,
	// max_crosstab_cells, max_estimated_memory, max_matrix_dim,
	// max_compose_slots, max_chain_stages, max_join_build_rows). Always
	// present: an instance without Options.Limits, a profile `limits`
	// section or `pulse mcp --limit` reports the built-in defaults.
	// Limits are behaviour, not features — they never enter
	// feature_set_digest; LimitsDigest identifies them.
	Limits []LimitMeta `json:"limits"`
}

Manifest is the root self-description of the Pulse system. One bootstrap call returns every fact an LLM needs to author a valid Pulse request: CLI command list, per-operator capabilities, per-test metadata (tier-1 and tier-2 as peer slices), synth distribution catalog, MCP tool list, cohort field-type catalog with operator cross-references, and embedded skill index. Error coverage is name-only — fetch per-code prose via the `pulse_errors_lookup` MCP tool or `pulse errors lookup CODE` CLI leaf on demand to keep the bootstrap payload lean.

The payload is deterministic and free of cohort data. Clients cache it for a session.

type MatrixCapability

type MatrixCapability struct {
	// Name is the canonical identifier ("matrices").
	Name string `json:"name"`
	// Encodings lists the MatrixSpec.Encoding values.
	Encodings []string `json:"encodings"`
	// DefaultEncoding is the encoding an empty MatrixSpec.Encoding
	// takes.
	DefaultEncoding string `json:"default_encoding"`
	// Kinds lists the MatrixValues.Kind values.
	Kinds []string `json:"kinds"`
	// MissingModes lists the missing-data modes.
	MissingModes []string `json:"missing_modes"`
	// MergeBlockSize is the record count of one merge block: rows fold
	// into per-block partials combined by one fixed tree.
	MergeBlockSize int `json:"merge_block_size"`
	// Limitations lists the v1 envelope notes for LLM clients.
	Limitations []string `json:"limitations"`
}

MatrixCapability is the matrix-slot capability descriptor (Manifest.Matrix): the Values encodings and matrix kinds a MatrixResult can carry, the missing-data modes, the blocked-merge block size and the v1 limitations. Omitted when the instance hides capability:matrices.

type MatrixMeta

type MatrixMeta struct {
	// Name is the operator identifier (MAT_COVARIANCE).
	Name string `json:"name"`

	// Description is a one-sentence prose summary for LLM-side
	// operator selection.
	Description string `json:"description"`

	// AcceptsTypes lists the schema field types valid as members
	// (packed_bool only under the vector's coerce "binary").
	AcceptsTypes []string `json:"accepts_types"`

	// Params lists the operator's params object keys.
	Params []Param `json:"params"`

	// OutputKeys names the MatrixResult parts the operator fills.
	OutputKeys MatrixOutputKeys `json:"output_keys"`

	// Streamable mirrors types.MatrixType.Streamable(): the result is
	// emitted at finalize (terminal flush on the streaming path).
	Streamable bool `json:"streamable"`

	// Mergeable mirrors types.MatrixType.Mergeable(): the operator's
	// state merges across DecodeWorkers / ShardWorkers partitions
	// through the blocked merge tree, bit-identical to serial.
	Mergeable bool `json:"mergeable"`

	// ComponentSchema declares the operator's
	// Response.Components.Matrices entry: the matrix floor
	// {n, n_null, n_listwise_dropped}, the optional pairwise and
	// weighted floor keys, then the operator's own keys. Mirrored under
	// Manifest.ComponentsSchemas.Matrices.
	ComponentSchema ComponentSchema `json:"component_schema"`

	// Intents lists the intent-taxonomy IDs (Manifest.Intents) the
	// operator answers, sorted. Omitted when it declares no purpose.
	Intents []string `json:"intents,omitempty"`

	// WeightKinds lists the weight kinds the operator computes a
	// weighted matrix under; omitted when it honours no weight and on an
	// instance that hides capability:weighting. See
	// Operator.WeightKinds.
	WeightKinds []types.WeightKind `json:"weight_kinds,omitempty"`
}

MatrixMeta describes a registered MAT_* matrix operator in the manifest (Manifest.Matrices). A matrix operator reads a virtual vector (Request.Vectors) or an inline member list and emits a MatrixResult; OutputKeys names what that result carries.

type MatrixOutputKeys

type MatrixOutputKeys struct {
	Primary   string   `json:"primary"`
	Auxiliary []string `json:"auxiliary,omitempty"`
	Vectors   []string `json:"vectors,omitempty"`
	Scalars   []string `json:"scalars,omitempty"`
}

MatrixOutputKeys names the parts of a MatrixResult a matrix operator fills: the meaning of Primary, the Auxiliary matrix names, the Vectors summary keys and the Scalars keys.

type MatrixPredict

type MatrixPredict struct {
	// Name is the result name (MatrixSpec.EffectiveName).
	Name string `json:"name"`
	// Type is the matrix operator.
	Type types.MatrixType `json:"type"`
	// Shape is [p, p]: rows × columns of every matrix in the result.
	Shape [2]int `json:"shape"`
	// AxisKeys are the member field names in axis order — the result's
	// row_keys and column_keys.
	AxisKeys []string `json:"axis_keys"`
	// Labels are the members' display labels in axis order when the
	// vector supplies them (the result's labels); omitted otherwise.
	Labels []string `json:"labels,omitempty"`
	// Missing is the missing-data mode, "listwise" or "pairwise".
	Missing string `json:"missing"`
	// Encoding is the effective Values layout.
	Encoding types.MatrixEncoding `json:"encoding"`
	// AccumulatorBytes estimates the payload bytes of one co-moment
	// state over the members (32 + 8·(p + p(p+1)/2) listwise,
	// 32 + 56·p(p+1)/2 pairwise). The run holds one per populated merge
	// block (Manifest.Matrix.merge_block_size rows) until finalize, so
	// its matrix state is this times its block count (grouped: times
	// its buckets too — EstimatedBytes).
	AccumulatorBytes int64 `json:"accumulator_bytes"`
	// BucketBasis says how EstimatedBuckets was derived from the
	// request and the schema: "ungrouped" (1), "dictionary" (the
	// Groups[0] field's dictionary size), "boolean" (2), "include"
	// (Group.Include's distinct reachable keys), "quantile_bins"
	// (GROUP_QUANTILE's bin count) or "unknown" (the keys depend on the
	// data — ranges, dates, numeric categories — and the three estimate
	// fields are omitted).
	BucketBasis string `json:"bucket_basis"`
	// EstimatedBuckets is the upper bound on the results this spec
	// emits — one per non-empty bucket of Groups[0]; a bucket no row
	// reaches is not emitted, so a run returns at most this many.
	EstimatedBuckets *int64 `json:"estimated_buckets,omitempty"`
	// EstimatedCells is EstimatedBuckets × p² — the primary cells the
	// spec's results carry in full encoding.
	EstimatedCells *int64 `json:"estimated_cells,omitempty"`
	// EstimatedBytes is the run's co-moment state: merge blocks ×
	// EstimatedBuckets × AccumulatorBytes, the blocks being
	// ceil(records / merge_block_size) — per shard for an archive, whose
	// block numbering restarts per shard. An upper bound (a bucket holds
	// one state per block its rows touch). Omitted with the bucket count
	// unknown. It is the matrix term of the MaxEstimatedMemory estimate.
	EstimatedBytes *int64 `json:"estimated_bytes,omitempty"`
	// Streamable reports whether the operator folds row by row (its
	// result is emitted at finalize — terminal flush when streamed).
	// Request-level routing (groupers, two-pass attributes) is
	// PredictResult.Streamable.
	Streamable bool `json:"streamable"`
	// PairwisePSDRisk reports whether the result can come back not
	// positive semidefinite — pairwise MAT_COVARIANCE at p ≥ 2,
	// pairwise MAT_CORRELATION at p ≥ 3 — the only shapes the runtime
	// checks for PULSE_MATRIX_NOT_PSD. False guarantees the warning
	// never fires; true says it can, depending on the data.
	PairwisePSDRisk bool `json:"pairwise_psd_risk"`
}

MatrixPredict is the per-spec predict surface for one entry of req.Matrices.

type Ontology

type Ontology struct {
	Nodes []OntologyNode `json:"nodes"`
	Edges []OntologyEdge `json:"edges"`
}

Ontology is a node / edge graph. Nodes are sorted by ID; edges by From, then Kind, then To; no duplicates.

type OntologyEdge

type OntologyEdge struct {
	// From is the source node ID.
	From string `json:"from"`
	// To is the target node ID.
	To string `json:"to"`
	// Kind classifies the edge.
	Kind OntologyEdgeKind `json:"kind"`
}

OntologyEdge is one directed, typed ontology edge between two node IDs.

type OntologyEdgeKind

type OntologyEdgeKind string

OntologyEdgeKind classifies a directed ontology edge.

const (
	// OntologyEdgeServesIntent: operator | example → intent.
	OntologyEdgeServesIntent OntologyEdgeKind = "serves_intent"
	// OntologyEdgeDocumentedBy: operator | mcp_tool | intent |
	// glossary_term → the skill that documents it.
	OntologyEdgeDocumentedBy OntologyEdgeKind = "documented_by"
	// OntologyEdgeExemplifiedBy: operator | skill → example.
	OntologyEdgeExemplifiedBy OntologyEdgeKind = "exemplified_by"
	// OntologyEdgeNotFor: operator → the operator or capability to use
	// instead (Purpose.NotFor).
	OntologyEdgeNotFor OntologyEdgeKind = "not_for"
	// OntologyEdgeUsesTerm: operator → glossary_term.
	OntologyEdgeUsesTerm OntologyEdgeKind = "uses_term"
	// OntologyEdgeRoutesTo: skill → skill, glossary_term → glossary_term
	// (a navigational "see also"), example → the operator its
	// description names without using it (a contrast).
	OntologyEdgeRoutesTo OntologyEdgeKind = "routes_to"
	// OntologyEdgeRequiresCapability: operator | capability | mcp_tool |
	// skill | example → the operator or capability it needs.
	OntologyEdgeRequiresCapability OntologyEdgeKind = "requires_capability"
)

The ontology edge kinds. Each comment gives the From → To node kinds.

type OntologyNode

type OntologyNode struct {
	// ID is the stable node identifier (see the package spelling rule).
	ID string `json:"id"`
	// Kind classifies the node.
	Kind OntologyNodeKind `json:"kind"`
	// Name is the node's name in its own registry.
	Name string `json:"name"`
}

OntologyNode is one ontology node.

type OntologyNodeKind

type OntologyNodeKind string

OntologyNodeKind classifies an ontology node.

const (
	// OntologyNodeIntent is an intent-taxonomy entry (Name = intent ID).
	OntologyNodeIntent OntologyNodeKind = "intent"
	// OntologyNodeOperator is a registered operator, overlay kind,
	// regression, regression spec modifier or synth distribution
	// (Name = its registered name).
	OntologyNodeOperator OntologyNodeKind = "operator"
	// OntologyNodeSkill is a skill-pack entry, virtual skills included
	// (Name = skill stem).
	OntologyNodeSkill OntologyNodeKind = "skill"
	// OntologyNodeExample is a runnable request example (Name = its
	// _meta.name).
	OntologyNodeExample OntologyNodeKind = "example"
	// OntologyNodeGlossaryTerm is a glossary term (Name = term ID).
	OntologyNodeGlossaryTerm OntologyNodeKind = "glossary_term"
	// OntologyNodeCapability is a non-operator feature (Name = its
	// feature-profile spelling, e.g. "capability:crosstab").
	OntologyNodeCapability OntologyNodeKind = "capability"
	// OntologyNodeMCPTool is an MCP tool (Name = tool name).
	OntologyNodeMCPTool OntologyNodeKind = "mcp_tool"
	// OntologyNodeTable is an instance-registered named table (Name =
	// `<label|range|lookup>/<table name>`, e.g. "label/region").
	OntologyNodeTable OntologyNodeKind = "table"
)

The ontology node kinds.

type Operator added in v0.5.0

type Operator struct {
	// Name is the operator identifier (e.g. "AGG_PERCENTILE").
	Name string `json:"name"`

	// Category is the family this operator belongs to. One of:
	// "aggregator", "attribute", "filterer", "grouper", "window",
	// "feature".
	Category string `json:"category"`

	// Description is a one-sentence prose summary for LLM-side selection.
	Description string `json:"description"`

	// Params lists every parameter the operator reads from its Params
	// blob in a request. Required and optional are both listed.
	Params []Param `json:"params"`

	// AcceptsTypes lists the cohort field types this operator can be
	// applied to. Values are field type name strings (e.g. "f64",
	// "categorical_u16", "date"). Empty means "no field input".
	AcceptsTypes []string `json:"accepts_types"`

	// IgnoresField marks an operator that never reads the request slot's
	// own Field — it takes its inputs from Params instead (AGG_RATIO
	// reads Params.numerator_field / denominator_field).
	//
	// Without it, AcceptsTypes is ambiguous in the one way that matters:
	// a client cannot tell "every type is welcome here" from "this slot
	// is not read at all", and the full AcceptsTypes list reads as a
	// type claim about a slot the operator discards. When IgnoresField is
	// set, AcceptsTypes says only that no type is REFUSED (the request
	// wire form still requires a field name), and the operator's real
	// type constraints live on its Params entries, which carry
	// Type=="field" plus a FieldFilter.
	//
	// The claim is held two ways: descriptor's
	// TestCapabilities_IgnoresFieldDeclarationsAreCoherent checks the
	// declaration shape, and processing's
	// TestAggregators_IgnoredFieldSlotIsReallyIgnored runs the operator
	// with different Field values and requires an identical result.
	IgnoresField bool `json:"ignores_field,omitempty"`

	// EmitsType is the field type produced for single-output operators.
	// Empty when the operator's emit type is conditional on input or
	// when it does not emit a typed column (e.g. an aggregator emits a
	// scalar).
	//
	// No execution path reads this: attribute outputs are float64 end to
	// end, window outputs land in a map[string]any and feature outputs in
	// feature.Output.Values. It is a claim MCP clients size destination
	// columns from, and the only thing that can contradict it is
	// processing.TestManifestEmitsTypeHoldsAtRuntime, which runs every
	// operator that declares one and checks the values it really emits
	// against the declared domain. A new declaration must arrive with a
	// probe there or that gate fails.
	EmitsType string `json:"emits_type,omitempty"`

	// EmitsTypeNote provides context when EmitsType is empty or
	// conditional (e.g. "matches input field type", "scalar float64").
	EmitsTypeNote string `json:"emits_type_note,omitempty"`

	// Streamable mirrors types.X.Streamable() for the operator's type.
	// Source of truth for the runtime gate.
	Streamable bool `json:"streamable"`

	// StreamableHint suggests the closest streaming-capable alternative
	// when Streamable is false (e.g. AGG_MEDIAN -> "Use AGG_AVERAGE for
	// streaming, or accept the buffered path."). Empty when Streamable
	// is true or no near-equivalent exists.
	StreamableHint string `json:"streamable_hint,omitempty"`

	// ComponentSchema declares the per-operator components contract
	// surfaced through the runtime ResponseComponents shell and the
	// MetaAggregator / MetaGrouper / MetaFilterer sibling interfaces.
	// The universal floor of {"n", "n_null"} is declared on every
	// aggregator entry so manifest consumers see a self-contained
	// schema without consulting orchestrator code; the operator-
	// specific keys are appended in emission order. Mergeability
	// classifies how the components map folds across streaming chunks.
	ComponentSchema ComponentSchema `json:"component_schema"`

	// Zone declares the operator's time-zone participation. "capable"
	// means the slot accepts a per-slot `tz` and resolves a zone through
	// the slot → request → Options.DefaultTimeZone → UTC precedence.
	// Empty (omitted) means the operator is not zone-capable and an
	// explicit `tz` on its slot is refused. Extension operators are
	// never zone-capable.
	Zone string `json:"zone,omitempty"`

	// WeightAware is true for an operator that computes a weighted
	// figure when a row weight (Request.Weight, a per-slot `weight`, or
	// Options.DefaultWeight) is in force on its slot, and omitted
	// otherwise: such an operator is skipped under an instance default
	// weight and refused under an explicit one (or, for an inferential
	// one, under any weight). Predict's per-slot `weights` status says
	// which happened.
	WeightAware bool `json:"weight_aware,omitempty"`

	// WeightKinds lists the weight kinds the operator computes a
	// weighted figure under: ["frequency","probability"] for an
	// operator weighted under both, ["frequency"] for a frequency-only
	// one (a probability weight in force on its slot is
	// PULSE_WEIGHT_UNSUPPORTED naming the kind). Omitted for an
	// operator that honours no weight, and on an instance that hides
	// capability:weighting.
	WeightKinds []types.WeightKind `json:"weight_kinds,omitempty"`

	// Intents lists the intent-taxonomy IDs (Manifest.Intents) the
	// operator answers, sorted. Omitted when the operator declares no
	// purpose.
	Intents []string `json:"intents,omitempty"`
}

Operator describes a single registered processing component (aggregator, attribute, filterer, grouper, window operator, or feature operator). The manifest exposes one Operator entry per registered component. LLM clients use this metadata at session start to author valid requests without further discovery round-trips.

type OperatorMeta added in v0.7.0

type OperatorMeta struct {
	Name        string `json:"name"`
	Namespace   string `json:"namespace"`
	Description string `json:"description,omitempty"`
	Streamable  bool   `json:"streamable"`
	// FansOut is the grouper-only projection of
	// pulse.GrouperRegistration.FansOut — the embedder-side sibling of
	// types.GroupType.FansOut(), which knows built-in constants only.
	// True means one record can land in more than one bucket, so the
	// bucket counts SUM to more than the record total. Omitted (and
	// meaningless) for every other operator category; absent reads as
	// false, which is also the registration default.
	FansOut bool `json:"fans_out,omitempty"`
	// Mergeable is the aggregator / grouper projection of
	// pulse.AggregatorRegistration.Mergeable and
	// pulse.GrouperRegistration.Mergeable — the embedder-side sibling
	// of types.AggregationType.Mergeable() / types.GroupType.Mergeable().
	// True means partial states fold (extend.MergeableAggregator.Merge,
	// extend.MergeableGrouper.MergeState), so the parallel reducers
	// (ShardWorkers / DecodeWorkers) and ProcessChain accept the
	// operator. Omitted for every other category; absent reads as
	// false, which is also the registration default.
	Mergeable bool `json:"mergeable,omitempty"`
	// MarginReducibility is the aggregator-only projection of
	// pulse.AggregatorRegistration.MarginReducibility — the
	// embedder-side sibling of types.AggregationType.MarginReducibility()
	// ("summable", "mean_reducible", "independent" or "recompute").
	// A non-recompute class admits the operator as a fused crosstab
	// cell. Omitted when undeclared, which reads as "recompute": the
	// crosstab cell runs buffered.
	MarginReducibility string `json:"margin_reducibility,omitempty"`
	// WeightAware is the aggregator / attribute / test projection of
	// the registration's WeightAware declaration — the extension half
	// of the built-in Operator.WeightAware. True means the operator
	// reads the resolved row weight (extend.Record.Weight) and computes
	// a weighted figure; omitted otherwise.
	WeightAware bool                `json:"weight_aware,omitempty"`
	Accepts     []string            `json:"accepts,omitempty"`
	Emits       string              `json:"emits,omitempty"`
	Mode        string              `json:"mode,omitempty"`
	Tier        string              `json:"tier,omitempty"`
	Params      []OperatorParamMeta `json:"params,omitempty"`
	// Intents lists the intent-taxonomy IDs (Manifest.Intents) the
	// registration's purpose declares, sorted. Omitted when the
	// registration declares none.
	Intents []string `json:"intents,omitempty"`
}

OperatorMeta is the manifest projection for an embedder-registered operator. The shape mirrors descriptor.Operator but adds Namespace (parsed from the registered name) and Mode (attribute-only) so reviewers can group operators by source without re-parsing.

type OperatorParamMeta added in v0.7.0

type OperatorParamMeta struct {
	Name        string `json:"name"`
	Description string `json:"description,omitempty"`
	JSONType    string `json:"json_type"`
	Required    bool   `json:"required,omitempty"`
	Default     any    `json:"default,omitempty"`
}

OperatorParamMeta is the manifest-friendly mirror of pulse.ParamMeta.

type OverlayAppliedDescriptor added in v0.19.0

type OverlayAppliedDescriptor struct {
	// Name echoes the renderer-facing label — either the spec's own
	// Name or a synthesised default (Kind + Scope + Ref) when empty.
	Name string `json:"name"`

	// Kind is the on-wire SCREAMING_SNAKE OverlayKind value.
	Kind types.OverlayKind `json:"kind"`

	// Scope echoes the spec's scope.
	Scope types.OverlayScope `json:"scope"`

	// Shape echoes the resolved OverlayShape the layer will carry —
	// derived from (Kind, Scope) via the capability catalog
	// (descriptor.overlayCapabilityFor). For single-shape kinds the
	// shape is unambiguous; for dual-shape kinds (OVERLAY_INDEX_VS_REF
	// / OVERLAY_DELTA_VS_REF / OVERLAY_PANEL_INDEX_VS_REF) the resolver
	// picks the shape matching the spec's scope (CELL ⇒ matrix, GROUP
	// ⇒ series); for whole-chain kinds (OVERLAY_INDEX_VS_STAGE /
	// OVERLAY_DELTA_VS_STAGE) the catalog declares every shape the
	// kind may emit (scalar / series / matrix) and the descriptor
	// leaves Shape empty — the realised shape depends on the target
	// stage's host shape, which Predict cannot determine without
	// inspecting the chain. Empty when the kind is unknown (the
	// validator surfaces PULSE_OVERLAY_KIND_UNKNOWN for the same
	// condition) or when no single shape can be resolved from the
	// spec at predict time.
	Shape types.OverlayShape `json:"shape,omitempty"`

	// Ref echoes the resolved OverlayRef discriminated union variant
	// as a renderer-friendly string. Format: "<family>:<discriminator>"
	// when the family carries an on-wire discriminator, "<family>" for
	// marker-only families, and "" for implicit-margin kinds (every
	// CHISQ_* / FISHER_EXACT_CELL / INDEX_VS_TOTAL / FORMULA) and the
	// COMPOSE-only catalog (Reference / Targets resolve via slot
	// labels, not via the OverlayRef discriminated union). Concrete
	// shapes:
	//
	//   - "margin:row" / "margin:column" / "margin:grand" — Ref.Margin
	//   - "sibling:<field>=<value>"                        — Ref.Sibling
	//   - "baseline_index:<position>"                      — Ref.BaselineIndex
	//   - "prior:<lag>"                                    — Ref.Prior
	//   - "rolling_mean"                                   — Ref.RollingMean (marker)
	//   - "yoy"                                            — Ref.YoY (marker)
	//   - "population:<cohort>"                            — Ref.Population
	//   - "stage:index:<index>" / "stage:name:<name>"      — Ref.Stage
	//   - "slot:<label>"                                   — Ref.Slot
	Ref string `json:"ref,omitempty"`

	// Streamable mirrors types.OverlayStreamable(kind). False for
	// the buffered crosstab catalog; streamable kinds surface the
	// per-kind decision uniformly.
	Streamable bool `json:"streamable"`
}

OverlayAppliedDescriptor describes one accepted overlay spec the predict path observed. Mirrors DefaultApplied / Suggestion in shape: JSON-serialisable, omitempty-free for slice-friendly emit, populated from the catalog entry rather than the raw spec so the caller cannot confuse the surface with the raw OverlaySpec it submitted.

Streamable echoes types.OverlayStreamable(kind). Note that a kind can be streamable in isolation but force buffered when its host (today: crosstab) is itself buffered — predict reports the kind's own streamability, not the composed host-overlay decision. Callers that need the composed decision should consult PredictResult.Streamable + StreamableReasons.

type OverlayCapability added in v0.19.0

type OverlayCapability struct {
	// Kind is the on-wire SCREAMING_SNAKE OverlayKind value.
	Kind types.OverlayKind `json:"kind"`

	// Shapes lists every OverlayShape this kind may emit. Sorted
	// alphabetically for golden stability.
	Shapes []types.OverlayShape `json:"shapes"`

	// Scopes lists every OverlayScope this kind supports. Sorted
	// alphabetically for golden stability.
	Scopes []types.OverlayScope `json:"scopes"`

	// RefKinds lists the OverlayRef union pointer-field names this
	// kind consumes (e.g. "Margin"). Sorted alphabetically for golden
	// stability. These are Go field-name strings — the on-wire
	// discriminator (MarginAxis, sibling field, etc.) lives one level
	// deeper inside the chosen pointer.
	RefKinds []string `json:"ref_kinds"`

	// Buffered reports whether the orchestrator must materialise
	// records before evaluating this kind. Derived from
	// types.OverlayStreamable(kind) — Buffered is !streamable.
	Buffered bool `json:"buffered"`

	// Fields lists the OverlaySpec slot names beyond Kind / Scope /
	// Ref / Params that the kind's runtime honours. Sorted
	// alphabetically for golden stability. "level" and "within" are
	// honoured by the share / index / delta / zscore family (prefix-axis
	// denominator dispatch) while the χ² / Fisher inferential family
	// leaves the list empty (Level / Within must stay zero; non-zero
	// values fire PULSE_OVERLAY_LEVEL_OUT_OF_RANGE). SHARE_OF_TOTAL
	// declares the fields for renderer-facing parity with the rest of
	// the share family even though the grand-axis denominator makes the
	// slots inert at runtime — the predict gate still accepts in-range
	// values without behavior change.
	Fields []string `json:"fields,omitempty"`

	// Description is a short human-readable summary of what the kind
	// computes. Mirrors the prose in the matching skill.
	Description string `json:"description"`

	// Zone declares the kind's time-zone participation. "following"
	// means the kind carries no zone of its own and inherits the
	// resolved zone of its host grouper (OVERLAY_YOY follows its host
	// GROUP_DATE). Empty (omitted) means the kind is zone-agnostic.
	Zone string `json:"zone,omitempty"`

	// Intents lists the intent-taxonomy IDs (Manifest.Intents) the kind
	// answers, sorted. Omitted when the kind declares no purpose.
	Intents []string `json:"intents,omitempty"`

	// Inferential is true for a hypothesis-test kind — its numbers are
	// test statistics and p-values, read with the shared p-value rules
	// and subject to multiple-comparison caveats — and omitted for a
	// descriptive kind (shares, indices, deltas, z-scores). Declared per
	// kind, never inferred from the name.
	Inferential bool `json:"inferential,omitempty"`

	// WeightKinds lists the weight kinds the overlay kind honours a
	// weight under (["frequency","probability"] or ["frequency"]);
	// omitted for a descriptive kind (it reads the already-weighted host
	// payload), a refused kind, and on an instance that hides
	// capability:weighting. See Operator.WeightKinds.
	WeightKinds []types.WeightKind `json:"weight_kinds,omitempty"`
}

OverlayCapability is the per-kind manifest entry describing one registered overlay catalog entry. The manifest carries one OverlayCapability per types.AllOverlayKinds() entry under Manifest.Overlays.

Field shape mirrors the kind-catalog-v1 PRD §I-FR-I2 contract: kind × supported shapes × supported scopes × valid ref kinds × buffered flag × description.

type PValueCount

type PValueCount struct {
	// Total is the number of inferential p-values the request emits.
	Total int `json:"total"`
	// Uncorrected is how many of Total no correction reaches (their
	// slot's resolved method is none).
	Uncorrected int `json:"uncorrected"`
	// Basis is how the counts were derived: PValueBasisExact,
	// PValueBasisDictionary or PValueBasisLowerBound.
	Basis string `json:"basis"`
	// Threshold is MultiplicityTriggerThreshold.
	Threshold int `json:"threshold"`
}

PValueCount is PredictResult.PValues.

type Param added in v0.5.0

type Param struct {
	// Name is the JSON key inside the operator's Params blob.
	Name string `json:"name"`

	// Type is the parameter's value type. One of:
	//   "float", "int", "string", "bool", "field", "enum", "list", "object".
	// "field" means the value names a cohort field; "enum" means the
	// value is one of EnumValues; "list" means a JSON array.
	Type string `json:"type"`

	// Required is true when the parameter must be supplied. Optional
	// parameters with a default carry Required=false and Default set.
	Required bool `json:"required"`

	// Default is the operator's default value when Required is false.
	// Omitted from JSON when nil.
	Default any `json:"default,omitempty"`

	// Description is a one-sentence prose explanation.
	Description string `json:"description"`

	// EnumValues lists the allowed values when Type=="enum".
	EnumValues []string `json:"enum_values,omitempty"`

	// FieldFilter constrains acceptable field types when Type=="field".
	// One of: "numeric", "categorical", "date", "any".
	FieldFilter string `json:"field_filter,omitempty"`
}

Param describes a single parameter accepted by an operator, test, or synth distribution.

type PredictResult

type PredictResult struct {
	Valid      bool               `json:"valid"`
	Request    *types.Request     `json:"request"`
	SchemaInfo *PredictSchemaInfo `json:"schema_info,omitempty"`
	// RecordCount reports the cumulative record total across all shards
	// when the cohort is a shard archive, and zero for single-file
	// cohorts (predict reads only headers/schemas, so single-file
	// counts are not computed by this no-execute path). The count is
	// derived by peeking each shard's own header.
	RecordCount int64 `json:"record_count"`
	// Shards mirrors InspectResult.Shards for archive-backed cohorts.
	// Empty (but non-nil) for single-file cohorts. Listed in zip
	// central-directory order.
	Shards []ShardInfo `json:"shards"`
	// Streamable reports whether ProcessStream / process --stream can
	// emit rows without buffering the entire result. False whenever the
	// request uses groups, attributes, windows, decimal fields, or any
	// non-streamable operator. Computed via per-type Streamable() methods
	// plus schema-aware checks.
	//
	// Streamability is a property of the request and the canonical
	// schema, not of the cohort shape — archive-backed cohorts inherit
	// the same streamability as a single-file cohort with the same
	// schema.
	Streamable bool `json:"streamable"`
	// StreamableReasons lists the gates that forced Streamable=false. Empty
	// when Streamable=true. Useful for users debugging why their request
	// is buffering.
	StreamableReasons []string `json:"streamable_reasons,omitempty"`
	// CrosstabFusable reports whether this instance would run the
	// request's crosstab on the fused in-decode arm (O(cells + margins)
	// memory, one decode pass) rather than the buffered arm that holds
	// every filter-passing record. Nil when the request carries no
	// crosstab spec. Computed from the same rule the engine dispatches
	// on (internal/crosstabfuse), on the defaults-resolved request, and
	// instance-aware: Options.DisableCrosstabFusion answers false with a
	// reason. Both arms produce identical output; only memory differs.
	//
	// The one fact predict cannot know is whether a keyable grouper's
	// factory accepts its params (e.g. a GROUP_DATE component the
	// runtime rejects): predict may then say true while the runtime
	// declines and the buffered path refuses the request with a coded
	// error — the request errors on either answer.
	CrosstabFusable *bool `json:"crosstab_fusable,omitempty"`
	// CrosstabFusionReasons lists, in rule order, every reason
	// CrosstabFusable is false — short stable prose. Empty when it is
	// true or nil.
	CrosstabFusionReasons []string `json:"crosstab_fusion_reasons,omitempty"`
	// Suggestions enumerates structured next-actions the caller can apply
	// to repair (or improve) the request. Suggestions fire on validation
	// issues — field-name typos, operator/type mismatches, date misuse,
	// missing required params — and on non-streamable but otherwise valid
	// requests (streamable-substitute hints). May be empty; never nil in
	// JSON output.
	Suggestions []Suggestion `json:"suggestions"`
	// DefaultsApplied lists every operator slot whose Type was inferred
	// from the named field's schema type. Predict computes this on a
	// clone of the request, so the echoed Request reflects exactly what
	// the engine would run; the DefaultsApplied list shows what would
	// have been filled in. Empty when no defaults fire; never nil in
	// JSON output.
	DefaultsApplied []DefaultApplied `json:"defaults_applied"`

	// Aggregations mirrors req.Aggregations in order, attaching the
	// per-slot ComponentSchema the engine will use at runtime (the
	// operator's `descriptor.ComponentSchema` projected from the
	// capabilities table — and via the manifest snapshot for
	// embedder-registered ops) plus the BufferedComponents flag.
	//
	// BufferedComponents is true iff the operator's
	// ComponentSchema.Mergeability is None — meaning the components
	// map cannot be reconstructed from per-chunk partials and the
	// orchestrator emits the Components block only on the terminal
	// buffered flush. The flag is a hint to streaming consumers; it
	// does NOT flip the overall Streamable axis (the data slice still
	// streams). Today the canonical None-mergeability operators are
	// AGG_MEDIAN and AGG_PERCENTILE.
	//
	// Empty when req.Aggregations is empty; never nil in JSON output.
	Aggregations []AggregationPredict `json:"aggregations"`

	// Groups mirrors req.Groups in order, attaching the per-slot
	// ComponentSchema the engine will use at runtime (the operator's
	// `descriptor.ComponentSchema` projected from the capabilities
	// table — same projection ComponentsSchemas.Groupers uses) plus
	// the BufferedComponents flag.
	//
	// BufferedComponents is true iff the operator's
	// ComponentSchema.Mergeability is None — meaning the components
	// map cannot be reconstructed from per-chunk partials and the
	// orchestrator emits the Components block only on the terminal
	// buffered flush. The flag is a hint to streaming consumers; it
	// does NOT flip the overall Streamable axis. Of the seven
	// registered groupers today, GROUP_QUANTILE is the canonical
	// None — quantile cutoffs need the sorted full input.
	//
	// Empty when req.Groups is empty; never nil in JSON output.
	Groups []GroupPredict `json:"groups"`

	// Filterers mirrors req.Filterers in order, attaching the per-slot
	// ComponentSchema the engine will use at runtime (the operator's
	// `descriptor.ComponentSchema` projected from the capabilities
	// table — same projection ComponentsSchemas.Filterers uses) plus
	// the BufferedComponents flag.
	//
	// Filterer components in v1 are uniform across every registered
	// filterer — the orchestrator emits the universal floor
	// {n_in, n_out, n_null_input} from the filter pass's per-record
	// counters. Mergeability is Mergeable for every entry today
	// (counters fold trivially via integer addition), so
	// BufferedComponents is always false. The flag is still surfaced
	// on the predict struct for symmetry with AggregationPredict and
	// GroupPredict so future per-filter specifics (e.g. n_below /
	// n_above for FILTER_RANGE) can downgrade the slot without
	// changing the predict shape.
	//
	// Empty when req.Filterers is empty; never nil in JSON output.
	Filterers []FiltererPredict `json:"filterers"`

	// OverlaysApplied lists every overlay spec the engine accepted, in
	// req.Overlays order. Each entry echoes the catalog identity (Name,
	// Kind, Scope) plus the streamability flag harvested from
	// types.OverlayStreamable(kind) so the caller can reason about the
	// buffered/streaming routing decision without re-parsing the spec.
	// Empty when req.Overlays is empty; never nil in JSON output.
	//
	// Per kind-catalog-v1 PRD §I-FR-I3 the predict surface is
	// `OverlaysApplied + OverlaysSchemaDivergence + OverlayCost`.
	// Predict emits one descriptor per spec in matching order across
	// the full overlay catalog; new kinds appended to
	// types.AllOverlayKinds() inherit the same per-spec emission
	// without re-opening this slot. The streamability-derived
	// dispatch routes streamable kinds to overlayCostStreamable and
	// buffered kinds to overlayCostBuffered; every kind in
	// types.AllOverlayKinds() carries a multiplier, and the
	// multi-ref scaling rule lives on the COMPOSE-host equivalent at
	// ComposeValidationResult.OverlayCost since Targets / MaxPanelTargets
	// only exist on ComposeOverlaySpec.
	OverlaysApplied []OverlayAppliedDescriptor `json:"overlays_applied"`

	// OverlaysSchemaDivergence lists every (left, right) overlay-spec
	// slot pair that produced incompatible result-schema shapes. Empty
	// for the Request-only Predict surface — divergence detection lives
	// on the Compose-host validator (ValidateCompose) — but the field is
	// present so consumers can rely on the shape today. Per PRD §I-FR-I3
	// the placeholder is honoured as "this slot is reserved and shipped
	// empty".
	OverlaysSchemaDivergence []SlotPair `json:"overlays_schema_divergence"`

	// OverlayCost maps each overlay-spec Name to a coarse cost score
	// the routing layer can consult before execution. The score is a
	// rough record-count multiplier per overlay slot — streamable kinds
	// fold inside the existing streaming pass (one extra accumulator per
	// record) and carry overlayCostStreamable; buffered kinds force a
	// post-host re-traversal of the materialised payload and carry
	// overlayCostBuffered. Per kind-catalog-v1 PRD §I-FR-I3 the value is
	// intentionally coarse — callers budgeting cost across slots sum the
	// map values; renderers showing a "this overlay will buffer the
	// host" warning branch on >= overlayCostBuffered. The map key is
	// the overlay spec's Name when set, and the synthesised default
	// (Kind + Scope + Ref) when empty — matching the Name field on
	// OverlayAppliedDescriptor below. Empty when req.Overlays is empty;
	// never nil in JSON output.
	OverlayCost map[string]float64 `json:"overlay_cost"`

	// TimeZones lists the resolved zone of every zone-capable slot
	// (GROUP_DATE, GROUP_DATE_RANGES, FILTER_DATE_RANGES, ATTR_DATE_PART,
	// FEAT_DATE_FEATURES — crosstab axes included) in request order:
	// filterers, features, attributes, groups, crosstab rows, crosstab
	// columns. Resolution is slot `tz` → Request.TimeZone →
	// Options.DefaultTimeZone → UTC, computed by the same function the
	// runtime validates with. Empty when no zone-capable slot is present;
	// never nil in JSON output.
	TimeZones []ResolvedZone `json:"time_zones"`

	// Weights lists how a row weight resolves on every weight-bearing
	// slot (aggregations, the crosstab cell and margin aggregations,
	// tests, post-tests, regressions, attributes, overlays) in that
	// order. Resolution is slot `weight` → Request.Weight →
	// Options.DefaultWeight → none, computed by the same function the
	// runtime validates with. Omitted when the request, its slots and
	// the instance name no weight at all, so an unweighted request's
	// predict output is unchanged.
	Weights []ResolvedWeight `json:"weights,omitempty"`

	// SuggestedWeight echoes inspect's suggested_weight as DATA (never
	// a warning, so strict mode is unaffected) when no slot resolves a
	// weight field. Never applied. Omitted when there is nothing to
	// suggest, a weight resolves, or the instance hides row weighting.
	SuggestedWeight *SuggestedWeight `json:"suggested_weight,omitempty"`

	// PValues counts the inferential p-values the request would emit —
	// one per test and post-test (the Tukey HSD post-test excluded: its
	// p-values are already corrected) plus every p-value an inferential
	// overlay layer carries — and how many of them no multiple-comparison
	// correction reaches. It is trigger data for suggesting a
	// `multiplicity` block, never an advisory: compare Uncorrected with
	// Threshold. Omitted when the request emits no p-value, when its
	// multiplicity blocks are refused, or when the instance hides the
	// multiple-comparison capability.
	PValues *PValueCount `json:"p_values,omitempty"`

	// ResolvedVectors maps each Request.Vectors name to its resolved
	// member list in axis order — literal `fields` entries in the
	// caller's order, glob and `pattern` expansions in schema order —
	// computed by the same resolver the runtime refuses with, against
	// the schema the request executes over. Omitted when the request
	// defines no vector or a vector is refused (the refusal is the
	// predict error).
	ResolvedVectors map[string][]string `json:"resolved_vectors,omitempty"`

	// Matrices mirrors req.Matrices in order once every spec resolves:
	// each matrix's shape, axis, missing mode, estimated accumulator
	// size, streamability and pairwise PSD risk, computed by the same
	// resolver and rules the runtime runs. Omitted when the request
	// carries no matrix or a spec is refused (the refusal is the
	// predict error).
	Matrices []MatrixPredict `json:"matrices,omitempty"`

	// Sizes estimates each top-level Response section the request
	// produces, unshaped and under the effective `return` plan, in
	// Response key order — reported on every request, with or without
	// a `return` block (no effective block: shaped equals full). A
	// section whose size depends on data predict cannot see (a grouper
	// with data-dependent keys, a join's column set, a single file's
	// row-level output with no record count) is omitted; a section the
	// request does not produce is omitted. Omitted entirely when the
	// `return` block is refused. Guidance only — never a byte-accurate
	// figure.
	Sizes []ResponseSectionSize `json:"sizes,omitempty"`

	// LimitFindings lists every instance resource limit (pulse.Limits)
	// the request's predictable figures exceed, in limit-declaration
	// order. A Certain finding is an exact figure the runtime will see —
	// predict adds the PULSE_LIMIT_EXCEEDED error the process pre-flight
	// raises and Valid is false. A Possible finding is an upper bound: a
	// warning only, Valid is untouched. Omitted when nothing exceeds a
	// limit.
	LimitFindings []LimitFinding `json:"limit_findings,omitempty"`

	// Return is the resolved `return` plan — the expanded selection the
	// runtime will apply, computed by the same resolver it calls.
	// Omitted when the request carries no `return` block or the block is
	// refused (the refusal is the predict error).
	Return *ReturnPlan `json:"return,omitempty"`
}

PredictResult holds the validated request and any diagnostics.

type PredictSchemaInfo

type PredictSchemaInfo struct {
	FieldCount int      `json:"field_count"`
	Fields     []string `json:"fields"`
}

PredictSchemaInfo summarizes the schema used for prediction.

type ProcessChainCapability added in v0.10.0

type ProcessChainCapability struct {
	// Name is the canonical entry identifier ("process_chain").
	Name string `json:"name"`

	// MaxStages is the upper bound on Stages length. Zero indicates
	// no compile-time cap; runtime memory is bounded by the largest
	// intermediate response.Data slice.
	MaxStages int `json:"max_stages"`

	// MergeableAggregators lists the aggregator names that pass the
	// chain gate (mergeable + single-scalar emit). Alphabetically
	// sorted, deterministic across calls.
	MergeableAggregators []string `json:"mergeable_aggregators"`

	// MergeableGroupers lists the grouper names that pass the chain
	// gate.
	MergeableGroupers []string `json:"mergeable_groupers"`

	// RowLocalAttributes lists the attribute names that pass the
	// chain gate (row-local only).
	RowLocalAttributes []string `json:"row_local_attributes"`

	// RejectionRules names the operator categories the chain gate
	// rejects today. Intended for LLM-side reasoning and fallback
	// routing; not a strict schema.
	RejectionRules []string `json:"rejection_rules"`

	// OverlayKinds lists the whole-chain overlay catalog entries
	// accepted on ChainRequest.Overlays today. Alphabetically sorted.
	// Minimal additive surface kept for backward compatibility; the
	// richer per-kind sub-block at Overlays carries the full surface
	// (shapes / scopes / ref kinds / buffered / description).
	OverlayKinds []string `json:"overlay_kinds"`

	// Overlays enumerates the per-kind capability rows for every
	// whole-chain overlay the catalog declares. Sourced from
	// descriptor.overlayCapabilityFor() so the chain capability stays
	// byte-equal with the corresponding entries on Manifest.Overlays —
	// single source of truth for kind metadata. Sorted alphabetically by
	// Kind for golden stability. Each row carries:
	//
	//   - Kind        — the OverlayKind constant (SCREAMING_SNAKE).
	//   - Shapes      — every OverlayShape the kind may emit; whole-chain
	//                   kinds inherit the target stage's host shape so
	//                   the catalog declares MATRIX + SCALAR + SERIES.
	//   - Scopes      — the OverlayScope values the kind supports;
	//                   whole-chain kinds decorate the entire chain
	//                   result so Scopes == [total].
	//   - RefKinds    — the OverlayRef union pointer-field names the kind
	//                   consumes; the chain family consumes Stage.
	//   - Buffered    — !types.OverlayStreamable(kind); whole-chain kinds
	//                   stay buffered today (the streamability flag is
	//                   the inverse of the table in
	//                   types/overlay_streamability.go).
	//   - Description — short human-readable summary; mirrors the prose
	//                   in skills/overlay-system.md.
	Overlays []OverlayCapability `json:"overlays"`
}

ProcessChainCapability describes the source-rooted linear chain endpoint surfaced by pulse.ProcessChain / pulse_process_chain. The manifest carries one ProcessChainCapability entry under Manifest.ProcessChain so LLM clients can detect the chain gate and choose between the chained or per-stage fallback path.

type Purpose

type Purpose struct {
	// Plain is the one sentence a developer reads first.
	Plain string `json:"plain"`
	// Intents lists the intent IDs the operator answers; at least one.
	Intents []string `json:"intents"`
	// Questions lists natural-language questions the operator answers.
	Questions []string `json:"questions"`
	// UseCases maps an audience domain to a one-line use case.
	UseCases map[Domain]string `json:"use_cases,omitempty"`
	// NotFor lists the situations where another choice fits better.
	NotFor []Alternative `json:"not_for,omitempty"`
	// Assumptions lists the operator's assumptions as plain sentences.
	Assumptions []string `json:"assumptions,omitempty"`
	// Level grades the background the operator assumes.
	Level Level `json:"level,omitempty"`
	// Glossary lists the glossary term IDs the guidance relies on.
	Glossary []string `json:"glossary,omitempty"`
}

Purpose is an operator's plain-language guidance: what it is for, the intents it answers, when to reach for something else, and what it assumes.

type RangeTableMeta added in v0.27.0

type RangeTableMeta struct {
	Name        string `json:"name"`
	Description string `json:"description,omitempty"`
	RangeCount  int    `json:"range_count"`
}

RangeTableMeta is the manifest projection of a registered labeled- date-range table. RangeCount surfaces how many {label, start, end} entries the table carries without exposing the boundaries themselves, mirroring how LabelTableMeta.HasRowsData characterises a label table.

type RegressionMeta added in v0.6.0

type RegressionMeta struct {
	// Name is the operator identifier (REG_OLS, REG_GLM,
	// REG_BAYES_LINEAR).
	Name string `json:"name"`

	// Description is a one-sentence prose summary for LLM-side
	// operator selection.
	Description string `json:"description"`

	// AcceptsTypes lists the schema field types valid as Target /
	// Predictors entries (numeric only in v1).
	AcceptsTypes []string `json:"accepts_types"`

	// EmitsTypeNote describes the structured output (RegressionResult)
	// since no scalar field type captures the fit summary.
	EmitsTypeNote string `json:"emits_type_note"`

	// Streamable mirrors types.RegressionType.Streamable(): true for
	// closed-form fits (REG_OLS, REG_BAYES_LINEAR), false for iterative
	// fits (REG_GLM). Spec-level modifiers (Resample, Selection) can
	// downgrade this further per request — see RegressionSpec.Streamable.
	Streamable bool `json:"streamable"`

	// StreamableHint flags the modifier downgrade rule so clients know
	// non-empty Resample / Selection forces the buffered path even on
	// streamable operators.
	StreamableHint string `json:"streamable_hint,omitempty"`

	// Params lists the per-operator parameter schema, including the
	// regularization, family, prior, resample, and selection knobs that
	// modify the underlying fit.
	Params []Param `json:"params"`

	// Modifiers names the spec-level orthogonal wrappers any regression
	// type supports. Each entry is a top-level RegressionSpec field
	// (Resample, Selection) plus its enum values; the field is
	// duplicated under Params for completeness but exposed at the top
	// level here so request authors can discover the composition story
	// without parsing the full param list.
	Modifiers []RegressionModifier `json:"modifiers,omitempty"`

	// Intents lists the intent-taxonomy IDs (Manifest.Intents) the
	// regression answers, sorted. Omitted when it declares no purpose.
	Intents []string `json:"intents,omitempty"`

	// WeightKinds lists the weight kinds the regression computes a
	// weighted fit under (["frequency","probability"] or ["frequency"]);
	// omitted when it honours no weight and on an instance that hides
	// capability:weighting. See Operator.WeightKinds.
	WeightKinds []types.WeightKind `json:"weight_kinds,omitempty"`
}

RegressionMeta describes a registered REG_* operator in the manifest. The shape mirrors Operator and TestMeta enough that LLM clients can reuse the same authoring path, but regression-specific knobs (family, link, penalty, modifier slots) live alongside the shared name + streamable + params trio so the catalog stays one fetch deep.

type RegressionModifier added in v0.6.0

type RegressionModifier struct {
	Name        string   `json:"name"`
	Description string   `json:"description"`
	EnumValues  []string `json:"enum_values"`
}

RegressionModifier describes a spec-level wrapper (Resample, Selection) that composes with any regression operator. Each modifier downgrades streamability when set; clients combine its EnumValues with their chosen Type to author the request.

type ResolvedWeight

type ResolvedWeight struct {
	// Slot addresses the slot by its JSON path inside the request, e.g.
	// "aggregations[0]", "crosstab.cell",
	// "crosstab.margin_aggregations[1]", "tests[0]", "overlays[2]".
	Slot string `json:"slot"`

	// Operator is the slot's operator type (post smart-defaults), or
	// the overlay kind on an overlay slot.
	Operator string `json:"operator"`

	// Field is the resolved weight field; empty when Status is
	// "opted_out" or "none".
	Field string `json:"field,omitempty"`

	// Kind is the resolved weight kind ("probability" or "frequency",
	// the default spelled out); empty when Field is.
	Kind string `json:"kind,omitempty"`

	// Status is "applied", "skipped_not_weight_aware", "opted_out" or
	// "none".
	Status string `json:"status"`

	// Source names where the weight came from: "slot", "request",
	// "options" or "none".
	Source string `json:"source"`
}

ResolvedWeight is the resolved row weight of one weight-bearing request slot.

type ResolvedZone

type ResolvedZone struct {
	// Slot addresses the slot by its JSON path inside the request, e.g.
	// "groups[0]", "filterers[1]", "attributes[2]", "features[0]",
	// "crosstab.rows[0]", "crosstab.columns[1]".
	Slot string `json:"slot"`

	// Operator is the slot's operator type (post smart-defaults).
	Operator string `json:"operator"`

	// FieldType is the schema type of the slot's field ("datetime",
	// "date", ...), or empty when the field is not in the cohort schema
	// (a derived column).
	FieldType string `json:"field_type,omitempty"`

	// TZ is the zone name that applies to the slot, spelled as the
	// caller (or Options.DefaultTimeZone) spelled it; "UTC" when nothing
	// named one. Null when the field is not `datetime` and the zone was
	// inherited: a calendar date carries no instant, so an inherited
	// zone is not applied to it.
	TZ *string `json:"tz"`

	// Source names where TZ came from: "slot", "request", "options" or
	// "default". On a null TZ it names the level the skipped zone was
	// inherited from.
	Source string `json:"source"`
}

ResolvedZone is the resolved time zone of one zone-capable request slot.

type ResponseSectionSize

type ResponseSectionSize struct {
	// Section is the top-level Response key.
	Section string `json:"section"`
	// FullBytes estimates the section's wire bytes without the
	// selection.
	FullBytes int64 `json:"full_bytes"`
	// ShapedBytes estimates the section's wire bytes under the
	// effective `return` plan — the same model with every node the plan
	// drops removed; 0 when the plan excludes the section, FullBytes
	// when there is no effective plan. Never above FullBytes.
	ShapedBytes int64 `json:"shaped_bytes"`
	// Basis says how the section's row / cell count was derived:
	// "exact" (fixed by the request and schema — an ungrouped
	// aggregation's one row, an ungrouped matrix), "upper_bound" (a
	// schema bound the run never exceeds — grouper dictionary sizes
	// clamped to the record count, crosstab axis dictionaries, the
	// record count of a row-level output) or "heuristic" (a typical-size
	// guess — test details, regression terms, overlay payloads,
	// component operator maps, metadata).
	Basis string `json:"basis"`
}

ResponseSectionSize is the size estimate of one top-level Response section (`data`, `metadata`, `tests`, `post_tests`, `regressions`, `matrices`, `crosstab`, `overlays`, `components`): rows or cells times a bytes-per-value model of its compact JSON form.

type ReturnPlan

type ReturnPlan struct {
	// Preset is the preset the block resolved from: full, standard,
	// minimal, or custom for explicit include / exclude without one.
	Preset string `json:"preset"`
	// Include is the selected paths (each with its whole subtree).
	Include []string `json:"include"`
	// Exclude is the removed paths; exclusion wins over inclusion.
	Exclude []string `json:"exclude"`
	// Keep is the nested `warnings` slots retained wherever their parent
	// object is emitted (unless excluded).
	Keep []string `json:"keep"`
	// Precision is the wire float significant-digit count; omitted when
	// unlimited.
	Precision int `json:"precision,omitempty"`
	// Identity reports that the plan changes nothing (the whole
	// response, unrounded).
	Identity bool `json:"identity"`
	// Digest identifies the resolved selection ("rp1:" + sha256 hex);
	// the preset name does not enter it.
	Digest string `json:"digest"`
	// UnresolvedIncludes lists the Open include paths (through a map key
	// or an open value) predict cannot resolve without data: at runtime
	// each may match nothing, which a buffered run reports as
	// PULSE_RETURN_PATH_UNMATCHED and a stream never reports. A
	// `data[*].<column>` include predict can match against the
	// request's known output columns is resolved and not listed.
	// Omitted when none.
	UnresolvedIncludes []string `json:"unresolved_includes,omitempty"`
}

ReturnPlan is a resolved `return` block: the canonical path sets the selection reduces to and a digest identifying it. Include, Exclude and Keep are deduplicated, sorted, and free of paths another one already covers, so equivalent spellings resolve to the same plan and digest.

type ReturnPresetMeta

type ReturnPresetMeta struct {
	// Name is the preset name (a types.ReturnPreset).
	Name string `json:"name"`
	// Paths is the selection the preset expands to on this instance, in
	// table order. Nested `warnings` slots are kept wherever their
	// parent is emitted and are not listed; full lists every visible
	// top-level key.
	Paths []string `json:"paths"`
}

ReturnPresetMeta is one named `return` preset as the manifest lists it.

type Role

type Role struct {
	// Name identifies the role within its shape (e.g. "outcome").
	Name string `json:"name"`
	// Kinds lists the field kinds that may fill the role; at least one.
	Kinds []FieldKind `json:"kinds"`
	// Min is the fewest fields the role takes; 0 marks it optional.
	Min int `json:"min"`
	// Max is the most fields the role takes, at least Min, or
	// RoleUnbounded for no upper bound.
	Max int `json:"max"`
}

Role is one named slot in an intent Shape — "outcome", "group", "measures" — with the field kinds that may fill it and how many fields it takes.

type Shape

type Shape struct {
	// Roles lists the shape's roles in authoring order.
	Roles []Role `json:"roles"`
}

Shape is one data shape an intent applies to: the set of roles a cohort's fields must fill for the intent's question to be answerable. An intent may declare several alternative shapes.

type ShardInfo added in v0.8.0

type ShardInfo struct {
	Filename    string `json:"filename"`
	RecordCount int64  `json:"record_count"`
}

ShardInfo is one shard inside a Pulse shard archive, surfaced by Inspect for archive-backed cohorts. Mirrors the public shape of internal/service.ShardEntry without importing it — descriptor/ is header-only and must not depend on the execution layer.

Filename is the basename of the shard inside the archive (e.g. "20190101.pulse"). RecordCount is the number of records carried by the shard, computed by peeking that shard's own header (the per-shard headers are authoritative; the canonical _schema.pulse aggregate is only a sanity check).

type SkillMeta

type SkillMeta struct {
	Name        string `json:"name"`
	Description string `json:"description"`
}

SkillMeta describes a bundled skill.

type SlotPair added in v0.19.0

type SlotPair struct {
	// ReferenceLabel is the slot label the spec's Reference field
	// resolves to. May be empty when the failure is the reference
	// itself being unknown / unresolvable.
	ReferenceLabel string `json:"reference_label"`

	// TargetLabel is the slot label the spec's Targets[j] resolves
	// to. May be empty when the failure is on the Reference arm.
	TargetLabel string `json:"target_label"`

	// Reason is a stable, machine-readable identifier for the failure
	// class. See the SlotPair doc for the enum.
	Reason string `json:"reason"`
}

SlotPair carries one rejected (Reference, Target) Compose-overlay slot pair from internal/descriptor.ValidateCompose's overlay walk. Mirrors ChainOverlaySchemaDivergence in spirit but flattens to a string-typed (ref label, target label, machine-readable reason) tuple so the PredictResult.OverlaysSchemaDivergence slot and the ComposeValidationResult.OverlaysSchemaDivergence slot can share one renderer-side shape.

PRD §I-FR-I3: the predict surface for Compose overlays is `OverlaysApplied + OverlaysSchemaDivergence + OverlayCost`. The divergence slot carries the per-spec failure reason alongside the envelope error entry so LLM planners can budget reshapes without stitching envelope details together.

Reason values are stable identifiers the renderer can branch on:

  • "kind-unknown" — spec.Kind absent from the overlay catalog.
  • "reference-unknown" — Reference label does not resolve to a slot.
  • "target-unknown" — Targets[j] does not resolve to a slot.
  • "slot-shape-divergent" — ref and target host shapes differ.
  • "slot-not-crosstab" — kind requires MATRIX but a slot isn't.
  • "schema-divergent" — per-axis grouper-kind tuples disagree.
  • "panel-targets-over-cap" — multi-ref target count exceeds the cap.

type SuggestedWeight

type SuggestedWeight struct {
	Field  string `json:"field"`
	Source string `json:"source"`
	Kind   string `json:"kind"`
}

SuggestedWeight is a row weight a cohort's metadata proposes. It is reported by inspect and echoed by predict; it is never applied — a caller opts in by naming Field in a request `weight`.

Kind is the weight kind the suggestion proposes: "probability", the request default. SPSS `WEIGHT BY` replicates cases (frequency-like), so a caller who means the SPSS semantics names kind "frequency".

type Suggestion added in v0.5.0

type Suggestion struct {
	Path       []string `json:"path"`
	Reason     string   `json:"reason"`
	Current    any      `json:"current,omitempty"`
	Proposed   []any    `json:"proposed,omitempty"`
	Confidence float64  `json:"confidence"`
}

Suggestion is a structured next-action attached to PredictResult. Predict computes suggestions inline so callers can repair a request without an additional inspect round-trip.

Path points at the offending request location using JSON-style segments — e.g. ["Aggregations", "0", "Field"] addresses the Field of the first aggregation.

Proposed is a ranked list of candidate values. Empty when no concrete proposal applies (e.g. ATTR_PERCENTILE has no streamable peer); the caller should treat empty Proposed as advisory.

Confidence is a static heuristic in [0, 1]: 0.9 for high-certainty single-candidate swaps and Levenshtein distance 1; 0.7 for distance 2; 0.6 for multi-candidate type-class swaps; 0.5 for missing-param fallbacks that hand the user a list to pick from; 0.8 for streamability substitutes.

type Term

type Term struct {
	// ID is the stable identifier purposes and other terms cite
	// (e.g. "p-value").
	ID string `json:"id"`
	// Short is a one-sentence definition.
	Short string `json:"short"`
	// WhyCare says why the reader should care, in plain words.
	WhyCare string `json:"why_care,omitempty"`
	// SeeAlso lists related term IDs.
	SeeAlso []string `json:"see_also,omitempty"`
	// Jargon marks a term that must be glossary-linked wherever it
	// appears in plain-language guidance.
	Jargon bool `json:"jargon,omitempty"`
	// Forms lists the surface spellings that count as the term
	// appearing in prose (e.g. "p-value", "p value").
	Forms []string `json:"forms,omitempty"`
}

Term is one glossary entry.

type TestMeta added in v0.5.0

type TestMeta struct {
	// Name is the canonical entry identifier. For tier-1 this is the
	// TestType string (e.g. "TEST_ANOVA_WELCH"). For tier-2 this is the
	// TestType plus the variant suffix
	// (e.g. "TEST_ANOVA_WELCH/welch_one_way_post").
	Name string `json:"name"`

	// Family is the canonical TestType the variant belongs to
	// (e.g. "TEST_PEARSON_R"). Tier-1 entries always have
	// Family == Name. Tier-2 entries set Family to the underlying
	// TestType so clients can pair siblings.
	Family string `json:"family"`

	// Tier is 1 for row tests in Request.Tests, 2 for post tests in
	// Request.PostTests.
	Tier int `json:"tier"`

	// Variant is the algorithm flavour for tier-2 entries
	// (e.g. "welch_one_way_post"). Empty for tier-1 entries.
	Variant string `json:"variant,omitempty"`

	// Description is a one-sentence prose summary.
	Description string `json:"description"`

	// Streamable mirrors types.TestType.Streamable() for tier-1 entries.
	// Always false for tier-2 entries.
	Streamable bool `json:"streamable"`

	// Params lists the operator-specific parameters (alpha,
	// success_value, etc.).
	Params []Param `json:"params"`

	// Requires lists the top-level Test fields that must be set for the
	// test to run (e.g. "Field", "Field2", "SplitBy", "Rows", "Cols").
	// Drives request authoring directly.
	Requires []string `json:"requires,omitempty"`

	// Intents lists the intent-taxonomy IDs (Manifest.Intents) the test
	// family answers, sorted. Tier-2 entries carry their Family's.
	// Omitted when the family declares no purpose.
	Intents []string `json:"intents,omitempty"`

	// WeightKinds lists the weight kinds the tier-1 test computes a
	// weighted figure under (["frequency","probability"] or
	// ["frequency"]); omitted when it honours no weight, on every tier-2
	// entry (a post-test reads aggregated rows, never row weights), and
	// on an instance that hides capability:weighting. See
	// Operator.WeightKinds.
	WeightKinds []types.WeightKind `json:"weight_kinds,omitempty"`
}

TestMeta describes a statistical test entry in the manifest. Tier-1 and tier-2 tests share this shape; they live in separate top-level slices (Manifest.Tests for tier-1, Manifest.PostTests for tier-2). The Family field ties variants of the same underlying test together across tiers so clients can filter by family.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL