pulse

package module
v1.0.0-alpha.8 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 9, 2026 License: MIT Imports: 58 Imported by: 0

README

Pulse

High-performance, self-describing tabular data processing engine written in Go. Ships as a Go library (github.com/frankbardon/pulse) and a CLI binary (cmd/pulse/). Library is primary; CLI is a thin adapter.

Pulse reads and writes .pulse files — a compact binary format with an inline schema, categorical dictionaries, and per-field descriptions. Import from CSV, TSV, NDJSON, JSON array, Parquet, Apache Arrow IPC, or Excel; run aggregations, filters, groupers, windows, features, statistical tests, and regressions; export back to any supported format.

LLM-native by design: every command supports --json, every operator declares its accepted types and streamability in the manifest, an embedded skill pack teaches agents how to operate Pulse, an embedded request-example library gives them runnable templates, and pulse mcp serves the full surface over Model Context Protocol.

Installation

From source
git clone https://github.com/frankbardon/pulse.git
cd pulse
make build
# Binary at ./bin/pulse
Go install
go install github.com/frankbardon/pulse/cmd/pulse@latest

Requires Go 1.22+.

Quickstart

Import a CSV into a .pulse file
pulse import csv --input data.csv --output data.pulse

Schema is inferred from a sample of rows (default 500). To supply an explicit schema:

# Generate a schema template from your data
pulse import schema-template data.csv > schema.json

# Edit schema.json — adjust types, add descriptions
pulse import csv --input data.csv --schema schema.json --output data.pulse
Inspect the file
pulse cohort inspect data.pulse --json

Returns field names, types, byte offsets, descriptions, and categorical dictionaries (truncated by default; pass --full-dict for the full mapping).

Run an aggregation

Create a request file (request.json):

{
  "cohort": {"filename": "data.pulse"},
  "aggregations": [
    {"type": "AGG_COUNT", "field": "id", "label": "total"},
    {"type": "AGG_AVERAGE", "field": "score", "label": "avg_score"}
  ]
}

pulse api process --request request.json --json
Filter, group, and aggregate
{
  "cohort": {"filename": "data.pulse"},
  "filterers": [
    {"type": "FILTER_RANGE", "field": "score", "values": ["80", "100"]}
  ],
  "groups": [
    {"type": "GROUP_CATEGORY", "field": "brand"}
  ],
  "aggregations": [
    {"type": "AGG_COUNT", "field": "id", "label": "count"},
    {"type": "AGG_AVERAGE", "field": "score", "label": "avg_score"},
    {"type": "AGG_STDDEV", "field": "score", "label": "stddev"}
  ]
}
Smart defaults

If you name a field but omit type, Pulse fills in a sensible operator from the schema type — AGG_SUM for numerics, AGG_MODE_COUNT for categoricals, GROUP_RANGE (interval 10) for numerics, GROUP_CATEGORY for categoricals, GROUP_DATE ("day") for dates. Disable with --no-defaults or pulse.Options{DisableDefaults: true}. The full rule table lives in internal/descriptor/defaults.go.

Validate before executing
pulse api predict --request request.json --json

Returns the proposed schema, applied defaults, streamability, warnings (e.g., numeric aggregation on a categorical field), and structural errors without touching record data. Pass --strict to treat warnings as errors.

Streaming

For requests Pulse can stream (single-pass aggregations on non-decimal fields, no buffered ops), use --stream to get NDJSON one row per line:

pulse api process --request request.json --stream
pulse api compose --request batch.json --stream

Library equivalent: pulse.ProcessStream(ctx, req) returns a RowIter.

Parallel compose
pulse api compose --request batch.json --parallel 4 --json

Library equivalent: pulse.ComposeParallel(ctx, req, pulse.ComposeOptions{MaxWorkers: 4}). Order-preserving by slot; --no-fail-fast aggregates errors instead of cancelling on first failure.

Export and convert
pulse export csv --input data.pulse --output results.csv
pulse export parquet --input data.pulse --output results.parquet

pulse convert data.csv data.parquet
pulse convert data.xlsx output.tsv --schema schema.json

Format auto-detected from extensions. convert does not write an intermediate .pulse unless --keep-pulse=path.

Sample rows / facet a field
pulse api sample --input data.pulse --count 10
pulse api facet --input data.pulse --field brand
Synthetic data

Generate a deterministic synthetic cohort from a schema spec or from a previously-captured profile:

pulse synth from-schema --spec spec.json --output synth.pulse --seed 42
pulse profile create --input real.pulse --output profile.json --include-correlations
pulse synth from-profile --profile profile.json --output synth.pulse --rows 100000 --seed 42

Distributions such as normal, lognormal, poisson, bernoulli, weighted_categorical, regex, uniform_date and monotonic_from (the manifest's synth_distributions block has the full list), pairwise correlations, value constraints. See the synthetic-data skill.

CLI Reference

pulse
├── --json [--slim]                       Root manifest (self-description)
├── import
│   ├── csv|tsv|ndjson|jsonarray|         --input FILE --output FILE [--schema FILE]
│   │   parquet|arrow|excel
│   ├── auto SOURCE                       Managed import (auto-detect format)
│   ├── list                              List managed import handles
│   ├── drop HANDLE                       Drop a managed handle
│   ├── predict                           --input FILE [--schema FILE] --json
│   └── schema-template INPUT             Generate editable schema from source
├── export
│   ├── csv|tsv|ndjson|jsonarray|         --input FILE --output FILE
│   │   parquet|arrow|excel
│   └── predict                           --input FILE --format FORMAT --json
├── convert INPUT OUTPUT [--from F] [--to F] [--schema FILE] [--keep-pulse PATH]
│   └── predict INPUT OUTPUT --json
├── cohort
│   ├── inspect PATH [--json] [--full-dict]
│   └── filter --input FILE --output FILE --filter EXPR
├── api
│   ├── process --request FILE [--json] [--stream] [--no-defaults]
│   ├── compose --request FILE [--json] [--stream] [--parallel N] [--no-fail-fast]
│   ├── sample --input FILE --count N
│   ├── facet --input FILE --field NAME
│   └── predict --request FILE --json [--strict]
├── synth
│   ├── from-schema --spec FILE --output FILE [--rows N] [--seed N]
│   └── from-profile --profile FILE --output FILE --rows N [--seed N]
├── profile
│   └── create --input FILE --output FILE [--top-k N] [--include-correlations]
├── skills
│   ├── list [--json]
│   └── show NAME
├── examples
│   ├── search [--query Q] [--tag T ...] [--category C] [--json]
│   └── show NAME [--json]
├── errors
│   ├── lookup CODE [--json]
│   └── list [--domain D] [--query Q] [--json]
└── mcp [--data-dir DIR] [--bind-on-open]   Serve MCP over stdio

Every leaf supports --json for output wrapped in a descriptor.Envelope with format_version: "1.1", data, errors, and warnings. pulse api compose --json wraps a ComposedResponse {responses[], overlays[]} on data — not the legacy []*Response array.

MCP Usage

Pulse ships an MCP (Model Context Protocol) server that exposes the full library surface to AI clients (Claude Desktop, Claude Code, Cursor, any MCP-aware host). One binary, one command:

pulse mcp --data-dir /path/to/data

PULSE_DATA_DIR is honored if --data-dir is omitted. The server speaks JSON-RPC over stdio (logs to stderr).

Wiring into a host

For Claude Desktop, add to ~/Library/Application Support/Claude/claude_desktop_config.json:

{
  "mcpServers": {
    "pulse": {
      "command": "pulse",
      "args": ["mcp"],
      "env": {
        "PULSE_DATA_DIR": "/Users/me/data"
      }
    }
  }
}

For Claude Code:

claude mcp add pulse --env PULSE_DATA_DIR=/Users/me/data -- pulse mcp

Restart the host. Pulse tools appear in the tool list.

Tool surface
Tool Purpose
pulse_manifest Call first. Self-description: commands, operators, accepted types, tests, regressions, distributions, error codes, MCP tools, cohort field types. Cache once per session.
pulse_inspect Read header + schema (no record bytes). Also binds session-scoped field-name enums on action tools.
pulse_predict Validate a request against the schema without executing.
pulse_process Execute one pre-built request.
pulse_compose Execute a batch of requests in one round trip.
pulse_sample Return up to N rows for preview.
pulse_facet Distinct values for a single field.
pulse_import Import a tabular source into a managed .pulse handle (TTL-tracked, default 7d).
pulse_dedup Deduplicate an existing cohort into parent groups, or suggest groups (read-only).
pulse_recommend Turn a question kind (intent ID) into ranked draft requests, predict-validated when bound to a cohort.
pulse_explain Say in plain words what a request will do, or what a result found (verdicts, strength bands, caveats).
pulse_drop Drop a managed handle.
pulse_imports_list Enumerate managed handles with sidecar metadata.
pulse_examples_search Search the embedded request-example library by query, tags, category.
pulse_examples_get Fetch one runnable example by name.
pulse_errors_lookup Per-code Message + Fixup detail (kept out of the manifest for context economy).
pulse_skills_list / pulse_skills_get Embedded skill pack.
Resources and prompts
URI What
pulse://<path> One per .pulse file under the data directory. Read returns descriptor.InspectResult JSON.
pulse-skill://<name> One per embedded skill. Read returns the markdown body.

Prompts pulse-bootstrap and pulse-author-request, plus one guided pulse-<intent> prompt per analytic intent (inspect → recommend → confirm → process → explain), are registered for hosts that surface them as slash commands.

  1. pulse_manifest once at session start. Cache the result — it is deterministic for a binary version and carries every fact needed to author a valid request.
  2. pulse_import when the user hands the LLM a raw tabular file; skip when the cohort already exists as a managed handle or .pulse under PULSE_DATA_DIR.
  3. pulse_inspect on the handle (or path). Reads header + schema only and binds session-scoped field-name enums on the action tools.
  4. pulse_predict with the authored request. Validates against the schema; each error code carries Fixup metadata so the LLM can repair the request without another round trip.
  5. pulse_process to execute. Use pulse_compose for batching, pulse_sample / pulse_facet for cheap probes.
Schema-bound enums

After a successful pulse_inspect, the server registers session-scoped variants of the action tools whose JSON Schemas embed enums on field-name parameters — picked from a typed list rather than free-texted. Works on SSE / Streamable HTTP transports; on stdio the session does not support tool overrides, so enums are advisory and pulse_predict remains the validation gate. Disable with --bind-on-open=false.

The session-bootstrap skill (pulse skills show session-bootstrap) and the tool-* skills are the authoritative reference.

Embedding Pulse in a Go Application

Pulse is library-first. The CLI is a thin adapter over the public API.

package main

import (
    "context"
    "fmt"
    "log"

    "github.com/frankbardon/pulse"
    pio "github.com/frankbardon/pulse/io"
    "github.com/frankbardon/pulse/types"
)

func main() {
    ctx := context.Background()

    p, err := pulse.New(pulse.Options{DataDir: "/var/data"})
    if err != nil {
        log.Fatal(err)
    }

    // Import a CSV. The io factory picks the adapter from a typed format.
    src, err := pio.NewReader(pio.FormatCSV, p.Fs(), "input.csv", pio.ReaderOptions{})
    if err != nil {
        log.Fatal(err)
    }
    importJob := pio.NewImportJob(src, "dataset.pulse")
    report, err := p.Import(ctx, importJob)
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("Imported %d rows\n", report.RowsImported)

    // Run an aggregation.
    resp, err := p.Process(ctx, &pulse.Request{
        Cohort: &types.Cohort{Filename: "dataset.pulse"},
        Aggregations: []*types.Aggregation{
            {Type: types.AGG_AVERAGE, Field: "score", Label: "avg"},
        },
    })
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(resp.Data)

    // Inspect a file.
    result, err := p.Inspect(ctx, "dataset.pulse")
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("Fields: %d\n", result.FieldCount)

    // Validate before execution.
    prediction, err := p.Predict(ctx, &pulse.Request{
        Cohort: &types.Cohort{Filename: "dataset.pulse"},
        Aggregations: []*types.Aggregation{
            {Type: types.AGG_SUM, Field: "revenue"},
        },
    })
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("Warnings: %v\n", prediction.Warnings)
}
Public facade

pulse.Pulse exposes: New, Open, Process, ProcessStream, Compose, ComposeParallel, Import, ImportFile, Drop, Imports, SweepImports, ResolveImport, Export, Convert, Inspect, Predict, Sample, Facet, Synth, Profile, Manifest, plus example/error lookup helpers.

Custom filesystem

Pulse accepts any afero.Fs for testing or non-local backends:

import "github.com/spf13/afero"

// In-memory filesystem for testing
p, _ := pulse.New(pulse.Options{
    FS: afero.NewMemMapFs(),
})

afero.NewMemMapFs() gives hermetic tests a complete in-memory filesystem — no disk I/O.

Operator catalogue

The manifest is the source of truth for what is registered — pulse manifest --json (or the pulse_manifest MCP tool) lists every operator with its params and capabilities. This README keeps categories and representative members only, never counts:

  • Aggregators (AGG_*): COUNT, SUM, AVERAGE, MIN, MAX, MEDIAN, STDDEV, PERCENTILE, FREQUENCY, DISTINCT_COUNT, WEIGHTED_MEAN, RATIO, set aggregators (SET_FREQUENCY, SET_UNION, …), …
  • Attributes (ATTR_*): ZSCORE, TSCORE, NORMALIZED, FORMULA, PERCENTILE, DATE_PART, regression diagnostics (REG_FITTED, REG_RESIDUAL, …), set attributes, …
  • Filterers (FILTER_*): INCLUDE, EXCLUDE, RANGE, EXPRESSION, NULL, DATE_RANGES, set-membership filters (SET_CONTAINS_ANY, …), …
  • Groupers (GROUP_*): CATEGORY, RANGE, ROUNDED, DATE, DATE_RANGES, QUANTILE, set groupers, …
  • Windows (WIN_*): LAG, LEAD, ROW_NUMBER, RANK, RUNNING_SUM, MOVING_AVG, EWMA, PCT_CHANGE, …
  • Features (FEAT_*): LOG, SQRT, BUCKETIZE, ONE_HOT, TARGET_ENCODE, DATE_FEATURES, TRAIN_TEST_SPLIT, POLY, …
  • Statistical tests (TEST_*): tier-1 row tests (T, WELCH, CHISQ, ANOVA_F, KS, PEARSON_R, MANN_WHITNEY_U, FISHER_EXACT, SHAPIRO_WILK, …) and tier-2 post-tests (TUKEY_HSD, TREND, summary-statistic variants)
  • Regressions (REG_*): OLS, GLM, BAYES_LINEAR — with Resample / Selection modifiers and FEAT_POLY composition covering the common textbook regression variants
  • Synth distributions: listed under synth_distributions in the manifest

LLM Skill Pack

Pulse bundles a skill pack that teaches LLM agents how to operate it. Skills are embedded via //go:embed — no external files. pulse skills list (or the manifest's skills block) is the authoritative listing; this README does not keep a count.

Discovering skills
pulse skills list
pulse skills list --json
pulse skills show session-bootstrap
Bundled skills

The pack has two shapes. Atomic skills cover one registered surface each, and their name says which: op-<category>-<kebab> per operator (op-agg-sum, op-group-date, op-test-t, …), tool-<kebab> per MCP tool (tool-process, tool-predict, …) and type-<kebab> per field type (type-categorical-u8, type-set-u256, …). Topical skills cover cross-cutting design:

Skill Purpose
session-bootstrap Canonical MCP session order — start here
request-envelope Request shapes, slot keys, smart defaults, streamability
cohort-schema-design Field-type selection, nullability, shard archives
aggregation-design Aggregator + filterer slot semantics
attribute-composition ATTR_* slot ordering and the formula environment
grouper-design Multi-grouper composition, smart defaults per type
window-design WIN_* partition / order / frame semantics
feature-engineering Pre-filter FEAT_* ordering and the leakage trap
statistical-testing Tier-1 tests vs tier-2 post-tests, assumption gates
regression-modeling REG_* operators, modifiers, textbook-name mapping
crosstab-guide Rows × columns grids, margins, normalization, fused vs buffered
compose-requests Batched requests against one cohort, parallel Compose
process-chain Source-rooted linear pipelines
join-design Pushdown hash joins
facet-design Per-field summaries and facet overlays
overlay-system Overlay composition and host wiring
pairwise-n-sources Sample-size vocabulary for pairwise overlays
response-components Response.Components — the parts behind every figure
label-display Display labels at output time
streaming-and-watching Request hashing, streaming, Watch, derived cohorts
request-templating Stored parameterised requests
financial-cohorts decimal128 money semantics
spss-cohorts SPSS .sav / .zsav import and .sav writing
synthetic-data synth from-schema / from-profile entry point; routes to the focused synth-* skills (capture, marginals, set fields, correlations, conflicts, fidelity report, determinism)
synth-models --fit-models entry point; routes to selection, draw, residual correlations and the two recovery sections
synth-structural-rules Synth spec rules[] and constraints[] entry point; routes to expressions, nulls, validation, claims and profile-derived rules
From Go

The skill pack is embedded in the binary (internal/skills/) and is not a Go import. Reach it through pulse skills list / pulse skills show <name>, the MCP skill tools and pulse-skill:// resources (mounted by gosdk.Register or mcpserve), or the skills block of (*pulse.Pulse).Manifest.

The root manifest (pulse --json) includes a skills[] array so agents can discover available skills in one call.

.pulse File Format

Binary, self-describing, fully transportable:

  • 9-byte header: magic bytes (PULSE\x00\x00\x00) + format version (0x01, or 0x02 when the schema declares a parent group)
  • Schema block: field count, per-field descriptors (type, name, byte offset, bit position, source column index, optional description capped at 1000 bytes)
  • Dictionary blocks: one per categorical field (string-to-integer mapping stored inline)
  • Record data: fixed-width binary records, one per row

Field types (the manifest's cohort_types block is the authoritative list; per-type detail lives in the type-* skills):

Type Bytes Notes
u4 0 4-bit unsigned; bit-packed
u8, u16, u32, u64 1, 2, 4, 8 Unsigned integers
f32, f64 4, 8 IEEE 754 floats
date 4 Days since Unix epoch
datetime 8 Seconds since Unix epoch
packed_bool 0 Bit-packed; shares bytes with adjacent packed fields
categorical_u8, categorical_u16, categorical_u32 1, 2, 4 Dictionary-encoded strings
decimal128 16 Fixed precision/scale; banker's rounding
set_u8 … set_u64, set_u128, set_u256 1–8, 16, 32 Multi-select bitmask over an inline dictionary

Nullability is orthogonal to type: any field can be nullable, carried by a per-record null bitmap. Categorical width auto-selected from sample cardinality during import. Bit-packed types report ByteSize() == 0 — they share bytes with adjacent packed fields. Schema reader rejects unknown type bytes at parse time with ENCODING_INVALID.

Configuration

Environment variables (pulse.Options always overrides; a .env at the repo root is auto-loaded):

export PULSE_DATA_DIR=/path/to/data        # Base directory for .pulse cohorts (fs.Default); bypass with Options{DataDir} or Options{FS}
export PULSE_IMPORTS_DIR=imports           # Managed-imports subdir under the fs root (default "imports")
export PULSE_IMPORT_TTL=7d                 # Default TTL for managed imports: Go duration ("24h"), day form ("7d") or "pin"; default 7d
export PULSE_LABEL_TABLES_DIR=/path/labels # Directory whose *.json files auto-load as LabelTables at pulse.New, keyed by filename
export PULSE_RANGE_TABLES_DIR=/path/ranges # Same shape for RangeTables ({label,start,end} array or {"description","ranges"} wrapper)
export PULSE_MCP_NO_COHORT_SCAN=1          # pulse mcp only: skip the startup cohort enumeration; pulse:// URIs stay readable
export PULSE_TEMPLATES_DIR=/a:/b           # Request-template roots, PATH-style precedence (first root wins)

Embedders override per-instance via pulse.Options{DataDir, ImportsDir, ImportTTL, FS, LabelTablesDir, RangeTablesDir, TemplateDirs}; the cohort-scan switch is gosdk.Config.DisableCohortScan / mcpserve.Options.DisableCohortScan (CLI pulse mcp --no-cohort-scan). No config files.

Output Format Contract

All --json output is wrapped in a descriptor.Envelope:

{
  "format_version": "1.1",
  "data": { ... },
  "errors": [],
  "warnings": []
}

format_version is "1.1" (bumped from "1.0" for the Compose facade lift — pulse api compose --json now wraps a ComposedResponse object {responses[], overlays[]} on data instead of a raw []*Response array). Errors/warnings use {"code", "message", "details"} entries (empty array when absent, never null). Additive-only: new data fields don't bump the version; renames or removals do.

Development

Build
make build    # Binary at ./bin/pulse
make test     # go test ./...
make fmt      # gofmt
make vet      # go vet
make lint     # staticcheck (auto-installed via go run)
make cover    # Coverage report
make docs     # Regenerate the Analysis Guide (docs/src/guide) and build mdBook
make dist     # 6-platform release archives + checksums.txt in ./dist (VERSION=v1.2.3)
make contrib  # vet + staticcheck + tidy check + test the contrib/ adapter modules
make clean    # Remove artifacts

A .env at repo root is auto-loaded by the Makefile.

Run tests
go test ./...
go test ./processing/...
go test ./internal/service/... -v -run TestProcess
go test ./encoding/... -fuzz FuzzPulseFileHeader -fuzztime 30s
Project structure
pulse/
├── pulse.go                Public facade
├── cmd/pulse/              CLI binary (thin adapter)
├── types/                  Request/response structs + streamability table
├── errors/                 Typed error codes (CodedError system)
├── encoding/               Schema nouns + ungrouped raw-byte primitives
├── descriptor/             Result + envelope types (manifest, predict, inspect)
├── io/                     Import/export jobs, Reader/Writer, io.Format factory
├── synth/                  Synthetic data specs, profiles, Synth
├── mcp/gosdk/, mcpserve/   Mount Pulse's MCP tools / run a ready-made server
├── extend/                 Public extension-authoring contract (Record, Rows, operator interfaces)
├── docs/                   mdBook source (GitHub Pages)
└── internal/               Everything else: service, descriptor and encoding
                            builders, io/<fmt> adapters, operator engine, synth engine, mcp core,
                            skills, examples, fs, imports, template, cli

Only the packages above internal/ are importable; the full tree is in .claude/reference/architecture.md.

Documentation: https://frankbardon.github.io/pulse/.

License

MIT. See LICENSE.

Documentation

Overview

Package pulse is a high-performance, self-describing tabular data processing engine.

Pulse ships as a CLI binary and as an embeddable Go library. The library is the primary deliverable; the CLI is a thin adapter over it.

Index

Constants

View Source 
const (
	DefaultRequestTimeout     time.Duration = limits.DefaultRequestTimeout
	DefaultMaxGroups          int64         = limits.DefaultMaxGroups
	DefaultMaxCrosstabCells   int64         = limits.DefaultMaxCrosstabCells
	DefaultMaxEstimatedMemory int64         = limits.DefaultMaxEstimatedMemory
	DefaultMaxMatrixDim       int64         = limits.DefaultMaxMatrixDim
	DefaultMaxComposeSlots    int64         = limits.DefaultMaxComposeSlots
	DefaultMaxChainStages     int64         = limits.DefaultMaxChainStages
	DefaultMaxJoinBuildRows   int64         = limits.DefaultMaxJoinBuildRows
)

Built-in limit defaults — the effective value of a Limits field left zero on every layer. RequestTimeout and MaxEstimatedMemory are opt-in (Unlimited); the rest are high enough that no legitimate request trips them. Raising a default is compatible; lowering one is not.

View Source 
const (
	LabelModeReplace = types.LabelModeReplace
	LabelModeAugment = types.LabelModeAugment
)

LabelMode constants re-exported for caller ergonomics.

View Source 
const (
	// LookupMultiplicityAssertUnique errors PULSE_LOOKUP_AMBIGUOUS when
	// the key matches more than one row. It is the zero-value default.
	LookupMultiplicityAssertUnique = types.LookupMultiplicityAssertUnique
	// LookupMultiplicityFirst returns the lowest-row-id match.
	LookupMultiplicityFirst = types.LookupMultiplicityFirst
	// LookupMultiplicityAll returns every match, ascending row-id order.
	LookupMultiplicityAll = types.LookupMultiplicityAll
)

LookupMultiplicity mode constants, re-exported so callers can set LookupRequest.Multiplicity without importing the types package.

View Source 
const (
	// SidecarKindPointLookupIndex is a `cohort.pulse.<keyhash>.idx`
	// point-lookup index. It carries a SHA-256 + size + mtime
	// fingerprint over the cohort and refuses to serve once the size
	// moves (PULSE_INDEX_STALE).
	SidecarKindPointLookupIndex = "point_lookup_index"
	// SidecarKindSPSSMetadata is a `cohort.pulse.spss.json` SPSS
	// metadata sidecar. Same fingerprint shape over the same cohort;
	// a size change is PULSE_SPSS_SIDECAR_STALE on the next
	// `pulse export spss`.
	SidecarKindSPSSMetadata = "spss_metadata"
)

Sidecar kind vocabulary for StaleSidecar.Kind. It is closed and small: these are the two documents beside a cohort whose validity is bound to the cohort's BYTES, so these are the two a cohort-rewriting operation invalidates. The index MANIFEST (internal/encoding.IndexManifestSuffix) is deliberately absent — it catalogues indexes rather than fingerprinting the cohort, so a rewrite leaves it accurate about which indexes exist while the indexes themselves go stale.

View Source 
const Unlimited = limits.Unlimited

Unlimited disables a limit. Untyped, so it is assignable to every Limits field (time.Duration and int64 alike).

Variables

This section is empty.

Functions

func ExampleFeatureProfiles 

func ExampleFeatureProfiles() []string

ExampleFeatureProfiles returns the names of the published example feature profiles, sorted. A name is the example's file name without the ".json" extension (for example "minimal"); pass it to ExampleFeatureProfile.

func Glossary 

func Glossary() []descriptor.Term

Glossary returns the plain-language glossary of the statistical terms Pulse's guidance relies on, in declaration order. Each call returns a fresh deep copy: mutating it cannot affect the registry. The glossary is static — not a feature, so no feature profile prunes it — and is also served as the "glossary" skill on every skill surface.

func Intents 

func Intents() []descriptor.Intent

Intents returns the closed intent taxonomy — the kinds of question an analysis can answer — in declaration order. Each call returns a fresh deep copy: mutating it cannot affect the registry. Operator purposes and the manifest's intents list cite these IDs; the taxonomy is also served as the "intents" skill on every skill surface.

func Version 

func Version() string

Version returns the Pulse build version. It resolves, in order, to the value injected at link time (`make build` sets it from `git describe` via -ldflags -X github.com/frankbardon/pulse/internal/buildinfo.version), then Pulse's own module version from the build info (Main when built via `go install module@vX.Y.Z`, the Pulse dependency entry — honouring replace — when linked into an embedder; never the embedder's version), and finally "devel" — suffixed "+<short vcs revision>" when the binary embeds VCS metadata. It never returns an empty string.

Types

type AddShardResult added in v0.38.0 

type AddShardResult = service.AddShardResult

AddShardResult carries the outcome of AddShard: the archive's new shard count, any set-field widenings the add forced, and the non-fatal warnings (PULSE_SHARD_DESCRIPTION_DIVERGENCE, PULSE_SHARD_SET_WIDENED) to lift onto a --json envelope.

type AggregatorComponentsFunc added in v0.20.0 

type AggregatorComponentsFunc func(instance extend.Aggregator) (map[string]any, error)

AggregatorComponentsFunc is the optional emitter an embedder may supply alongside an AggregatorRegistration to make the operator participate in the per-operator components contract. The orchestrator invokes the func ONCE after Aggregate / Finalize and routes the result onto Response.Components.Aggregations[i].Operator; the universal floor ({"n", "n_null"}) is filled unconditionally by the orchestrator. Returning (nil, nil) is the canonical signal for "no operator-specific keys; orchestrator's universal floor is the entire payload" (floor-only operators).

The instance passed in is the extend.Aggregator the registration's Factory returned (after Aggregate / Finalize), so a type assertion to the embedder's concrete type succeeds; the func should not call Aggregate / UpdateRow / Finalize on it. The returned map's keys must be a SUBSET of the registration's ComponentSchema.Keys (universal- floor keys allowed but not required) — probe-validation enforces this contract at pulse.New time and runtime mismatches surface as PULSE_EXTENSION_COMPONENT_SCHEMA_MISMATCH.

type AggregatorRegistration added in v0.7.0 

type AggregatorRegistration struct {
	Name        types.AggregationType
	Description string
	// Purpose is the optional plain-language guidance for the
	// operator — what it is for, the intents it answers, when to use
	// something else. It is validated at pulse.New with the same rules
	// as the built-in tier (NotFor.Use resolves against built-ins, the
	// other registered extensions and the feature table); a broken rule
	// fails with PULSE_EXTENSION_PURPOSE_INVALID. Only its intent IDs
	// reach the manifest entry's intents list; nil declares no intents.
	Purpose *descriptor.Purpose
	// DependsOn names the features this operator needs, each an AND
	// edge (a single-name group): a feature profile that enables the
	// operator must enable every named feature too, or pulse.New fails
	// with PULSE_FEATURE_PROFILE_DEPENDENCY. Every entry must name a
	// feature this build knows — a built-in feature or another
	// registered extension operator — whether or not a profile is set;
	// otherwise pulse.New fails with PULSE_FEATURE_PROFILE_UNKNOWN.
	// Optional; nil adds no dependency. Extensions carry no Since.
	DependsOn  []string
	Factory    extend.AggregatorFactory
	Streamable bool
	// Mergeable declares that partial states fold via
	// extend.MergeableAggregator.Merge, admitting the operator to the
	// parallel reducers and ProcessChain. Requires Streamable.
	Mergeable bool
	// MarginReducibility declares the crosstab margin class; a fusable
	// class admits the operator as a fused crosstab cell. Requires
	// Mergeable unless empty or types.MarginRecompute.
	MarginReducibility types.MarginReducibility
	// WeightAware declares that the operator computes a weighted
	// figure: on a slot where a row weight is in force (slot weight,
	// request weight or pulse.Options.DefaultWeight) the factory sees
	// the resolved weight on spec.Weight and every row's VALID weight
	// is readable through extend.Record.Weight() — rows with an invalid
	// weight are excluded before they reach the operator, and the
	// orchestrator emits the weighted floor keys (sum_weights, n_eff,
	// n_weight_invalid). Probe-validation constructs the factory both
	// with and without a weight. Undeclared, an explicit (slot or
	// request) weight on the slot is PULSE_EXTENSION_NOT_WEIGHT_AWARE
	// and the instance default is skipped. Projected as the manifest's
	// weight_aware.
	WeightAware bool
	Accepts     []encoding.FieldType
	Params      []ParamMeta
	// FieldInputs is the optional buffered-projection introspection
	// hook. See FieldInputsFunc. Omit to keep the operator opaque to
	// projection (runtime widens the field set when this operator
	// appears in a request).
	FieldInputs FieldInputsFunc
	// ComponentSchema declares the per-operator components contract
	// surfaced through Response.Components.Aggregations[i].Operator.
	// Leaving the schema empty (zero-value Keys plus
	// empty Mergeability) makes the operator floor-only — the
	// orchestrator emits the universal floor ({"n", "n_null"}) and no
	// extra keys. When ComponentSchema.Keys is non-empty, the
	// ComponentsFunc emitter MUST be supplied and its returned map
	// keys MUST be a subset of the declared key set; probe-validation
	// enforces both at pulse.New time.
	ComponentSchema descriptor.ComponentSchema
	// ComponentsFunc is the optional sibling-interface emitter. When
	// set, the orchestrator routes the returned map onto
	// Response.Components.Aggregations[i].Operator after the
	// aggregator's Aggregate / Finalize call terminates. Nil is the
	// floor-only path (universal floor fills the entire payload).
	ComponentsFunc AggregatorComponentsFunc
}

AggregatorRegistration installs a custom AGG_* operator. The factory is an extend.AggregatorFactory: it builds a fresh extend.Aggregator per Process call against the supplied Aggregation spec + schema.

When Streamable=true the factory MUST return a value that also implements extend.OnlineAggregator. Probe-validation at registration time enforces the contract via PULSE_EXTENSION_STREAMABLE_MISMATCH. The declaration decides the run-time path: Streamable=false runs buffered even when the value implements extend.OnlineAggregator. Optional siblings (extend.OnlineAggregator, extend.RichAggregator, extend.MergeableAggregator) are honoured whether or not ComponentsFunc is set.

When Mergeable=true (which requires Streamable=true) the factory MUST return an extend.MergeableAggregator and any ComponentSchema keys must be classified "mergeable" or "partial"; probe-validation refuses anything else with PULSE_EXTENSION_MERGEABLE_MISMATCH. The declaration decides whether ShardWorkers / DecodeWorkers fan the request out and whether a ProcessChain stage accepts it; omitted, the request runs serially.

MarginReducibility declares the operator's crosstab margin class — the embedder-side sibling of types.AggregationType.MarginReducibility. A fusable class (types.MarginSummable, types.MarginMeanReducible, types.MarginIndependent) admits the operator as a crosstab CELL to the fused in-decode arm and requires Mergeable=true (and so Streamable=true); probe-validation refuses an unknown class or a fusable class without Mergeable with PULSE_EXTENSION_MARGIN_REDUCIBILITY_MISMATCH. The fused walk feeds each margin its own accumulator record by record, so the class gates admission only and never changes a margin figure. Omitted (or types.MarginRecompute) keeps the crosstab cell on the buffered arm.

type AttributeEmitType added in v0.7.0 

type AttributeEmitType string

AttributeEmitType is a manifest hint declaring the dtype the attribute emits per record. Defaults to "float64" when empty. 

const (
	AttributeEmitFloat64 AttributeEmitType = "float64"
	AttributeEmitBool    AttributeEmitType = "bool"
	AttributeEmitString  AttributeEmitType = "string"
)

type AttributeMode added in v0.7.0 

type AttributeMode string

AttributeMode declares which streaming tier an attribute factory promises to satisfy. 

const (
	// AttributeModeRowLocal — factory returns an extend.RowLocalAttribute.
	// Equivalent to built-in ATTR_FORMULA / ATTR_DATE_PART.
	AttributeModeRowLocal AttributeMode = "row_local"
	// AttributeModeTwoPass — factory returns an extend.TwoPassAttribute.
	// Equivalent to ATTR_ZSCORE / ATTR_TSCORE / ATTR_NORMALIZED.
	AttributeModeTwoPass AttributeMode = "two_pass"
	// AttributeModeBuffered — factory returns a plain
	// extend.AttributeComputer; no streaming. Equivalent to
	// ATTR_PERCENTILE.
	AttributeModeBuffered AttributeMode = "buffered"
)

type AttributeRegistration added in v0.7.0 

type AttributeRegistration struct {
	Name        types.AttributeType
	Description string
	// Purpose is the optional plain-language guidance for the
	// operator — what it is for, the intents it answers, when to use
	// something else. It is validated at pulse.New with the same rules
	// as the built-in tier (NotFor.Use resolves against built-ins, the
	// other registered extensions and the feature table); a broken rule
	// fails with PULSE_EXTENSION_PURPOSE_INVALID. Only its intent IDs
	// reach the manifest entry's intents list; nil declares no intents.
	Purpose *descriptor.Purpose
	// DependsOn names the features this operator needs, each an AND
	// edge (a single-name group): a feature profile that enables the
	// operator must enable every named feature too, or pulse.New fails
	// with PULSE_FEATURE_PROFILE_DEPENDENCY. Every entry must name a
	// feature this build knows — a built-in feature or another
	// registered extension operator — whether or not a profile is set;
	// otherwise pulse.New fails with PULSE_FEATURE_PROFILE_UNKNOWN.
	// Optional; nil adds no dependency. Extensions carry no Since.
	DependsOn []string
	Factory   extend.AttributeFactory
	Mode      AttributeMode
	Accepts   []encoding.FieldType
	Emits     AttributeEmitType
	Params    []ParamMeta
	// WeightAware declares that the attribute reads the row weight:
	// on a slot where a weight is in force the factory sees it on
	// spec.Weight and extend.Record.Weight() reports each row's valid
	// weight (false on a row whose weight is invalid — the attribute
	// still owes that row a value). Undeclared, ANY weight in force on
	// the slot — the instance default included — is
	// PULSE_EXTENSION_NOT_WEIGHT_AWARE (a non-aware attribute may
	// depend on the whole population, whose weighted form it cannot
	// give); `weight: null` on the slot opts out. Probe-validation
	// constructs the factory both with and without a weight.
	WeightAware bool
	// FieldInputs is the optional buffered-projection introspection
	// hook. See FieldInputsFunc.
	FieldInputs FieldInputsFunc
}

AttributeRegistration installs a custom ATTR_* operator. The factory is an extend.AttributeFactory; Mode declares which extend sibling the returned value implements, and probe-validation at pulse.New refuses a mismatch with PULSE_EXTENSION_STREAMABLE_MISMATCH.

type BuildIndexResult added in v0.27.0 

type BuildIndexResult = service.BuildIndexResult

BuildIndexResult re-exports internal/service.BuildIndexResult — the outcome of a successful point-lookup sidecar index build: the derived sidecar path plus the in-memory SidecarIndex that was serialized there.

type ChainOverlaySchemaDivergence 

type ChainOverlaySchemaDivergence = descx.ChainOverlaySchemaDivergence

ChainOverlaySchemaDivergence is one chain-overlay (ref, target) pair ChainPredictResult rejects.

type ChainPredictResult 

type ChainPredictResult = descx.ChainValidationResult

ChainPredictResult is PredictChain's envelope Data: the verdict, the source schema and each stage's inferred output columns.

type ChainRequest added in v0.10.0 

type ChainRequest = types.ChainRequest

ChainRequest re-exports types.ChainRequest so callers can use pulse.ChainRequest as the input to ProcessChain.

type ChainResponse added in v0.10.0 

type ChainResponse = types.ChainResponse

ChainResponse re-exports types.ChainResponse.

type ChainStage added in v0.10.0 

type ChainStage = types.ChainStage

ChainStage re-exports types.ChainStage.

type ChangeEvent added in v0.11.0 

type ChangeEvent struct {
	Path      string
	Kind      ChangeKind
	Hash      string
	Timestamp time.Time
}

ChangeEvent is one observed mutation against a watched .pulse path (or any file, when WatchDir is rooted at a non-.pulse directory).

type ChangeKind added in v0.11.0 

type ChangeKind int

ChangeKind identifies the kind of change a Watch / WatchDir emits. 

const (
	// ChangeCreated reports the appearance of a new file in the
	// watched scope. Hash is populated when the file is readable.
	ChangeCreated ChangeKind = iota
	// ChangeModified reports an in-place content change on an
	// already-tracked path. Hash carries the post-change content
	// hash.
	ChangeModified
	// ChangeRemoved reports that a previously-tracked path is gone.
	// Hash is empty.
	ChangeRemoved
	// ChangeRenamed reports an atomic write that promoted a temp
	// file to the watched path. Coalesces a Removed-or-Created
	// pair into a single Renamed event aimed at the destination.
	ChangeRenamed
)

func (ChangeKind) String added in v0.11.0 

func (k ChangeKind) String() string

String returns a stable lowercase identifier for the kind.

type CohesionWarning 

type CohesionWarning = encx.CohesionWarning

CohesionWarning is one non-fatal shard-archive diagnostic (for example PULSE_SHARD_DESCRIPTION_DIVERGENCE or PULSE_SHARD_SET_WIDENED), carried by AddShardResult.Warnings, CreateShardArchiveResult.Warnings and VerifyResult.Warnings. Code is the coded-error code, Details its structured payload.

type Cohort 

type Cohort struct {
	// contains filtered or unexported fields
}

Cohort represents an opened .pulse file with its parsed schema. It wraps the service-layer Cohort to provide a clean public API.

func (*Cohort) Categorical 

func (c *Cohort) Categorical(name string) (*encoding.Dictionary, bool)

Categorical returns the dictionary for a named categorical field. Returns nil, false if the field is not found or is not categorical.

func (*Cohort) Field 

func (c *Cohort) Field(name string) *encoding.Field

Field returns a pointer to the named field, or nil if not found.

func (*Cohort) Reader 

func (c *Cohort) Reader() (*CohortReader, error)

Reader opens an index-addressed record reader over the cohort. It supports single-file cohorts, ungrouped (format 0x01) and grouped (0x02) alike — a grouped cohort reads identically to its ungrouped twin — whole shard archives and anchored shards (`archive.pulse#shard.pulse`).

Over an archive, index i is global: shards are concatenated in archive order (Cohort.Shards), Len is the sum of their record counts, and every record decodes against the archive's canonical schema. An anchored shard addresses that one shard, also against the canonical schema, so its rows equal the whole-archive reader's rows for that shard; Reader().Schema() can therefore carry longer dictionaries than the anchored Cohort.Schema() (a stored shard's own header dictionaries are a prefix of the canonical ones). A shard whose header is not decode-compatible with the canonical schema is refused with the code `shard verify` reports (PULSE_SHARD_SCHEMA_MISMATCH / PULSE_SHARD_DICT_DIVERGENCE). The caller must Close the returned reader.

func (*Cohort) RecordCount added in v0.8.3 

func (c *Cohort) RecordCount() (int64, error)

RecordCount returns the number of records in the cohort. For single-file cohorts this is derived from the byte length of the record region divided by the per-record size implied by the schema. For archive-backed cohorts the caller should sum per-shard RecordCount values from Shards() — the underlying service Cohort errors on RecordCount for archives because the byte-region path doesn't apply across shards.

func (*Cohort) Schema 

func (c *Cohort) Schema() *encoding.Schema

Schema returns the cohort's schema.

func (*Cohort) Shards added in v0.8.0 

func (c *Cohort) Shards() []ShardEntry

Shards returns the shard manifest for an archive-backed cohort. Empty for single-file cohorts. The returned slice is a defensive copy.

type CohortBuildResult 

type CohortBuildResult struct {
	// Target is the cohort path written, as given to NewCohortBuilder
	// (the anchored `archive.pulse#name.pulse` for an append).
	Target string `json:"target"`
	// Records is the number of rows written: the accepted Append calls.
	Records int64 `json:"records"`
	// FormatVersion is the cohort header's format version byte
	// (encoding.FormatVersionV1 for an ungrouped cohort). It is derived
	// from the schema's content, never chosen.
	FormatVersion byte `json:"cohort_format_version"`
	// Schema is the schema written: the caller's fields with the layout
	// recomputed and the final dictionaries (pre-seeded entries first,
	// then labels in first-seen order). For an anchored append it (and
	// FormatVersion) is the shard as built, before AddShard reconciled
	// it to the archive; read the archive's schema back with Open.
	Schema *encoding.Schema `json:"-"`
	// Warnings are the non-fatal coded findings, in order: one
	// PULSE_FIELD_DESCRIPTION_LOW_QUALITY per field with a weak
	// description, the group gate's PULSE_GROUP_TOO_NARROW then
	// PULSE_DEDUP_LOW_RATIO findings (each fatal under Strict instead),
	// then, for a sharded build, CreateShardArchive's warnings — for an
	// anchored append, AddShard's — unchanged (PULSE_SHARD_*, with their
	// details: PULSE_SHARD_SET_WIDENED when a set field outgrew the
	// archive's rung, PULSE_SHARD_GROUPS_REWRITTEN when the shard was
	// re-encoded to the archive's group layout or a constant group was
	// promoted), then a failure to enumerate
	// the sidecars an overwrite invalidated.
	Warnings []*errors.CodedError `json:"warnings,omitempty"`
	// Groups describes every declared group in declaration order — its
	// verdict and the figures behind it — as an import report does.
	// Empty when no group was declared.
	Groups []pio.GroupReport `json:"groups,omitempty"`
	// ElidedConstants names the fields ElideConstants stored once, in
	// schema order. Empty when nothing was elided.
	ElidedConstants []string `json:"elided_constants,omitempty"`
	// Shards names a sharded build's (ShardSplit) archive entries, in
	// archive order, or the one entry an anchored append added. Empty
	// for a single-file build.
	Shards []string `json:"shards,omitempty"`
	// InvalidatedSidecars names each sidecar beside the target that an
	// Overwrite of an existing cohort invalidated — the point-lookup
	// index and its manifest, the SPSS metadata sidecar — with the
	// command that rebuilds it, exactly as Pulse.Dedup reports them.
	// Nothing is rebuilt: each sidecar self-invalidates through its own
	// fingerprint. Empty when no cohort was replaced.
	InvalidatedSidecars []StaleSidecar `json:"invalidated_sidecars,omitempty"`
}

CohortBuildResult is what a successful CohortBuilder.Close wrote.

type CohortBuilder 

type CohortBuilder struct {
	// contains filtered or unexported fields
}

CohortBuilder writes a new cohort from rows appended one at a time. Obtain one from Pulse.NewCohortBuilder and finish it with Close (or discard it with Abort). It is not safe for concurrent use.

func (*CohortBuilder) Abort 

func (b *CohortBuilder) Abort() error

Abort discards the build: the spool is removed and the target is left untouched. It is idempotent and a no-op after Close.

func (*CohortBuilder) Append 

func (b *CohortBuilder) Append(row CohortRow) error

Append adds one row: one value per schema field, in field order, under the CohortRow type mapping (a row read by CohortReader.RecordAt is accepted unchanged). No coercion is applied.

A row that does not fit is rejected with PULSE_IMPORT_ROW_ERROR whose details carry row (the 1-based Append call), field and reason — "arity" (wrong value count), "type" (wrong Go type; details also carry expected and got), "null" (nil in a non-nullable field) or "overflow" (an integer past its type, a decimal past its precision, a new label past the dictionary rung). A rejected row leaves no trace — nothing is written and no dictionary entry it introduced is kept — and the builder stays usable. A filesystem failure while spooling is returned by every later call.

func (*CohortBuilder) Close 

func (b *CohortBuilder) Close() (*CohortBuildResult, error)

Close writes the cohort atomically — with Groups or ElideConstants it first makes the full pass (constant detection, group encoding into a second spool, the ratio gate) — then the schema block and every spooled row, into a temp file beside the target that is fsynced and renamed over it — and returns what was written. With Shards it instead cuts the rows into shard files staged in a directory beside the target and publishes them with one CreateShardArchive call (its errors are Close's, unchanged); the staging directory is removed whatever the outcome. For an anchored target it writes the rows as one shard file staged beside the archive and calls AddShard once, which rewrites the archive atomically: its errors — cohesion (PULSE_SHARD_SCHEMA_MISMATCH, PULSE_SHARD_DICT_WIDTH_OVERFLOW, …) and a name added meanwhile (PULSE_SHARD_NAME_COLLISION) — are Close's, unchanged, and on failure the archive is left as it was. On any failure nothing is left at the target (an overwritten cohort stays as it was) and the spool is removed. The builder is finished either way: a second Close returns SERVICE_RESOURCE.

type CohortBuilderOptions 

type CohortBuilderOptions struct {
	// Strict turns every PULSE_FIELD_DESCRIPTION_LOW_QUALITY finding
	// into a fatal error of the same code, raised by NewCohortBuilder
	// before anything is written (as `--strict` does for import). It
	// makes the group viability gate strict too: PULSE_GROUP_TOO_NARROW
	// fails NewCohortBuilder, PULSE_DEDUP_LOW_RATIO fails Close, and
	// neither leaves anything on disk.
	Strict bool
	// Overwrite lets the build replace an existing cohort at the
	// target. Without it an existing target is refused with
	// SERVICE_VALIDATION, at NewCohortBuilder and again at Close. When
	// the replaced cohort is a managed import, its expiry slides by its
	// TTL (a pinned import stays pinned), as an in-place Pulse.Dedup
	// does.
	Overwrite bool
	// Groups declares parent groups (key + the members it determines),
	// exactly as `pulse import --group` / pio.ImportJob.Groups do: the
	// same PULSE_GROUP_* declaration checks (at NewCohortBuilder), the
	// same viability gate — a group no wider than its index is dropped
	// with PULSE_GROUP_TOO_NARROW and its members stay row fields; a
	// group below RatioFloor rows per tuple is written with
	// PULSE_DEDUP_LOW_RATIO — and the same encoder. A key violation
	// (PULSE_GROUP_MEMBER_NOT_CONSTANT) surfaces at Close, whose
	// details carry source_row: the 1-based Append call. The format
	// version follows: 0x02 iff a group is written.
	Groups []pio.GroupDecl
	// ElideConstants stores every field holding one value on every row
	// once, as `pulse import --elide-constants` does, decided by a full
	// pass over the appended rows at Close.
	ElideConstants bool
	// RatioFloor is the rows-per-distinct-tuple floor below which a
	// declared group draws PULSE_DEDUP_LOW_RATIO (import's
	// --dedup-ratio-floor); 0 selects the default (2).
	RatioFloor float64
	// Shards, when set, makes Close publish a shard archive at the
	// target instead of a single-file cohort. See ShardSplit.
	Shards *ShardSplit
}

CohortBuilderOptions configures Pulse.NewCohortBuilder.

type CohortFingerprint 

type CohortFingerprint = encx.Fingerprint

CohortFingerprint is the SHA-256 content fingerprint of a cohort's bytes, as recorded in SidecarIndex.Fingerprint at build time and recomputed by VerifyIndex to decide freshness.

type CohortReader 

type CohortReader struct {
	// contains filtered or unexported fields
}

CohortReader reads an opened cohort record by record, by index. Obtain one from Cohort.Reader and release it with Close.

RecordAt is safe for concurrent use from multiple goroutines; every returned row is freshly allocated and owned by the caller.

func (*CohortReader) Close 

func (r *CohortReader) Close() error

Close releases the reader's file handle, waiting for in-flight RecordAt calls. It is idempotent.

func (*CohortReader) Len 

func (r *CohortReader) Len() int64

Len returns the number of records the reader addresses — for a shard archive, the sum over its shards (unlike Cohort.RecordCount, which counts single-file cohorts only).

func (*CohortReader) RecordAt 

func (r *CohortReader) RecordAt(i int64) (CohortRow, error)

RecordAt returns record i (0-based) as exact typed values. An index outside [0, Len()) returns a SERVICE_VALIDATION coded error and a call after Close a SERVICE_RESOURCE coded error.

func (*CohortReader) Schema 

func (r *CohortReader) Schema() *encoding.Schema

Schema returns the schema the reader decodes against; CohortRow slot i holds Schema().Fields[i].

type CohortRow 

type CohortRow []any

CohortRow is one cohort record as exact typed values, one slot per field in schema LOGICAL order (index it by Schema().Fields). It is the row type of CohortReader and is distinct from Record, the name-keyed map Sample returns.

Per-type Go values (the EXACT stored value, never a float64 echo): 

u4, u8, u16, u32, u64        uint64
f32 / f64                    float32 / float64
date                         int32 epoch days
datetime                     int64 epoch seconds
decimal128                   encoding.Decimal128
categorical_*                string (the dictionary label)
set_* (every rung)           []string labels in dictionary order;
                             an empty non-nil slice is "no selection"
packed_bool                  bool
null (any type)              nil

type ComposeOptions added in v0.2.0 

type ComposeOptions = service.ComposeOptions

ComposeOptions controls parallel execution. See internal/service.ComposeOptions.

type ComposePredictResult 

type ComposePredictResult = descx.ComposeValidationResult

ComposePredictResult is PredictCompose's envelope Data: the batch verdict, the rejected overlay pairs, the per-spec overlay cost, the Compose-host p-value count and every slot's advisories.

type ComposedRequest 

type ComposedRequest = types.ComposedRequest

Type aliases re-exported from the types package so embedders can use pulse.Request instead of types.Request.

type ComposedResponse added in v0.22.0 

type ComposedResponse = types.ComposedResponse

Type aliases re-exported from the types package so embedders can use pulse.Request instead of types.Request.

type CreateShardArchiveResult added in v0.38.0 

type CreateShardArchiveResult = service.CreateShardArchiveResult

CreateShardArchiveResult carries the outcome of CreateShardArchive: the archive's shard count, any set-field widenings the seed forced, and the non-fatal warnings (PULSE_SHARD_DESCRIPTION_DIVERGENCE, PULSE_SHARD_SET_WIDENED) to lift onto a --json envelope.

type DateRangeSpec added in v0.27.0 

type DateRangeSpec struct {
	Label string  `json:"label"`
	Start *string `json:"start,omitempty"`
	End   *string `json:"end,omitempty"`
}

DateRangeSpec is the wire-level shape of a single labeled date range ({label, start, end}). It is the shared model authored inline on a GROUP_DATE_RANGES grouper or FILTER_DATE_RANGES filter and registered in an Extensions.RangeTables entry. Start / End are ISO date literals; nil / empty is an open bound; both bounds are inclusive.

type DedupOptions added in v0.39.0 

type DedupOptions struct {
	// Groups declares the parent groups (key + the members it
	// determines). A cohort that is already grouped is regrouped from
	// scratch: its existing groups are replaced, not extended.
	Groups []pio.GroupDecl
	// Out writes the converted cohort to a NEW path instead of
	// rewriting the source in place. It must not exist. Empty (the
	// default) rewrites in place.
	Out string
	// ElideConstants also stores every field holding a single value on
	// every record once (as `pulse import --elide-constants`).
	ElideConstants bool
	// RatioFloor is the rows-per-distinct-tuple floor below which a
	// group draws PULSE_DEDUP_LOW_RATIO; 0 selects
	// internal/encoding.DefaultDedupRatioFloor (2).
	RatioFloor float64
	// Strict turns the gate's warnings (PULSE_GROUP_TOO_NARROW,
	// PULSE_DEDUP_LOW_RATIO) into a fatal error of the same code, raised
	// before anything is written.
	Strict bool
	// SuggestGroups detects candidate parent groups over the cohort's
	// records and returns them on DedupResult.GroupCandidates, each with
	// a ready-to-use declaration. Alone (no Groups, no ElideConstants)
	// the call is read-only.
	SuggestGroups bool
}

DedupOptions configures Pulse.Dedup. It carries the same parent-group declaration import takes (pio.GroupDecl) and the same viability-gate policy, so a declaration that works on `pulse import --group` works unchanged on an existing cohort.

type DedupResult added in v0.39.0 

type DedupResult struct {
	pio.DedupReport
	// InvalidatedSidecars names each sidecar beside the cohort that an
	// IN-PLACE rewrite invalidated — the point-lookup index and its
	// manifest, the SPSS metadata sidecar — with the command that
	// rebuilds it. Nothing is rebuilt: a dedup changes the cohort's
	// length, so each sidecar self-invalidates through its own size
	// fingerprint, and rebuilding is the caller's to schedule. Empty
	// for an --out rewrite (the source's sidecars still describe the
	// untouched source) and when nothing was written.
	InvalidatedSidecars []StaleSidecar `json:"invalidated_sidecars,omitempty"`
	// SidecarWarnings carries a failure to ENUMERATE those sidecars.
	// The rewrite has already succeeded when this is set, so it is a
	// warning, never an error: any sidecar beside the cohort must be
	// assumed stale.
	SidecarWarnings []*errors.CodedError `json:"sidecar_warnings,omitempty"`
}

DedupResult is Pulse.Dedup's report: the rewrite's figures (pio.DedupReport, inlined in JSON) plus the sidecars the rewrite invalidated.

func (*DedupResult) Warnings added in v0.39.0 

func (r *DedupResult) Warnings() []*errors.CodedError

Warnings returns every non-fatal coded finding of the run in report order: the viability gate's, then the sidecar scan's.

type DistributionRegistration added in v0.7.0 

type DistributionRegistration struct {
	Name        string
	Description string
	// Purpose is the optional plain-language guidance for the
	// operator — what it is for, the intents it answers, when to use
	// something else. It is validated at pulse.New with the same rules
	// as the built-in tier (NotFor.Use resolves against built-ins, the
	// other registered extensions and the feature table); a broken rule
	// fails with PULSE_EXTENSION_PURPOSE_INVALID. Only its intent IDs
	// reach the manifest entry's intents list; nil declares no intents.
	Purpose *descriptor.Purpose
	// Factory is the sampler-construction callback. Its concrete
	// signature is finalised when the synth distribution overlay
	// lands; until then this field is held opaque to keep the API
	// shape stable.
	Factory any
	Params  []ParamMeta
}

DistributionRegistration installs a custom synthetic-data distribution kind. The factory shape is finalised alongside the synth wiring in a later phase; for now this registration is validated for naming + duplicates only and the Factory field is held opaque.

type ErrorFixup added in v0.5.0 

type ErrorFixup = errors.Fixup

ErrorFixup is one repair template attached to an error code.

type ErrorMetadata added in v0.5.0 

type ErrorMetadata = errors.LookupResult

ErrorMetadata is the depth-on-demand projection returned by ErrorLookup, ErrorsByDomain, and ErrorsSearch. Carries the code, domain, canonical Message, and materialised Fixup templates.

type Example added in v0.5.0 

type Example = examples.Example

Example is the full record returned by ExampleGet — runnable request JSON plus the indexed metadata.

type ExampleSummary added in v0.5.0 

type ExampleSummary = examples.ExampleSummary

ExampleSummary is the lightweight projection returned by ExamplesSearch.

type ExamplesQuery 

type ExamplesQuery = examples.Query

ExamplesQuery is ExamplesSearchWith's filter set — query, tags, category and intent, every one optional and all ANDed.

type ExportReferenceOptions 

type ExportReferenceOptions struct {
	// OmitSkills leaves out skills.md and the skills/ tree; no exported
	// page then links to a skill. The zero value includes the skills.
	OmitSkills bool
}

ExportReferenceOptions tunes one ExportReference call. The zero value exports the full reference: catalog, reading pages, glossary and the skill pack.

type ExprFunction added in v0.7.0 

type ExprFunction struct {
	Name        string
	Description string
	// Signature is a human-readable signature surfaced in the
	// manifest and in predict error messages when typecheck fails
	// (e.g. `rank_familiarity(value float64, total_pop bool) float64`).
	Signature string
	// Fn is the Go function. expr-lang accepts any func value;
	// the runtime resolves argument types against the call site.
	Fn any
	// Pure declares the function is side-effect-free and depends
	// only on its arguments. Reserved for a future memoization
	// optimisation; declaring it has no runtime effect today.
	Pure bool
}

ExprFunction describes a function callable from runtime expressions (ATTR_FORMULA / FILTER_EXPRESSION). At pulse.New time the Fn value is registered with the expr-lang engine under Name.

type Extensions added in v0.7.0 

type Extensions struct {
	Aggregators        []AggregatorRegistration
	Attributes         []AttributeRegistration
	Filterers          []FiltererRegistration
	Groupers           []GrouperRegistration
	Windows            []WindowRegistration
	Features           []FeatureRegistration
	Tests              []TestRegistration
	SynthDistributions []DistributionRegistration

	// ExprFunctions are merged into the runtime expression environment
	// used by ATTR_FORMULA and FILTER_EXPRESSION (plus any future
	// expression hook). Each entry is callable from request expressions
	// under its declared Name.
	ExprFunctions []ExprFunction

	// LookupTables expose static keyed maps to the runtime expression
	// environment. Callers reference them as
	//   lookup("table_name", key1, key2, ...)
	// The built-in lookup() function is added to the expression env
	// whenever LookupTables is non-empty. Tables are read-only after
	// pulse.New.
	LookupTables map[string]LookupTable

	// LabelTables expose static ID→label maps used by the output-time
	// categorical-label overlay (per-request Labels slot on Request /
	// SampleRequest / FacetRequest / ExportJob). Each table accepts a
	// single string key (the categorical's resolved dictionary value)
	// and returns a display string. Tables are read-only after
	// pulse.New and do not affect filter / formula / sort semantics —
	// only end-user output rendering.
	LabelTables map[string]LabelTable

	// RangeTables expose named sets of labeled date ranges so a
	// GROUP_DATE_RANGES grouper or FILTER_DATE_RANGES filter can
	// reference an ordered {label, start, end} bucketing by name
	// instead of authoring it inline per request. Each table is
	// validated at pulse.New time via the shared range-compilation
	// pass (the same overlap / duplicate-label / empty / invalid-
	// boundary rules the inline path enforces); a table that nothing
	// references is valid and simply visible in the manifest. Tables
	// are read-only after pulse.New. Operator→table resolution is
	// wired separately; this slot only registers the tables.
	RangeTables map[string]RangeTable

	// Skills is an optional fs.FS of embedder skill files, served to
	// agents exactly like the built-in pack (p.Skills / p.Skill,
	// pulse_skills_list / pulse_skills_get, pulse-skill:// resources, the
	// manifest skills list, p.Ontology). Only top-level .md files, in
	// two shapes: an atomic op-<category>-<kebab>.md (kind: operator)
	// documenting ONE operator registered here — its stem is op- plus
	// the operator name lowercased with _ → - — and a topical
	// ext-<kebab>.md (kind: design, optional requires:). Every file is
	// validated HARD at pulse.New (frontmatter, name = stem, the
	// family's required ## sections, body budget, feature fences and
	// their coverage, ## See stems): a failure is
	// PULSE_EXTENSION_SKILL_INVALID, a stem a built-in skill or another
	// file already takes PULSE_EXTENSION_SKILL_COLLISION. A feature
	// profile hiding the operator hides its skill — it then reads as
	// nonexistent everywhere. Read once at pulse.New; nil adds none.
	Skills iofs.FS

	// Examples is an optional fs.FS of embedder request examples, served
	// to agents exactly like the built-in library (p.ExamplesSearch /
	// p.ExampleGet, pulse_examples_search / pulse_examples_get, the
	// manifest examples count / categories / tags, p.Ontology). Only
	// top-level .json files, each a request body plus a _meta block —
	// name, category, description, tags, operators, optional intents and
	// capabilities — exactly as in the built-in library. Every file is
	// validated HARD at pulse.New (the _meta keys and values, operators
	// registered on this instance and equal to the body's "type" values,
	// a strict decode of the body as its request root, every operator
	// it names joined to it by an ontology edge): a failure is
	// PULSE_EXTENSION_EXAMPLE_INVALID, a name a built-in example or
	// another file already takes PULSE_EXTENSION_EXAMPLE_COLLISION. A
	// feature profile hiding an operator or capability the example uses
	// hides it — it then reads as nonexistent everywhere. Read once at
	// pulse.New; nil adds none.
	Examples iofs.FS
}

Extensions bundles every per-category registration slot plus the expression-environment additions an embedder wants exposed to runtime ATTR_FORMULA and FILTER_EXPRESSION evaluation.

Zero value is the zero-extension case — pulse.New(Options{}) with no Extensions field behaves identically to a Pulse instance shipped without this surface. The registration types live in a separate namespace from the built-in registries (no collision risk) and the runtime treats them identically to built-ins.

type FacetDiscrete added in v0.7.0 

type FacetDiscrete = types.FacetDiscrete

FacetDiscrete is the per-value count list for discrete fields.

type FacetField added in v0.7.0 

type FacetField = types.FacetField

FacetField wraps either a discrete or numeric per-field summary.

type FacetHistogram added in v0.7.0 

type FacetHistogram = types.FacetHistogram

FacetHistogram is the fixed-width binning of a numeric field.

type FacetNumeric added in v0.7.0 

type FacetNumeric = types.FacetNumeric

FacetNumeric is the streaming-stats summary for numeric fields.

type FacetPredictResult 

type FacetPredictResult = descx.FacetValidationResult

FacetPredictResult is PredictFacet's envelope Data.

type FacetRequest added in v0.7.0 

type FacetRequest = types.FacetRequest

FacetRequest is the input to FacetSchema — multi-field, with optional filters, percentiles, histograms, and additive contribution counts.

type FacetResult added in v0.7.0 

type FacetResult = types.FacetResult

FacetResult is the response from FacetSchema.

type FacetValueCount added in v0.7.0 

type FacetValueCount = types.FacetValueCount

FacetValueCount is one (value, count) tuple inside FacetDiscrete.

type FeatureDescription 

type FeatureDescription struct {
	Name string `json:"name"`
	// Kind is operator, capability, io_format or mcp_extra.
	Kind string `json:"kind"`
	// Category is an operator's category (AGG, GROUP, OVERLAY …);
	// empty for other kinds.
	Category string `json:"category,omitempty"`
	// Source is builtin, extension or unknown.
	Source string `json:"source"`
	// Since is the release that introduced a built-in; empty otherwise.
	Since string `json:"since,omitempty"`
	// DependsOn is the dependency edges: an AND of any-of groups — for
	// every group at least one member must be enabled. Never null.
	DependsOn [][]string `json:"depends_on"`
	// Unknown explains a name the running Pulse does not resolve; nil
	// for a resolvable one.
	Unknown *FeatureProfileUnknownName `json:"unknown,omitempty"`
}

FeatureDescription describes one feature a profile lists.

type FeatureProfile 

type FeatureProfile struct {
	// Profile is a free-form label naming the profile, e.g.
	// "self-serve". Informational only.
	Profile string `json:"profile,omitempty"`

	// WrittenWith records the Pulse version the profile was authored
	// against. Informational only: it is never compared and an
	// unparseable value is not an error.
	WrittenWith string `json:"written_with,omitempty"`

	// Features lists the feature names the instance offers. It is
	// REQUIRED: a nil slice (or an absent / null "features" key in the
	// file) is refused with PULSE_FEATURE_PROFILE_INVALID. An empty,
	// non-nil slice is valid and offers no optional feature. A name may
	// appear at most once.
	Features []string `json:"features"`

	// Behaviour carries engine switches the profile turns on. Nil
	// leaves every switch to Options.
	Behaviour *FeatureProfileBehaviour `json:"behaviour,omitempty"`

	// Return is the instance `return` default the profile supplies: the
	// response selection a request without its own `return` block is
	// shaped by. Options.DefaultReturn wins over it; a request block
	// replaces either entirely. Nil leaves the library default `full`.
	// pulse.New (and CheckFeatureProfile) resolve it against the
	// profile's own feature set after the dependency class: a bad
	// preset, precision or path syntax, or a path the profile's
	// instance does not have — one a hidden feature owns included — is
	// PULSE_FEATURE_PROFILE_INVALID reason "invalid_return".
	Return *types.Return `json:"return,omitempty"`

	// Limits is the instance resource-limit layer the profile supplies.
	// Per field, a non-zero Options.Limits value wins over it and an
	// absent or 0 key falls back to the built-in default. Nil leaves
	// every limit to Options and the defaults. A value below -1 (or an
	// unparseable request_timeout) is PULSE_FEATURE_PROFILE_INVALID
	// reason "invalid_limits". Limits are not features: they never
	// enter the feature-set digest.
	Limits *FeatureProfileLimits `json:"limits,omitempty"`
}

FeatureProfile declares the set of features a Pulse instance offers. It mirrors the JSON profile file accepted through Options.FeatureProfileFile key for key, so a Go value and a file are interchangeable.

pulse.New validates the profile and stores it on the instance. The feature list is not applied yet — no surface is hidden or filtered by it — but the Behaviour switches take effect immediately.

The JSON form is decoded strictly: a key outside profile, written_with, features, behaviour, return and limits — or an unknown key inside one of those sections — is refused with PULSE_FEATURE_PROFILE_INVALID.

func ExampleFeatureProfile 

func ExampleFeatureProfile(name string) (*FeatureProfile, error)

ExampleFeatureProfile returns the published example feature profile called name, decoded with the same strict decode as ParseFeatureProfile. Each call returns a fresh value the caller owns.

Examples are frozen: each lists exact feature names as of the release in its WrittenWith, and Pulse never edits a published example in place. Every example passes pulse.New validation when handed over as Options.FeatureProfile.

An unknown name is refused with PULSE_FEATURE_PROFILE_INVALID, reason "unknown_example", naming the available examples under "examples".

func InitFeatureProfile 

func InitFeatureProfile(from string, ext ...Extensions) (*FeatureProfile, error)

InitFeatureProfile generates a feature profile for the running Pulse.

With from empty it lists every feature the running build offers — every built-in whose Since it has reached, plus the operator registrations of ext when given — by exact name. With from naming a published example (ExampleFeatureProfiles) it starts from that example's label, features and behaviour instead; extension names are not added to a seeded profile. Features are in canonical profile order (by kind, then operator category, then feature-table position).

WrittenWith is the running version's release core (`major.minor.patch`, pre-release and build suffixes cut — the comparison Since uses); a build with no release core (`devel`) stamps the newest release line its feature table knows.

ext is the value handed to pulse.New through Options.Extensions; several values are merged registration by registration. The result passes CheckFeatureProfile with the same ext, or InitFeatureProfile returns that check's coded error. An unknown example is PULSE_FEATURE_PROFILE_INVALID reason "unknown_example".

func ParseFeatureProfile 

func ParseFeatureProfile(data []byte) (*FeatureProfile, error)

ParseFeatureProfile strictly decodes the JSON form of a feature profile — the same decode, with the same refusals, that pulse.New applies to Options.FeatureProfileFile. Every refusal is PULSE_FEATURE_PROFILE_INVALID with a "reason" detail: malformed_json (including trailing data after the object), unknown_key, or missing_features (an absent or null "features" key).

It only decodes: feature names and dependencies are validated when the result is handed to pulse.New through Options.FeatureProfile.

It exists for entry points that read a profile from somewhere other than the instance filesystem. Options.FeatureProfileFile stays relative to the instance afero Fs (hermetic, and inside DataDir when DataDir is set); a caller holding a host OS path — the MCP server's --feature-profile flag and PULSE_FEATURE_PROFILE — reads the bytes itself and passes the parsed value through Options.FeatureProfile.

type FeatureProfileBehaviour 

type FeatureProfileBehaviour struct {
	// DisableDefaults ORs into Options.DisableDefaults.
	DisableDefaults bool `json:"disable_defaults,omitempty"`

	// DisableComponents ORs into Options.DisableComponents.
	DisableComponents bool `json:"disable_components,omitempty"`

	// DisableProjection ORs into Options.DisableProjection.
	DisableProjection bool `json:"disable_projection,omitempty"`

	// DisableCohortScan has no effect on the engine pulse.New builds:
	// the cohort resource scan belongs to the MCP server. It is stored
	// on the instance and ORed into the MCP adapter's own setting by
	// gosdk.Register (and so by mcpserve and `pulse mcp`): true skips
	// the startup scan even when Config.DisableCohortScan is false.
	DisableCohortScan bool `json:"disable_cohort_scan,omitempty"`
}

FeatureProfileBehaviour holds the engine switches a FeatureProfile can set. Each switch combines with the matching Options field by OR: true in the profile turns the switch on, and Options cannot force it back off. False (the default) defers to Options.

type FeatureProfileCheck 

type FeatureProfileCheck struct {
	// Profile and WrittenWith echo the checked profile.
	Profile     string `json:"profile,omitempty"`
	WrittenWith string `json:"written_with,omitempty"`

	// Version is the running Pulse version the profile was checked
	// against (pulse.Version()).
	Version string `json:"version"`

	// Valid is true when the profile passes every validation class.
	Valid bool `json:"valid"`

	// Features is the number of feature names the profile lists.
	Features int `json:"features"`

	// Unverified lists, sorted, the names an offline check accepted
	// without verifying them. Never null.
	Unverified []string `json:"unverified"`

	// Warnings carries one entry per unverified name, coded
	// PULSE_FEATURE_PROFILE_UNKNOWN with details {name, reason:
	// "unverified_extension"} — ready for an envelope's warnings. Never
	// null.
	Warnings []*descriptor.EnvelopeEntry `json:"warnings"`
}

FeatureProfileCheck is the report CheckFeatureProfile returns.

func CheckFeatureProfile 

func CheckFeatureProfile(fp *FeatureProfile, opts FeatureProfileCheckOptions) (*FeatureProfileCheck, error)

CheckFeatureProfile validates fp exactly as pulse.New does — the extension registrations in opts first, then the profile's classes in order: structural (PULSE_FEATURE_PROFILE_INVALID), name resolution (PULSE_FEATURE_PROFILE_UNKNOWN), dependencies (PULSE_FEATURE_PROFILE_DEPENDENCY), stopping at the first failing class. It returns the report (never nil) and that coded error, nil when the profile is valid. Offline relaxes only the name class (see FeatureProfileCheckOptions.Offline).

type FeatureProfileCheckOptions 

type FeatureProfileCheckOptions struct {
	// Extensions is the same value handed to pulse.New through
	// Options.Extensions. Its operator registrations are features a
	// profile may list; the registrations are validated exactly as
	// pulse.New validates them.
	Extensions Extensions

	// Offline checks a profile without the embedder's extensions in
	// hand (the `pulse features check` CLI). A listed name that is not
	// registered but follows the extension naming policy — a
	// non-SYNTH operator category, a non-reserved namespace — is
	// reported as a warning ("unverified extension name, check
	// in-process") instead of PULSE_FEATURE_PROFILE_UNKNOWN; every other
	// unknown name stays an error. For the dependency class such a name
	// counts as enabled and needs only a request host, like an extension
	// with no DependsOn.
	Offline bool
}

FeatureProfileCheckOptions configures CheckFeatureProfile.

type FeatureProfileDescription 

type FeatureProfileDescription struct {
	Profile     string `json:"profile,omitempty"`
	WrittenWith string `json:"written_with,omitempty"`
	Version     string `json:"version"`
	// Features describes every listed name, in canonical profile order.
	// Never null.
	Features []FeatureDescription `json:"features"`
}

FeatureProfileDescription is the report DescribeFeatureProfile returns.

func DescribeFeatureProfile 

func DescribeFeatureProfile(fp *FeatureProfile, ext ...Extensions) (*FeatureProfileDescription, error)

DescribeFeatureProfile describes every feature fp lists: kind, operator category, source, Since and dependency edges, in canonical profile order. A name the running Pulse does not resolve is described with what is known and an Unknown explanation; it is not an error. ext is the value handed to pulse.New through Options.Extensions. Only a nil profile or a nil Features list is an error (PULSE_FEATURE_PROFILE_INVALID).

type FeatureProfileDiff 

type FeatureProfileDiff struct {
	Profile     string `json:"profile,omitempty"`
	WrittenWith string `json:"written_with,omitempty"`
	Version     string `json:"version"`

	// Missing lists, in canonical profile order, every feature the
	// running Pulse offers (built-ins it has reached plus the given
	// extension operators) that the profile does not list. Never null.
	Missing []FeatureProfileMissing `json:"missing"`

	// Unknown lists, sorted by name, the profile's names the running
	// Pulse does not resolve. Reported, not fatal. Never null.
	Unknown []FeatureProfileUnknownName `json:"unknown"`
}

FeatureProfileDiff is the report DiffFeatureProfile returns.

func DiffFeatureProfile 

func DiffFeatureProfile(fp *FeatureProfile, ext ...Extensions) (*FeatureProfileDiff, error)

DiffFeatureProfile compares fp with the running Pulse: the features the running build offers that fp does not list (flagged New when introduced after fp's written_with), and the names fp lists that the running build does not resolve. Unknown names are reported, never fatal. ext is the value handed to pulse.New through Options.Extensions. Only a nil profile or a nil Features list is an error (PULSE_FEATURE_PROFILE_INVALID).

type FeatureProfileLimits 

type FeatureProfileLimits struct {
	// RequestTimeout is a Go duration string ("30s", "2m"). "" or "0"
	// means the default; "-1" or "unlimited" means no timeout. Any other
	// negative or unparseable value is refused.
	RequestTimeout string `json:"request_timeout,omitempty"`

	// MaxGroups is Options.Limits.MaxGroups' profile twin.
	MaxGroups int64 `json:"max_groups,omitempty"`

	// MaxCrosstabCells is Options.Limits.MaxCrosstabCells' profile twin.
	MaxCrosstabCells int64 `json:"max_crosstab_cells,omitempty"`

	// MaxEstimatedMemory is Options.Limits.MaxEstimatedMemory's profile
	// twin, in bytes.
	MaxEstimatedMemory int64 `json:"max_estimated_memory,omitempty"`

	// MaxMatrixDim is Options.Limits.MaxMatrixDim's profile twin.
	MaxMatrixDim int64 `json:"max_matrix_dim,omitempty"`

	// MaxComposeSlots is Options.Limits.MaxComposeSlots' profile twin.
	MaxComposeSlots int64 `json:"max_compose_slots,omitempty"`

	// MaxChainStages is Options.Limits.MaxChainStages' profile twin.
	MaxChainStages int64 `json:"max_chain_stages,omitempty"`

	// MaxJoinBuildRows is Options.Limits.MaxJoinBuildRows' profile twin.
	MaxJoinBuildRows int64 `json:"max_join_build_rows,omitempty"`
}

FeatureProfileLimits is a feature profile's `limits` section: the instance resource limits the profile supplies. Keys are the snake_case limit names (the spelling `pulse mcp --limit` and every PULSE_LIMIT_* error's `limit` detail use). The encoding matches Options.Limits: an absent or 0 key means "use the built-in default", -1 (pulse.Unlimited) means no limit, and any other negative value is refused with PULSE_FEATURE_PROFILE_INVALID reason "invalid_limits".

Per field, a non-zero Options.Limits value wins over the profile's, which wins over the built-in default. Limits are behaviour, not features: they never enter the feature-set digest.

type FeatureProfileMissing 

type FeatureProfileMissing struct {
	Name     string `json:"name"`
	Kind     string `json:"kind"`
	Category string `json:"category,omitempty"`
	// Since is the release that introduced a built-in feature; empty for
	// an extension operator.
	Since string `json:"since,omitempty"`
	// New is true when Since is later than the profile's written_with —
	// the feature did not exist when the profile was written. Always
	// false when written_with is absent or has no release core.
	New bool `json:"new"`
}

FeatureProfileMissing is a feature the running Pulse offers that a profile does not list.

type FeatureProfileUnknownName 

type FeatureProfileUnknownName struct {
	Name       string `json:"name"`
	Reason     string `json:"reason"`
	DidYouMean string `json:"did_you_mean,omitempty"`
	Since      string `json:"since,omitempty"`
}

FeatureProfileUnknownName explains why a listed name does not resolve against the running Pulse. Reason is one of the PULSE_FEATURE_PROFILE_UNKNOWN reasons: unregistered, pattern, wrong_kind (DidYouMean names the valid spelling), core_surface, newer_than_running (Since names the release that introduces it).

type FeatureRegistration added in v0.7.0 

type FeatureRegistration struct {
	Name        types.FeatureType
	Description string
	// Purpose is the optional plain-language guidance for the
	// operator — what it is for, the intents it answers, when to use
	// something else. It is validated at pulse.New with the same rules
	// as the built-in tier (NotFor.Use resolves against built-ins, the
	// other registered extensions and the feature table); a broken rule
	// fails with PULSE_EXTENSION_PURPOSE_INVALID. Only its intent IDs
	// reach the manifest entry's intents list; nil declares no intents.
	Purpose *descriptor.Purpose
	// DependsOn names the features this operator needs, each an AND
	// edge (a single-name group): a feature profile that enables the
	// operator must enable every named feature too, or pulse.New fails
	// with PULSE_FEATURE_PROFILE_DEPENDENCY. Every entry must name a
	// feature this build knows — a built-in feature or another
	// registered extension operator — whether or not a profile is set;
	// otherwise pulse.New fails with PULSE_FEATURE_PROFILE_UNKNOWN.
	// Optional; nil adds no dependency. Extensions carry no Since.
	DependsOn  []string
	Factory    extend.FeatureFactory
	Streamable bool
	Accepts    []encoding.FieldType
	Params     []ParamMeta
	// FieldInputs is the optional buffered-projection introspection
	// hook. See FieldInputsFunc.
	FieldInputs FieldInputsFunc
}

FeatureRegistration installs a custom FEAT_* operator. Set Streamable=true when the factory returns an extend.FeatureComputer that also implements extend.StreamingFeatureComputer. The operator only reads rows and returns extend.FeatureOutput columns; the engine writes them.

type FieldInputsFunc added in v0.8.0 

type FieldInputsFunc func(raw json.RawMessage) []string

FieldInputsFunc is the optional introspection callback an extension registration may supply so the buffered-projection extractor can determine which extra schema fields the operator reads beyond the spec's explicit Field/Field2/PartitionBy/etc. references. raw is the operator's Params block. Return value lists schema field names to include; nil/empty means "no extra fields."

Embedders that omit this hook leave their operator opaque to the projection extractor — the runtime then widens the buffered-decode field set to "every field" so the operator stays correct.

type FilterToFileRequest added in v0.11.0 

type FilterToFileRequest struct {
	// SourcePath identifies the cohort to filter. Single-file path,
	// shard archive, or `archive.pulse#shard.pulse` anchor — same
	// dispatch rules as Pulse.FilterToFile.
	SourcePath string `json:"source_path"`

	// Expression is the FILTER_EXPRESSION-style predicate evaluated
	// per row. Empty when Filterers is supplied instead.
	Expression string `json:"expression,omitempty"`

	// Filterers is the structured-predicate alternative. When set,
	// the request is translated into the equivalent expression and
	// passed to the existing FilterToFile engine. Mutually exclusive
	// with Expression.
	Filterers []*types.Filterer `json:"filterers,omitempty"`

	// OutputDir is the directory the result file lands in. Required.
	OutputDir string `json:"output_dir"`

	// OutputName is the optional override for the result basename.
	// When empty, defaults to `{source-hash}_{predicate-hash}.pulse`
	// so dedup is automatic. When set, the engine still validates the
	// caller-supplied path against the deterministic hash and refuses
	// to overwrite an existing file with mismatching content.
	OutputName string `json:"output_name,omitempty"`
}

FilterToFileRequest is the structured request shape for the deterministic filter-to-file operation. It deduplicates by `{source-hash}_{predicate-hash}.pulse` so two callers issuing the same filter against the same source land on the same output file and the engine skips redundant work.

Exactly one of Expression / Filterers must be set. Both empty is an error; both set is rejected so the predicate hash is unambiguous.

type FilterToFileResult added in v0.11.0 

type FilterToFileResult struct {
	OutputPath string `json:"output_path"`
	OutputHash string `json:"output_hash"`
	RowCount   int64  `json:"row_count"`
	ElapsedMs  int64  `json:"elapsed_ms"`
	// Reused is true when the engine returned an existing output file
	// without re-running the filter (content-hash match).
	Reused bool `json:"reused"`
}

FilterToFileResult is the structured response from FilterToFileWithRequest.

type FiltererComponentsFunc added in v0.20.0 

type FiltererComponentsFunc func(instance extend.FiltererBuilder) (map[string]any, error)

FiltererComponentsFunc is the filterer-shape sibling. In v1 every built-in filterer leaves its operator-specific payload empty (uniform floor of {n_in, n_out, n_null_input}); extensions may opt in by supplying a ComponentSchema + this emitter. The instance passed in is the extend.FiltererBuilder the registration's Factory returned.

type FiltererRegistration added in v0.7.0 

type FiltererRegistration struct {
	Name        types.FiltererType
	Description string
	// Purpose is the optional plain-language guidance for the
	// operator — what it is for, the intents it answers, when to use
	// something else. It is validated at pulse.New with the same rules
	// as the built-in tier (NotFor.Use resolves against built-ins, the
	// other registered extensions and the feature table); a broken rule
	// fails with PULSE_EXTENSION_PURPOSE_INVALID. Only its intent IDs
	// reach the manifest entry's intents list; nil declares no intents.
	Purpose *descriptor.Purpose
	// DependsOn names the features this operator needs, each an AND
	// edge (a single-name group): a feature profile that enables the
	// operator must enable every named feature too, or pulse.New fails
	// with PULSE_FEATURE_PROFILE_DEPENDENCY. Every entry must name a
	// feature this build knows — a built-in feature or another
	// registered extension operator — whether or not a profile is set;
	// otherwise pulse.New fails with PULSE_FEATURE_PROFILE_UNKNOWN.
	// Optional; nil adds no dependency. Extensions carry no Since.
	DependsOn []string
	Factory   extend.FiltererFactory
	Accepts   []encoding.FieldType
	Params    []ParamMeta
	// FieldInputs is the optional buffered-projection introspection
	// hook. See FieldInputsFunc. Filterers don't carry a Params
	// block today; the callback receives nil raw bytes and should
	// return the static set of extra fields the filterer reads
	// beyond Filterer.Field.
	FieldInputs FieldInputsFunc
	// ComponentSchema declares the per-operator components contract
	// surfaced through Response.Components.Filterers[i].Operator.
	// v1 ships every built-in filterer as floor-only
	// (uniform {n_in, n_out, n_null_input}); extensions may opt in by
	// declaring a non-empty ComponentSchema alongside a ComponentsFunc.
	// Mismatch between declared schema and emitted keys is rejected at
	// pulse.New time.
	ComponentSchema descriptor.ComponentSchema
	// ComponentsFunc is the optional sibling-interface emitter. When
	// set, the orchestrator routes the returned map onto
	// Response.Components.Filterers[i].Operator after the filter pass
	// terminates. Nil is the floor-only path (the orchestrator's
	// universal floor is the entire payload). Mirrors
	// the engine's per-operator Components() emission in shape.
	ComponentsFunc FiltererComponentsFunc
}

FiltererRegistration installs a custom FILTER_* operator. Filterers are always row-local streamable today — every built-in filterer evaluates per-row. The runtime records custom filterers as streamable; a buffered-filterer registration shape lands the day Pulse grows a buffered-filterer path.

type GroupIndexHeadroom 

type GroupIndexHeadroom = encx.GroupIndexHeadroom

GroupIndexHeadroom reports how much of one parent group's dictionary index space a grouped archive's canonical schema has consumed. VerifyShardArchive returns one per group in VerifyResult.GroupIndexHeadroom.

type GroupReconciliation added in v0.39.0 

type GroupReconciliation = service.GroupReconciliation

GroupReconciliation records one parent-group layout change a CreateShardArchive or AddShard made to fit a shard into the archive (reason incoming_flattened / incoming_regrouped / constant_promoted), with the shards and records the rewrite cost.

type GrouperComponentsFunc added in v0.20.0 

type GrouperComponentsFunc func(instance extend.Grouper) (map[string]any, error)

GrouperComponentsFunc is the grouper-shape sibling of AggregatorComponentsFunc. The orchestrator invokes the func ONCE after the grouper's terminal partition pass and routes the result onto Response.Components.Groupers[i].Operator. The universal floor ({total_n, n_null}) is filled unconditionally by the orchestrator. Returning (nil, nil) is the canonical signal for "no operator- specific keys". The instance passed in is the extend.Grouper the registration's Factory returned, so a type assertion to the embedder's concrete type succeeds.

type GrouperRegistration added in v0.7.0 

type GrouperRegistration struct {
	Name        types.GroupType
	Description string
	// Purpose is the optional plain-language guidance for the
	// operator — what it is for, the intents it answers, when to use
	// something else. It is validated at pulse.New with the same rules
	// as the built-in tier (NotFor.Use resolves against built-ins, the
	// other registered extensions and the feature table); a broken rule
	// fails with PULSE_EXTENSION_PURPOSE_INVALID. Only its intent IDs
	// reach the manifest entry's intents list; nil declares no intents.
	Purpose *descriptor.Purpose
	// DependsOn names the features this operator needs, each an AND
	// edge (a single-name group): a feature profile that enables the
	// operator must enable every named feature too, or pulse.New fails
	// with PULSE_FEATURE_PROFILE_DEPENDENCY. Every entry must name a
	// feature this build knows — a built-in feature or another
	// registered extension operator — whether or not a profile is set;
	// otherwise pulse.New fails with PULSE_FEATURE_PROFILE_UNKNOWN.
	// Optional; nil adds no dependency. Extensions carry no Since.
	DependsOn  []string
	Factory    extend.GrouperFactory
	Streamable bool
	// FansOut declares that a single record can land in MORE THAN ONE
	// bucket of this grouper — the embedder-registered sibling of
	// types.GroupType.FansOut(), which knows built-in constants only.
	//
	// The runtime expression of the same fact is the optional
	// extend.MultiKeyStreamingGrouper interface (KeysForRow). A
	// registration whose factory returns that interface MUST declare
	// FansOut=true, and one that declares FansOut=true MUST return it:
	// probe-validation at pulse.New() rejects either mismatch with
	// PULSE_EXTENSION_FANOUT_MISMATCH.
	//
	// The declaration exists because consumers that reason about
	// per-record denominators cannot assert a runtime interface —
	// under a fan-out grouper the bucket counts SUM to more than the
	// record total, so an n taken from a slab total double-counts
	// records. Omitting the field defaults it to false, so a multi-key
	// factory is refused rather than silently over-counting.
	FansOut bool
	// Mergeable declares that the grouper may run under the parallel
	// reducers (ShardWorkers / DecodeWorkers) and in ProcessChain
	// stages: one instance per partition, components state folded
	// through extend.MergeableGrouper.MergeState. Requires Streamable
	// (the reducers key rows one at a time). A Mergeable grouper that
	// emits components (ComponentsFunc, or a value with its own
	// Components() method) MUST return an extend.MergeableGrouper,
	// and its ComponentSchema must not classify the keys "none"; a
	// grouper that emits none needs no merge method. Probe-validation
	// refuses anything else with PULSE_EXTENSION_MERGEABLE_MISMATCH.
	// Omitted, a request naming the grouper runs serially.
	Mergeable bool
	Accepts   []encoding.FieldType
	Params    []ParamMeta
	// FieldInputs is the optional buffered-projection introspection
	// hook. See FieldInputsFunc.
	FieldInputs FieldInputsFunc
	// ComponentSchema declares the per-operator components contract
	// surfaced through Response.Components.Groupers[i].Operator.
	// Leaving the schema empty makes the operator
	// floor-only — the orchestrator emits the universal floor
	// ({total_n, n_null}) without operator-specific extras. When
	// ComponentSchema.Keys is non-empty, the ComponentsFunc emitter
	// MUST be supplied and its keys MUST match the declared schema.
	ComponentSchema descriptor.ComponentSchema
	// ComponentsFunc is the optional sibling-interface emitter. When
	// set, the orchestrator routes the returned map onto
	// Response.Components.Groupers[i].Operator after the grouper's
	// terminal partitioning pass. Nil is the floor-only path. Mirrors
	// the engine's per-operator Components() emission in shape.
	ComponentsFunc GrouperComponentsFunc
}

GrouperRegistration installs a custom GROUP_* operator. Set Streamable=true when the factory returns an extend.Grouper that also implements extend.StreamingGrouper (KeyForRow) or extend.MultiKeyStreamingGrouper (KeysForRow); the grouped Process request then streams. Probe-validation refuses Streamable=true without either sibling (PULSE_EXTENSION_STREAMABLE_MISMATCH); Streamable=false runs buffered through Group.

Set FansOut=true when the factory returns a value that also implements extend.MultiKeyStreamingGrouper — see the field comment. Probe-validation at registration time enforces the claim in BOTH directions via PULSE_EXTENSION_FANOUT_MISMATCH, mirroring the Streamable contract on AggregatorRegistration.

type ImportEntry added in v0.5.0 

type ImportEntry = imports.Entry

Type aliases re-exported from the imports package so embedders can use pulse.ImportSpec instead of internal/imports.Spec.

type ImportResult added in v0.5.0 

type ImportResult = imports.Result

Type aliases re-exported from the imports package so embedders can use pulse.ImportSpec instead of internal/imports.Spec.

type ImportSidecar 

type ImportSidecar = imports.Sidecar

ImportSidecar is the managed-import metadata document an ImportEntry carries.

type ImportSpec added in v0.5.0 

type ImportSpec = imports.Spec

Type aliases re-exported from the imports package so embedders can use pulse.ImportSpec instead of internal/imports.Spec.

type IndexFreshnessReason added in v0.27.0 

type IndexFreshnessReason = service.IndexFreshnessReason

IndexFreshnessReason re-exports internal/service.IndexFreshnessReason — why VerifyIndex reached its Fresh/stale verdict ("stat_mismatch"/"fingerprint_match"/"fingerprint_mismatch").

type IndexInfo added in v0.27.0 

type IndexInfo = service.IndexInfo

IndexInfo re-exports internal/service.IndexInfo — one entry in a Service.ListIndexes result: a sidecar's derived path, its ordered key column names, and its distinct-key / indexed-record summary.

type LabelBinding added in v0.10.1 

type LabelBinding = types.LabelBinding

LabelBinding pairs a categorical field with a label table for output-time translation. See types.LabelBinding for semantics.

type LabelMatch added in v0.11.2 

type LabelMatch struct {
	Key   string `json:"key"`
	Value string `json:"value"`
	// Score is the match confidence in [0,1]: 1.0 is an exact (or
	// exact-key) hit, ~0.9+ a prefix/near-typo, and lower values are
	// fuzzy matches. Callers should treat a high top score that is
	// clearly ahead of the runner-up as confident, and present options
	// to the user when the best score is low or several are close.
	Score float64 `json:"score"`
}

LabelMatch is one (key, value) pair from a label table, where key is the on-disk categorical dictionary value and value is the display label. ResolveLabel returns these ranked by Score.

type LabelMode added in v0.10.1 

type LabelMode = types.LabelMode

LabelMode selects replace vs augment rendering for a binding.

type LabelTable added in v0.10.1 

type LabelTable struct {
	Description string
	// Rows is the canonical static map. Key is the categorical's
	// resolved dictionary value (the on-disk string the field
	// presents, e.g. "US"); value is the display label (e.g.
	// "United States").
	Rows map[string]string
	// Lookup is the function-driven accessor. Returns the label and
	// true on hit; the second return is the miss signal. An error
	// from Lookup surfaces as PULSE_LABEL_LOOKUP_MISS with the
	// embedder's message attached.
	Lookup func(key string) (label string, ok bool, err error)
}

LabelTable exposes an ID→label map for display-time categorical translation. Exactly one of Rows / Lookup must be non-nil.

Rows is the simple path — a string→string map keyed by the resolved categorical dictionary value. Lookup is the escape hatch for tables that wrap an external store (e.g. a code-system service) or that compose the label from multiple sources.

LabelTable is intentionally distinct from LookupTable: LookupTable returns float64 for the numeric expression environment, LabelTable returns string for output rendering. Conflating them would force every embedder to choose one shape over the other; the two surfaces are independent.

type LabelTableInfo added in v0.11.2 

type LabelTableInfo struct {
	Name     string `json:"name"`
	RowCount int    `json:"row_count"`
	// Enumerable reports whether the table exposes a static Rows map
	// (true) or only a function-driven Lookup closure (false). Reverse
	// search via ResolveLabel works only on enumerable tables.
	Enumerable bool `json:"enumerable"`
}

LabelTableInfo is one registered label table as surfaced by LabelTables. RowCount and Enumerable describe whether the table can be reverse-searched (ResolveLabel requires a static Rows map).

type Limits 

type Limits = limits.Limits

Limits holds the instance resource limits set through Options.Limits. On input a zero field means "use the built-in default" and Unlimited (-1) means no limit; any other negative value fails New() with PULSE_LIMIT_INVALID. The effective limits (Pulse.Limits) never carry a zero: every field is a positive bound or Unlimited. A breach raises PULSE_LIMIT_EXCEEDED with details {limit, configured, observed, option}.

type LoadMemberSetResult added in v0.8.3 

type LoadMemberSetResult struct {
	Set             MemberSet
	Lines           int
	NotInDictionary int
	Invalid         int
}

LoadMemberSetResult is the per-load report returned by LoadMemberSetFromReader. Lines is the total of non-blank lines processed; NotInDictionary counts categorical values absent from the field's dictionary (they can never match a record, so they are dropped); Invalid counts numeric lines that failed to parse on the integer path. Callers decide whether to surface each counter as a warning or a hard error.

func LoadMemberSetFromReader added in v0.8.3 

func LoadMemberSetFromReader(r io.Reader, schema *encoding.Schema, fieldName string) (LoadMemberSetResult, error)

LoadMemberSetFromReader reads newline-delimited values from r and returns the best MemberSet for the named field on schema (bitset for categorical, uint64 map for integer / date, string map for decimal / fallback). Lines are whitespace-trimmed, a leading UTF-8 BOM is stripped and blank lines are skipped. Float fields are rejected.

type LookupKey added in v0.27.0 

type LookupKey = types.LookupKey

LookupKey re-exports types.LookupKey — one ordered key column/value pair in a composite LookupRequest.Keys tuple, so callers can build a composite key without importing the types package.

type LookupMultiplicity added in v0.27.0 

type LookupMultiplicity = types.LookupMultiplicity

LookupMultiplicity re-exports types.LookupMultiplicity — the duplicate-key handling mode on LookupRequest (assert-unique by default; opt into "first" or "all" for a key that may match more than one row).

type LookupRequest added in v0.27.0 

type LookupRequest = types.LookupRequest

LookupRequest re-exports types.LookupRequest so callers can use pulse.LookupRequest as the input to Lookup.

type LookupResult added in v0.27.0 

type LookupResult = types.LookupResult

LookupResult re-exports types.LookupResult — the response from Lookup.

type LookupTable added in v0.7.0 

type LookupTable struct {
	Description string
	// Rows is the canonical static map. The map key is the joined
	// composite key (caller joins arguments with the table's
	// configured separator, conventionally "|").
	Rows map[string]float64
	// Lookup is the function-driven accessor. Returns the value
	// and true on hit; the second return is the miss signal. An
	// error from Lookup surfaces as PULSE_LOOKUP_MISS with the
	// embedder's message attached.
	Lookup func(keys ...string) (value float64, ok bool, err error)
}

LookupTable exposes a static keyed map to the runtime expression environment. Exactly one of Rows / Lookup must be non-nil.

Rows is the simple path — a composite-key map where the caller of lookup() supplies the joined key. Lookup is the escape hatch for embedders that compose keys, perform partial-match fallback, or pull from an external store.

type MemberSet added in v0.8.3 

type MemberSet interface {
	// Len is the number of distinct members in the set.
	Len() int
	// Kind names the representation: "bitset", "uint64" or "string".
	Kind() string
}

MemberSet is a read-only set of field values consumed by FilterToFileBySetAndExpr as an include-set membership test. Build one with LoadMemberSetFromReader, which picks the fastest representation for the field's type (bitset for categorical, uint64 map for integer / date, string map for decimal / fallback). Only sets returned by LoadMemberSetFromReader are accepted by FilterToFileBySetAndExpr; any other implementation is refused when the membership predicate is built.

type Options 

type Options struct {
	// DataDir is the base directory for cohort files.
	// Defaults to PULSE_DATA_DIR if empty and FS is not set.
	DataDir string

	// FS is an optional custom filesystem.
	// When set, DataDir is ignored for filesystem construction.
	FS afero.Fs

	// DisableDefaults turns off the smart-defaults pass that infers
	// operator Type from the named field's schema type when the caller
	// omits it. Defaults to false (defaults enabled). Predict still
	// computes and reports DefaultsApplied independently — this flag
	// governs only what the runtime mutates on the live request.
	DisableDefaults bool

	// DisableComponents suppresses the Response.Components block emitted
	// by every Process / Compose / ProcessChain / Facet response —
	// per-aggregator, per-grouper, per-filterer, crosstab, and run-wide
	// constituent metadata. Defaults to false (components emitted).
	//
	// When true, Response.Components stays nil and the wire form is
	// byte-identical to the pre-Components baseline; format_version is
	// not bumped. The gate sits at the processor's attach helpers and
	// the per-shard merge path, so the MetaAggregator.Components and
	// MetaGrouper.Components construction work is skipped entirely —
	// not built-then-discarded.
	//
	// Per-request override via Request.DisableComponents (*bool): nil
	// inherits this engine setting; explicit true / false wins.
	DisableComponents bool

	// DisableProjection turns off buffered-decode field projection.
	//
	// Projection is ON by default because it is output-transparent: the
	// runtime walks each request to compute the set of schema fields the
	// operators actually read (processing.NeededFields) and skips map
	// writes for fields outside that set. The result payload is identical
	// — only faster and lighter on per-record memory. Extension operators
	// without a registered FieldInputs hook widen the projection to the
	// full schema, so the default is always safe (worst case degenerates
	// to the full-decode behaviour).
	//
	// Set DisableProjection: true to force full-record decode (every
	// schema field populated in the record map regardless of what the
	// request references). Defaults to false (projection enabled).
	DisableProjection bool

	// DisableCrosstabFusion forces every crosstab request onto the
	// buffered path, skipping the fused in-decode streaming arm even
	// when the fusion gate (processing.CanFuseCrosstab) admits the
	// request. Output is identical either way — fusion is a peak-heap
	// optimisation — so this is a diagnostic / benchmarking knob, e.g.
	// to compare fused against buffered memory on the same request.
	// Defaults to false (fusion engages whenever the gate accepts).
	// Predict and PredictBytes honour it: PredictResult.CrosstabFusable
	// then answers false with a reason.
	DisableCrosstabFusion bool

	// ImportsDir overrides the managed-imports directory. Defaults to
	// internal/imports.DefaultImportsDir (resolved relative to the Pulse fs
	// root). Honoured before the PULSE_IMPORTS_DIR env var.
	ImportsDir string

	// ImportTTL overrides the default TTL applied to managed imports
	// when the caller does not pass one. Zero falls back to the
	// PULSE_IMPORT_TTL env var, then to internal/imports.DefaultTTL. Negative
	// values pin imports (never expire) by default.
	ImportTTL time.Duration

	// ImportSourceFS is the afero.Fs used to read source files when
	// ImportFile / pulse_import receives an absolute path. Defaults to
	// afero.NewOsFs() on real OS installs, or to FS when FS is an
	// in-memory MemMapFs (so tests stay hermetic). Setting this
	// disables the default jail: the explicit fs IS the boundary.
	ImportSourceFS afero.Fs

	// ImportSourceJailRoot confines absolute source paths to a
	// directory tree. Empty string defaults to os.Getwd() at New
	// time — the natural sandbox for an MCP server or CLI invocation.
	// Ignored when ImportSourceFS is set explicitly.
	ImportSourceJailRoot string

	// Extensions registers domain-specific operators + expression
	// extensions that the runtime treats identically to built-ins.
	// Zero value disables the extension path entirely. See
	// extensions.go for the full surface.
	Extensions Extensions

	// ShardWorkers caps the per-shard parallel worker pool used when
	// Process operates on a shard archive. Zero means runtime.NumCPU();
	// 1 forces strictly serial execution (same byte-for-byte semantics
	// as the serial path). Negative values are rejected at New() time.
	//
	// The parallel reducer engages only when every operator in the
	// request is mergeable — built-ins per their type, an extension
	// aggregator per its AggregatorRegistration.Mergeable declaration.
	// Non-mergeable requests (percentile aggregators, window operators,
	// tier-2 tests, two-pass attributes combined with groupers, ...)
	// fall through to the serial shardIter path with no worker
	// spawning. Worker count is also capped at the shard count — no
	// point spawning more workers than shards.
	//
	// Order semantics: partials merge in shard insertion order (zip
	// central-directory order). Associative+commutative aggregators
	// (count, sum, min, max, null_count, frequency, distinct_count,
	// mode) produce byte-equal results vs the serial path; Welford
	// mean / variance / stddev drift within ULP on well-conditioned
	// inputs (parallel formula, see processing.MergeOnline docstrings).
	ShardWorkers int

	// DecodeWorkers caps the per-cohort parallel decode worker pool the
	// buffered Process path spawns when a single-file cohort is large
	// enough to benefit from segmenting record-decode across workers.
	// Zero means runtime.NumCPU() at dispatch time when the cohort
	// exceeds service.parallelDecodeRecordThreshold (currently 100_000
	// records); 1 forces strictly serial execution regardless of cohort
	// size. Negative values are rejected at New() time.
	//
	// Below the threshold the buffered Process path stays serial
	// regardless of this knob — worker spawn + merge overhead
	// dominates savings on small inputs. Sharded cohorts continue to
	// route through ShardWorkers; the two knobs are orthogonal (one
	// fans out across shards, the other fans out across record
	// segments within a single file or shard payload).
	DecodeWorkers int

	// DefaultTimeZone is the engine-wide IANA zone ("UTC" or an
	// Area/Location name such as "Europe/Berlin") that zone-capable
	// request slots inherit when neither the slot's `tz` nor the
	// request's `time_zone` names one. Empty means UTC. New() validates
	// the name and refuses an unknown one with PULSE_TIMEZONE_UNKNOWN; a
	// valid non-UTC zone is accepted here (whether a request may resolve
	// onto it is decided per request).
	DefaultTimeZone string

	// DefaultWeight is the engine-wide row weight that weight-bearing
	// request slots inherit when neither the slot's `weight` nor the
	// request's `weight` names one; a slot's `weight: null` opts out.
	// It reaches Process, Compose slots and ProcessChain stages, never
	// Facet / FacetSchema. Nil means none. New() refuses a spec with no
	// Field or an unknown Kind (PROCESSING_CONFIG); the field itself is
	// judged per request against the cohort it runs over, where the
	// default applies. See .claude/reference/weighting.md.
	DefaultWeight *types.WeightSpec

	// DefaultMultiplicity is the engine-wide multiple-comparison
	// correction block that tests, post-tests and overlays inherit,
	// field by field, when neither their own `multiplicity` nor the
	// request's (a Compose slot: then the composed request's) sets the
	// field. It reaches Process, Compose, ProcessChain stages and
	// FacetSchema overlays. Nil (the shipped default) means none: no
	// p-value is corrected unless a request asks. New() refuses an
	// unknown method or family or an alpha outside (0, 1)
	// (PULSE_MULTIPLICITY_INVALID); a family a surface does not offer
	// falls back to that surface's default there.
	DefaultMultiplicity *types.Multiplicity

	// SuppressAdvisories lists PULSE_ADVISORY_* codes this instance's
	// predict never reports in PredictResult.Advisories — for an
	// embedder whose product handles that analysis-fit question itself.
	// Advisories are coded, non-blocking notes (never warnings, never
	// escalated by Strict, never a change to execution); suppressing one
	// changes nothing else. Engine-level only: there is no per-request
	// or CLI knob. New() refuses a code that is not a registered
	// advisory code (PULSE_SUPPRESS_ADVISORY_UNKNOWN). Nil or empty
	// suppresses nothing.
	SuppressAdvisories []string

	// Limits are the instance resource limits: request timeout, group
	// count, crosstab cells, estimated memory, matrix dimension, Compose
	// slots, chain stages and join build rows. A zero field uses the
	// built-in default (DefaultMaxGroups, ...), Unlimited (-1) disables
	// that limit, and any other negative value fails New() with
	// PULSE_LIMIT_INVALID. Precedence per field: a non-zero value here >
	// the feature profile's value > the built-in default. Requests can
	// never override a limit; a breach is PULSE_LIMIT_EXCEEDED.
	// Pulse.Limits() reports the effective values.
	Limits Limits

	// DefaultReturn is the instance `return` default: the response
	// selection a request WITHOUT its own `return` block is shaped by.
	// A request block replaces it entirely (it never merges). Nil
	// defers to the feature profile's `return` section, else the
	// library default `full` (identity). Precedence: Request.Return >
	// DefaultReturn > FeatureProfile.Return > full.
	//
	// DisableComponents folds into this layer: when the components
	// compute gate is off for a request (the engine switch with no
	// request override, or a request disable_components: true), the
	// effective selection also excludes `components` — a request
	// `return` block included: it never re-opens the engine switch. With no `return`
	// on any layer DisableComponents stays the compute gate alone.
	//
	// New() refuses a bad preset, precision or path syntax
	// (PULSE_RETURN_INVALID) and a path the instance does not have —
	// including one a hidden feature owns (PULSE_RETURN_PATH_UNKNOWN).
	// A `data[*].<column>` path is accepted (no schema to judge it
	// against); the runtime warns PULSE_RETURN_PATH_UNMATCHED when it
	// matches nothing. Applies at Process today.
	DefaultReturn *types.Return

	// Strict promotes request-validation warnings into hard errors at
	// runtime. Today this covers the numeric-aggregation-on-categorical
	// check (PULSE_AGG_NOT_MEANINGFUL_FOR_CATEGORICAL); future runtime
	// validations follow the same flag. Defaults to false — Process
	// runs the request and emits warnings through the descriptor
	// Envelope (visible via --json at the CLI boundary). Predict's
	// PredictOptions.Strict remains independently controllable.
	Strict bool

	// ProjectBufferedFields enables buffered-decode field projection.
	//
	// Deprecated: projection is on by default; retained for compat.
	// Setting true is a harmless no-op; use DisableProjection to opt out.
	ProjectBufferedFields bool

	// LabelTablesDir points at a directory of JSON files that the
	// engine auto-registers as LabelTables at pulse.New time. Empty
	// disables the loader (the only source of LabelTables is then
	// Options.Extensions.LabelTables, set programmatically).
	//
	// File format: each *.json file is a mapping of source value to
	// label string, or a wrapped object {"description": "...",
	// "rows": {"k": "v"}}. The filename without the .json extension
	// becomes the registered table name.
	//
	// Honoured after PULSE_LABEL_TABLES_DIR — programmatic value
	// wins. Tables loaded from disk merge into Options.Extensions.
	// LabelTables; collisions are rejected with
	// PULSE_EXTENSION_DUPLICATE.
	LabelTablesDir string

	// RangeTablesDir points at a directory of JSON files that the
	// engine auto-registers as RangeTables at pulse.New time. Empty
	// disables the loader (the only source of RangeTables is then
	// Options.Extensions.RangeTables, set programmatically).
	//
	// File format: each *.json file is a bare array of range objects
	// [{"label":"Q1","start":"2024-01-01","end":"2024-03-31"}, ...] or
	// a wrapped object {"description": "...", "ranges": [ ... ]}. The
	// filename without the .json extension becomes the registered
	// table name.
	//
	// Honoured after PULSE_RANGE_TABLES_DIR — the programmatic value
	// wins. A table name declared both programmatically and on disk is
	// a hard error at pulse.New; the registered ranges are then
	// validated via the shared range-compilation pass, surfacing the
	// matching PULSE_RANGE_* code on any structural failure.
	RangeTablesDir string

	// TemplateDirs is the ordered list of directory roots the engine
	// scans for request templates at pulse.New time. Empty disables the
	// loader entirely — no store is built, and template lookups answer
	// PULSE_TEMPLATE_NOT_FOUND.
	//
	// File format: every *.json file beneath a root is a template
	// document. A template's name is its path relative to its OWN root,
	// minus the .json extension, forward-slash separated — a file at
	// <root>/finance/revenue.json is named "finance/revenue" — so
	// subdirectories namespace for free and the root's own location never
	// leaks into the name.
	//
	// Roots are a precedence list and the FIRST root wins: the same name
	// under a later root is shadowed, not rejected, which is what lets a
	// site override directory sit ahead of a shipped default. A root that
	// does not exist is skipped (an absent optional layer is not a
	// fault); a root that exists but is not a directory is an error.
	//
	// Honoured before PULSE_TEMPLATES_DIR — the programmatic value wins,
	// and the env var (roots separated by os.PathListSeparator) is
	// consulted only when this slice is empty. The store is built
	// eagerly, so a malformed template fails pulse.New with the offending
	// file named.
	//
	// The roots stay live after startup: a lookup re-walks them when its
	// cached snapshot has aged past the store's rescan interval, so files
	// added, changed, and removed are picked up without restarting the
	// process. ReloadTemplates forces that walk immediately for callers
	// who need determinism rather than eventual visibility.
	//
	// Breakage after startup degrades per file rather than globally: a
	// template whose file becomes malformed keeps serving its last-good
	// parse, and the broken state surfaces on ListTemplates rather than
	// failing the catalog. That split is deliberate — at startup a broken
	// document is a deploy error, and afterwards it is almost always a
	// half-written editor save.
	TemplateDirs []string

	// EchoRequest causes execution paths and descriptor operations to
	// populate descriptor.Envelope.Request with the *normalized* request
	// that was executed — smart defaults resolved, per-stage forms
	// captured for ProcessChain. Off by default to keep wire size
	// unchanged for hot paths and existing callers. The streaming
	// process / compose paths skip the echo unconditionally (NDJSON
	// emit per row, no envelope construction). Predict / inspect /
	// facet / chain / join descriptor endpoints honor the flag via
	// PredictOptions and the equivalent option structs.
	//
	// Intended for automation callers that want to log or replay the
	// exact request the engine ran without re-deriving defaults
	// themselves.
	EchoRequest bool

	// AutoLabels are default LabelBindings the engine injects into every
	// read request (Process / Compose / Facet / Sample) before
	// validation, so registered label tables render display strings
	// without the caller specifying bindings per request. Each binding
	// is applied only when its Field is present and categorical in the
	// target cohort's schema and the caller has not already bound that
	// field — a default that does not fit a given cohort is silently
	// skipped, never an error. Bindings whose Table is not registered are
	// rejected at New time. Empty (the default) disables auto-binding
	// entirely; existing behaviour is unchanged.
	AutoLabels []LabelBinding

	// FeatureProfile declares the features this instance offers, as a
	// Go value. pulse.New validates it structurally and stores a copy on
	// the instance; the feature list is not applied yet, but its
	// Behaviour switches take effect (OR semantics: a profile switch set
	// true turns the matching Options switch on, and Options cannot turn
	// it back off). Nil means no profile — behaviour is unchanged.
	//
	// Mutually exclusive with FeatureProfileFile: setting both fails
	// New with PULSE_FEATURE_PROFILE_INVALID. pulse.New never reads a
	// profile from the environment.
	FeatureProfile *FeatureProfile

	// FeatureProfileFile names a JSON feature-profile file read through
	// the instance filesystem (Options.FS, or DataDir's OS-backed
	// filesystem — so a relative path resolves under the data root)
	// after extensions are probed and the filesystem is resolved. The
	// body is decoded strictly; a missing or unreadable file, malformed
	// JSON, an unknown key, an absent "features" array or a repeated
	// feature fails New with PULSE_FEATURE_PROFILE_INVALID. Empty means
	// no file. Mutually exclusive with FeatureProfile.
	FeatureProfileFile string

	// SetInferenceMinPct configures the delimited-cell heuristic used
	// when an importer must classify a column as set_* vs categorical.
	// Threshold is the minimum percentage of non-null sampled cells
	// that must contain the inferred delimiter for the column to be
	// classified as set_*; combined with two other gates (post-split
	// unique token count ≤ 64 + average post-split cardinality > 1)
	// the threshold trades off false-positive misclassifications
	// against missed multi-select columns. Zero falls back to the
	// package default of 30. Values > 100 clamp to 100. Per-import
	// overrides via the managed-import Spec still take precedence over
	// this default.
	SetInferenceMinPct int

	// Logger receives the instance's structured log records (lifecycle,
	// plan decisions, warnings and failures by error code — never row
	// data or error messages). Nil (the default) logs nothing and does
	// no handler work.
	Logger *slog.Logger

	// Hooks are optional callbacks fired around every operation (and
	// its phases). Nil (the default) fires nothing; each field of a
	// non-nil Hooks is independently optional. Hooks run synchronously
	// and must be fast; a panicking hook is recovered.
	Hooks *observe.Hooks

	// Metrics is the instrument factory the instance records its
	// operation metrics through. Nil (the default) records nothing. The
	// factory may be called lazily after New, at most once per label
	// combination, and must be safe for concurrent use.
	Metrics observe.Metrics
}

Options configures a Pulse instance.

type ParamMeta added in v0.7.0 

type ParamMeta struct {
	Name        string `json:"name"`
	Description string `json:"description,omitempty"`
	JSONType    string `json:"json_type"`
	Required    bool   `json:"required,omitempty"`
	Default     any    `json:"default,omitempty"`
}

ParamMeta describes one operator-specific parameter for manifest emission. It is descriptive only — runtime parameter resolution happens inside each factory.

type Profile added in v0.2.0 

type Profile = synth.Profile

Profile is the cohort statistical summary used by from-profile.

type ProfileOptions added in v0.2.0 

type ProfileOptions = synth.ProfileOptions

ProfileOptions modulate which statistics the profiler captures.

type Pulse 

type Pulse struct {
	// contains filtered or unexported fields
}

Pulse is the top-level library facade. It wraps the service layer and provides a clean API for embedding Pulse into Go programs.

func New 

func New(opts Options) (*Pulse, error)

New creates a new Pulse instance with the given options.

func (*Pulse) AddShard added in v0.8.0 

func (p *Pulse) AddShard(ctx context.Context, archivePath, shardPath string) (*AddShardResult, error)

AddShard validates the incoming single-file shard against the archive's canonical schema and appends it. Dict growth that the incoming shard introduces is reflected in the rewritten `_schema.pulse` payload before the new shard payload is appended. v1 reads the whole archive into memory and writes it back via temp+fsync+rename — semantically equivalent to true in-place append and crash-safe at the canonical-path level.

A merged dictionary that outgrows a set_* field's bitmask WIDENS the field across the canonical schema and every shard payload instead of refusing the add, and reports it as a mandatory PULSE_SHARD_SET_WIDENED warning on the returned result — the rewrite is far cheaper than the re-import it replaces but far more expensive than an append, and a caller must be able to tell which one it paid for. A union above the widest set rung stays fatal (PULSE_SHARD_DICT_WIDTH_OVERFLOW).

AddShard returns a result rather than a bare error precisely so that warning has nowhere to be dropped.

A grouped (format 0x02) shard is conformed to the archive's layout: an ungrouped archive stores it flattened, a grouped archive re-encodes it into its own groups and union-merges the group dictionaries (a union past the u32 index space is PULSE_SHARD_DICT_WIDTH_OVERFLOW). A constant group the shard disagrees with is promoted to indexed across the whole archive; a declared key it violates is refused (PULSE_GROUP_MEMBER_NOT_CONSTANT). Layout changes are reported as a mandatory PULSE_SHARD_GROUPS_REWRITTEN warning and on Regrouped.

func (*Pulse) ApplySeriesOverlays added in v0.36.0 

func (p *Pulse) ApplySeriesOverlays(ctx context.Context, req *SeriesOverlayRequest) (*SeriesOverlayResult, error)

ApplySeriesOverlays applies the SERIES overlay catalog to a series the caller already holds.

It is the same fold the grouped-Process path runs at host finalize, reached without a Process: build the host view from Points, dispatch each spec through the per-kind SERIES handler table, return the layers. A caller with pre-aggregated values — stored metric cells read by point lookup, a series assembled from several archives — gets Pulse-computed statistics over them without re-deriving values it already has.

Empty Overlays returns an empty result and no error, mirroring processing.ApplyOverlaysSeries' own unconditional-call contract.

func (*Pulse) BuildIndex added in v0.27.0 

func (p *Pulse) BuildIndex(ctx context.Context, path string, keyFields []string) (*BuildIndexResult, error)

BuildIndex builds a point-lookup sidecar index for the cohort at path over the ordered key columns named in keyFields (a single element is the degenerate single-key case; more than one produces a composite key, in column order). Delegates to internal/service.Service.BuildIndex for the full algorithm and error surface — see that method's doc comment for the scan/bucket/write contract, the PULSE_INDEX_UNSUPPORTED_SHARDED shard-archive rejection, and the PROCESSING_CONFIG disallowed-key-type rejection.

func (*Pulse) CohortArtifacts 

func (p *Pulse) CohortArtifacts(ctx context.Context, cohort string) ([]string, error)

CohortArtifacts returns every Pulse-owned sidecar that currently exists beside the cohort, sorted, excluding the cohort itself:

  • cohort.pulse.<keyhash>.idx — each sidecar point-lookup index
  • cohort.pulse.indexes.json — the index discovery manifest
  • cohort.pulse.spss.json — the SPSS metadata sidecar
  • cohort.pulse.meta.json — the managed-import handle metadata

It is the answer to "what must travel with this cohort": moving or copying the cohort together with every returned path keeps Lookup hitting and SPSS export faithful at the new location — provided the move carries the cohort's modification time across (a rename or `cp -p` does; a plain copy does not). Both sidecars fingerprint the cohort's size and mtime, and the SPSS sidecar refuses a cohort whose mtime moved (PULSE_SPSS_SIDECAR_STALE). Only files that exist are returned, each spelled relative to the same root as cohort (the cohort's directory joined with the sidecar's name); a cohort with none returns an empty, non-nil slice.

Nothing is opened or validated — a stale or corrupt sidecar is still listed, because it still has to move with (or be deleted beside) its cohort. Discovery is by the owners' own naming rules, never a "<cohort>.*" glob, so an unrelated file that merely shares the cohort's name as a prefix is not swept up. A missing cohort is a SERVICE_RESOURCE error.

func (*Pulse) CompactShardArchive added in v0.8.0 

func (p *Pulse) CompactShardArchive(ctx context.Context, archivePath string) error

CompactShardArchive rewrites the archive to eliminate orphaned bytes from prior in-place mutations and refreshes the canonical metadata (aggregate_record_count + shard_count). v1 AddShard / RemoveShard already use temp+rename (no orphan bytes in v1 archives), so Compact primarily serves to refresh canonical metadata that may have drifted if the archive was edited outside Pulse. The whole-archive rewrite pattern is the explicit reclaim path per the design contract §7.1.

func (*Pulse) Compose 

func (p *Pulse) Compose(ctx context.Context, req *ComposedRequest) (*ComposedResponse, error)

Compose executes multiple requests against a cohort and returns a structured ComposedResponse carrying both per-request results and any composition-level overlay output.

Returned shape:

  • Responses []*Response — one entry per request in req.Requests, in input order. Per-request errors surface as a Response with non-empty Errors; the call returns a top-level error only on orchestration failures (validation, cohort open, etc.).
  • Overlays []OverlayLayer — one layer per OverlaySpec in req.Overlays, in declaration order. The Compose-host overlay fold (service.applyComposeOverlays) writes layers post-execution; when req.Overlays is empty the slot is nil/omitempty.
  • Overlays[i].Warnings []OverlayWarning — per-overlay diagnostics emitted by the handler (cohesion failures, missing host coordinates, threshold breaches). Empty when the layer produced no diagnostics.

Shaping: each Requests[i].Return (else the instance default) shapes Responses[i] exactly as Process would; req.Return shapes the top-level Overlays (paths rooted at ComposedResponse, `overlays…` only) and stamps ComposedResponse.Returned. Both apply after the overlay fold, so every layer is computed from the unshaped slots.

func (*Pulse) ComposeParallel added in v0.2.0 

func (p *Pulse) ComposeParallel(ctx context.Context, req *ComposedRequest, opts ComposeOptions) (*ComposedResponse, error)

ComposeParallel runs every request in req concurrently across a bounded worker pool and returns a structured ComposedResponse. Workers share the engine's read-only registries; each Process call constructs fresh stateful operators per request, so concurrent execution is safe.

Defaults: MaxWorkers = runtime.GOMAXPROCS(0), no per-request timeout, FailFast = true (set FailFast=false to collect every request's outcome instead of cancelling siblings on first error).

Returned shape mirrors Compose:

  • Responses []*Response — one entry per request in req.Requests, in input order (not completion order). Workers compose the slice deterministically before returning.
  • Overlays []OverlayLayer — one layer per OverlaySpec in req.Overlays, in declaration order. The Compose-host overlay fold runs after every per-request Process has settled, so overlay handlers always observe the full Responses slice.
  • Overlays[i].Warnings []OverlayWarning — per-overlay diagnostics surfaced by distributeComposeWarnings (cohesion failures, missing host coordinates, panel-target overflow). Empty when the layer produced no diagnostics.

Shaping is Compose's, applied after every slot settled and the overlay fold ran.

func (*Pulse) Convert 

func (p *Pulse) Convert(ctx context.Context, job *pio.ConvertJob) (*pio.ConvertReport, error)

Convert chains import and export with no intermediate file on disk. The job's FS field is set to the Pulse instance's filesystem if not already set.

Labels apply to the export half only — see Export.

func (*Pulse) CountRecords added in v0.10.0 

func (p *Pulse) CountRecords(ctx context.Context, path string) (uint64, error)

CountRecords returns the number of records in the cohort at path without decoding payload bytes. For single-file cohorts this is O(1) — bytes read is bounded by header + schema size, independent of the record count. For shard archives the cost is O(N shards) (zip central-directory walk + reserved `_schema.pulse` SHRD trailer). Anchor paths (`archive#shard.pulse`) report the named shard's count only.

Use this when a planner needs to make a cardinality-dependent decision (sample injection, smaller-side hash join, batch sizing) without paying the per-row decode cost of a full Process.

func (*Pulse) CreateShardArchive added in v0.8.0 

func (p *Pulse) CreateShardArchive(ctx context.Context, archivePath string, shardPaths []string) (*CreateShardArchiveResult, error)

CreateShardArchive writes a fresh Pulse shard archive at archivePath containing the supplied single-file shardPaths. The first shard seeds the canonical schema; remaining shards are validated via structural cohesion + the append-only dictionary prefix rule. The archive is written atomically (temp file + rename) so partial writes never appear at archivePath. See internal/service.CreateShardArchive for the full error surface.

Set-width auto-widen applies at CREATE exactly as it does at ADD: a set column whose merged dictionary outgrows its bitmask, or whose rung differs between two of the seeded shards, is promoted rather than refused, and the promotion is reported as a mandatory PULSE_SHARD_SET_WIDENED warning on the returned result. One rule, no asymmetry — the same two files must not produce an archive when passed together and an error when passed one after the other.

CreateShardArchive returns a result rather than a bare error precisely so that warning has nowhere to be dropped.

Grouped (format 0x02) shards are accepted: the archive keeps ONE parent-group layout (the first shard's), group dictionaries union-merge canonical-first, and any layout change a shard needs is reported as a mandatory PULSE_SHARD_GROUPS_REWRITTEN warning plus a GroupReconciliation on Regrouped — the same rule AddShard applies.

func (*Pulse) Dedup added in v0.39.0 

func (p *Pulse) Dedup(ctx context.Context, path string, opts DedupOptions) (*DedupResult, error)

Dedup converts an existing single-file cohort to a deduplicated (format 0x02) cohort grouped by opts.Groups — retro-dedup, the existing-cohort twin of `pulse import --group`.

The byte work is pio.DedupJob: the source is read through its logical record stream, encoded through the same internal/encoding.GroupEncoder import uses into a temp-file spool (memory stays O(dictionaries)), judged by the same internal/encoding.DedupGate (width floor before the pass, ratio floor after it and BEFORE anything is written, Strict escalating either), then assembled into a temp file beside the target, fsynced and renamed over it. Any failure leaves the original byte-identical.

Refusals are coded errors: SERVICE_VALIDATION for a shard archive or an anchored shard (grouped shards are not accepted by the archive tooling yet) and for an --out path that already exists; SERVICE_RESOURCE for a missing cohort; the PULSE_GROUP_* family for a bad declaration or a member that varies within its key; and, under Strict, PULSE_GROUP_TOO_NARROW / PULSE_DEDUP_LOW_RATIO.

Output is transparent: every request returns the same Response against the converted cohort as against the original.

func (*Pulse) Drop added in v0.5.0 

func (p *Pulse) Drop(ctx context.Context, handle string) error

Drop removes a managed import handle (and its sidecar) from the pool. Returns PULSE_IMPORT_SOURCE_MISSING when the handle is unknown.

func (*Pulse) DropIndex added in v0.27.0 

func (p *Pulse) DropIndex(ctx context.Context, path string, keyFields []string) error

DropIndex removes the sidecar point-lookup index built for keyFields against the cohort at path. Delegates to internal/service.Service.DropIndex — see that method's doc comment for the non-interactive (no-confirmation-prompt) contract. Returns PULSE_INDEX_MISSING when no sidecar exists at the derived path and PULSE_INDEX_UNSUPPORTED_SHARDED for shard archive cohorts.

func (*Pulse) ErrorLookup added in v0.5.0 

func (p *Pulse) ErrorLookup(code string) (ErrorMetadata, bool)

ErrorLookup returns the metadata projection for a single error code. Case-sensitive exact match. Returns (ErrorMetadata{}, false) when the code is unknown.

The manifest carries only the alphabetized code-name list; per-code Message + Fixup detail lives behind this facade so per-session bootstrap stays lean. Use ErrorsByDomain / ErrorsSearch to enumerate in bulk.

The view is the instance's: under a feature profile a code every owning feature of which is hidden is not found, and a fixup naming a hidden feature is stripped from the result. With no profile it is the full registry.

func (*Pulse) ErrorsByDomain added in v0.5.0 

func (p *Pulse) ErrorsByDomain(domain string) []ErrorMetadata

ErrorsByDomain returns every code's metadata in the named domain (CLI, DATA, ENCODING, PROCESSING, PULSE, SERVICE). Match is case-insensitive. Returns a non-nil empty slice when nothing matches; results are sorted alphabetically by code. Instance-scoped as ErrorLookup: hidden codes are absent, hidden-naming fixups stripped.

func (*Pulse) ErrorsSearch added in v0.5.0 

func (p *Pulse) ErrorsSearch(query string) []ErrorMetadata

ErrorsSearch returns codes whose Message or Fixup hints contain the query (case-insensitive substring). Results are ranked by match source: description hits before fixup hits before code-name hits; ties resolve alphabetically. Returns a non-nil empty slice when nothing matches. Instance-scoped as ErrorLookup; the query matches only the text the instance renders, never a stripped fixup.

func (*Pulse) ExampleGet added in v0.5.0 

func (p *Pulse) ExampleGet(name string) (*Example, bool)

ExampleGet returns the example whose _meta.name matches name. The returned Body is the request JSON with the _meta block stripped so it can be handed directly to Process / Predict. Under a feature profile an example that uses a hidden operator answers (nil, false), exactly as a name that does not exist. Embedder examples (Extensions.Examples) are served the same way.

func (*Pulse) ExamplesSearch added in v0.5.0 

func (p *Pulse) ExamplesSearch(query string, tags []string, category string) []ExampleSummary

ExamplesSearch returns summaries from the request-example library matching the given filters; it is ExamplesSearchWith without an intent, so it never fails. An empty filter is treated as "no constraint" for that dimension; tags is ANDed; category is an exact match. A one-word query keeps its substring match across name, description and operators; ExamplesSearchWith documents the full matching rules. Always returns a non-nil slice (possibly empty) for safe JSON marshaling. Under a feature profile an example that uses an operator the instance hides is not returned. Embedder examples (Extensions.Examples) are searched with the built-ins, ranked by the same rules.

func (*Pulse) ExamplesSearchWith 

func (p *Pulse) ExamplesSearchWith(q ExamplesQuery) ([]ExampleSummary, error)

ExamplesSearchWith searches the request-example library. Every filter of q is optional and all are ANDed: Tags (every tag), Category (exact directory), Intent (an intent-taxonomy ID, see Intents, the example's intents must carry) and Query.

Query matching runs as ordered tiers, the first with any hit answering. Exact: a one-word query (letters and digits only) is a case-insensitive substring of the name, description or an operator, as ExamplesSearch always matched; any other query is tokenized (lower-case, split on non-alphanumerics, no stemming, stop words dropped) and every token must be an exact token of the name, description, operators, intents or tags. Synonym: a query phrase that is an operator's Purpose.KnownAs alias ("anova", "chisq.test") or sits inside an intent's Sounds ("move together") expands the query to that operator or intent. Score = distinct matched fields; ties break alphabetically by name; no query lists alphabetically.

An unknown intent, or one the instance's feature profile hides, is PULSE_RECOMMEND_INTENT_UNKNOWN (details.valid lists the instance's intents). Under a feature profile a hidden operator's aliases and a hidden intent's Sounds never expand a query, and hidden examples are absent. Never nil on success.

func (*Pulse) Explain 

func (p *Pulse) Explain(ctx context.Context, req descriptor.ExplainRequest) (*descriptor.ExplainResult, error)

Explain says, in plain words, what a request will do — without running it — or what a result found. req names exactly one request root (Request, Composed, Chain, Facet or Sample), or exactly one result root (Response, ComposedResponse, ChainResponse or FacetResult) with at most its own request companion (Request, Composed, Chain or Facet respectively); anything else, or a Detail other than terse / full, is SERVICE_VALIDATION.

Response mode reads the response into Findings — one per test, regression coefficient and fit, overlay layer, matrix and aggregation — each with a closed verdict read at the result's own alpha (adjusted when a correction ran), a strength band only where the figure's Interpretation names a convention, and the figures it rests on (undefined ones null). Without the companion an aggregation is described by count only and a caveat says the reading is partial. A ComposedResponse is read slot by slot then its batch overlays, a ChainResponse stage by stage with the request it echoes (else its companion), a FacetResult field by field with the overlays that decorate each; every finding's Slot is the wire path it reads, and the many-tests caveat counts the whole result. Response mode opens no cohort.

The result's Steps describe each slot from its structure and its operator's plain purpose. When the root names a cohort Explain predicts it in-process with the instance's options and the cohort's sidecar facts — header, schema and sidecar only, never a record of a single-file cohort (a shard archive or anchored shard is read whole, as Predict reads it) — to name each operator smart defaults infer (DefaultsApplied, and the step marked Defaulted), the advisories the request raises and predict's verdict (Valid, with each refusal a caveat). Without a cohort nothing is checked, and a caveat says so. A cohort that cannot be read is DATA_FILE (ENCODING_INVALID for a malformed header or schema). A Compose root predicts each slot over its own cohort and validates the batch; a chain root predicts its first stage and validates the chain; a Sample root names no operator and is never predicted.

Terse detail (the default) returns the summary sentence and the structured parts; full detail adds the narrative sentences, the glossary terms the operators link, their declared follow-ups and the assumptions of the request's tests and models. Under a feature profile no hidden operator or capability is named. Explain is not an observed operation: its predicts run in-process and fire no predict hooks or metrics.

func (*Pulse) Export 

func (p *Pulse) Export(ctx context.Context, job *pio.ExportJob) (*pio.ExportReport, error)

Export converts a .pulse file into tabular output. The job's FS field is set to the Pulse instance's filesystem if not already set.

When job.Labels is non-empty, the facade builds the runtime label resolver from the Service's registered LabelTables and attaches it to job.LabelResolver before Run. The resolver applies replace / augment translation to categorical column values during export.

func (*Pulse) ExportReference 

func (p *Pulse) ExportReference(fsys afero.Fs, dir string, opts ExportReferenceOptions) error

ExportReference writes the instance's analysis reference — the operator catalog, the "Reading your results" pages, the glossary and (unless opts.OmitSkills) every skill as the instance serves it — as a deterministic Markdown tree under dir on fsys, with an mdBook SUMMARY.md fragment at its root. A nil fsys writes through the instance filesystem (Fs). Two exports of the same instance are byte-identical.

The export is instance-scoped exactly like Ontology, Skills and Skill: under a feature profile no page, row, link or prose line names a hidden operator, MCP tool or a hidden capability's slot token.

Overwrite rule: the export writes the marker file .pulse-docs-export at dir. A missing or empty dir, or one carrying the marker, is written in place, and every file a previous export recorded in the marker that this export no longer renders is deleted (with any directory left empty by it); files the marker does not list are never touched. A non-empty dir without the marker is refused with PULSE_DOCS_EXPORT_DIR_NOT_EMPTY before anything is written. A filesystem failure is a DATA_FILE error naming the path.

func (*Pulse) ExportTransfer added in v0.39.0 

func (p *Pulse) ExportTransfer(ctx context.Context, job *pio.TransferExportJob) (*pio.TransferReport, error)

ExportTransfer compresses a cohort's exact bytes into a zstd transfer artifact (`.pulse.zst`) for moving it between machines. The job's FS field is set to the Pulse instance's filesystem if not already set.

Compression is TRANSPORT-ONLY: the artifact is never opened as a cohort (every read surface refuses it with PULSE_COHORT_COMPRESSED), and ImportTransfer turns it back into a byte-identical `.pulse` at rest. Works for single-file cohorts (0x01 and 0x02) and whole shard archives alike, streaming in bounded memory.

func (*Pulse) ExtractShard added in v0.8.0 

func (p *Pulse) ExtractShard(ctx context.Context, archivePath, shardBasename string) (io.ReadCloser, error)

ExtractShard returns an io.ReadCloser over the named shard's standalone single-file `.pulse` bytes. Suitable for piping to `pulse inspect -` or writing back to disk.

func (*Pulse) Facet 

func (p *Pulse) Facet(ctx context.Context, path string, field string) ([]string, error)

Facet returns distinct values for the named field in the cohort. Categorical fields short-circuit through the dictionary; numeric fields stream the file collecting distinct float values. For richer summaries (counts, null tallies, statistics, histograms, additive contributions) call FacetSchema instead.

func (*Pulse) FacetSchema added in v0.7.0 

func (p *Pulse) FacetSchema(ctx context.Context, req *FacetRequest) (*FacetResult, error)

FacetSchema runs a multi-field rich facet against the cohort named in req.Cohort. Returns per-field summaries (discrete value counts or numeric statistics), with optional percentiles, fixed-width histograms, and "additive" contribution counts that report what each distinct value of an additive field would yield if it were added to the base filter.

Streamability: requests with no NumericPercentiles run in a single pass; requests with percentiles buffer the requested numeric fields' non-null values and sort once before percentile interpolation.

func (*Pulse) FeatureProfile 

func (p *Pulse) FeatureProfile() (*FeatureProfile, bool)

FeatureProfile returns a copy of the feature profile the instance was built with and true, or nil and false when it was built without one. Mutating the returned value does not affect the instance.

func (*Pulse) FeatureSetDigest 

func (p *Pulse) FeatureSetDigest() string

FeatureSetDigest returns the instance's feature_set_digest: "fs1:" followed by the hex SHA-256 of its sorted enabled feature names and its effective behaviour switches (a feature profile's behaviour ORed with Options). Every instance has one, a profile-free instance included. Two instances offering the same features with the same switches share a digest — across processes too — so it can key caches of self-description output; any difference in either changes it.

func (*Pulse) FilterToFile added in v0.8.1 

func (p *Pulse) FilterToFile(ctx context.Context, src, dst, filterExpr string) (int64, error)

FilterToFile reads the .pulse cohort at src, evaluates filterExpr (FILTER_EXPRESSION semantics — same operators, identifiers, expr functions, and lookup tables available to pulse.Process with a FILTER_EXPRESSION filterer) against every record, and writes a new .pulse cohort at dst containing only matching records.

Dispatch matches the rest of the facade. Single-file inputs produce single-file outputs whose header + schema bytes are copied byte-for- byte from src. Shard archives produce shard archives that preserve the per-shard layout: one input shard maps to one output shard at the same basename, in central-directory (insertion) order, with per-shard header + schema + categorical-dictionary bytes copied verbatim. Empty shards survive so shard_count metadata stays stable; the canonical `_schema.pulse` trailer's aggregate_record_count is refreshed to the surviving total. The anchor form `archive.pulse#shard.pulse` resolves a single shard and writes a single-file output.

Returns the number of records written to dst (sum across shards for archive inputs).

func (*Pulse) FilterToFileBySetAndExpr added in v0.8.3 

func (p *Pulse) FilterToFileBySetAndExpr(ctx context.Context, src, dst, includeField string, set MemberSet, filterExpr string) (int64, error)

FilterToFileBySetAndExpr applies an optional MemberSet membership test (record's value for includeField must be in set) combined with an optional FILTER_EXPRESSION (filterExpr) to every record in src, writing the survivors to dst. At least one of (set, filterExpr) must be supplied; if both are present they are AND-combined and the set is tested first so per-row work short-circuits on misses without paying the expr eval cost.

Input-shape dispatch is identical to FilterToFile (single-file, shard archive, anchor). The set must be built against the same canonical schema as src.

Returns the number of records written to dst (sum across shards for archive inputs).

func (*Pulse) FilterToFileWithRequest added in v0.11.0 

func (p *Pulse) FilterToFileWithRequest(ctx context.Context, req *FilterToFileRequest) (*FilterToFileResult, error)

FilterToFileWithRequest is the deterministic, dedup-aware variant of FilterToFile. Same (source content, predicate) inputs produce the same output file hash and output path; if the expected output already exists at the target path, the engine returns it without re-doing the work.

Determinism comes from three guarantees:

  • The .pulse byte format embeds no timestamps and no compression-level variation, so the underlying FilterToFile engine produces byte-identical output for identical inputs.
  • The output is written via atomic rename: a temp file is staged in OutputDir and renamed to the final path on success. A partially-written .pulse file never appears at the target path.
  • The output name defaults to `{source-hash}_{predicate-hash}.pulse` so independent consumers reach the same expected path without coordination.

func (*Pulse) Fs added in v0.2.0 

func (p *Pulse) Fs() afero.Fs

Fs returns the underlying afero.Fs. Embedders (e.g. the MCP server) need this to enumerate .pulse files; processing methods route through service and never expose the filesystem directly.

func (*Pulse) GetTemplate added in v0.28.0 

func (p *Pulse) GetTemplate(name string) (*Template, error)

GetTemplate returns the parsed declaration registered under name — target, description, and the full Variable list — so a caller can build a form (or a prompt) from the declaration alone before rendering.

Lookup is exact and case-sensitive, and the name is the derived one: a template's path relative to its own directory root, minus the .json extension, forward-slash separated. A file at <root>/finance/revenue.json is named "finance/revenue" — not "finance/revenue.json" and not the absolute path.

An unregistered name — including every name on an engine with no template directories configured — is PULSE_TEMPLATE_NOT_FOUND carrying the requested name in its details. A name whose file has broken since it was loaded still resolves, to the last-good parse (see ReloadTemplates); only a name that has NEVER parsed is PULSE_TEMPLATE_INVALID here, naming the path so the operator knows which file to open.

The returned template is the engine's own copy and must be treated as read-only; rendering never mutates it.

func (*Pulse) Import 

func (p *Pulse) Import(ctx context.Context, job *pio.ImportJob) (*pio.ImportReport, error)

Import converts tabular source data into a .pulse file. The job's FS field is set to the Pulse instance's filesystem if not already set.

func (*Pulse) ImportFile added in v0.5.0 

func (p *Pulse) ImportFile(ctx context.Context, spec ImportSpec) (*ImportResult, error)

ImportFile auto-detects the source format, converts the source into a managed .pulse file under the imports pool, and returns the resulting handle. Pulse-native sources pass through unchanged (no copy, no sidecar). Sliding-window TTL applies to managed handles — every subsequent Inspect / Predict / Process / Sample / Facet against the handle bumps the expiry forward.

func (*Pulse) ImportTransfer added in v0.39.0 

func (p *Pulse) ImportTransfer(ctx context.Context, job *pio.TransferImportJob) (*pio.TransferReport, error)

ImportTransfer decompresses a transfer artifact produced by ExportTransfer into a byte-identical `.pulse` at rest (temp file, fsync, rename — a damaged artifact leaves nothing behind). The job's FS field is set to the Pulse instance's filesystem if not already set. The returned report's SHA256 matches the sender's.

func (*Pulse) Imports added in v0.5.0 

func (p *Pulse) Imports(ctx context.Context) ([]ImportEntry, error)

Imports returns a snapshot of the managed-imports pool. Sweep is not invoked; expired entries are flagged via Entry.Expired so callers can render them. Results are sorted by handle name.

func (*Pulse) Inspect 

func (p *Pulse) Inspect(ctx context.Context, path string) (*descriptor.InspectResult, error)

Inspect reads a .pulse file header and schema, returning structured field information. It never reads record data.

Callers that need the inspect WARNINGS (a truncated payload tail raises one) or dictionary options want InspectEnvelope — this wrapper keeps the result and discards the rest of the envelope.

func (*Pulse) InspectBytes 

func (p *Pulse) InspectBytes(ctx context.Context, data []byte, opts *descriptor.InspectOptions) (*descriptor.Envelope, error)

InspectBytes inspects an in-memory .pulse cohort — a single file or a whole shard archive, detected by its leading magic bytes — and returns the descriptor envelope WHOLE, warnings included (a payload whose length is not a whole multiple of the record stride reports its floored record count beside an ENCODING_INVALID warning). It is the byte-level twin of InspectEnvelope for a caller that already holds the bytes: no filesystem read, no anchor resolution, no managed-import TTL slide, and no sidecar — so never a SuggestedWeight. opts may be nil (defaults).

Inspect has no request and no strict mode, so it reads nothing else from the instance's Options. A returned error is a cancelled ctx; every fault in the bytes themselves comes back as env.Errors so a --json caller can emit the coded envelope verbatim.

func (*Pulse) InspectEnvelope added in v0.34.0 

func (p *Pulse) InspectEnvelope(ctx context.Context, path string, opts *descriptor.InspectOptions) (*descriptor.Envelope, error)

InspectEnvelope is Inspect's envelope-returning sibling: same no-record read (header + schema, plus the SPSS metadata sidecar beside the cohort for InspectResult.SuggestedWeight), same anchor resolution, same afero.Fs, but it returns the descriptor envelope WHOLE and accepts InspectOptions (nil means defaults).

It exists because the two things a caller can need beyond the result are unreachable through Inspect: the InspectOptions.FullDict knob, and env.Warnings — a cohort whose payload length is not a whole multiple of the record stride reports its floored record count with an ENCODING_INVALID warning beside it, and Inspect drops that warning on the floor. `pulse cohort inspect --json` / `--full-dict` used to hand bytes from its own os.ReadFile to descriptor's byte-level inspect for exactly this reason, which cost it anchor resolution (archive.pulse#shard.pulse resolved in text mode and failed under --json) and the injected filesystem along with it.

A returned error is a READ failure (missing file, unresolvable anchor). An envelope-level fault — a bad header, a corrupt archive — comes back as a non-nil envelope carrying env.Errors, so a --json caller can emit the coded envelope verbatim.

func (*Pulse) InvalidatedSidecars added in v0.38.0 

func (p *Pulse) InvalidatedSidecars(ctx context.Context, cohortPath string) ([]StaleSidecar, error)

InvalidatedSidecars reports the sidecar documents beside the cohort at cohortPath whose validity is bound to the cohort's bytes — the ones a rewrite such as WidenSetField invalidates.

Discovery goes through each sidecar's OWN naming authority, never a filename guess: point-lookup indexes come from ListIndexes (which reads the keyless discovery manifest first and unions any on-disk sidecar it does not name), and the SPSS metadata sidecar from spsssidecar.Path. Guessing would be wrong in the one direction that matters — an index filename is a hash of its key tuple, so a guess cannot recover the tuple the rebuild command needs.

The result is ordered deterministically (indexes first, by path, then the SPSS sidecar) so a caller can print or diff it. An empty slice means there is nothing beside the cohort to rebuild; callers should print NOTHING rather than an empty section.

A cohort with no sidecars, and a shard archive (which cannot carry a point-lookup index at all), both answer with an empty slice rather than an error: this is advisory reporting attached to an operation that has already succeeded, and failing it would turn a hint into an outage.

func (*Pulse) LabelTables added in v0.11.2 

func (p *Pulse) LabelTables() []LabelTableInfo

LabelTables returns the registered label tables in name order. It reports each table's row count and whether it is reverse-searchable. Returns nil when no label tables are registered.

func (*Pulse) Limits 

func (p *Pulse) Limits() Limits

Limits returns a copy of the instance's effective resource limits: per field, a non-zero Options.Limits value, else the feature profile's value, else the built-in default.

func (*Pulse) ListIndexes added in v0.27.0 

func (p *Pulse) ListIndexes(ctx context.Context, path string) ([]IndexInfo, error)

ListIndexes enumerates every sidecar point-lookup index built against the cohort at path. Delegates to internal/service.Service.ListIndexes — see that method's doc comment for the directory-glob + sidecar-read discovery algorithm. Returns an empty (non-nil) slice, not an error, when no sidecar indexes have been built yet. Returns PULSE_INDEX_UNSUPPORTED_SHARDED for shard archive cohorts.

func (*Pulse) ListShards added in v0.8.0 

func (p *Pulse) ListShards(ctx context.Context, archivePath string) ([]ShardEntry, error)

ListShards returns the archive's shard manifest in central- directory order (which equals shard insertion order). Single-file cohorts return an empty slice.

func (*Pulse) ListTemplates added in v0.28.0 

func (p *Pulse) ListTemplates() []TemplateSummary

ListTemplates returns one summary per registered request template, sorted by name so the order is deterministic across runs and platforms.

Each Summary projects everything a caller needs to CHOOSE a template and build a form for it — name, description, target, declared variable names, and the source file it was loaded from — without carrying the body. The full declarations (types, defaults, enum values) come from GetTemplate.

Shadows names the source paths of same-named templates the listed entry takes precedence over. Template directories are an ordered precedence list and the first root wins; the losing entries are reported here rather than discarded, which is what makes "why is my override not taking effect?" answerable from the listing alone. A shadowed entry deliberately gets no summary of its own — it is not renderable, and a listing whose entries cannot all be fetched would be a trap.

Broken is how a post-startup breakage becomes visible. A template file that has stopped parsing since it was loaded keeps ANSWERING GetTemplate with its last-good copy, so nothing about fetching it would reveal the fault; the flag, with the fault text in Error, is what lets an operator find the bad file without rendering all fifty templates one at a time. A broken entry with an empty Target never parsed at all — the file was already malformed the first time the engine saw it — so it is listed to be SEEN rather than fetched, and asking for it by name returns PULSE_TEMPLATE_INVALID.

Always returns a non-nil slice (possibly empty) for safe JSON marshaling. An engine with no template directories configured lists nothing; that is an ordinary deployment, not a fault.

func (*Pulse) Lookup added in v0.27.0 

func (p *Pulse) Lookup(ctx context.Context, req *LookupRequest) (*LookupResult, error)

Lookup resolves a point lookup (single-key or composite, via req.Keys) against the cohort named in req.Cohort, using the prebuilt sidecar index (see BuildIndex). Returns PULSE_INDEX_MISSING when no sidecar index exists for the requested key fields, PULSE_LOOKUP_NOT_FOUND when the index exists but no record matches, or PULSE_LOOKUP_AMBIGUOUS when more than one record matches and req.Multiplicity is (or defaults to) LookupMultiplicityAssertUnique. See internal/service.Service.Lookup for the full algorithm.

func (*Pulse) Manifest added in v0.5.0 

func (p *Pulse) Manifest(ctx context.Context) *descriptor.Manifest

Manifest returns the instance's self-description: only the features it offers. Without a feature profile that is the full registry plus the instance's extensions; with one, hidden operators, capabilities, I/O formats, commands and MCP tools are absent (a hidden capability's block is omitted) and no prose names them. FeatureSetDigest identifies the described set; the always-present Limits block lists the instance's effective resource limits and LimitsDigest identifies them. The manifest is deterministic per instance and does not depend on cohort data or the filesystem; callers cache it keyed by (PulseVersion, FeatureSetDigest, LimitsDigest).

func (*Pulse) ManifestForIntent 

func (p *Pulse) ManifestForIntent(ctx context.Context, intent string) (*descriptor.Manifest, error)

ManifestForIntent returns the instance's manifest narrowed to one intent-taxonomy ID (see Intents) — the bootstrap pulse_manifest {intent} serves, a small fraction of the full manifest. The operator-keyed sections (components, tests, post_tests, regressions, matrices, overlays, components_schemas) keep only operators whose purpose names the intent; skills is the intent's ranked list (SkillsForIntent); crosstab and matrix stay only when a remaining operator can use them. The fixed sections — commands, operations, error codes, cohort_types, mcp_tools, synth_distributions, extensions, limits and the facet / process_chain / join / export / import blocks — are dropped: zero here, absent on the wire. Scope carries the full intent record and Elided lists every dropped key. Version, digests, intents and return_presets are kept, so a client caches it under (pulse_version, feature_set_digest, limits_digest, intent).

It composes with the instance's feature profile: nothing the profile hides appears. An unknown intent, or one the profile hides, is PULSE_RECOMMEND_INTENT_UNKNOWN (details.valid lists the instance's intents). Manifest itself is unchanged. Observed as a manifest operation.

func (*Pulse) NewCohortBuilder 

func (p *Pulse) NewCohortBuilder(ctx context.Context, target string, schema encoding.Schema, opts CohortBuilderOptions) (*CohortBuilder, error)

NewCohortBuilder starts a single-file cohort at target — ungrouped (format 0x01) unless opts declares a group that survives the gate or elides constants (0x02) — or, with opts.Shards, a new shard archive (see ShardSplit), resolved against the instance filesystem (Options.FS / Options.DataDir), with the given schema.

The schema is the contract, as an explicit-schema import's is: field order, names, types, nullability, descriptions, decimal precision / scale and optional pre-seeded dictionaries (Field.Dictionary). The layout fields — ByteOffset, BitPosition, CsvColumnIdx — are RECOMPUTED from field order and types, so caller values are ignored, and the caller's schema and dictionaries are never mutated. Dictionaries grow from the pre-seeded entries in first-seen order; the declared rung is a ceiling, never auto-promoted.

The cohort is byte-identical to an import of the same rows with the same schema (pio.ImportJob with Schema set, and the same Groups / ElideConstants): the builder runs the import path's own dictionary assignment, row encoding, group gate and encoder. Shared checks are the description length and the row conversion; the schema-shape refusals below are the builder's own (import trusts its schema).

An anchored target — `archive.pulse#name.pulse` — instead builds ONE new shard and Close appends it to that EXISTING archive with Pulse.AddShard (see Close). The archive is never created implicitly (build a new one with ShardSplit), name must be a plain file name the archive does not hold yet, and the options that would shape the archive are refused: Groups, ElideConstants and Shards (the archive's group layout wins — AddShard re-encodes the shard to it), and Overwrite (an append never replaces anything). The spool and staging live beside the archive.

Refusals: SERVICE_VALIDATION for an empty or `.zst` target, an existing target without Overwrite, and a malformed schema (no fields, empty or duplicate names, unknown type, bad decimal precision / scale, a pre-seeded dictionary longer than its rung, parent groups in the schema); PULSE_IMPORT_DESCRIPTION_TOO_LONG for a description past 1000 bytes; PULSE_FIELD_DESCRIPTION_LOW_QUALITY under Strict; a bad group declaration (PULSE_GROUP_*) and, under Strict, a too-narrow group (PULSE_GROUP_TOO_NARROW); SERVICE_VALIDATION for a ShardSplit whose MaxRecords is not positive. For an anchored target: SERVICE_VALIDATION (details.reason) for a missing archive (archive_not_found), a file that is not a shard archive (not_an_archive), a malformed shard name (invalid_shard_name, anchored_target) and a refused option (append_option, details.option naming it); PULSE_SHARD_RESERVED_NAME for `_schema.pulse`; PULSE_SHARD_NAME_COLLISION for a name the archive already holds; and a schema AddShard is certain to refuse, checked up front against the archive's canonical schema with AddShard's own merge rules and refused with its code, message and details unchanged (PULSE_SHARD_SCHEMA_MISMATCH for a field name, count, non-set type, nullability or categorical width that diverges; PULSE_SHARD_DICT_WIDTH_OVERFLOW / PULSE_SHARD_DICT_DIVERGENCE for pre-seeded dictionaries that cannot merge). What AddShard accepts — set widening, dictionary union, the archive's group re-layout — is accepted here too, and Close re-runs AddShard as the authority.

func (*Pulse) Ontology 

func (p *Pulse) Ontology() descriptor.Ontology

Ontology returns the instance's skill ontology: the typed node / edge graph over the intent taxonomy, the registered operators (extensions included), the skill pack, the runnable examples, the glossary, the capabilities, the MCP tools and the named tables. It carries no prose — read bodies through Skill, ExampleGet and Glossary.

It is always the instance's PRUNED view: under a feature profile every node for a hidden surface — and every skill, example or intent left with nothing to describe — is absent, together with its edges, exactly as if it had never been registered. There is no full-graph variant. Each call returns a fresh deep copy the caller may mutate. Nodes are sorted by ID; edges by From, then Kind, then To.

func (*Pulse) Open 

func (p *Pulse) Open(ctx context.Context, path string) (*Cohort, error)

Open reads a .pulse file and returns a Cohort with the parsed schema.

Anchor syntax: a path of the form "archive.pulse#shard.pulse" opens the archive, locates the named shard inside it, and returns a single- shard cohort whose schema comes from the shard's own header (not the canonical schema in `_schema.pulse`). The returned Cohort has an empty Shards slice — anchor-resolved shards stand alone for the purposes of facade methods. Anchors require an archive backing the path; using `#` against a single-file `.pulse` raises PULSE_ARCHIVE_MAGIC_INVALID. A literal `#` in a filename is not supported in v1.

Anchor parsing happens inside internal/service.Service.Open as well, so the other facade methods (Process, Sample, Facet, ...) that receive an anchored Cohort path resolve consistently.

func (*Pulse) PayloadSchema 

func (p *Pulse) PayloadSchema() ([]byte, error)

PayloadSchema returns the instance's payload JSON Schema (draft 2020-12) as raw JSON: only what it offers. The operator, overlay-kind and regression enums list only enabled names; a hidden request slot (crosstab, joins, overlays) is not a property; a hidden capability's root (compose, process_chain, facet, sample, lookup) is not an entry point; and no def reachable only through an omitted part remains. Request, Response and Envelope are always present and $id is unchanged. The root $comment carries the instance's feature_set_digest ("feature_set_digest: fs1:…"), equal to FeatureSetDigest and to the manifest's, so the two self-descriptions cache under one key. Without a feature profile the output is the published full-registry schema.

func (*Pulse) Predict 

func (p *Pulse) Predict(ctx context.Context, req *Request) (*descriptor.PredictResult, error)

Predict validates a request against a .pulse file without executing it. It reads only the header and schema, never record data.

func (*Pulse) PredictBytes 

func (p *Pulse) PredictBytes(ctx context.Context, data []byte, req *Request) (*descriptor.Envelope, error)

PredictBytes validates req against an in-memory .pulse cohort — a single file or a whole shard archive, detected by its leading magic bytes — without executing it, and returns the descriptor envelope whole.

Unlike Predict, every predict option comes from the instance: the extension snapshot (so an embedder-registered operator is KNOWN, not flagged unknown), Options.Strict (warnings promoted to errors) and Options.EchoRequest (envelope.Request carries the normalized request). req.Cohort is not read — the bytes ARE the cohort.

A returned error is a nil req or a cancelled ctx; every validation fault comes back as env.Errors with PredictResult.Valid false.

func (*Pulse) PredictChain 

func (p *Pulse) PredictChain(ctx context.Context, req *ChainRequest) (*descriptor.Envelope, error)

PredictChain validates a ProcessChain request against its cohort without executing it: stage 0 over the cohort's schema, each later stage over the schema the one before it produces. It reads what PredictFacet reads and returns errors the same way; the envelope's Data is a *ChainPredictResult. A chain raises no advisories.

func (*Pulse) PredictCompose 

func (p *Pulse) PredictCompose(ctx context.Context, req *ComposedRequest) (*descriptor.Envelope, error)

PredictCompose validates a Compose batch without executing it: each slot is predicted over its own cohort (header and schema only, read through the runtime's opener) with the instance's options, then the batch overlays are checked. Each slot's advisories ride the result, the sidecar-fed ones included — every slot cohort's SPSS sidecar is read for its measure levels and weight variable.

The returned envelope is whole: its Data is a *ComposePredictResult whose Valid mirrors env.Errors emptiness, and each refusal is a coded env.Errors entry. A returned error is a cancelled ctx. Under a feature profile a hidden operator or slot predicts exactly as a never-registered one.

func (*Pulse) PredictFacet 

func (p *Pulse) PredictFacet(ctx context.Context, req *FacetRequest) (*descriptor.Envelope, error)

PredictFacet validates a facet request against its cohort without executing it — the cohort's header and schema only, never a record of a single-file cohort (a shard archive or anchored shard is read whole, as Predict reads it). The envelope's Data is a *FacetPredictResult.

A returned error is a cancelled ctx, a nil request or cohort (SERVICE_VALIDATION), or a cohort that cannot be read (DATA_FILE; ENCODING_INVALID for a malformed header or schema). Every request fault is a coded env.Errors entry with Valid false.

func (*Pulse) Process 

func (p *Pulse) Process(ctx context.Context, req *Request) (*Response, error)

Process executes a single processing request against a cohort.

A `return` block on req — else the instance default (Options.DefaultReturn, then the feature profile's `return`) — shapes the response here, at the outermost facade, after the engine finished: excluded slots are pruned from the Go value and absent from its JSON, and Response.Returned stamps the selection. No block on any layer (or one resolving to the full response) returns the response untouched.

func (*Pulse) ProcessChain added in v0.10.0 

func (p *Pulse) ProcessChain(ctx context.Context, req *ChainRequest) (*ChainResponse, error)

ProcessChain executes a source-rooted linear chain of Process requests. The first stage runs against the cohort identified by req.Cohort; each subsequent stage receives the previous stage's rows as its input. All stages must be mergeable per the v1 chain gate (an extension aggregator qualifies when its registration declares Mergeable); a non-mergeable stage surfaces PULSE_CHAIN_NOT_MERGEABLE with the offending stage index so the caller can fall back to per-stage Process calls.

Why chain? Building a linear plan as N independent Process calls pays N file-open + schema-rebind + per-stage validate costs. ProcessChain collapses those into one open + one synth-schema per intermediate stage, keeping the streaming iterator stack alive across stages.

Shaping: each Stages[i].Request.Return (else the instance default) shapes Stages[i], applied only after the WHOLE chain completed — a stage that excludes `data` still feeds the next stage its rows. Final follows the last stage's return; there is no chain-level knob.

func (*Pulse) ProcessStream added in v0.2.0 

func (p *Pulse) ProcessStream(ctx context.Context, req *Request) (RowIter, error)

ProcessStream executes a request and returns a pull-based row iterator over the result. Equivalent to Process for any request shape — same gates, same errors — but streaming consumers (HTTP responders, NDJSON writers, downstream pipelines) can drain rows one at a time without buffering the full result in their own memory.

Predict's Streamable flag reports whether the underlying execution avoids buffering inside the engine; ProcessStream wraps the result regardless, so the API is stable for non-streamable requests too.

Shaping: the request's `return` block (else the instance default) shapes the stream exactly as Process shapes the buffered response — each row is a pruned clone equal to the shaped response's `data` element (no rows when `data` is excluded), Components / Metadata are shaped copies (the run keeps accumulating an excluded slot), and the iterator reports the `returned` marker only once exhausted, through a `Returned() *types.ReturnedMarker` method. Precision is wire-only: rows stay full float64, and the shaped iterator's `MarshalRow(Row) ([]byte, error)` writes one at the plan's precision (the `pulse api process --stream` NDJSON writer uses it). With no effective block the service iterator is returned untouched.

Observability: the operation starts at the call and ends exactly once — when the iterator is drained (or fails), closed, or ctx is cancelled, whichever comes first. Its scan phase runs from the first Next to that end, so it includes the caller's own time between Next calls. An iterator that is never drained, closed or cancelled never ends the operation (a leaked iterator is a caller bug).

func (*Pulse) ProcessStreamResult added in v0.11.0 

func (p *Pulse) ProcessStreamResult(ctx context.Context, req *Request) (StreamResult[Row], error)

ProcessStreamResult executes req and returns a StreamResult that yields each result row as a StreamChunk[Row]. The header is built before any rows are read; the chunks channel closes when the operation completes, and the Done channel delivers exactly one StreamTerminator describing the terminal state.

Cancellation: closing ctx delivers a StreamTerminator with Status: StreamCancelled and a non-nil Error matching ctx.Err().

Options.Limits.RequestTimeout bounds the whole call — the run and the drain. A deadline that fires during the run returns the coded PULSE_LIMIT_EXCEEDED error directly; one that fires mid-stream delivers Status: StreamErrored carrying that coded error (never StreamCancelled, which stays the caller's own cancel or deadline).

Observability: the operation starts at the call and ends exactly once, just before the terminator is delivered — completed, errored or cancelled. Its scan phase runs from the first row read to that end, so it includes the time the producer waits on a slow consumer (the Chunks buffer is small, so backpressure reaches it quickly). A stream whose Chunks are never drained and whose ctx is never cancelled never ends the operation (a caller bug).

func (*Pulse) Profile added in v0.2.0 

func (p *Pulse) Profile(ctx context.Context, path string, opts ProfileOptions) (*Profile, error)

Profile reads a .pulse file at path and returns a statistical summary suitable for from-profile synthesis. The profile retains no individual rows from the source data.

func (*Pulse) RangeTables added in v0.27.0 

func (p *Pulse) RangeTables() []RangeTableInfo

RangeTables returns the registered range tables in name order. Each entry carries the table's range count and its ordered {label, start, end} ranges — the INPUT-direction discovery surface a caller consults before authoring a GROUP_DATE_RANGES / FILTER_DATE_RANGES request that references the table by name. Returns nil when none are registered.

func (*Pulse) Recommend 

func (p *Pulse) Recommend(ctx context.Context, req descriptor.RecommendRequest) (*descriptor.RecommendResult, error)

Recommend turns a question kind into ranked draft requests. req.Intent is an intent-taxonomy ID (Intents); an unknown one is the coded error PULSE_RECOMMEND_INTENT_UNKNOWN, whose details.valid lists every ID.

Without a cohort the answer is UNBOUND (Bound false): for each operator the instance offers that serves the intent, a request skeleton whose caller-supplied values are "<placeholder>" strings named after their wire keys, a why sentence from the operator's purpose, the intent's field shapes, and the alternatives and follow-ups its guidance declares. Nothing is predict-run. A non-analytic intent (prepare, simulate, lookup) or one no operator serves yet returns an empty Recommendations list plus RoutesTo, the tooling that answers it instead — that is not an error.

With req.Cohort the answer is BOUND: each serving operator's slots are bound to the cohort's fields — req.Fields hints pin roles, the rest take schema order, at most three bindings per operator — and every draft is validated by an in-process predict carrying the instance's options and the cohort's sidecar facts. A draft predict refuses is dropped; a survivor carries predict's advisories. A draft that still needs a value only the caller can choose (a success value, a reference mean, a model family, a percentile) keeps its bound fields with Bound false and Needs naming each value, and ranks after a fully bound draft of the same hint, level and advisory standing. A hint that is not a cohort field, or whose kind no role of the intent takes, is SERVICE_VALIDATION; a cohort that cannot be read is DATA_FILE (ENCODING_INVALID for a malformed header or schema). Bound mode reads the header, the schema and the sidecar only — never a record of a single-file cohort; a shard archive (or an anchored shard) is read whole, as Predict reads it.

Under a feature profile no hidden operator or capability appears in a recommendation, an alternative, a follow-up or a route. Recommend is not an observed operation: its draft predicts run in-process and fire no predict hooks or metrics.

func (*Pulse) ReloadTemplates added in v0.28.0 

func (p *Pulse) ReloadTemplates() error

ReloadTemplates rescans every configured template directory immediately and swaps in the result.

It is an escape hatch, not the mechanism. Templates hot-reload on their own: a lookup whose cached snapshot has aged past the store's rescan interval re-walks the directories first, so a file dropped into a scanned directory becomes renderable, a changed file starts serving its new content, and a deleted one stops resolving — all without restarting the process. The interval is a package constant rather than an Option, because a dial nobody can set better than the store can is a permanent public surface bought for nothing.

What the interval cannot give is determinism. A newly written file can be invisible for up to that interval, which is fine for an operator editing a directory and wrong for a deployment step that writes a template and must render it on the next line, or for a test that would otherwise have to sleep. ReloadTemplates covers exactly that case.

A rescan is a directory walk plus one stat per candidate file; a file whose size and modification time both match the copy already parsed is carried over rather than re-read, so calling this on an unchanged directory costs syscalls and no JSON parsing.

A file going bad after startup degrades PER FILE and does not come back from this call. A template that parsed once and whose file later becomes malformed keeps serving its last-good parse, every other template is untouched, and this returns nil: an error here would tell a caller its whole catalog failed over one half-written editor save. The broken state is observable through ListTemplates instead — Summary.Broken with the fault in Summary.Error — and through GetTemplate for a name that never parsed at all. Repairing the file clears it on the next rescan. An unreadable file degrades the same way; it is also one file.

What this DOES return is a whole-walk fault: a configured root that exists but is not a directory, or a directory that cannot be walked. Those are misconfigurations rather than transient edits, and a failed walk leaves the previously loaded templates entirely in place.

Startup keeps the opposite rule: pulse.New still fails outright on a malformed template, because at startup a broken document is a deploy error the operator should see immediately rather than a keystroke.

An engine with no template directories configured has nothing to rescan and returns nil.

func (*Pulse) RemoveShard added in v0.8.0 

func (p *Pulse) RemoveShard(ctx context.Context, archivePath, shardBasename string) error

RemoveShard rewrites the archive omitting the named shard. The canonical schema is preserved (dictionary entries are never shrunk). Returns PULSE_SHARD_MISSING when the named shard is not in the archive.

func (*Pulse) RenderTemplate added in v0.28.0 

func (p *Pulse) RenderTemplate(name string, vars map[string]any) (*RenderedTemplate, error)

RenderTemplate resolves the named template and renders it against the supplied variable map, returning the substituted JSON plus the typed request it decoded into. It is the general form, covering all five targets; RenderTemplateRequest is the shorthand for the common one.

Exactly one of Rendered's typed pointers is populated, selected by Rendered.Target — read that pointer (or Rendered.Typed()) and hand it to the matching execution method: Process, Compose, ProcessChain, FacetSchema, SampleWithRequest. There are deliberately no per-execution- mode convenience wrappers: N execution modes would mean N wrappers to keep in sync forever, for no capability gain.

Rendered.JSON is the rendered body before decode, retained because re-marshaling the typed value would not reproduce it — every request struct is dense with omitempty, so a slot that rendered to an explicit zero would silently vanish from a round trip.

Errors are the PULSE_TEMPLATE_* family, surfaced with their codes and details intact: PULSE_TEMPLATE_NOT_FOUND for an unknown name, then whatever the render raises — PULSE_TEMPLATE_VAR_MISSING / _VAR_UNKNOWN / _VAR_TYPE / _VAR_ENUM for the variable map, PULSE_TEMPLATE_UNRESOLVED for a marker with nothing to substitute, and PULSE_TEMPLATE_RENDER_INVALID when the substituted JSON does not fit the target request type.

Rendering never opens a cohort: a template that renders is well-formed against the request SHAPE. Whether it is executable against a particular cohort stays Predict's question.

On an engine whose feature profile hides a request slot (crosstab, joins, overlays), a rendered body that sets that slot is refused with the PULSE_TEMPLATE_RENDER_INVALID a key the target type does not declare gets — same message, details and document-order position. An operator name is not a slot: a hidden operator renders and fails at execution, exactly as a never-registered one does.

func (*Pulse) RenderTemplateRequest added in v0.28.0 

func (p *Pulse) RenderTemplateRequest(name string, vars map[string]any) (*Request, error)

RenderTemplateRequest renders the named template and returns the typed *Request directly. It is the 95% path — the caller hands the result straight to Process, Predict, or ProcessStream.

It is RenderTemplate restricted to the "request" target. A template declaring any other target is PULSE_TEMPLATE_TARGET_UNKNOWN: the target is a valid one, just not one this method can return, so the message names both the target the template actually declares and RenderTemplate as the method that handles it. Every other fault is RenderTemplate's, unchanged.

func (*Pulse) ResolveCanonicalSchema added in v0.8.3 

func (p *Pulse) ResolveCanonicalSchema(ctx context.Context, src string) (*encoding.Schema, error)

ResolveCanonicalSchema returns the canonical encoding.Schema for the cohort at src without streaming records. Resolves single-file, shard-archive, and `archive#shard` anchor inputs identically to the rest of the facade.

Useful for callers that need to build an include-set before invoking FilterToFileBySetAndExpr: the set loader requires the schema to pick the best MemberSet impl (bitset / uint64 / string) for the field's type.

func (*Pulse) ResolveImport added in v0.5.0 

func (p *Pulse) ResolveImport(ctx context.Context, handle string) (string, error)

ResolveImport returns the managed-pool path for a handle, or PULSE_IMPORT_SOURCE_MISSING when no such handle exists. Embedders who want to address a managed handle by name (instead of by path) run path through this resolver before passing it to Inspect/Process.

func (*Pulse) ResolveLabel added in v0.11.2 

func (p *Pulse) ResolveLabel(table, query string, limit int) ([]LabelMatch, error)

ResolveLabel reverse-searches a registered label table: given a human-readable query — including a minor misspelling — it returns up to limit matching (key, value) pairs ranked by a confidence Score in [0,1]. Matching is typo-tolerant: the query and each label are normalized (lowercased, punctuation folded to spaces, whitespace collapsed), and the score is the best of an exact/prefix/substring match, a Damerau edit-distance ratio (catches typos and adjacent transpositions), and a character-trigram Dice coefficient (catches word reordering and partial overlap). An exact key match also scores 1.0.

Candidates scoring at or above an internal floor are returned best first. When nothing clears the floor the closest few are returned anyway (with their low scores) so the caller can present "did you mean" options rather than a dead end. An empty query returns the first limit rows in label order (a cheap "browse" mode, Score 0). limit <= 0 defaults to 10.

Returns PULSE_LABEL_TABLE_UNKNOWN when the table is not registered and PULSE_LABEL_TABLE_NOT_ENUMERABLE when the table exposes only a Lookup closure (no static Rows to scan).

func (*Pulse) Sample 

func (p *Pulse) Sample(ctx context.Context, path string, n int) ([]Record, error)

Sample returns up to n rows from the cohort as maps of field name to value.

func (*Pulse) SampleWithRequest added in v0.10.1 

func (p *Pulse) SampleWithRequest(ctx context.Context, req *SampleRequest) (*SampleResult, error)

SampleWithRequest is the labelled variant of Sample. The req struct carries the cohort path (via Cohort.Filename), the row cap (N), and LabelBinding entries that translate categorical values to display labels in the returned rows.

When Labels is empty the call degenerates to the same shape as Sample, returning a SampleResult with no Warnings and no transformation applied to the rows.

func (*Pulse) Skill 

func (p *Pulse) Skill(name string) (string, bool)

Skill returns the markdown body of the named skill (no ".md" suffix) as the instance serves it — the body pulse_skills_get and a pulse-skill://<name> read return. A skill the instance hides answers ("", false), exactly as a name that does not exist. Under a feature profile an atomic (operator, tool, type) body drops the lines naming a hidden operator or tool, and the "intents" / "glossary" bodies render only what the instance keeps.

func (*Pulse) Skills 

func (p *Pulse) Skills() []SkillMetadata

Skills returns the metadata of every skill the instance exposes, sorted by name: the embedded skill pack plus the virtual "glossary" and "intents" skills. Under a feature profile a skill for a hidden surface is absent, and each listed skill's description and applies_to / covers / examples_tags entries are rendered so they name no hidden operator or MCP tool. It is the list pulse_skills_list and the pulse-skill:// resource enumeration serve. Each call returns a fresh slice the caller may mutate.

func (*Pulse) SkillsForIntent 

func (p *Pulse) SkillsForIntent(intent string) ([]SkillMetadata, error)

SkillsForIntent returns the skills relevant to one intent-taxonomy ID (see Intents), ranked — the list pulse_skills_list {intent} serves. Order: the skill documenting the intent itself (the virtual "intents" skill), then the design skills whose covers names an operator serving the intent or its category (TEST, AGG, ...), by name, then those operators' atomic skills, simplest first by the operator's Purpose.Level (basic, intermediate, advanced, then operators with none), then by name. Every edge is read off the instance's pruned ontology, so under a feature profile a hidden operator — and its skill — takes no part, and each entry is the rendered metadata Skills lists. An intent no operator serves answers its reference skill only.

An unknown intent, or one the instance's feature profile hides, is PULSE_RECOMMEND_INTENT_UNKNOWN (details.valid lists the instance's intents) — the same refusal ExamplesSearchWith and Recommend give. Skills itself is unchanged. Never nil on success; each call returns a fresh slice the caller may mutate.

func (*Pulse) SweepImports added in v0.5.0 

func (p *Pulse) SweepImports(ctx context.Context) ([]string, error)

SweepImports removes every expired managed handle and returns the list of swept handle names. Invoked opportunistically by ImportFile and exposed here for callers that want explicit control (CLI maintenance, periodic ticker, etc.).

func (*Pulse) Synth added in v0.2.0 

func (p *Pulse) Synth(ctx context.Context, spec *SynthSpec, output string, opts SynthOptions) (*SynthResult, error)

Synth materializes a synthetic .pulse file at output from spec. The generator is deterministic for a given (spec, opts.Seed) pair: same seed produces a byte-identical file.

Setting opts.SourceCohort activates the tagged top-up path used by `synth from-profile`: output becomes source's real rows (tagged _synthetic=false) plus spec.RowCount newly generated rows (tagged _synthetic=true), output must be a path distinct from SourceCohort, and SourceCohort itself is never opened for write. Leaving SourceCohort empty (the default) is unchanged plain synthesis, used by `synth from-schema` and any caller that built the Spec directly.

Setting opts.FidelityReportPath alongside opts.SourceCohort additionally writes a JSON fidelity report after generation completes: a per-field marginal comparison between the source and generated partitions of output, driven through TEST_KS (numeric fields) / TEST_CHISQ (categorical fields) against the tagged _synthetic column rather than new comparison math, plus — when spec carries any Correlations — a pairwise numeric-numeric correlation delta section comparing each pair's captured/target rho against its realized rho in the newly generated partition (synth.PairwiseFidelity), plus — when spec carries any CategoricalPairs / CategoricalNumericPairs — the categorical-categorical contingency-table delta and categorical-numeric conditional-mean/std delta sections (synth.CategoricalPairFidelity / synth.CategoricalNumericPairFidelity, E3-S4). opts.FidelityWarnings, when set, is copied verbatim onto the report's Warnings slot — the mechanism `synth from-profile` uses to surface Profile.Warnings (e.g. a thin numeric/categorical-pair warning) beside the pairwise deltas they qualify, across all three pair kinds. Ignored when SourceCohort is empty — the plain synthesis path has no _synthetic partition to compare against.

func (*Pulse) SynthStream added in v0.11.0 

func (p *Pulse) SynthStream(ctx context.Context, spec *SynthSpec, opts SynthOptions) (StreamResult[Row], error)

SynthStream generates synth respondents as a StreamResult. Chunks carry one map[string]any per generated row, the same shape returned by the buffered Synth path. EstimatedTotal mirrors spec.RowCount.

The current implementation drives the buffered synth pipeline behind the scenes and yields rows from the materialized result; it does not expose true row-at-a-time generation yet but is API-stable so callers can adopt the streaming shape today.

Observability is ProcessStreamResult's: the operation ends exactly once, just before the terminator is delivered, and its scan phase (generation and emission) includes any backpressure wait.

func (*Pulse) VerifyIndex added in v0.27.0 

func (p *Pulse) VerifyIndex(ctx context.Context, path string, keyFields []string) (*VerifyIndexResult, error)

VerifyIndex reports whether the sidecar point-lookup index built for keyFields against the cohort at path is still fresh, using the size+mtime fast-path before paying for a full content-hash recompute. Delegates to internal/service.Service.VerifyIndex — see that method's doc comment for the full fast-path decision tree. Returns PULSE_INDEX_MISSING when no sidecar exists for keyFields and PULSE_INDEX_UNSUPPORTED_SHARDED for shard archive cohorts.

func (*Pulse) VerifyShardArchive added in v0.8.0 

func (p *Pulse) VerifyShardArchive(ctx context.Context, archivePath string) (*VerifyResult, error)

VerifyShardArchive opens the archive and re-validates every shard's header (magic + format_version), structural cohesion against the canonical schema, dictionary prefix rule, and cross-checks each shard's record count against the canonical aggregate. Returns a VerifyResult carrying any errors (PULSE_SHARD_HEADER_INVALID, PULSE_SHARD_SCHEMA_MISMATCH, PULSE_SHARD_DICT_DIVERGENCE) and any non-fatal warnings (PULSE_SHARD_DESCRIPTION_DIVERGENCE, aggregate drift). Returns a non-nil error only when the archive itself cannot be opened (archive corrupt, file missing, etc.); per-shard issues are reported through the result struct so the caller can render the full diagnosis.

func (*Pulse) Watch added in v0.11.0 

func (p *Pulse) Watch(ctx context.Context, target string) <-chan ChangeEvent

Watch observes the cohort path on the Pulse instance's filesystem and returns a channel of ChangeEvent records. Closing the context drains and closes the channel.

func (*Pulse) WatchDir added in v0.11.0 

func (p *Pulse) WatchDir(ctx context.Context, dir string, recursive bool) <-chan ChangeEvent

WatchDir observes every file under dir (and subdirectories when recursive=true) for change events, filtered to WatchOptions.Suffix (defaults to ".pulse"). Closing the context drains and closes the channel.

func (*Pulse) WatchDirWithOptions added in v0.11.0 

func (p *Pulse) WatchDirWithOptions(ctx context.Context, dir string, opts WatchOptions) <-chan ChangeEvent

WatchDirWithOptions is the configurable variant of WatchDir.

func (*Pulse) WatchWithOptions added in v0.11.0 

func (p *Pulse) WatchWithOptions(ctx context.Context, target string, opts WatchOptions) <-chan ChangeEvent

WatchWithOptions is the configurable variant of Watch. PollInterval and CoalesceWindow control responsiveness vs CPU; HashPrefixBytes caps the per-event hash size.

func (*Pulse) WidenSetField added in v0.38.0 

func (p *Pulse) WidenSetField(ctx context.Context, path, field, targetType string) (*WidenReport, error)

WidenSetField widens the set column named field in the cohort at path to the wider set rung named by targetType ("set_u16", "set_u32", "set_u64", "set_u128", "set_u256"), rewriting the cohort IN PLACE and returning a report of what moved.

A widen changes the field's stride, so every record is re-laid-out and every field after the widened one moves. The rewrite is atomic — temp file beside the cohort, fsync, rename — so any failure leaves the original byte-identical; see internal/encoding.WidenSetFieldFile.

targetType is a type NAME rather than an encoding.FieldType because this is the boundary where a caller-supplied string arrives (a CLI flag, an embedder's config), and the resolution rule belongs to the library: an unknown name is ENCODING_TYPE_MISMATCH, never a silent fallback to some default type. Callers holding an encoding.FieldType pass its String(); the name table round-trips by contract.

Refusals are coded errors reusing existing codes — SERVICE_RESOURCE (no such cohort), SERVICE_VALIDATION (the path is a shard archive or an anchored shard within one), ENCODING_INVALID (no such field) and ENCODING_TYPE_MISMATCH (not a set, not wider, or already at the widest rung). See internal/service.Service.WidenSetField.

Sidecars are not rebuilt: a widened cohort changes length, so the point-lookup index and the SPSS metadata sidecar invalidate themselves through their own fingerprints on the next read.

type RangeTable added in v0.27.0 

type RangeTable struct {
	// Description is an optional human-readable summary surfaced in the
	// manifest projection.
	Description string
	// Ranges is the ordered list of labeled date ranges. Must be
	// non-empty and pass the shared validation pass.
	Ranges []DateRangeSpec
}

RangeTable is a named, ordered set of labeled date ranges. It is the reusable, register-once form of the inline range list authored on a GROUP_DATE_RANGES grouper or FILTER_DATE_RANGES filter: a caller registers the table under a name and references it by that name from any number of requests.

Ranges are the ordered {label, start, end} entries. Start / End are ISO date literals (nil / empty means an open bound; both bounds are inclusive). The set is validated at pulse.New time via the shared range-compilation pass — overlap, duplicate label, empty set, and unparseable / inverted boundaries each surface the matching PULSE_RANGE_* code. Order is preserved as authored; the compiler sorts internally for deterministic matching.

type RangeTableInfo added in v0.27.0 

type RangeTableInfo struct {
	Name       string          `json:"name"`
	RangeCount int             `json:"range_count"`
	Ranges     []DateRangeSpec `json:"ranges"`
}

RangeTableInfo is one registered range table as surfaced by RangeTables — the discovery companion for the GROUP_DATE_RANGES / FILTER_DATE_RANGES operators. It carries the table name, its labeled- range count, and the ordered ranges themselves so a caller can turn a table name into the {label, start, end} tuples an operator resolves.

type Record 

type Record = map[string]any

Record is a row of field→value data returned by Sample.

type RenderedTemplate 

type RenderedTemplate = template.Rendered

RenderedTemplate is a RenderTemplate result: the substituted JSON plus the decoded, validated request for the template's target.

type Request 

type Request = types.Request

Type aliases re-exported from the types package so embedders can use pulse.Request instead of types.Request.

type Response 

type Response = types.Response

Type aliases re-exported from the types package so embedders can use pulse.Request instead of types.Request.

type Row added in v0.2.0 

type Row = service.Row

Row is a single result row in a processing stream.

type RowIter added in v0.2.0 

type RowIter = service.RowIter

RowIter is a pull-based iterator over a processing result. Each call to Next returns the next row or (nil, false, nil) on exhaustion. Close releases underlying resources. Metadata returns the run metadata once available (always present after the iterator is drained).

type SampleRequest added in v0.10.1 

type SampleRequest = types.SampleRequest

SampleRequest is the labelled variant of the Sample entry point.

type SampleResult added in v0.10.1 

type SampleResult struct {
	Rows     []Record
	Warnings []SampleWarning
}

SampleResult bundles the rows and the resolver-side warnings from a labelled Sample call. Warnings carry PULSE_LABEL_COLLISION and PULSE_LABEL_LOOKUP_MISS records; an empty slice means the labels resolved cleanly (or no Labels were requested).

type SampleWarning added in v0.10.1 

type SampleWarning struct {
	Code    string
	Message string
	Details map[string]any
}

SampleWarning is the envelope-ready projection of a single resolver warning. The shape mirrors descriptor.EnvelopeEntry so callers can fold it into a descriptor.Envelope at the CLI / MCP boundary.

type SeriesOverlayRequest added in v0.36.0 

type SeriesOverlayRequest struct {
	// Points is the series IN AXIS ORDER. The windowed kinds are
	// order-dependent — "prior" means the preceding element of THIS slice —
	// so a caller that assembles points out of order gets a well-formed
	// answer to a different question. Sort before calling.
	Points []SeriesPoint

	// Overlays is the spec list, applied in order. One layer per spec comes
	// back in matching order.
	Overlays []types.OverlaySpec
}

SeriesOverlayRequest is the input to Pulse.ApplySeriesOverlays.

type SeriesOverlayResult added in v0.36.0 

type SeriesOverlayResult struct {
	Layers   []types.OverlayLayer
	Warnings []types.OverlayWarning
}

SeriesOverlayResult is what ApplySeriesOverlays returns: one layer per requested spec, in matching order, plus the flat warning slice the fold produced.

type SeriesPoint added in v0.36.0 

type SeriesPoint struct {
	// Key is the group key this point sits at — the ordered axis position,
	// carried through to the emitted SeriesEntry so a caller can align
	// layers back onto its own series.
	Key types.AxisKey

	// Value is the host metric's value at this ordinal. Read only when
	// Present is true.
	Value float64

	// Present reports whether this ordinal has a value at all.
	Present bool
}

SeriesPoint is one ordered point of a caller-materialised series.

Present distinguishes "this ordinal has no value" from "this ordinal's value is zero", which the windowed kinds treat very differently: an absent point does not advance a lag carrier, where a present zero does. A caller reading stored cells sets Present=false for a period that holds no row — never Value=0, which would assert that the measured value was zero.

type SetWidening added in v0.38.0 

type SetWidening = service.SetWidening

SetWidening records one set field promoted to a wider rung during a CreateShardArchive or an AddShard, with the shard and record counts the rewrite cost. From == To marks the one-shard case: the arriving shard declared a narrower rung and was promoted to the archive's, leaving the archive itself untouched.

type SetWidthHeadroom added in v0.38.0 

type SetWidthHeadroom = encoding.SetWidthHeadroom

SetWidthHeadroom reports how much of a set field's bitmask capacity the canonical dictionary has consumed, and which rung a widen would promote it to. VerifyShardArchive returns one per set field so an impending archive-wide widen is foreseeable rather than a surprise the next `shard add` bills for.

type ShardEntry added in v0.8.0 

type ShardEntry = service.ShardEntry

ShardEntry is one shard inside a Pulse shard archive. Re-exported from service so embedders can address pulse.ShardEntry directly.

type ShardInfo added in v0.8.0 

type ShardInfo = descriptor.ShardInfo

ShardInfo is one shard entry as surfaced by Inspect / Predict. Re- exported from descriptor so embedders consuming the no-execute surface can address pulse.ShardInfo directly. Mirrors ShardEntry's shape (filename + record count); the two types are parallel because descriptor/ cannot import internal/service/.

type ShardSplit 

type ShardSplit struct {
	// MaxRecords is the number of rows per shard.
	MaxRecords int
}

ShardSplit makes a CohortBuilder write a NEW shard archive, split automatically: the appended rows are cut, in order, into consecutive shards of MaxRecords rows — part-00001.pulse, part-00002.pulse, … — the last one partial (an empty build is one empty shard), and Close publishes them with ONE Pulse.CreateShardArchive call, so the archive appears whole or not at all. Groups and ElideConstants are decided ONCE over all rows and applied to every shard: each shard carries the same schema, dictionaries and parent-group layout, and a group the gate drops is dropped in every shard. MaxRecords must be positive (SERVICE_VALIDATION otherwise); a build holds at most 65,535 shards (the archive's shard-count limit), and the row that would open one more is refused with SERVICE_VALIDATION.

type SidecarIndex 

type SidecarIndex = encx.Index

SidecarIndex is the in-memory point-lookup sidecar index carried by BuildIndexResult.Index: the source cohort's content fingerprint, the ordered key spec, the hash-bucket table and the source-stat snapshot (size + modification time) taken at build time.

type SidecarIndexBucket 

type SidecarIndexBucket = encx.IndexBucket

SidecarIndexBucket is one hash bucket of a SidecarIndex (SidecarIndex.Buckets): the entries whose key hashed to it.

type SidecarIndexEntry 

type SidecarIndexEntry = encx.IndexEntry

SidecarIndexEntry is one distinct key inside a SidecarIndexBucket: the key's raw on-wire bytes and the record IDs that carry it.

type SidecarIndexKeySpec 

type SidecarIndexKeySpec = encx.IndexKeySpec

SidecarIndexKeySpec is one ordered key column of a SidecarIndex (SidecarIndex.Keys): the column name and its field type.

type SkillMetadata 

type SkillMetadata = skills.Metadata

SkillMetadata is one skill-pack entry as Skills lists it — name, description, kind and the frontmatter cross-references.

type StaleSidecar added in v0.38.0 

type StaleSidecar struct {
	// Path is the sidecar's path, derived the same way its owner
	// derives it. Never guessed from a filename pattern.
	Path string `json:"path"`
	// Kind is one of the SidecarKind* constants.
	Kind string `json:"kind"`
	// Keys is the point-lookup index's ordered key tuple, empty for
	// every other kind. Order is significant: a reversed tuple is a
	// different index at a different path.
	Keys []string `json:"keys,omitempty"`
	// Rebuild is the command that regenerates this sidecar.
	//
	// For a point-lookup index it is exact and runnable, which is the
	// whole reason the key tuple has to be recovered first: the
	// sidecar's filename is a HASH of that tuple, so the command cannot
	// be reconstructed from the path. For the SPSS metadata sidecar it
	// names the import that produces one and leaves the source `.sav`
	// as a placeholder — the sidecar holds dictionary detail the
	// `.pulse` format has no slot for, so it cannot be regenerated from
	// the cohort at all, and pretending otherwise would be worse than
	// the placeholder.
	Rebuild string `json:"rebuild"`
}

StaleSidecar names one sidecar document beside a cohort that a cohort-rewriting operation has invalidated.

Nothing here rebuilds anything, and that is the decision rather than an omission: a widen is a fast, bounded metadata operation, and rebuilding an index over a multi-million-row cohort behind it would turn it into an arbitrarily long one with no way for the caller to opt out. Both sidecars refuse to serve stale data on their own — their fingerprints cover the cohort's SIZE, which a widen always changes — so the risk being closed is not silent corruption but silent BREAKAGE: a corpus that stops answering lookups with nothing having said why.

type StreamChunk added in v0.11.0 

type StreamChunk[T any] struct {
	Sequence   int
	Data       T
	Progress   float64
	Components *types.ResponseComponents `json:"components,omitempty"`
	Returned   *types.ReturnedMarker     `json:"returned,omitempty"`
}

StreamChunk wraps a single incremental payload. Sequence is a monotonic 0-based counter so consumers can detect gaps or reorder out-of-band telemetry; Progress reports a best-effort fraction in [0.0, 1.0] (or -1.0 when unknown).

Components, when populated, carries the per-operator constituent-parts metadata at chunk boundary — same shape as Response.Components on a buffered Process run. Mergeable / partial aggregators surface their running state on every chunk so consumers can render mid-stream; non-mergeable operators (AGG_MEDIAN, AGG_PERCENTILE) appear only on the terminal chunk (last row before close), as do a grouped run's per-group aggregation figures (components.aggregations[*].groups). nil when the underlying operation produced no Components payload (e.g. SynthStream).

Returned, set on the TERMINAL chunk only, is the `return` marker of a stream shaped by a non-identity `return` block (see Pulse.ProcessStream); nil — and absent on the wire — otherwise.

type StreamHeader added in v0.11.0 

type StreamHeader struct {
	// RequestHash is the canonical hash of the originating request,
	// as returned by Request.Hash / SynthSpec.Hash. Identifies the
	// stream's input for caching and deduplication.
	RequestHash string

	// EstimatedTotal is the best-effort row-count estimate for the
	// stream. -1 when unknown (predict not available, source is a
	// live cohort whose size is dynamic, etc.).
	EstimatedTotal int64

	// StartedAt is the wall-clock time the stream began producing.
	StartedAt time.Time
}

StreamHeader is the single-shot prelude describing what the stream is about to produce. Sent before any chunks.

type StreamResult added in v0.11.0 

type StreamResult[T any] struct {
	Header StreamHeader
	Chunks <-chan StreamChunk[T]
	Done   <-chan StreamTerminator
}

StreamResult is the canonical streaming-output shape. Every operation that produces incremental data exposes a *Stream variant returning StreamResult[T]. The receiver pattern: 

res, err := p.ProcessStreamResult(ctx, req)
if err != nil { ... }
for chunk := range res.Chunks {
    ... handle chunk.Data
}
term := <-res.Done
if term.Status != pulse.StreamCompleted { ... }

Backpressure: the Chunks channel carries a small buffer (4). Slow consumers slow the producer; the producer never drops chunks.

type StreamStatus added in v0.11.0 

type StreamStatus int

StreamStatus reports the terminal state of a StreamResult. 

const (
	// StreamCompleted means the operation produced all chunks and
	// finished without error.
	StreamCompleted StreamStatus = iota
	// StreamCancelled means the caller's context was cancelled or
	// timed out before the operation finished.
	StreamCancelled
	// StreamErrored means the operation aborted mid-stream. The
	// terminator's Error field carries the underlying cause.
	StreamErrored
)

func (StreamStatus) String added in v0.11.0 

func (s StreamStatus) String() string

String returns a stable lowercase identifier for the status, suitable for logging and JSON marshalling.

type StreamTerminator added in v0.11.0 

type StreamTerminator struct {
	CompletedAt time.Time
	TotalRows   int64
	Status      StreamStatus
	Error       error
}

StreamTerminator is the single-shot epilogue describing how the stream ended. Sent after the chunks channel closes; the Done channel itself is closed immediately after delivering this value so a receiver that selects on Done sees one value and then channel-closed.

type SynthOptions added in v0.2.0 

type SynthOptions = synth.Options

SynthOptions modulate the deterministic seed and other knobs.

type SynthResult added in v0.2.0 

type SynthResult = synth.Result

SynthResult is the result of a successful Synth call.

type SynthSpec added in v0.2.0 

type SynthSpec = synth.Spec

SynthSpec is the parsed synthesis request shape.

type Template 

type Template = template.Template

Template is one parsed request-template document (GetTemplate).

type TemplateSummary 

type TemplateSummary = template.Summary

TemplateSummary is one ListTemplates entry.

type TemplateTarget 

type TemplateTarget = template.Target

TemplateTarget names the request root a template renders into. 

const (
	// TemplateTargetRequest renders into Request (process / predict).
	TemplateTargetRequest TemplateTarget = template.TargetRequest
	// TemplateTargetComposed renders into ComposedRequest (Compose).
	TemplateTargetComposed TemplateTarget = template.TargetComposed
	// TemplateTargetChain renders into ChainRequest (ProcessChain).
	TemplateTargetChain TemplateTarget = template.TargetChain
	// TemplateTargetFacet renders into FacetRequest (the facet endpoints).
	TemplateTargetFacet TemplateTarget = template.TargetFacet
	// TemplateTargetSample renders into SampleRequest (record sampling).
	TemplateTargetSample TemplateTarget = template.TargetSample
)

TemplateTarget values — the closed set of request roots a template can render into. Each equals its internal counterpart, so a Template.Target, TemplateSummary.Target or RenderedTemplate.Target compares against these directly; no embedder needs to match on String().

type TemplateVarType 

type TemplateVarType = template.VarType

TemplateVarType is a template variable's declared type. 

const (
	// TemplateVarString accepts any JSON string.
	TemplateVarString TemplateVarType = template.VarString
	// TemplateVarNumber accepts any JSON number.
	TemplateVarNumber TemplateVarType = template.VarNumber
	// TemplateVarInteger accepts a JSON number with no fractional part.
	TemplateVarInteger TemplateVarType = template.VarInteger
	// TemplateVarBoolean accepts a JSON bool.
	TemplateVarBoolean TemplateVarType = template.VarBoolean
	// TemplateVarField accepts a JSON string naming a cohort field.
	TemplateVarField TemplateVarType = template.VarField
	// TemplateVarEnum accepts a JSON string from the declaration's Values.
	TemplateVarEnum TemplateVarType = template.VarEnum
	// TemplateVarList accepts a JSON array of the declaration's Items type.
	TemplateVarList TemplateVarType = template.VarList
	// TemplateVarDate accepts a JSON string parsing as an ISO date.
	TemplateVarDate TemplateVarType = template.VarDate
	// TemplateVarPeriod accepts a labeled-date-range object (ranges XOR table).
	TemplateVarPeriod TemplateVarType = template.VarPeriod
)

TemplateVarType values — the closed set of declared variable types. Each equals its internal counterpart, so a TemplateVariable.Type (or Items) compares against these directly.

type TemplateVariable 

type TemplateVariable = template.Variable

TemplateVariable is one declared template variable.

type TestRegistration added in v0.7.0 

type TestRegistration struct {
	Name        types.TestType
	Description string
	// Purpose is the optional plain-language guidance for the
	// operator — what it is for, the intents it answers, when to use
	// something else. It is validated at pulse.New with the same rules
	// as the built-in tier (NotFor.Use resolves against built-ins, the
	// other registered extensions and the feature table); a broken rule
	// fails with PULSE_EXTENSION_PURPOSE_INVALID. Only its intent IDs
	// reach the manifest entry's intents list; nil declares no intents.
	Purpose *descriptor.Purpose
	// Interpretation optionally says how to read the test's outputs,
	// one entry per normalised output path (e.g. "statistic",
	// "details.effect_size.cohens_d"). Checked for structure only at
	// pulse.New — path syntax, Means or Shared, Bands with a
	// Convention — since an extension declares no output keys to probe;
	// a broken rule fails with PULSE_EXTENSION_PURPOSE_INVALID.
	Interpretation []descriptor.Interpretation
	// DependsOn names the features this operator needs, each an AND
	// edge (a single-name group): a feature profile that enables the
	// operator must enable every named feature too, or pulse.New fails
	// with PULSE_FEATURE_PROFILE_DEPENDENCY. Every entry must name a
	// feature this build knows — a built-in feature or another
	// registered extension operator — whether or not a profile is set;
	// otherwise pulse.New fails with PULSE_FEATURE_PROFILE_UNKNOWN.
	// Optional; nil adds no dependency. Extensions carry no Since.
	DependsOn   []string
	Tier        TestTier
	RowFactory  extend.RowTestFactory
	PostFactory extend.PostTestFactory
	Streamable  bool
	Accepts     []encoding.FieldType
	Params      []ParamMeta
	// WeightAware declares that the test computes a weighted result:
	// on a slot where a weight is in force the factory sees it on
	// spec.Weight and a tier-1 row test reads each row's valid weight
	// through extend.Record.Weight() (rows with an invalid weight never
	// reach UpdateRow). Undeclared, ANY weight in force on the slot —
	// the instance default included — is
	// PULSE_EXTENSION_NOT_WEIGHT_AWARE, as the built-in tests refuse
	// one; `weight: null` on the slot opts out. Probe-validation
	// constructs a WeightAware test's factory both with and without a
	// weight (a test that does not declare it is not probed).
	WeightAware bool
	// FieldInputs is the optional buffered-projection introspection
	// hook for tier-1 row tests. Tier-2 post-tests run on materialized
	// result rows rather than source records, so projection doesn't
	// apply — leave nil for tier-2 registrations.
	FieldInputs FieldInputsFunc
}

TestRegistration installs a custom TEST_* operator. Exactly one of RowFactory / PostFactory must be non-nil, matching Tier.

Streamable applies to tier-1 only and indicates whether the test can co-stream with online aggregators (no extra pass over the data). The flag alone decides: Streamable=false forces the request onto the buffered path.

type TestTier added in v0.7.0 

type TestTier string

TestTier selects which factory shape a TestRegistration provides. Tier-1 row tests fold per-row during the streaming aggregation pass; tier-2 post-tests consume the materialized result row set after windows. 

const (
	TestTierRow  TestTier = "tier1"
	TestTierPost TestTier = "tier2"
)

type VerifyIndexResult added in v0.27.0 

type VerifyIndexResult = service.VerifyIndexResult

VerifyIndexResult re-exports internal/service.VerifyIndexResult — the outcome of a Service.VerifyIndex freshness check.

type VerifyResult added in v0.8.0 

type VerifyResult = service.VerifyResult

VerifyResult carries the structured outcome of VerifyShardArchive. Errors aggregate every fatal cohesion failure discovered while walking the archive's shards; Warnings carry non-fatal divergences (per-field description drift, aggregate-record-count mismatch). An empty Errors slice means the archive is structurally sound.

type WatchOptions added in v0.11.0 

type WatchOptions struct {
	// PollInterval is the stat-poll cadence. Defaults to 250ms.
	// Network filesystems should pass a larger interval (recommend
	// 30s).
	PollInterval time.Duration

	// CoalesceWindow caps how long the watcher waits for a quick
	// follow-up modify before emitting a single event. Defaults to
	// 100ms; pass 0 to disable coalescing entirely.
	CoalesceWindow time.Duration

	// HashPrefixBytes caps how many leading bytes the hash digest
	// reads from each file. Defaults to 64 KiB — enough to cover
	// the .pulse header + schema for any realistic cohort. Pass a
	// negative value to hash the entire file.
	HashPrefixBytes int

	// Recursive controls whether WatchDir descends into
	// subdirectories. Ignored by Watch.
	Recursive bool

	// Suffix narrows the watched fileset by suffix (case-sensitive).
	// Empty matches every file. WatchDir defaults this to ".pulse"
	// to keep traffic focused on cohort files; callers can override
	// with an empty string to watch every file.
	Suffix string
}

WatchOptions tunes the watcher cadence. Zero values fall back to the defaults documented on each field.

type WidenReport added in v0.38.0 

type WidenReport = encoding.WidenReport

WidenReport re-exports encoding.WidenReport — the outcome of a WidenSetField call: the field's identity, the rung it moved from and to, the number of records re-laid-out and the record stride on each side of the rewrite.

type WindowRegistration added in v0.7.0 

type WindowRegistration struct {
	Name        types.WindowType
	Description string
	// Purpose is the optional plain-language guidance for the
	// operator — what it is for, the intents it answers, when to use
	// something else. It is validated at pulse.New with the same rules
	// as the built-in tier (NotFor.Use resolves against built-ins, the
	// other registered extensions and the feature table); a broken rule
	// fails with PULSE_EXTENSION_PURPOSE_INVALID. Only its intent IDs
	// reach the manifest entry's intents list; nil declares no intents.
	Purpose *descriptor.Purpose
	// DependsOn names the features this operator needs, each an AND
	// edge (a single-name group): a feature profile that enables the
	// operator must enable every named feature too, or pulse.New fails
	// with PULSE_FEATURE_PROFILE_DEPENDENCY. Every entry must name a
	// feature this build knows — a built-in feature or another
	// registered extension operator — whether or not a profile is set;
	// otherwise pulse.New fails with PULSE_FEATURE_PROFILE_UNKNOWN.
	// Optional; nil adds no dependency. Extensions carry no Since.
	DependsOn []string
	Factory   extend.WindowFactory
	Accepts   []encoding.FieldType
	Params    []ParamMeta
	// FieldInputs is the optional buffered-projection introspection
	// hook. See FieldInputsFunc.
	FieldInputs FieldInputsFunc
}

WindowRegistration installs a custom WIN_* operator. Window operators run buffered today; the runtime records custom windows as non-streamable. A streaming-window shape lands when the runtime gains a streaming-window pipeline.

Source Files

View all Source files

Directories

Path Synopsis
cmd
pulse command
Package main is the entry point for the pulse CLI binary.
 Package main is the entry point for the pulse CLI binary.
Package descriptor holds the result and envelope types of Pulse's no-execute self-description surface: the --json Envelope, the manifest (Manifest and its capability blocks), InspectResult and PredictResult, and the per-operator ComponentSchema contract.
 Package descriptor holds the result and envelope types of Pulse's no-execute self-description surface: the --json Envelope, the manifest (Manifest and its capability blocks), InspectResult and PredictResult, and the per-operator ComponentSchema contract.
Package encoding handles the binary .pulse file format: reading, writing, and schema management.
 Package encoding handles the binary .pulse file format: reading, writing, and schema management.
Package errors provides structured error codes and error handling for pulse.
 Package errors provides structured error codes and error handling for pulse.
Package extend is the public contract for authoring Pulse extension operators.
 Package extend is the public contract for authoring Pulse extension operators.
internal
apigolden
Package apigolden renders the exported Go surface of every public package in the Pulse module as a deterministic, line-oriented text dump.
 Package apigolden renders the exported Go surface of every public package in the Pulse module as a deterministic, line-oriented text dump.
buildinfo
Package buildinfo is the single source of the Pulse build version.
 Package buildinfo is the single source of the Pulse build version.
cli
Package cli provides internal CLI wiring and command construction for the pulse binary.
 Package cli provides internal CLI wiring and command construction for the pulse binary.
crosstabfuse
Package crosstabfuse is the one fused-crosstab eligibility rule shared by the engine and the no-execute predict layer.
 Package crosstabfuse is the one fused-crosstab eligibility rule shared by the engine and the no-execute predict layer.
dategroup
Package dategroup is the one reading of GROUP_DATE's params, shared by the operator factory (internal/processing) and predict (internal/descriptor) so both refuse the same request with the same code and message.
 Package dategroup is the one reading of GROUP_DATE's params, shared by the operator factory (internal/processing) and predict (internal/descriptor) so both refuse the same request with the same code and message.
datepart
Package datepart is the one reading of ATTR_DATE_PART's params and field, shared by the operator factory (internal/processing) and predict (internal/descriptor) so both refuse the same request with the same code and message.
 Package datepart is the one reading of ATTR_DATE_PART's params and field, shared by the operator factory (internal/processing) and predict (internal/descriptor) so both refuse the same request with the same code and message.
daterange
Package daterange is the single compiled {label, start, end} model shared by GROUP_DATE_RANGES, FILTER_DATE_RANGES and the named RangeTables the facade validates at pulse.New time.
 Package daterange is the single compiled {label, start, end} model shared by GROUP_DATE_RANGES, FILTER_DATE_RANGES and the named RangeTables the facade validates at pulse.New time.
descriptor
Package descriptor is the internal twin of the public descriptor package: the no-execute builders (manifest, payload JSON Schema, capability declarations), predict, inspect and the request validators.
 Package descriptor is the internal twin of the public descriptor package: the no-execute builders (manifest, payload JSON Schema, capability declarations), predict, inspect and the request validators.
docgen
Package docgen renders an instance's analysis reference — the operator catalog, the glossary and the skill pack — as a deterministic Markdown tree (U21).
 Package docgen renders an instance's analysis reference — the operator catalog, the glossary and the skill pack — as a deterministic Markdown tree (U21).
encoding
Package encoding is the module-internal twin of the public github.com/frankbardon/pulse/encoding package.
 Package encoding is the module-internal twin of the public github.com/frankbardon/pulse/encoding package.
encodingbridge
Package encodingbridge carries the few unexported helpers of the public encoding package that its internal twin (internal/encoding) needs, without the public package exporting them.
 Package encodingbridge carries the few unexported helpers of the public encoding package that its internal twin (internal/encoding) needs, without the public package exporting them.
examples
Package examples embeds the catalogue of runnable Pulse request JSON files and exposes a search/get API over them.
 Package examples embeds the catalogue of runnable Pulse request JSON files and exposes a search/get API over them.
facadebridge
Package facadebridge carries the few in-module hooks that sibling packages (the MCP go-sdk adapter) need from a *pulse.Pulse without the root facade exporting its engine handle.
 Package facadebridge carries the few in-module hooks that sibling packages (the MCP go-sdk adapter) need from a *pulse.Pulse without the root facade exporting its engine handle.
fs
Package fs provides the filesystem abstraction layer for pulse storage backends.
 Package fs provides the filesystem abstraction layer for pulse storage backends.
guide
Package guide holds the logic behind Pulse's guidance surfaces — Recommend (an intent to ranked draft requests) and Explain (a request to plain-language steps).
 Package guide holds the logic behind Pulse's guidance surfaces — Recommend (an intent to ranked draft requests) and Explain (a request to plain-language steps).
imports
Package imports manages tabular-source imports with a TTL-tracked on-disk pool.
 Package imports manages tabular-source imports with a TTL-tracked on-disk pool.
io
Package io defines the I/O pipeline framework for Pulse: Reader/Writer interfaces, schema inference, and job types (ImportJob, ExportJob, ConvertJob).
 Package io defines the I/O pipeline framework for Pulse: Reader/Writer interfaces, schema inference, and job types (ImportJob, ExportJob, ConvertJob).
io/arrow
Package arrow provides Arrow IPC (Feather V2) import and export for the pulse I/O pipeline, plus shared Arrow<->Pulse type-mapping helpers used by both this package and internal/io/parquet.
 Package arrow provides Arrow IPC (Feather V2) import and export for the pulse I/O pipeline, plus shared Arrow<->Pulse type-mapping helpers used by both this package and internal/io/parquet.
io/csv
Package csv provides CSV format adapters for the Pulse I/O pipeline.
 Package csv provides CSV format adapters for the Pulse I/O pipeline.
io/excel
Package excel provides Excel import and export for the pulse I/O pipeline.
 Package excel provides Excel import and export for the pulse I/O pipeline.
io/exportoverlay
Package exportoverlay holds the higher-level integration tests for the ExportJob and ConvertJob overlay-embedding wiring.
 Package exportoverlay holds the higher-level integration tests for the ExportJob and ConvertJob overlay-embedding wiring.
io/jsonarray
Package jsonarray provides JSON-array import and export for the pulse I/O pipeline.
 Package jsonarray provides JSON-array import and export for the pulse I/O pipeline.
io/jsonshared
Package jsonshared holds value coercion helpers shared by the ndjson and jsonarray packages.
 Package jsonshared holds value coercion helpers shared by the ndjson and jsonarray packages.
io/ndjson
Package ndjson provides NDJSON (newline-delimited JSON) import and export for the pulse I/O pipeline.
 Package ndjson provides NDJSON (newline-delimited JSON) import and export for the pulse I/O pipeline.
io/nullcell
Package nullcell holds the cross-adapter integration tests for the TWO absent-looking states of a `categorical_*` column — a null cell and a cell whose value IS the empty string.
 Package nullcell holds the cross-adapter integration tests for the TWO absent-looking states of a `categorical_*` column — a null cell and a cell whose value IS the empty string.
io/parquet
Package parquet provides Parquet import and export for the pulse I/O pipeline.
 Package parquet provides Parquet import and export for the pulse I/O pipeline.
io/settristate
Package settristate holds the cross-adapter integration tests for the THREE states of a `set_*` column — null, empty mask ("answered, selected nothing") and non-empty mask.
 Package settristate holds the cross-adapter integration tests for the THREE states of a `set_*` column — null, empty mask ("answered, selected nothing") and non-empty mask.
io/setwide
Package setwide holds the per-adapter wide-set round-trip tests: one test per tabular format, proving that a 206-member `set_u256` column survives an export and a re-import with every bit in the place it started.
 Package setwide holds the per-adapter wide-set round-trip tests: one test per tabular format, proving that a 206-member `set_u256` column survives an export and a re-import with every bit in the place it started.
io/spss
Package spss reads SPSS `.sav` system files for the Pulse I/O pipeline.
 Package spss reads SPSS `.sav` system files for the Pulse I/O pipeline.
io/tsv
Package tsv provides TSV import and export for the pulse I/O pipeline.
 Package tsv provides TSV import and export for the pulse I/O pipeline.
iocore
Package iocore is the leaf that holds the tabular I/O contracts every format adapter implements: Reader, Writer and the optional interfaces the shared import / export / convert paths discover by type assertion, plus the handful of cell conventions (set delimiter, empty-set marker, null-cell test, stop-iteration sentinel, set-width ladder) the adapters and the jobs must agree on.
 Package iocore is the leaf that holds the tabular I/O contracts every format adapter implements: Reader, Writer and the optional interfaces the shared import / export / convert paths discover by type assertion, plus the handful of cell conventions (set delimiter, empty-set marker, null-cell test, stop-iteration sentinel, set-width ladder) the adapters and the jobs must agree on.
jsonfinite
Package jsonfinite is the read-side inverse of types.MarshalFinite.
 Package jsonfinite is the read-side inverse of types.MarshalFinite.
limits
Package limits is the one definition of Pulse's instance resource limits: the Limits struct the root facade aliases as pulse.Limits, the built-in defaults, the per-field precedence resolver, the construction-time validation and the shared constructors every PULSE_LIMIT_* raise site uses.
 Package limits is the one definition of Pulse's instance resource limits: the Limits struct the root facade aliases as pulse.Limits, the built-in defaults, the per-field precedence resolver, the construction-time validation and the shared constructors every PULSE_LIMIT_* raise site uses.
mcp
Package mcp is the SDK-free core of the Pulse MCP surface.
 Package mcp is the SDK-free core of the Pulse MCP surface.
mcp/toolmeta
Package toolmeta holds the metadata table for the MCP tools registered by the Pulse MCP layer.
 Package toolmeta holds the metadata table for the MCP tools registered by the Pulse MCP layer.
mergegate
Package mergegate is the one pure mergeability rule shared by the engine and the no-execute validators.
 Package mergegate is the one pure mergeability rule shared by the engine and the no-execute validators.
obsprom
Package obsprom is Pulse's stdlib-only Prometheus exporter: an observe.Metrics implementation that keeps its instruments in memory and renders them in the Prometheus text exposition format (version 0.0.4) over net/http.
 Package obsprom is Pulse's stdlib-only Prometheus exporter: an observe.Metrics implementation that keeps its instruments in memory and renders them in the Prometheus text exposition format (version 0.0.4) over net/http.
processing
Package processing provides the single dynamic processing engine for Pulse.
 Package processing provides the single dynamic processing engine for Pulse.
processing/arena
Package arena provides a bump-allocator backed by a single contiguous []byte.
 Package arena provides a bump-allocator backed by a single contiguous []byte.
processing/feature
Package feature implements the FEAT_* operators that run pre-filter to add derived columns to a record stream.
 Package feature implements the FEAT_* operators that run pre-filter to add derived columns to a record stream.
processing/multiplicity
Package multiplicity is the pure multiple-comparison correction core: it maps one vector of raw p-values to its adjusted twin under a named method (bonferroni, holm, bh, by, none), matching R's stats::p.adjust.
 Package multiplicity is the pure multiple-comparison correction core: it maps one vector of raw p-values to its adjusted twin under a named method (bonferroni, holm, bh, by, none), matching R's stats::p.adjust.
processing/regression
Package regression hosts the REG_* operators that fit a regression model against the filtered record set.
 Package regression hosts the REG_* operators that fit a regression model against the filtered record set.
processing/window
Package window implements the WIN_* window operators for Pulse.
 Package window implements the WIN_* window operators for Pulse.
profilefile
Package profilefile reads a feature profile from a host OS path.
 Package profilefile reads a feature profile from a host OS path.
returnplan
Package returnplan is the schema-free core of response shaping (`Request.Return`): the path grammar, the compiled selection plan and its digest, and the per-node verdict a pruner or wire encoder asks while walking a response.
 Package returnplan is the schema-free core of response shaping (`Request.Return`): the path grammar, the compiled selection plan and its digest, and the per-node verdict a pruner or wire encoder asks while walking a response.
returnshape
Package returnshape applies a resolved `return` plan to a finished Response.
 Package returnshape applies a resolved `return` plan to a finished Response.
service
Package service provides the orchestration layer for pulse operations.
 Package service provides the orchestration layer for pulse operations.
shardfixtures
Package shardfixtures is the single source of truth for the canonical shard archives committed under testdata/sharding/.
 Package shardfixtures is the single source of truth for the canonical shard archives committed under testdata/sharding/.
skills
Package skills provides the embedded skill pack for LLM-driven agents.
 Package skills provides the embedded skill pack for LLM-driven agents.
spsssidecar
Package spsssidecar names the SPSS metadata sidecar that sits beside a cohort.
 Package spsssidecar names the SPSS metadata sidecar that sits beside a cohort.
spsstest
Package spsstest builds spec-conformant SPSS `.sav` system files from a declarative spec, for use as test fixtures.
 Package spsstest builds spec-conformant SPSS `.sav` system files from a declarative spec, for use as test fixtures.
statdist
Package statdist holds the distribution primitives shared by the statistical operators: the Student-t tail, CDF, quantile and two-sided critical value, and the regularized incomplete beta they rest on.
 Package statdist holds the distribution primitives shared by the statistical operators: the Student-t tail, CDF, quantile and two-sided critical value, and the regularized incomplete beta they rest on.
synth
Package synth produces synthetic .pulse cohorts from either a schema declaration ("from-schema") or a statistical profile of a real cohort ("from-profile").
 Package synth produces synthetic .pulse cohorts from either a schema declaration ("from-schema") or a statistical profile of a real cohort ("from-profile").
template
Package template turns a stored, parameterised JSON document into a validated Pulse request.
 Package template turns a stored, parameterised JSON document into a validated Pulse request.
temporal
Package temporal is the single home of Pulse's epoch-day and calendar arithmetic.
 Package temporal is the single home of Pulse's epoch-day and calendar arithmetic.
tools/docsummary command
Command docsummary splices the SUMMARY.md fragment a `pulse docs export` writes into the book's own SUMMARY.md, between the docgen.BookSpanBegin / BookSpanEnd lines, prefixing every path with the export directory.
 Command docsummary splices the SUMMARY.md fragment a `pulse docs export` writes into the book's own SUMMARY.md, between the docgen.BookSpanBegin / BookSpanEnd lines, prefixing every path with the export directory.
tools/pkgsplit command
Command pkgsplit is the in-module AST rewriter used to split a public package in place: the kept subset stays in the public directory, the remainder moves to an internal twin that imports the public package.
 Command pkgsplit is the in-module AST rewriter used to split a public package in place: the kept subset stays in the public directory, the remainder moves to an internal twin that imports the public package.
vectors
Package vectors is the one resolution of Request.Vectors: it turns each types.VectorSpec into its ordered member list against a schema, refusing malformed specs and unsupported member types with the PULSE_VECTOR_* codes.
 Package vectors is the one resolution of Request.Vectors: it turns each types.VectorSpec into its ordered member list against a schema, refusing malformed specs and unsupported member types with the PULSE_VECTOR_* codes.
weighting
Package weighting is the one declaration of how built-in operators treat a resolved row weight, plus the per-row weight-validity rule.
 Package weighting is the one declaration of how built-in operators treat a resolved row weight, plus the per-row weight-validity rule.
Package io is Pulse's public tabular I/O surface: the Reader / Writer contracts and their optional interfaces, schema inference, and the job types (ImportJob, ExportJob, ConvertJob, the transfer and dedup jobs).
 Package io is Pulse's public tabular I/O surface: the Reader / Writer contracts and their optional interfaces, schema inference, and the job types (ImportJob, ExportJob, ConvertJob, the transfer and dedup jobs).
Package linalg is Pulse's one linear-algebra core: Pulse-owned dense (Matrix), symmetric (Sym) and vector (Vec) types plus the kernels the engine and the synthetic-data generator factor, solve and invert with.
 Package linalg is Pulse's one linear-algebra core: Pulse-owned dense (Matrix), symmetric (Sym) and vector (Vec) types plus the kernels the engine and the synthetic-data generator factor, solve and invert with.
mcp
gosdk
Package gosdk is the thin, reusable adapter that mounts the SDK-free Pulse MCP catalog (github.com/frankbardon/pulse/mcp) onto a caller-supplied github.com/modelcontextprotocol/go-sdk server.
 Package gosdk is the thin, reusable adapter that mounts the SDK-free Pulse MCP catalog (github.com/frankbardon/pulse/mcp) onto a caller-supplied github.com/modelcontextprotocol/go-sdk server.
Package mcpserve exposes Pulse's Model Context Protocol server as a public entry point.
 Package mcpserve exposes Pulse's Model Context Protocol server as a public entry point.
Package observe holds the vocabulary a host uses to watch a Pulse instance: the hook set (Hooks), what each hook is told (OperationInfo, OperationResult, PhaseTiming), the bounded enums those carry (OperationKind, Phase, Arm, Scope) and the metric instrument interfaces an adapter implements (Metrics, Counter, Histogram, UpDownCounter, Label).
 Package observe holds the vocabulary a host uses to watch a Pulse instance: the hook set (Hooks), what each hook is told (OperationInfo, OperationResult, PhaseTiming), the bounded enums those carry (OperationKind, Phase, Arm, Scope) and the metric instrument interfaces an adapter implements (Metrics, Counter, Histogram, UpDownCounter, Label).
Package synth produces synthetic .pulse cohorts from either a schema declaration ("from-schema") or a statistical profile of a real cohort ("from-profile").
 Package synth produces synthetic .pulse cohorts from either a schema declaration ("from-schema") or a statistical profile of a real cohort ("from-profile").
Package types provides shared type definitions for pulse.
 Package types provides shared type definitions for pulse.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.