examples

Pulse Examples

Runnable JSON request files organized by topic. Every example resolves its cohort from the shared fixture set under fixtures/ so adding a new example is just authoring JSON — no new CSV, schema, or build step.

Layout

internal/examples/
├── README.md                  # this file
├── fixtures/                  # shared CSVs + schemas + generator + build script
│   ├── gen.go                 # deterministic CSV generator (run once)
│   ├── build.sh               # CSV -> .pulse importer (writes ./.data)
│   ├── *.csv                  # checked-in source data
│   └── schemas/*.json         # hand-tuned schemas with descriptions
├── aggregations/              # AGG_* aggregations, incl. vector aggregators (21)
├── attributes/                # ATTR_* per-record derivations (23)
├── compose/                   # Compose batches: sweeps, fits, populations, the MMM decay sweep, the SEM invariance ladder, SEM scores from a bound fit (10)
├── crosstab/                  # Crosstab tables, margins and inference (17)
├── facet/                     # Facet requests (3)
├── features/                  # FEAT_* feature engineering (12)
├── filterers/                 # FILTER_* row-level predicates (9)
├── groupers/                  # GROUP_* partitioning (11)
├── matrices/                  # MAT_* matrix operators over virtual vectors (16)
├── overlays/                  # OVERLAY_* layers on Request / Compose / Facet hosts (48)
├── regression/                # REG_* regressions (18)
├── synth/                     # synth specs (7)
├── tests/                     # TEST_* tier-1 and tier-2 statistical tests (37)
├── windows/                   # WIN_* per-row analytics (12)
└── <new-category>/            # each: README.md + run-all.sh + *.json

One-time setup

Build the binary, generate fixtures, import to .pulse:

make build
./internal/examples/fixtures/build.sh

That writes transactions.pulse, customers.pulse, orders.pulse, training_data.pulse, all_types.pulse, experiment.pulse, repeated_measures.pulse, card_issuers.pulse, tier_migrations.pulse, survey.pulse, subscribers.pulse, warehouse_orders.pulse, plan_history.pulse, plan_history_wide.pulse, battery.pulse, intake.pulse, mmm_weekly.pulse and brand_survey.pulse into .data/ at the repo root, plus the point-lookup index on experiment.pulse (id). Every example request has cohort.data_dir = ".data" so the requests run unmodified.

Available cohorts

The generator (fixtures/gen.go) is deterministic — re-running with the same seed produces byte-identical CSVs. Re-generate from scratch:

go run internal/examples/fixtures/gen.go
./internal/examples/fixtures/build.sh

Running

A category-specific runner exercises everything in its folder:

./internal/examples/attributes/run-all.sh
./internal/examples/features/run-all.sh
./internal/examples/filterers/run-all.sh
./internal/examples/groupers/run-all.sh
./internal/examples/windows/run-all.sh
./internal/examples/tests/run-all.sh

Or invoke individual requests:

bin/pulse api predict --request internal/examples/features/01_log_transform.json --json
bin/pulse api process --request internal/examples/features/01_log_transform.json --json

The _meta block

Every example JSON carries a top-level _meta object; it is stripped before the body is served or run (examples.Get), and request decoding ignores it.

capabilities exists so an example is joined to every capability it needs by an explicit edge in the discovery graph rather than by a scan of its prose. The graph builder (internal/descriptor/ontology_examples.go) already DETECTS capability:crosstab (a crosstab key), capability:compose (a root requests), capability:process_chain (a root stages), capability:joins (a joins key) and capability:facet (category or tag facet) from the body, plus every overlays[].kind; declare the rest — capability:stream, watch, shard, index, sample, labels, range_tables, filter_to_file — in capabilities. An example with an empty operators list must declare at least one capability. Gates (internal/descriptor): TestExamples_CapabilitiesFromFeatures (every value is a known non-operator feature), TestExamples_EdgeCoverage (every operator a body or description names is an edge; a declared capability never contradicts its detector), and TestExamples_EmptyOperatorExamplesDeclareCapabilities.

Story examples

An example is a story iff its _meta declares both question and interpretation; one without the other is refused. A story's request sets "return": {"precision": 4} (examples.StoryPrecision, four significant digits), so a float's last few ulps — which differ between amd64 and arm64 — never reach its output. Rules: examples.StoryProblems, gated by TestStories_LibraryHoldsTheRules. They bind the shipped library only: an embedder example (Extensions.Examples) may declare question / interpretation, and pulse.New serves them as declared without applying these rules.

Each story's response is pinned by a harness-produced golden, testdata/stories/<name>.json — the response exactly as json.MarshalIndent writes it. The root package's TestExamples_StoryGoldens builds the fixture cohorts, runs every story and fails on a missing, extra or differing golden. Never hand-edit one; regenerate after changing a story or a fixture:

go test -count=1 . -run TestExamples_StoryGoldens -update

The goldens are embedded: examples.StoryGolden(name) returns one, so the generated docs quote real figures without executing a request.

Adding a new example category

1. Pick a category name (e.g. windows, aggregation, attributes).
2. Create internal/examples/<category>/ with a README.md and request *.json files.
3. Each JSON sets cohort.data_dir = ".data" and references one of the shared cohort filenames above.
4. If your category needs a field that's not in any shared cohort, add it to fixtures/gen.go and the matching schema in fixtures/schemas/. Re-run the generator and check in the new CSV.
5. Drop in a run-all.sh for the category (copy from features/ and adjust the glob).
6. Optionally add an end-to-end test in examples_test.go mirroring TestFeatureExamples_RunEndToEnd.

CI coverage

TestExamples_RunEndToEnd (in examples_test.go) builds the shared fixtures into a temp directory and runs every example across every category through pulse.Process, asserting no errors. Adding a new category is one line in the test's categories slice.

go test -run TestExamples -v .

One sub-test runs per example file, grouped by category.