examples

package
v1.0.0-alpha.6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 9, 2026 License: MIT Imports: 11 Imported by: 0

README

Pulse Examples

Runnable JSON request files organized by topic. Every example resolves its cohort from the shared fixture set under fixtures/ so adding a new example is just authoring JSON — no new CSV, schema, or build step.

Layout

internal/examples/
├── README.md                  # this file
├── fixtures/                  # shared CSVs + schemas + generator + build script
│   ├── gen.go                 # deterministic CSV generator (run once)
│   ├── build.sh               # CSV -> .pulse importer (writes ./.data)
│   ├── *.csv                  # checked-in source data
│   └── schemas/*.json         # hand-tuned schemas with descriptions
├── attributes/                # ATTR_* per-record derivations (6)
├── features/                  # FEAT_* feature engineering (10)
├── filterers/                 # FILTER_* row-level predicates (6)
├── groupers/                  # GROUP_* partitioning (7)
├── windows/                   # WIN_* per-row analytics (10)
├── aggregations/              # AGG_* requests over the all_types cohort (5)
├── tests/                     # TEST_* tier-1 and tier-2 statistical tests (10)
├── matrices/                  # MAT_* matrix operators over virtual vectors (1)
└── <new-category>/            # each: README.md + run-all.sh + *.json

One-time setup

Build the binary, generate fixtures, import to .pulse:

make build
./internal/examples/fixtures/build.sh

That writes transactions.pulse, customers.pulse, orders.pulse, training_data.pulse, all_types.pulse, and experiment.pulse into .data/ at the repo root. Every example request has cohort.data_dir = ".data" so the requests run unmodified.

Available cohorts

File Fields
transactions.pulse id (u32), amount (f64)
customers.pulse id (u32), age (u8), income (f64), region (categorical), city (categorical)
orders.pulse id (u32), order_date (date), revenue (f64), ordered_at (datetime)
training_data.pulse id (u32), region/occupation/category (categorical), label (u8 binary), price/income (f64), signup_date (date)
all_types.pulse One column for every one of the 17 supported types (u8/u16/u32/u64, f32/f64, nullable_bool/u4/u8/u16, date, packed_bool, categorical_u8/u16/u32, decimal128, nullable_decimal128). Hand-curated 5-row CSV — exists so a single fixture exercises every type-aware code path.
experiment.pulse A/B test cohort: id (u32), treatment/region/segment/converted (categorical), revenue/session_minutes (f64), period (date). Planted effects across treatment, region, segment, and time so every TEST_* example produces a meaningful, non-trivial result.

The generator (fixtures/gen.go) is deterministic — re-running with the same seed produces byte-identical CSVs. Re-generate from scratch:

go run internal/examples/fixtures/gen.go
./internal/examples/fixtures/build.sh

Running

A category-specific runner exercises everything in its folder:

./internal/examples/attributes/run-all.sh
./internal/examples/features/run-all.sh
./internal/examples/filterers/run-all.sh
./internal/examples/groupers/run-all.sh
./internal/examples/windows/run-all.sh
./internal/examples/tests/run-all.sh

Or invoke individual requests:

bin/pulse api predict --request internal/examples/features/01_log_transform.json --json
bin/pulse api process --request internal/examples/features/01_log_transform.json --json

The _meta block

Every example JSON carries a top-level _meta object; it is stripped before the body is served or run (examples.Get), and request decoding ignores it.

Key Required Meaning
name yes unique example ID (pulse_examples_get name=...)
category yes must equal the parent directory
description yes one-sentence summary served by search
tags yes drawn from CanonicalTags
operators yes (may be []) exactly the "type": "AGG_/ATTR_/FILTER_/GROUP_/WIN_/FEAT_/TEST_/REG_..." values in the body (TestExamples_OperatorsMatchBody); overlay kinds ride overlays[].kind instead
intents expected intent-taxonomy IDs (TestExamples_IntentsFromTaxonomy)
capabilities optional non-operator feature names in feature-profile spelling (capability:stream), for capabilities the body carries no structural signal for

capabilities exists so an example is joined to every capability it needs by an explicit edge in the discovery graph rather than by a scan of its prose. The graph builder (internal/descriptor/ontology_examples.go) already DETECTS capability:crosstab (a crosstab key), capability:compose (a root requests), capability:process_chain (a root stages), capability:joins (a joins key) and capability:facet (category or tag facet) from the body, plus every overlays[].kind; declare the rest — capability:stream, watch, shard, index, sample, labels, range_tables, filter_to_file — in capabilities. An example with an empty operators list must declare at least one capability. Gates (internal/descriptor): TestExamples_CapabilitiesFromFeatures (every value is a known non-operator feature), TestExamples_EdgeCoverage (every operator a body or description names is an edge; a declared capability never contradicts its detector), and TestExamples_EmptyOperatorExamplesDeclareCapabilities.

Adding a new example category

  1. Pick a category name (e.g. windows, aggregation, attributes).
  2. Create internal/examples/<category>/ with a README.md and request *.json files.
  3. Each JSON sets cohort.data_dir = ".data" and references one of the shared cohort filenames above.
  4. If your category needs a field that's not in any shared cohort, add it to fixtures/gen.go and the matching schema in fixtures/schemas/. Re-run the generator and check in the new CSV.
  5. Drop in a run-all.sh for the category (copy from features/ and adjust the glob).
  6. Optionally add an end-to-end test in examples_test.go mirroring TestFeatureExamples_RunEndToEnd.

CI coverage

TestExamples_RunEndToEnd (in examples_test.go) builds the shared fixtures into a temp directory and runs every example across every category through pulse.Process, asserting no errors. Adding a new category is one line in the test's categories slice.

go test -run TestExamples -v .

One sub-test runs per example file, grouped by category.

Documentation

Overview

Package examples embeds the catalogue of runnable Pulse request JSON files and exposes a search/get API over them.

Every embedded JSON carries a top-level _meta block describing the example's name, category, taxonomy tags, the operators it uses, and a one-sentence description. Pulse's types.Request unmarshaler ignores unknown fields by default, so _meta is invisible at execution time — the examples remain runnable verbatim.

Get returns the example with the _meta block stripped, so callers can hand the JSON straight to pulse_process / pulse_predict.

Index

Constants

This section is empty.

Variables

View Source 
var CanonicalTags = []string{

	"time-series", "cohort-analysis", "experiment-analysis", "correlation-analysis",
	"comparison", "before-after", "top-n", "distribution-shape", "cross-tabulation",
	"proportion-analysis", "trend-detection", "outlier-detection", "cardinality-analysis",
	"data-quality", "financial", "feature-engineering",

	"hypothesis-test", "t-test", "parametric", "nonparametric", "paired", "one-sample",
	"two-sample", "k-sample", "repeated-measures", "post-hoc",
	"normality-test", "homogeneity-test", "exact-test",

	"regression", "ecological",
	"ols", "glm", "logistic", "bayesian",
	"regularization", "ridge", "lasso", "elasticnet",
	"polynomial", "resampling", "jackknife", "selection", "stepwise",

	"tier-1-test", "tier-2-test", "composed", "pre-filter", "feature-pipeline",
	"window-operator", "streaming-friendly", "buffered-pipeline",

	"matrix", "covariance",

	"leakage-risk", "small-sample",

	"facet",

	"sharded", "anchor",

	"overlay",
	"byte-equal-test",
	"compose",
	"crosstab",
	"welch",
	"welford-triple",
	"z",
}

CanonicalTags is the curated taxonomy every annotated example must draw from. Kept here (rather than embedded in a separate file) so the validation gate can compare directly without round-tripping through a disk asset.

View Source 
var ErrNoMeta = errors.New("missing _meta block")

ErrNoMeta is Parse's error for a JSON object without a _meta block.

Functions

func AllCategories 

func AllCategories() []string

AllCategories returns every directory the library indexes, sorted alphabetically.

func AllTags 

func AllTags() []string

AllTags returns every taxonomy tag the library currently uses, sorted alphabetically. Derived from the indexed examples — not the canonical allowlist — so callers see exactly what is tagged today.

func Capabilities 

func Capabilities() map[string][]string

Capabilities returns every example's optional _meta.capabilities, keyed by example name; an example without the key is absent. Values are feature-profile spellings of non-operator features (`capability:stream`), validated by internal/descriptor (TestExamples_CapabilitiesFromFeatures), which this package cannot import. The key exists for capabilities a request body carries no structural signal for; internal/descriptor's ontology builder turns each value into an `example requires_capability <feature>` edge.

func Count 

func Count() int

Count returns the number of embedded examples.

func DeriveOperators 

func DeriveOperators(body []byte) []string

DeriveOperators returns the sorted, distinct operator names a body carries as a `"type"` value — what `_meta.operators` must equal (TestExamples_OperatorsMatchBody; embedder examples at pulse.New).

func Intents 

func Intents() map[string][]string

Intents returns every example's optional _meta.intents, keyed by example name; an example without the key is absent. Values are validated against the intent taxonomy by internal/descriptor (TestExamples_IntentsFromTaxonomy), which this package cannot import.

func IsCanonicalTag 

func IsCanonicalTag(tag string) bool

IsCanonicalTag reports whether tag belongs to the curated taxonomy.

func Parse 

func Parse(data []byte, strict bool) (*Example, Meta, error)

Parse reads one example file: a JSON object carrying a _meta block plus the request body. It returns the Example — Body is the object minus _meta, re-marshaled deterministically — and the parsed Meta. strict refuses an unknown _meta key (embedder examples); the embedded library parses leniently. Parse does not validate Meta's values.

Types

type Example 

type Example struct {
	Name        string          `json:"name"`
	Category    string          `json:"category"`
	Tags        []string        `json:"tags"`
	Operators   []string        `json:"operators"`
	Description string          `json:"description"`
	Body        json.RawMessage `json:"body"`
}

Example is the full record returned by Get. Body carries the request JSON with the _meta block stripped, so it can be handed verbatim to pulse_process / pulse_predict.

func All 

func All() []*Example

All returns a copy of every embedded example, sorted by name.

func Get 

func Get(name string) (*Example, bool)

Get returns the example with the given name. The returned Body is the request JSON minus the _meta field, runnable verbatim against pulse.Process. Returns false when no example carries the name.

type ExampleSummary 

type ExampleSummary struct {
	Name        string   `json:"name"`
	Category    string   `json:"category"`
	Tags        []string `json:"tags"`
	Operators   []string `json:"operators"`
	Description string   `json:"description"`
}

ExampleSummary is the lightweight projection used by Search results. 

func Search(query string, tags []string, category string) []ExampleSummary

Search returns summaries matching all three filters. An empty filter is treated as "no constraint" for that dimension.

  • query (case-insensitive substring) matches against name, description, and every operator. Score = number of distinct matches across the three fields; ties break alphabetically by name.
  • tags is ANDed: an example must carry every requested tag.
  • category is an exact directory match.

Returns a non-nil empty slice when nothing matches.

func SearchIn 

func SearchIn(lib []*Example, query string, tags []string, category string) []ExampleSummary

SearchIn is Search over lib, which must be sorted by name (the no-query order and the score tie-break). An instance serving embedder examples searches the merged library through it.

type Meta 

type Meta struct {
	Name        string   `json:"name"`
	Category    string   `json:"category"`
	Tags        []string `json:"tags"`
	Operators   []string `json:"operators"`
	Description string   `json:"description"`
	// Intents is optional; every value must be an intent-taxonomy ID
	// (validated by internal/descriptor's TestExamples_IntentsFromTaxonomy,
	// since this package cannot import the taxonomy).
	Intents []string `json:"intents,omitempty"`
	// Capabilities is optional; every value must be a non-operator
	// feature name (validated by internal/descriptor's
	// TestExamples_CapabilitiesFromFeatures).
	Capabilities []string `json:"capabilities,omitempty"`
}

Meta mirrors the _meta block every example JSON carries — built-in and embedder (pulse.Extensions.Examples) alike.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.