extensions/

directory
v0.20.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 8, 2026 License: Apache-2.0

README

Flow extensions

Every extension is one self-contained package under extensions/<name>/ that plugs into the compiler through core.Bundle. Core knows only the bundle contract; the feature lives entirely here.

Anatomy

extensions/loop/
├── library.go          # Package comment, ID, init(), Bundle()
├── keyword.go          # Feature logic (parser, action, operator, ...)
├── nflows/             # Optional DSL fixtures, auto-discovered
│   └── basic.nflow
├── keyword_test.go     # Tests for this package only
└── library_test.go     # Wiring: Bundle() returns what it claims

library.go never contains feature logic. Feature files are named after what they provide: keyword.go, directive.go, modifiers.go, actions.go, primary.go.

The Bundle contract

const ID = "loop"

func init() {
    core.Register(ID, Bundle)
}

func Bundle(_ map[string]string) core.Bundle {
    return core.Bundle{
        ID:        ID,
        Libraries: []action.Library{{Name: ID}},
        Primaries: []core.PrimaryExtension{loopKeyword{}},
        Fixtures:  fixturesFS,  // optional
    }
}

ID is a short label — "loop", "macros", "fs" — never an import path. It survives forks, module moves, and vendor dirs. Two extensions with the same ID panic at init().

Fixtures vs SelfTest

Prefer nflows/*.nflow fixtures over inline SelfTest(). A fixture is a real .nflow file the runner executes like any pipeline:

@description "Loop loop(...) until(...)"
@assert: result.n == 3
{ n: 0 } -> loop( { n: .n + 1 } ) until( .n >= 3 )

The @description names the feature in nflow self test. The file grows with the DSL — no Go code to change when the feature evolves.

Use inline SelfTest() only when the feature cannot be expressed in DSL: files from outside the repo (Files: map), a running process or network (external.exec, http.request), or runtime-only middleware where the DSL grammar accepts a modifier but no assertion can observe its effect (:cache=, :retry=).

Tests

Extension tests verify this package's code only:

  • Parser output — does loop(...) produce the expected AST node.
  • Translation — does :timeout=5s set Meta.Timeout = 5s.
  • Wiring — does Bundle() return the promised directives, primaries, libraries, and fixtures.

Never test kernel runtime here (retries, cache hits, concurrency, context semantics). That is kernel/*'s job; its suite already covers it. Never use time.Sleep in these tests.

Three rules that prevent 90% of the flaky tests in this package:

  • Reading from map[string]any: assert the type first (v, ok := m[key].(string)) before passing to ktest.RequireEqual. Generic inference cannot resolve any against a concrete want.
  • Absolute paths: use t.TempDir(), not a hardcoded /abs/path. filepath.Abs("/x") returns C:\x on Windows; a test comparing against /x will fail.
  • Stream operators are not predicates. action.StreamOp returns an iter.Seq2, not func(T) bool. Test through a collectX helper that iterates to exhaustion.

Errors

One rule, no exceptions:

  • Compile-time with a position: core.SourceError(pos, ...).
  • Runtime, reaches the user: xerr.<Kind>(...).
  • Internal wiring (bug, not user input): xerr.Internal(...).
  • fmt.Errorf is allowed only inside helpers that are immediately wrapped by SourceError in the same file (tag parsers, field parsers). Anywhere else it means the error escapes unclassified.

Before commit: rg 'fmt\.Errorf' flow/extensions/ — every hit must be justified by the rule above.

External repositories

Any repo can become a Flow extension by adding a nexssflow/ package:

my-repo/
├── go.mod                  # module github.com/nexssp/my-repo
└── nexssflow/
    └── library.go      # exposes Bundle(opts) core.Bundle

Then @require github.com/nexssp/my-repo v1.2.3 selects package github.com/nexssp/my-repo/nexssflow. Go determines whether the package comes from the root module or an independently versioned nested module; the requested version applies to the module that provides it. An explicit variant such as @require github.com/nexssp/my-repo/nexssflow_v2 v1.4.0 selects that Go package path, and Go determines its provider. Flow does not infer or truncate module paths from repository URL shape. Remote requirements are version-pinned; unversioned subpackages are only for the current module or Go workspace, not an implicit latest request.

The shim can be a thin wrapper over external.ExecAction, a sandbox engine, or a direct Go import if the repo is already in Go.

Rules

  • Package comment: // Package <name> ships <what>. + Typical use: block.
  • No const ImportPath. No hardcoded module URLs anywhere.
  • No silent anything — collisions panic; missing contracts fail loudly.
  • Files split by responsibility, not by line count.

Directories

Path Synopsis
Package assert provides the `assert(cond, msg)` keyword and the `@assert: expr` directive.
Package assert provides the `assert(cond, msg)` keyword and the `@assert: expr` directive.
Package config provides the @config and @config.load directives that populate meta["config"], which OnPreprocess forwards to the compiler as a compile-time key-value map.
Package config provides the @config and @config.load directives that populate meta["config"], which OnPreprocess forwards to the compiler as a compile-time key-value map.
Package config_yaml registers a YAML decoder with extensions/config.
Package config_yaml registers a YAML decoder with extensions/config.
Package decide provides the `decide` action: it evaluates a state against a registered decision backend and returns structured answers.
Package decide provides the `decide` action: it evaluates a state against a registered decision backend and returns structured answers.
Package description provides the `@description "..."` directive, which records a human-readable summary of a .nflow file into meta["description"].
Package description provides the `@description "..."` directive, which records a human-readable summary of a .nflow file into meta["description"].
Package external provides three actions for calling code that lives outside the Flow process: exec (shell), http.request (HTTP client), and wasm (Wazero WASI modules).
Package external provides three actions for calling code that lives outside the Flow process: exec (shell), http.request (HTTP client), and wasm (Wazero WASI modules).
Package flow_version provides the @flow_version directive, which rejects a .nflow file at compile time when the running compiler does not satisfy the declared version constraint.
Package flow_version provides the @flow_version directive, which rejects a .nflow file at compile time when the running compiler does not satisfy the declared version constraint.
Package fs ships the file-system stream source and operators: fs.walk as a source; fs.filter, fs.read, fs.sort, fs.write, and the terminal sinks out.stdout and out.file as stream operators.
Package fs ships the file-system stream source and operators: fs.walk as a source; fs.filter, fs.read, fs.sort, fs.write, and the terminal sinks out.stdout and out.file as stream operators.
Package hook provides the `@hook:name` directive, which attaches every named hook from the kernel registry to the compiled pipeline, and the hook.probe action that reads HookProbeKey from the context to prove a hook fired.
Package hook provides the `@hook:name` directive, which attaches every named hook from the kernel registry to the compiled pipeline, and the hook.probe action that reads HookProbeKey from the context to prove a hook fired.
Package include provides the `@include "file.nflow"` directive.
Package include provides the `@include "file.nflow"` directive.
Package io ships process-stdio stream primitives: io.stdin reads newline-delimited items from standard input; io.stdout and io.stderr write each item as a line and pass it through.
Package io ships process-stdio stream primitives: io.stdin reads newline-delimited items from standard input; io.stdout and io.stderr write each item as a line and pass it through.
Package loop provides the `loop(body) until(cond)` keyword.
Package loop provides the `loop(body) until(cond)` keyword.
Package macros extends the compiler with a @macro engine.
Package macros extends the compiler with a @macro engine.
Package modifiers_auth ships the identity guards :auth, :role=, :perm=, and :feature=.
Package modifiers_auth ships the identity guards :auth, :role=, :perm=, and :feature=.
Package modifiers_core ships the execution modifiers :timeout=, :retry=, :concurrency=, :cache=, :coalesce, :dedup, :idempotent, and :rate_limit=.
Package modifiers_core ships the execution modifiers :timeout=, :retry=, :concurrency=, :cache=, :coalesce, :dedup, :idempotent, and :rate_limit=.
Package modifiers_meta ships the metadata modifiers :name=, :desc=, :description=, :status=, :tag=, :scope=, and the flags :read_only, :audit, :debug, :deprecated, :strict, :lenient.
Package modifiers_meta ships the metadata modifiers :name=, :desc=, :description=, :status=, :tag=, :scope=, and the flags :read_only, :audit, :debug, :deprecated, :strict, :lenient.
Package nodes_bench provides bench.run (measure latency distribution of an action), bench.save (write results to a file), and bench.compare (diff against a stored baseline).
Package nodes_bench provides bench.run (measure latency distribution of an action), bench.save (write results to a file), and bench.compare (diff against a stored baseline).
Package nodes_dispatch provides the `dispatch.run` action, which tries explicitly listed actions in order and returns the first successful result.
Package nodes_dispatch provides the `dispatch.run` action, which tries explicitly listed actions in order and returns the first successful result.
Package nodes_distribute provides distribute.map (bounded-concurrency fan-out over a slice) and distribute.reduce (fold the map output into a single value).
Package nodes_distribute provides distribute.map (bounded-concurrency fan-out over a slice) and distribute.reduce (fold the map output into a single value).
Package nodes_log provides log.info, log.warn, and log.error as pass-through nodes.
Package nodes_log provides log.info, log.warn, and log.error as pass-through nodes.
Package nodes_supervisor provides the supervisor action: it compiles and runs a set of named child pipelines concurrently, isolating each child's panic, error, and timeout.
Package nodes_supervisor provides the supervisor action: it compiles and runs a set of named child pipelines concurrently, isolating each child's panic, error, and timeout.
Package on provides the `@on event "protocol:target"` directive, which records an EventTrigger into meta["on_event"].
Package on provides the `@on event "protocol:target"` directive, which records an EventTrigger into meta["on_event"].
Package on_error provides the `@on_error { when ...
Package on_error provides the `@on_error { when ...
Package pipeline provides the `@pipeline NAME ...
Package pipeline provides the `@pipeline NAME ...
Package pool provides the `@pool NAME [member1, member2] { strategy: "round_robin" }` directive.
Package pool provides the `@pool NAME [member1, member2] { strategy: "round_robin" }` directive.
Package progress ships a TTY progress reporter and two actions that publish progress events through kernel/xctx.
Package progress ships a TTY progress reporter and two actions that publish progress events through kernel/xctx.
Package projection adds the `{ ...
Package projection adds the `{ ...
Package render provides the render.markdown stream operator, which reformats FileContent into a single Markdown document.
Package render provides the render.markdown stream operator, which reformats FileContent into a single Markdown document.
Package retry adds retry resilience to individual Flow atoms through the AtomAdvise contract.
Package retry adds retry resilience to individual Flow atoms through the AtomAdvise contract.
Package runtime ships the primitive actions every .nflow pipeline starts from: runtime.const, runtime.noop, runtime.debug, runtime.fail, runtime.pick, runtime.wrap, runtime.with, runtime.env, runtime.uuid, runtime.call, runtime.dispatch_by_prefix, json.clean, runtime.sleep.
Package runtime ships the primitive actions every .nflow pipeline starts from: runtime.const, runtime.noop, runtime.debug, runtime.fail, runtime.pick, runtime.wrap, runtime.with, runtime.env, runtime.uuid, runtime.call, runtime.dispatch_by_prefix, json.clean, runtime.sleep.
Package schema provides the `@schema NAME { Field Type `json:"..." validate:"..."` ...
Package schema provides the `@schema NAME { Field Type `json:"..." validate:"..."` ...
Package scope provides the `@scope :mods { ...
Package scope provides the `@scope :mods { ...
Package selftestkit ships the coverage fixtures the self-test suite references as `cov.*`.
Package selftestkit ships the coverage fixtures the self-test suite references as `cov.*`.
Package syntax ships the four binary operators every .nflow file relies on: ->, |, &, ||.
Package syntax ships the four binary operators every .nflow file relies on: ->, |, &, ||.
Package template provides a reusable Go text-template renderer and exposes it to Flow as the template.render action.
Package template provides a reusable Go text-template renderer and exposes it to Flow as the template.render action.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL