README
¶
BPMN conformance suite
A curated collection of small BPMN models plus the oracles that prove Atlas
executes them correctly — the Milestone-1 roadmap item "Conformance tests
against a curated BPMN model set". See the package doc in doc.go for
the design rationale; this file is the how-to.
The suite keeps two questions apart:
- Completeness — do we cover every BPMN execution feature? Answered by an
explicit register (
scenario.go) mapped onto the recognized workflow control-flow patterns. A feature with no covering scenario shows up as a gap inCOVERAGE.md, so "we cover everything" is a checkable claim, not a vibe. - Correctness — does the engine produce the right result, not merely a self-consistent one? Answered by four layered oracles.
The five oracles
| Oracle | What it proves | Where |
|---|---|---|
| Golden trace | The token path, variables, and data objects a model produces match a reviewed baseline. | golden/*.trace, compared in TestScenarios |
| Replay equivalence | State rebuilt from the log alone equals the live state — invariant I4, on every model for free. | Run → replayLog |
| Structural invariants | Model-independent truths, e.g. a completed instance leaves no orphan tokens. | executeLive |
| Metamorphic | Behaviorally equivalent models (concurrent vs. sequential independent effects) reach the same effect projection despite different shapes — no reference engine needed. | TestMetamorphic |
| Differential | The outcome matches an independent engine (Node's bpmn-engine) — the one oracle that catches Atlas being consistently wrong. Control-flow subset, opt-in. | differential/ |
The first four trust Atlas alone (self-consistency); the differential is the only
one that compares against a second implementation. See differential/README.md.
Runs are deterministic — the precondition for golden files. Self-completing
models (inline FEEL scripts) need no help; models that park a token carry a
Driver: an ordered list of steps that advance the wait deterministically. How
the instance is born is a separate axis — an explicit start by default, or a
message/timer start event that springs it from a trigger.
A model may declare more than one <process> (a call activity needs its child
deployed alongside the caller); the runner compiles and deploys them all, and the
scenario's Root names which one to instantiate. The captured trace is the root
instance's — child instances a call activity spawns are filtered out by definition
key.
The driver
A scenario's Driver is a list of steps, applied to the live run in order with a
RunUntilIdle after each. The steps only touch the live run — replay rebuilds
from the log the events they produced, so replay equivalence still holds
unchanged. Three step kinds cover the parking mechanisms:
| Constructor | Drives | Engine call |
|---|---|---|
Complete("task", Str("k","v")…) |
a parked job (user or service task), writing outputs | CompleteJob |
Publish("msg", "corrKey", …) |
a waiting message subscription | PublishMessage |
Wait(30*time.Second) |
the clock past a timer's due date, firing it | clock advance + TickTimers |
Fail("task", "message") |
a job to failure with no retries, raising an incident | FailJob |
Resolve("task") |
the incident on a task, re-activating its job | ResolveIncident |
ThrowError("task", "CODE") |
a job to throw a business error a boundary catches | ThrowJobError |
Complete resolves the job by the task's BPMN id (job → element instance →
compiled element), and fails loudly if the id names no parked job or an ambiguous
one — a mis-authored step is a test error, not a wrong-token run. Resolve is
self-verifying the same way: it errors if there is no incident on the named task,
so an incident scenario can't pass without the incident actually being raised.
For start events the instance has no CreateInstance at all; the scenario's
Start field says how it is born:
| Start | Births the instance by | Engine call |
|---|---|---|
| (zero value) | an explicit create — the default for a none start event | CreateInstance |
MessageStart("msg", "corrKey") |
publishing to a message start event | PublishMessage |
TimerStart(30*time.Second) |
arming the start timer and advancing the clock past it | ArmStartTimers + TickTimers |
SignalStart("thrower") |
instantiating a trigger process whose signal throw broadcasts to the root's signal start event | CreateInstance (on the trigger) |
Running
go test ./conformance/ # assert against the committed goldens
go test ./conformance/ -update # regenerate goldens + COVERAGE.md
go test -race ./conformance/ # with the race detector
A regenerated golden is a behavior change: review the diff before committing.
COVERAGE.md is generated too — never hand-edit it; a test fails if it drifts.
Adding a scenario
- Drop a BPMN model under
models/. If it parks a token, note which step advances each wait; otherwise make it self-completing with inline<zeebe:script>tasks. Fixtures intentionally omit BPMN-DI (they are executed, not rendered). - Register it in
Scenariosinscenario.go, listing theFeaturesit exercises and, for a parking model, theDriversteps that carry it to an end. Add newFeature/Patternentries if needed. For a metamorphic pair, give both members the sameEquivClass. go test ./conformance/ -updateto mint the golden and refreshCOVERAGE.md.- Review both generated files, then commit.
Negative models
The adversarial half of the collection. A NegativeModel is well-formed XML that
is nonetheless structurally invalid; TestNegativeModels asserts the compiler
rejects each one, because "garbage refused at deploy" is as much a correctness
property as "valid model runs" (invariant 5). They are listed in COVERAGE.md.
Add one by dropping a neg-*.bpmn under models/ and registering it in
NegativeModels with the reason it must fail.
What's next
The suite covers self-completing control flow (exclusive, parallel, and inclusive gateways), the parking features the driver reaches (user/service tasks, messages, timers, receive tasks, boundary timer/message/error/signal events, event-based gateway, all four start-event kinds), the incident lifecycle, signal throw/catch, compensation, embedded subprocess, parallel and sequential multi-instance, call activity, transaction/cancel, first-class data objects (whole-object, field-level, and collection), and a growing set of negative models. Planned extensions, roughly in order:
- Broader coverage (data-object lineage, error/message end events, manual/send
tasks) — each a row that flips from 🔲 to ✅ in
COVERAGE.md. - More differential reference translations, growing the cross-engine subset
beyond pure control flow (see
differential/README.md). - More negative models as unsupported constructs are pinned (terminate end is the latest — the compiler rejects it rather than silently degrading it).
- Escalation events once the engine supports them — there is no
escalationEventDefinitionin the compiler yet, so there is no feature to exercise; this stays out until it lands. - More negative models as the compiler's validation grows (unroutable gateways, multiple defaults, cross-scope references).
- Optionally, a differential job comparing outcomes against a reference engine (Camunda/Zeebe) as the strongest external oracle.
Documentation
¶
Overview ¶
Package conformance is Atlas's curated BPMN conformance suite: a collection of small, deterministic process models plus the oracles that prove the engine executes them correctly. It fills the Milestone-1 roadmap item "Conformance tests against a curated BPMN model set".
The suite answers two separate questions that must not be conflated:
- Completeness — do we cover every BPMN execution feature? Tracked by an explicit register (Features, Patterns, Scenarios) mapped onto the externally recognized workflow control-flow patterns, so a feature with no scenario is a visible gap in COVERAGE.md rather than a blind spot.
- Correctness — does the engine produce the right result, not merely a self-consistent one? Checked by four layered oracles in the runner.
The four oracles, weakest-to-strongest independence:
- Golden trace. The engine is event-sourced, so a run's observable behavior is its token path plus the variables it wrote. Each scenario's RunResult is serialized and compared against a committed golden file. Regenerate with `go test ./conformance -update`; a changed golden is a behavior change and must be reviewed in the diff.
- Replay equivalence. Every scenario is replayed from its own log into a fresh store and must reproduce the live result byte-for-byte — the engine's load-bearing invariant "state after replay == state built live" (I4), exercised for free on every model.
- Structural invariants. Model-independent properties that must hold for any run: a completed instance leaves no orphan tokens, for example.
- Metamorphic equivalence. Behaviorally equivalent models (e.g. independent effects run concurrently vs. sequentially) must reach the same effect projection despite different control-flow shapes — a correctness check that needs no reference engine.
- Differential. The outcome must match an independent engine (Node's bpmn-engine) — the only oracle that compares against a second implementation rather than trusting Atlas alone. A control-flow subset, opt-in behind a build tag; see the differential subpackage.
Adding a scenario: drop a self-completing BPMN model under models/, register it in Scenarios (and any new Feature/Pattern), run `go test ./conformance -update` to mint its golden and refresh COVERAGE.md, and review both.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var Features = []Feature{ {"start-end-event", "None start and end events", nil}, {"sequence-flow", "Sequence flow between activities", []string{"WCP-1"}}, {"script-task", "Inline FEEL script task (in-engine, no worker)", nil}, {"exclusive-gateway", "Data-based exclusive gateway with default flow", []string{"WCP-4", "WCP-5"}}, {"parallel-gateway", "Parallel fork and synchronizing join", []string{"WCP-2", "WCP-3"}}, {"user-task", "User task (human-completed job)", nil}, {"service-task", "Service task (worker-completed job with outputs)", nil}, {"message-catch", "Intermediate message catch event", nil}, {"timer-catch", "Intermediate timer catch event", nil}, {"receive-task", "Receive task (message wait as an activity)", nil}, {"boundary-timer-interrupting", "Interrupting boundary timer event", nil}, {"boundary-message-noninterrupting", "Non-interrupting boundary message event", nil}, {"event-based-gateway", "Event-based gateway (deferred choice)", []string{"WCP-16"}}, {"message-start", "Message start event", nil}, {"timer-start", "Timer start event", nil}, {"incident", "Job failure raises an incident; resolve resumes it", nil}, {"boundary-error", "Interrupting boundary error event", nil}, {"signal", "Signal throw and catch (1:n broadcast)", nil}, {"embedded-subprocess", "Embedded subprocess", nil}, {"multi-instance", "Parallel multi-instance activity with output collection", nil}, {"multi-instance-sequential", "Sequential multi-instance activity", nil}, {"standard-loop", "Standard loop activity (repeat while a condition holds)", []string{"WCP-21"}}, {"call-activity", "Call activity invoking a child process", nil}, {"compensation", "Compensation via a boundary and a compensation throw", nil}, {"signal-boundary", "Interrupting boundary signal event", nil}, {"inclusive-gateway", "Inclusive (OR) gateway split and synchronizing join", []string{"WCP-6", "WCP-7"}}, {"signal-start", "Signal start event (broadcast births an instance)", nil}, {"data-object", "First-class data object: output/input associations and data state", nil}, {"field-level-data-object", "Field-level data-object writes (accrue members)", nil}, {"collection-data-object", "Collection data object (isCollection list)", nil}, {"transaction-cancel", "Transaction subprocess with cancel end and cancel boundary", nil}, }
Features is the register of execution features. A feature with no covering scenario surfaces as a gap in COVERAGE.md — that visibility is the point.
var NegativeModels = []NegativeModel{
{"neg-dangling-flow", "neg-dangling-flow.bpmn", "a sequence flow targets an element that does not exist"},
{"neg-boundary-bad-host", "neg-boundary-bad-host.bpmn", "a boundary event attaches to a host that does not exist"},
{"neg-unknown-message", "neg-unknown-message.bpmn", "a receive task references a message that is not declared"},
{"neg-loop-unbounded", "neg-loop-unbounded.bpmn", "a standard loop has neither a loop condition nor a loop maximum, so it could never end"},
}
NegativeModels is the adversarial half of the collection: each must fail to compile. TestNegativeModels asserts it; COVERAGE.md lists them.
var Patterns = []Pattern{
{"WCP-1", "Sequence"},
{"WCP-2", "Parallel Split"},
{"WCP-3", "Synchronization"},
{"WCP-4", "Exclusive Choice"},
{"WCP-5", "Simple Merge"},
{"WCP-6", "Multi-Choice"},
{"WCP-7", "Structured Synchronizing Merge"},
{"WCP-16", "Deferred Choice"},
{"WCP-21", "Structured Loop"},
}
Patterns is the subset of control-flow patterns the suite tracks so far. Grow it as scenarios reach further into the catalog.
var Scenarios = []Scenario{ {Name: "sequence", Model: "sequence.bpmn", Features: []string{"start-end-event", "sequence-flow", "script-task"}}, {Name: "exclusive-gateway", Model: "exclusive-gateway.bpmn", Features: []string{"exclusive-gateway", "script-task"}}, {Name: "parallel-independent", Model: "parallel-independent.bpmn", Features: []string{"parallel-gateway", "script-task"}, EquivClass: "independent-effects"}, {Name: "linear-independent", Model: "linear-independent.bpmn", Features: []string{"sequence-flow", "script-task"}, EquivClass: "independent-effects"}, {Name: "user-task", Model: "user-task.bpmn", Features: []string{"user-task"}, Driver: []Step{Complete("approve")}}, {Name: "service-task", Model: "service-task.bpmn", Features: []string{"service-task"}, Driver: []Step{Complete("charge", Str("status", "captured"))}}, {Name: "message-catch", Model: "message-catch.bpmn", Features: []string{"message-catch"}, Driver: []Step{Publish("payment-received", "K")}}, {Name: "timer-catch", Model: "timer-catch.bpmn", Features: []string{"timer-catch"}, Driver: []Step{Wait(31 * time.Second)}}, {Name: "receive-task", Model: "receive-task.bpmn", Features: []string{"receive-task"}, Driver: []Step{Publish("reply", "K")}}, {Name: "boundary-timer-interrupting", Model: "boundary-timer-interrupting.bpmn", Features: []string{"boundary-timer-interrupting"}, Driver: []Step{Wait(31 * time.Second)}}, {Name: "boundary-message-noninterrupting", Model: "boundary-message-noninterrupting.bpmn", Features: []string{"boundary-message-noninterrupting"}, Driver: []Step{Publish("ping", "K"), Complete("review")}}, {Name: "event-gateway-message", Model: "event-based-gateway.bpmn", Features: []string{"event-based-gateway"}, Driver: []Step{Publish("go", "")}}, {Name: "event-gateway-timer", Model: "event-based-gateway.bpmn", Features: []string{"event-based-gateway"}, Driver: []Step{Wait(31 * time.Second)}}, {Name: "message-start", Model: "message-start.bpmn", Features: []string{"message-start"}, Start: MessageStart("order-placed", "")}, {Name: "timer-start", Model: "timer-start.bpmn", Features: []string{"timer-start"}, Start: TimerStart(31 * time.Second)}, {Name: "incident", Model: "incident.bpmn", Features: []string{"incident"}, Driver: []Step{Fail("risky", "boom"), Resolve("risky"), Complete("risky")}}, {Name: "boundary-error", Model: "boundary-error.bpmn", Features: []string{"boundary-error"}, Driver: []Step{ThrowError("call", "BOOM")}}, {Name: "signal-throw-catch", Model: "signal-throw-catch.bpmn", Features: []string{"signal"}}, {Name: "subprocess", Model: "subprocess.bpmn", Features: []string{"embedded-subprocess"}}, {Name: "multi-instance", Model: "multi-instance.bpmn", Features: []string{"multi-instance"}}, {Name: "multi-instance-sequential", Model: "multi-instance-sequential.bpmn", Features: []string{"multi-instance-sequential"}, Driver: []Step{Complete("step"), Complete("step"), Complete("step")}}, {Name: "standard-loop", Model: "standard-loop.bpmn", Features: []string{"standard-loop"}}, {Name: "call-activity", Model: "call-activity.bpmn", Features: []string{"call-activity"}, Root: "call-parent"}, {Name: "compensation", Model: "compensation.bpmn", Features: []string{"compensation"}, Driver: []Step{Complete("charge"), Complete("refund")}}, {Name: "signal-boundary", Model: "signal-boundary.bpmn", Features: []string{"signal-boundary"}}, {Name: "inclusive-gateway", Model: "inclusive-gateway.bpmn", Features: []string{"inclusive-gateway"}}, {Name: "signal-start", Model: "signal-start.bpmn", Features: []string{"signal-start"}, Root: "on-signal", Start: SignalStart("thrower")}, {Name: "data-object", Model: "data-object.bpmn", Features: []string{"data-object"}}, {Name: "data-object-fields", Model: "data-object-fields.bpmn", Features: []string{"field-level-data-object"}}, {Name: "collection-data-object", Model: "collection-data-object.bpmn", Features: []string{"collection-data-object"}}, {Name: "transaction-cancel", Model: "transaction-cancel.bpmn", Features: []string{"transaction-cancel"}, Driver: []Step{Complete("reserve"), Complete("unreserve")}}, }
Scenarios is the curated collection. Self-completing models (inline scripts) carry a nil Driver; models that park a token carry the deterministic steps that advance them — a completed job, a delivered message, an elapsed timer.
Functions ¶
func CoverageReport ¶
func CoverageReport() string
CoverageReport renders the feature/pattern coverage matrix as Markdown from the live register (Features, Patterns, Scenarios), so a feature added without a covering scenario shows up as an explicit gap. The report is committed as COVERAGE.md and a test fails if it drifts.
Types ¶
type Feature ¶
Feature is one BPMN execution feature the engine must cover. Patterns lists the control-flow patterns it realizes, if any (many features — inline scripts, events — are not control-flow patterns and leave it empty).
type NegativeModel ¶
NegativeModel is a well-formed BPMN model that is nonetheless invalid and must be rejected at compile — proving the engine refuses garbage at deploy rather than running it into undefined behavior (invariant 5). Reason documents why.
type Pattern ¶
Pattern is a workflow control-flow pattern (van der Aalst et al.) — the external yardstick the collection measures its coverage against, so "we cover all features" is a claim against a recognized catalog rather than a self-defined checklist.
type RunResult ¶
type RunResult struct {
State model.ProcessInstanceState
Path []string
Variables map[string]string
DataObjects []string // "name[state]=value", sorted; empty for models with none
}
RunResult is a scenario's observable behavior: the terminal instance state, the ordered token path (BPMN element ids a token activated), and the final root-scope variables. It is the unit both the golden and replay oracles compare.
func Run ¶
Run compiles the model (every executable process in it — a call activity needs its child deployed too), executes the root process live while applying the driver steps, replays its log into a fresh store, and returns the live result. It fails if replay diverges from live (invariant I4) or if a completed instance left orphan tokens. root is the BPMN id of the process to instantiate; pass "" when the model has exactly one process. base must be an empty directory unique to this call.
type Scenario ¶
type Scenario struct {
Name string // model base name; also the golden file base name
Model string // file under models/ (several scenarios may share one model)
Features []string // feature IDs this scenario exercises
EquivClass string // non-empty: metamorphic equivalence group
Root string // BPMN id of the process to instantiate; "" = the sole process
Start Start // how the instance is born; zero value = explicit CreateInstance
Driver []Step // ordered actions that drive parked tokens; nil = self-completing
}
Scenario binds a BPMN model to the features it exercises, how its instance is born, and the driver steps that carry it through any parked waits. A non-empty EquivClass marks it as one of a metamorphic group: every scenario sharing that class must produce the same effect projection (see RunResult.Effect).
func (Scenario) LoadModel ¶
LoadModel exposes a scenario's embedded BPMN model for tooling (the TCK generator, the catalog) that lives outside this package.
func (Scenario) PatternsOf ¶
PatternsOf returns the sorted-unique control-flow patterns a scenario realizes, resolved through the features it exercises.
type Start ¶
type Start struct {
// contains filtered or unexported fields
}
Start says how a scenario's instance is born. The zero value is an explicit CreateInstance (the default for models with a none start event); message-, timer-, and signal-start events instead spring the instance from a trigger, so no CreateInstance is ever called on the root for them.
func MessageStart ¶
MessageStart births the instance by publishing a message to a message start event, rather than by an explicit CreateInstance.
func SignalStart ¶
SignalStart births the instance by instantiating the trigger process (named by its BPMN id), whose signal throw broadcasts to the root's signal start event.
func TimerStart ¶
TimerStart arms the definition's timer start event and advances the clock by after so the timer comes due and births the instance.
func (Start) MarshalJSON ¶
MarshalJSON renders how the instance is born, e.g. {"kind":"signal","trigger":"thrower"}.
type Step ¶
type Step struct {
// contains filtered or unexported fields
}
A Step drives a parked instance one action forward during the live run: it completes a job, delivers a message, or advances the clock past a timer. Steps only touch the live run — replay rebuilds from the log the events they produced, so replay equivalence (I4) still holds unchanged. Build steps with the Complete, Publish, and Wait constructors so the register reads declaratively.
func Complete ¶
Complete completes the parked job of the task with the given BPMN id, writing any vars as outputs — the driver stands in for the human (user task) or worker (service task).
func Fail ¶
Fail fails the parked job of the given task with no retries left, raising an incident (message becomes the incident message).
func Publish ¶
Publish delivers a message by (name, correlation key) with an optional payload, correlating any waiting subscription.
func Resolve ¶
Resolve clears the incident on the given task's element with fresh retries, re-activating its job. It fails if no incident is present there.
func ThrowError ¶
ThrowError makes the given task's job throw the business error code instead of completing, so a matching boundary error event catches it (code carried in the message field).
func Wait ¶
Wait advances the clock by d and fires every timer that has come due — the deterministic stand-in for wall-clock time passing.
func (Step) MarshalJSON ¶
MarshalJSON renders a driver step as a tagged object, e.g. {"action":"complete","element":"charge","vars":{"status":"captured"}}.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package differential is the conformance suite's cross-engine oracle: it runs the same process on Atlas and on an independent reference BPMN engine and compares a normalized outcome, so a control-flow bug shows up as disagreement with a second implementation — the one oracle the suite's other checks (golden, replay, invariants, metamorphic) can't provide, since they all trust Atlas alone.
|
Package differential is the conformance suite's cross-engine oracle: it runs the same process on Atlas and on an independent reference BPMN engine and compares a normalized outcome, so a control-flow bug shows up as disagreement with a second implementation — the one oracle the suite's other checks (golden, replay, invariants, metamorphic) can't provide, since they all trust Atlas alone. |