37_mvcc_write_contention

Example 37 — MVCC under concurrent WRITERS

Scenario

An order-processing service ingests from several producers at once. Each producer writes to its own customer records, but every order also touches a small set of shared inventory nodes — so most transactions are independent and a predictable fraction genuinely collide. Readers run throughout, answering queries while the ingest continues.

That shape is deliberate. A workload with no contention cannot demonstrate that conflict detection works; one where everything contends cannot demonstrate that independent writers scale. This example has both, in a ratio the operator sets.

Objective

Exercise and measure the write side of GoGraph's MVCC, which is the half the two existing MVCC examples do not reach:

• example 35 measures reader latency under a mixed workload;
• example 36 checks snapshot isolation on the topology dimension.

Neither drives concurrent writers, so neither can observe a write-write conflict, a retriable serialization error, a logical abort, read-your-own-writes inside an open transaction, or a commit timestamp surviving a restart. Every one of those is a property sprint 334 added.

Purpose

Produce evidence, not a demonstration. The example must let a reader answer, from its output alone:

1. Does write throughput scale with writer count? commits/s at 1, 2, 4, 8, 16 writers, with the ratio against one writer stated. This is the number the sprint exists to move.
2. What does contention actually cost? conflict rate by store, retry count, and retry success — measured, with the hot-spot fraction that produced it.
3. Are aborts correct? a rolled-back transaction must leave no trace. Asserted by a conservation invariant that no external oracle can supply: the sum of a quantity that every transaction preserves.
4. What do writers cost readers? reader latency percentiles taken beside the writers, not after them.
5. What does versioning retain? version-chain depth distribution and version count against the bound, sampled while the workload runs.
6. Does it survive a restart? the data AND the MVCC commit clock, so a post-restart transaction cannot re-mint an instant a previous process published.

Why it must be able to FAIL

The standing rule in this project is that an instrument which cannot fail on the defective build proves nothing — three of GoGraph's own instruments have been caught reporting a number they could only ever have produced.

So this example is validated in both directions, and the result is recorded here:

• run against a worktree at the pre-sprint head, or with the property deliberately broken, it must FAIL;
• the failure output is reproduced in this README so a reader can see what a broken build looks like, not merely be told it would differ.

The self-contradictory check

The bracket-invariant discipline that made example 36 able to detect — bracketing each observation between acknowledged commits — is structurally blind to a read of the wrong legal instant: a stale-but-valid snapshot passes every bracket.

So this example also carries at least one check that needs no external oracle: a statement whose two halves must agree with each other within one transaction, so a disagreement is self-evidently a defect regardless of which instant was read. The conservation invariant in (3) is that check — a transfer that debits one node and credits another cannot change the total, whatever instant observes it.

Its cost is stated, and it is sized for the gate that has to run it. A check nobody can afford to run is not a check.

Evidence collected

Per the project's examples mandate, every relevant indicator is measured:

Vector	How
CPU	runtime/pprof CPU profile over the workload
Heap	runtime/pprof heap profile plus runtime.MemStats sampling
Scheduling	runtime/trace, to see writers blocking and GC pauses
Latency	per-operation histograms, reported as percentiles
Storage	on-disk store size before and after, and its growth
Coverage	go build -cover + GOCOVERDIR, reported with go tool covdata

Running it

go run ./examples/37_mvcc_write_contention \
    -producers 8 -ops-per-producer 200 -customers 1000 -readers 4 \
    -profile-dir /tmp/ex37 -trace /tmp/ex37/trace.out

Bare lines carry deterministic facts — invariant verdicts and counts that hold on any machine, and which example_test.go asserts. Lines prefixed with # carry volatile telemetry that varies per run and per machine, and which no test pins: a gate that asserted on throughput would fail on a loaded machine and teach the next reader to ignore a red test.

A representative run (Apple M4, 10 cores):

## phase 1 — writer scaling
# scaling writers=1  commits=1600 commits_per_sec=834891  ratio_vs_1=1.00x
# scaling writers=2  commits=1600 commits_per_sec=1109313 ratio_vs_1=1.33x
# scaling writers=8  commits=1600 commits_per_sec=1675540 ratio_vs_1=2.01x
scaling.unrecovered_conflicts=0
## phase 2 — contention, reader latency, version retention
# contention commits=1600 hot_orders=410 hot_pct_actual=25.6
# contention retried_orders=20 retry_succeeded=20 unrecovered=0
# conflicts.total=20 aborts=20 commits=2604
# conflicts.by_store.node properties=20
# versions.samples=25 versions.max_retained=2331 versions.bound=4096 versions.ceiling=16384 max_chain_depth=10 max_concurrent_writers=8
# reader.samples=8606 reader.p50=416ns reader.p95=1.083µs reader.p99=10.208µs
contention.unrecovered_conflicts=0
versions.sampled=true
versions.writer_peak_consistent=true
## phase 3 — conservation under concurrent transfers
# conservation transfers=1483 refused=0 observations=25842
conservation.torn_observations=0
conservation.final_total_correct=true
## phase 4 — restart: the data and the MVCC clock
# restart nodes=200 store_bytes=25536 clock_before=200 clock_after_reopen=400 clock_after_write=401
restart.all_nodes_recovered=true
restart.clock_not_rewound=true
restart.post_restart_instant_is_new=true

Reading it against the six questions: writers scale (2.01× at eight, on a workload where a quarter of orders contend); contention costs 20 conflicts over 2604 transactions, every one recovered by the retry loop and every one attributed to the node-property store; readers see p99 10.2 µs beside the writers; versioning retained 1854 versions against a 4096 bound with a deepest chain of 7; and the clock came back at 400 after a restart, above the 200 the previous process published.

Validated in both directions — and it caught a real defect

The check earned its place before the example was finished.

The first draft read balances with the present-time g.GetNodeProperty inside the transfer, rather than through the transaction's own view — a textbook lost update: two transfers read the same balance, both write, and the second silently overwrites the first. The conservation invariant reported it immediately:

# conservation transfers=221 refused=0 observations=109
conservation.torn_observations=97
conservation.final_total_correct=false
# conservation.first_torn_total=15999019 want=16000000

981 cents gone, from a run that reported no errors and no refused transactions. The check caught the example's own bug before it could be mistaken for the engine's, which is exactly what a self-contradictory check is for: no external oracle was consulted, and none was needed, because the two halves of one transaction must agree with each other.

TestConservationCheckCanFail keeps that ability under test. It injects the same shape into a real graph — a debit with no matching credit — and asserts the real observer reports it, with a positive arm that fails if the observer is simply wrong about an untouched fixture. Asserting the arithmetic instead would be a tautology that passes however broken the observer became.

The version sampler could not answer question 5, and said nothing about it

Question 5 — what does versioning retain? — is answered by sampling MVCCStats while the workload runs. The sampler ticked every 2 ms, and phase 2 takes 2 ms. It therefore got between zero and one observation, and published whichever it happened to get.

Measured over twelve runs of the documented default shape above:

runs	max_retained	max_chain_depth	max_concurrent_writers
7 of 12	0	0	0
5 of 12	568 – 3983	0, 5, 5, 5, 20	0, 2, 5, 8

Every one of the seven zero runs had between 18 and 192 conflicts. A write-write conflict is two writers overlapping, so max_concurrent_writers=0 beside a non-zero conflict count is a self-contradiction: all three zeros were a sampling miss published as a measurement. The representative run in this README — 1854 retained, depth 7 — was one draw from that distribution, not a typical result.

This is the defect class the project keeps rediscovering, and the one the rest of this example already defends against: contention.readers_sampled and conservation.observed_any exist precisely so a phase that observed nothing cannot pass vacuously. The version sampler had no such guard.

Three changes, and the third is the one that matters:

1. the tick is 100 µs, so a 2 ms phase yields ~20 observations rather than one;
2. the phase is bracketed — one sample taken on the calling goroutine before the writers start, one after they stop — so even a phase shorter than a tick observes something;
3. versions.sampled and versions.writer_peak_consistent are deterministic fact lines the gate asserts, so a miss now fails the test instead of printing zeros.

After the change, twelve runs of the same shape sampled 17–65 times each, reported max_concurrent_writers=8 — the true producer count — in all twelve, and gave max_retained in a 2278–4023 band with a non-zero chain depth in nine. The guard is validated in the failing direction too: with the sampler's tick set beyond the run's lifetime and the brackets removed, TestRun fails on the missing versions.sampled=true line rather than passing on zeros.

Note that max_retained may legitimately exceed versions.bound. Bound is the settled-churn threshold at which the vacuum is woken, not a cap; Ceiling (16384) is the instantaneous bound. A maximum-contention run measured 10184 retained — above the bound, well under the ceiling, which is exactly the documented catch-up state.

Examples 35 and 36 under non-serialising writers

Both still pass, and both remain meaningful now that writers overlap rather than serialise — verified rather than assumed:

• 36 brackets every observation between the ingest's acknowledged-commit counter before and after the query, which is precisely a construction that does not assume serialisation, and it refuses to report success when observations == 0 or when the self-contradiction query never ran. Its verdicts are unaffected.
• 35 measures throughput collapse and latency amplification — ratios whose whole subject is concurrency — and guards the zero-sample case. Nothing in it assumed a single writer.

Neither needed a change. What they did need was checking, because a verdict that was sound under exclusion is not automatically sound without it.

The zero-unrecovered checks

scaling.levels printed the length of a constant slice and contention.accounted compared committed + unrecovered with a total the loop makes true by construction; neither could fail. Both are replaced by scaling.unrecovered_conflicts=0 and contention.unrecovered_conflicts=0: the number of orders that did not commit, computed as the target minus the commits. An order is unrecovered when its retry budget (2 s of conflicts) expires, or when the phase's hang budget expires before it runs. The hang budget is 30 s per scaling level and for the contention phase, three to four orders of magnitude above a healthy phase, so a stuck or livelocked writer ends the phase with a non-zero count instead of hanging the run.

Phase 5 — deterministic MVCC scenario catalogue (rmp #2933)

catalogue.go ports the write-write (§1.1) and snapshot-isolation (§1.2) rows of docs/mvcc-scenario-catalogue.md, catalogue_mgix.go its MERGE/UNIQUE (§1.3) and index (§1.4) rows, catalogue_ddhz.go its DDL-vs-DML (§1.5), abort (§1.6) and horizon, read-only-transaction and session (§1.7) rows, and catalogue_rigg.go its referential-integrity and graph-shape (§1.8) rows, the frontier proposals FP01-FP03 (§5) and the frontier rows of §1.7 that need a held commit (SE02, SE04 and SE03's fsync arm), from PostgreSQL's isolation specs and InnoDB's tests. The PostgreSQL scenarios are re-implemented, not copied. Each row is an internal/isolationtest spec run over every order-preserving interleaving of its steps (or over named interleavings where the catalogue requires them) and pinned by a golden transcript. The transcript ends with a final block — the state the permutation left behind — added to the harness as Spec.Final for this catalogue.

The example prints one deterministic line per scenario:

catalogue.WW02.cypher spec=ww02-stale-snapshot-write permutations=6 steps=30 ok=18 conflicts=6 poisoned=6 other_errors=0 blocked=0 violations=0

steps counts every observed step, final included; conflicts are mvcc.ErrSerializationConflict; poisoned are cypher.ErrTxPoisoned; blocked counts <waiting ...>, invalid-permutation and never-completed lines, and must be 0 (GoGraph takes no DML locks, F7); violations counts property-check failures. The summary lines catalogue.scenarios, catalogue.blocked_steps=0 and catalogue.violations=0 are printed by the binary; TestRun does not run phase 5 (rmp #2993): TestCatalogue runs the same scenarios through the same runner, checks every property, and asserts every transcript against its golden.

Harness

• Two drivers (H3). Every row runs through cypher.Engine. Where the shape exists in the lpg API — property reads and writes, label removal and property delete inside Session.BeginVersionedTx — a second spec (-lpg suffix) drives the same graph through lpg.Graph, using the harness's Probe step (a Go step that reports rows; added to internal/isolationtest for this purpose). Rows that need ROLLBACK run through Cypher only: the undo log a rollback replays belongs to the Cypher engine; an abort through lpg (WriteTx.Abandon, then EndVersionedTx) is used where no eager write has to be undone (HZ03). The DD rows, AB01, AB02, AB04 and AB05 run through Cypher only: the lpg API has no DDL, no constraint registry, no statement that can fail midway and no undo of eager writes. The MG and IX rows run through Cypher only: the lpg API has no MERGE, the UNIQUE and NOT NULL constraints are the Cypher engine's registry, and a secondary index is maintained only by the change fan-out the Cypher engine drives at commit (lpg.ErrIndexedRawWrite), so an lpg arm would test that contract rather than the row.
• Vacuum drain (H1). A Hook step calls lpg.Graph.ReclaimNow. Every write-after-rollback row places it in the rolling-back session directly after the ROLLBACK, and runs only the interleavings in which no other step falls between the two, so the drain is at a fixed point in every run.
• Goldens live in internal/isolationtest/testdata. That is decided by the harness: isolationtest.Check resolves testdata/<spec>.golden against its own source directory. Catalogue goldens carry the row ID as a prefix (ww02-…, sk12-…, mg02-…, ix04-…).
• Property checks. WW15 (pinned reader byte-identical), SK10 (seek = scan), SK12 (repeatable count; read-skew freedom), SK14 (fast path = scan), every IX row, DD01-DD05, DD09, AB02 and AB05 (seek = scan), AB04 (final value = acknowledged increments), HZ01 (versions rise while the reader is held and fall after), RO01 (repeatable read) and RO03 (parallel count = scan) also assert a property on every step, independently of the golden. Every RI and GG row asserts no dangling edge; GG01, GG03 and GG04 also assert repeatable reads, and GG03 and GG04 seek = scan; FP02, SE02 and SE04 assert that their background wait was parked behind the held straggler.
• Go steps that drive the engine (§1.5-§1.7). A sessionless read (SE01, SE03), a statement cancelled while it executes (AB02: the statement parks in a procedure, cat.hold, registered on every engine, and is cancelled there), a client retry loop (AB04), and MVCCStats readings taken after a vacuum drain (HZ01, HZ04, SE03) reach the engine of the permutation in force through world.eng and world.g.
• The dangling-edge check (§1.8). Every RI and GG row ends with a dangling probe: at a fresh snapshot it walks every interned node through the lpg API and counts the arcs whose source or destination is not alive there, plus the in-neighbour entries of dead nodes. Every RI and GG scenario carries the noDanglingEdge property, so an edge to a dead node committed in any permutation fails the run (catalogue §9 point 3). The probe reads the adjacency below Cypher on purpose: a Cypher pattern binds only live nodes, so a dangling arc would vanish from a MATCH rather than show in it.
• Drivers for §1.8. The lpg arms use AddEdge, an lpg spelling of DETACH DELETE (RemoveEdge of each in-arc, RemoveAllEdgesFrom, RemoveNode), and the by-handle surfaces SetEdgePropertyByHandle and RemoveEdgeByHandle. GG01 (variable-length traversal), GG03 (degree and relationship count store) and GG05 (delete everything) are Cypher only: the lpg API has no traversal operator, no relationship count store and no bulk delete.
• The commit hold (FP01-FP03, SE02, SE04, SE03's fsync arm). lpg.Graph.AllocateCommitTS reserves a transaction's commit instant without publishing it — the first half of the WAL commit path, which allocates before its fsync and publishes after it. Called on an in-memory graph, with the transaction's EndVersionedTx in a later step, it holds a commit between allocation and publication at a fixed point of the interleaving. The straggler is therefore always an lpg transaction; the later commits and the reads run through either driver. A session statement that must wait for the straggler (SE02, SE04) runs on a background goroutine: a waiting probe reports it parked (SessionsWaiting = 1, not returned) and a later step collects its result, so no step of the transcript blocks.
• COMMIT of a poisoned transaction. The engine refuses it with ErrTxPoisoned and leaves the transaction open for the caller's ROLLBACK. The harness's COMMIT control now performs that rollback and still reports the COMMIT's error (internal/isolationtest/runner.go; TestCommitOfPoisonedTxRollsItBack fails without it: the peer's later write is refused by the leaked transaction). One golden depended on the leak: four permutations of ww10-crossing-writes, in which one transaction is poisoned and COMMITs before the other writes the node it held. The other transaction's write now succeeds and commits, as snapshot isolation allows after an abort; before, the leaked transaction refused it.
• Seek = scan in one statement (IX). Each index read returns two columns: seek, the predicate served from the index, and scan, the same predicate spelled so no index can serve it (m.s + '', m.k + 0, left(m.s, n)). The scan arm is an OPTIONAL MATCH: with a plain MATCH, a scan that finds nothing leaves no row to group, the statement returns no row, and a stale index entry would vanish from the transcript instead of disagreeing in it. TestIXAccessPaths proves that every seek arm is planned as an index access and every scan arm as a label scan. The planner estimates a btree or numeric predicate from the committed value domain and plans a label scan for a bound outside it, so the btree and numeric rows write values inside that domain ('v999x', k = 16).

Rows

The catalogue's #2933 section has 89 rows. 88 are pinned: 84 by a catalogue spec and four (WW01, SK01, SK06, SK09) by goldens the harness already had. One is deferred: MG11, to random load (#2934), because no hook exists inside MERGE between its probe and its create (G5). The three §5 proposals FP01-FP03 are pinned as well.

Expected outcomes are snapshot isolation (F1), not serializable snapshot isolation: GoGraph permits write skew and refuses a write-write conflict.

Row	Spec (golden)	Driver	Source	Expected SI outcome	GoGraph outcome, and why it differs from the source
WW01	lost-update (existing)	Cypher	PG deadlock-simple, lock-committed-update; MY concurrent.inc	Second writer refused	As expected. InnoDB REPEATABLE READ permits the lost update; SI cannot.
WW02	ww02-stale-snapshot-write, -lpg	both	PG lock-committed-update, lock-committed-keyupdate, eval-plan-qual	Writer refused when the peer's version is in flight or committed after its snapshot	As expected. PG READ COMMITTED re-fetches and succeeds.
WW03	ww03-blind-increment, -lpg	both	MY innodb_bug52663, innodb_bug49164; PG eval-plan-qual	Second increment refused; final 1	As expected. InnoDB and PG READ COMMITTED reach 2 without an error.
WW04	ww04-update-delete-commit, ww04-update-rollback-then-delete, ww04-delete-rollback-then-update, both rollback arms also -explicit	Cypher	PG eval-plan-qual, lock-update-delete, merge-delete; MY concurrent.inc	Later writer refused; allowed after the earlier rolls back and the vacuum drains	As expected. A DETACH DELETE that conflicts reports it at COMMIT, not at the statement; nothing is applied. Each rollback arm runs twice: with an autocommit second writer, and (-explicit) with an explicit transaction whose snapshot predates the rollback (D1, rmp #2973).
WW05	ww05-update-delete-chain-commit, ww05-update-delete-chain-rollback	Cypher	PG lock-update-delete, lock-update-traversal, aborted-keyrevoke	Older-snapshot peer refused once the chain commits; allowed after the chain rolls back and the vacuum drains	As expected in both arms. The rollback arm is defect D1's shape and witnesses its fix (rmp #2973).
WW06	ww06-predicate-moved-out	Cypher	PG eval-plan-qual, merge-match-recheck, merge-update; MY innodb-semi-consistent	Older snapshot still matches and is refused	As expected. PG READ COMMITTED and InnoDB's semi-consistent read skip the row instead.
WW07	ww07-non-matching-not-blocked	Cypher	MY innodb-semi-consistent, innodb-consistent	No wait, no conflict	As expected.
WW08	ww08-different-properties-one-node, -lpg	both	MY concurrent.inc; PG update-locked-tuple	Second refused: the node is the conflict unit	As expected; same verdict as PG and InnoDB row granularity.
WW09	ww09-disjoint-nodes, -lpg	both	docs/isolation-design.md	Zero conflicts in every interleaving	As expected.
WW10	ww10-crossing-writes, -lpg	both	PG deadlock-simple, fk-deadlock, fk-deadlock2; MY deadlock_detect, innodb_deadlock	No wait; refused instead	As expected; in some interleavings both transactions are refused, where PG and InnoDB pick one deadlock victim. When the refused transaction COMMITs (and is rolled back) before the other writes the node it held, that write succeeds.
WW11	ww11-ring-3, ww11-ring-8	Cypher	PG deadlock-hard; MY long_deadlock_cycle, undetected_deadlock, hp_deadlock	No wait; each write succeeds or is refused; progress after rollback	As expected. Named interleavings (round-robin, sequential; staggered for 3).
WW12	ww12-write-after-peer-rollback, -explicit	Cypher	PG eval-plan-qual, eval-plan-qual-trigger, multixact-no-forget; MY innodb_mysql_rbk	Write after rollback + drain succeeds	As expected, both for a writer whose snapshot follows the rollback and (-explicit) for an explicit transaction whose snapshot predates it (D1, rmp #2973). The no-drain arm is not pinned (a race, F5).
WW13	ww13-void-label-removal, -lpg; ww13-void-property-delete, -lpg; ww13-void-edge-property	both	GoGraph rmp #2354 (no PG/InnoDB counterpart)	Statement OK, COMMIT refused, nothing applied	As expected for label removal and property delete, through both drivers. The edge-property write is refused at the statement, not at COMMIT: the catalogue's F3 premise (edge side stores record the conflict) does not hold for SET r.w on an existing edge property. The outcome is still a refusal that applies nothing.
WW14	ww14-chain-walk-in-flight-head	Cypher	PG update-conflict-out, multiple-row-versions; MY innodb-read-view, lob_mvcc_undo	Pinned reader sees the original; a new reader sees the committed value, never the in-flight one	As expected.
WW15	ww15-long-chain-large-values	Cypher	PG multiple-row-versions; MY lob_mvcc_undo, lob_partial_update_concurrent	Pinned reader returns the 64 KiB value of its BEGIN, byte-identical	As expected. The value is written inline in the CREATE map (rmp #2975, fixed).
SK01	write-skew (existing)	Cypher	Berenson et al. A5B	Permitted	As expected.
SK02	sk02-swap-write-skew	Cypher	PG simple-write-skew	Permitted: both commit, sets swapped	As expected. PG SERIALIZABLE aborts one.
SK03	sk03-booking-overlap	Cypher	PG classroom-scheduling, temporal-range-integrity	Permitted: overlapping bookings	As expected.
SK04	sk04-cross-label-write-skew	Cypher	PG project-manager	Permitted	As expected.
SK05	sk05-total-cash, -lpg	both	PG total-cash	Permitted: total −200	As expected.
SK06	read-only-anomaly-named (existing)	Cypher	PG read-only-anomaly, read-only-anomaly-2	Permitted	As expected; PG SERIALIZABLE aborts.
SK07	sk07-receipt-report	Cypher	PG receipt-report	Permitted: the report misses a receipt that later commits into its batch	As expected, in two of the four named interleavings. The receipt's date is read inline in the CREATE map, as in the source spec; that read is the witness for D2 (rmp #2974, fixed).
SK08	sk08-two-ids	Cypher	PG two-ids	All commit in every interleaving	As expected.
SK09	bank-transfer (existing)	Cypher	examples/27	Total constant	As expected.
SK10	sk10-write-skew-hash-seek, sk10-write-skew-btree-range	Cypher	PG index-only-scan, predicate-hash, insert-conflict-serializable, matview-write-skew	Permitted; seek = scan in every step	As expected. 1 100 padding nodes make the planner seek; TestSK10AccessPaths proves one arm seeks and the other scans. The scan arm is an OPTIONAL MATCH, as in the IX rows, so an empty scan cannot hide a mismatch; the goldens changed only in the query text.
SK11	sk11-materialised-conflict-increment, sk11-materialised-conflict-value-preserving	Cypher	Fekete et al., TODS 2005; MY t/locking_clause	Increment arm: second refused, rule holds	Increment arm as expected. Value-preserving arm (SET g.v = g.v): both commit and the rule breaks — a write of the current value writes no version and claims nothing, so it is not a remedy (G4 pinned).
SK12	sk12-no-phantom-no-non-repeatable, sk12-read-skew-lpg	both	MY t/consistent_snapshot, select_count_perf, innodb-read-view	Repeated reads equal; no read skew	As expected. The lpg arm reads two keys (A5A) and is the negative control's target.
SK13	sk13-snapshot-at-begin, -lpg	both	MY t/consistent_snapshot; PG fk-snapshot	Snapshot at BEGIN	As expected. InnoDB's plain START TRANSACTION takes it at the first read.
SK14	sk14-fast-path-aggregates	Cypher	MY select_count_perf, parallel_read; PG index-only-bitmapscan	Fast path = scan, at the snapshot	As expected.
SK15	sk15-aborted-read-g1a, sk15-intermediate-read-g1b	Cypher	TU hermitage_tests.rs:150, :200, :862 (MIT; from Hermitage)	G1a: T2 reads 10 throughout; T3 reads 10 while T1 is open, after its ROLLBACK before the vacuum drain, and after the drain. G1b: 101 is never read; T3 reads 10 before the commit, 11 after	As expected, in 3 named interleavings each.
SK16	sk16-circular-information-flow-g1c	Cypher	TU hermitage_tests.rs:246 (MIT; from Hermitage)	Each writer reads the other's node at its snapshot (b = 20, a = 10); both commit; final (11, 22)	As expected, in all 20 interleavings.
SK17	sk17-observed-transaction-vanishes-otv	Cypher	TU hermitage_tests.rs:295 (MIT; from Hermitage)	T2 refused at its statement; T3 reads (10, 20) at every step; final (12, 18)	As expected. Turso rolls the refused transaction back itself; GoGraph poisons it until ROLLBACK (F8), so the spec scripts the ROLLBACK.
MG01	mg01-merge-no-constraint	Cypher	PG merge-insert-update	Both succeed; duplicates when both snapshots precede both commits; one node when serial	As expected: 2 nodes in 18 of 20 interleavings, 1 in the 2 serial ones, where a BEGIN follows the other's COMMIT.
MG02	mg02-unique-merge-winner-commits, -winner-rolls-back, -autocommit-loser	Cypher	PG insert-conflict-do-nothing, insert-conflict-do-update; MY iodku, innodb_replace, constraint_check_locks_in_read_committed	Loser gets a ConstraintViolation, not retriable (F10)	As expected in all three arms. PG waits and then does nothing or updates; InnoDB waits on a shared lock. After the winner rolls back, a new MERGE creates the node: no reservation leaks. The autocommit arm is refused too while the winner is open (G6); it matches the node once the winner has committed.
MG03	mg03-unique-committed-after-snapshot	Cypher	PG insert-conflict-do-nothing-2, read-write-unique-2	MERGE cannot see the committed value and gets a ConstraintViolation	As expected. PG REPEATABLE READ raises a serialization failure instead.
MG04	mg04-read-then-insert-unique	Cypher	PG read-write-unique, -2, -3	Second CREATE is a ConstraintViolation at the statement	As expected; exactly one node in all 20 interleavings. PG SERIALIZABLE waits, then raises a serialization failure.
MG05	mg05-gapless-sequence-no-constraint, -unique	Cypher	PG read-write-unique-4	Without a constraint: duplicate numbers (write skew); with UNIQUE: loser refused	As expected: [1, 2, 2] in all 20 interleavings without the constraint, [1, 2] with it.
MG06	mg06-merge-on-match-vs-update	Cypher	PG insert-conflict-do-update, -3, merge-match-recheck, merge-update	ON MATCH SET over a version the snapshot cannot see is refused (F2)	As expected: the later writer is refused at its statement in every order. PG -3 updates a tuple its snapshot cannot see.
MG07	mg07-on-match-changes-unique-key	Cypher	PG insert-conflict-do-update-2	One node per key; loser gets a typed error	As expected in all 30 interleavings; the release of the old key is deferred to commit, so a CREATE of it is refused while the renamer is open (rmp #2366).
MG08	mg08-merge-vs-delete	Cypher	PG merge-delete	MERGE onto a node deleted after its snapshot is refused; no resurrection	As expected. The refusal of the DETACH DELETE that comes second surfaces at its COMMIT. PG READ COMMITTED turns the MERGE into an INSERT.
MG09	mg09-delete-recreate-same-tx, mg09-delete-commit-then-recreate, mg09-delete-open-then-rollback	Cypher	MY innodb-lock-inherit-read_commited, index-create-dml-rollback, lock-inherit-existing; PG read-write-unique-3	(a) one holder; (b) re-create succeeds after the delete commits; (c) refused while the delete is open, one holder after its rollback	As expected in all three arms. In (b) the re-create succeeds once the delete has committed even when the creator's snapshot still sees the old node: the check reads the reservation set, not the snapshot.
MG10	mg10-intra-statement-duplicate	Cypher	MY innodb-index, create_table_select	Statement rejected, nothing applied	As expected. Between the failed statement and the client's ROLLBACK (F8) the value stays reserved and a peer is refused (G6); after the ROLLBACK the peer succeeds.
MG11	— (phase 6, arm MG11)	Cypher, random load	PG insert-conflict-specconflict, PG-inj on_conflict_probe_window	One node; loser ConstraintViolation	Implemented under random load (phase 6). No hook exists inside MERGE between its probe and its create (G5), so the race is reached by explicit-transaction MERGE storms on four keys at every ladder level: one node per key, every refusal a ConstraintViolation or a serialization conflict.
MG12	mg12-not-null-at-commit, mg12-not-null-peer-removal	Cypher	PG alter-table-1; MY constraint_check_locks_in_read_committed	Set-later commits; removed arm refused at COMMIT; peer removal refused	As expected. A peer's REMOVE n.p is refused by the write-write conflict at COMMIT when the setter committed first, and by the constraint at COMMIT otherwise; p is never null after a commit.
IX01	ix01-own-write-hash-seek	Cypher	GoGraph #2814 (efd32fb9); MY innodb_fts/transaction	Own writes visible to the hash seek; seek = scan	As expected. InnoDB's full-text index hides own uncommitted rows by design. Negative control below.
IX02	ix02-own-write-btree-seek	Cypher	GoGraph #2814	Range and prefix seek = scan after own writes	As expected. Negative control below.
IX03	ix03-own-write-autocommit	Cypher	GoGraph #2814	One autocommit statement counts its own CREATE and SET	As expected. Negative control below.
IX04	ix04-peer-rollback-label-add-hash, -btree	Cypher	GoGraph #2931; MY index-create-dml-rollback, innodb-index-online; PG partial-index	Node indexed under its committed value; seek = scan	As expected. Negative control below. Without a UNIQUE constraint the label add does not conflict with the open property write on the same node, in either order.
IX05	ix05-rollback-leaves-no-trace	Cypher	MY innodb-index-online, index-create-dml-rollback, lob_rollback_update, innodb_mysql_rbk; PG partial-index	Every seek = scan = pre-state plus the peer's commits	As expected. Since #3008 the per-node constraint stamp is taken only for a write of a label or property key that a constraint names. This row's UNIQUE constraint names (:K).k alone, so adding L to m and writing m.s no longer conflict, and the peer's SET m.s commits beside s1's open label add (see the golden). The negative control at 43c69dbe predates #3008, so whether this row now catches #2931 is not measured.
IX06	ix06-moved-out-of-index-domain	Cypher	PG partial-index, partition-key-update-4; MY multi_value_index_merge_mvcc	Pinned reader finds the old values; a new reader the new ones	As expected.
IX07	ix07-two-index-predicate-pinned	Cypher	MY multi_value_index_merge_mvcc, bug32554667	Pinned reader gains no row	As expected.
IX08	ix08-index-created-after-snapshot	Cypher	PG drop-index-concurrently-1, reindex-concurrently; MY innodb-read-view	Only snapshot rows	As expected; CREATE INDEX does not wait for the open read-only transaction.
IX09	ix09-label-index-scan-agree	Cypher	GoGraph #2931 churn; MY lock_impl_to_expl_case_sensitivity	Label count = seek = scan at quiescence	As expected.
IX10	ix10-parameter-seek-own-write	Cypher	GoGraph 3fd78c5e	Parameter seek = literal seek = scan	As expected, for a string and an integer parameter.
IX11	ix11-large-indexed-commit-hash, -btree	Cypher	TU tests.rs:4940, :5550 (MIT)	One commit re-values s on all 1 100 :L nodes (each to its successor's value); a pinned reader and autocommit readers see the old state before the publish and the new one after, seek = scan, never a mix	As expected. The btree arm compares node NAMES (n5 before, n4 after): the rewrite leaves every count unchanged. The instant inside the index delivery is covered by the IX11 ladder arm (phase 6).

| DD01 | dd01-create-index-under-open-writer-commit, -rollback | Cypher | PG multiple-cic, amcheck t/002_cic.pl; MY innodb-index-online, innodb-table-online, innodb-index-online-delete, bulk_create_index_online, index-create-dml-rollback | Index holds the committed arm, not the rolled-back one; seek = scan | As expected in all 20 interleavings of each arm. G9 settled: the backfill reads a committed snapshot (negative control below). | | DD02 | dd02-two-indexes-writer-between | Cypher | PG multiple-cic | Both indexes seek = scan | As expected. | | DD03 | dd03-drop-index-between-seeks | Cypher | PG drop-index-concurrently-1, vacuum-concurrent-drop | A read after the drop scans, with the same rows; no error | As expected; DROP INDEX does not wait for the open transaction. | | DD04 | dd04-drop-recreate-index-with-writers | Cypher | PG reindex-concurrently | Rebuilt index holds every commit | As expected. | | DD05 | dd05-unique-under-open-duplicate-commit, -rollback | Cypher | PG alter-table-1; MY alter_table_rebuild_duplicate_record, index-create-dml-rollback, innodb-alter-debug | No committed state violates the constraint | As expected. The uncommitted duplicate never refuses the DDL. The DDL is refused only when the duplicate committed first; otherwise s1 is refused, at its statement (DDL first) or at its COMMIT (the DDL straddles it, rmp #2936). | | DD06 | dd06-dml-not-blocked-by-idle-tx (DML half), dd06-ddl-bounded-by-context (DDL half) | Cypher | PG timeouts, truncate-conflict; MY innodb-timeout, innodb_lock_wait_timeout_1, innodb-lock | DML never waits on an idle open transaction; a DDL behind an in-flight statement returns its context error within the deadline | As expected in both halves. The DDL half parks an autocommit write in cat.hold and runs each DDL kind (CREATE and DROP INDEX, hash and btree; CREATE and DROP CONSTRAINT, UNIQUE and NOT NULL) with a 50 ms deadline: each returns context deadline exceeded while the write is still parked, the schema is unchanged before and after the release, and every retry succeeds. It witnesses the fix of D3 (rmp #2982). | | DD07 | dd07-ddl-in-write-tx, dd07-ddl-in-read-tx | Cypher | MY t/implicit_commit, t/trans_read_only | DDL rejected (F9); the transaction stays usable | As expected. The rejection does not poison the write transaction: its COMMIT succeeds with its other writes. MySQL commits the transaction implicitly instead. | | DD08 | dd08-ddl-on-one-object | Cypher | PG ddl-dependency-locking | Consistent catalog; the losing DDL gets a typed error | As expected in all 24 orders. The harness runs one step at a time, so the overlap of the DDL statements is #2934's. | | DD09 | dd09-create-index-vs-vacuum | Cypher | MY alter_table_rebuild_missing_record, innodb-index-online-purge, virtual_debug_purge | Seek = scan; no lost node | As expected with the drain between steps. A drain inside the backfill needs a seam that does not exist; that point is #2934's. | | AB01 | ab01-failed-statement-commit-then-retry, -rollback-then-retry | Cypher | PG delete-abort-savept, -2, aborted-keyrevoke; MY t/func_rollback, innodb_mysql_rbk | COMMIT is ErrTxPoisoned; nothing applied; the retry commits | As expected. | | AB02 | ab02-autocommit-statement-fails-midway | Cypher | MY t/func_rollback, t/kill | None applied, indexes clean | As expected for the failing statement and for the statement cancelled while it executes. | | AB03 | ab03-refused-statement-then-continue, -lpg | both | MY t/innodb_deadlock, innodb_mysql_rbk | Reads keep the snapshot; COMMIT applies nothing | As expected. G10 pinned: through Cypher the COMMIT returns ErrTxPoisoned, through lpg the serialization conflict. | | AB04 | ab04-retry-loop-converges | Cypher | docs/isolation-design.md:141-150 | Final = acknowledged increments | As expected. A client whose peer still holds the node is refused on every retry and gives up after three attempts; the final value still equals the acknowledged count. | | AB05 | ab05-rollback-of-created-subgraph | Cypher | MY lob_big_rollback, zlob_big_rollback, undo_log_temp_table | Invisible; counts and UNIQUE set restored | As expected. While s1 is open its UNIQUE value refuses the peer (G6). | | AB06 | ab06-create-delete-recreate-commit, -rollback; -commit-lpg | both | GoGraph graph/lpg/mvcc_life_test.go:149-265 | Commit: visible once; rollback: never visible | As expected. The lpg driver runs the commit arm only (no undo of eager writes). | | HZ01 | hz01-long-reader-retains-versions, -lpg | both | PG horizons, vacuum-no-cleanup-lock; MY bug120529, purge_on_replica, flush-hang, lob_purge | Versions rise while held, fall after | As expected: 0, then 10 while the reader is held, then 0 after it ends (each reading after a drain). One named interleaving. | | HZ02 | hz02-autocommit-read-releases-slot, -lpg | both | MY innodb-ac-non-locking-select, innodb_i_s_innodb_trx; PG horizons | 0 after an autocommit read; 1 while a read transaction is open | As expected. | | HZ03 | hz03-aborted-versions-withdrawn, -lpg | both | GoGraph docs/design-mvcc-abort-withdrawal.md; MY lob_rollback_update, zlob_rollback_update | Reader sees the pre-state; writer succeeds after the drain | As expected; the writer is refused while the aborting transaction is still open. | | HZ04 | hz04-vacuum-during-drop-index | Cypher | PG vacuum-concurrent-drop; MY innodb_bug26818787 | No error; backlog drains to 0 | As expected: backlog 10 while the reader is pinned, 0 after it ends. Three named interleavings. | | RO01 | ro01-read-only-tx-stable-and-rejects-writes; ro01-read-only-snapshot-stable-lpg | both | MY t/trans_read_only; PG read-only-anomaly-3 | Stable reads; ErrWriteInReadOnlyTx | As expected. The lpg driver runs the stable-snapshot half: an lpg read snapshot has no write surface. | | RO02 | ro02-read-view-before-two-writers, -lpg | both | MY innodb-read-view | The reader sees neither commit; a new reader sees both | As expected in all 90 interleavings. | | RO03 | ro03-read-only-tx-parallel-count | Cypher | PG serializable-parallel | Same answer as the serial path | As expected. The engine's ParallelScanThreshold is lowered to 2 048 over 3 000 nodes; TestRO03TakesTheParallelPath proves the count is planned as ParallelCountScan. | | SE01 | se01-session-reads-own-commit | Cypher | docs/isolation-design.md:236-248; PG-rec 057_snapshot_commit_race | The session sees its commit | As expected. In a serial script the frontier is always contiguous, so the sessionless reader sees it too; the case where they differ needs SE02's straggler. | | SE02 | se02-straggler-holds-frontier, -lpg | both | PG-rec 057_snapshot_commit_race, PG-inj repack_commit_race; MY binlog_gtid/binlog_group_commit_gtid_order, binlog/binlog_after_commit_order_info_schema, hp_deadlock | Sessionless read excludes T1; session read waits for T0 | As expected, on the commit hold: T1's commit acknowledges while T0 is held; a sessionless read sees only s at its old value; InFlightCommits is 2; T1's session read (cypher.Session.RunInTx, lpg.Session.Await) is parked (SessionsWaiting = 1) and returns only when T0 publishes, after which T0's write and t1 are visible together. The control interleaving publishes T0 first. | | SE03 | se03-abandoned-commit-does-not-stall-frontier (NOT NULL arm), se03-abandoned-fsync-does-not-stall-frontier (fsync arm) | Cypher | docs/isolation-design.md:389-395; MY binlog/binlog_group_commit_flush_crash | T1 visible to a sessionless reader; InFlightCommits returns to 0 | As expected in both arms. The fsync arm is the WAL commit path's failure on the commit hold: T0 reserves its instant and then abandons it (WriteTx.Abandon, then EndVersionedTx). In all 35 interleavings InFlightCommits is 0 once T0 has abandoned and T0's write is never visible; T1's commit is hidden from a sessionless reader only while T0's lower instant is still held (3 interleavings). | | SE04 | se04-self-conflict-through-frontier, -lpg | both | docs/isolation-design.md:141-150,199-213 | Sessionless self-write may be refused; session write is not | As expected, on the commit hold: with T0 held, T1's sessionless rewrite of its own node is refused with a serialization conflict (its snapshot is below T1's own commit); its session rewrite waits for T0 and then commits. With T0 published first both rewrites commit. | | RI01 | ri01-edge-create-vs-endpoint-delete, -lpg | both | PG fk-snapshot-2, fk-concurrent-pk-upd, fk-partitioned-1, PG-inj ri_fastpath_snapshot | Older-snapshot edge create refused after the endpoint delete commits | As expected in all 6 interleavings, through both drivers. When the edge create runs first, the Cypher DETACH DELETE returns its row and is refused at COMMIT; the lpg removal is refused at the statement. No dangling edge. | | RI02 | ri02-edge-into-deleted-node, ri02-edge-out-of-deleted-node, both -lpg | both | PG fk-snapshot, fk-partitioned-1; MY update-cascade | Later writer refused; no edge to a dead node | As expected for both endpoints, in every order. | | RI03 | ri03-hub-same-target, ri03-hub-same-source, both -lpg | both | PG fk-contention; MY innodb_cats | G8: the code says the second appender is refused; a comment said appends never conflict | Second appender refused, same target and same source, in every order, through both drivers: the code is right and the comment was stale (G8 below). PG's foreign-key contention waits instead. | | RI04 | ri04-application-check-vs-edge-insert, -lpg | both | PG referential-integrity, ri-trigger, temporal-range-integrity | One refused; no orphan | As expected. The application check (WHERE NOT ()-->(b)) passes at the deleter's snapshot, and the native edge RI still refuses one of the two. PG without a foreign key permits the orphan. The lpg arm reads InNeighbours in the transaction's view. | | RI05 | ri05-edge-to-parent-then-update-parent, -lpg | both | PG fk-deadlock, fk-deadlock2 | Second refused at once, no wait | As expected in all 20 interleavings: the second writer is refused at its edge create, before its SET; the parent holds the committed writer's value. | | RI06 | ri06-same-edge-property, ri06-parallel-edges-disjoint, both -lpg | both | GoGraph graph/lpg/mvcc_conflict_stores_test.go | Same edge: second refused. Parallel edges: no conflict | Same edge: as expected; Cypher refuses at the statement, lpg (SetEdgePropertyByHandle, a void primitive) at COMMIT. Parallel edges: the drivers differ. lpg commits both. Cypher refuses the second: its SET r.w also writes the per-pair property column (lpg.Graph.SetEdgeProperty, cypher/exec), a copy-on-write of the source node's adjacency entry, which claims that node (F4). The refusal applies nothing, so it is a false conflict, not an isolation defect. | | GG01 | gg01-traversal-at-pinned-snapshot | Cypher | GoGraph-specific | Both pinned counts equal the snapshot's | As expected: 7 paths at the snapshot in all 10 interleavings, 4 after the churn. | | GG02 | gg02-detach-delete-hub-vs-edge-create, -lpg | both | GoGraph-specific (#2725, #2694) | One refused; no dangling edge; counts consistent | As expected. The relationship count agrees with the surviving arcs. | | GG03 | gg03-degree-and-count-at-snapshot | Cypher | GoGraph-specific (#2081) | Degree and count equal the snapshot's | As expected in all 21 interleavings; the count-store answer equals a typed scan in every read. | | GG04 | gg04-label-churn-at-pinned-snapshot, -lpg | both | GoGraph-specific (#2687) | Pinned reader unchanged | As expected: 10 labelled nodes at the snapshot in every interleaving; label store = label scan. | | GG05 | gg05-delete-everything-vs-insert | Cypher | PG truncate-conflict (adapted) | The new node survives | As expected in every order. | | GG06 | gg06-delete-disjoint-parallel-edges, gg06-delete-self-loop-and-parallel-edge, gg06-delete-same-edge-instance, each also -lpg | both | GoGraph #2018 | Different instances commit; same instance: second refused | Same instance: as expected through both drivers; through Cypher exactly one delete commits and the other is refused at COMMIT, in all 6 orders (D4, fixed by rmp #2986). Different instances: the second deleter is refused (at COMMIT through Cypher, at the statement through lpg): every instance has source a, and an arc removal claims its source node exclusively (F4). The refusal applies nothing. | | FP01 | fp01-later-commits-ack-behind-held-straggler, -lpg | both | GoGraph #2932 (catalogue §5) | 2 048 later commits acknowledge while one commit is held; one-step catch-up | As expected: 8 goroutines × 256 commits all return while s0 is held (no <waiting ...>), none is visible to a sessionless reader, InFlightCommits is 2 049, and all 2 048 become visible together when s0 publishes. The control interleaving runs the fan before the hold. | | FP02 | fp02-straggler-abandons-lpg | lpg | GoGraph (catalogue §5); MY binlog/binlog_group_commit_flush_crash | Frontier catches up; the waiting session is released | As expected: the session wait is parked behind the held straggler (SessionsWaiting = 1); when the straggler abandons (WriteTx.Abandon, the fsync-failure path), the wait returns with no error, t1 is visible, s's write is not, and no commit is in flight. Through lpg only: the hold is an lpg transaction and Cypher has no abandon of an allocated commit. | | FP03 | fp03-session-read-bounded-by-context, fp03-session-wait-bounded-by-context-lpg | both | GoGraph (catalogue §5); CLAUDE.md context-aware blocking | A session read behind a held straggler returns the context error | As expected: with a 50 ms deadline, cypher.Session.RunInTx and lpg.Session.Await return context deadline exceeded; after the straggler publishes the same read returns t1. |

Negative control

world.perKeySnapshot (example code, set only by TestReadSkewNegativeControl) makes every lpg-driver read take a fresh snapshot — each key read at its own instant. With it, sk12-read-skew-lpg reports the anomaly; without it, none:

permutation "s1x s2dx s2cy s2c s1y s1x2 s1c" step s1y: read skew: x=50 y=60 sum to 110, want 100

Run against the golden with the seam on, the same spec fails on twelve property violations and on a transcript diff; with the seam off it passes.

Negative controls for the index rows

Each spec and golden at HEAD was copied into a temporary worktree of an earlier revision and run there. The worktree needed one change to compile: lpgSession.removeLabel and lpgSession.delProp wrap RemoveNodeLabel and DelNodeProperty, which returned no error then; no MG or IX spec uses either.

• #2814 at efd32fb9^. IX01, IX02 and IX03 all fail. IX01 reports 21 property violations, for example permutation "s1w s1new s1old s1cr s1crr s1c s2r" step s1old: index seek counted 1, scan counted 0; IX02 reports 32, for example step s1rg: index seek counted 1, scan counted 2; IX03 diverges from its golden at line 11 of permutation s1cr s1set s2r (want: 1, got: 0).
• #2931 at 43c69dbe. Both IX04 specs fail, 18 property violations and a diff at line 59 of permutation s1w s2l s1rb s1dr s2v s2p (hash: want: 1 |1, got: 0 |1; btree: want: 11 |11, got: 10 |11). IX05 passes there. A rolled-back transaction writes no index entry of its own, and at that revision the peer route #2931 took was closed in IX05 by the "node constraint" conflict that any registered UNIQUE constraint then brought. Since #3008 that conflict is taken only for a label or property key a constraint names, which IX05's L and s are not, so IX05's peer write now commits. This control has not been re-run since #3008, so whether IX05 now catches #2931 is not measured. The catalogue's §6 named IX05 as a row that catches #2931; it did not at 43c69dbe, and §6 names IX04 instead.

Negative control for G9 (DD01)

A test-only mutant, applied with go test -overlay and never written to the tree, makes Engine.beginIndexBuild (cypher/index_binding.go) hand the backfill the live graph, e.g.ReadAt(nil), instead of the snapshot it opens. Against it both DD01 specs and DD09 fail: the rollback arm with 36 property violations, the commit arm with 11, DD09 with 8, each also diverging from its golden. For example:

permutation "s1w s1cr s2ix s1rb s2x s2v7" step fx: index seek counted 1, scan counted 0
permutation "s1w s1cr s2ix s1rb s2x s2v7" step fv7: index seek counted 0, scan counted 1

The index kept n7 under s1's rolled-back 'x' and lost it under its committed 'v7'. Without the mutant all three pass.

Negative control for the dangling-edge check (RI, GG)

A test-only mutant, applied with go test -overlay and never written to the tree, makes Graph.appendEdgeInfo (graph/lpg/lpg.go) skip all three destination tests of an edge create: the destination is not claimed, its existence head is not tested when it is interned, and its existence is not cross-checked after the insert. Against it RI01, RI02 (into), RI04 and GG02 fail through both drivers, each with 3 property violations — the 3 interleavings in which the delete runs first — for example:

permutation "s1d s1c s2e s2c" step dangling: an edge to or from a dead node was committed: [[0 2]]

RI03 (same target) and RI05 also fail, on their goldens: the second appender to a shared target is no longer refused. Without the mutant every RI and GG row passes.

Negative controls for SK15-SK17 and IX11 (rmp #3016, #3017)

Seeded mutants applied with go test -overlay, never written to the tree:

• Dirty read — mvcc.Visible (graph/mvcc/mvcc.go) returns true for another transaction's uncommitted or aborted change. SK15 (both arms), SK16 and SK17 fail on their goldens: T2 reads 101|20 while T1 is open (G1a, G1b), each writer reads the other's uncommitted write (G1c), and T2's refused write is no longer refused (OTV).
• Aborted version visible only — mvcc.Visible returns true for AbortedTS. SK15 G1a does NOT fail: the Cypher ROLLBACK replays its undo log and restores a = 10 in the present state, so no read resolves through the aborted version. This mutant is detected by no catalogue row; SK15's G1a arm is held by the dirty-read mutant above.
• No index proof — index.Manager.DescribesSnapshot (graph/index/manager.go) always returns true, so a seek never declines to the scan. Both IX11 arms fail: seekEqualsScan reports, for the pinned reader after the commit, index seek counted 1, scan counted 0 (hash) and index seek counted [], scan counted ["n500"] (btree). An earlier btree arm that compared counts (prefixS('v110')) did NOT fail against this mutant and was replaced by the name comparison.

Defects found

D1, D2, D3 and the first parser defect are fixed; the second parser defect and D4 are open.

• D1 — ExplicitTx.Rollback publishes its commit record (rmp #2973). After a Cypher ROLLBACK, MVCCStats().Write counts one more commit and no abort, and a transaction whose snapshot predates the rollback is refused when it writes a node the rolled-back transaction touched — even after ReclaimNow. Snapshot isolation (and PostgreSQL, which lets the waiter proceed once the peer aborts) allows that write. The same shape through the lpg API (a doomed transaction ended with EndVersionedTx) is allowed. Reproduction: CREATE (:Item {name:'n', x:0}); T2 BeginTx; T1 BeginTx, SET n.x = 1, Rollback; ReclaimNow; T2 SET n.x = 2 → mvcc: serialization conflict in node properties. Expected: success. Fixed: every rollback path of an explicit transaction on which nothing is durable — Rollback, Commit's refusals, a panic during Exec — marks the transaction with lpg.WriteTx.Abandon, so EndVersionedTx aborts it. The -explicit arms of WW04 and WW12 and the WW05 rollback arm pin it.
• D2 — a property read inline in a CREATE map ignores the snapshot (rmp #2974). Inside an explicit transaction, after a peer committed c.date = 2 over the transaction's snapshot value 1, MATCH (c:Control) CREATE (r:Receipt {date: c.date, amount: 4}) RETURN r.date stores and returns 2, while MATCH (c:Control) RETURN c.date in the same transaction returns 1. CREATE (r:Receipt {date: c.date}) RETURN r.date also returns 2; binding the value first (WITH c.date AS d CREATE …) or CREATE … SET r.date = c.date returns 1. A read of data committed after the snapshot is a non-repeatable read inside one transaction. Fixed (rmp #2974): a bound entity read by a write clause's expression — CREATE and MERGE node and relationship maps, ON CREATE SET, ON MATCH SET — now resolves through the transaction's view. SK07 uses the inline spelling.
• Parser (rmp #2975). A CREATE map entry whose value contains a comma — CREATE (:Doc {blob: reduce(s = 'z', i IN range(1, 16) | s + s)}) — failed with parse properties … missing ':' in map item. Fixed: the map-item splitter treats a parenthesised group as one item. WW15 uses the inline spelling.
• Parser: STARTS WITH / ENDS WITH after a WITH clause (open). WITH 1 AS x MATCH (m:P) WHERE m.name STARTS WITH 'a' RETURN m.name fails with cypher: parse: unexpected "RETURN" at 1:53, expected one of {'WITH', 'UNION'} through Engine.Run; ENDS WITH fails the same way, while CONTAINS, the same predicate in parentheses, and the same MATCH … WHERE … STARTS WITH without a preceding WITH parse. openCypher admits the statement. The IX prefix rows spell their scan arm with left() instead; the seek arm, which precedes the WITH, keeps STARTS WITH.
• D3 — a DDL statement ignores its deadline while an autocommit write is in flight (rmp #2982). Reproduction: register a procedure that blocks, start CREATE (n:T {name:'t'}) WITH n CALL cat.hold() YIELD x RETURN x through Engine.RunInTx on one goroutine, and once it is parked run CREATE INDEX l_s2 FOR (n:L) ON (n.s) through Engine.RunInTx with a 50 ms deadline. Observed: the DDL is still blocked after 1 s and returns context deadline exceeded only once the write statement is released. Expected: the DDL returns its context error within the deadline plus a margin (catalogue DD06; F7; the "Context-aware blocking" rule of CLAUDE.md). Cause, from the source: an autocommit write holds Engine.schemaGate shared for its whole statement, and every DDL path took it with schemaGate.StrongLock() (cypher/api.go, runCreateBTreeIndex and its siblings; cypher/index_binding.go), which takes no context; the context was checked only after the gate is acquired. Fixed (rmp #2982): every DDL path takes the gate through Engine.lockSchemaForDDL, which waits with mvcc.Gate.StrongLockCtx and returns the context error holding nothing, before any schema, index, constraint or WAL state is touched. dd06-ddl-bounded-by-context pins it.
• D5 — autocommit MERGE under UNIQUE fails callers under concurrency (rmp #2987; found by phase 6, L13). F10 and cypher/merge_race_test.go state that concurrent autocommit MERGE under a UNIQUE constraint converges on one node with every caller succeeding. Reproduction (in-memory engine, cypher.NewEngine over lpg.New): CREATE CONSTRAINT k_u FOR (n:K) REQUIRE n.k IS UNIQUE; 8 goroutines each run RunInTx("MERGE (n:K {k:0})") once → 1 to 7 of the 8 fail with exec: Merge: ON CREATE SetNodeProperty: exec: constraint violation: UNIQUE constraint on (K).k: value 0 already exists (5 of 5 repetitions; one node remains). On a WAL-backed engine the 8×1 shape of the existing test passes, but the L13 storm (8 or 64 goroutines, 32 or 4 calls each, 4 keys) fails 16 of 256 and 74 of 256 callers the same way. One node per key always holds and stays gated. Fixed (rmp #2987): the MERGE operator marks a UNIQUE refusal of its own pattern key whose holder the statement's snapshot cannot see (cypher/exec/merge_unique_race.go), and an autocommit statement so marked is re-run, after waiting for every allocated commit to become visible, on a snapshot that sees the winner (cypher/merge_unique_retry.go); the re-run is bounded and context-aware. A statement of an explicit transaction is not re-run (MG11). L13 gates every_caller_succeeds.
• D6 — DETACH DELETE of a node with two parallel in-edges from one source leaves one arc behind (fixed, rmp #2988; found by phase 6, L15). Reproduction: CREATE (:Hub {id:1}), (:X {id:53}); twice MATCH (x:X {id:53}), (h:Hub {id:1}) CREATE (x)-[:R]->(h); MATCH (h:Hub {id:1}) DETACH DELETE h; ReclaimNow. Cypher counts 0 relationships, but at a fresh snapshot the live x still lists the dead hub in its out-neighbours and the dead hub lists x as an in-neighbour (the dangling probe counts 2). With one edge, or with one of the two deleted first, the probe counts 0. The in-edge index names each source once, and DETACH DELETE removed one slot per name. Fixed (rmp #2988): the transaction-visible in-neighbour read lists a source once per incoming arc (inArcSourcesInTx, cypher/api.go), so every parallel instance is removed. L15 creates its edges with CREATE, so repeated draws of one pair build parallel in-edges, and gates no_dangling_edge.
• D7 — the hash and label indexes diverge from the graph under random churn (fixed, rmp #2989; found by phase 6, L06). At quiescence, after the L06 workload (label add/remove, indexed writes, 30% rollbacks, a UNIQUE set), the equality seek on (:L).s misses nodes a scan finds — hash "s3" seek=40 scan=41, hash "own-920-0-1" seek=0 scan=1, and once the label count seek=694 scan=692. Measured: 3 of 3 runs at 1024 goroutines, 1 of 1 soak run at 1024 (0 at 256), 2 of 3 -race runs at 64, 1 of 2 coverage-build runs at 64. In-transaction seeks of own writes (L07) never diverged (0 of 1 457 at 1024). Reproduction: go test -tags soak -run TestLadderSoak ./examples/37_mvcc_write_contention/ (or -race -run TestLadder), or the binary with -ladder-levels 1024 -ladder-rows L06 -ladder-ops 4096. Four causes. A write that changes nothing — a SET to the stored value, a REMOVE of an absent label — conflicts with no concurrent writer, yet its commit applied its own index payload after a peer's commit had changed the node: the SET re-indexed the value the node had given up, the REMOVE deleted the peer's entry. The old value of an indexed write was read from the graph's present, which can still hold a rolled-back peer's value, so the committed value was never deleted. And a REMOVE of an absent label re-stamped the deferred label-bitmap removal of a committed one; two such transactions aborting dropped it, and the node stayed in the label scan. Last, the undo of a SET n:L was decided by a probe taken outside the label store's lock: when a peer restored the label in between, the add changed nothing, yet its rollback removed the label and its bitmap entry, and only the bag got the label back when the abort was withdrawn. The engine now enqueues an index change, and records a label undo, only for a write that wrote a version; reads the old value through the writing transaction's view; and an absent label defers no removal. L06 gates seek_equals_scan.
• D4 — a Cypher DELETE of a relationship another transaction removed is a silent no-op (fixed, rmp #2986). Reproduction: CREATE (a:N {name:'a'})-[:R {id:1}]->(b:N {name:'b'}), (a)-[:R {id:2}]->(b), (a)-[:R {id:3}]->(a); T1 and T2 BeginTx; T1 MATCH (:N {name:'a'})-[r:R {id:1}]->() DELETE r RETURN count(*) → 1; T2 the same statement → 1; T2 Commit → ok; T1 Rollback. Observed: relationship id 1 still exists, so T2's acknowledged, committed delete is lost. With T1 committing instead, both transactions commit a delete of the same relationship. Expected: T2's delete is refused with ErrSerializationConflict while T1's removal is pending or committed after T2's snapshot (first-updater-wins, F2), as the lpg arm (gg06-delete-same-edge-instance-lpg, Graph.RemoveEdgeByHandle) does. Cause, from the source: removeBoundRelationship (cypher/exec/rel_instance.go) tests whether the handle is stored in the PRESENT adjacency (relStoredOrder); a handle a peer already removed is treated as "deleted by an earlier row of this statement" (rmp #2940) and nothing is called, so no conflict is recorded — the shape of rmp #2943. Fixed (rmp #2986): the removal by handle is issued in that case too; the engine's first-updater-wins check on the endpoints runs before its presence probe, so a peer's removal records the conflict, while a removal this transaction already made finds the claim held and removes nothing. cypher/rel_delete_peer_conflict_test.go pins it on both engines, and gg06-delete-same-edge-instance is now in the catalogue.
• D8 — two committed CREATEs could name one node (rmp #3015, fixed; catalogue GG07, gated by phase 8). A CREATE or MERGE gives each new node a hidden key drawn from a process-wide counter that was seeded once per process, from the first graph a write operator ran against. A graph whose keys another process minted — a second store opened in the same process, or a store recovered from disk and loaded into memory — holds keys at and above the counter, and interning one returned the existing node, so the CREATE overwrote it. Reproduction: one process writes store B with 20 CREATEs and exits; a fresh process runs one CREATE in an empty store A, opens B and runs 3 CREATEs. B still holds 20 nodes, and three of the first ones carry the new nodes' properties. The same counter re-minted, after a reopen, the key of a deleted node whose key was the largest in the store, so its id() named a new node. Fixed: every key is minted by mintNodeKey (cypher/exec/create_node.go), which rejects a key the target graph already holds, live or deleted (GraphMutator.ResolveNodeID), and doubles its stride over a run of held keys. cypher/node_key_mint_3015_test.go pins it for CREATE, MERGE and a MERGE pattern, in memory and over a reopened store.

Gaps pinned

• G1 (commit-hold seam): confirmed, and H2 checked. The only route is a WAL-backed engine whose wal.WALFile.Sync blocks on a gate (wal.OpenFS takes a caller filesystem; txn.NewStoreWithOptions and cypher.NewEngineWithStore take the result). It can hold T0 between its timestamp allocation and its publication, but one WAL writer runs one fsync at a time: a committer whose frames follow the held leader's waits for the leader to finish (store/wal/writer.go, syncToLocked), so T1 can never acknowledge while T0 is held, and SE02's shape cannot be built from it. G1 is refuted: frontier scenarios are not blocked by a missing hook. lpg.Graph.AllocateCommitTS reserves a commit instant without publishing it, on an in-memory graph, and the transaction's later EndVersionedTx publishes it or, after WriteTx.Abandon, abandons it. FP01-FP03, SE02, SE04 and SE03's fsync arm are built on it, deterministically.
• G2 (vacuum control): resolved with a drain Hook at a fixed point. The withdrawal of an aborted version now runs at abort (withdrawAbortedNow), so F5's race is narrower than the catalogue states; the no-drain arm is still not pinned.
• G4 (value-preserving write as a skew remedy): pinned by SK11 — it is not a remedy.
• G5 (no hook inside MERGE): confirmed — cypher/exec/merge*.go has no hook between probe and create. MG11 is left to random load (#2934).
• G6 (UNIQUE against an in-flight reservation): pinned by MG02 (all three arms), MG03, MG07, MG09 and MG10 — always a ConstraintViolation, never a serialization conflict, and refused even when the holder later rolls back. An autocommit MERGE is refused too while an explicit transaction holds the value, so F10's "autocommit MERGE converges" holds only between autocommit callers.
• G7 (no online index verifier): unchanged. Each IX row derives seek = scan in the statement itself, and TestIXAccessPaths guards that the seek arm is an index access.
• G8 (adjacency append conflicts): settled by RI03 — the code is right. The second of two transactions appending an edge to one node is refused, for a shared target and for a shared source, through both drivers, in every interleaving. The addEdgeInfo godoc the catalogue quoted had already been corrected (commit 1ad20d03); the stale text left was the comment on the Graph.adjVer field in graph/lpg/lpg.go, which said an adjacency append "is commutative and must not conflict with another append". It now states the rmp #2445 rule.
• G9 (backfill source of CREATE INDEX / CONSTRAINT): settled — a committed snapshot. DD01 (index) and DD05 (UNIQUE constraint) pass in every interleaving, and DD01 fails against a mutant whose backfill reads the live graph (negative control above). The source agrees: Engine.beginIndexBuild waits out the commits deciding and scans e.g.ReadAt(snap); the constraint build scans a snapshot taken after HoldCommitDecisions. Not an engine defect.
• G10 (state after a refused statement): pinned by AB03 through both drivers — through Cypher, COMMIT returns ErrTxPoisoned; through lpg, the refused write dooms the transaction and its commit returns the serialization conflict. Both apply nothing, and a read after the refusal stays at the transaction's snapshot.

Phase 6 — concurrency ladder (rmp #2934)

ladder.go (driver, sampler, output) and ladder_rows.go (arms) run the §2 rows of docs/mvcc-scenario-catalogue.md — L01-L20 — and MG11 as randomised concurrent arms at a ladder of goroutine counts. The short layer (TestLadder) runs 1, 8 and 64; the soak layer (ladder_soak_test.go, -tags soak, TestLadderSoak) runs 256 and 1024 with 4 096 operations per arm and level, adds L09 and the full-size L17, and turns on the self-conflict streak gate. The binary runs the phase after phase 5 with -ladder-levels (default 1,8,64), -ladder-ops, -ladder-rows and -ladder-soak.

Every arm runs on the durable store (store.Open: WAL, durable commit, recovery) in its own directory under TMPDIR, removed when the arm ends (rmp #2993). Pointing TMPDIR at a RAM drive changes only the medium. Because a RAM drive shrinks an fsync to microseconds, and with it the window between a commit's timestamp and its visibility, TestLadder and TestLadderSoak open every arm's store with an injected fsync latency drawn uniformly from 1-5 ms (store.Options.SyncLatency, set through internal/synclatency; rmp #3022). GOGRAPH_FSYNC_LATENCY=off disables it, and GOGRAPH_FSYNC_LATENCY_SEED=<n> replays the seed a failing test logs. The binary runs without it. The short layer and the binary's default run 256 operations per arm and level, except L13 (32) and L15 (64); see "Sizes" below. -ladder-ops or -ladder-soak applies one total to every arm.

Arms

Arm	Rows	Workload	Gates (bare ladder.<arm> level=<n> <check>=true lines)
L01	L01, L02, L03	Bank transfers (BeginTx, two reads, two writes, a Log insert; 10% rolled back) and read transactions (BeginReadTx or read-only BeginTx) that read every account and the Log count twice; one doctors write-skew round per goroutine, two transactions interleaved by the goroutine itself. Every attempt is recorded with internal/anomaly (aborts as aborted) and the history checked at SnapshotIsolation	history_clean (0 forbidden, not truncated), write_skew_permitted (G2-item in Report.Permitted), repeatable_reads, no_phantoms, conservation
L04.<arm>	L04	Hot counter, SET c.n = c.n + 1 with retries, session and sessionless arms	Final value = acknowledged increments
L05	L05	One transaction over 16 hot nodes against single-node writers	Sum = acknowledged; the large transaction's success rate and refused streak are reported (G3: no fairness mechanism)
L06	L06, L07	Label add/remove, hash- and btree-indexed writes, UNIQUE writes, 30% rollbacks; 1 in 5 statements writes a fresh value and seeks it inside its transaction	No duplicate UNIQUE value; in-transaction seek = scan = 1. Seek = scan at quiescence for the hash index, btree range and prefix, label scan, count store and UNIQUE backing index (seek_equals_scan, gated since rmp #2989, D7). The btree index's CONTENT (Lookup cardinality per value) = a scan (btree_index_equals_scan, rmp #2995): the btree seek/scan pairs cannot fail, because the range seek is planned only with >= 64 labelled nodes and <= 10% selectivity and keeps the full predicate as a residual filter. Before the UNIQUE constraint is declared, one #2989 interleaving is CONSTRUCTED on two nodes the churn never draws (a no-op SET committed over a peer's commit, noop_set_over_peer_committed), so the btree gate does not depend on the churn producing it
L08	L08	A read transaction held while writers leave 3 x Bound versions	retention_shown (Total > Bound while held), repeatable read, reclaimed after release
L09	L09 (soak)	1 032 read transactions held while the graph churns	UnregisteredSnapshots > 0, every read correct, released after close
L10.<arm>, L20.<arm>	L10, L20	WAL-backed store (store.Open), disjoint writers, a checkpointer triggered back to back	Final = acknowledged; session arm 0 self-conflicts; checkpoints ran; commit tail during a checkpoint below hangBudget; storage size before and after
L11.<arm>	L11, L12	Disjoint writers, no checkpointer, session and sessionless	Session arm 0 self-conflicts (and so 0 conflicts)
L13, MG11, L14	L13, MG11, L14	MERGE storms on 4 keys: autocommit under UNIQUE, explicit under UNIQUE, autocommit without a constraint	One node per key under UNIQUE; every failure typed; L13 and L14 every caller succeeds (D5), L14 duplicates counted
L15, L16	L15, L16	Edge CREATE (parallel edges)/delete on hubs and DETACH DELETE + recreate of hubs; read transactions repeat a one-hop and a *1..3 traversal. After the churn, two interleavings are CONSTRUCTED once on nodes the churn never draws (rmp #3007): a DETACH DELETE whose snapshot predates a committed arc, run after a third transaction appended to the hub and rolled back (rmp #2997); and a read transaction that reads before and after a hub is created after its snapshot and a DETACH DELETE of it is rolled back (rmp #3001)	No dangling arc at quiescence; repeated traversals identical; the stale DETACH DELETE refused with a serialization conflict (stale_detach_refused); the read transaction never sees the young hub (young_node_stays_invisible)
L17	L17	8 transactions hold 16 000 (soak: 150 000) uncommitted nodes; counts; a 9 M-row statement cancelled after 2 ms	Count = committed only; cancelled statement returns context.Canceled within 1 s
L18	L18	Property writes, edge creates and DETACH DELETEs, 50% rolled back	Total <= Bound after quiescence; 0 dangling arcs; every hub writable by a lone writer
L19	L19, DD08, DD09	CREATE/DROP INDEX cycles and an overlapping CREATE INDEX on the same object while writers churn and the vacuum is swept in a loop	Seek = scan; every DDL refusal classified (already_exists); DDL latency below hangBudget; no writer starved
IX11	IX11 (Turso tests.rs:4940, :5550, MIT)	One writer re-values the hash-indexed s and the btree-indexed b of all 64 :L nodes in one autocommit statement, generation after generation; every goroutine opens read transactions at every instant, the index delivery included, reads the generation its snapshot holds and seeks the current and the next generation's value of one node through each index	seeks_planned_as_index, commits_overlapped_reads, never_mixed (one generation per snapshot), seek_equals_scan
L21	L21 (Turso shuttle_mvcc.rs:749, tests.rs:792, MIT)	One writer SET n.v = i in a loop; a goroutine calls ReclaimNow without pause; every goroutine opens read transactions that read n three times, 2 ms apart	reclaimed_during_reads, node_seen_exactly_once, value_committed_by_start (between the value acknowledged before BEGIN and the value attempted after it), repeatable, and the sampler's watermark_regressions_zero, horizon_stale_leaves_zero
L22, L22.crash	L22 (Turso tests.rs:5403, :18953, MIT)	Bank transfers (two accounts per goroutine, at least 8; each also creates a :Tr node) on the durable store while RunCheckpoint runs back to back; read transactions repeat sum and count three times, 1 ms apart; half-way, after a checkpoint, a crash image is copied between checkpoints under the commit lock and recovered	checkpoints_ran, snapshots_pinned_across_reads, snapshot_total_and_count_constant, crash_image_after_checkpoint; after recovery snapshot_used, total_and_count_unchanged, acked_present

Every sampled arm also runs a sampler on lpg.MVCCStats every 500 µs and gates WatermarkRegressions = 0 and HorizonStaleLeaves = 0 at every sample, WithinCeiling() at every sample with no reader open (not in L08), SessionsWaiting <= goroutines, visibility lag (time for the frontier to reach the newest commit allocated at the sample) below the 2 s retry budget, and, after quiescence, InFlightCommits = 0 and Total <= Bound. It reports peak InFlightCommits, peak SessionsWaiting, OutOfOrderPublications, HelpedPublications, max visibility lag, peak versions, active and unregistered snapshots, conflicts by store and the snapshot capacity (1 024). Each arm reports commits/s, refused attempts, rollbacks, unrecovered operations, constraint violations, the longest retry streak in attempts and in wall time, and commit latency percentiles. Each level emits a runtime/trace region per arm (ladder/<arm>/<level>) and a runtime.MemStats line.

Not observable: ring waits. The engine has no ring-wait counter since bfaf3ed6 (#2932): a commit-timestamp claim never waits for a ring slot; it chains behind the previous lap's record, counted only by the unexported commitRegistry.chained.

InFlightCommits is a window, not a count. Clock.InFlightCommits returns the newest allocated timestamp minus the frontier, so finished commits above an unfinished one stay in it. A gate of "peak <= goroutines" failed on the first soak run (729 at 256 goroutines, WAL-backed L13) and was removed; the window is gated by the lag measurement and by InFlightCommits = 0 after quiescence. The godoc of Clock.InFlightCommits and of MVCCStats.InFlightCommits opens with "how many allocated commit timestamps have not yet finished", which describes a count; the code returns the window. This is noted, not changed here.

The streak gate and where its threshold comes from

A disjoint writer conflicts only with its own previous commit, which it can meet only while the frontier is held below that commit. The soak gate is that the longest such run of refusals, in wall time, stays below the writers' retry budget (retryBudget, 2 s, rmp #2330) and no operation exhausts it. The attempt count is reported but not gated: it measures how fast a refused WAL attempt returns (94 attempts in 1.2 ms at 64 writers). Measured at HEAD: 19.5 ms at 256 and 82.6 ms at 1 024 goroutines (WAL-backed, sessionless), 0 in-memory and in every session arm.

The gate does not detect the absence of #2932. Built against 43c69dbe, which predates it, the longest sessionless streak measured 13–28 ms (0 with GOMAXPROCS=2), far inside the 2 s budget. The gate stays as a guard against retry stalls; #2932's own tests (graph/mvcc/publish_convoy_test.go) cover its regression.

Negative controls

• #2931 (index churn), shown failing. The ladder files were compiled against a copy of the tree at 43c69dbe extracted with git archive (no git write on this tree), with ladder_frontier.go replaced by a stub returning zeros and store.Open replaced by its parts (wal.Open, txn.NewStoreWithOptions, cypher.NewEngineWithStore), which that revision predates. L06 failed at 8 (hash "s4" seek=5 scan=4) and at 64 (hash "s7" seek=4 scan=5) goroutines.
• #2932 (streak gate), NOT shown failing — accepted. The same copy at 43c69dbe passed the soak streak gate: longest sessionless self-conflict streak 13.2 ms (L10) and 27.8 ms (L11) at 256 goroutines, 12.8 ms at 1 024; with GOMAXPROCS=2, 0 — against the 2 s budget, and against 19.5 / 82.6 ms at HEAD. The gate cannot detect #2932's absence with this workload; the deterministic catch remains graph/mvcc/publish_convoy_test.go.
• Long-reader retention, shown. retention_shown holds at every level: with the reader open, Total reached 12 288 against Bound 4 096 (64 goroutines), and fell to 0 after release.

Sizes (rmp #2993)

Each arm's short-layer size is the smallest measured to still fail with its target defect restored. Detection was measured on the durable store under -race, by building the example with go build -overlay over a copy of the engine file with the fix reversed (git diff <fix>^ <fix> applied in reverse to the current file), seeds 1-5:

Gate	Target defect (fix reverted)	Total ops, old → new	Detection at the new size	Wall clock, 1+8+64, old → new
L13 every_caller_succeeds	rmp #2987 (fe8864b0)	256 → 32	6 of 6 runs at 8 and at 64 goroutines (10 of 10 at 256)	0.47 s → 0.13 s
L15 no_dangling_edge	rmp #2988 (9d0d315e)	256 → 64	9 of 9 runs at every level (8 of 9 at 32, so 32 was rejected)	0.71 s → 0.50 s
L06 btree_index_equals_scan	rmp #2989 (8999e4b4), the property-write and the whole fix	256 (kept)	15 of 15 at 1, 8 and 64 for both reverts, at 32, 64, 128 and 256: the interleaving is constructed	—
L06 seek_equals_scan	rmp #2989, the whole fix	256 (kept)	at 64 goroutines only: 2 of 5 at 128, 1 of 5 at 64, 0 of 5 at 32, so not reduced	—
L15 stale_detach_refused, no_dangling_edge	rmp #2997 (32efbe73)	64	9 of 9 at 1, 8 and 64 goroutines: the interleaving is constructed	—
L16 young_node_stays_invisible	rmp #2999/#3001 (676a6b94)	64	9 of 9 at 1, 8 and 64 goroutines: the interleaving is constructed	—
TestRun phase 5 (catalogue)	rmp #2986 (8f933554)	the whole catalogue → not run	with the fix reverted, TestCatalogue fails (GG06/gg06-delete-same-edge-instance) while TestRun with phase 5 passed: phase 5 added no detection	28.7 s → 0 s
TestDurability checkpoint arm (D09, D16)	rmp #2990 and #2991 (828f1661)	384 → 24 transactions	24 of 24 runs fail (seeds 1-3 at 192, 96, 48 and 24, writers 8 and 64): D09.missing_segment refused_loudly, D09 capture_not_refused (since WAL v2 step 1 the refusal no longer exists and D09 checkpoint_ran gates the first attempt); at 64 writers the 4-per-writer floor keeps 256	4.4 s + 8.7 s → 0.25 s + 2.3 s

Turso arms (rmp #3017, #3018)

The Turso arms have no historical fix to revert, so each is sized against a seeded mutant applied with go test -overlay (never written to the tree), under -race, with the injected fsync latency, levels 1, 8 and 64, three runs per size:

Gate	Seeded mutant	Total ops	Detection	Green
IX11 seek_equals_scan	index.Manager.DescribesSnapshot always true (the seek never declines to the scan)	16	3 of 3 runs at every level; at 4, level 1 runs too few commits to overlap a read (commits_overlapped_reads fails). Since rmp #3061 the btree arm's read of the next generation's key, absent from the reader's snapshot, is planned as a seek, not a label scan, so both btree reads compare a seek with a scan; re-measured: 3 of 3 runs at every level	3 of 3
L21 value_committed_by_start, repeatable	Horizon.Oldest returns its fallback without scanning the readers (reclaim past the watermark)	4 (the 4-per-goroutine floor)	3 of 3 runs at every level (repeatable; value_committed_by_start at 8 and 64 in every run)	3 of 3
L22.crash acked_present, total_and_count_unchanged	Checkpointer.awaitCommitQuiescence returns at once (checkpoint capture skew: the durable-but-unpublished window is not waited out)	32	5 of 5 runs fail at one level or more, never at the same levels every run (for example 1 of 174 acknowledged transfers missing, recovered sum=12802); at 16, 2 of 3	3 of 3

IX11 runs 64 nodes because of the planner, not the delivery: at 32 and 48 nodes the parameterised btree prefix seek is planned as a label scan, and seeks_planned_as_index fails.

At the soak layer's 256 and 1 024 goroutines (4 096 operations) the three arms passed 3 of 3 runs. The first soak run of L22 caught the arm's OWN defect: a transfer that exhausted its retry budget was logged as acknowledged, and acked_present reported 21 of 2 178 (256) and 576 of 2 706 (1 024) "acknowledged" transfers missing, against 33 and 865 unrecovered. Only a transfer whose commit returned nil is logged now, and unrecovered transfers are telemetry, as in every other arm.

Not reduced, because no defect was available to prove that a smaller size still detects: L01, L04, L05, L08, L10, L11, L17, L18, L19, and the abandon and fsync arms of phase 7 (the D14 seam proves only acked_present). The churn of L15 and L16 — no_dangling_edge by chance and traversal_repeatable — detects neither rmp #2997 nor rmp #2999 reverted: 0 of 15 runs at 1, 8 and 64 goroutines and 0 of 4 at 256 and 1 024 (4 096 operations). Both are detected only by the constructed interleavings above.

Phases 1 to 3 and 5 run on the in-memory engine: they drive lpg.Graph.ApplyVersioned and lpg.Session directly (phase 5 also cypher.NewEngineWithOptions), which the durable store does not route through its WAL, so they cannot be moved to store.Open without changing what they measure. Phase 4 restarts a WAL-backed store under TMPDIR.

Coverage

docs/benchmarks/ex37-mvcc-coverage-2026-10-06.md gives the command and the per-package figures for graph/mvcc, graph/lpg and cypher.

Phase 7 — durability under concurrent writers across a crash (rmp #2935)

Examples 17 and 25 crash a store that has one writer. durability.go and durability_kill.go crash a WAL-backed store (store.Open) while 8 to 64 writers commit, and hold the recovered directory to the durability contract. They implement the 16 rows of docs/mvcc-scenario-catalogue.md §3 (D01-D16). The short layer (durability_test.go, TestDurability) runs the in-process arms at 8 and 64 writers. TestDurabilityNegativeControl runs the D14 control. The soak layer (durability_soak_test.go, -tags soak, TestDurabilityKill9) runs five kill -9 runs of a child process with 32 writers. The binary runs the phase after phase 6 with -durability-levels (default 8,64), -durability-txns, -durability-kill-runs (default 0) and -durability-kill-level.

Output follows the rest of the example: durability.<row> level=<n> <check>=true lines are verdicts, and # durability.<row> level=<n> ... lines are telemetry.

Harness

• Workload. Each regular transaction is an explicit transaction with three statements: CREATE (:D {id, g, tag}), MATCH (d:D {id}) CREATE (d)-[:R {id}]->(:E {id}) and MATCH (h:H {k}) SET h.v = h.v + 1 on one of 4 shared counters. The counters make writers collide, so attempts are refused and retried, and each attempt takes a fresh id. One transaction in eight is rolled back on purpose. The schema has a UNIQUE constraint on :D(id), a hash index on :E(id) and a btree index on :L(id).
• Acknowledged-commit log. The log is kept outside the store. An id is recorded as acknowledged only after its Commit returned nil. Refusals, rollbacks and other failures are recorded with their outcome.
• Bracket. Each observation is bracketed between two counts of acknowledged commits, one read before the image and one read after it. Only the ids acknowledged before the image are owed by the image. An id acknowledged later is in doubt.
• Crash image. In the in-process arms, the crash is a copy of the store directory taken while the writers run. There are two variants. The durable image cuts the WAL at wal.Writer.DurableOffset, read after the "before" count; this models a power loss, in which only fsynced bytes survive. The written image copies the WAL as written to the OS; this models a process crash, in which the page cache survives. The live process is then abandoned: its writers stop, the open transaction rolls back, and the store closes. Nothing it does after the copy can reach the image. The checkpoint images are copied inside the store's commit lock, so each one is an exact instant.
• Scoring. Every acknowledged commit is present, whole. Every refused, rolled-back or failed attempt and every transaction open at the crash is absent. An in-doubt transaction is present or absent as a whole, never partially. The sum of the shared counters equals the number of whole regular transactions. A new session after recovery sees every acknowledged commit, and its state fingerprint equals the one recovery produced. Seek equals scan on every index and on the count store, and UNIQUE holds. The clock after recovery is not below the clock read with the "before" count, and a post-recovery commit takes a higher instant.
• Telemetry per image. The image line reports acknowledged (before and after), recovered, missing, phantom, partial, in doubt (present and absent), the seek/scan comparisons and mismatches, the clock before the crash, after recovery and after a new write, the store size before the writers, at the crash and after recovery, recovery wall time, WAL ops replayed, the tail error and whether a snapshot was used.

Rows

Row	Arm (durability.<arm>)	What runs	Gates
D01	D01, D01.durable, D01.written	In-process crash image at half of 384 transactions, 8 and 64 writers	acked_present, refused_absent, writers_running_at_crash, no_unexpected_errors, acknowledged_seen
D02	D02.run0..run4 (soak)	kill -9 of a child with 32 writers, an open transaction and a checkpointer triggered back to back; killed after 100 + 60 x run acknowledgements. Every A line the child wrote after its commit returned nil is owed	Same gates as D01 on the killed directory
D03	D03, D01.durable, D01.written	Crash points: after append and before fsync (bytes beyond the durable offset, durable image), after fsync and before acknowledgement (durable but unacknowledged), and bytes written but not yet fsynced (written image)	In-doubt transactions whole or absent (whole_or_absent); counts reported as durable_but_unacknowledged and appended_not_durable_or_in_flight
D04	D04, D04.reopen	Tail WAL segment over internal/testfs with fsyncs failing after 12 successes, which discards the unsynced suffix	failure_seen, post_poison_commit_refused, failed_not_visible (live engine), recovers_exactly_the_acknowledged, and every image gate on the reopened directory
D05	every image	Three-statement transactions in flight at the crash	whole_or_absent, counters_conserved
D06	D01.*, D02.*	One transaction left open across the crash (the kill child reopens it every 5 ms)	open_absent
D07	every image	Clock read with the "before" count; new session; new commit	clock_not_rewound, new_session_sees_acked, post_recovery_commit_is_new
D08	D08.reference, D08.torn, D08.garbled	The durable image with its last frame cut by 3 bytes, or with its last byte inverted	damaged_record_discarded_alone: against the undamaged image, at most one transaction is lost, none is gained and none is partial. The garbled image must be opened for writing exactly when recovery reports it clean (refused_unless_clean)
D09	D09, D09.pre_capture, D09.post_checkpoint, D09.missing_segment	One checkpoint under load, with no retry, imaged under the commit lock before the capture and after the checkpoint returns (its segment unlink takes no commit lock, so there is no image between the control-file write and the unlink); the post-checkpoint image is then recovered without its snapshot directory	checkpoint_ran (the single attempt succeeds), every image gate at each phase, and missing_segment.refused_loudly (recovery returns recovery.ErrMissingSnapshot and is not clean). The kill runs add crashes at random checkpoint points
D10	every fully opened image	Seek = scan for every attempted id on :D(id) and :E(id), on :L(id), and on the :D/:E count store	seek_equals_scan, unique_holds
D11	D11, D01.*	CREATE/DROP INDEX cyc_g and CREATE/DROP CONSTRAINT cyc_e cycled during the abandon arm	ddl_ran; after recovery, seek = scan on cyc_g when it is present (seek_equals_scan); no duplicate :E(id) while cyc_e is present (unique_holds)
D12	D01.*	Four transactions, each with one 1 MiB string	blobs_identical
D13	D13	The torn image: recovery interrupted inside the WAL replay (a context that reports cancellation from its second check, so at the first replayed frame; structural, rmp #3000), then opened for writing (which repairs the torn tail) and closed, then recovered again	recovery_interrupted, double_recovery_identical
D14	D14	Negative control: the seam cuts the durable image at the last WAL frame carrying an acknowledged transaction's tag. Off by default (durabilityConfig.dropLastAcked)	TestDurabilityNegativeControl: acked_present must fail
D15	D01.*, D02.*	Transactions ordered by the WAL offset of their tag	wal_tags_seen, no_hole: no acknowledged transaction is absent below a recovered one
D16	D16	The pre-truncate image recovered with its snapshot, and without it and its WAL prefix marker (a full WAL replay)	snapshot_used, checkpoint_plus_tail_equals_full_replay (identical state fingerprints)
D17	D17 (Turso tests.rs:2751, logical_log.rs:4741, MIT)	Epoch 1: 2 commits per writer; crash image with the last WAL frame torn by 3 bytes; the image opened for writing; epoch 2: 2 commits per writer; second crash image; recovery	torn_record_discarded_alone (the tear loses at most one acknowledged commit), epoch1_survives_second_recovery, epoch2_present (the second recovery does not stop at the first epoch's torn bytes)
D18	D18 (Turso tests.rs:3302, MIT)	2 commits per writer; T0 reserves a commit instant with lpg.Graph.AllocateCommitTS on the store's graph and does not publish it (H2); T1 commits; RunCheckpoint starts and T0 is held 50 ms; T0 released; crash image after the checkpoint; recovery	t1_acked_while_t0_held, checkpoint_waited (not finished and nothing truncated while T0 was held), checkpoint_ran_after_release, acked_present_after_crash (snapshot used)
D19	D19 (Turso group_commit_tests.rs:378, tests.rs:14581, shuttle_mvcc.rs:349, MIT)	txn.Tx.CommitCtx with its context cancelled (a) before the call, then retried; (b) at a random point within twice the fsync latency; (c) after it returned; 2 rounds per writer; crash image; recovery	a_refused_with_canceled, a_retry_commits, c_completes, no_unexpected_errors, committed_equals_ok_in_memory and committed_equals_ok_after_crash (no ghost, no lost commit), later_commit_not_blocked

D18 and seam H2. AllocateCommitTS cannot hold a commit inside store/txn between its fsync and its publish: Tx.CommitCtx runs lpg.Graph.ApplyDurable, whose durable step fsyncs with commit timestamp 0 and publishes when the apply bracket ends, with no caller-visible point between them. The Cypher engine's commit (Tx.CommitWALOnly plus an lpg bracket) allocates through AllocateCommitTS after its fsync, but exposes no hook to hold it there. D18 therefore holds the window directly on the store's graph: an allocated, unpublished instant is exactly what AwaitCommitQuiescence waits for, whatever path allocated it. T0 writes nothing, so nothing outside the WAL reaches the snapshot. The other D18 branch — a T0 never released fails the checkpoint without truncating — is not exercised: its bound (commitQuiesceTimeout) is 30 s.

D19 (b) in practice. In the measured runs every phase-(b) commit returned OK (b_cancelled=0): the apply takes its claims within microseconds, so a cancellation drawn across the fsync lands after them, where CommitCtx no longer consults its context. Phase (a) is the cancellation-before-claims path.

Negative controls (D17-D19, rmp #3019)

Seeded faults applied with go test -overlay, never written to the tree, at 8 and 64 writers:

• D17 — wal.Open keeps the torn tail (store/wal/writer.go, the ErrTornFrame truncation skipped): epoch2_present fails at both levels.
• D18 — Checkpointer.awaitCommitQuiescence returns at once: checkpoint_waited and acked_present_after_crash fail at both levels (T1, durable below the watermark and invisible behind T0, is missing from the snapshot).
• D19 — Tx.CommitCtx reports a completed commit as cancelled when its context is done by then (a ghost commit): committed_equals_ok_in_memory and committed_equals_ok_after_crash fail at both levels, 3 of 3 runs.

Negative control (D14)

With the seam on, the durable image at 8 writers was cut at offset 4 343 889. Recovery reported missing=1 (id 1261), and durability.D01.durable level=8 acked_present=false and new_session_sees_acked=false were printed. TestDurabilityNegativeControl asserts that acked_present fails. The seam is off in every other run. D04 is the second negative control: 8 waiters of one failed fsync all failed, and none of them was visible to a new reader.

Defects found

Both were found by this phase, are fixed, and are now gated.

• D09 — a lost snapshot directory was not detected after a WAL prefix truncation (rmp #2990, fixed). After a checkpoint truncated the WAL prefix it folded, removing snapshot/ left a directory that recovery opened with a nil error and IsClean() = true; the commits in the truncated prefix were missing: 130 acknowledged commits at both 8 and 64 writers. PostgreSQL refuses to start when a needed WAL segment is missing (050_redo_segment_missing). Reproduction: store.Open; commit; checkpoint.New(...) wired with WithCommitSerialiser(o.Store().RunUnderCommitLock) and the codecs, then RunCheckpoint(); close; rm -r <dir>/snapshot; recovery.Open(dir). The fix made a marker file (wal.PrefixTruncatedMarkerPath) durable before the first discard of history; since the segmented WAL (docs/design-wal-v2.md) the same record is the WAL control file's prefix-truncated flag, written before any segment is unlinked. Recovery refuses a directory that records a truncated prefix but holds no snapshot with recovery.ErrMissingSnapshot; store.Open refuses it with store.ErrUncleanRecovery. Gated as durability.D09.missing_segment ... refused_loudly.
• D09 — a checkpoint capture was refused under explicit-transaction load (rmp #2991, fixed). RunCheckpoint, wired with RunUnderCommitLock while writers run three-statement explicit transactions, intermittently returned snapshot.ErrCaptureNotQuiesced. One observed message was: "capture instant taken while a write transaction was open: shard 98 drops node 2402 (interned, not visible at instant 132) but keeps node 2658 above it". It occurred on the first attempt at 8 writers in one run, and 3 times in 4 attempts at 64 writers under -race. The capture walks the mapper after the commit lock is released; an id interned after the instant by a transaction that then rolled back lost its birth record when the abort was withdrawn, so it read as interned before the instant and was kept above a dropped id. The checkpointer now opens its instant with lpg.Graph.BeginCaptureRead, which reads the mapper watermark just after the instant, and the capture keeps exactly the per-shard prefix the watermark names. The drain does not stop every interning: an id the watermark covered but that was not visible at the instant was then captured as a tombstone with its key, which WAL replay did not revive when its transaction created the node through an edge alone (docs/design-wal-v2.md risk 7, confirmed). The arm makes one attempt, with no retry, gated by checkpoint_ran. WAL v2 step 1 then retired the refusal itself: the capture carries only the ids ever born as of its instant, every other assigned id is a hole, and recovery accepts holes (docs/design-wal-v2.md §3).

Observed, by contract

An explicit transaction registers as a writer from BeginTx until it commits or rolls back, and RunUnderCommitLock closes admission and waits for every registered writer. A transaction held open therefore holds back the checkpoint and, once admission is closed, every writer. The first kill child held one transaction open for its whole life, and the child deadlocked. The child now reopens its open transaction every 5 ms.

Running

go test -race -run 'TestDurability' ./examples/37_mvcc_write_contention/
go test -tags soak -run TestDurabilityKill9 ./examples/37_mvcc_write_contention/
go run ./examples/37_mvcc_write_contention -ladder-levels "" -durability-levels 8,64 -durability-kill-runs 5

Every image and every derived copy is created under TMPDIR and removed at the end of its arm. TestDurability opens the live stores of the in-process arms (D01, D09) with the same injected fsync latency as the ladder (1-5 ms, GOGRAPH_FSYNC_LATENCY=off to disable, GOGRAPH_FSYNC_LATENCY_SEED to replay; rmp #3022); the binary and the kill -9 child run without it.

Phase 8 — node identity across processes (GG07, rmp #3015)

Catalogue row GG07: node identity under concurrent CREATE, rollback and store reopen (identity.go). A CREATE gets a hidden node key from a counter that is process-wide, so the scenario needs fresh processes: every workload runs in a child — this binary with -identity-child, or the test binary re-executed — whose counter starts at zero, as a restarted application's would. The stores are persisted (store.Open) under TMPDIR.

Workload

8 concurrent sessions, 6 explicit transactions each; each transaction runs one CREATE (n:P:T_<tag> {tag, s, i}), and the odd ones are rolled back. The committed CREATEs whose transaction number is a multiple of four are then deleted.

Arms

• reopen — child 1 writes store R; child 2 reopens R and runs the workload again.
• two_stores — child 1 writes store B; child 2 runs one CREATE in a fresh store A, then opens B and runs the workload on A and B at the same time.
• memory — child 2 runs one CREATE in a fresh in-memory graph, then recovers B into a second in-memory graph (recovery.Open, read-only) and runs the workload on both at the same time. It checks both graphs itself, because they end with it.

Each arm makes its collision certain rather than likely. A seeding child runs four committed anchor CREATEs before its sessions, so keys 1 to 4 of its store are live, and the first CREATE on the second graph after the one-shot seed draws key 2. The seeding child also commits and then deletes one last CREATE, so the largest key of its store belongs to a deleted node.

Gates

Every acknowledged CREATE reports the hidden key, id() and elementId() it was given, read in the process that committed it. The parent reopens each persisted store and gates, for every arm:

Check	Holds when
exercised	the store has commits, rollbacks and deletes
keys_distinct	no hidden key was given to two acknowledged CREATEs over the store's history, deleted nodes included
acked_ids_distinct_in_process	no id() or elementId() was given to two acknowledged CREATEs of one process
acked_present_once	every acknowledged, undeleted CREATE is present exactly once
nothing_else_present	no other node is present (rolled-back and deleted nodes absent)
no_merged_identity	every node has only its own two labels and three properties
ids_distinct	the live nodes' id() and elementId() values are all distinct

With the fix reverted (go test -overlay over cypher/exec/create_node.go, merge.go and merge_pattern.go), keys_distinct failed in every arm, and acked_present_once, nothing_else_present and no_merged_identity failed in two_stores and memory, in 5 of 5 runs. With the fix, the gate passed in 30 of 30 runs. The defect is D8 under phase 5's "Defects found".

Gated since WAL v2 step 3

ids_reused_across_processes and ids_moved_across_processes compare id() and elementId() across processes: an id given to CREATEs in two different processes, and a live node whose id differs from the one its CREATE returned. Before WAL v2 step 3 (rmp #3021 A) they were reported, not gated: id() was not stable across a reopen when rolled-back CREATEs preceded committed ones in a mapper shard, because a rolled-back CREATE kept its position in the process that ran it while recovery appended only committed keys. In one probe, 62 of 200 committed nodes came back with a different id(). Every commit marker now names the exact id of each node its transaction created, recovery places each key at that id, and both are gates at 0.

Running

go test -race -run '^TestIdentity$' ./examples/37_mvcc_write_contention/
go run ./examples/37_mvcc_write_contention -ladder-levels "" -durability-levels ""

The binary runs phase 8 unless -skip-identity is set.

Status

IMPLEMENTED (rmp #2313). main.go holds the configuration, the workload primitives and the telemetry; phases.go holds the four measurement phases; example_test.go is the short-layer regression gate.

The specification above was committed before any code, so the acceptance criteria were fixed in advance rather than fitted to whatever the implementation produced.