sim-xrelease-helper

Overview ¶

checkpoint.go is the REMOVABLE half of the cross-release helper (rmp #2477, reworked by rmp #2531).

It installs [publishCheckpointHook] so the helper publishes a durable checkpoint — a snapshot directory under <dir>/snapshot plus a WAL truncated to the snapshot's watermark — before it exits. That is what puts a PRIOR RELEASE'S snapshot bytes in front of the current code: manifest.json, csr.bin, labels.bin, properties.bin, mapper.bin and edgehandles.bin as that release wrote them, opened by the current reader.

Why Start/TriggerCtx/Stop and not RunCheckpoint (rmp #2531) ¶

This file is compiled against the target TAG's packages, so every symbol it names must exist at that tag. The obvious entry point, checkpoint.Checkpointer.RunCheckpoint, is the WRONG choice for exactly that reason: it was only exported from v0.6.0 onwards. At v0.1.0..v0.5.0 the same body exists solely as the unexported runCheckpoint, so naming it there fails the build with

cp.RunCheckpoint undefined (type *checkpoint.Checkpointer[string, float64]
has no field or method RunCheckpoint, but does have unexported method
runCheckpoint)

which cost this file its build at every tag the harness actually exercises and silently reduced cross-release coverage to HEAD-as-prior.

Start/Trigger/TriggerCtx/Stop, by contrast, have been exported with an UNCHANGED shape since v0.1.0 — as have checkpoint.New and checkpoint.Config's Dir field — so this route reaches every release tag the repository holds. It is also the same body: the checkpoint loop's triggerCh arm calls precisely the runCheckpoint that RunCheckpoint later exposed, so nothing about the artefact on disk depends on which door was used.

The sequence is New → Start → TriggerCtx → Stop, and each step is load-bearing:

• Start is REQUIRED, not optional. TriggerCtx submits a request on a buffered channel and then waits for the loop to answer it. With no loop running, the submit succeeds into the buffer and the wait never completes: Trigger alone would HANG rather than fail, so the loop must be up first.
• TriggerCtx rather than Trigger, so the wait is bounded. A checkpoint that cannot complete becomes a reported error at [checkpointPublishTimeout] instead of a wedged subprocess the harness can only kill on its own outer deadline.
• Stop joins the loop goroutine before this function returns, so the checkpoint has demonstrably finished writing through the WAL writer that main.go closes immediately afterwards.

Why this is a second file and not part of main.go ¶

Both files are staged into a git worktree of the target tag and compiled against THAT tag's packages. main.go is pinned to the API stable across v0.2.0..HEAD; the checkpoint API is younger and has moved more. If the checkpoint call did not compile at some tag and lived in main.go, the whole binary would fail to build and the harness would report the tag as "unbuildable" — a clean SKIP that silently removes an entire release from cross-release coverage, which is exactly the vacuity this task exists to avoid. Kept apart, the harness builds with both files, and on failure drops THIS one and rebuilds: the tag still runs, still writes a WAL image, and the wire protocol reports checkpoint=false so the caller knows which of the two shapes the image has. See github.com/FlavioCFOliveira/GoGraph/internal/sim BuildPriorReleaseHelper for the two-stage build.

That fallback is retained deliberately even though every tag in the repository now builds WITH this file. It is the harness's insurance against a FUTURE tag whose checkpoint API moves again, and its loudness — BuildFallbackErr surfaced in the run report — is what made the rmp #2531 gap visible in the first place.

Why the option set is deliberately minimal ¶

Every option passed here is another symbol that must exist at every tag, and each one that does not costs the whole checkpoint at that tag. Only what the helper's fixed shape actually needs is used:

• No WithMapperCodec. N is string, for which the checkpointer already writes the self-sufficient string mapper, so the WAL prefix is truncated anyway.
• No WithWeightCodec. W is float64, a fixed-width primitive the dense CSR weights column persists natively; a codec is only needed for weight types that column cannot size (rmp #2526). It is also younger than most tags — it does not exist before HEAD — so naming it would reintroduce the very build failure rmp #2531 removed.
• No WithCommitSerialiser. The helper drives the store from ONE goroutine and the checkpoint runs after the last op, so there is no concurrent writer for the serialiser to drain and the storeMu fallback's precondition ("the caller serialises its own writes under that same mutex") holds vacuously — the checkpoint now runs on the loop goroutine, but it is still the only party touching the store while it runs.
• No constraint or index specs. The cross-release op stream is CREATE / MERGE / MATCH / SET / DELETE over nodes and edges and declares no DDL, so there is no durable schema for the truncated WAL prefix to strand.

Command sim-xrelease-helper is the prior-release subprocess driver for the DST cross-release harness (internal/sim). It is built FROM A PRIOR GIT TAG'S SOURCE TREE — the harness copies this file verbatim into a temporary git worktree checked out at the target tag and runs `go build` there, so the resulting binary embeds that release's store/txn/wal/cypher code. The current (HEAD) process then drives it over a small stdin/stdout protocol to obtain a store image written by the prior release and the prior release's observable per-op results.

Why a copied-in file rather than a tag-resident command ¶

A git tag is immutable and prior tags ship no headless CLI to drive. Adding an UNTRACKED file to a worktree of the tag does not modify the tag; it simply compiles against that tag's packages (every relevant tag shares the current module path github.com/FlavioCFOliveira/GoGraph and the same store/cypher API surface, verified by the harness build step). The file therefore uses ONLY the API that is stable across v0.2.0..HEAD: wal.Open, txn.NewStoreWithOptions, txn.New{String,Float64Weight}Codec, cypher.NewEngineWithStore, and the cypher.Result reader. It must not reference anything newer, or the build at an older tag fails (which the harness reports as "tag unbuildable", a clean skip).

v0.2.0 is the floor this file is PINNED to, not the floor it was measured at. rmp #2531 swept every release tag the repository holds and this file, together with checkpoint.go, builds and drives the full pipeline at v0.1.0 as well. The pinned range is left as the contract — it is the promise future edits must keep — while the measured reach is recorded here so a decision to add older tags to the harness's list does not have to re-derive it.

Protocol ¶

Invocation: sim-xrelease-helper write <dir>

stdin   one JSON object per line, each a {"kind","cypher","params"} op.
        params values are string | float64 (JSON number) | bool.
stdout  one JSON object per line, each {"i","committed","rows"} where rows
        is a canonical order-independent signature of the op's result rows
        (empty for ops that produced none). A trailing line
        {"done":true,"nodes":N,"edges":E,"checkpoint":B} reports the final
        engine counts and whether a checkpoint was published.
exit    0 on success (store written to <dir> and fsynced via store close);
        non-zero with a diagnostic on stderr on any harness-level failure.

The helper writes a real WAL under <dir>/"wal" via the prior release's txn.Store, so the current code can reopen <dir> with recovery.Open — the genuine cross-version data-compatibility boundary.

The checkpoint (rmp #2477) ¶

Before exiting, the write path also publishes a durable CHECKPOINT: the prior release writes a SNAPSHOT DIRECTORY under <dir>/snapshot and truncates the WAL prefix the snapshot now covers. Without it the cross-release harness only ever handed the current code a WAL, so a prior release's manifest.json, csr.bin, labels.bin, properties.bin and mapper.bin had never been parsed by current code at all — the whole snapshot format was outside the cross-version test.

The checkpoint lives in a SEPARATE staged file (checkpoint.go) reached through [publishCheckpointHook], so a tag whose checkpoint API differs degrades to a WAL-only image that says so on the wire instead of becoming an unbuildable skip. See the hook's own documentation.