README
¶
title: Work Queue protocol model description: TLA+ model, safety proof argument, and bounded verification for the work queue.
Work Queue protocol model
The current design is the mandatory priority/fairness DAG protocol: one causal QueueCommit log, exact native Go/JavaScript scheduling conformance, immutable Claim arrays, verified Work Results and typed Issue/PR observations. Its implementation coverage table tracks unfinished runtime/verification work and user-deferred writer-restriction enforcement. Do not infer deployment-security completion from functional tests.
The original WorkQueue.tla is retained as historical model evidence for
issue #64852. Its best-effort
oldest-first selection, competing Claim arbitration and scalar worker behavior
do not describe the new current protocol.
The design rationale and trade-offs are recorded in ADR-64955.
Current operator interface
The current operator commands and keyboard browser are documented in the queue reference. Policy installation is documented in the deployment guide. This directory retains executable specifications and formal evidence, not a second current user guide. The historical material below is model provenance, not supported deployment guidance.
Local stress simulator
Run the simulator from the repository root with Node.js 24 or later and Git. It uses temporary local Git repositories, concurrent worker threads and a simulated GitHub Git/Actions API; it does not contact GitHub or need credentials.
node --test .github/scripts/work-queue-stress.test.cjs
node .github/scripts/work-queue-stress.cjs --mode history --items 100000 --workers 2
node .github/scripts/work-queue-stress.cjs --items 1024 --workers 4 --queue-items 64 --seed 7
node .github/scripts/work-queue-stress.cjs --mode saturation --items 100000 --workers 4
The history profile counts retained Work plus Claim items, not successful
worker completions. The lifecycle profile submits, grants, launches simulated
original native runs, finishes Claims, verifies no-op delivery receipts and
releases reservations. Its total workload is explicitly divided into bounded
queue branches; --items 100000 is a long-running aggregate workload, not a
promise that one queue admits 100,000 pending Work items. The saturation profile
instead reports the first real admission refusal, accepted batch count and
unattempted remainder without fabricating Claims or completions.
Seeded faults force competing real Git reference updates and lose one successful publication response. JSON diagnostics report CAS conflicts, recovered requests, native POST counts, ledger bytes, cold replay timings, memory and workload scope. Workers have bounded deadlines and fixtures are cleaned up on completion or failure. Canonicalization must preserve the complete causal history and projection.
The separate Work Queue Stress workflow
runs on pull requests and main pushes changing actions/setup/js JavaScript,
the simulator, its workflow or protocol fixtures. It runs the 100,000-item history
profile, 1,024 live lifecycles and admission saturation independently. Its step
summary reports profile outcomes, measured workload, elapsed time, throughput and
peak memory, with phase timings and Git contention in a collapsible section.
History validation throughput is separate from live Work completion throughput;
failed or skipped profiles never imply successful completion. The summary and
JSON diagnostics are uploaded even on failure. It uses read-only checkout
credentials and does not dispatch actual GitHub workflow runs.
Render saved JSON diagnostics locally with
node .github/scripts/work-queue-stress-summary.cjs stress-results. The command
writes stress-results/summary.md and appends the same report to
GITHUB_STEP_SUMMARY when that environment variable is set.
Runtime scaling and compaction
The JavaScript replay maintains private indexes for available Work, unfinished delivery barriers, open Claims, native reservations, graph membership and original run bindings. Scheduling copies only unfinished Work, its direct predecessors and held reservations, rather than cloning the entire ledger. Graph admission checks new nodes against indexed accepted graphs and traverses new edges iteratively. These indexes are derived from validated commits, never stored as authority. Detached public projections rebuild indexes when used again, so caller mutation cannot leave a stale scheduling cache.
The Git store parses and replays a read once, validates an appended candidate once, and serializes that checked history without another replay. Custom candidate generators still receive isolated copies. Multi-MiB base64 blobs use linear validation rather than a repeated-quartet regular expression that can overflow the JavaScript engine's stack.
Compaction remains canonical ordering and identical-commit deduplication only. It does not erase terminal Work, historical Claims, request identities, FIFO positions or scheduling debt. Native ledger, graph, pending-node and recovery bounds still apply; 100,000 submitted Work items are not interchangeable with 100,000 combined Work/Claim records. A workload that reaches admission limits must report that boundary, not raise policy ceilings or silently drop history.
Historical queue inspection and operator commands
[!WARNING] The commands, facts, storage backends and scalar assignments in this historical section are superseded. They are not compatibility modes of the current implementation. Use the current queue reference and the current command's help; old queues are rejected unchanged.
gh aw work-queue operates on a dedicated branch without using the current checkout. Supply
--repo owner/repo; use --branch to select a different
queue branch (default: gh-aw-work-queue). The experimental command uses authenticated GitHub Git APIs to
read and validate work-queue.jsonl, create trees and commits, and
publish changes with non-force reference updates. It needs neither a checkout nor
a Git executable, and accepts GitHub repositories rather than local Git remotes.
Rejected concurrent updates are retried against a fresh branch snapshot and replay.
In an initialized repository, an absent queue branch is initialized with a
parentless commit; other files in an existing queue branch are preserved.
GitHub Git APIs cannot create the first reference in an entirely empty repository.
Authentication uses the GitHub
CLI configuration or GH_TOKEN/GITHUB_TOKEN, with repository contents write
permission required for mutations.
All subcommands support --json for machine-readable output. Workflows enable the
read-only snapshot MCP server with tools.work-queue: true.
For the workflow runtime (not the operator CLI), tools.work-queue: {storage: issues}
selects issue storage instead of the default Git branch. Each Work is an issue with
the aw:work-queue label; its Work transaction is in the issue body and later
transactions are comments. aw:work-queue:available, :claimed, :completed,
and :cancelled labels display the projected state. Comments are replayed in
publication order; only issues and comments authored by github-actions[bot]
with a valid HMAC signature participate. Configure the repository secret
GH_AW_WORK_QUEUE_HMAC_SECRET with a random value (for example, generate one
with openssl rand -hex 32) so trusted workflow steps can sign and verify each
record. Keep this secret unchanged while records exist; rotating it invalidates
their signatures. Labels are not used to authorize a worker. The same
snapshot and MCP tools are used with either storage choice. Both backends must
not be used on the same logical queue without an explicit migration. Issue
storage needs issues read access at activation and conclusion, and issues write
access at trusted safe-output publication. Each refresh lists the issues and
comments in full, so the Git backend is preferable for large queues.
| Command | Arguments |
|---|---|
replay |
Display the projected Work and Claims |
stats |
Count Work, Claims, and distinct transactions |
compact |
Canonically order facts and remove only identical duplicates |
submit-work |
--file work.json (or --file - for stdin); derives an id from the canonical JSON object |
claim |
--run-id RUN [--work-id ID]; defaults to the oldest available Work |
finish |
--claim-id ID --attempt-id ATTEMPT [--outcome TEXT] |
cancel-work |
--work-id ID |
cancel-claim |
--claim-id ID |
These are operator commands; finish writes a Completion fact, but is not a
worker safe-output authorization mechanism and does not execute external effects.
The caller must independently establish the provenance of --run-id and
--attempt-id. Worker authorization and MCP/safe-output integration remain
separate implementation obligations described in the ADR.
The historical operator transaction wire format was Transaction; it is no
longer defined in the current transactions.tsp or accepted
by the current embedded schemas. The historical workflow runtime used a
separate work-queue branch and the versioned WorkQueueTransaction format:
required fields version: 2, kind, work, claim, and attempt, plus optional
enqueued on Work. Unused claim/attempt
fields are explicitly null; identities are nonempty strings. The Git loader upgrades
unversioned/version-0/version-1 workflow records through successive codemods before
validation and replay. Version 1 remains the closed five-field format without
enqueued; historical records with extra fields are rejected before upgrading.
The 1-to-2 upgrade changes only the version, leaving historical Work at age zero
without inventing enqueue metadata. The snapshot envelope version is independent
of the transaction version. These formats
are not interchangeable; the CLI branch also preserves Work payloads and run
provenance, which the workflow fact format does not contain.
Trusted workflow assignments use the compiler-managed work_queue_claim
workflow-dispatch input, containing work_id, claim_id, and the work payload
object (WorkQueueAssignment). aw_context remains reserved for caller metadata.
Activation checks that the Claim is currently effective before capturing the assignment.
When tools.work-queue: true and safe-outputs.dispatch-workflow are both
configured, a dispatcher can read an available identity with work_queue_read
and pass work_queue: {work_id: "..."} to an allowed same-repository worker's
dispatch tool. The worker enables tools.work-queue: true; the compiler adds its
reserved work_queue_claim input so activation admission and completion
reconciliation are compiled. Workflows must not declare aw_context or
work_queue_claim themselves.
Trusted safe-output processing refreshes the queue, publishes a new Claim for
that identity, verifies it is effective, and injects the assignment with
work: {id: work_id} through work_queue_claim. The work_queue selector is
not forwarded as a workflow input.
Dispatch errors attempt to cancel the Claim. An ordinary dispatch without a
selection and a staged preview create no Claim; call-workflow does not create
one either. Concurrent Claims may supersede an assignment before worker
activation, so admission and completion still recheck the durable queue.
The MCP tools are work_queue_read and work_queue_claim_finish; the latter
records a WorkQueueFinishIntent containing only outcome: "completed" or
outcome: "cancelled". Artifacts use work-queue.snapshot.json and
work-queue.finish.jsonl; the Git-backed durable transaction file is work-queue.jsonl.
When the runtime prompt advertises work-queue under <mcp-clis>, invoke these
tools as work-queue work_queue_read '{"work":"example"}' and
work-queue work_queue_claim_finish '{"outcome":"completed"}'. The tool names
are subcommands of the server's CLI wrapper, not standalone executables.
Copilot advertises this wrapper when CLI mounting is active; other engines
advertise it with tools.cli-proxy: true.
Both tools return JSON serialized in MCP text content. The finish-intent file
contains only the outcome and is readable by the runner artifact collector even
when the MCP container runs as a different user.
Pre-rename aw_context.work_claim assignments are schema-checked and normalized
to work_queue, not treated as unassigned. Existing runtime storage on
dispatch-coordinator / dispatch-work-coordinator.jsonl is read and updated in
place with the same checked publication protocol. It is not silently copied to
an independent queue. Explicit migration requires quiescing all writers and old
workflows, renaming the existing branch/log, and deploying recompiled workflows
before resuming. Both branch names or both log filenames together are ambiguous
and fail closed. Logs/audit retain read compatibility for historical artifacts.
Current wire contract and schema checks
transactions.tsp defines the current version-3
QueueCommit contract shared by the operator and workflow runtime. Historical
Transaction, WorkQueueTransaction and scalar assignment formats above are
not accepted by these schemas.
The checked-in schemas are generated by the dependency-free, fail-closed TypeSpec-subset generator, not copied from the official emitter:
python3 specs/work-queue/generate_contract.py
python3 specs/work-queue/generate_contract.py --check
--check compares the complete generated schema set with pkg/workqueue/schema/
and fails on drift. A separate validation runs the genuine
@typespec/compiler@1.16.0 and @typespec/json-schema@1.16.0 with
file-type=json and seal-object-schemas=true into a disposable output
directory. The fail-closed comparison below resolves references and compares
validation constraints for the supported schema subset, including custom
identity bounds. The current 41-schema set passes this comparison; this is not
a general JSON Schema equivalence solver or a proof of runtime validation.
Do not overwrite the checked-in schemas with the separate official output.
For official validation, TOOL_DIR denotes a disposable directory containing
those two exact pinned packages, and OUT_DIR denotes a separate output
directory:
cp specs/work-queue/transactions.tsp "$TOOL_DIR/transactions.tsp"
"$TOOL_DIR/node_modules/.bin/tsp" compile "$TOOL_DIR/transactions.tsp" \
--emit @typespec/json-schema \
--option @typespec/json-schema.file-type=json \
--option @typespec/json-schema.seal-object-schemas=true \
--output-dir "$OUT_DIR"
python3 specs/work-queue/verify_official_contract.py \
--official-dir "$OUT_DIR/@typespec/json-schema"
The current wire profile restricts numbers recursively, including payloads: only plain safe integers are accepted. Other exact quantities require explicit strings. All accepted payload fields and decoded values are retained without coercion.
Historical arbitration model
The retained WorkQueue.tla model abstracts identities as integers;
the historical CLI used stable string identities and canonical JSON Work payloads. Protocol
versions and payload data are outside the model's arbitration abstraction.
Verification status: the module includes parameterized safety theorem statements and the inductive proof argument below. TLC exhaustively checks the supplied finite configurations. The theorem statements are not mechanically checked by TLAPS; bounded model checking is not an unbounded proof.
Historical model's concrete protocol choices
The model fixes arbitration and makes the worker lifecycle, publication guards, and terminal-state protection explicit:
| Boundary | Modeled rule |
|---|---|
| Arbitration | The least stable, uncancelled Claim identity wins on nonterminal Work. A persisted Completion fixes the winner; WorkCancellation removes authority. |
| Queue selection | Stage a Claim for the oldest available Work in the dispatcher's local replay. Use an immutable enqueue-time key with stable Work identity as the tie-breaker; skip claimed and terminal Work. |
| Terminal Work | Reject new state-changing transactions for completed/cancelled Work. Identical physical records may be duplicated without changing the fact set. |
| Dispatcher | The activation job reads the queue branch and packages its log and branch version into the activation artifact. The work-queue MCP server reads only that immutable snapshot and never accesses Git; the view may be stale while the agent runs. Mutations are revalidated and published only by trusted safe_outputs; pending competing Claims may become durable on nonterminal Work. |
| Worker | Each worker has one immutable inbound Claim and one pass through safe-output processing; it records at most one distinct Completion. Finalize carries no authority parameters. |
| Authorization | The winning worker verifies its newly committed Completion before outputs. Finished, stopped, and failed workers cannot restart or receive authorization again. |
| Compaction | Canonicalize order and remove identical duplicate records only. Preserve the entire fact set, including cancelled/superseded Claim history. More aggressive compaction needs a separate proof. |
These are protocol refinements, not claims that an implementation already enforces them. Claim ordering is by stable identity, not arrival time. Arrival order can change which transactions are accepted; it cannot change replay of the same accepted fact set.
WorkOf, Inbound, and Origin provide small, deterministic identity mappings for TLC. Inbound represents validated trusted context, not agent-selected input. Claims also identify their owning run; multiple worker attempts may share one Claim/run. Payloads are abstracted to stable Work identities, assuming collision-free canonical identity and idempotent submission.
Best-effort queue ordering
Each Work fact carries immutable enqueued metadata, set once at submission and preserved through retries, replay, and compaction. The finite model uses the Work integer as the rank of the (enqueue time, Work identity) key, not as a physical log position or a globally allocated sequence number.
JavaScript and Go represent enqueued as Unix milliseconds, restricted to nonnegative integers no larger than 9007199254740991 so both languages compare exactly. Both replay projections expose an oldest-first available identity list, with UTF-8 Work identity as the tie-breaker. Historical Work without metadata has age zero and sorts before timestamped Work; replay never invents timestamps from record position or the current clock. Repeated submissions preserve the first durable Work fact's age, while conflicting metadata already present in the durable log is rejected.
JavaScript callers construct Work with createWorkTransaction and select or stage Claims with oldestAvailableWork / claimOldestAvailableWork, passing their local view including earlier pending intents. Go callers use NewWork and OldestAvailable. gh aw work-queue claim --run-id RUN selects once from its initial view and retains that Work identity across publication retries; --work-id ID remains an explicit operator override. work_queue_read lists available Work first in enqueue order by default, includes enqueue metadata, and returns the snapshot's recommended next_work identity or null. Its optional work selector reads a single identity, while optional sort reorders available Work and the queue-wide next_work recommendation. sort accepts one to four ordered keys (id, enqueued, or id_length), each with a direction (asc or desc): {"sort":[{"key":"enqueued","direction":"desc"}]} recommends newest available Work. Later keys break ties, then the default oldest-first order does. ID length counts Unicode code points. Non-available Work remains after available Work. Sorting is read-only: it does not change Claim selection, safe-output dispatch authority, or publication order, and the recommendation is not durable authority.
OldestAvailable chooses the least key among Work whose replayed state is available. Stage applies that preference to the local view, including earlier pending intents, so a batch cannot repeatedly select the same Work. Claimed Work does not block selection of newer available Work; cancellation of its last active Claim makes it eligible again with its original age. Work and claim arbitration remain separate.
Publication deliberately does not enforce oldest-first ordering. Allowed continues to accept competing Claims on nonterminal Work, and a version-conflict retry replays already selected intents without moving a Claim to another Work. An older submission can become visible after a newer item was selected; concurrent workers can also complete out of order. The model's local view is current durable facts plus pending intents at staging, but even that view may be outdated before publication. Implementations selecting from immutable activation snapshots have the same limitation.
This is a queue preference, not strict FIFO, a bounded-overtaking guarantee, or a claim that reordering has a particular probability. Reordering should be uncommon with fresh views and prompt publication, but TLA+ explores adversarial schedules and does not measure frequency. No global sequence allocator, head-of-line lock, or completion barrier is introduced.
Historical model's state and job boundaries
log is the only authoritative durable state. head abstracts an opaque, non-reused Git branch version. Branch creation is a version-checked write against the initial absent-branch token. All successful mutations change that version; no force-push or reuse of an old version is permitted.
The activation artifact is an immutable snapshot of the queue log, branch version, and validated worker assignment read during activation. The MCP server mounts that snapshot read-only and derives its query results with shared replay; it has no Git client or repository credentials. Its only writable mount is the safe-output intent directory, where work_queue_claim_finish(outcome?) records an outcome without accepting Work or Claim identity. The snapshot is only an early view and can be stale by the time the agent asks a question or submits work. Any future local pending-intent view remains non-authoritative. Every trusted writer records both the source log and its branch version; CandidateDerivation verifies that the candidate was generated from that latest source, not from the activation snapshot.
Workers progress through activation snapshot capture/admission, execution, finalize/no-finalize, preparation, push, verification, and effects. Admission reads the immutable activation snapshot and is not retained authority: WorkerCandidate checks current ownership against the latest log again. A stale candidate must be regenerated. Missing finalize cancels an effective Claim without permitting outputs; losing attempts stop without effects. In the implementation, safe-output processing downloads the activation and agent artifacts, reconciles against the latest Git-backed log, and gates user steps and handlers until a Completion for the trusted inbound Claim is verified. The finish outcome cancelled and an absent finish intent both map to finalize = FALSE; completed maps to finalize = TRUE.
Each worker's inbound Claim is fixed by Inbound. SingleCompletionPerWorker bounds its distinct Completion facts, and WorkerOneShot makes finished, stopped, and failed phases absorbing. The former reauthorization example added an impossible restart transition; it was not a reachable protocol failure and has been removed.
Compaction and recovery have independent prepared snapshots and bounded retries. Recovery cancels unresolved Claims whose owning runs terminated, including superseded Claims on nonterminal Work. Compaction discards stale snapshots and rebuilds from current facts.
terminalHistory records each Work item's complete fact set at its first terminal decision. It and authorizations/effects are observer histories, not additional queue files or decision-making state. Authorization/effect histories are sequences, so repeated execution of the same record cannot disappear through set deduplication. One ExternalEffect represents entry into one attempt's ordinary safe-output batch, not one GitHub API call.
Apply abstracts explicit rejection and idempotent no-op outcomes as no append. The model admits an identity-based Work resubmission with different enqueue metadata as idempotent and checks that staging it does not change replayed facts or the first durable Work age. Other exact duplicate intents are abstracted as no append; the model does not track implementation diagnostic counts. To keep the finite search bounded, each exact intent can be staged at most once per dispatcher.
JavaScript applyTransactions returns the candidate transactions, invalid intents in rejected, and an idempotent count. Exact duplicates of every kind and identity-based Work resubmissions are idempotent, not rejected, even when resubmitted Work carries a different enqueue time. Publication logs count requested intents as new, rejected, or idempotent on each attempt, independently of duplicate physical records in the source log. Retry outcomes describe only the refreshed attempt, not accumulated counts. All-no-op publication skips writes unless the log needs a protocol upgrade or canonicalization.
Required protections
This section describes the retained historical WorkQueue.tla abstraction,
not the current QueueCommit wire contract or batching algorithm.
Activation snapshot: the activation job reads the queue branch and validates any trusted inbound assignment before uploading the activation artifact. The queue MCP process receives a read-only mount of the packed snapshot plus a writable safe-output intent directory; it does not receive a Git client or repository token. Snapshot queries are informative, not authority, and may be stale.
Publication: publish only if the branch still matches the version originally read. Otherwise fetch the latest log, replay it, and regenerate the proposed changes.
All dispatcher, worker, recovery, and compaction pushes use Publish. It requires both snapshot.base = head and snapshot.source = log. Their retry actions capture the latest version/log and rebuild the entire candidate through replay; none reuse stale output or splice it into newer state. Retry exhaustion fails without publication or external effects.
Terminal Work: reject new state-changing transactions after Work becomes completed or cancelled. Late competing Claims cannot reopen the decision.
Allowed applies the nonterminal guard to every new transaction type. TerminalFreeze requires a terminal Work item's current fact set to remain identical to its recorded terminal snapshot. Physical duplicate records and compaction may change representation, but cannot change those facts.
Invariants
| Predicate | Guarantee |
|---|---|
TypeOK, ValidLog |
Valid identities/references; at most one terminal transaction per Work; Completion belongs to the trusted inbound Claim and the selected claimant. |
SnapshotValidity, SnapshotVersions, ActivationSnapshotValidity, CandidateDerivation |
A matching writer snapshot contains the current source log and an exactly regenerated candidate; activation admission reflects its immutable artifact snapshot; no snapshot is based on a future version. |
Serialization |
Every locally proposed dispatcher transaction is serialized identically for safe-output processing. |
TerminalPersistence |
Previously committed Completion/WorkCancellation facts remain durable. |
TerminalHistoryValid, TerminalFreeze |
All facts for a terminal Work item are frozen; neither late Claims nor other new transactions can change the decision. |
SingleCompletionPerWorker, WorkerOneShot |
One fixed inbound Claim and at most one distinct Completion per worker; terminal worker phases never restart. |
SingleEffectiveClaim |
At most one effective Claim per Work. |
QueueSelection (action property) |
Every staged Claim selects the oldest available Work in the local view at that step, not necessarily the oldest at publication or completion. |
WorkResubmissionNoOp (action property) |
A same-identity Work resubmission with different enqueue metadata may be staged, but does not change replayed facts or the original Work age. |
WorkerOrigin, FinishRequired |
Worker mutations concern only the inbound Claim; authorization requires finalize. |
AuthorizationSoundness, EffectSoundness |
Authorization follows durable Completion; effects follow authorization. |
LifecycleAccounting |
Each attempt authorizes/emits at most once, without resetting its lifecycle. |
SingleAuthorization, SingleEffect |
At most one safe-output attempt per Work receives authorization or enters its output batch. |
Inductive proof argument
The argument is parameterized by the identity-domain sizes and retry limit. It does not use MaxHead or MaxLog, which appear only in TLC's exploration constraint.
Replay and compaction lemmas
Replay(s) is defined as Projection(Facts(s)). Substituting equal fact sets gives ReplayDeterminism immediately; physical record order and duplicates have no influence.
For finite f, induction on its cardinality proves Facts(Canonical(f)) = f. The empty case returns the empty sequence. The nonempty case emits one chosen member and recursively emits exactly the remaining members. Thus CompactionEquivalence follows by the replay lemma.
For any intent list, induction on its length also shows that applying it to logs with equal fact sets produces equal resulting fact sets: Allowed depends only on facts, and append adds the same accepted fact in either case. Consequently the modeled compaction preserves future mutation decisions, not just current displayed state. It may change HEAD and provoke a retry, as any concurrent write can.
OldestAvailable also depends only on facts and immutable Work metadata. Equal fact sets therefore give the same next selection, and compaction preserves the ordering preference even when it rearranges physical records.
Accepted-extension lemma
Assume ValidFacts(f). One accepted transaction preserves it:
- Work adds an identity without references.
- Claim and ClaimCancellation require their existing references and nonterminal Work. No existing Completion can have its arbitration result changed.
- WorkCancellation requires nonterminal Work, so it cannot coexist with a prior terminal decision.
- Completion requires nonterminal Work, the current winner, and the trusted attempt/Claim binding. It becomes the sole terminal decision.
Duplicate records preserve the fact set. Induction over the serialized batch gives the same result for Apply.
Initial state and snapshot preservation
The empty log, empty histories, zero HEAD/bases, and initial phases satisfy Safety.
The activation artifact contains the then-current branch version and full replayable transaction log. The finite model abstracts this to the version and admission result for the worker's inbound Claim. Activate derives admission from the captured log and stores the snapshot version only for admitted workers; no later action changes that field. Consequently ActivationSnapshotValidity and the bound on activation snapshot versions are inductive, while later execution can observe a stale version. Worker preparation derives its candidate from the latest log, never the artifact. Preparing or retrying a dispatcher/worker/recovery candidate applies the accepted-extension lemma to the current log. Preparing a compaction uses the canonicalization lemma. Both the source log and its version are captured, and CandidateDerivation records their exact relationship to the regenerated candidate. Recovery also captures the observed terminated runs.
A local/read/lifecycle step does not change remote facts or HEAD. A successful write either leaves the log unchanged or increases HEAD. Existing snapshot versions cannot equal the new HEAD because SnapshotVersions bounds them by the old HEAD. Their matching-source implications become false. Newly prepared snapshots again satisfy those implications. Therefore snapshot validity is inductive.
Durable safety
A push requires equality with both its recorded source version and source log. SnapshotValidity and CandidateDerivation therefore supply a valid extension generated from actual current facts, not a stale projection. Compaction supplies exactly the current fact set. Neither removes a terminal fact, preserving ValidLog and TerminalPersistence.
When Work first becomes terminal, Commit records all its facts in terminalHistory. Any later Apply rejects new facts for that Work through the universal nonterminal guard; worker preparation also stops on terminal Work. Duplicate records and compaction preserve fact sets. The stored terminal snapshot therefore remains equal to current Work facts, proving TerminalFreeze. This applies to higher-ranked and lower-ranked late Claims alike.
The winner is a single-valued function: a stored Completion's Claim, no Claim for cancelled Work, or the unique minimum active identity. Valid references and terminal exclusion prevent ambiguity. Hence SingleEffectiveClaim.
Authorization and effects
WorkerOrigin restricts a fresh worker append to its own Completion or ClaimCancellation. A newly committed Completion requires finalize. A terminal/no-op candidate moves to stopped, not committed.
Inbound is immutable, and Completion(a) is one fixed fact for worker a, so SingleCompletionPerWorker follows. Workers cannot return to preparation after committing; finished, stopped, and failed phases have no outgoing restart transition. Inspection of every action and stuttering proves WorkerOneShot. Reauthorization is therefore unreachable, not an additional failure scenario.
Only VerifyCompletion extends authorization history. It requires the attempt's committed phase and its exact durable Completion. The phase becomes authorized; it cannot return to committed. A second attempt for the same Claim cannot newly append Completion to terminal Work. Thus authorization records are durable, finalize-bound, and unique per attempt.
ValidLog allows only one Completion record per Work. Since an authorization contains that exact attempt-bound record and each attempt authorizes at most once, SingleAuthorization follows.
Only ExternalEffect extends effect history. It requires authorized, then moves irrevocably to done. The effect is backed by its authorization and is emitted at most once per attempt. Together with SingleAuthorization, this proves SingleEffect.
Remaining actions
Agent staging changes only equal local/serialized logs. Only Stage extends these sequences, and its Claim guard selects OldestAvailable from the preceding local view; every other action leaves them unchanged. Thus QueueSelection holds without imposing a publication-order invariant. Activation snapshot capture and admission do not grant authorization; execution-time worker preparation rechecks current Git facts. Retry exhaustion changes phase to failed without writing or emitting effects. Run termination can stop a worker but neither invents nor removes a terminal fact or authorization. Maintenance uses the same guarded write lemmas. Every action in Next, and stuttering, therefore preserves the strengthened Safety predicate.
By induction on execution length, Spec => []Safety follows under the modeled assumptions. This is a reviewable proof argument, not a TLAPS proof certificate.
Reproduce verification
The 2026-10-07 source-bound review record
contains exact model/configuration/runner hashes, tool identity, bounded-run
verdicts and trace counts. The initial review ran all 60 registered configurations:
14 positive searches exhausted; all 33 negative controls and nine guarded
witnesses produced their exact expected outcome; four searches were interrupted
at explicit local resource budgets. WorkQueue, QueueOrdering,
FairDAGFork and FairDAGGitHub were not passes in that initial run.
An independent longer FairDAGFork rerun subsequently exhausted 70,602 states
at depth 28 in 6m47s. It did not resume or add the earlier partial search.
That review's per-configuration result was 15 exhausted searches,
42 exact expected controls/witnesses and three unfinished searches:
WorkQueue, QueueOrdering and FairDAGGitHub.
The separate 2026-10-07 refinement record
supersedes that status without rewriting the earlier capture. The registered
suite now includes the projection-corruption control: 16 positive searches
exhausted, 34 negative controls and nine guarded witnesses matched their exact
expected outcomes, and two searches remain unfinished. FairDAGGitHub
exhausted 1,055,182 distinct states at depth 20 in 8m39s, with zero states
remaining and its original bounds, constraints and safety conjuncts retained.
TLC uses 64-bit fingerprints; that run reports optimistic collision probability
5.2E-7 and actual-fingerprint estimate 3.2E-8, not mathematical certainty.
Both historical searches successfully restored complete, pinned local TLC
checkpoints and continued for another 1,200 seconds. WorkQueue reached
38,640,894 distinct states at depth 22 with 7,079,332 queued; QueueOrdering
reached 51,756,372 at depth 17 with 23,003,758 queued. Neither exhausted or
passed. These are actual continuations, not sums of independent searches.
Checkpoint recovery rolls back to the saved checkpoint, which can precede the
last interrupted progress report. Local restore succeeded; portable checkpoint
archives remain unvalidated. The earlier nominal 1,800-second searches actually
ran about 2,226–2,227 seconds; the record preserves both budget and measured time.
The refined sources also produced 18 seeded simulations with 309 sampled states
and nine exact guarded witness traces. Their model/configuration hashes still
match after merging main at 303b4028106137af47924a4bc079b8d3fdd8884c.
Earlier counts below remain historical evidence. Neither finite model checking
nor sampled traces verify Go/JavaScript runtime or supported-host refinement.
Evaluation refinements without state-space reduction
WorkQueue now dispatches each actor's actions by its existing phase guard.
Every action retains its original guard and update; the CASE branches merely
avoid evaluating actions whose first guard is false. Run termination and
duplicate-record publication remain separate alternatives. No variable,
transition, bound, safety invariant or action property is removed.
In FairWorkQueue default mode, Class(w) and Key(w) are always 1.
Consequently active classes/keys are either empty or {1}, and selecting among
them cannot depend on their pass values. NextWork directly chooses the same
oldest eligible Work. Normalize retains the exact class-1/key-1 wake-up clocks,
inactive clocks and active-set updates using the simplified formulas.
Weighted/strict selection is unchanged. The recursive DAG closure also
evaluates its preceding finite set before reusing it; TLCEval(x) = x.
There is no VIEW, symmetry reduction or extra exploration constraint.
projection is a deterministic, non-authoritative cache: initialization
sets it to Replay(log), every nonstuttering action sets it to Replay(log'),
and ProjectionSoundness independently checks complete replay equality in
every state. Safety retains every preceding conjunct and adds this check.
Normal, guarded-witness and deliberately broken protocol transitions all
update the cache; a separate corruption control bypasses that update and must
violate ProjectionSoundness. Scheduling, observations and accounting remain
solely in log. This adds a functional state component, not an independent
choice or a reduced state space. It avoids replaying the same log separately
for each guard and projected-state invariant.
compare-evaluation.mjs independently instantiates
the original and revised modules over shared protocol variables/constants,
lifting the original relation with the same replay-derived cache update. It explores
the union of their transitions and checks NextEquivalence on each edge,
so a missing revised transition cannot disappear from the comparison. Both
initial-state predicates and safety predicates must agree/hold; fair-model
replay projections must be exactly equal, including accounting clocks.
The five comparison configurations cover historical recovery, default batches,
weighted/strict selection and typed external gates. Missing-transition and
changed-clock mutations must produce the exact action-property/invariant
diagnostics. Removing the deterministic cache maps each revised state to
exactly one original state; complete replay reconstructs it uniquely.
These finite comparisons supplement this extension/algebraic equivalence
argument. They are separate from the larger searches above and do not prove
runtime refinement. All five comparisons and both exact mutation controls
passed; the default FairBatch graph still has 13,662 distinct states at depth 20.
To reproduce the comparison against the last unrefined source:
baseline="$(mktemp -d)"
git archive 4ae415cc7046e215b6056fed90c26e4b915ef06b specs/work-queue |
tar -x -C "$baseline"
export JAVA_BIN=/path/to/java
export TLA2TOOLS_JAR=/absolute/path/to/tla2tools.jar
node specs/work-queue/compare-evaluation.mjs \
"$baseline/specs/work-queue" \
"$PWD/.queue-validation-cache/evaluation-comparison"
The comparison rejects changed configurations, an unexpected TLC checksum, timeouts, tooling failures and unexpected mutation diagnostics. Its manifest is a pass only when every case finished. Use a fresh results directory rather than overwriting an earlier capture.
The long-run collector .github/scripts/work-queue-formal-check.cjs accepts
FORMAL_CONFIG=WorkQueue, QueueOrdering or FairDAGGitHub. It executes the
model/configuration copies in its evidence bundle, not live checkout files,
so concurrent source edits cannot change the inputs after their hashes are
recorded. RESULTS_DIR and TLA2TOOLS_JAR must be absolute paths. A natural
successful exit with an empty remaining queue is required for a pass; a timeout
or unvalidated checkpoint never establishes exhaustion or resumability.
Successor: mandatory fair scheduling and batched workers
The priority/fairness specification defines the
replacement protocol. FairWorkQueue.tla models its causal
atomic Claim batches, FIFO defaults, two-level integer-pass selection, fresh
selection after CAS conflicts, request deduplication, and independent Claim
completion/effect authorization within one worker assignment.
Each Claim consumes a logical slot and one service charge. A worker group consumes one native slot until trusted termination/release; completing one member does not release that slot. Missing finish intent cancels only that member. Published Completions survive recovery, and effects require the same assigned Claim's authorization.
The model has first-class immutable Work dependency sets and normalized external
Issue/PR gates. Only a verified-delivery Result releases a Work successor;
Completion alone does not. Unknown/unavailable external observations block
admission, and a closed-unmerged PR does not satisfy its merged predicate.
It abstracts a fixed policy epoch, one compatible worker profile, one request/group per dispatcher, native run identity as its group identity, and canonical causal log order. Work is seeded in the first commit; immutable graph edges and resolved resource identities are model parameters representing its admitted metadata. External resources abstract an authorized foreign Issue and PR as typed host/repository/number/identity references, with at most two observations each. Weighted cases abstract two classes and two accounting keys with 2:1 shares; default mode collapses both. Effect entry and trusted delivery verification are separate observer steps.
It does not model dynamic graph admission, multi-profile packing, policy edits,
idempotency fingerprints/replayed transport responses, actual GitHub credentials,
resource IDs, observation freshness, artifact delivery, JSON codecs, OTLP delivery,
or unbounded fairness/liveness. Work positions are causal FIFO positions.
The protocol review revision
additionally specifies selection-before-packing, authenticated activation
binding recovery, terminal DeliveryFailure/replacement rules, operational
Control, batch trust domains, and replay/resource budgets. These are acceptance
obligations, not behavior added to or verified by this model. In particular,
DecisionValidity checks conformance to NextWork; it is not an independent
proportional-service or eventual-grant assertion. The dedicated fairness release
gate is specified in
the validation plan.
The preceding WorkQueue.tla checks
remain regression evidence for the existing implementation, not an operational
legacy mode in the replacement.
For a focused successor check, set TLC_MODEL_FILTER=FairWorkQueue. Omit this
filter to run the complete original, successor, and Claim-scope suite. Set
TLC_CONFIG_FILTER=FairBatch to run one named configuration; unknown or
incompatible filters fail rather than returning a no-op success.
The same check.sh additionally runs:
| Configuration | Expected result |
|---|---|
FairBatch.cfg |
Two dispatchers compete for a two-Claim worker batch; safety, independent logical/native limits, and per-Claim authorization hold |
FairThreeClaim.cfg |
One worker receives three Claims and independently completes, cancels, or omits members across crash/recovery interleavings |
FairPriority.cfg |
Weighted class/key decisions and Claim limits hold |
FairStrict.cfg |
Strict class selection with weighted keys holds |
FairDAGChain.cfg |
A dependent is admitted only after its predecessor's verified Result |
FairDAGForward.cfg |
An older child referencing a later-numbered parent waits while that ready parent runs |
FairDAGFork.cfg |
Siblings become independently schedulable after their common predecessor succeeds |
FairDAGGitHub.cfg |
A child needs its Work predecessor and both foreign Issue/PR conditions; bounded to one observation/resource plus checked boundary successors |
FairGitHubDependencies.cfg |
Both Issue-completed and PR-merged observations must satisfy the external gate |
BrokenProjection.cfg |
A cache-only forged completion must violate complete log-replay equality |
BrokenBatchSelection.cfg |
Deliberately bypass selection; DecisionValidity must fail |
BrokenClaimEffects.cfg |
Use another unfinished Claim's effects after one Claim completes; EffectAuthorization must fail |
BrokenAssignmentHandle.cfg |
Finish another worker's Claim; ClaimClosureAuthority must fail |
BrokenBatchCAS.cfg |
Overwrite from a stale batch snapshot after Completion; TerminalPersistence must fail |
BrokenBatchRelease.cfg |
Release a live group's native slot after only some Claims finish; RunReleaseAuthority must fail |
BrokenDAGDependency.cfg |
Claim a not-ready successor; DependencyAuthorization must fail |
BrokenDAGResult.cfg |
Publish a Result before delivery verification; ResultEffectSoundness must fail |
BrokenDAGCycle.cfg |
Cyclic constant graph is rejected during setup with exit 151 and the exact DAGValidity-is-FALSE diagnostic, not a parser failure |
BrokenExternalDependency.cfg |
Admit Work with unknown/unsatisfied external gates; ExternalAuthorization must fail |
BrokenPRClosedAsMerged.cfg |
Treat a closed-unmerged PR as ready; ExternalTruth must fail |
BatchedAssignmentWitness.cfg |
Guarded model reaches a multi-Claim assignment; the deliberately false NoBatchedAssignment fails |
PartialCompletionWitness.cfg |
Guarded model reaches one completed and one open member; the deliberately false NoPartialCompletion fails |
DAGJoinWitness.cfg |
A success-path subset of guarded actions, with fixed root/sibling/join dispatcher roles, reaches the join after both Results; false NoJoinClaim fails |
Before the DAG extension, successor checks completed on 2026-10-05 at commit
f50550e8dc with the pinned TLC 2.19/Java 21:
FairBatch exhausted 4,902 distinct states at depth 17; FairThreeClaim exhausted
4,144 at depth 17; FairPriority and FairStrict each exhausted 283 at depth 14.
All five negative controls produced
their named violations; the guarded batching and partial-completion witnesses
produced their expected counterexamples without a safety failure. These are
bounded model checks, not an unbounded proof or runtime conformance result.
The current DAG/result/Issue/PR cases exhaust the following positive state spaces:
| Configuration | Distinct states | Graph depth |
|---|---|---|
FairBatch |
13,662 | 20 |
FairThreeClaim |
34,075 | 23 |
FairPriority |
675 | 17 |
FairStrict |
675 | 17 |
FairDAGChain |
13,846 | 24 |
FairDAGForward |
13,846 | 24 |
FairDAGFork |
70,602 | 28 |
FairGitHubDependencies |
9,749 | 14 |
Total exhausted: 157,130 distinct positive states. The ten negative controls require nine
named invariant violations and one exact constant-cycle setup rejection. The
three reachability witnesses demonstrate batching, partial completion, and a
guarded diamond join. JoinWitnessSpec restricts to successful root/sibling/join
roles while using only guarded actions; it proves reachability, not exhaustive
coverage of all diamond failure/retry interleavings.
Full-suite verification is incomplete. Two additional exhaustive searches were stopped without reaching exhaustion on 2026-10-05:
| Configuration | Distinct states explored | Last depth | Result |
|---|---|---|---|
FairDAGGitHub |
588,795 | 14 | No violation observed; not a passed/exhausted check |
Existing QueueOrdering |
109,949,148 | 18 | No violation observed; not a passed/exhausted check |
These state-space-heavy cases remain in check.sh; their bounds were not reduced
to manufacture a pass. Checkpoints/logs are retained in the session artifacts.
The other original checks completed, including WorkQueue (59,444,361 distinct
states, depth 35, resumed from its checkpoint), Recovery (147,542, depth 20),
the original negative controls/witness, and traces.sh. Passing successor
checks do not substitute for the two incomplete searches.
Re-run a named case with TLC_CONFIG_FILTER. To resume a retained TLC checkpoint,
use the same jar/model/configuration and worker count with
-recover /path/to/checkpoint-directory; the checks here used two workers.
Claim-scoped safe outputs and mixed DAG outcomes
ClaimScopedWorker.tla separately formalizes
enforced output attribution
and mixed outcomes.
Its fixed trusted assignment contains one or three Claims. Omitted selectors
normalize automatically only for an originally one-Claim assignment; explicit
foreign handles and missing multi-Claim selectors are rejected. A multi-Claim
assignment never becomes implicitly single-Claim when other members close.
flowchart LR
Single["One-Claim assignment"] --> Auto["Omitted selector: attach sole handle"]
Multi["Multi-Claim assignment"] --> Explicit["Explicit member handle required"]
Auto --> Scope["Canonical Claim-scoped output"]
Explicit --> Scope
Scope --> Finished["Same Claim's verified Completion"]
Finished --> Effects["Authorized effects and verified Result"]
Cancelled["Cancelled Claim"] --> NoEffects["No effects or Result from this attempt"]
Effects --> DAG["Only declared Result-dependent successors become ready"]
All closures and Results are facts in one abstract log. Safe outputs preserve
their resolved Claim and original selector; effects require that same Claim's
Completion. Cancelled Claims cannot emit effects or Results. The mixed witness
finishes Claims 1/3, cancels Claim 2 despite its staged output, and admits the
successor requiring Results 1/3 while the Claim-2-dependent successor waits.
Cancellation is not batch failure or terminal WorkCancellation; the existing
FairWorkQueue model separately covers returning that Work to availability.
Run this focused suite with TLC_MODEL_FILTER=ClaimScopedWorker, or select a
single configuration with TLC_CONFIG_FILTER.
| Configuration | Expected result |
|---|---|
ClaimScopeSingle.cfg |
Automatic/explicit one-Claim attribution, foreign-selector rejection, closure/effect/Result safety |
ClaimScopeMixed.cfg |
Three independently completed/cancelled/omitted Claims; scoped outputs and Result-only DAG admission |
BrokenMissingClaimScope.cfg |
Assign an omitted multi-Claim selector to the first member; OutputScope fails |
BrokenForeignClaimScope.cfg |
Replace a foreign explicit selector with the sole handle; OutputScope fails |
BrokenLastOpenClaimScope.cfg |
Infer an omitted selector from the last open member of a multi-Claim assignment; OutputScope fails |
BrokenCancelledClaimOutput.cfg |
Execute a cancelled member's staged output; EffectAuthorization fails |
BrokenMixedDAGAdmission.cfg |
Use Claim closure instead of verified Results to admit a successor; DAGAuthorization fails |
SingleClaimScopeWitness.cfg |
Guarded model reaches a sole-Claim effect with automatically attached scope; false NoAutomaticScope fails without violating Safety |
MixedClaimDAGWitness.cfg |
Guarded model reaches finished/cancelled/finished members, scoped effects only for 1/3, and the corresponding ready successor; false NoMixedDAGProgress fails without violating Safety |
The two positive cases exhaust 144 and 120,832 distinct states respectively (depths 12 and 25), totaling 120,976 on 2026-10-05 with the pinned TLC/Java setup. The controls/witnesses require their exact named violations, not parser/tool errors. These checks do not substitute for the two unfinished searches above.
The model abstracts an already validated assignment/native binding, one required scoped handler batch per root, trusted delivery verification, and two fixed successor dependency sets. It does not model concrete output schemas, every handler type, no-write task contracts, temporary-ID codecs, Git CAS, retries in a new assignment, actual API delivery, fair scheduling, or liveness. Handler-wide enforcement remains a replacement runtime conformance obligation, not a deployed feature established by this model.
Independent recurring service evidence
QueueService.tla checks one sibling competition set with
exact strides for weights 5:3:2, independently of the queue lifecycle model.
Its count invariants do not compare a selector with itself: they check a
three-grant prefix-deviation bound and exact shares over each ten-grant cycle.
At a virtual threshold, each fixed-set stream differs from its ideal event count
by at most one; summing those stream errors gives the stated conservative
three-competitor bound. This justification and the finite checks apply to the
stated fixed set, not arbitrary dynamic fairness intervals.
The dynamic case allows arbitrary membership changes and checks eventual grants for continuously eligible competitors under weak fairness of packable Grant. Each competitor's service bit must recurrently take both values; a stale last winner cannot satisfy the property without new grants. Relative pass clocks and discarded inactive past debt form a finite quotient of the anti-idle-credit rule. Inactive future debt remains; reactivation uses the specified clamp.
| Configuration | Expected result | Evidence |
|---|---|---|
ServiceFixed |
Share/deviation invariants and eventual service pass | 30 exhausted states, depth 30 |
ServiceDynamic |
Eventual service under eligibility changes passes | 60,360 exhausted states, depth 26 |
BrokenServiceShare |
ServiceDeviation violation |
Exact invariant diagnostic, exit 12 |
StrictServiceStarvation |
Lower-priority eventual service fails | Exact temporal diagnostic, exit 13 |
UnfairServiceStarvation |
Eventual service without fair opportunities fails | Exact temporal diagnostic, exit 13 |
UnpackableServiceStarvation |
Eligibility without packable opportunities fails | Exact temporal diagnostic, exit 13 |
flowchart LR
Eligible["Continuously eligible competitor"] --> Opportunity{"Packable recurring Grant?"}
Opportunity -->|No| NoPromise["No eventual-service guarantee"]
Opportunity -->|Yes| Policy{"Weighted positive shares?"}
Policy -->|Strict lower class| Starvation["Starvation is permitted"]
Policy -->|Yes, with service fairness| Service["Bounded model checks recurring grants"]
Reproduce this model alone with TLC_MODEL_FILTER=QueueService, using the Java
and jar settings in the reproduction instructions. Full class/key composition,
arbitrary weights, native publication/release and runtime refinement remain
separate obligations. The new checks do not finish FairDAGGitHub or
QueueOrdering, and the earlier interrupted dynamic exploration is not added to
these exhausted counts.
Bounded launch, delivery and recovery evidence
QueueLifecycle.tla separates launch-boundary safety from
fairness arithmetic. Its independent FIFO packing oracle uses profile order
X/Y/X: one available dispatch admits the first X only; two dispatches permit
X/X and Y without skipping the middle winner. Immutable assignments, one
selected sender, activation recovery, conflicting bindings, attempt-1 effects,
Controls, logical/native release, delivery outcomes and replacement admission
are checked together.
| Configuration | Expected result | Exhausted evidence |
|---|---|---|
LifecycleRecurring |
Safety across two fully drained epochs | 496 states, depth 20 |
LifecyclePacked |
Safety for the bounded packed assignment | 207,612 states, depth 33 |
Eleven negative controls require the exact named violations for fair-prefix packing, one sender, binding authority, unauthenticated activation, reruns, premature release, conflict release, pause/preemption confusion, delivery exclusivity, replacement safety and delivery-verification budgets. Three guarded witnesses reach mixed DAG outcomes, recovered activation and retained capacity after conflicting activation. Witness violations are expected reachability diagnostics, not failures of the guarded safety properties.
flowchart LR
Prefix["Independent X/Y/X packing oracle"] --> Assignment["Immutable group membership"]
Assignment --> Marker["One sender marker"]
Marker --> Unknown["Launch uncertain: retain native slot"]
Unknown --> Activation["Checked activation recovers binding"]
Activation --> Members["Independent Claim closures and delivery barriers"]
Activation --> Conflict["Conflicting run: retain reservation"]
Members --> Terminal["Exact terminal evidence releases shared slot"]
Members --> DAG["Only verified Results unblock Work successors"]
Reproduce these checks with TLC_MODEL_FILTER=QueueLifecycle. The positive
total is 208,108 exhausted states, not an unbounded proof or runtime test.
Terminal, activation and receipt predicates abstract independently checked
evidence; they do not verify real GitHub responses, credentials, artifact
authenticity or resource targets. The model does not cover actual Git CAS,
wire codecs, arbitrary graphs, the native scheduler, all handler types or a
liveness theorem. These checks do not complete the two interrupted large
searches or the supported-host integration gate.
Daily evidence collection
daily-work-queue-formal-verification.md
collects FairDAGGitHub and QueueOrdering on separate parallel runners daily,
with manual dispatch available. Each verification job has a five-hour limit.
Its deterministic collector interrupts TLC after 4h40m, preserving time to
finalize and upload the artifact instead of losing it to a job timeout.
Artifacts are retained for 30 days and named
work-queue-formal-<configuration>-<run-id>-<attempt>. They contain exact
model/config snapshots and hashes, Java/TLC provenance, command, raw log,
normalized verdict/counts, and checkpoint inventory. A full state archive is
included only when a checkpoint diagnostic and basic file presence checks pass
and the state fits the 256 MiB uncompressed cap; otherwise the explicit omission
reason is retained. Large state directories
are never uploaded indiscriminately.
A read-only agent writes a separate work-queue-formal-handoff-<run-id>-<attempt>
artifact for subsequent agents. It does not rerun the checker, modify the model,
or reinterpret timeouts/partial searches as successful verification. Setup
failures keep an explicit setup_incomplete record; artifact availability
remains visible even if agent analysis fails.
GitHub resource safe outputs are preview-only; the deterministic verification
and handoff artifact uploads remain real.
Each daily run currently starts fresh; it does not resume a prior artifact.
Repeated timeouts on identical sources do not accumulate verification coverage
or prove exhaustion. Archives are labeled unvalidated_checkpoint_candidate,
with resumable: false and recovery_validation: "not_attempted". Basic file
presence and packaging do not validate model/worker checkpoint completeness or
actual restore capability. The collector step uses continue-on-error:
artifact collection/workflow success must be
distinguished from the result.json verification verdict. The five-hour limit
applies to each matrix job, not the whole workflow including later analysis.
See the evidence contract
for follow-up recovery and failure-signaling requirements.
Safety includes state types, single open owner per Work, causal history,
recomputed selection, logical/native capacities, exact Claim charge counts,
one dispatch request per group, assignment integrity, one effect authorization
per Claim, per-handle closure and run-release authority, terminal persistence, and independent
FIFO/strict-priority assertions. The DAG extension additionally checks graph
acyclicity, readiness at each Claim's causal prefix, Result authority/verified
delivery, and external condition truth/admission. Positive cases exhaust their bounded state
space; a trace showing partial completion is reachability evidence, not a proof
that arbitrary partial completions eventually happen.
Use Java 21 and the official tla2tools.jar v1.7.4, which reports TLC 2.19. Jar SHA-256:
936a262061c914694dfd669a543be24573c45d5aa0ff20a8b96b23d01e050e88
TLA2TOOLS_JAR=/path/to/tla2tools.jar \
JAVA_BIN=/path/to/java \
bash specs/work-queue/check.sh
The runner checks every registered configuration across the five models,
including the historical configurations below. Positive configurations require
exhaustive successful termination; negative controls and guarded witnesses
require their exact named diagnostic and exit status, not a parse/tooling failure.
Use TLC_MODEL_FILTER and TLC_CONFIG_FILTER to select a subset; unmatched
filters fail. Full reports are saved under a printed temporary path; set
TLC_RESULTS_DIR to retain them at a chosen location. The historical controls
below deliberately bypass branch-version, terminal-state or selection protection;
those are not reachable behaviors of the guarded historical protocol.
| Configuration | Scope / expected result |
|---|---|
WorkQueue.cfg |
One Work, two competing Claims, two dispatchers, three workers (two share a Claim), three branch changes, four records; Safety, WorkerOneShot, QueueSelection, and WorkResubmissionNoOp hold. |
Recovery.cfg |
One Work, two Claims/workers, one dispatcher, four branch changes, five records; Safety holds through recovery/compaction interleavings. |
QueueOrdering.cfg |
Two Work items, two Claims/workers/dispatchers, two branch changes, four records; Safety, WorkerOneShot, and QueueSelection hold across selection, retry, cancellation, and compaction interleavings. |
BrokenCAS.cfg |
Bypass the branch-version check and overwrite with a stale snapshot; TerminalPersistence fails. |
BrokenTerminal.cfg |
Bypass terminal protection and append a competing Claim after Completion; TerminalFreeze fails. |
BrokenQueueSelection.cfg |
Bypass oldest-available selection while retaining publication safety; QueueSelection fails. |
WeakOrderingWitness.cfg |
Under guarded Spec, a newer Work is claimed before an older submission becomes visible; the deliberately false NoOutOfOrderClaim invariant fails. |
Earlier records reported three positive searches on 2026-10-02, checking
Safety, WorkerOneShot and QueueSelection: 7,218,153 distinct states at graph
depth 33 for concurrency, 49,087 at depth 19 for recovery, and 1,696,812 at depth
24 for two-Work ordering. These retained historical counts are not a fresh,
source-bound verification verdict for the current checkout. The branch-version,
terminal-state and queue-selection controls reported expected violations at
depths 7, 10 and 4; the guarded weak-ordering witness reported an out-of-order
Claim at depth 6. Use the dated current-source review evidence, not these older
counts, to identify exhausted versus unfinished searches.
Bound constrains branch changes and physical log size, not execution depth. TLC also checks immediate successor states before pruning them. Deadlock checking is disabled because stopped/failed workflows are intentional; no fairness or liveness theorem is asserted.
Inspect execution traces
Generate bounded textual simulations of six configurations and nine reachable
counterexamples to deliberately false witness invariants. WorkQueue is
historical; the other models abstract parts of the current protocol:
TLA2TOOLS_JAR=/path/to/tla2tools.jar \
JAVA_BIN=/path/to/java \
TLC_TRACE_DEPTH=16 TLC_TRACE_COUNT=3 \
bash specs/work-queue/traces.sh
The script prints a temporary results directory (or uses TLC_RESULTS_DIR when
set). simulation_* files are historical WorkQueue traces; current abstraction
traces use prefixes FairBatch, FairDAGGitHub, ClaimScopeMixed,
ServiceDynamic and LifecyclePacked. Each has at most TLC_TRACE_DEPTH states;
TLC_TRACE_COUNT sets the number of seeded random simulations per configuration.
The runner requires emitted traces and successful TLC termination, records
Java identity/settings and SHA-256 hashes, and rejects source/tool drift.
These samples check configured invariants along the emitted trajectories. They
are not exhaustive coverage, a liveness proof, native queue replay, or guaranteed
occurrences of particular actions. In the 2026-10-07 run with depth 32 and count
3, all 18 traces were emitted: 272 sampled states across 15 current-abstraction
traces and 31 across three historical traces. All nine guarded witnesses produced
their exact expected diagnostic; no unexpected safety or tooling failure was
accepted. sources-before.sha256 and sources-after.sha256 bind this evidence
to unchanged model/configuration/runner sources and the TLC jar.
The root *Witness.log files contain the four historical model-checked
counterexamples below. Current guarded witnesses are in current-witnesses/:
| Configuration | Reachable current behavior |
|---|---|
PartialCompletionWitness |
One immutable assignment member completes while another remains open |
MixedClaimDAGWitness |
Per-Claim success/failure independently permits or blocks DAG progress |
LifecycleMixedDAGWitness |
Mixed lifecycle and delivery outcomes settle independent successors |
LifecycleActivationWitness |
Checked activation recovers an uncertain launch binding |
LifecycleConflictWitness |
Conflicting activation retains the native reservation |
Each witness configuration also checks Safety; the script accepts only its
named witness violation, not a safety violation or a TLC failure.
| Witness | What to inspect in its counterexample |
|---|---|
NoCompetingClaims |
Two dispatchers select the same Work in their local views before either Claim is durable. Both Claims persist, but replay still selects one effective winner. |
NoRecoveredOrphan |
A run terminates, then recovery prepares and publishes its ClaimCancellation. |
NoExternalEffect |
A worker finalizes, commits Completion, verifies it, and enters its output batch. |
NoOutOfOrderClaim |
A dispatcher selects newer Work from its local view before staging an older submission; publication preserves the Claim rather than imposing strict FIFO. |
These invariants are intentionally not protocol requirements: their violation demonstrates reachability of legitimate behavior under each guarded Spec. Unlike the deliberately unsafe Broken*.cfg configurations, the witness configurations do not add unsafe actions. Inspect the preceding states, not just the final state, to verify the ordering claimed by the ADR. Witnesses show that the model permits these paths; they do not establish that a runtime implementation follows them or that progress is guaranteed.
Runtime smoke coverage
The private .github/workflows/smoke-work-queue.md workflow is a read-only
observer. It exercises snapshot read/explain tools through the compiled MCP
mount and verifies that unassigned finish is unavailable or rejected without
recording an intent. Its verification job rejects a finish artifact or failure
report and requires a noop result. Ordinary reports use the observer's normal
workflow authorization; they do not acquire worker or queue-control authority.
This smoke test checks observer wiring and unassigned-finish rejection. It does not establish dispatcher submission, native binding, verified worker delivery, recovery, or compaction.
Limits
Safety does not imply eventual dispatch, successful external effects, or eventual orphan recovery. Those require fairness, available workflows, and successful retries. A crash after Completion but before outputs can leave delivery uncertain; the current runtime retains independent barriers and reconciles Result or DeliveryFailure instead of replaying completed effects. This is not atomic or exactly-once delivery. The activation artifact is a snapshot, not a live subscription; reads can be stale and are never used as final authority.
The independent ESLint factory model compares the configured factory with executable observations of its real sources and ESLint behavior. It separates runtime enforcement from prompt assumptions and external queue installation; it is not proof that the factory automatically admits an Issue-to-Work DAG or that arbitrary agents follow their prompts.
The model does not prove GitHub authentication, aw_context provenance validation, payload canonicalization, parser behavior, transport errors, or the implementation's refinement of these abstract actions. These remain implementation obligations. It does not make arbitrary external API batches atomic or exactly-once.
Directories
¶
| Path | Synopsis |
|---|---|
|
Command native_probe is a local verification adapter, not a workflow helper.
|
Command native_probe is a local verification adapter, not a workflow helper. |