ouroboros

What it is

CSF runs AI coding agents under rules: every agent works in a harness session, and every shell command and commit it makes passes a gate. Gates are only as good as the mistakes someone thought of in advance. Ouroboros finds the ones nobody did.

Ouroboros uses evaluated results to propose the next version of an agent's instructions or an allowed workflow; RRSI is its method.

That sentence is the ontology's definition, unchanged. This directory is its mining half. A miner reads a corpus (harness event logs, tickets, pull requests, files at a revision), turns it into facts, and derives verdicts with a Datalog rule. A miner is accepted only when a walk-forward backtest finds every labeled offense it should, and an accepted miner becomes the next gate. The gated runs then become the corpus for the next miner: the snake eats its tail.

You write two files, extract.ml and rules.dl, on a contract that is already built.

What you can do with it

• Turn a complaint into a gate. A mistake the operator flags becomes labeled instances; a miner that finds every one of them, and nothing clean, closes the ticket.
• Prove a rule before you enforce it. Thresholds are fitted on the earlier half of the record and judged on the later half, so a miner cannot pass by memorizing its examples.
• See the evidence behind every finding. Each verdict carries its proof: the facts it used and the source line of each.
• Measure how early a rule would have fired. The backtest reports lead time: how long before the operator noticed, the miner would have.
• Close tickets only on proof. tools/close-ticket.sh closes a ticket only when the merged pull request's backtest shows no missed offense.

The loop

Figure 1 shows one turn of the loop.

Figure 1. The Ouroboros loop. A miner extracts facts from the corpus, a Datalog rule derives verdicts with proofs, and a walk-forward backtest decides: a miner with no missed offense becomes a typed finding that can close its ticket, and the next runs become the next corpus.

Each arrow is one typed artifact, and each has a check: extract.ml and rules.dl must pass miner_test, the backtest block is generated (never typed), and tools/close-ticket.sh closes a ticket only when the merged pull request's Backtest FN row is 0 and every check row exits 0.

Quick start

Copy the template and give it your miner's name (in extract.ml, set name, package and verdicts):

cp -r services/ouroboros/miners/_template services/ouroboros/miners/my_miner

Test, build and run the template as it ships. This is real output from this branch, with Bazel's progress lines, the findings' proof arrays and the backtest's first rows trimmed (…):

$ tools/bazel.sh test //services/ouroboros/miners/_template:miner_test --test_output=all
…
draft-pr-late: 4 tests passed
…
//services/ouroboros/miners/_template:miner_test                (cached) PASSED in 0.0s

$ tools/bazel.sh build //services/ouroboros/miners/_template:miner
INFO: Build completed successfully, 4 total actions

$ bazel-bin/services/ouroboros/miners/_template/miner.exe findings --knee 468 'services/ouroboros/miners/_template/fixtures/*/events.jsonl'
{"miner":"draft-pr-late","rule":"invisible","subject":[{"text":"6c1f2a10-3b7e-4d6a-9e51-0a1b2c3d4e31"}],"severity":"SEVERITY_S3","proof":[…]}
{"miner":"draft-pr-late","rule":"invisible","subject":[{"text":"6c1f2a10-3b7e-4d6a-9e51-0a1b2c3d4e60"}],"severity":"SEVERITY_S3","proof":[…]}

$ bazel-bin/services/ouroboros/miners/_template/miner.exe backtest services/ouroboros/miners/_template/fixtures/labels.tsv 'services/ouroboros/miners/_template/fixtures/*/events.jsonl'
…
| Backtest $\kappa^\star$ | 468 (argmax precision on $\Lambda_{\le t}$ s.t. FN = 0; 3 candidates from `score`) |
| Backtest TP | 1: 6c1f2a10-3b7e-4d6a-9e51-0a1b2c3d4e60 |
| Backtest FP | 0 |
| Backtest FN | 0 |

The miner's verbs are facts, findings [--knee K], backtest [--json] LABELS ITEM... mutate [--json] LABELS ITEM... and scope. Swap the fixture glob for '<state>/*/events.jsonl', where <state> is the harness state directory, to run over every recorded run.

Worked example: DRAFT-PR-LATE

The template flags a harness run whose work stayed invisible to the operator, with no draft pull request, for longer than a knee measured from the runs themselves (#102). Its fixtures are three real runs.

Facts (miner.exe facts on the fixtures; score is seconds from start to the first draft pull request, or to the last event when there is none):

Rule (rules.dl):

invisible(R) :- gated(R), score(R, S), knee(K), gt(S, K).

Labels (fixtures/labels.tsv), ordered by start and split at $t$ = 2026-10-02T18:04:45Z:

Knee. The candidates are the measured scores, $K = {468, 1259, 3089}$, and the fit half chooses among them:

$$\kappa^\star = \arg\max_{\kappa \in K} \mathrm{precision}(\kappa; \Lambda_{\le t}) \ \text{s.t.}\ \mathrm{FN}(\kappa; \Lambda_{\le t}) = 0$$

Ties keep the smallest knee, which fires earliest, so $\kappa^\star = 468$.

Backtest (miners/_template/backtest.md, generated, and checked current by miner_test):

Lead is $t_{\mathrm{flag}} - (t_{\mathrm{start}} + \kappa^\star) = 772 - 468 = 304$ s: the miner would have fired five minutes before the operator flagged 4e60. The finding carries its proof, gated(4e60), score(4e60, 1259) and knee(468), each with its source line.

Writing a miner

1. Read the ticket and every comment: its named instances are your labels and its predicate is your rule.
2. Copy miners/_template to miners/<name>; in extract.ml set name, package and verdicts.
3. Write extract: one corpus item in, facts out, each with its source span. Emit a score fact if the rule needs a knee.
4. Write rules.dl with its /* predicate */ block; thresholds read knee(K).
5. Put each labeled instance in fixtures/<instance>/events.jsonl (only the lines a fact needs) and list it in fixtures/labels.tsv: instance, + or -, start, flagged time or -, source.
6. Run tools/bazel.sh test //services/ouroboros/miners/<name>:miner_test until it passes; paste the blocks it prints into backtest.md and mutation.md. A surviving mutant is printed with the rules or labels that survived: strengthen the fixtures or the rule until the test kills it.
7. Write the README from the template's, with the agent-drafted marker and the backtest block.
8. Run python3 tools/check_operator_identifiers.py, commit with the CSF-Session and CSF-Model trailers, push, and open a draft pull request.
9. Run bash tools/check-merge.sh; on exit 0 mark the pull request ready. Never merge.

Pitfalls

Each row is a mistake seen in tonight's proposals or in the template's own history.

The engine checks neither safety nor stratification, so Contract.check_rules does: an unbound negation raises Invalid_rules "negation before its variables are bound", and every miner's test runs it first.

The service

The Go package in this directory is the loop itself, mounted into csf serve (#120): miners run all the time over the harness corpus, a pre-check decides at no model cost which tickets a fixer can take, fixers run through the harness under a daily budget, the merge train merges what passes the gate, and the compounding rate is measured and shown on the ops view. It replaces the ad hoc scripts that launched fixers outside the session gates and receipts.

csf serve -database-config database.json -ouroboros-repository /path/to/checkout            # detector, pre-check, measure, merge train
csf serve -database-config database.json -ouroboros-repository /path/to/checkout -ouroboros-fixers   # and fixer launch
csf ouroboros precheck -repository /path/to/checkout [-ticket N]...                             # the pre-check by hand, one row per ticket
curl http://127.0.0.1:14120/api/ouroboros                                                        # the latest numbers as JSON

The ledger is six csf_ouroboros_* tables of CSF's schema (ipc/db/csfpg); every proposal, verdict, fixer session, merge result and series point is a row there. Fixers launch unattended only when sandboxed launch is available; until then -ouroboros-fixers is the switch, and without it the loop detects, pre-checks, measures and merges.

Triggers

Four cron triggers in America/Los_Angeles, the zone #330's measurement uses; every cadence is derived from a measurement, and every occurrence skips while the previous one runs.

The pre-check

Before any model is paid for a ticket, its latest proposal (the comment the overnight loop posted on every mining ticket) must label at least one instance that exists in the corpus and is of a kind the proposed extractor reads: a run directory for jsonl, an existing issue or pull request other than the ticket itself for issue and pull_request, a commit in the repository for file_at. A proposal that names none gets the verdict needs_labels and waits in the labeler's queue; one that does gets launch.

Replayed over the 107 tickets whose fixers ran on 2026-10-03 (csf ouroboros precheck -ticket ..., 0 model calls):

The two that pass are #249 (its proposal names pull requests #226 and #245, and its fixer reached a ready pull request) and #122 (its proposal names commit a1c14af; its fixer still wrote synthetic labels, which no pre-check can foresee). The one ready pull request the check would have skipped, #236, came from a proposal that named no instance: the fixer found one on its own.

Fixers and the budget

A fixer is one harness session: the fixer agent on claude-opus-5-5 (the only model the harness admits, by the operator's ruling of 2026-10-05), a worktree of its own on miner/<name>-<assignment prefix>, the session gates on, and a brief that names the instances the pre-check found. The loop launches at most one per ticket, stops when the harness's admission reports no room, and stops for the day when the spend reaches the cap:

A fixer's cost is folded from its event log: the executor reports total_cost_usd cumulatively within one process and from zero when the process is reopened, so the cost is the sum of the maximum of each non-decreasing run. Once a fixer's one turn is done the loop closes the session and reads the outcome from its pull request: ready, draft or no_pull_request.

The merge train

The train runs tools/merge-pr.sh over every ready, non-LANG pull request, oldest first, once per head; a refused head gets a comment with the refusal lines and is retried only after its author pushes a new one. NO-SELF-MERGE (#249): the merger is csf-serve/<pid>, never a session, and a pull request whose commits carry the merger's own identity in a CSF-Session: trailer is refused without running the path.

The compounding number

The objective of #120 is the struggle rate's compounding rate, shown with its interval. Amendment 1 (operator rulings, 2026-10-04) fixes the primary series as #330's weekly one and adds an internal check. What the loop measures, exactly:

Series: ouroboros.struggle_rate_per_1k_tool_calls_weekly (one point per week) and ouroboros.struggle_rate_per_1k_tool_calls (per day), ouroboros.compounding_per_week and ouroboros.compounding_per_day, ouroboros.structure_per_intervention (per week) and ouroboros.structure_exponent, ouroboros.findings, ouroboros.fixer_spend_usd, ouroboros.cost_per_ready_pull_request_usd, ouroboros.fixer_yield. The snapshot is written to <state>/ouroboros.json, which the ops view follows into its panel, and served at /api/ouroboros.

Generic and tenant gates

A miner's scope is decided when it is accepted and carried on every finding (scope in extract.ml, Scope in records.proto, miner.exe scope): a generic gate holds for any harness user (a draft pull request opened late, a sleep loop, a self-matching pgrep); a tenant gate encodes one operator's intents, rulings or vocabulary and never leaves the tenant. The ledger keeps the scope on every finding and the snapshot counts the accepted miners by scope. Today's accepted miners: 1 generic (_template, DRAFT-PR-LATE), 0 tenant. The tenant record, its opt-in field (default off) and the export of generic findings into a shared miner set (after the operator-identifier gate, facts and spans only, instances re-keyed) are planned: CSF has no tenant record yet to carry the field.

The labeler's queue

csf_ouroboros_proposals rows with the verdict needs_labels are the queue the labeler (#318) reads; the record is LabelRequest in contract/records.proto: ticket, proposal, miner, corpus kinds, the instances that named nothing, the reason and when. The labeler never edits the row: it posts a new proposal comment with real instances, and the loop pre-checks that as a new proposal.

Labeler

Labels are the bottleneck. On 2026-10-03, 85 of 107 fixer sessions ($61.90 of $79.37) ended on a ticket that named no real instance, so each invented one. The labeler (labeler/, #318) proposes real ones at no model spend: a local model on the host's otherwise idle GPU, reached through the Ollama provider behind the brain contract.

csf label -repo OWNER/NAME -model-container ollama-1 -model-endpoint http://HOST:11434 \
  -checkout "$PWD" -ticket 105 -ticket 71            # or -tickets-file landed.tsv

One ticket is one call, in four steps:

1. Parse. The request is a LabelRequest: the loop's queue row (ticket, proposal, miner, corpus, the instances the proposal named, the pre-check's reason) plus the labeler's own reading of the ticket (repository, predicate, facts). A row that carries only the ticket is completed here: the labeler reads the ticket through gh, takes the predicate from its title and body, and the miner, corpus kinds and fact lines from its latest proposal comment, keeping every value the row already carries.
2. Prefilter, deterministically. The fact lines' quoted literals, snake_case names and flags, and the predicate's backticked spans, are the search terms. Every line of the corpus (<state>/*/events.jsonl through the file capability; issues, pull requests and git log through the process capability) that carries one is a candidate, scored by inverse document frequency, so a term on every line counts for nothing and a rare one for much, and interleaved across instances so one verbose run cannot fill a batch. No threshold is picked.
3. Label. The model reads a batch of candidates, each with its instance, source line and an excerpt centred on the rarest matched term, and answers a schema-constrained JSON object: for each candidate + or -, a verbatim quote and a reason. The prompt asks for the concrete evidence the predicate describes; shared vocabulary is -.
4. Accept, with no human in the loop. A positive stands only when it reproduces: through the ticket's miner when one is registered (-miner name=executable; its facts must name the instance), otherwise by its quote being verbatim on the line it names: every fragment of the quote, split at the excerpt's cut mark, on the line in order, read as written or with the JSON line's string escapes decoded. A quote lifted from the ticket's own words fails. Everything else is a ProposedLabel with ACCEPTANCE_REJECTED and why, or ACCEPTANCE_UNCHECKED for a negative.

Every label goes to <state>/labeler.jsonl and the run record LabelerRun to <state>/labeler.json, which the ops view's panel shows live. Both records are typed in contract/records.proto, the shape the LOOP slice and the labeler share.

GPU sharing. Before each batch the labeler lists the running containers and yields while any other than the model server was created with the GPU (Docker's --runtime nvidia, or a device request for the nvidia driver or the gpu capability); it looks again every batch's worth of model time. Every request carries a keep-alive, so the model unloads soon after a run ends: twice the median gap between batch starts, floored at twice the measured load.

Derivations. Every threshold is measured, and the derivation stands beside it in labeler.go:

Model choice, by measurement. The host holds one local model, qwen3:8b (the 27B GGUF the exited llama.cpp container names is not on disk), so the choice was between its two configurations, measured on tickets #105, #71 and #199 on 2026-10-04 with the same corpus and prompt:

Thinking labels fewer lines positive, so it loses whole tickets for three points of precision and a fifth more time; the labeler runs with it off.

The proof run. On 2026-10-04 the labeler ran over the 85 tickets whose fixer session had invented an instance (landed.tsv, gate column "synthetic labels"), on this host's RTX 5070, at $0:

Every 21st proposed positive, 20 in all, was read against its source line. All 19 accepted quotes are verbatim on their lines and the one rejected quote is not, so the rule judged 20 of 20 correctly. Only 4 of the 20 lines are an instance of their predicate (a session refusing with the ticket's exact mkdir error, a cold-cache usage record, a list of terms minted after the fact, a brief stating an operator's intent and deadline); the other 15 reproduce a line that carries the ticket's vocabulary: a brief, a tool result, a commit footer. The rule guarantees that a positive is real text at a real span, not that the span is an offense; that judgment waits for the ticket's extractor, which is the rule's first clause. The same rule read strictly (no fragments, no decoded escapes) over the first 11 tickets kept 54 of 88 positives (0.61); the final rule kept 58 of 68 on the same tickets (0.85).

Status

The contract, the template miner, its gates and the loop service are built; the PBO guard and dreaming are planned.

First mining run

On 2026-10-03 the orchestrator ran a first pass over the mining tickets. These are proposals, not miners: the orchestrator's miner loop asked one stateless safe-mode Haiku call per ticket batch to write a predicate, facts, rules and labels for each open mining ticket, and posted each as a ticket comment. No backtest has run on any of them.

Source: the loop's state/costs.tsv (one row per call: family, tickets, artifacts, input, cache read, cache creation, output tokens, USD, wall seconds, load1), state/posted.tsv (one row per comment) and no state/refused.tsv. Reproduce the sums from the loop's directory:

python3 -c "R=[l.split('\t') for l in open('state/costs.tsv') if l.strip()]; s=lambda i: sum(float(r[i]) for r in R); print(len(R), s(1), max(int(r[1]) for r in R), s(3), s(4), s(5), s(6), round(s(7), 4), round(s(8) / s(1), 1))"

Call seconds per ticket sum each call's wall time; calls ran four at a time, so elapsed time was shorter. Every call wrote its whole prompt into the cache and none read it back. The census on #41 measures these proposals against the contract: 0 of 110 can reach TP > 0 with the template's extractor, because each needs its own.

Reference

The contract

Every record is typed in contract/contract.mli, with its wire form in contract/records.proto; Corpus reads event logs and transcripts in place, gh tickets and pull requests, and files at a revision.

An argument is Text or Number, and a span is a file and a line. Severity runs S0 to S3 (#105): S0 is a gate escape, S3 an offense found only by mining. families counts the $\kappa$ candidates tried, so multiple testing is visible on every backtest.

The gates this service owns

tools/close-ticket.sh --dry-run 211 23 exits 1 with the pull request body has no | Backtest FN | row; without --dry-run it comments that row and leaves the ticket open. Walk-forward acceptance follows Bergmeir and Benítez 2012; the acceptance first written for #390 chose the knee on the record it then judged, and #86 replaced it.

Mutation score

A miner's test is itself tested (#317): Mutate (contract/mutate.ml) changes the rules or the labels one way at a time, reruns the miner's checks on each mutant, and counts.

A mutant the checks fail is killed; one they pass has survived; the score is killed over killed plus survived. Two kinds leave the denominator, each with its reason in mutation.md: a mutant check_rules refuses (an unbound variable after a drop or a negation), and an equivalent mutant, one that derives the same verdicts as the original under no knee and under every knee candidate.

The gate is Mutate.rejections: the score must reach the floor, and no surviving mutant may change a labeled instance's verdict; the survivor is printed with its rules or labels. The floor is derived, not picked: it is the template miner's measured score, recorded as Mutate.template and checked by the template's own test, so it moves only when the template's measurement does.

Mutation analysis found one blind spot in the template's test as it first shipped: a knee written into the rules, gt(S, 468), passed every check, because the fitted knee was 468 too. The knee-binds check closes it; mutation.md records that it is the only check that kills that mutant. Mutants change rule text and labels only, so they run inside the test's own Bazel sandbox; the extractor runs no further.

The $h_{\mathrm{fleet}}$ floor

$h_{\mathrm{fleet}}$ is the fleet's prompt-cache hit ratio (#235) over every harness result event's usage:

$$h_{\mathrm{fleet}} = \frac{\sum \mathrm{cache_read}}{\sum (\mathrm{input} + \mathrm{cache_creation} + \mathrm{cache_read})}$$

The 0.9867 floor is a stated number, so it carries its reality check (#241):

Measured on 2026-10-03 with:

jq -n '[inputs | select(.event_type == "result") | .event.usage // {}
  | {r: (.cache_read_input_tokens // 0),
     t: ((.input_tokens // 0) + (.cache_creation_input_tokens // 0) + (.cache_read_input_tokens // 0))}]
  | (map(.r) | add) / (map(.t) | add)' <state>/*/events.jsonl

The fleet clears the floor (0.9870 ≥ 0.9867), yet 40 of the 73 runs fall below it on their own ratio $h_r$, so a fleet-wide pass hides a majority of offending runs; #235 ranks those offenders for a miner to drive down. Tonight's 17 proposal calls read 0 cached tokens, so their ratio is 0.

PBO guard (planned)

Walk-forward answers in-sample selection (Bailey, Borwein, López de Prado and Zhu 2014) but not multiple testing. The probability of backtest overfitting (Bailey et al. 2016) splits the record into $S$ blocks; each of the $\binom{S}{S/2}$ halvings $c$ picks the best candidate in sample, with relative rank $\omega_c$ out of sample:

$$\mathrm{PBO} = \Pr_c\left[\lambda_c \le 0\right], \qquad \lambda_c = \ln\frac{\omega_c}{1 - \omega_c}$$

With $S = 2$ each half holds one or two labels, too few to rank candidates; the guard waits for a label set with a positive in each of $S \ge 4$ blocks.

Dreaming (planned)

Dreaming is the loop pointed at the language and the queue rather than at one gate (#140): a dreamer classifies operator intents and proposes terms, gates and ranked work, and an actor executes turns and never edits the language. Its objective is minimum description length (Rissanen 1978), choosing the language $L$ that minimizes $|L| + \sum_{i} |\mathrm{encode}(i \mid L)|$. It runs at two speeds, after complementary learning systems (McClelland, McNaughton and O'Reilly 1995): the nightly dream ratifies, and online dreaming (#141) mints provisional session gates that expire unless ratified.

• Example (build it, #141): swap the gates at a tool-call boundary; the hook binary reads its rules on every call, so a rule published between two calls binds from the next one.
• Counterexample (dropped, #141): swap code into the Go process during a GC pause; Go has no JIT, and a plugin can never be unloaded.

Neither speed has code here yet; a miner's findings are the dream's input once it lands.