stteval

Overview ¶

cmd/stteval evaluates the unreviewed transmissions in cmd/sttreview's queue against the current STT decoder. It partitions them into trusted entries (current output agrees with the stored output and passes consistency checks), suspects (disagreement or a failed check), and unusable records; selects the trusted entries that add statement coverage over the existing stt/tests corpus; and stages new test files and a suspects review queue for human sign-off.

Phases (run in order; each reads the earlier phases' outputs from -evaldir):

-phase decode    decode every queued entry, write report.jsonl
-phase coverage  per-entry coverage novelty + greedy selection
                 (requires a binary built with
                 go build -cover -covermode=atomic -coverpkg=github.com/mmp/vice/stt)
-phase emit      write staged-tests/, staged-index.json, suspects.json
-phase apply     -accepted <file>: install accepted staged tests in
                 stt/tests/, divert the rest to suspects.json, mark
                 all processed entries Seen in the main state