Documentation
¶
Overview ¶
Command corpusstat derives the engine-consistent record, URL, host, and host-key-range counts over a url-only JSONL corpus, keyed the way the scale and bench harness key it: full-host grouping via frontier.HostOf and meguri.HostKeyOf. It fills a profile manifest's records, urls, hosts, hostkey_lo, and hostkey_hi fields with the engine's view of the corpus, which differs from a naive hostname or registrable-domain split. Run it when pinning a new scale profile.
go run ./cmd/corpusstat corpus/profiles/scale-1m.jsonl
Source Files
Click to show internal directories.
Click to hide internal directories.