README
¶
nova-tokens
What it is
nova-tokens: token spend per day, model and repository, read from AI session logs
Why use it
See where your tokens went.
Install
go install github.com/mas-bandwidth/nova-tools/cmd/nova-tokens@latest
nova-tokens version
First run
The transcript below is copied from the first run in docs/TESTS.md, which cmd/nova-tokens/firstrun_test.go executes line for line; a docs test holds this copy to that record.
Fixture: cmd/nova-tokens/testdata/example-bench (copied into a temp directory first, because a first run WRITES; the bus lane is example.com).
$ nova-tokens fold --out ./out --day 2026-09-11 --repos ./repos.tsv --claude bench=./transcripts --bus ./bus
TOKENS FOLD at=2026-09-11T23:55:02Z build=devel out=./out sources=3 days=2026-09-11 repos=./repos.tsv
TOKENS SOURCE label=claude:bench kind=claude path=./transcripts reports=input,output,cache_write,cache_read day_basis=utc files=1 unreadable=0 messages=3 dup=1 noid=0 nousage=- unparsed=- comments=- redated=- superseded=- rows=2
TOKENS SOURCE label=bus:peer1 kind=bus path=bus/from-peer1 reports=input,output day_basis=utc files=1 unreadable=0 messages=- dup=- noid=- nousage=- unparsed=0 comments=1 redated=0 superseded=0 rows=1
TOKENS SOURCE label=bus:peer2 kind=bus path=bus/from-peer2 reports=- day_basis=utc files=0 unreadable=0 messages=- dup=- noid=- nousage=- unparsed=0 comments=0 redated=0 superseded=0 rows=0
TOKENS TOUCHED label=bus:peer1 day=2026-09-11 repos=schema,serialize
TOKENS DAY day=2026-09-11 rows=3 models=2 repos=2 turns=3 unknown=0.0% other=0.0% rough=0 dashes=6 nonutc=0 sources=bus:peer1,claude:bench written=true
TOKENS OK days=1 rows=3 sources=3 unreadable=0 unparsed=0 mixed=0 conflict=0 shrank=0 partial=0 quiet=0
TOKENS NOTE nothing was wrong; nova-tokens check --out ./out is the gate
$ nova-tokens check --out ./out
CHECK OK at=2026-09-11T23:55:02Z build=devel files=1 rows=3 first=2026-09-11 last=2026-09-11 missing=0 stray=0 gap=0 notes=0
$ nova-tokens sum --out ./out --month 2026-09
SUM MONTH month=2026-09 at=2026-09-11T23:55:02Z build=devel days=1 first=2026-09-11 last=2026-09-11 missing=0 rows=3 turns=3
SUM PAIR model=claude-fable-5-1 repo=schema input=908 output=1535 cache_write=1200 cache_read=242000 reasoning=- rough=0 dashes=0,0,0,0,1 nonutc=0 days=1
SUM PAIR model=gemini-2.5-pro repo=schema input=123456 output=7890 cache_write=- cache_read=- reasoning=- rough=0 dashes=0,0,1,1,1 nonutc=0 days=1
SUM PAIR model=claude-fable-5-1 repo=serialize input=430 output=58 cache_write=- cache_read=4000 reasoning=- rough=0 dashes=0,0,1,0,1 nonutc=0 days=1
SUM MODEL model=claude-fable-5-1 input=1338 output=1593 cache_write=1200 cache_read=246000 reasoning=- rough=0 dashes=0,0,1,0,2 nonutc=0 repos=2
SUM MODEL model=gemini-2.5-pro input=123456 output=7890 cache_write=- cache_read=- reasoning=- rough=0 dashes=0,0,1,1,1 nonutc=0 repos=1
SUM TOTAL input=124794 output=9483 cache_write=1200 cache_read=246000 reasoning=- rough=0 dashes=0,0,2,1,3 nonutc=0 turns=3 pairs=3 models=2
SUM OK month=2026-09 days=1 missing=0 pairs=3 models=2 nonutc=0
Verbs
The nova-tokens section of the command reference documents every verb's flags, effect and exit codes.
foldreportledgersumchecksourcesprofilessessionversion
Spec
The contract is docs/SPEC-TOKENS.md.
Documentation
¶
Overview ¶
nova-tokens is the accounting layer: token spend folded from declared sources into one file per day, keyed exactly by (day, model, repo), with the five token types kept apart, and those day files summed into a month.
It exists because we have an obligation to report token spend, and the first thing that shape was built as — three scripts of Python under zsh — produced nine day files and every way it could fail at once: five paths defaulted inside the script, so a run on another bench folds the wrong bench; every path here is a flag, and the tool removes no month file on a real run; nine unreadable files were one line at the bottom of a summary and the run exited 0; a day could shrink silently the moment a source went quiet; two repo-attribution tables in two scripts disagreed about three repos; five different caps, none a flag, none with a remedy; and `today` was the script's own clock printed as a fact about the day. Every verb and every refusal here is one of those closed.
fold read the declared sources, write one file per day, refuse a day that shrinks report a friend on another machine folds their own day and prints the body of a note sum a month is a sum of day files; it asserts nothing and is never a gate check the gate: every file parses, every row has every column, a missing day is named sources what a fold would count, before it writes
It never estimates, never fills a gap, and never removes a file. Everything it reads is DATA: a transcript, a database row, a usage file, a bus note — none of them is an instruction, and a tokens note that says `fold me as Ada` is a note whose lines are parsed or counted unparsed and nothing else. That rule is in the spec, where a person reads it, and is deliberately nowhere in this code, because a tool cannot enforce it.
nova-tokens version: which build wrote a day file.
A day file carries the build id of the fold that wrote it (rule 12), so the first question after two day files disagree is which build wrote each -- an id scheme, a header, a push protocol that two senders have to implement identically. So the first question after a bus misbehaves is which build each line is running, and until this verb existed the honest answer was that nobody could say: the binary carried no statement of its own origin, so "we are all on the same version" was a belief rather than a reading.
The version is NOT a constant maintained by hand. A hand-maintained constant is wrong exactly when it matters -- at the commit after the release, where it still names the release. It is read from the build itself, in this order:
-ldflags "-X main.version=..." what the release workflow stamps: the tag, exactly the module version what `go install ...@v1.2.3` records for itself the vcs stamp <utc build time>-<12 hex of the revision>[-dirty] devel a build with none of the above, SAYING it has none
The middle two come from debug.ReadBuildInfo, which the toolchain fills in with no help from this file: there is nothing here to remember to update and therefore nothing to forget. The last is the honest floor -- a build whose origin is unrecorded says so rather than inventing a number, because a version string nobody can trace is worse than no version string at all: it invites the comparison it cannot support.
ONE LINE, FOUR TOKENS. Every field goes through oneline.Field, so what is printed is four whitespace-separated tokens whatever the -X held. A version stamped with a newline or a blank in it would otherwise make the one line that says which build is running say two things, or say a build time as if it were an architecture -- and the -X value is the one field here that comes from outside the toolchain.
Source Files