giteval

Overview ¶

Command giteval scores the served stack against labels nobody on this project authored: each case is a real commit — query = the commit subject, gold = the symbols whose enclosing-function hunk headers appear in that commit's diff, resolved against the index. If the tool can take the words a developer used to DESCRIBE a change and surface the symbols the change touched, it works on real questions about real data.

Label independence caveat: commit subjects describe deltas ("fix X when Y"), agent queries describe code ("where is X decided") — related but not the same distribution. Treat absolute numbers as a floor, deltas as the signal.

Usage:

giteval -bin .task/build/contextmaxxer -repo ../public-project [-scan 400] [-n 100]