deepprobe

Overview ¶

Command deepprobe measures what ranking metrics structurally cannot see: whether the answer is inside the excerpt the response actually shows.

Every retrieval channel (vector, FTS, rerank) reads body_excerpt, which the extractor caps, and evidence-span trimming picks its window from that same capped text. So for a long symbol the response can only ever show the head. Our self-retrieval corpus cannot detect this: it queries a symbol by its docstring, and docstrings sit in the head.

A case here is (symbol, probe line, query), where the probe line is real code from the part of the body that the excerpt drops. Two rates are reported and must not be conflated:

retrieved — the owning symbol reached top-k at all (a recall property of
            the vector/FTS channels)
covered   — the shown span contains the probe line (a presentation
            property of evidence-span selection)