Documentation
¶
Overview ¶
@index BM25 scoring of recorded reasons against a plain-language question, shared by every search backend.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func MatchesByPrefix ¶
MatchesByPrefix reports whether a term of an intent question is long enough to prefix-match safely.
A short Latin term is the one that misfires. `get*` reaches only 3 of 1702 recorded reasons, so it is not a common word any frequency cut would catch — but all three matches are the identifiers getAnnotation, getImpactRadius, and getAffectedFlows written inside prose, not the word "get" being used.
Length alone is not the rule, because a word boundary is not written the same way everywhere. English separates words with a space, so the indexed token already is the whole word and a prefix only buys inflections. Korean glues the particle onto the noun, so "네임스페이스가" is one token and a prefix is the only way a question about "네임스페이스" can reach it. Anything outside ASCII keeps prefix matching for that reason.
It is exported because the query sanitizers and this scorer both have to apply it: if the index matched a term one way and the score is computed the other, the answer is ordered by evidence from a query that never ran. @intent measure a term against the misfire that motivated the rule, not against a raw length. @domainRule non-ASCII terms always match by prefix, whatever their length.
Types ¶
type Doc ¶
type Doc struct {
NodeID uint
Content string
FilePath string
QualifiedName string
Kind graph.NodeKind
Namespace string
StartLine int
}
Doc is one candidate the index admitted: a node and one recorded reason that was indexed for it. A node that recorded several reasons arrives as several Docs sharing a node id.
It carries the node's identity alongside its id because scoring ties are the normal case here, and the tie-break has to mean the same thing after a re-index. An id cannot: it is handed out in the order rows were written. @intent carry the exact indexed text into scoring so the score is computed over what was matched.
type Match ¶
Match is one declaration the question reached, and the terms of the question written in its recorded reasons.
It is per node, not per reason. The index holds one document per reason, so a question touching two of a node's reasons reaches it twice; naming it twice would tell the reader there are two answers where there is one declaration. @intent say what earned a declaration its place, not only that it earned one.
type Result ¶
Result is a ranked answer plus the evidence for it. @intent hand back what matched alongside what ranked, so a weak answer can be recognised as one.
func Rank ¶
Rank scores candidate reasons against a question and returns the answer best first, at most limit declarations of it, with the evidence that produced it.
The limit counts declarations because that is what the caller is asking for. One node can arrive as several documents — one per recorded reason — and if those spent the caller's slots, a node whose author wrote three reasons down would shorten the page for everybody else.
This runs in Go rather than in the database because the two databases do not agree. SQLite's FTS5 orders by bm25, which discounts a word that appears in many recorded reasons; PostgreSQL's ts_rank reads one document at a time and never learns that a word is common. Same index, same question, different answer — and the deployed server runs PostgreSQL while the golden set was measured on SQLite. Scoring here gives both backends one answer to be judged by, and leaves the databases doing what they both do well: finding candidates.
corpusSize is how many documents the whole index holds, which is what makes a word "common". Pass 0 and only the candidates are counted, which overstates how rare every term is.
The evidence comes back rather than a confidence score or a cutoff. A cutoff would be a number fitted to whichever codebase it was measured on; the term counts are recounted against whatever corpus is in front of them, so they mean the same thing in a repository nobody has ever measured. "Twenty files, all of them matched on `code` alone, and `code` is written in 812 of 1751 recorded reasons" is something the reader can act on. A score of 0.31 is not.
@requires docs must be every document the index matched, not a truncated page. @return returns one match per declaration in answer order, dropping any declaration no term of the question reaches, and every question term with its corpus count either way.
type Term ¶
Term is one term of the question and how many recorded reasons in the whole index hold it.
A term nobody wrote down is reported with a count of zero rather than left out, because that is the reader's answer to why the question came back thin. @intent let a reader weigh a match by how common the word that earned it is.
Source Files