Documentation
¶
Overview ¶
@index BM25 scoring of recorded reasons against a plain-language question, shared by every search backend.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func MatchesByPrefix ¶
MatchesByPrefix reports whether a term of an intent question is long enough to prefix-match safely.
A short Latin term is the one that misfires. `get*` reaches only 3 of 1702 recorded reasons, so it is not a common word any frequency cut would catch — but all three matches are the identifiers getAnnotation, getImpactRadius, and getAffectedFlows written inside prose, not the word "get" being used.
Length alone is not the rule, because a word boundary is not written the same way everywhere. English separates words with a space, so the indexed token already is the whole word and a prefix only buys inflections. Korean glues the particle onto the noun, so "네임스페이스가" is one token and a prefix is the only way a question about "네임스페이스" can reach it. Anything outside ASCII keeps prefix matching for that reason.
It is exported because the query sanitizers and this scorer both have to apply it: if the index matched a term one way and the score is computed the other, the answer is ordered by evidence from a query that never ran. @intent measure a term against the misfire that motivated the rule, not against a raw length. @domainRule non-ASCII terms always match by prefix, whatever their length.
Types ¶
type Doc ¶
Doc is one candidate the index admitted: a node and the recorded reason that was indexed for it. @intent carry the exact indexed text into scoring so the score is computed over what was matched.
type Match ¶
Match is one document the question reached, and the terms of the question written in it. @intent say what earned a document its place, not only that it earned one.
type Result ¶
Result is a ranked answer plus the evidence for it. @intent hand back what matched alongside what ranked, so a weak answer can be recognised as one.
func Rank ¶
Rank scores candidate reasons against a question and returns the answer best first, at most limit documents of it, with the evidence that produced it.
This runs in Go rather than in the database because the two databases do not agree. SQLite's FTS5 orders by bm25, which discounts a word that appears in many recorded reasons; PostgreSQL's ts_rank reads one document at a time and never learns that a word is common. Same index, same question, different answer — and the deployed server runs PostgreSQL while the golden set was measured on SQLite. Scoring here gives both backends one answer to be judged by, and leaves the databases doing what they both do well: finding candidates.
corpusSize is how many documents the whole index holds, which is what makes a word "common". Pass 0 and only the candidates are counted, which overstates how rare every term is.
The evidence comes back rather than a confidence score or a cutoff. A cutoff would be a number fitted to whichever codebase it was measured on; the term counts are recounted against whatever corpus is in front of them, so they mean the same thing in a repository nobody has ever measured. "Twenty files, all of them matched on `code` alone, and `code` is written in 812 of 1751 recorded reasons" is something the reader can act on. A score of 0.31 is not.
@requires docs must be every document the index matched, not a truncated page. @return returns matches in answer order, dropping any document no term of the question reaches, and every question term with its corpus count either way.
type Term ¶
Term is one term of the question and how many recorded reasons in the whole index hold it.
A term nobody wrote down is reported with a count of zero rather than left out, because that is the reader's answer to why the question came back thin. @intent let a reader weigh a match by how common the word that earned it is.
Source Files