Documentation
¶
Overview ¶
Package search provides ranking, formatting, and token-counting utilities for documentation query results returned by the defSource library.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ApproxTokenCount ¶
ApproxTokenCount returns an approximate token count for the given text. It multiplies the whitespace-delimited word count by 1.5 to account for the higher token-per-word ratio typical of code-heavy documentation (PHP signatures, type declarations, and source snippets tokenize more densely than prose). The result is an approximation suitable for token-budget gating; it is not calibrated to any specific LLM tokeniser.
func FormatDocSnippets ¶
func FormatDocSnippets(snippets []source.DocSnippet, tokenBudget int) string
FormatDocSnippets renders snippets as LLM-friendly Markdown text and joins them with "---" separators. It respects tokenBudget by stopping before appending a snippet that would push the running token total over the limit.
Token-budget contract:
- The first snippet is always included regardless of its size. A caller that passes tokenBudget=0 (or any value smaller than the first snippet's token count) will still receive one snippet, never an empty string. This at-least-1 guarantee exists so callers always have something meaningful to return even when the budget is very tight.
- Subsequent snippets are added only when totalTokens+snippetTokens <= tokenBudget. Once the budget is exhausted the remaining snippets are silently dropped.
- An empty snippets slice returns "".
Token counts are estimated by ApproxTokenCount (a 1.5× word-count heuristic).
func RankLibraries ¶
func RankLibraries(query string, libs []store.LibraryRecord) []store.LibraryRecord
RankLibraries scores and sorts libs by relevance to query, returning a new slice ordered highest-score first. Scoring uses a four-tier name-match system: exact match (100 points), prefix match (80), substring match in the library name (60), or substring match in the library ID (40). Libraries that score zero on the name tiers are still included but ranked last.
Two continuous tiebreakers are added on top of the tier score:
- SnippetCount × 0.01: larger, more fully-indexed libraries rank slightly higher when name-match quality is equal.
- TrustScore × 10: explicitly curated quality signal overrides snippet volume when present.
The sort is not stable; libraries with identical total scores may appear in any order between calls.
Types ¶
This section is empty.
Source Files