Package exemplars implements BM25-based few-shot exemplar mining with
greedy-diverse k-center selection. It maintains per-task JSONL sidecars of
verified-good (input, output) pairs and, on demand, selects a diverse ~50
finalists subset and persists it for fast retrieval.
Concurrency: Append uses O_APPEND for cross-process safety plus an in-process
mutex per task file. Select and Retrieve are read-only after the selected.json
is written and safe to call concurrently.
Append appends a verified-good (input, output) pair to the per-task sidecar.
It is concurrency-safe across goroutines (mutex) and across processes (O_APPEND).
Select builds a BM25 index over the sidecar for `task`, then runs a
greedy k-center diverse pick targeting ~50 finalists, and writes the result
to <dir>/<task>.selected.json. It returns a human-readable report.
func Retrieve(dir, task, input string, k int) []Pair
Retrieve loads <dir>/<task>.selected.json, BM25-ranks the entries against
`input`, and returns the top-k diverse results. Never errors; missing file
or any I/O issue returns nil.
Documentation
¶
Source Files