Documentation
¶
Overview ¶
Package distill turns the raw review findings behind lessons into candidate groups for LLM distillation (RFC-001 §5.4 Tier 2). Exact clustering cannot merge ten differently-worded findings about one mistake; a reader of the raw material can — this package prepares that reader's batches.
The pipeline is built around one economic invariant: an unchanged evidence set is never distilled twice. Findings carry stable ids (GitHub comment ids), a group's Signature hashes its member ids, and the store remembers which signatures were already processed. A new finding changes its group's signature — exactly that group is re-read, nothing else.
Grouping is a Grouper: the contract (deterministic output, stable signatures) is the architecture, the strategy is replaceable. The built-in lexical grouper connects findings that share salient tokens — identifiers survive in finding bodies, and "reset"/"pooled"/"Free" recurring across files is precisely the trace a semantic pattern leaves — then buckets the remainder by directory. An embedding-based grouper can replace it without touching the pipeline or the stored state.
Index ¶
- func ApplyPins(root string, ps []model.Proposal) error
- func Clusters(ps []model.Proposal) [][]model.Proposal
- func CountEvents(cited []model.Finding) int
- func RenderPins(ps []model.Proposal) (string, error)
- type Config
- type Group
- type GroupPlan
- type Grouper
- type Known
- type Options
- type PinKey
- type Preflight
- type Result
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ApplyPins ¶
ApplyPins inserts the proposals as pins into <root>/.seamark/ lessons.yaml, preserving everything already there: entries are inserted directly under an existing bare `pin:` line (list items at the head of the list — hand-written entries and their comments are untouched), or a new pin section is appended when none exists. The result must parse as a lessons config before one byte is written; a file this function cannot safely edit (a flow-style `pin: []`, say) is an error, and the caller falls back to printing the block.
func Clusters ¶
Clusters groups patterns that restate each other, largest first; singletons are omitted. It is the audit of an existing pin file: what is already there in several wordings, for a human to prune (seamark never edits lessons.yaml unasked).
func CountEvents ¶
CountEvents collapses cited findings into distinct events: findings sharing a pull request are one event (the review comment and the fix commit that answered it), while findings without a pr number are independent. The prompt states this rule; validation enforces it, because the model's arithmetic is not evidence — the same reason cited ids are checked against the group rather than trusted. Exported so the report counts a proposal's evidence exactly as the recurrence bar did.
Types ¶
type Config ¶
type Config struct {
Distill struct {
// Write lets `lessons --apply` edit .seamark/lessons.yaml. Off
// by default: without it, apply prints the pin block for the
// human to paste — seamark never modifies a reviewed config
// file without the workspace having opted in, in that same
// reviewed config.
Write bool `yaml:"write"`
} `yaml:"distill"`
}
Config is the distill section of .seamark/config.yaml.
func LoadConfig ¶
LoadConfig reads the distill section of the shared config file. Same contract as the other section readers: absent file means defaults, malformed file is loud.
type Group ¶
type Group struct {
// Region is the deepest directory common to every member, "" when
// the members span top-level trees (a repo-wide theme batch).
Region string
// Findings are the members, ordered by id.
Findings []model.Finding
// Signature identifies the evidence set: the hash of the member
// ids. Same members — same signature, regardless of ordering or of
// anything outside the group.
Signature string
}
Group is one distillation batch: findings that plausibly share a theme (or at least a neighborhood), with a stable identity.
type Grouper ¶
Grouper buckets findings into candidate groups. Implementations must be deterministic: the same findings yield the same groups with the same signatures, in the same order.
func NewLexicalGrouper ¶
func NewLexicalGrouper() Grouper
NewLexicalGrouper returns the default token-overlap Grouper.
The thresholds are measured, not guessed (graphql-go-tools, 1517 findings + the pooled-state benchmark): minShared 3 or 4 halves the number of blank-region cap-sized slices — less transitive chaining — but drops the pooled-state theme to 6/8 then 5/8 members, sacrificing exactly the cross-wording recall this package exists for. 2 keeps the benchmark at 8/8 and accepts one large weak-link component that the size cap turns into bounded, id-ordered (≈chronological) batches. A smarter Grouper (two-tier edge strength, embeddings) is the upgrade path if mixed batches prove to dilute distillation quality.
type Known ¶
type Known struct {
// contains filtered or unexported fields
}
Known is the set of patterns already captured: every pin in lessons.yaml (hand-written ones included) and every proposal already decided or pending. A distilled pattern matching one of these is not news — including one the user dismissed, since a dismissal is a decision the distiller must not relitigate.
func NewKnown ¶
NewKnown builds the set. Pins arrive as Proposals carrying just Rule and Note, so config pins and stored proposals share one shape.
func (*Known) Add ¶
Add records a pattern as known — used as a run proceeds, so two groups in the same pass cannot both propose the same thing.
func (*Known) Labels ¶
Labels lists the rule names already captured for an area, newest first, at most limit of them. They are fed to the distiller so it can skip what is known and spend the call looking for something else — labels only, because the notes would cost more tokens than the duplicates they prevent.
type Options ¶
type Options struct {
// Region restricts the run to groups whose Region sits within this
// prefix ("" = everywhere). Cross-tree theme groups (Region "") only
// run when no region filter is set.
Region string
// Limit caps how many new groups one run reads (0 = all). The cap
// is the budget lever: each group is one agent invocation.
Limit int
// Logf receives progress; nil discards it.
Logf func(format string, args ...any)
// OnGroupStart/OnGroupDone drive interactive surfaces through the
// long silent stretch of each agent call: start fires with the
// group's description before the call, done with the outcome after.
// When nil, the same information flows through Logf as plain lines.
OnGroupStart func(desc string)
OnGroupDone func(outcome string)
// Pins are the patterns already captured in .seamark/lessons.yaml,
// hand-written ones included, as Rule/Note pairs. A distilled
// pattern that restates one of them is dropped: groups are read
// independently, so a repo-wide mistake would otherwise be
// re-proposed under a new name by every group it appears in.
Pins []model.Proposal
// Agent is the resolved agent command line, for the preflight
// disclosure only — the Invoker is what actually runs it.
Agent []string
// OnPreflight receives the plan before the first agent call: what
// would be sent, where, and roughly what it would cost.
OnPreflight func(Preflight)
// DryRun stops after the preflight: nothing is sent, marked, pruned,
// or persisted. The Invoker may be nil on a dry run.
DryRun bool
}
Options tunes one distillation run.
type PinKey ¶
PinKey identifies one pin entry. A rule name alone is not an identity: the same rule is legitimately pinned in several regions (RUF001 for scripts and for api), and pruning one must not take the others with it.
func RemovePins ¶
RemovePins deletes the named pins from <root>/.seamark/lessons.yaml, with their provenance comments, leaving every other byte alone. It is the inverse of ApplyPins and deliberately more careful: adding a wrong entry is visible in a diff, while deleting a neighbouring one destroys work. A pin already absent is not an error (the user may have pruned it by hand); the result must parse AND contain exactly the expected remaining pins, or nothing is written.
type Preflight ¶
type Preflight struct {
// Agent is the command line each group's prompt would be piped to.
Agent []string
// Groups describes each evidence group that would be sent this run.
Groups []GroupPlan
// PromptChars totals the prompts exactly as they would be sent now.
// Execution primes later prompts with labels learned from earlier
// groups, so treat it as a close lower bound.
PromptChars int
// Findings counts the finding bodies across all planned groups.
Findings int
// BodyCap is the per-finding truncation applied inside prompts — the
// only bound on what a body carries: bodies are NOT redacted.
BodyCap int
}
Preflight is the pre-invocation disclosure. Metadata only — never finding bodies.
type Result ¶
type Result struct {
GroupsTotal int // candidate groups in the current corpus
GroupsSkipped int // already distilled (signature known)
GroupsRead int // sent to the agent this run
GroupsFailed int // agent or parse errors (not marked; retried next run)
GroupsPending int // new groups left unread (limit or region filter)
PrunedStale int // pending proposals dropped because their group changed
// PromptChars/ReplyChars meter the run's agent traffic — the basis
// for the ~token estimate shown to the user. Failed groups count
// too: their cost was paid.
PromptChars int
ReplyChars int
Duration time.Duration
// Duplicates counts distilled patterns dropped as restatements of
// something already pinned, proposed, or decided.
Duplicates int
Proposals []model.Proposal
}
Result reports what a run did.
func Run ¶
func Run(ctx context.Context, st *store.Store, grouper Grouper, inv agent.Invoker, opts Options) (*Result, error)
Run executes the plan half of distillation: group the findings, skip evidence sets already read, send each remaining group to the agent, validate what comes back, and persist the survivors as proposals. It never touches .seamark/lessons.yaml — applying is a separate, human decision.