Help us shape the future of Go! Take the 2026 Go Developer Survey

distill

package
v0.8.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 18, 2026 License: Apache-2.0 Imports: 27 Imported by: 0

Documentation

Overview

Package distill turns the raw review findings behind lessons into candidate groups for LLM distillation (RFC-001 §5.4 Tier 2). Exact clustering cannot merge ten differently-worded findings about one mistake; a reader of the raw material can — this package prepares that reader's batches.

The pipeline is built around one economic invariant: an unchanged evidence set is never distilled twice. Findings carry stable ids (GitHub comment ids), a group's Signature hashes its member ids, and the store remembers which signatures were already processed. A new finding changes its group's signature — exactly that group is re-read, nothing else.

Grouping is a Grouper: the contract (deterministic output, stable signatures) is the architecture, the strategy is replaceable. The built-in lexical grouper connects findings that share salient tokens — identifiers survive in finding bodies, and "reset"/"pooled"/"Free" recurring across files is precisely the trace a semantic pattern leaves — then buckets the remainder by directory. An embedding-based grouper can replace it without touching the pipeline or the stored state.

Index

Constants

View Source
const DefaultGroupLimit = 10

DefaultGroupLimit is the number of new evidence groups one run sends when the operator names no limit. Each group is one agent call, so the default bounds what a run costs. The CLI flag and the console share it.

View Source
const (
	// TriggerPromptVersion is persisted with an answered trigger question.
	// Version 3 bounds evidence-path metadata; version 2 permits a directly
	// cited companion path, while version 1 told the model to omit every
	// trigger that was also an evidence path. Version 3 also covers the
	// indented continuation lines of a multi-line rule or note
	// (quotedField). A one-line field keeps its bytes, so that change
	// reopens no stored negative answer and asks no question again.
	TriggerPromptVersion = 3
)

Variables

View Source
var (
	// ErrInvalidLimit reports a negative group limit. The legacy CLI
	// reads a negative limit as "all"; a reviewed plan must not guess.
	ErrInvalidLimit = errors.New("the group limit must be 0 (all groups) or positive")
	// ErrStale reports that an input of the plan changed after planning.
	// The wrapped text names each changed input. The caller must make a
	// new plan instead of running the old one.
	ErrStale = errors.New("distillation inputs changed since the plan")
)
View Source
var ErrExtractRoot = errors.New("trigger extraction needs the workspace root: " +
	"without it every answer would be stored as a negative")

ErrExtractRoot reports a plan without a workspace root. The existence rung needs the root: without it no named path is verified, and every answer would be stored as a permanent negative.

View Source
var ErrExtractStale error = extractStaleError{}

ErrExtractStale reports that an input of an extraction plan changed after planning. The wrapped text names each changed input. It also matches ErrStale, so one check serves both kinds of plan.

View Source
var ErrNoInvoker = errors.New("distill: the plan has groups but no agent invoker")

ErrNoInvoker reports a plan with groups but no invoker to run them.

View Source
var ErrNoTriggersKey = errors.New(`reply carries no "triggers" key — not a negative answer, the batch retries`)

ErrNoTriggersKey marks a reply without a "triggers" key. Such a reply answers nothing, so no row is stamped. The CLI prints this text as it is, and its golden tests pin it. The console matches the error with errors.Is and uses its own words, because the console retries nothing by itself.

View Source
var ErrRegionEscapes = errors.New("the region is outside the workspace")

ErrRegionEscapes reports a region outside the workspace root. Such a region matches no finding, so accepting it would plan an empty run that looks like "nothing new" instead of a mistyped path.

Functions

func AuditScopes added in v0.4.0

func AuditScopes(st *store.Store, cfg *reviews.Config, root string,
	ps []model.Proposal, meta map[int64]model.Finding,
) (map[int64]ScopeAdvisory, error)

AuditScopes runs the trigger-scope check for a set of proposals. Pending rows use their stored note and regions. Applied rows use the live yaml pin's note — a reworded note is what fires — and rows whose pin is pruned or re-scoped by hand are skipped. Only living findings feed the check. Keyed by proposal ID; unflagged proposals are absent.

func Clusters

func Clusters(ps []model.Proposal) [][]model.Proposal

Clusters groups patterns that restate each other, largest first; singletons are omitted. It audits an existing pin file for guidance repeated in different words. Seamark reports the duplicates but never edits lessons.yaml without an explicit request.

func CountEvents

func CountEvents(cited []model.Finding) int

CountEvents re-exports model.CountEvents — the recurrence bar counts events, and older callers reach it through the distill package.

func CoverageRegions added in v0.2.0

func CoverageRegions(cited []model.Finding) []string

CoverageRegions computes a proposal's region set from its cited evidence: a small set of directories (≤ maxRegions, depth ≤ maxRegionDepth) covering at least coverTarget of the voting events, chosen greedily — most new events first. Greedy is not guaranteed minimal in general, but under this need/cap geometry no input has been found where it misses a cover an exhaustive search would find, and it stays linear in candidates where exhaustion is combinatorial. Nil means repo-wide — rendered `*`, exactly as before.

func NormalizeRegion added in v0.8.0

func NormalizeRegion(root, input string) (string, error)

NormalizeRegion turns operator input into the repository-relative region form that PlanOptions.Region expects. A relative input is relative to root, not to the process directory, because the console can start anywhere. An absolute input must be inside root. The empty input, ".", "./", and root itself all mean the whole repository and return "". The result uses slashes and has no trailing slash.

The CLI keeps its own lenient toRepoRel: its output must not change.

func PinsFromConfig added in v0.8.0

func PinsFromConfig(cfg *reviews.Config) []model.Proposal

PinsFromConfig converts the pins of .seamark/lessons.yaml into the proposal shape that deduplication reads. The CLI and the console both call it, so a pin counts as known in the same way on both surfaces.

func RenderPins

func RenderPins(ps []model.Proposal) (string, error)

RenderPins renders proposals as ready-to-paste pin entries, each with its provenance comment. The YAML values are marshaled, not hand-quoted — a note containing quotes or colons must not be able to corrupt the file.

The region keys follow reviews.PinForRegions, the layout that the manual pin renderer (pinfile.RenderPin) also writes. The bytes of this render stay fixed, because the CLI apply goldens pin them.

Types

type Config

type Config struct {
	Distill struct {
		// Write lets `lessons --apply` edit .seamark/lessons.yaml. Off
		// by default: without it, apply prints the pin block for manual
		// insertion. Seamark edits a reviewed config only when that same
		// config opts in.
		Write bool `yaml:"write"`
	} `yaml:"distill"`
}

Config is the distill section of .seamark/config.yaml.

func LoadConfig

func LoadConfig(root string) (*Config, error)

LoadConfig reads the distill section of the shared config file. Same contract as the other section readers: absent file means defaults, malformed file is loud.

func ParseConfig added in v0.8.0

func ParseConfig(data []byte) (*Config, error)

ParseConfig decodes the distill section from the bytes of config.yaml. LoadConfig calls it, so both entry points report the same error text. The settings editor calls it to validate planned bytes before a write. Empty input yields the defaults, the same as an absent file.

type Duplicate added in v0.8.0

type Duplicate struct {
	// Rule is the dropped pattern's label.
	Rule string
	// Of is the label of the known pattern it restates.
	Of string
}

Duplicate is one returned pattern that restates a known one. The executor drops it before persistence.

type Event added in v0.8.0

type Event struct {
	Kind EventKind
	// Index is the 0-based position in Plan.Groups. For EventStopped it
	// is the first group that did not run.
	Index int
	// Total is len(Plan.Groups), a known denominator for progress.
	Total int
	// Group is set on EventGroupStarted.
	Group *GroupPlan
	// Outcome is set on EventGroupFinished.
	Outcome *GroupOutcome
	// Dup is set on EventDuplicate.
	Dup *Duplicate
	// Removed is set on EventCleanupDone.
	Removed int
	// Stop and Err are set on EventStopped.
	Stop StopReason
	Err  error
}

Event is one typed progress report. The pointers refer to copies, so a receiver on another goroutine never shares memory with the run.

type EventKind added in v0.8.0

type EventKind int

EventKind names one progress event.

const (
	// EventCleanupDone follows the stale-proposal cleanup.
	EventCleanupDone EventKind = iota
	// EventGroupStarted precedes the agent call of one group.
	EventGroupStarted
	// EventDuplicate reports one dropped restatement.
	EventDuplicate
	// EventGroupFinished reports a group outcome.
	EventGroupFinished
	// EventStopped reports that no further group runs.
	EventStopped
)

func (EventKind) String added in v0.8.0

func (k EventKind) String() string

String returns a stable lower-case name for the kind.

type ExecuteOptions added in v0.8.0

type ExecuteOptions struct {
	// Progress receives events synchronously on the executing goroutine,
	// never while a transaction is open. A slow receiver delays the next
	// group, not the current agent call. Nil discards the events.
	Progress func(Event)
	// Revalidate runs before the cleanup and before each group. A
	// non-nil error stops the run with StopConflict. A service passes
	// its file and workspace checks here. Nil checks nothing.
	Revalidate func() error
	// Boundary turns on the store checks: the full plan revalidation
	// before the cleanup, the per-group checks before each group, and
	// the guarded save that refuses a group another writer recorded.
	Boundary bool
	// SkipCleanup leaves the planned cleanup undone. The legacy Run
	// sets it, because it already pruned during planning.
	SkipCleanup bool
}

ExecuteOptions tune one execution.

type Execution added in v0.8.0

type Execution struct {
	// Groups are the outcomes, parallel to Plan.Groups.
	Groups []GroupOutcome
	// CleanupIDs are the planned cleanup rows.
	CleanupIDs []int64
	// Removed counts the rows the cleanup actually deleted. A row that
	// another writer decided meanwhile stays, so it can be lower.
	Removed int
	Stop    StopReason
	StopErr error
	// Result holds the legacy counters, derived from the outcomes.
	Result Result
}

Execution is the typed result of one execution.

func Execute added in v0.8.0

func Execute(ctx context.Context, st *store.Store, inv agent.Invoker, p *Plan, opts ExecuteOptions) (*Execution, error)

Execute runs exactly the planned cleanup and groups. It checks ctx before the cleanup and before each group, and it starts no agent call after a stop. A stop that cancels a group, the last one included, ends the run with StopCancelled and EventStopped. A reply that returns before the stop is persisted and counted as completed, never also as cancelled.

It always returns a non-nil *Execution. The error is non-nil only for a store failure or an unusable call (no plan, no invoker); a stop or a conflict is data in Execution.Stop, not an error.

type ExtractBatch added in v0.8.0

type ExtractBatch struct {
	// Proposals are the frozen rows in send order. The question sends
	// their recorded rule and note, and their guards bind the answers.
	Proposals []model.Proposal
	// Items are parallel to Proposals.
	Items []ExtractItem
	// Prompt is the exact text that ExecuteExtract sends.
	Prompt      string
	PromptChars int
}

ExtractBatch is one planned agent call.

type ExtractBatchOutcome added in v0.8.0

type ExtractBatchOutcome struct {
	// Index is the position in ExtractPlan.Batches.
	Index   int
	Status  ExtractBatchStatus
	Failure FailureKind
	// Err is the cause of a failed or cancelled batch. It can hold agent
	// output, so a surface must sanitize and redact it.
	Err error
	// Items are parallel to the planned batch's proposals.
	Items []ExtractItemOutcome
	// Named counts the items whose answer in a valid reply names a path
	// that the syntax rung accepts. It counts the whole reply before any
	// write, so a store failure in the batch does not lower it. The CLI
	// prints it in the batch outcome line, as it did before a write.
	Named int
	// PromptChars and ReplyChars meter the agent traffic. They stay 0
	// when no agent call started for the batch.
	PromptChars int
	ReplyChars  int
	// Started and Finished bound the batch's work. Both are zero for an
	// unattempted batch.
	Started  time.Time
	Finished time.Time
}

ExtractBatchOutcome is what happened to one planned agent call.

type ExtractBatchStatus added in v0.8.0

type ExtractBatchStatus int

ExtractBatchStatus is the result class of one planned agent call.

const (
	// BatchUnattempted means the call never started.
	BatchUnattempted ExtractBatchStatus = iota
	// BatchCompleted means the reply was valid and every item was
	// handled. An item can still be a conflict.
	BatchCompleted
	// BatchFailed means the call or a store write failed.
	// ExtractBatchOutcome.Failure says why.
	BatchFailed
	// BatchCancelled means a stop ended the batch before or during its
	// agent call.
	BatchCancelled
)

func (ExtractBatchStatus) String added in v0.8.0

func (s ExtractBatchStatus) String() string

String returns a stable lower-case name for the status.

type ExtractEvent added in v0.8.0

type ExtractEvent struct {
	Kind ExtractEventKind
	// Index is the 0-based position in ExtractPlan.Batches. For
	// ExtractStopped it is the first batch that did not run.
	Index int
	// Total is len(ExtractPlan.Batches), a known denominator.
	Total int
	// Batch is set on ExtractBatchDone. It points to a deep copy, slices
	// included, so a receiver on another goroutine never shares memory
	// with the run. Only Err values are shared, because errors are
	// immutable.
	Batch *ExtractBatchOutcome
	// Stop and Err are set on ExtractStopped.
	Stop StopReason
	Err  error
}

ExtractEvent is one typed progress report. A batch sends at most one ExtractBatchStarted and one ExtractBatchDone, and a run at most one ExtractStopped, so a receiver can size a queue for 2*batches+1 events.

type ExtractEventKind added in v0.8.0

type ExtractEventKind int

ExtractEventKind names one extraction progress event.

const (
	// ExtractBatchStarted precedes the agent call of one batch.
	ExtractBatchStarted ExtractEventKind = iota
	// ExtractBatchDone reports a batch outcome after its answers are
	// stored, so a receiver never shows an unsaved answer as done.
	ExtractBatchDone
	// ExtractStopped reports that no further batch runs.
	ExtractStopped
)

func (ExtractEventKind) String added in v0.8.0

func (k ExtractEventKind) String() string

String returns a stable lower-case name for the kind.

type ExtractExecuteOptions added in v0.8.0

type ExtractExecuteOptions struct {
	// Check runs before the first batch and before each batch: the
	// caller's cheap checks, such as the file hashes and the invocation.
	// A non-nil error stops the run with StopConflict. Nil checks nothing.
	Check func() error
	// OnEvent receives events synchronously on the executing goroutine,
	// never while a transaction is open. Nil discards the events.
	OnEvent func(ExtractEvent)
	// Now is the clock for the stamps and the batch times. Nil means
	// time.Now.
	Now func() time.Time
}

ExtractExecuteOptions tune one extraction execution.

type ExtractExecution added in v0.8.0

type ExtractExecution struct {
	// Batches are the outcomes, parallel to ExtractPlan.Batches.
	Batches []ExtractBatchOutcome
	Stop    StopReason
	StopErr error
	// Persisted lists the items with a stored answer, positive or
	// negative. The other lists name the items in each remaining class.
	// Every list is in send order.
	Persisted   []int64
	Failed      []int64
	Cancelled   []int64
	Conflicts   []int64
	Unattempted []int64
	// Result holds the legacy counters, derived from the outcomes.
	// Examined counts the items of the batches whose agent call started.
	Result ExtractResult
}

ExtractExecution is the typed result of one extraction execution.

func ExecuteExtract added in v0.8.0

func ExecuteExtract(ctx context.Context, st *store.Store, inv agent.Invoker, p *ExtractPlan,
	opts ExtractExecuteOptions,
) (*ExtractExecution, error)

ExecuteExtract runs exactly the planned batches. Before the first batch it checks ctx, revalidates the whole plan, runs opts.Check, and checks ctx again. Before each batch it checks ctx, runs opts.Check, compares the batch's rows and evidence with the store, and checks ctx again. A change stops the run with StopConflict, and the remaining items need a new plan. A row that changes during an agent call is a conflict too: its answer is discarded, the rest of its batch is stored, and the run stops, because another writer is active (D-20). A stop that ends an agent call cancels its batch; a call that already replied is stored for all its items.

It always returns a non-nil *ExtractExecution. The error is non-nil only for a store failure or an unusable call (no plan, no root, no invoker); a stop or a conflict is data in ExtractExecution.Stop, not an error.

type ExtractItem added in v0.8.0

type ExtractItem struct {
	// ID is the proposal ID, equal to the parallel proposal's ID.
	ID int64
	// Reason says why the question is open.
	Reason TriggerReason
	// Living are the member findings that are still in the index, with
	// metadata only. Region recomputation reads these frozen values.
	Living []model.Finding
	// Evidence are the evidence paths that the prompt names for the item,
	// after the shared batch budget.
	Evidence []string
	// Coverage is the evidence coverage of a pending row. A negative or
	// unconfirmed answer gives the row this region set. It is nil for an
	// applied row, whose regions never change here.
	Coverage []string
	// CoverageChanges reports that Coverage differs from the stored
	// regions of a pending row, so a negative answer changes them.
	CoverageChanges bool
	// contains filtered or unexported fields
}

ExtractItem is what one planned item sends and what a negative answer changes. It is parallel to ExtractBatch.Proposals.

type ExtractItemOutcome added in v0.8.0

type ExtractItemOutcome struct {
	// Proposal is the row as planned.
	Proposal model.Proposal
	Status   ExtractItemStatus
	Failure  FailureKind
	// Err is the cause of a failed, cancelled, or conflicting item. It can
	// hold agent output, so a surface must sanitize and redact it. It is
	// nil for a stored answer.
	Err error
	// Omitted is true when a valid reply had no entry for the item. The
	// prompt asks the model to list only the items that qualify, so the
	// item is a stored negative answer (D-19). A surface shows "omitted"
	// as the reason of that negative.
	Omitted bool
	// NamedRaw counts the paths of the item's answer as the model wrote
	// them. An answer of only invalid paths stays visible this way.
	NamedRaw int
	// Named are the answer's paths after the syntax rung.
	Named []string
	// Valid are the named paths that exist in the working tree. They are
	// the stored paths when Status is ExtractStored.
	Valid []string
	// Facts describe each valid path under today's history: direct,
	// confirmed by co-change, or unverified, and selected or not.
	Facts []TriggerFact
	// RegionsBefore and RegionsAfter are the effective region sets before
	// and after the answer. They differ only for a pending row whose
	// regions this run changed; an applied row keeps its regions.
	RegionsBefore  []string
	RegionsAfter   []string
	RegionsChanged bool
}

ExtractItemOutcome is what happened to one planned item.

type ExtractItemStatus added in v0.8.0

type ExtractItemStatus int

ExtractItemStatus is the result class of one planned item.

const (
	// ExtractUnattempted means the item was never sent: a stop, a
	// conflict, or an earlier store failure ended the run first.
	ExtractUnattempted ExtractItemStatus = iota
	// ExtractStored means a positive answer was stored and stamped.
	ExtractStored
	// ExtractNegative means a negative answer was stored and stamped: an
	// empty list, only invalid or absent paths, or no entry in a valid
	// reply (ExtractItemOutcome.Omitted). The question is closed under the
	// current prompt version.
	ExtractNegative
	// ExtractConflict means the row changed during the agent call, so the
	// answer was discarded and nothing was written. It also stops the run.
	ExtractConflict
	// ExtractFailed means the item has no stored answer: the agent call
	// failed, the reply was not valid, or a store write failed. The
	// question stays open.
	ExtractFailed
	// ExtractCancelled means a stop ended the item's batch before or
	// during its agent call. The question stays open.
	ExtractCancelled
)

func (ExtractItemStatus) String added in v0.8.0

func (s ExtractItemStatus) String() string

String returns a stable lower-case name for the status.

type ExtractOptions added in v0.4.0

type ExtractOptions struct {
	// Root is the workspace root; named paths are verified against it.
	Root string
	// DryRun stops after the preflight: nothing is sent.
	DryRun bool
	// Agent is the resolved agent command line, for disclosure only.
	Agent []string
	// OnPreflight receives the plan before the first call — always,
	// so sending note text to a model-backed CLI is never a surprise.
	OnPreflight func(ExtractPreflight)
	// OnBatchStart/OnBatchDone drive interactive surfaces through the
	// long silent stretch of each agent call — the same contract as
	// distill's group callbacks. When nil, the same information flows
	// through Logf as plain lines.
	OnBatchStart func(desc string)
	OnBatchDone  func(outcome string)
	// Logf receives progress; nil discards it.
	Logf func(format string, args ...any)
}

ExtractOptions configures ExtractTriggers.

type ExtractPlan added in v0.8.0

type ExtractPlan struct {
	Options ExtractPlanOptions
	// Batches are the agent calls in execution order. NewExtractPlan sends
	// pending rows newest first, then applied rows newest first, as the
	// CLI orders them. ExtractTriggers keeps the order of its caller.
	Batches []ExtractBatch
	// Ineligible lists the selected proposals that are not asked, in
	// ascending ID order.
	Ineligible []ExtractSkip
	Snapshot   ExtractSnapshot
	// Preflight is the disclosure total. Its BodyCap is 0, because the
	// plan reads finding metadata only and sends no finding text. The
	// CLI disclosure of ExtractTriggers keeps its excerpt cap, because
	// its goldens pin that text (A0 defect D4).
	Preflight ExtractPreflight
	PlannedAt time.Time
}

ExtractPlan is one frozen extraction run.

func NewExtractPlan added in v0.8.0

func NewExtractPlan(st *store.Store, opts ExtractPlanOptions) (*ExtractPlan, error)

NewExtractPlan selects the open questions among opts.IDs and freezes their batches and prompts. It reads finding metadata only, as the CLI does, so no finding text is sent. A selected proposal is not asked when it is not found, not pending or applied, already answered under the current question, excluded by opts.Exclude, or without living evidence. NewExtractPlan only reads the store, so a read-only store works. It never invokes an agent and never stamps a row.

func (*ExtractPlan) IDs added in v0.8.0

func (p *ExtractPlan) IDs() []int64

IDs returns the planned proposal IDs in send order.

func (*ExtractPlan) Revalidate added in v0.8.0

func (p *ExtractPlan) Revalidate(st *store.Store) error

Revalidate reads the planned rows and their findings again. It returns an error that wraps ErrExtractStale and names each changed input. A read failure returns that error unwrapped, because it says nothing about staleness.

type ExtractPlanOptions added in v0.8.0

type ExtractPlanOptions struct {
	// IDs are the explicitly selected proposals. The plan never adds a
	// proposal, so an empty list plans no call.
	IDs []int64
	// Root is the workspace root. Named paths are verified against this
	// tree. NewExtractPlan refuses "" (see ErrExtractRoot).
	Root string
	// Spec is the agent invocation the plan discloses. ExecuteExtract
	// runs the invoker its caller passes, so the caller must build that
	// invoker from this spec.
	Spec agent.CommandSpec
	// Exclude adds the caller's own exclusions, such as an absent or
	// ambiguous pin. It runs only for pending and applied rows whose
	// question is open. It returns the reason and true to exclude the
	// row. Nil excludes nothing more.
	Exclude func(model.Proposal) (reason string, excluded bool)
}

ExtractPlanOptions are the frozen inputs of one extraction plan.

type ExtractPreflight added in v0.4.0

type ExtractPreflight struct {
	Proposals   int
	Batches     int
	PromptChars int
	Agent       []string
	BodyCap     int
}

ExtractPreflight is the disclosure: what would be sent, where, and roughly what it costs.

func (ExtractPreflight) Tokens added in v0.4.0

func (p ExtractPreflight) Tokens() string

Tokens estimates the preflight's prompt volume.

type ExtractResult added in v0.4.0

type ExtractResult struct {
	Examined   int // proposals sent to the agent
	Named      int // proposals whose reply named at least one path
	Stored     int // proposals with at least one validated path stored
	Retargeted int // pending rows whose regions changed
	// AppliedStored counts applied pins that stored new triggers —
	// the rows whose drift the user must apply with --retarget. An
	// explicit counter: inferring it from Stored minus Retargeted
	// miscounts pending rows whose regions did not change.
	AppliedStored int
	BatchesFailed int // batches lost to agent or parse errors (retryable)
	// Conflicts counts the proposals that another writer changed during
	// their agent call. The run discarded their answers and wrote nothing
	// to them.
	Conflicts int
	// Unattempted counts the proposals that a stop left unsent. The run
	// wrote nothing to them, so a later run can ask them.
	Unattempted int
	PromptChars int
	ReplyChars  int
	Duration    time.Duration
}

ExtractResult reports what one extraction run did.

func ExtractTriggers added in v0.4.0

func ExtractTriggers(ctx context.Context, st *store.Store, inv agent.Invoker,
	ps []model.Proposal, meta map[int64]model.Finding, opts ExtractOptions,
) (*ExtractResult, error)

ExtractTriggers runs the backfill over the given proposals. The caller selects them (the idempotency filter — rows already carrying triggers — belongs there) and provides the living-findings map the other surfaces already hold. Batches fail independently: a lost batch is logged and retried on the next run, like a distill group. On a store error it returns the partial result together with the error. The counters name what completed before the failure, and stamped rows stay done.

ExtractTriggers is the CLI entry. It plans and executes in one call. It shares the plan step of NewExtractPlan and the ExecuteExtract step of a reviewed extraction. A legacy adapter turns the typed events into the callback strings that the CLI prints. The plan asks every given row in the given order. It does not apply the console exclusions, such as aged-out evidence or a duplicate pin. The CLI keeps this wider set until a separate decision moves it to the console selection (A0 defects D5 and D6).

The executor checks the rows before each batch and saves each answer with the row's guard, as in the console. A row that another writer changed is a conflict, not an error: the run leaves the row unchanged and stops. Before, the CLI stamped such a row anyway (A0 defect D14). The counters and the conflict lines report it.

func (ExtractResult) TokensBack added in v0.4.0

func (r ExtractResult) TokensBack() string

TokensBack estimates the run's reply traffic for the summary line.

func (ExtractResult) TokensSent added in v0.4.0

func (r ExtractResult) TokensSent() string

TokensSent estimates the run's prompt traffic for the summary line.

type ExtractSkip added in v0.8.0

type ExtractSkip struct {
	ID     int64
	Reason string
}

ExtractSkip is one selected proposal that the plan does not ask.

type ExtractSnapshot added in v0.8.0

type ExtractSnapshot struct {
	Rows     [32]byte
	Findings [32]byte
}

ExtractSnapshot identifies the store inputs of a plan: every planned row with its trigger stamp, and every living member finding as the prompts show it.

type FailureKind added in v0.8.0

type FailureKind int

FailureKind classifies a failed group, so a surface can show a distinct and actionable outcome for each cause.

const (
	// FailNone is the kind of every group that did not fail.
	FailNone FailureKind = iota
	// FailAgent means the invoker returned an error while the run was
	// live: a missing login, a rate limit, or an exit status.
	FailAgent
	// FailTimeout means the per-group time limit ended the call.
	FailTimeout
	// FailInvalidReply means the reply was not the requested JSON.
	FailInvalidReply
	// FailStore means a store read or write failed after the reply. The
	// run stops, because later groups would build on unsaved patterns.
	FailStore
)

func (FailureKind) String added in v0.8.0

func (k FailureKind) String() string

String returns a stable lower-case name for the kind.

type FindingPlan added in v0.5.0

type FindingPlan struct {
	ID     int64
	Source string
	PR     int
	Path   string
	Paths  []string
}

FindingPlan identifies one finding without disclosing its body. It lets a user verify the evidence boundary before approving a model call.

type Group

type Group struct {
	// Region is the deepest directory common to every member, "" when
	// the members span top-level trees (a repo-wide theme batch).
	Region string
	// Findings are the members, ordered by id.
	Findings []model.Finding
	// Signature identifies the evidence set: the hash of the member
	// ids. Same members — same signature, regardless of ordering or of
	// anything outside the group.
	Signature string
	// Area marks a directory bucket: thematically unconnected findings
	// batched so distillation still covers them. Membership in an area
	// group only means "same directory", so consumers that treat group
	// membership as "same mistake" (the outcome loop) must skip these.
	Area bool
}

Group is one distillation batch: findings that plausibly share a theme (or at least a neighborhood), with a stable identity.

type GroupOutcome added in v0.8.0

type GroupOutcome struct {
	// Plan is the group's preflight entry.
	Plan    GroupPlan
	Status  GroupStatus
	Failure FailureKind
	// Err is the cause of a failed, cancelled, or skipped group. It can
	// hold agent output, so a surface must sanitize and redact it.
	Err error
	// Proposals are the persisted rows with their IDs.
	Proposals  []model.Proposal
	Duplicates []Duplicate
	// PromptChars and ReplyChars meter the agent traffic. They stay 0
	// when no agent call started for the group.
	PromptChars int
	ReplyChars  int
	// Started and Finished bound the group's work. Both are zero for an
	// unattempted group.
	Started  time.Time
	Finished time.Time
}

GroupOutcome is what happened to one planned group.

type GroupPlan

type GroupPlan struct {
	Signature   string
	Region      string
	Findings    int
	PromptChars int
	BodyCap     int
	Evidence    []FindingPlan
}

GroupPlan is one group's slice of the preflight.

type GroupStatus added in v0.8.0

type GroupStatus int

GroupStatus is the result class of one planned group.

const (
	// GroupUnattempted means the group never started: a stop, a
	// conflict, or an earlier store failure ended the run first.
	GroupUnattempted GroupStatus = iota
	// GroupCompleted means the reply was valid and its proposals and
	// signature mark were committed, with zero to five proposals.
	GroupCompleted
	// GroupFailed means the group is not marked; a later plan selects
	// it again. GroupOutcome.Failure says why.
	GroupFailed
	// GroupCancelled means a stop ended the group before or during its
	// agent call. The group is not marked, and the run stops with
	// StopCancelled.
	GroupCancelled
	// GroupSkipped means another writer recorded the group, so this run
	// stored nothing for it.
	GroupSkipped
)

func (GroupStatus) String added in v0.8.0

func (s GroupStatus) String() string

String returns a stable lower-case name for the status.

type Grouper

type Grouper interface {
	Group(findings []model.Finding) []Group
}

Grouper buckets findings into candidate groups. Implementations must be deterministic: the same findings yield the same groups with the same signatures, in the same order.

func NewLexicalGrouper

func NewLexicalGrouper() Grouper

NewLexicalGrouper returns the default token-overlap Grouper.

Three shared tokens form an unconstrained strong edge. Two-token edges are useful wording bridges (the pooled-state corpus needs them), but may only build a small component. This two-tier rule prevents the weak transitive chains observed in large public repositories without giving up the recall that motivated lexical grouping.

type Known

type Known struct {
	// contains filtered or unexported fields
}

Known is the set of patterns already captured: every pin in lessons.yaml (hand-written ones included) and every proposal already decided or pending. A distilled pattern matching one of these is not news — including one the user dismissed, since a dismissal is a decision the distiller must not relitigate.

func NewKnown

func NewKnown(patterns ...[]model.Proposal) *Known

NewKnown builds the set. Pins arrive as Proposals carrying just Rule and Note, so config pins and stored proposals share one shape.

func (*Known) Add

func (k *Known) Add(p model.Proposal)

Add records a pattern as known — used as a run proceeds, so two groups in the same pass cannot both propose the same thing.

func (*Known) Labels

func (k *Known) Labels(region string, limit int) []string

Labels lists the rule names already captured for an area, newest first, at most limit of them. They are fed to the distiller so it can skip what is known and spend the call looking for something else — labels only, because the notes would cost more tokens than the duplicates they prevent.

func (*Known) Restated

func (k *Known) Restated(p model.Proposal) (string, bool)

Restated returns the label of the known pattern p duplicates, and whether it duplicates one at all. Evidence identity is checked before wording: it is the stronger claim, and it catches the pairs wording cannot.

Identity deliberately skips patterns from p's own reply batch (same signature): one finding can flag two mistakes, so a model reading a group may honestly cite the same members for distinct patterns — the wording check still guards that batch against padding. Across batches the same citations mean the corpus was re-carved and the theme re-derived, which is exactly the duplication to stop.

type Options

type Options struct {
	// Region restricts the finding corpus before grouping ("" =
	// everywhere). This keeps unrelated findings outside a requested tree
	// from changing token frequencies, bridging components, or consuming a
	// budgeted call.
	Region string
	// Limit caps how many new groups one run reads (0 = all). The cap
	// is the budget lever: each group is one agent invocation.
	Limit int
	// Logf receives progress; nil discards it.
	Logf func(format string, args ...any)
	// OnGroupStart/OnGroupDone drive interactive surfaces through the
	// long silent stretch of each agent call: start fires with the
	// group's description before the call, done with the outcome after.
	// When nil, the same information flows through Logf as plain lines.
	OnGroupStart func(desc string)
	OnGroupDone  func(outcome string)
	// Pins are the patterns already captured in .seamark/lessons.yaml,
	// hand-written ones included, as Rule/Note pairs. A distilled
	// pattern that restates one of them is dropped: groups are read
	// independently, so a repo-wide mistake would otherwise be
	// re-proposed under a new name by every group it appears in.
	Pins []model.Proposal
	// Agent is the resolved agent command line, for the preflight
	// disclosure only — the Invoker is what actually runs it.
	Agent []string
	// Root is the workspace root. Trigger paths named by the agent are
	// verified against this tree; empty drops them all — an unverified
	// name must not reach storage.
	Root string
	// OnPreflight receives the plan before the first agent call: what
	// would be sent, where, and roughly what it would cost.
	OnPreflight func(Preflight)
	// DryRun stops after the preflight: nothing is sent, marked, pruned,
	// or persisted. The Invoker may be nil on a dry run.
	DryRun bool
}

Options tunes one distillation run.

type Plan added in v0.8.0

type Plan struct {
	Options PlanOptions
	// Preflight is the disclosure of the selected groups. It is the
	// same value that the legacy OnPreflight callback receives.
	Preflight Preflight
	// Groups are the selected groups in execution order, parallel to
	// Preflight.Groups.
	Groups []PlannedGroup
	// Cleanup lists the pending proposals whose evidence group no longer
	// exists. Execute deletes exactly these rows before the first group.
	Cleanup []model.Proposal
	// Skipped lists the signatures already distilled, in coverage order.
	Skipped []string
	// Deferred lists the groups beyond the limit, in coverage order. An
	// entry carries identity and size only: Signature, Region, Findings,
	// and BodyCap. Its prompt is not built, so PromptChars and Evidence
	// stay empty.
	Deferred []GroupPlan
	// Total counts every group in scope, skipped and deferred included.
	Total int
	// Corpus counts the findings in scope before grouping. Findings
	// without a group are not sent, and this count makes that visible.
	Corpus    int
	Snapshot  Snapshot
	PlannedAt time.Time
	// contains filtered or unexported fields
}

Plan is one frozen distillation run: what it sends, in which order, what it skips, and which pending proposals it removes first.

func NewPlan added in v0.8.0

func NewPlan(st *store.Store, grouper Grouper, opts PlanOptions) (*Plan, error)

NewPlan groups, orders, and selects evidence exactly as a real run does after its stale-proposal cleanup. It models that cleanup instead of doing it: the pending rows to delete are listed in Plan.Cleanup and left out of the known patterns. NewPlan only reads the store, so a read-only store works. It never invokes an agent, prunes, or marks.

func (*Plan) Revalidate added in v0.8.0

func (p *Plan) Revalidate(st *store.Store) error

Revalidate reads the store again and compares every snapshot digest except Pins, which the store does not hold. It returns an error that wraps ErrStale and names each changed input. A read failure returns that error unwrapped, because it says nothing about staleness.

type PlanOptions added in v0.8.0

type PlanOptions struct {
	// Region restricts the corpus before grouping. It must already be
	// repository-relative, as NormalizeRegion returns it; "" means the
	// whole repository.
	Region string
	// Limit caps how many new groups the plan selects; 0 selects all.
	// NewPlan rejects a negative limit.
	Limit int
	// Pins are the patterns of .seamark/lessons.yaml as proposals, as
	// PinsFromConfig returns them.
	Pins []model.Proposal
	// Spec is the agent invocation the plan discloses. Execute runs the
	// invoker its caller passes, so the caller must build that invoker
	// from this spec.
	Spec agent.CommandSpec
	// Root is the workspace root. Trigger paths are verified against
	// this tree; "" drops them all.
	Root string
}

PlanOptions are the frozen inputs of one plan.

type PlannedGroup added in v0.8.0

type PlannedGroup struct {
	Group Group
	// Digest identifies the member findings as the prompt would show
	// them. A boundary check compares it with the stored rows.
	Digest [32]byte
}

PlannedGroup is one selected group with its members frozen. Execute sends exactly these findings, never a later version of the group.

type Preflight

type Preflight struct {
	// Agent is the command line each group's prompt would be piped to.
	Agent []string
	// Groups describes each evidence group that would be sent this run.
	Groups []GroupPlan
	// PromptChars totals the prompts exactly as they would be sent now.
	// Execution primes later prompts with labels learned from earlier
	// groups, so treat it as a close lower bound.
	PromptChars int
	// Findings counts the finding bodies across all planned groups.
	Findings int
	// BodyCap is the largest per-finding evidence cap across planned groups.
	// Groups share a fixed total budget, so smaller groups can show more
	// of each finding. Path metadata and body share this cap; dispatch
	// reapplies best-effort secret redaction before the body is sent.
	BodyCap int
}

Preflight is the pre-invocation disclosure. Metadata only — never finding bodies.

func (Preflight) Tokens

func (p Preflight) Tokens() string

Tokens renders the preflight's approximate token cost.

type Result

type Result struct {
	GroupsTotal   int // candidate groups in the current corpus
	GroupsSkipped int // already distilled (signature known)
	GroupsRead    int // sent to the agent this run
	GroupsFailed  int // agent or parse errors (not marked; retried next run)
	GroupsPending int // new groups left unread (limit or region filter)
	PrunedStale   int // pending proposals dropped because their group changed
	// StalePending lists the IDs of the stale pending proposals that a
	// real run prunes first. Only a dry run fills it, because it plans
	// as if they were gone but must not delete them.
	StalePending []int64
	// PromptChars/ReplyChars meter the run's agent traffic — the basis
	// for the ~token estimate shown to the user. Failed groups count
	// too: their cost was paid.
	PromptChars int
	ReplyChars  int
	Duration    time.Duration
	// Duplicates counts distilled patterns dropped as restatements of
	// something already pinned, proposed, or decided.
	Duplicates int
	Proposals  []model.Proposal
}

Result reports what a run did.

func Run

func Run(ctx context.Context, st *store.Store, grouper Grouper, inv agent.Invoker, opts Options) (*Result, error)

Run executes the plan half of distillation: group the findings, skip evidence sets already read, send each remaining group to the agent, validate what comes back, and persist the survivors as proposals. It never touches .seamark/lessons.yaml — applying is a separate, operator decision.

Run is the CLI entry. It plans and executes in one call. It shares the planning step of NewPlan and the Execute step of a reviewed run. A legacy adapter turns the typed events into the callback strings that the CLI prints. A real run prunes stale pending proposals during planning, at the point where it always did. A dry run models that prune, so its preflight names the groups that the real run sends.

func (*Result) CostNote

func (r *Result) CostNote() string

CostNote renders what the run cost: estimated tokens both ways and wall time. Empty when no agent traffic happened — a fully-skipped run was free and says nothing.

type ScopeAdvice added in v0.8.0

type ScopeAdvice struct {
	Kind ScopeKind
	// Regions is the computed set in the order of the region engine. Nil
	// means repository-wide. It is nil for ScopeNoEvidence too; RepoWide
	// tells the two cases apart.
	Regions []string
	// Facts are the trigger facts of RecomputeRegions: which stored
	// trigger paths the evidence or the history confirms, and which of
	// them the region set delivers.
	Facts []TriggerFact
	// RepoWide is true when the computed set is repository-wide.
	RepoWide bool
	// Broader is true for a change that reaches a path the recorded set
	// does not reach. A preview shows it as a warning.
	Broader bool
	// Narrower is true for a change that no longer reaches a path the
	// recorded set reaches.
	Narrower bool
}

ScopeAdvice is the computed scope of one proposal with its reasons.

func RecommendScope added in v0.8.0

func RecommendScope(st *store.Store, root string, p model.Proposal, living []model.Finding) (ScopeAdvice, error)

RecommendScope computes the scope that a retarget writes for p. No living finding gives ScopeNoEvidence. Otherwise RecomputeRegions gives the set. A set equal to p.RegionSet() under the pin-key equivalence gives ScopeCurrent. living holds the cited findings that are still in the index. It writes nothing; it reads the store (co-change history) and the working tree under root.

type ScopeAdvisory added in v0.4.0

type ScopeAdvisory struct {
	// NotePath is the path the note names, outside the regions. It
	// exists in the working tree — a missing path is never a target.
	NotePath string

	// Partner is the co-change partner file that agrees with
	// NotePath: equal to it, or under it when NotePath is a
	// directory. Co-change names files; notes often name directories.
	Partner string

	// Evidence is the cited finding file whose partner agreed.
	Evidence string

	// Together counts the shared commits on the Evidence–Partner
	// edge, so the printed advisory can show its strength.
	Together int

	// Suggested is the pin's current region set with the trigger
	// region appended. Regions the trigger region contains leave
	// first: delivery regions form a union, so a contained region
	// adds nothing and wastes a cap slot. Survivors keep their order
	// — order only feeds the legacy region: field; pin identity
	// sorts regions. Nil when the result would still pass
	// maxRegions: the user decides what to remove.
	Suggested []string
}

ScopeAdvisory says that a pin's delivery regions likely miss the trigger site. Two signals must agree on the same place: the note names a repo path outside the regions, and the co-change history of a cited evidence file points at it too. One signal alone is not enough.

func AuditScope added in v0.4.0

func AuditScope(st *store.Store, root, note string, regions []string, cited []model.Finding) (ScopeAdvisory, bool, error)

AuditScope runs the trigger-scope check for one pin. regions must be the pin's live region set; nil means repo-wide, which already delivers everywhere, so the check reports nothing. When several partner edges agree, the one with the most shared commits wins. ok is false when the signals do not agree — the common case.

func (ScopeAdvisory) Line added in v0.4.0

func (a ScopeAdvisory) Line() string

Line renders the advisory sentence

type ScopeKind added in v0.8.0

type ScopeKind int

ScopeKind classifies the scope that today's inference computes for a proposal against the scope that the proposal records.

const (
	// ScopeNoEvidence means that no cited finding is living, so nothing
	// can be recomputed. The recorded scope stays: missing evidence is
	// never a reason to widen a pin. It is the zero value, so an unset
	// advice never reads as a change.
	ScopeNoEvidence ScopeKind = iota
	// ScopeCurrent means that the computed set equals the recorded set.
	ScopeCurrent
	// ScopeChanged means that the computed set differs from the recorded
	// set. A retarget can apply it.
	ScopeChanged
)

func (ScopeKind) String added in v0.8.0

func (k ScopeKind) String() string

String returns a stable lower-case phrase for the kind.

type Snapshot added in v0.8.0

type Snapshot struct {
	// Corpus covers the findings in scope, with every field that reaches
	// grouping or the prompt.
	Corpus [32]byte
	// Ledger covers every stored proposal: the cleanup set, the known
	// patterns for deduplication, and the coverage order all read it.
	Ledger [32]byte
	// Distilled covers the recorded group signatures, the skip set.
	Distilled [32]byte
	// Pins covers PlanOptions.Pins. The store holds no pins, so
	// Revalidate does not check it; the caller guards the pin file.
	Pins [32]byte
}

Snapshot identifies the store inputs a plan was made from. Revalidate recomputes each digest and names every one that differs.

type StopReason added in v0.8.0

type StopReason int

StopReason says why an execution ended before its last group.

const (
	// StopNone means every planned group had its turn and no stop
	// cancelled one. A stop that comes after the last reply prevents no
	// work, so the run keeps StopNone.
	StopNone StopReason = iota
	// StopCancelled means the context ended before the cleanup, before
	// a group, or during a group. The position of the group does not
	// matter: a stop in the last group also gives StopCancelled.
	StopCancelled
	// StopConflict means an input changed; StopErr wraps ErrStale.
	StopConflict
	// StopStoreError means a store read or write failed.
	StopStoreError
)

func (StopReason) String added in v0.8.0

func (r StopReason) String() string

String returns a stable lower-case name for the reason.

type TriggerFact added in v0.4.0

type TriggerFact struct {
	Path     string // the stored trigger path
	Region   string // its exact delivery region; "" for vanished paths
	Together int    // strongest confirming co-change edge; 0 = unconfirmed
	Direct   bool   // true when the trigger is itself cited evidence
	Selected bool   // true when the recomputed delivery set reaches it
}

TriggerFact reports one stored trigger path's state under today's history, for the plan and ledger surfaces.

func RecomputeRegions added in v0.4.0

func RecomputeRegions(st *store.Store, root string, p model.Proposal, living []model.Finding,
) ([]string, []TriggerFact, error)

RecomputeRegions returns the regions today's inference assigns a proposal. Verified trigger paths are the most precise delivery surface: a trigger is accepted when it is directly cited evidence or when history confirms it co-changes with that evidence. If none are accepted, evidence coverage is the conservative fallback. Every reader of "regions now" — distillation, the ledger, --retarget — must use this one function. root locates the working tree and rejects vanished trigger paths.

func (TriggerFact) BlockedLine added in v0.4.0

func (f TriggerFact) BlockedLine() string

BlockedLine renders the confirmed-but-undeliverable sentence every surface prints — the plan, the ledger, and the HTML report must not phrase the same fact differently. Empty for selected or unconfirmed facts.

type TriggerReason added in v0.8.0

type TriggerReason int

TriggerReason says why the trigger question of a proposal is open or answered under the current TriggerPromptVersion.

const (
	// TriggerOpen means the question was never answered.
	TriggerOpen TriggerReason = iota
	// TriggerOpenObsoleteNegative means the stored answer is a negative
	// answer to an older question. The current question asks again once.
	TriggerOpenObsoleteNegative
	// TriggerAnsweredPositive means validated trigger paths are stored.
	// A positive answer is never bought again, whatever its version.
	TriggerAnsweredPositive
	// TriggerAnsweredCurrentNegative means the current question was
	// answered and named no valid path.
	TriggerAnsweredCurrentNegative
)

func TriggerQuestionOpen added in v0.8.0

func TriggerQuestionOpen(p model.Proposal) (bool, TriggerReason)

TriggerQuestionOpen reports whether the trigger question of p is open, and why. It reads only the trigger fields of p: the status and the pin file are the caller's checks. Stored paths close the question at any version. A negative answer closes it only at TriggerPromptVersion or later, because a newer question can change a negative answer.

func (TriggerReason) String added in v0.8.0

func (r TriggerReason) String() string

String returns a stable lower-case phrase for the reason. A surface can show it as the reason that a proposal is or is not asked.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL