Documentation
¶
Overview ¶
Package ideator — orchestrator. See source.go for the IdeaSource interface and Idea wire shape.
Run executes every enabled source in parallel under a shared context, dedupes by Idea.DedupeKey (first-write-wins), scores by SuggestedPriority, and returns the top-K. RunAndQueue is the same pipeline plus a Propose call into autopilot per surviving Idea — every queued row lands at StatusProposed.
Package ideator — top of clawtool's three-layer self-direction stack:
Ideator ──── what to work on (mines repo signals → Idea list) Autopilot ─── when to work on it (proposed → pending → claimed) Autonomous ── how to work on it (one goal → bounded iteration loop)
The Ideator surveys cheap, repo-local signals that the operator would normally surface by hand — open ADR questions, TODO/FIXME comments, recent CI failures, doc/manifest drift, benchmark regressions — and emits Idea structs ranked by signal strength.
Output is NEVER auto-claimed. RunAndQueue posts every selected Idea into the autopilot backlog at status=proposed; only an operator running `clawtool autopilot accept <id>` (or the MCP AutopilotAccept tool) can flip the row to pending. That gate is the safety boundary — it pins the agent to "I can suggest, the human approves, then I work" rather than the unbounded loop the 2025-11 autonomy literature warns against.
Sources are pluggable via the IdeaSource interface. New signals (e.g. "dependabot alerts", "log analytics anomaly") drop in as a new file under internal/ideator/sources/ + a registration in the CLI wire-up (DefaultSources is assembled at the edge to avoid an import cycle between this package and its sources subpackage).
Each source MUST be cheap-on-fail: missing tool, missing dir, network error → empty result + warning log, never a hard error that takes down the whole orchestrator.
Index ¶
Constants ¶
const DefaultMaxConcurrency = 4
DefaultMaxConcurrency caps source parallelism by default. 4 is the sweet spot in benchmarks: above ~4 concurrent fork-execs the /mnt/c (Windows-mounted) and afs/sshfs filesystems start to thrash from concurrent stat/open syscalls; below 2 the slowest source (govulncheck @ 16s) dominates wall time. 4 keeps fast-host wall time within ~10% of the slowest source while bounding the pathological case.
const DefaultTopK = 10
DefaultTopK is the cap when Options.TopK is unset.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Idea ¶
type Idea struct {
// Title is a short label for `ideate` print output and the
// queue list. Empty → orchestrator derives one from the first
// line of SuggestedPrompt.
Title string
// Summary is a 1–3 sentence rationale; the operator reads this
// when deciding whether to Accept. Stored as Item.Note in the
// queue.
Summary string
// SourceName names the IdeaSource that produced this Idea.
// Optional in the source-side struct; orchestrator backfills
// from IdeaSource.Name() when empty.
SourceName string
// Evidence carries the file:line / test name / ADR section
// reference / bench-baseline diff — anything the operator can
// grep to verify the suggestion isn't fabricated.
Evidence string
// SuggestedPriority is the base score (typically 0..10);
// orchestrator may apply a small evidence-recency bump before
// final ranking.
SuggestedPriority int
// SuggestedPrompt is the verbatim text the agent will receive
// when it eventually claims the item. Written in the second
// person ("Investigate the TODO at ..."). Empty → idea is
// silently dropped by the orchestrator.
SuggestedPrompt string
// DedupeKey is a stable hash combining source + evidence;
// re-running ideate over the same repo state must not multiply
// the queue. Source authors hash once at construction; the
// orchestrator AND queue.Propose both deduplicate by this key.
DedupeKey string
}
Idea is one feature candidate the Ideator emits.
type IdeaSource ¶
type IdeaSource interface {
Name() string
Scan(ctx context.Context, repoRoot string) ([]Idea, error)
}
IdeaSource is the pluggable signal-mining contract. Implementations live under internal/ideator/sources/ and are wired into the orchestrator at the CLI / MCP edge (see internal/cli/ideate.go's defaultSources()). Two ground rules:
Scan MUST be cheap-on-fail. Missing dir / missing CLI tool / timed-out network → empty slice + nil error. The orchestrator surfaces this via the per-source warning log; one broken source must never poison the rest.
Scan MUST respect ctx.Done(). The orchestrator runs all sources in parallel under a single deadline; ignoring the ctx blocks `clawtool ideate` past the operator's patience.
type Options ¶
type Options struct {
RepoRoot string
SourceFilter string
TopK int
Sources []IdeaSource
Warn io.Writer
// MaxConcurrency caps how many sources run at once. 0 → use
// DefaultMaxConcurrency. Sources fork-exec heavy CLIs (gh,
// govulncheck, deadcode); on slow filesystems (WSL2 /mnt/c,
// network shares) unbounded parallelism saturates I/O and
// bloats wall time 5× vs. running individually. The default
// is conservative — operators on fast hosts can raise it via
// CLAWTOOL_IDEATOR_MAX_CONCURRENCY.
MaxConcurrency int
// SuppressDryDiagnostic disables the synthetic meta-Idea that
// fires when every configured source returns zero. Default
// false — the diagnostic is exactly what the autonomous loop
// needs so it doesn't go structurally silent. Set to true for
// operator-driven `clawtool ideate` invocations where an empty
// result should print "no ideas" honestly without pollution.
SuppressDryDiagnostic bool
}
Options is the orchestrator entry-point shape.
SourceFilter (when non-empty) restricts execution to a single source by name — drives `clawtool ideate --source adr_questions`. Empty means "all enabled sources".
TopK caps the returned slice (after dedupe + score). 0 means the package default (DefaultTopK).
Sources defaults to DefaultSources(); tests override it to inject stubs.
Warn receives one line per cheap-on-fail (missing CLI, missing dir, scan error). Defaults to io.Discard when nil — the orchestrator never panics on a quiet caller.
type RunResult ¶
type RunResult struct {
Ideas []Idea `json:"ideas"`
PerSource map[string]int `json:"per_source"`
Deduped int `json:"deduped"`
Added int `json:"added"`
Skipped int `json:"skipped"`
SourceErrors map[string]string `json:"source_errors,omitempty"`
}
RunResult is the wire shape returned by Run / RunAndQueue. Counts surface what the orchestrator did to the operator without making them re-run with -v: how many ideas each source emitted, how many got deduped, how many ended up queued (RunAndQueue only).
func Run ¶
Run walks every (filtered) source in parallel, dedupes, scores, and returns the top-K. No side effects on the autopilot queue — use RunAndQueue for that. Caller MUST populate opts.Sources; the CLI / MCP edge wires the canonical bundle (see internal/cli/ideate.go's defaultSources()) so this package stays free of an import cycle into internal/ideator/sources.
func RunAndQueue ¶
RunAndQueue runs the orchestrator then writes each surviving Idea into the autopilot queue at StatusProposed. q may be nil — in which case autopilot.Open() is used (the default per-host store). Returns the same RunResult shape with Added/Skipped populated.
Skipped counts items the queue rejected as duplicates (DedupeKey already lived on a non-terminal proposed/pending/in_progress row). That's by design: re-running ideate after operator inaction must be idempotent.