ideator

package
v0.22.147 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 2, 2026 License: MIT Imports: 10 Imported by: 0

Documentation

Overview

Package ideator — orchestrator. See source.go for the IdeaSource interface and Idea wire shape.

Run executes every enabled source in parallel under a shared context, dedupes by Idea.DedupeKey (first-write-wins), scores by SuggestedPriority, and returns the top-K. RunAndQueue is the same pipeline plus a Propose call into autopilot per surviving Idea — every queued row lands at StatusProposed.

Package ideator — top of clawtool's three-layer self-direction stack:

Ideator  ──── what to work on (mines repo signals → Idea list)
Autopilot ─── when to work on it (proposed → pending → claimed)
Autonomous ── how to work on it (one goal → bounded iteration loop)

The Ideator surveys cheap, repo-local signals that the operator would normally surface by hand — open ADR questions, TODO/FIXME comments, recent CI failures, doc/manifest drift, benchmark regressions — and emits Idea structs ranked by signal strength.

Output is NEVER auto-claimed. RunAndQueue posts every selected Idea into the autopilot backlog at status=proposed; only an operator running `clawtool autopilot accept <id>` (or the MCP AutopilotAccept tool) can flip the row to pending. That gate is the safety boundary — it pins the agent to "I can suggest, the human approves, then I work" rather than the unbounded loop the 2025-11 autonomy literature warns against.

Sources are pluggable via the IdeaSource interface. New signals (e.g. "dependabot alerts", "log analytics anomaly") drop in as a new file under internal/ideator/sources/ + a registration in the CLI wire-up (DefaultSources is assembled at the edge to avoid an import cycle between this package and its sources subpackage).

Each source MUST be cheap-on-fail: missing tool, missing dir, network error → empty result + warning log, never a hard error that takes down the whole orchestrator.

Index

Constants

View Source
const DefaultMaxConcurrency = 4

DefaultMaxConcurrency caps source parallelism by default. 4 is the sweet spot in benchmarks: above ~4 concurrent fork-execs the /mnt/c (Windows-mounted) and afs/sshfs filesystems start to thrash from concurrent stat/open syscalls; below 2 the slowest source (govulncheck @ 16s) dominates wall time. 4 keeps fast-host wall time within ~10% of the slowest source while bounding the pathological case.

View Source
const DefaultTopK = 10

DefaultTopK is the cap when Options.TopK is unset.

Variables

This section is empty.

Functions

This section is empty.

Types

type Idea

type Idea struct {
	// Title is a short label for `ideate` print output and the
	// queue list. Empty → orchestrator derives one from the first
	// line of SuggestedPrompt.
	Title string
	// Summary is a 1–3 sentence rationale; the operator reads this
	// when deciding whether to Accept. Stored as Item.Note in the
	// queue.
	Summary string
	// SourceName names the IdeaSource that produced this Idea.
	// Optional in the source-side struct; orchestrator backfills
	// from IdeaSource.Name() when empty.
	SourceName string
	// Evidence carries the file:line / test name / ADR section
	// reference / bench-baseline diff — anything the operator can
	// grep to verify the suggestion isn't fabricated.
	Evidence string
	// SuggestedPriority is the base score (typically 0..10);
	// orchestrator may apply a small evidence-recency bump before
	// final ranking.
	SuggestedPriority int
	// SuggestedPrompt is the verbatim text the agent will receive
	// when it eventually claims the item. Written in the second
	// person ("Investigate the TODO at ..."). Empty → idea is
	// silently dropped by the orchestrator.
	SuggestedPrompt string
	// DedupeKey is a stable hash combining source + evidence;
	// re-running ideate over the same repo state must not multiply
	// the queue. Source authors hash once at construction; the
	// orchestrator AND queue.Propose both deduplicate by this key.
	DedupeKey string
}

Idea is one feature candidate the Ideator emits.

type IdeaSource

type IdeaSource interface {
	Name() string
	Scan(ctx context.Context, repoRoot string) ([]Idea, error)
}

IdeaSource is the pluggable signal-mining contract. Implementations live under internal/ideator/sources/ and are wired into the orchestrator at the CLI / MCP edge (see internal/cli/ideate.go's defaultSources()). Two ground rules:

  1. Scan MUST be cheap-on-fail. Missing dir / missing CLI tool / timed-out network → empty slice + nil error. The orchestrator surfaces this via the per-source warning log; one broken source must never poison the rest.

  2. Scan MUST respect ctx.Done(). The orchestrator runs all sources in parallel under a single deadline; ignoring the ctx blocks `clawtool ideate` past the operator's patience.

type Options

type Options struct {
	RepoRoot     string
	SourceFilter string
	TopK         int
	Sources      []IdeaSource
	Warn         io.Writer
	// MaxConcurrency caps how many sources run at once. 0 → use
	// DefaultMaxConcurrency. Sources fork-exec heavy CLIs (gh,
	// govulncheck, deadcode); on slow filesystems (WSL2 /mnt/c,
	// network shares) unbounded parallelism saturates I/O and
	// bloats wall time 5× vs. running individually. The default
	// is conservative — operators on fast hosts can raise it via
	// CLAWTOOL_IDEATOR_MAX_CONCURRENCY.
	MaxConcurrency int
}

Options is the orchestrator entry-point shape.

SourceFilter (when non-empty) restricts execution to a single source by name — drives `clawtool ideate --source adr_questions`. Empty means "all enabled sources".

TopK caps the returned slice (after dedupe + score). 0 means the package default (DefaultTopK).

Sources defaults to DefaultSources(); tests override it to inject stubs.

Warn receives one line per cheap-on-fail (missing CLI, missing dir, scan error). Defaults to io.Discard when nil — the orchestrator never panics on a quiet caller.

type RunResult

type RunResult struct {
	Ideas        []Idea            `json:"ideas"`
	PerSource    map[string]int    `json:"per_source"`
	Deduped      int               `json:"deduped"`
	Added        int               `json:"added"`
	Skipped      int               `json:"skipped"`
	SourceErrors map[string]string `json:"source_errors,omitempty"`
}

RunResult is the wire shape returned by Run / RunAndQueue. Counts surface what the orchestrator did to the operator without making them re-run with -v: how many ideas each source emitted, how many got deduped, how many ended up queued (RunAndQueue only).

func Run

func Run(ctx context.Context, opts Options) (RunResult, error)

Run walks every (filtered) source in parallel, dedupes, scores, and returns the top-K. No side effects on the autopilot queue — use RunAndQueue for that. Caller MUST populate opts.Sources; the CLI / MCP edge wires the canonical bundle (see internal/cli/ideate.go's defaultSources()) so this package stays free of an import cycle into internal/ideator/sources.

func RunAndQueue

func RunAndQueue(ctx context.Context, opts Options, q *autopilot.Queue) (RunResult, error)

RunAndQueue runs the orchestrator then writes each surviving Idea into the autopilot queue at StatusProposed. q may be nil — in which case autopilot.Open() is used (the default per-host store). Returns the same RunResult shape with Added/Skipped populated.

Skipped counts items the queue rejected as duplicates (DedupeKey already lived on a non-terminal proposed/pending/in_progress row). That's by design: re-running ideate after operator inaction must be idempotent.

Directories

Path Synopsis
Package sources — adr_drafting source.
Package sources — adr_drafting source.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL