Documentation
¶
Overview ¶
Package sources — adr_drafting source.
Mirrors adr_questions: walks wiki/decisions/*.md, parses YAML frontmatter via the same line-scan approach (no yaml.v3 import pulled into this hot path — we only need two scalar fields: `status:` and `updated:`), and emits one Idea per ADR that has stayed in `status: drafting` for > 30 days. Stale drafts are a real source of decision-debt: an ADR that's been "thinking out loud" for over a month is either supersedable, promotable, or abandonable — but it shouldn't keep masquerading as in-flight.
Wiki is gitignored on this repo per operator policy, so a missing wiki/ directory is a silent no-op rather than an error — the source returns an empty slice and lets the orchestrator move on (cheap-on-fail contract per source.go).
Package sources — concrete IdeaSource implementations.
adr_questions: parses every ADR markdown under wiki/decisions/ for a "## Open questions" section and emits one Idea per numbered or bulleted item. Wiki is gitignored on this repo per operator policy, so a missing wiki/ directory is a silent no-op rather than an error — the source returns an empty slice and lets the orchestrator move on.
Package sources — bench_regression source.
Reads /tmp/clawtool-toolsearch-bench.tsv (the toolsearch BM25 rank-1 hit-rate fixture the autodev cron writes after each manifest tweak) and compares the latest run against a stored baseline at $XDG_CONFIG_HOME/clawtool/ideator/bench-baseline.json. A drop greater than DefaultRegressionThreshold (5pp) emits one Idea so the operator decides whether to chase the regression or roll the baseline forward.
Missing TSV → silent no-op (CI hasn't written one yet). Missing baseline → silent no-op (operator hasn't run `clawtool ideate --baseline-set` yet).
Package sources — ci_failures source.
Calls `gh run list --branch main --status failure --limit 10 --json databaseId,name,headSha,conclusion,event,createdAt` and emits one Idea per failed run. Falls back to a quiet no-op when the gh CLI isn't installed or unauthenticated — the spec calls this a cheap-on-fail signal, not a hard requirement.
Superseded-run filter: a failed run is dropped from the result set when ANY of (a) its head sha is no longer reachable from HEAD (the commit was force-pushed or rewritten away — re-running won't help), (b) its head sha is more than MaxCommitsBehind commits behind the current branch tip, or (c) the same workflow has had a later successful run on the same branch. All probes use `gh`/`git` so they share the same auth + cheap-on-fail story as the main query.
Package sources — deadcode_hits source.
Calls `deadcode -test ./...` from repoRoot and converts each reported unreachable function into an Idea. The deadcode tool (golang.org/x/tools/cmd/deadcode) does Rapid Type Analysis on the program's call graph; anything it lists is statically unreachable from any main + every test binary.
Cheap-on-fail: a missing `deadcode` binary on PATH or any non-zero exit from the tool returns `[]Idea{}, nil` with a silent no-op — the orchestrator's per-source warn pipeline only surfaces hard errors, and an unreachable-function survey is a nice-to-have, not a release gate.
Filters (applied before Idea emission, so the orchestrator's dedupe / top-K only sees survivors):
- `_test.go` — test helpers go through deadcode-via-`-test`, but a test fixture being unreachable is rarely a real finding; skip to keep the queue actionable.
- `*_gen.go` — generator-owned code; the generator decides when to delete, not the operator.
- `internal/mcpgen/` — template scaffolds: the body of an adapter file is emitted as user-code, deadcode flags it because nothing in this repo calls it.
- `internal/checkpoint/` — has uncalled-yet-deliberate helpers (the `wip!` autocommit/autosquash branch in resolve.go landed pre-wired; the wiring lives in a feature flag still being staged). Filter rather than re-flag every run.
Package sources — canonical default-source list.
Both Ideator edges (`internal/cli/ideate.go` for the CLI verb + `internal/tools/core/ideator_tool.go` for the MCP `IdeateRun` tool) historically maintained their own `defaultIdeatorSources()` function. Each one was a literal slice of `New*()` constructor calls. When a new source landed, the operator had to update both — and forgetting to update one was a silent drift bug: the CLI would surface 12 source signals while MCP showed 8. Operator hit this at v0.22.150 (PR #12 triage cycle).
DefaultSources is the single canonical list. Both edges call it. Adding a new source is one new entry here, not two.
`manifest_drift` still depends on the package-global manifest provider that edges wire via `SetDefaultManifestProvider` in their init(). The constructor returns an empty struct; the provider lookup happens at Scan time. Edges remain responsible for wiring the provider before any IdeateRun call lands.
Package sources — deps_outdated source.
Runs `go list -m -u -json all` from repoRoot, parses the JSON stream of module records, and emits one Idea per module whose Update.Version is set. The cheap-on-fail rule applies: missing `go` binary, network failure, or a parser error returns an empty slice + nil error rather than poisoning the orchestrator.
Priority logic (lower number = lower priority because the autopilot queue scores higher first):
- patch / minor bump → priority 4 (safe, mostly mechanical)
- major bump → priority 2 (needs manual review, breaking changes likely)
Filter: golang.org/toolchain is auto-managed by `go` itself — surfacing it as an Idea is noise. The Main module is also skipped (Update never fires on it, but the guard is cheap). Indirect modules (transitive deps not declared in go.mod's require block as direct) are skipped too — the operator can't bump them with a `go get` against go.mod, so they'd surface as un-actionable noise after a bulk-update batch.
Package sources — manifest_drift source.
Compares the manifest's hardcoded ToolSpec.Description against the *live* description each tool's Register fn emits via `mcp.WithDescription(...)`. Drift here is the v0.22.110 bug class: a hand-edit to one side leaves the search index ranking on stale prose. SyncDescriptionsFromRegistration auto-mirrors at boot, but this source guards against the inverse — someone removing the sync call or accidentally re-introducing the two hand-maintained copies.
Cycle note: this package can't import internal/tools/core (core registers IdeateRun / IdeateApply, which would import sources). The CLI / MCP edge injects the manifest builder via SetManifestBuilder before the orchestrator runs; when nothing is injected the source is a quiet no-op.
Package sources — pr_review_pending source.
Calls `gh pr list --search "review:none" --json number,title,createdAt,author,reviewDecision` and emits one Idea per PR awaiting review for more than MinAge (default 24h). Every open PR that hasn't been reviewed yet is real signal — review queues silently grow when nobody triages them.
Cheap-on-fail: missing gh CLI / unauthenticated / no PRs → no ideas, no error.
Package sources — stale_branches source.
Surfaces remote feature branches whose tip is already merged into the default branch — i.e. PRs that landed but the branch was never deleted. Each one is a one-line `git push origin --delete` away from being cleaned up; in aggregate they pollute `git branch -a`, confuse `gh pr list`, and slow `git fetch`.
Signal-driven, not heuristic: a merged branch is *definitely* removable. Priority 3 — same tier as deadcode, below ci_failures but above stale_files.
Cheap-on-fail: missing git / non-repo cwd → no ideas, no error.
Package sources — stale_files source.
Surfaces .go files whose newest commit is older than DefaultStaleAge (default 90 days). Files this old are review candidates: doc rot, slow regressions, dead-but-still-used helpers the deadcode source can't catch (because they're reachable but nobody touches them).
This source exists specifically so the Ideator never goes silent. When the eight signal-driven sources (CI failures, deadcode, etc.) converge to zero, stale_files keeps the loop productive by surfacing existing-code review opportunities. The operator can always skip the proposed item if it's not interesting; the cost is one autopilot row, the win is a never-dry signal feed.
`git log -1 --format=%ct -- <path>` per file would be O(N) shells. Instead we run `git ls-files -z '*.go'` once + a single `git log --name-only --format=%ct%n --since=<cutoff> -- *.go` pass and treat any file NOT printed in that window as stale.
Missing git / non-repo cwd → silent no-op (cheap-on-fail).
Package sources — todos source.
Walks repoRoot for *.go files (using a Go-native walker — no shell-out, no rg dependency) and emits one Idea per TODO / FIXME / XXX comment. (ideator-skip) DedupeKey is sha1(path + line + comment text) so the same comment surfaces the same id across runs even when other comments above it shift line numbers.
Package sources — vuln_advisories source.
Runs `govulncheck -json ./...` and emits one Idea per (osv, module) finding so the operator decides whether to bump the affected module / stdlib pin. Findings are joined against the in-stream `osv` advisory blocks so each Idea carries the CVE alias + summary, not just the GO-YYYY-NNNN id.
Missing govulncheck binary → silent no-op (cheap-on-fail, same shape as ci_failures / deps_outdated).
Dedupe key is `vuln_advisories:<osv-id>:<module>` — bumping the module across multiple `finding` blocks (one per call site) collapses to one Idea, so 12 stdlib finds turn into 1 actionable "bump Go toolchain" item, not 12 noise rows.
Index ¶
- Constants
- func DefaultBenchBaselinePath() string
- func DefaultSources() []ideator.IdeaSource
- func SetDefaultManifestProvider(p ManifestProvider)
- type ADRDrafting
- type ADRQuestions
- type Baseline
- type BenchRegression
- type CIFailures
- type DeadcodeHits
- type DepsOutdated
- type ManifestDrift
- type ManifestProvider
- type PRReviewPending
- type StaleBranches
- type StaleFiles
- type TODOs
- type VulnAdvisories
Constants ¶
const DefaultBenchTSV = "/tmp/clawtool-toolsearch-bench.tsv"
DefaultBenchTSV is the canonical TSV path the autodev cron writes. Columns: query, expected_top1, observed_top1, score.
const DefaultPRMinAge = 24 * time.Hour
DefaultPRMinAge is how old an unreviewed PR must be before it surfaces. 24h is operator-friendly: a PR opened in the morning and reviewed by EOD doesn't ring the bell.
const DefaultRegressionThreshold = 0.05
DefaultRegressionThreshold is the rank-1 hit-rate drop that promotes a TSV row into an Idea. 5pp is loud enough that the operator wants to know, quiet enough that single-test variance (one query per ideate cycle) doesn't trip it constantly.
const DefaultStaleAge = 90 * 24 * time.Hour
DefaultStaleAge is how long a file must go untouched before it surfaces. 90 days is roughly one quarter — short enough that real maintenance gaps appear, long enough that active development doesn't ring the bell for every helper.
const DefaultStaleBranchMaxIdeas = 5
DefaultStaleBranchMaxIdeas caps the result set; merged-but-undeleted branches accumulate fast on a busy autonomous loop, and 5 per ideate run is enough to drain the backlog over a few cycles without drowning the autopilot queue in single-line cleanup items.
const DefaultStaleBranchPrefix = "origin/autodev/"
DefaultStaleBranchPrefix scopes the source to autodev branches by default — those are the ones the autonomous loop creates and the loop should clean up after itself. Operators with different naming conventions (`feature/`, `wip/`) override the field.
const DefaultStaleMaxIdeas = 5
DefaultStaleMaxIdeas caps how many stale files we report per run. 5 is small enough to not drown the operator, large enough that the loop stays productive across multiple ideate cycles.
Variables ¶
This section is empty.
Functions ¶
func DefaultBenchBaselinePath ¶
func DefaultBenchBaselinePath() string
DefaultBenchBaselinePath returns $XDG_CONFIG_HOME/clawtool/ideator/bench-baseline.json.
func DefaultSources ¶ added in v0.22.152
func DefaultSources() []ideator.IdeaSource
DefaultSources returns every Ideator signal source enabled by default, in priority-conscious order. Edges (`internal/cli` for the `clawtool ideate` verb, `internal/tools/core` for the MCP `IdeateRun` tool) call this so they're always in sync. Tests can override the list by passing their own `[]ideator.IdeaSource`.
Source order is NOT load-bearing — the orchestrator dedupes by `Idea.DedupeKey` and re-sorts by priority before returning. The order here is documentary: roughly highest-signal-first so the operator reading the list understands "what does Ideator notice most reliably?"
func SetDefaultManifestProvider ¶
func SetDefaultManifestProvider(p ManifestProvider)
SetDefaultManifestProvider lets the CLI / MCP edge inject a manifest builder. internal/cli/ideate.go calls this at init so every freshly-constructed ManifestDrift picks up the canonical builder; the function is safe to call multiple times.
Types ¶
type ADRDrafting ¶ added in v0.22.138
type ADRDrafting struct{}
ADRDrafting implements IdeaSource. Construct via NewADRDrafting.
func NewADRDrafting ¶ added in v0.22.138
func NewADRDrafting() *ADRDrafting
NewADRDrafting returns a ready-to-use stale-drafting-ADR miner.
func (ADRDrafting) Name ¶ added in v0.22.138
func (ADRDrafting) Name() string
Name returns the canonical source name used by --source filters.
func (ADRDrafting) Scan ¶ added in v0.22.138
Scan walks repoRoot/wiki/decisions/*.md. Missing directory is a silent no-op. For each ADR with `status: drafting` whose `updated:` date is more than 30 days behind the wall clock, one Idea is emitted at priority 5 with evidence pointing at the ADR path. Per-file parse errors degrade gracefully: a malformed frontmatter block on one ADR doesn't kill the rest of the scan.
type ADRQuestions ¶
type ADRQuestions struct{}
ADRQuestions implements IdeaSource. Construct via NewADRQuestions.
func NewADRQuestions ¶
func NewADRQuestions() *ADRQuestions
NewADRQuestions returns a ready-to-use ADR question miner.
func (ADRQuestions) Name ¶
func (ADRQuestions) Name() string
Name returns the canonical source name used by --source filters.
func (ADRQuestions) Scan ¶
Scan walks repoRoot/wiki/decisions/*.md. Missing directory is a silent no-op (wiki/ is gitignored on this repo). Each line under "## Open questions" that begins with `-`, `*`, or a numbered list marker becomes one Idea. The DedupeKey is sha1(adr-path + question text) so an unedited file produces stable IDs across runs.
type Baseline ¶
type Baseline struct {
HitRate float64 `json:"hit_rate"`
NumQueries int `json:"num_queries"`
WrittenAt string `json:"written_at"`
}
Baseline is the on-disk JSON shape the operator writes via `clawtool ideate --baseline-set`. HitRate is rank-1 (top-1 correct count / total queries).
func SaveBaseline ¶
SaveBaseline writes the current TSV's hit rate to the baseline path. CLI's `--baseline-set` calls this. Exposed at package level so the orchestrator + CLI share one writer.
type BenchRegression ¶
type BenchRegression struct {
// TSVPath overrides DefaultBenchTSV (tests inject tmpdir).
TSVPath string
// BaselinePath overrides the default baseline location.
BaselinePath string
// Threshold is the hit-rate drop (0..1) at which an Idea fires.
Threshold float64
}
BenchRegression implements IdeaSource for the BM25 baseline diff.
func NewBenchRegression ¶
func NewBenchRegression() *BenchRegression
NewBenchRegression returns a ready-to-use BM25 baseline-diff source with default paths and threshold.
func (BenchRegression) Name ¶
func (BenchRegression) Name() string
Name returns the canonical source name.
type CIFailures ¶
type CIFailures struct {
// Branch is the branch to inspect (default "main").
Branch string
// Limit caps how many failed runs gh returns (default 10).
Limit int
// GHBinary lets tests inject a stub binary; defaults to "gh".
GHBinary string
// Timeout caps the gh subprocess (default 10s).
Timeout time.Duration
// MaxCommitsBehind drops failures whose head sha is more than
// this many commits behind the current branch tip (default 20).
// Set to 0 to disable the distance probe.
MaxCommitsBehind int
// GitBinary lets tests inject a stub `git`; defaults to "git".
// Used by the distance probe (`git rev-list --count A..HEAD`).
GitBinary string
// SkipSupersededByGreen toggles the per-workflow "later green
// run" probe. Default true. Tests that don't want to stub gh's
// success query can disable it.
SkipSupersededByGreen bool
}
CIFailures implements IdeaSource. Construct via NewCIFailures.
func NewCIFailures ¶
func NewCIFailures() *CIFailures
NewCIFailures returns a ready-to-use ci_failures miner.
type DeadcodeHits ¶ added in v0.22.138
type DeadcodeHits struct {
// Binary lets tests inject a stub script; defaults to
// "deadcode". The lookup falls back to whatever `exec.LookPath`
// finds on $PATH, which on Go developer hosts typically
// resolves into $GOPATH/bin.
Binary string
// Args overrides the argv passed to the binary; defaults to
// `-test ./...`. Tests that pin a fixture file feed
// `-test ./fixturepkg` etc.
Args []string
// SkipPathFragments are forward-slash substrings; any finding
// whose file path contains one (case-insensitive) is dropped.
// Defaults documented at NewDeadcodeHits.
SkipPathFragments []string
}
DeadcodeHits implements IdeaSource by shelling out to the `deadcode` tool and converting each finding into an Idea.
func NewDeadcodeHits ¶ added in v0.22.138
func NewDeadcodeHits() *DeadcodeHits
NewDeadcodeHits returns a ready-to-use deadcode-hits miner with the canonical filter list (see package doc).
func (DeadcodeHits) Name ¶ added in v0.22.138
func (DeadcodeHits) Name() string
Name returns the canonical source name.
type DepsOutdated ¶ added in v0.22.138
type DepsOutdated struct {
// GoBinary lets tests inject a stub binary; defaults to "go".
GoBinary string
// Timeout caps the `go list` subprocess (default 60s — module
// proxy queries can be slow on cold caches).
Timeout time.Duration
// contains filtered or unexported fields
}
DepsOutdated implements IdeaSource. Construct via NewDepsOutdated.
func NewDepsOutdated ¶ added in v0.22.138
func NewDepsOutdated() *DepsOutdated
NewDepsOutdated returns a ready-to-use outdated-module miner.
func (DepsOutdated) Name ¶ added in v0.22.138
func (DepsOutdated) Name() string
Name returns the canonical source name.
type ManifestDrift ¶
type ManifestDrift struct {
// Provider returns the manifest to inspect. nil → silent no-op.
Provider ManifestProvider
}
ManifestDrift implements IdeaSource by diffing the manifest spec descriptions against what each Register fn registers live.
func NewManifestDrift ¶
func NewManifestDrift() *ManifestDrift
NewManifestDrift returns a ready-to-use manifest-drift detector. Provider must be wired separately via SetProvider or by direct field assignment; the CLI / MCP edge does this in internal/cli/ideate.go and internal/tools/core/ideator_tool.go (those layers can import core without creating a cycle).
func (ManifestDrift) Name ¶
func (ManifestDrift) Name() string
Name returns the canonical source name.
func (ManifestDrift) Scan ¶
Scan compares each registered spec description against the live `mcp.WithDescription` value. Mismatches surface as Ideas pointing the operator at SyncDescriptionsFromRegistration.
func (*ManifestDrift) SetProvider ¶
func (m *ManifestDrift) SetProvider(p ManifestProvider)
SetProvider lets a caller swap the manifest builder (tests use this to feed a hand-built manifest with deliberately drifted descriptions).
type ManifestProvider ¶
ManifestProvider returns the live MCP tool manifest. The CLI / MCP edge wraps core.BuildManifest into this shape so this package stays free of an import cycle.
type PRReviewPending ¶ added in v0.22.147
type PRReviewPending struct {
// GHBinary lets tests inject a stub `gh`; defaults to "gh".
GHBinary string
// Limit caps how many PRs gh returns; default 10.
Limit int
// MinAge is how long a PR must sit unreviewed; default 24h.
MinAge time.Duration
// Timeout caps the gh subprocess; default 10s.
Timeout time.Duration
// Now is overridable for tests; defaults to time.Now.
Now func() time.Time
}
PRReviewPending implements IdeaSource for review-queue triage.
func NewPRReviewPending ¶ added in v0.22.147
func NewPRReviewPending() *PRReviewPending
NewPRReviewPending returns a ready-to-use review-queue miner.
func (PRReviewPending) Name ¶ added in v0.22.147
func (PRReviewPending) Name() string
Name returns the canonical source name.
type StaleBranches ¶ added in v0.22.150
type StaleBranches struct {
// GitBinary lets tests inject a stub; defaults to "git".
GitBinary string
// DefaultBranch is the merge target tested against (default
// "origin/main"). Branches whose tip is already in this ref's
// history are considered merged.
DefaultBranch string
// Prefix is the ref-name prefix that scopes which branches are
// candidates. Default `origin/autodev/`. Set to "origin/" to
// scope to the whole remote.
Prefix string
// MaxIdeas caps the result set; default DefaultStaleBranchMaxIdeas.
MaxIdeas int
// SkipBranches lists ref names to never surface (default
// includes `origin/main`, `origin/HEAD`).
SkipBranches []string
}
StaleBranches implements IdeaSource for "remote branch tip already reachable from default branch HEAD".
func NewStaleBranches ¶ added in v0.22.150
func NewStaleBranches() *StaleBranches
NewStaleBranches returns a ready-to-use stale-branch miner.
func (StaleBranches) Name ¶ added in v0.22.150
func (StaleBranches) Name() string
Name returns the canonical source name.
type StaleFiles ¶ added in v0.22.142
type StaleFiles struct {
// GitBinary lets tests inject a stub; defaults to "git".
GitBinary string
// MinAge overrides DefaultStaleAge.
MinAge time.Duration
// MaxIdeas caps the result set; default DefaultStaleMaxIdeas.
MaxIdeas int
// Now lets tests pin "current time" for deterministic cutoffs.
Now func() time.Time
// Pattern restricts the file scan; defaults to "*.go".
Pattern string
// SkipPaths drops files whose path contains any of these
// substrings. Default skips test fixtures + generated code.
SkipPaths []string
}
StaleFiles implements IdeaSource for "untouched .go files".
func NewStaleFiles ¶ added in v0.22.142
func NewStaleFiles() *StaleFiles
NewStaleFiles returns a ready-to-use stale-files miner.
func (StaleFiles) Name ¶ added in v0.22.142
func (StaleFiles) Name() string
Name returns the canonical source name.
type TODOs ¶
type TODOs struct {
// MaxIdeas caps the source's emit count so a repo with hundreds
// of TODOs doesn't drown the orchestrator's top-K. Default 50.
MaxIdeas int
// SkipDirs are directory names skipped during the walk
// (default: vendor, node_modules, .git, .clawtool, dist, bin).
SkipDirs []string
// SkipPathFragments are forward-slash substrings; any path that
// contains one (case-insensitive) is dropped before parsing.
// Default value covers Go module cache, Claude Code plugin
// install/cache dirs, Obsidian wiki, vendor / node_modules,
// and .git in case the walker misses them via a symlink.
SkipPathFragments []string
}
TODOs implements IdeaSource by mining TODO/FIXME/XXX comments from *.go files.
type VulnAdvisories ¶ added in v0.22.142
type VulnAdvisories struct {
// Binary lets tests inject a stub `govulncheck`; defaults to
// the resolved one from $PATH.
Binary string
// Timeout caps the subprocess. Govulncheck loads the package
// graph + queries the vulndb, so 90s is the realistic
// default — cron-friendly but bounded.
Timeout time.Duration
// Pattern is the package pattern to scan; defaults to ./...
Pattern string
// WorkflowGoVersionFile points to a YAML file whose
// `GO_VERSION:` line is treated as the authoritative toolchain
// pin for stdlib advisories. When set + readable, stdlib
// findings whose `fixed_version` is <= the workflow pin are
// dropped — the published binary already runs the fix even if
// the developer's local Go install is older. Defaults to
// `.github/workflows/ci.yml`. Set to "" to disable.
WorkflowGoVersionFile string
// CachePath stores the most-recent raw govulncheck output. When
// the cache is fresher than CacheTTL AND the cache key (go.sum
// hash + govulncheck version) matches, Scan skips the subprocess
// entirely. Default `$TMPDIR/clawtool-govulncheck-cache.json`.
// Set to "" to disable caching.
CachePath string
// CacheTTL is how long the on-disk govulncheck output stays
// authoritative. Default 12h: the Go vulndb publishes daily, so
// 12h gives at most one stale call between db refreshes; on a
// slow filesystem (WSL2 /mnt/c) the win is enormous (~16s → ~0.1s).
CacheTTL time.Duration
// Now is overridable for tests; defaults to time.Now.
Now func() time.Time
}
VulnAdvisories implements IdeaSource for govulncheck output.
func NewVulnAdvisories ¶ added in v0.22.142
func NewVulnAdvisories() *VulnAdvisories
NewVulnAdvisories returns a ready-to-use vuln_advisories miner.
func (VulnAdvisories) Name ¶ added in v0.22.142
func (VulnAdvisories) Name() string
Name returns the canonical source name.
func (VulnAdvisories) Scan ¶ added in v0.22.142
Scan calls govulncheck, parses the JSON stream, joins findings against advisory metadata, and emits deduped Ideas. Empty + nil error on missing binary / non-zero exit / parse failure (cheap-on-fail).
When a cache hit on the local go.sum hash is fresh enough (CacheTTL), the subprocess is skipped and the cached output is re-parsed — turning a 16s scan into ~50ms.
Source Files