Documentation
¶
Overview ¶
Package sources — adr_drafting source.
Mirrors adr_questions: walks wiki/decisions/*.md, parses YAML frontmatter via the same line-scan approach (no yaml.v3 import pulled into this hot path — we only need two scalar fields: `status:` and `updated:`), and emits one Idea per ADR that has stayed in `status: drafting` for > 30 days. Stale drafts are a real source of decision-debt: an ADR that's been "thinking out loud" for over a month is either supersedable, promotable, or abandonable — but it shouldn't keep masquerading as in-flight.
Wiki is gitignored on this repo per operator policy, so a missing wiki/ directory is a silent no-op rather than an error — the source returns an empty slice and lets the orchestrator move on (cheap-on-fail contract per source.go).
Package sources — concrete IdeaSource implementations.
adr_questions: parses every ADR markdown under wiki/decisions/ for a "## Open questions" section and emits one Idea per numbered or bulleted item. Wiki is gitignored on this repo per operator policy, so a missing wiki/ directory is a silent no-op rather than an error — the source returns an empty slice and lets the orchestrator move on.
Package sources — bench_regression source.
Reads /tmp/clawtool-toolsearch-bench.tsv (the toolsearch BM25 rank-1 hit-rate fixture the autodev cron writes after each manifest tweak) and compares the latest run against a stored baseline at $XDG_CONFIG_HOME/clawtool/ideator/bench-baseline.json. A drop greater than DefaultRegressionThreshold (5pp) emits one Idea so the operator decides whether to chase the regression or roll the baseline forward.
Missing TSV → silent no-op (CI hasn't written one yet). Missing baseline → silent no-op (operator hasn't run `clawtool ideate --baseline-set` yet).
Package sources — ci_failures source.
Calls `gh run list --branch main --status failure --limit 10 --json databaseId,name,headSha,conclusion,event,createdAt` and emits one Idea per failed run. Falls back to a quiet no-op when the gh CLI isn't installed or unauthenticated — the spec calls this a cheap-on-fail signal, not a hard requirement.
Superseded-run filter: a failed run is dropped from the result set when ANY of (a) its head sha is no longer reachable from HEAD (the commit was force-pushed or rewritten away — re-running won't help), (b) its head sha is more than MaxCommitsBehind commits behind the current branch tip, or (c) the same workflow has had a later successful run on the same branch. All probes use `gh`/`git` so they share the same auth + cheap-on-fail story as the main query.
Package sources — deadcode_hits source.
Calls `deadcode -test ./...` from repoRoot and converts each reported unreachable function into an Idea. The deadcode tool (golang.org/x/tools/cmd/deadcode) does Rapid Type Analysis on the program's call graph; anything it lists is statically unreachable from any main + every test binary.
Cheap-on-fail: a missing `deadcode` binary on PATH or any non-zero exit from the tool returns `[]Idea{}, nil` with a silent no-op — the orchestrator's per-source warn pipeline only surfaces hard errors, and an unreachable-function survey is a nice-to-have, not a release gate.
Filters (applied before Idea emission, so the orchestrator's dedupe / top-K only sees survivors):
- `_test.go` — test helpers go through deadcode-via-`-test`, but a test fixture being unreachable is rarely a real finding; skip to keep the queue actionable.
- `*_gen.go` — generator-owned code; the generator decides when to delete, not the operator.
- `internal/mcpgen/` — template scaffolds: the body of an adapter file is emitted as user-code, deadcode flags it because nothing in this repo calls it.
- `internal/checkpoint/` — has uncalled-yet-deliberate helpers (the `wip!` autocommit/autosquash branch in resolve.go landed pre-wired; the wiring lives in a feature flag still being staged). Filter rather than re-flag every run.
Package sources — deps_outdated source.
Runs `go list -m -u -json all` from repoRoot, parses the JSON stream of module records, and emits one Idea per module whose Update.Version is set. The cheap-on-fail rule applies: missing `go` binary, network failure, or a parser error returns an empty slice + nil error rather than poisoning the orchestrator.
Priority logic (lower number = lower priority because the autopilot queue scores higher first):
- patch / minor bump → priority 4 (safe, mostly mechanical)
- major bump → priority 2 (needs manual review, breaking changes likely)
Filter: golang.org/toolchain is auto-managed by `go` itself — surfacing it as an Idea is noise. The Main module is also skipped (Update never fires on it, but the guard is cheap). Indirect modules (transitive deps not declared in go.mod's require block as direct) are skipped too — the operator can't bump them with a `go get` against go.mod, so they'd surface as un-actionable noise after a bulk-update batch.
Package sources — manifest_drift source.
Compares the manifest's hardcoded ToolSpec.Description against the *live* description each tool's Register fn emits via `mcp.WithDescription(...)`. Drift here is the v0.22.110 bug class: a hand-edit to one side leaves the search index ranking on stale prose. SyncDescriptionsFromRegistration auto-mirrors at boot, but this source guards against the inverse — someone removing the sync call or accidentally re-introducing the two hand-maintained copies.
Cycle note: this package can't import internal/tools/core (core registers IdeateRun / IdeateApply, which would import sources). The CLI / MCP edge injects the manifest builder via SetManifestBuilder before the orchestrator runs; when nothing is injected the source is a quiet no-op.
Package sources — todos source.
Walks repoRoot for *.go files (using a Go-native walker — no shell-out, no rg dependency) and emits one Idea per TODO / FIXME / XXX comment. (ideator-skip) DedupeKey is sha1(path + line + comment text) so the same comment surfaces the same id across runs even when other comments above it shift line numbers.
Index ¶
Constants ¶
const DefaultBenchTSV = "/tmp/clawtool-toolsearch-bench.tsv"
DefaultBenchTSV is the canonical TSV path the autodev cron writes. Columns: query, expected_top1, observed_top1, score.
const DefaultRegressionThreshold = 0.05
DefaultRegressionThreshold is the rank-1 hit-rate drop that promotes a TSV row into an Idea. 5pp is loud enough that the operator wants to know, quiet enough that single-test variance (one query per ideate cycle) doesn't trip it constantly.
Variables ¶
This section is empty.
Functions ¶
func DefaultBenchBaselinePath ¶
func DefaultBenchBaselinePath() string
DefaultBenchBaselinePath returns $XDG_CONFIG_HOME/clawtool/ideator/bench-baseline.json.
func SetDefaultManifestProvider ¶
func SetDefaultManifestProvider(p ManifestProvider)
SetDefaultManifestProvider lets the CLI / MCP edge inject a manifest builder. internal/cli/ideate.go calls this at init so every freshly-constructed ManifestDrift picks up the canonical builder; the function is safe to call multiple times.
Types ¶
type ADRDrafting ¶ added in v0.22.138
type ADRDrafting struct{}
ADRDrafting implements IdeaSource. Construct via NewADRDrafting.
func NewADRDrafting ¶ added in v0.22.138
func NewADRDrafting() *ADRDrafting
NewADRDrafting returns a ready-to-use stale-drafting-ADR miner.
func (ADRDrafting) Name ¶ added in v0.22.138
func (ADRDrafting) Name() string
Name returns the canonical source name used by --source filters.
func (ADRDrafting) Scan ¶ added in v0.22.138
Scan walks repoRoot/wiki/decisions/*.md. Missing directory is a silent no-op. For each ADR with `status: drafting` whose `updated:` date is more than 30 days behind the wall clock, one Idea is emitted at priority 5 with evidence pointing at the ADR path. Per-file parse errors degrade gracefully: a malformed frontmatter block on one ADR doesn't kill the rest of the scan.
type ADRQuestions ¶
type ADRQuestions struct{}
ADRQuestions implements IdeaSource. Construct via NewADRQuestions.
func NewADRQuestions ¶
func NewADRQuestions() *ADRQuestions
NewADRQuestions returns a ready-to-use ADR question miner.
func (ADRQuestions) Name ¶
func (ADRQuestions) Name() string
Name returns the canonical source name used by --source filters.
func (ADRQuestions) Scan ¶
Scan walks repoRoot/wiki/decisions/*.md. Missing directory is a silent no-op (wiki/ is gitignored on this repo). Each line under "## Open questions" that begins with `-`, `*`, or a numbered list marker becomes one Idea. The DedupeKey is sha1(adr-path + question text) so an unedited file produces stable IDs across runs.
type Baseline ¶
type Baseline struct {
HitRate float64 `json:"hit_rate"`
NumQueries int `json:"num_queries"`
WrittenAt string `json:"written_at"`
}
Baseline is the on-disk JSON shape the operator writes via `clawtool ideate --baseline-set`. HitRate is rank-1 (top-1 correct count / total queries).
func SaveBaseline ¶
SaveBaseline writes the current TSV's hit rate to the baseline path. CLI's `--baseline-set` calls this. Exposed at package level so the orchestrator + CLI share one writer.
type BenchRegression ¶
type BenchRegression struct {
// TSVPath overrides DefaultBenchTSV (tests inject tmpdir).
TSVPath string
// BaselinePath overrides the default baseline location.
BaselinePath string
// Threshold is the hit-rate drop (0..1) at which an Idea fires.
Threshold float64
}
BenchRegression implements IdeaSource for the BM25 baseline diff.
func NewBenchRegression ¶
func NewBenchRegression() *BenchRegression
NewBenchRegression returns a ready-to-use BM25 baseline-diff source with default paths and threshold.
func (BenchRegression) Name ¶
func (BenchRegression) Name() string
Name returns the canonical source name.
type CIFailures ¶
type CIFailures struct {
// Branch is the branch to inspect (default "main").
Branch string
// Limit caps how many failed runs gh returns (default 10).
Limit int
// GHBinary lets tests inject a stub binary; defaults to "gh".
GHBinary string
// Timeout caps the gh subprocess (default 10s).
Timeout time.Duration
// MaxCommitsBehind drops failures whose head sha is more than
// this many commits behind the current branch tip (default 20).
// Set to 0 to disable the distance probe.
MaxCommitsBehind int
// GitBinary lets tests inject a stub `git`; defaults to "git".
// Used by the distance probe (`git rev-list --count A..HEAD`).
GitBinary string
// SkipSupersededByGreen toggles the per-workflow "later green
// run" probe. Default true. Tests that don't want to stub gh's
// success query can disable it.
SkipSupersededByGreen bool
}
CIFailures implements IdeaSource. Construct via NewCIFailures.
func NewCIFailures ¶
func NewCIFailures() *CIFailures
NewCIFailures returns a ready-to-use ci_failures miner.
type DeadcodeHits ¶ added in v0.22.138
type DeadcodeHits struct {
// Binary lets tests inject a stub script; defaults to
// "deadcode". The lookup falls back to whatever `exec.LookPath`
// finds on $PATH, which on Go developer hosts typically
// resolves into $GOPATH/bin.
Binary string
// Args overrides the argv passed to the binary; defaults to
// `-test ./...`. Tests that pin a fixture file feed
// `-test ./fixturepkg` etc.
Args []string
// SkipPathFragments are forward-slash substrings; any finding
// whose file path contains one (case-insensitive) is dropped.
// Defaults documented at NewDeadcodeHits.
SkipPathFragments []string
}
DeadcodeHits implements IdeaSource by shelling out to the `deadcode` tool and converting each finding into an Idea.
func NewDeadcodeHits ¶ added in v0.22.138
func NewDeadcodeHits() *DeadcodeHits
NewDeadcodeHits returns a ready-to-use deadcode-hits miner with the canonical filter list (see package doc).
func (DeadcodeHits) Name ¶ added in v0.22.138
func (DeadcodeHits) Name() string
Name returns the canonical source name.
type DepsOutdated ¶ added in v0.22.138
type DepsOutdated struct {
// GoBinary lets tests inject a stub binary; defaults to "go".
GoBinary string
// Timeout caps the `go list` subprocess (default 60s — module
// proxy queries can be slow on cold caches).
Timeout time.Duration
// contains filtered or unexported fields
}
DepsOutdated implements IdeaSource. Construct via NewDepsOutdated.
func NewDepsOutdated ¶ added in v0.22.138
func NewDepsOutdated() *DepsOutdated
NewDepsOutdated returns a ready-to-use outdated-module miner.
func (DepsOutdated) Name ¶ added in v0.22.138
func (DepsOutdated) Name() string
Name returns the canonical source name.
type ManifestDrift ¶
type ManifestDrift struct {
// Provider returns the manifest to inspect. nil → silent no-op.
Provider ManifestProvider
}
ManifestDrift implements IdeaSource by diffing the manifest spec descriptions against what each Register fn registers live.
func NewManifestDrift ¶
func NewManifestDrift() *ManifestDrift
NewManifestDrift returns a ready-to-use manifest-drift detector. Provider must be wired separately via SetProvider or by direct field assignment; the CLI / MCP edge does this in internal/cli/ideate.go and internal/tools/core/ideator_tool.go (those layers can import core without creating a cycle).
func (ManifestDrift) Name ¶
func (ManifestDrift) Name() string
Name returns the canonical source name.
func (ManifestDrift) Scan ¶
Scan compares each registered spec description against the live `mcp.WithDescription` value. Mismatches surface as Ideas pointing the operator at SyncDescriptionsFromRegistration.
func (*ManifestDrift) SetProvider ¶
func (m *ManifestDrift) SetProvider(p ManifestProvider)
SetProvider lets a caller swap the manifest builder (tests use this to feed a hand-built manifest with deliberately drifted descriptions).
type ManifestProvider ¶
ManifestProvider returns the live MCP tool manifest. The CLI / MCP edge wraps core.BuildManifest into this shape so this package stays free of an import cycle.
type TODOs ¶
type TODOs struct {
// MaxIdeas caps the source's emit count so a repo with hundreds
// of TODOs doesn't drown the orchestrator's top-K. Default 50.
MaxIdeas int
// SkipDirs are directory names skipped during the walk
// (default: vendor, node_modules, .git, .clawtool, dist, bin).
SkipDirs []string
// SkipPathFragments are forward-slash substrings; any path that
// contains one (case-insensitive) is dropped before parsing.
// Default value covers Go module cache, Claude Code plugin
// install/cache dirs, Obsidian wiki, vendor / node_modules,
// and .git in case the walker misses them via a symlink.
SkipPathFragments []string
}
TODOs implements IdeaSource by mining TODO/FIXME/XXX comments from *.go files.
Source Files