Directories
¶
| Path | Synopsis |
|---|---|
|
whetstone
Package whetstone implements the whetstone CLI: the authoring front end for the skill specification, generation, lint, contract, and suite engine under internal/eval, plus the evaluation commands (eval, drift, repair, report) built on top of it.
|
Package whetstone implements the whetstone CLI: the authoring front end for the skill specification, generation, lint, contract, and suite engine under internal/eval, plus the evaluation commands (eval, drift, repair, report) built on top of it. |
|
cmd
|
|
|
migrate
command
|
|
|
server
command
|
|
|
skael
command
|
|
|
skael-worker
command
Command skael-worker runs the eval queue's claim/materialise/evaluate/report loop against real infrastructure: a Docker sandbox, the direct Anthropic API gateway, and the linked-in agent adapter registry.
|
Command skael-worker runs the eval queue's claim/materialise/evaluate/report loop against real infrastructure: a Docker sandbox, the direct Anthropic API gateway, and the linked-in agent adapter registry. |
|
whetstone
command
Command whetstone is the standalone skill authoring and evaluation CLI.
|
Command whetstone is the standalone skill authoring and evaluation CLI. |
|
internal
|
|
|
eval/agent
Package agent defines the interface every coding-agent CLI is driven through.
|
Package agent defines the interface every coding-agent CLI is driven through. |
|
eval/agent/claudecode
Package claudecode adapts the Claude Code CLI.
|
Package claudecode adapts the Claude Code CLI. |
|
eval/agent/codex
Package codex reserves the Codex CLI adapter.
|
Package codex reserves the Codex CLI adapter. |
|
eval/agent/cursor
Package cursor reserves the Cursor CLI adapter.
|
Package cursor reserves the Cursor CLI adapter. |
|
eval/agent/opencode
Package opencode reserves the OpenCode CLI adapter.
|
Package opencode reserves the OpenCode CLI adapter. |
|
eval/bundlepath
Package bundlepath holds the one bundle-containment check every package that writes model-authored paths into a skill bundle shares.
|
Package bundlepath holds the one bundle-containment check every package that writes model-authored paths into a skill bundle shares. |
|
eval/contract
Package contract compiles a spec.SkillSpec into a behavioural contract of deterministic matchers.
|
Package contract compiles a spec.SkillSpec into a behavioural contract of deterministic matchers. |
|
eval/drift
Package drift measures how closely an agent's actual trajectory followed the behavioural contract compiled from a skill's specification.
|
Package drift measures how closely an agent's actual trajectory followed the behavioural contract compiled from a skill's specification. |
|
eval/gen
Package gen writes a skill bundle — SKILL.md plus scripts/, references/, and assets/ — from an approved spec.SkillSpec.
|
Package gen writes a skill bundle — SKILL.md plus scripts/, references/, and assets/ — from an approved spec.SkillSpec. |
|
eval/lint
Package lint validates a generated or imported skill bundle.
|
Package lint validates a generated or imported skill bundle. |
|
eval/llm
Package llm is the single seam through which every model call passes: interview, generation, suite drafting, judging, and repair.
|
Package llm is the single seam through which every model call passes: interview, generation, suite drafting, judging, and repair. |
|
eval/llm/agentcli
Package agentcli implements the LLM gateway by shelling out to a coding-agent CLI in print mode, so every model call is billed to the operator's existing subscription rather than requiring an API key.
|
Package agentcli implements the LLM gateway by shelling out to a coding-agent CLI in print mode, so every model call is billed to the operator's existing subscription rather than requiring an API key. |
|
eval/llm/api
Package api implements the LLM gateway against a direct Anthropic-compatible HTTP endpoint.
|
Package api implements the LLM gateway against a direct Anthropic-compatible HTTP endpoint. |
|
eval/llm/fake
Package fake provides an in-memory Gateway for tests: these tests run with no LLM subscription, no API key, and no network, so the fake is how generation and suite behaviour get exercised.
|
Package fake provides an in-memory Gateway for tests: these tests run with no LLM subscription, no API key, and no network, so the fake is how generation and suite behaviour get exercised. |
|
eval/repair
Package repair clusters evaluation failures into fixable defects and audits which assertions actually discriminate a skill's effect, so a repair loop spends its diff budget on real skill defects rather than on restating the same failure three times or rewording a broken verifier.
|
Package repair clusters evaluation failures into fixable defects and audits which assertions actually discriminate a skill's effect, so a repair loop spends its diff budget on real skill defects rather than on restating the same failure three times or rewording a broken verifier. |
|
eval/report
Package report defines the machine-readable result of an eval run: the schema a CI job reads, a later phase posts, and a future UI charts.
|
Package report defines the machine-readable result of an eval run: the schema a CI job reads, a later phase posts, and a future UI charts. |
|
eval/runner
Package runner plans and executes an evaluation: which sessions to run, in what environment, with what resumed from a previous attempt.
|
Package runner plans and executes an evaluation: which sessions to run, in what environment, with what resumed from a previous attempt. |
|
eval/sandbox
Package sandbox is the seam every evaluation run executes through.
|
Package sandbox is the seam every evaluation run executes through. |
|
eval/sandbox/docker
Package docker implements the sandbox driver on the Docker CLI.
|
Package docker implements the sandbox driver on the Docker CLI. |
|
eval/sandbox/imagespec
Package imagespec renders everything an image needs as text: the base Dockerfile, a per-skill layer over it, the proxy configuration that enforces a network allowlist, and the digest that makes a layer cacheable.
|
Package imagespec renders everything an image needs as text: the base Dockerfile, a per-skill layer over it, the proxy configuration that enforces a network allowlist, and the digest that makes a layer cacheable. |
|
eval/sandbox/sprites
Package sprites is the hosted, hardware-isolated sandbox driver: Firecracker microVMs with checkpoint/restore, which is what makes a 60-session tier one environment build instead of sixty, and what makes untrusted skills evaluable at all.
|
Package sprites is the hosted, hardware-isolated sandbox driver: Firecracker microVMs with checkpoint/restore, which is what makes a 60-session tier one environment build instead of sixty, and what makes untrusted skills evaluable at all. |
|
eval/score
Package score implements the LLM judge protocol: position-swapped pairwise comparisons and per-rule semantic scoring, both anchored on a worked rubric and both requiring a verbatim evidence quote before a verdict counts.
|
Package score implements the LLM judge protocol: position-swapped pairwise comparisons and per-rule semantic scoring, both anchored on a worked rubric and both requiring a verbatim evidence quote before a verdict counts. |
|
eval/spec
Package spec holds the SkillSpec intermediate representation — the source of truth every other eval component derives from.
|
Package spec holds the SkillSpec intermediate representation — the source of truth every other eval component derives from. |
|
eval/store
Package store owns the .whetstone/ workspace: a SQLite index for versioned specs and the completion cache, plus the on-disk artifact layout.
|
Package store owns the .whetstone/ workspace: a SQLite index for versioned specs and the completion cache, plus the on-disk artifact layout. |
|
eval/suite
Package suite drafts an evaluation suite from an approved spec.SkillSpec: a set of task packages, each with its own oracle and verifier, plus a trigger set used to measure whether a skill fires when it should (and stays silent when it shouldn't).
|
Package suite drafts an evaluation suite from an approved spec.SkillSpec: a set of task packages, each with its own oracle and verifier, plus a trigger set used to measure whether a skill fires when it should (and stays silent when it shouldn't). |
|
eval/suite/assets
Package assets embeds the static suite assets.
|
Package assets embeds the static suite assets. |
|
eval/trajectory
Package trajectory defines the normalized event schema every agent adapter maps its native stream onto.
|
Package trajectory defines the normalized event schema every agent adapter maps its native stream onto. |
|
evalqueue
Package evalqueue is the eval job queue.
|
Package evalqueue is the eval job queue. |
|
evalsuite
Package evalsuite is the registry-side counterpart to internal/eval/suite: evaluation task-suites become content-addressable artifacts stored alongside skill bundles, so a quality score can be re-run later against the exact same tasks with a different model panel.
|
Package evalsuite is the registry-side counterpart to internal/eval/suite: evaluation task-suites become content-addressable artifacts stored alongside skill bundles, so a quality score can be re-run later against the exact same tasks with a different model panel. |
|
gate
Package gate decides whether a skill version may be published, held for review, or refused outright.
|
Package gate decides whether a skill version may be published, held for review, or refused outright. |
|
ownership
Package ownership answers two questions and nothing else: who owns a skill name, and may this actor manage this pattern.
|
Package ownership answers two questions and nothing else: who owns a skill name, and may this actor manage this pattern. |
|
quality
Package quality maps an eval report onto a stored quality record and persists it, so a score can be read back per skill version and compared across versions.
|
Package quality maps an eval report onto a stored quality record and persists it, so a score can be read back per skill version and compared across versions. |
|
worker
Package worker implements the eval queue worker's run loop: claim a job from the server, materialise a local whetstone workspace from the downloaded skill bundle and suite, run the evaluation, and post the report back.
|
Package worker implements the eval queue worker's run loop: claim a job from the server, materialise a local whetstone workspace from the downloaded skill bundle and suite, run the evaluation, and post the report back. |
Click to show internal directories.
Click to hide internal directories.




