Documentation
¶
Overview ¶
Package sourceconformance provides shared conformance test suites for the content-source seams: tool.SkillSource (RunSkillSource), prompt.SoulSource (RunSoulSource), tool.AgentDefSource (RunAgentSource), and prompt.CommandSource (RunCommandSource). Adapters (the filesystem skills/ agents sources, the soul file store, remote drivers, ...) call the Run functions with a factory that constructs a fresh source, and the suites exercise only the port interfaces.
Importing "testing" in a non-_test.go file is intentional here: this is a test-helper package whose sole purpose is to be imported by adapter tests, the conventional Go pattern for shared conformance suites (cf. testing/fstest and the sibling fsconformance/memconformance/storeconformance packages).
The suites pin the CONTRACT, not the implementation: where a skill bundle or soul body comes from (directories, a database, a remote process) is adapter-internal and deliberately NOT asserted. RunSkillSource is driven by the exported canonical Fixture so every backend — the in-memory reference (NewFixtureSource), the filesystem source over a written-out fixture tree, and a driver client over a wire server — answers for the SAME bundles.
Index ¶
- Variables
- func NewAgentFixtureSource() tool.AgentDefSource
- func NewCommandFixtureSource() prompt.CommandSource
- func NewFixtureSource() tool.SkillSource
- func NewRuleFixtureSource() prompt.RulesSource
- func RunAgentSource(t *testing.T, newSource func(t *testing.T) tool.AgentDefSource)
- func RunCommandSource(t *testing.T, newSource func(t *testing.T) prompt.CommandSource)
- func RunRulesSource(t *testing.T, newSource func(t *testing.T) prompt.RulesSource)
- func RunSkillSource(t *testing.T, newSource func(t *testing.T) tool.SkillSource)
- func RunSoulSource(t *testing.T, newSource func(t *testing.T, body string) prompt.SoulSource)
- type FixtureAsset
- type FixtureCommand
- type FixtureSkill
Constants ¶
This section is empty.
Variables ¶
var AgentFixture = []tool.AgentDef{ { Name: "echo", Description: "Repeats the task back with findings.", Body: "Restate the task, then answer it directly.", }, { Name: "full-stack", Description: "A fully-loaded specialist exercising every optional field.", Tools: []string{"Read", "Grep", "Glob"}, DisallowedTools: []string{"Write"}, Model: "sonnet", Provider: "openrouter", PermissionMode: "plan", MaxTurns: 7, MaxToolCalls: 21, Color: "blue", Skills: []string{"review", "research"}, Hooks: map[string]string{ "PreToolUse": "echo pre", "Stop": "echo done", }, Body: "You are a meticulous full-stack specialist.\nInspect before you conclude.", }, { Name: "mcp-ops", Description: "Operates over scoped MCP servers.", MCPServers: []tool.AgentMCPServer{ {Name: "github"}, { Name: "inline-auth", URL: "https://example.test/mcp", Headers: map[string]string{"Authorization": "Bearer fixture-token"}, }, }, Body: "Use the scoped MCP tools to inspect the project boards.", }, }
AgentFixture is the canonical agent-definition set RunAgentSource asserts against, sorted by Name. It covers the three shapes the suite needs: a minimal def (name + description + body only), a fully-loaded def (tools, disallowed tools, model, provider, permission mode, limits, color, skills, hooks), and an MCP-bearing def (one REFERENCE entry + one INLINE entry with secret-shaped headers).
The fixture is authored to ROUND-TRIP the filesystem frontmatter parser: bodies are trimmed, descriptions are single-line and under the always-in-context cap, and hooks/headers are pre-normalized (trimmed keys and values, no empties) — so the FS backend writing these out as <name>.md and re-discovering them answers byte-identically. Origin is deliberately UNSET here: each backend stamps its own admission tier, and the suite asserts only that it is non-empty.
var CommandFixture = []FixtureCommand{
{
Name: "fix",
Description: "Fix a reported bug.",
Body: "Fix the bug in $1 and add a regression test.",
},
{
Name: "pr.merge-2",
Description: "Merge a pull request (grammar: letters, digits, '-', '_', '.').",
Body: "Merge PR $1 once checks pass.",
},
{
Name: "review",
Description: "Run a structured review.",
Body: "---\ndescription: Run a structured review.\n---\nReview $ARGUMENTS for correctness first, style second.",
},
}
CommandFixture is the canonical command set RunCommandSource asserts against, sorted by Name. It covers a frontmatter-less positional template, a punctuation-bearing name (the full invocation grammar), and a frontmatter-bearing template with $ARGUMENTS — the raw body keeps its frontmatter so the round-trip subtest proves the source never strips.
var Fixture = []FixtureSkill{ { Name: "commit-style", Description: "Write conventional commits.", Body: "type(scope): subject\nWrap the body at 72 columns.", }, { Name: "research", Description: "Deep research with nested references.", Body: "Start from references/deep/sources.md and cite everything.", Assets: []FixtureAsset{ {Name: "references/deep/sources.md", Content: "primary sources only\n"}, }, }, { Name: "review", Description: "Run a structured code review.", Body: "Follow references/checklist.md, then run scripts/lint.sh.", License: "MIT", Compatibility: "mecatl >= 0.1", Metadata: map[string]string{ "author": "stacklok", "version": "1", }, AllowedTools: []string{"Read", "Grep", "Bash"}, Assets: []FixtureAsset{ {Name: "references/checklist.md", Content: "- correctness first\n- style second\n"}, {Name: "scripts/lint.sh", Content: "#!/bin/sh\necho lint\n", Executable: true}, }, }, }
Fixture is the canonical skill set RunSkillSource asserts against, sorted by Name. It covers the three shapes the suite needs: a skill with a text asset AND an executable asset (review), an asset-less skill (commit-style), and a skill with a multi-segment logical asset name (research).
var RuleFixture = []prompt.Rule{ { Name: "api", Body: "Follow the API style guide for every endpoint.", }, { Name: "go-style", Body: "Run gofmt and go vet before committing.", Paths: []string{"**/*.go", "go.mod"}, }, { Name: "testing", Body: "Write table-driven tests with t.Run subtests.", Paths: []string{"**/*_test.go"}, }, }
RuleFixture is the canonical rule set RunRulesSource asserts against, sorted by Name. It covers the three shapes the suite needs: an unconditional rule (no paths), a single-glob path-scoped rule, and a two-glob path-scoped rule.
The fixture is authored to ROUND-TRIP the filesystem frontmatter parser: bodies are trimmed and globs are written as a YAML sequence — so the FS backend writing these out as <name>.md and re-discovering them answers byte-identically. Origin is deliberately UNSET here: each backend stamps its own admission tier, and the suite asserts only that it is non-empty.
Functions ¶
func NewAgentFixtureSource ¶
func NewAgentFixtureSource() tool.AgentDefSource
NewAgentFixtureSource returns the in-memory REFERENCE tool.AgentDefSource serving exactly the canonical AgentFixture, with the explicit tier stamped. It lives here (not in a _test.go file) because the driver conformance fixtures mount it behind a wire server; it is also the suite's self-test subject, so the suite cannot smuggle filesystem-shaped assumptions.
func NewCommandFixtureSource ¶
func NewCommandFixtureSource() prompt.CommandSource
NewCommandFixtureSource returns the in-memory REFERENCE prompt.CommandSource serving exactly the canonical CommandFixture. It lives here (not in a _test.go file) because the driver conformance fixtures mount it behind a wire server; it is also the suite's self-test subject.
func NewFixtureSource ¶
func NewFixtureSource() tool.SkillSource
NewFixtureSource returns the in-memory REFERENCE tool.SkillSource serving exactly the canonical Fixture. It lives here (not in a _test.go file) because the driver conformance fixtures mount it behind a wire server; it is also the suite's self-test subject, so the suite cannot smuggle filesystem-shaped assumptions.
func NewRuleFixtureSource ¶
func NewRuleFixtureSource() prompt.RulesSource
NewRuleFixtureSource returns the in-memory REFERENCE prompt.RulesSource serving exactly the canonical RuleFixture, with the user tier stamped. It lives here (not in a _test.go file) because a future driver conformance fixture mounts it behind a wire server; it is also the suite's self-test subject, so the suite cannot smuggle filesystem-shaped assumptions.
func RunAgentSource ¶
RunAgentSource executes the shared AgentDefSource conformance table against the source produced by newSource. newSource must return a fresh source serving EXACTLY the canonical AgentFixture each call.
func RunCommandSource ¶
RunCommandSource executes the shared CommandSource conformance table against the source produced by newSource. newSource must return a fresh source serving EXACTLY the canonical CommandFixture each call.
NOTE on lifecycle: the PORT is LIVE-semantics (the set MAY change between calls); the fixture backend is FIXED, so the stability subtest pins only "a fixed backend lists stably", not snapshot semantics.
func RunRulesSource ¶
RunRulesSource executes the shared RulesSource conformance table against the source produced by newSource. newSource must return a fresh source serving EXACTLY the canonical RuleFixture each call.
func RunSkillSource ¶
RunSkillSource executes the shared SkillSource conformance table against the source produced by newSource. newSource must return a fresh source serving EXACTLY the canonical Fixture each call.
func RunSoulSource ¶
RunSoulSource executes the shared SoulSource conformance table. newSource must return a fresh source whose backing soul content is exactly body (e.g. a temp file holding body, or a wire server returning it); the suite asserts the FAIL-SOFT load discipline every implementation shares: a clean body round-trips trimmed, and an empty/whitespace-only/fence-breakout body yields ("", nil) — never an error that would abort a run.
Types ¶
type FixtureAsset ¶
FixtureAsset is one auxiliary payload of a fixture skill, addressed by its LOGICAL (slash-separated, relative) name.
type FixtureCommand ¶
FixtureCommand is one canonical fixture slash command: invocation metadata plus the RAW template body (optional YAML frontmatter INCLUDED — stripping is the expander's job, never the source's, so the body must round-trip verbatim).
type FixtureSkill ¶
type FixtureSkill struct {
Name string
Description string
Body string
Assets []FixtureAsset
// License, Compatibility, Metadata, and AllowedTools mirror the like-named
// SkillMeta fields; advisory only.
License string
Compatibility string
Metadata map[string]string
AllowedTools []string
}
FixtureSkill is one canonical fixture skill: metadata + instruction body + auxiliary payloads. The Body is a trimmed string (no leading/trailing whitespace) so filesystem backends that trim on parse round-trip it exactly.
License, Compatibility, Metadata, and AllowedTools are the OPTIONAL ADVISORY frontmatter fields (issue #419); they are advisory/observability only and mirror SkillMeta. Populate them on at least one fixture so the conformance contract pins their round-trip across every backend; zero values are exercised by the fixtures that omit them. AllowedTools (agentskills.io Experimental) is NEVER a permission grant — advisory only.
Source Files