Documentation
¶
Overview ¶
Package extensiontest is the test kit of an extension repository, as Test kit in contract.md defines. One test of the repository calls Run. Run checks the links between the manifest, the skill, the scenarios, the spec, and the Nicely versions. It installs the real ncly-<slug> and the ncly of go tool ncly in scratch folders. It runs manifest examples there, then the scenarios of testdata/script/. Coverage requires a checked JSON answer or a completed success with a declared text result.
A scenario runs the extension through the commands that the kit adds, so that the kit checks each answer against the schema of its command:
ncly <arg>... runs ncly, which exits 0, or not 0 after ! ncly-<slug> <arg>... runs the extension directly, the same way exits <code> <command> <arg> runs a command, these two included, and checks its exact exit code signal INT|TERM <marker> <command> <arg> checks the host's signal exit and answer survives <file>... requires finished files before and after the next signal appears <marker> <path> <command> <arg> requires a failure that keeps a new file protect <path>... selects paths that refuses and the next signal or appears keep refuses <code> <command> <arg> checks the same refusal in a dry run and a run grant <service> grants the key of a declared service to the extension fake <name> puts the fake child in $WORK/bin as name
The rules of its code:
- No case reads the user's keychain, HOME, or XDG folders, or finds the user's extensions. Each scenario gets a scratch HOME and scratch XDG folders, and a PATH without the folders that hold an ncly-* file. Each service that the manifest declares gets the sentinel key sentinel-<service> in its variable, so ncly never reads the keychain for it. On Linux, D-Bus names no bus, so a read that a scenario forces finds no keychain
- The kit imports no package of core's internal/, which an extension repository cannot import either
- The kit builds every program that it runs from Go source: the extension, ncly, and the fake child of the fake/ folder
- The kit copies each program to where it runs, and never makes a hard link. macOS checks a new program at its first start through one name of its file. A scenario deletes its names when it ends. A check through such a name fails, and macOS kills the start with exit 137. See #371
- Each failure names its file, the missing link, and its fix
Index ¶
- func Run(t *testing.T, e extension.Extension)
- type Coverage
- func (c *Coverage) Missing() []string
- func (c *Coverage) MissingCodes() []string
- func (k *Coverage) Observe(words []string, exit int, stdout, stderr, jsonVariable string) string
- func (c *Coverage) Reservations() []string
- func (c *Coverage) RunExamples(t *testing.T, fixture string, params testscript.Params)
- func (c *Coverage) ScenarioCommands(host string) map[string]func(*testscript.TestScript, bool, []string)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Coverage ¶ added in v0.23.4
type Coverage struct {
// contains filtered or unexported fields
}
Coverage checks answers and records calls across parallel scenarios. Other scenario commands use Observe after execution and Missing after all scenarios finish.
func NewCoverage ¶ added in v0.23.4
NewCoverage prepares the manifest and answer checks that Run uses.
func (*Coverage) Missing ¶ added in v0.23.4
Missing reports commands and declared codes without validated observations, in manifest order.
func (*Coverage) MissingCodes ¶ added in v0.23.4
MissingCodes checks declared codes without requiring foreground coverage for commands that a runner exercises only in the background.
func (*Coverage) Observe ¶ added in v0.23.4
Observe checks and records a completed call's answer. words holds the arguments after ncly, including the root for direct calls. It returns a problem without recording an invalid answer. A declared text result counts a completed success without JSON.
func (*Coverage) Reservations ¶ added in v0.23.4
Reservations lists each untested declaration and its reason for the UAT report, in manifest order.
func (*Coverage) RunExamples ¶ added in v0.23.4
RunExamples runs manifest examples with the runner's scenario setup and commands. The optional fixture supplies setup and files. Example answers never earn scenario coverage.
func (*Coverage) ScenarioCommands ¶ added in v0.23.4
func (c *Coverage) ScenarioCommands(host string) map[string]func(*testscript.TestScript, bool, []string)
ScenarioCommands supplies signal, appears, refuses, protect, and survives to a testscript runner. host is the registered ncly's absolute path, independent of the scenario's PATH.