Documentation
¶
Overview ¶
Package extensiontest is the test kit of an extension repository, as Test kit in contract.md defines. One test of the repository calls Run. Run checks the links between the manifest, the skill, the scenarios, the spec, and the Nicely versions. It installs the real ncly-<slug> and the ncly of go tool ncly in scratch folders. It runs manifest examples there, then the scenarios of testdata/script/. Coverage requires a checked JSON answer or a completed success with a declared text result.
A scenario runs the extension through the commands that the kit adds, so that the kit checks each answer against the schema of its command:
ncly <arg>... runs ncly, which exits 0, or not 0 after !
ncly-<slug> <arg>... runs the extension directly, the same way
exits <code> <command> <arg> runs a command, these two included, and checks its exact exit code
signal INT|TERM <marker> <command> <arg> checks the host's signal exit and answer
survives <file>... requires finished files before and after the next signal
appears <marker> <path> <command> <arg> requires a failure that keeps a new file
protect <path>... selects paths that refuses and the next signal or appears keep
refuses <code> <command> <arg> checks the same refusal in a dry run and a run
predicts <command> <arg>... checks that a dry run predicts the publication of its run
reads [-case <case>] [-limit <id>] [-private <text>]... <output> <reference> <bad> <reader> [<arg>...]
reads the output with an independent reader, as Acceptance cases defines
keeps <exit> <command> <arg> checks the exact exit, the protected paths, and the children of any call
holds <marker> <path> <command> <arg> checks the public path while the worker waits, then the commit
without <operation> <check> <arg> runs refuses, predicts, appears, or keeps without that operation, on Linux
grant <service> grants the key of a declared service to the extension
fake <name> puts the fake child in $WORK/bin as name
The rules of its code:
- No case reads the user's keychain, HOME, or XDG folders, or finds the user's extensions. Each scenario gets an owned scratch run of extensionrepo.NewOwnedRun. It has a scratch HOME, TMPDIR, and XDG folders, and a PATH without the folders that hold an ncly-* file. Each service that the manifest declares gets the sentinel key sentinel-<service> in its variable, so ncly never reads the keychain for it. On Linux, D-Bus names no bus, so a read that a scenario forces finds no keychain
- No code of the kit reads the environment of the caller as a whole. The analyzer callerenv refuses os.Environ here
- The kit imports no package of core's internal/, which an extension repository cannot import either
- The kit builds every program that it runs from Go source: the extension, ncly, and the fake child of the fake/ folder. On Linux, it also builds the guard of the guard/ folder
- The kit copies each program to where it runs, and never makes a hard link. macOS checks a new program at its first start through one name of its file. A scenario deletes its names when it ends. A check through such a name fails, and macOS kills the start with exit 137. See #371
- Each failure names its file, the missing link, and its fix
Index ¶
- func Run(t *testing.T, e extension.Extension)
- type Coverage
- func (c *Coverage) Missing() []string
- func (c *Coverage) MissingCodes() []string
- func (k *Coverage) Observe(words []string, exit int, stdout, stderr, jsonVariable string) string
- func (c *Coverage) Reservations() []string
- func (c *Coverage) RunExamples(t *testing.T, fixture string, params testscript.Params)
- func (c *Coverage) ScenarioCommands(host string) map[string]func(*testscript.TestScript, bool, []string)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Coverage ¶ added in v0.23.4
type Coverage struct {
// contains filtered or unexported fields
}
Coverage checks answers and records calls across parallel scenarios. Other scenario commands use Observe after execution and Missing after all scenarios finish.
func NewCoverage ¶ added in v0.23.4
NewCoverage prepares the manifest and answer checks that Run uses.
func (*Coverage) Missing ¶ added in v0.23.4
Missing reports commands and declared codes without validated observations, in manifest order.
func (*Coverage) MissingCodes ¶ added in v0.23.4
MissingCodes checks declared codes without requiring foreground coverage for commands that a runner exercises only in the background.
func (*Coverage) Observe ¶ added in v0.23.4
Observe checks and records a completed call's answer. words holds the arguments after ncly, including the root for direct calls. It returns a problem without recording an invalid answer. A declared text result counts a completed success without JSON.
func (*Coverage) Reservations ¶ added in v0.23.4
Reservations lists each untested declaration and its reason for the UAT report, in manifest order.
func (*Coverage) RunExamples ¶ added in v0.23.4
RunExamples runs manifest examples with the runner's scenario setup and commands. The optional fixture supplies setup and files. Example answers never earn scenario coverage.
func (*Coverage) ScenarioCommands ¶ added in v0.23.4
func (c *Coverage) ScenarioCommands(host string) map[string]func(*testscript.TestScript, bool, []string)
ScenarioCommands supplies signal, appears, refuses, protect, and survives to a testscript runner. host is the registered ncly's absolute path, independent of the scenario's PATH.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Command fake is the fake child of the test kit.
|
Command fake is the fake child of the test kit. |
|
Command guard runs one call of a scenario for the test kit, which builds it from this source, as it builds the fake child.
|
Command guard runs one call of a scenario for the test kit, which builds it from this source, as it builds the fake child. |