Documentation
¶
Overview ¶
Package scripttest runs a managed script's tests (#1939) and measures the statements they reach (#1940).
A test is a top-level def named test_*, versioned with the script's source. It names at most one recording with testing.replay("<run id>"), as a string literal, calls main() or any other function, and asserts on what the execution produced with the assert module and testing.outputs(). Every host call is answered from the recording (internal/platform/scriptrec), so a test never reaches an upstream, and a write binding records what it would have written: an export is kept as its rows, a notify as its arguments. A call the recording holds no answer for fails the test, naming the call.
Each test runs in a fresh module, so nothing one test does is seen by the next. Every test is instrumented for statement coverage; the statements of the tests themselves are not counted.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func RecordingsNamed ¶
RecordingsNamed is every recording the source's tests name, sorted, or nil when the source does not parse.
Types ¶
type AssertionError ¶
AssertionError is an assertion that did not hold: what it expected, and the line of the test that asserted it.
func (*AssertionError) Error ¶
func (f *AssertionError) Error() string
type Contracts ¶
type Contracts interface {
Check(tool string, args, answer map[string]any) (checked bool, err error)
}
Contracts holds a declared answer to what its tool always answers (internal/toolanswer). checked is false when no contract covers the call.
type Coverage ¶
type Coverage struct {
Statements int `json:"statements"`
Covered int `json:"covered"`
Percent float64 `json:"percent"`
MissedLines []int `json:"missed_lines"`
}
Coverage is the statements the tests reached.
type Loader ¶
Loader reads the recording a test names, applying who may read it: a recording is its script owner's and an administrator's.
type Outcome ¶
type Outcome struct {
Exports []scriptrun.ExportRequest
Publishes []scriptrun.PublishRequest
// State is what the execution staged with platform.save_state, nil when
// it staged nothing.
State map[string]any
// Calls is every tool call the execution made, in order.
Calls []scriptrec.Made
// Result is what platform.result handed back, empty when nothing was.
Result json.RawMessage
// Failure is the error the execution stopped with, empty when it
// finished. Missing names the call it stopped at when that call is one
// the recording holds no answer for.
Failure string
Missing string
}
Outcome is what one execution of a source against a recording produced, in the platform's terms rather than a test's: the regression replay a save runs compares two of them (#1942).
type Report ¶
type Report struct {
Tests []Result `json:"tests"`
Passed int `json:"passed"`
Failed int `json:"failed"`
Coverage Coverage `json:"coverage"`
// Unread is every output the tests' executions produced that no test
// read (#1952), as output "weekly" column "region".
Unread []string `json:"unread"`
}
Report is what running a source's tests found.
type Request ¶
type Request struct {
Source string
Name string
// Destinations is the deployment's bucket destinations, which an export a
// test reaches resolves against as a run's would.
Destinations []script.Destination
Load Loader
// MaxMemoryBytes is the memory one test may hold; zero sets no budget.
MaxMemoryBytes int64
// Contracts is what a declared answer is held to; nil checks none.
Contracts Contracts
// Libraries is where the libraries the source loads are read from
// (#1941), the same pinned source a run reads.
Libraries scriptlib.Source
}
Request is one run of a source's tests.
type Result ¶
type Result struct {
Name string `json:"name"`
Passed bool `json:"passed"`
Recording string `json:"recording,omitempty"`
// Failure is what failed: the assertion, or the error the execution
// stopped with. Line is the line of the script it happened at.
Failure string `json:"failure,omitempty"`
Line int `json:"line,omitempty"`
Log string `json:"log,omitempty"`
// Notes is what the author is told about a test that passed or failed
// alike: an answer declared for a tool that declares no answer contract,
// which nothing checked.
Notes []string `json:"notes,omitempty"`
}
Result is one test's outcome.
Source Files