README
¶
Testy
Docs incoming :)
We couldn't find a framework that addressed our API acceptance testing desires, so we're making one ourselves. This project is still a rapid work in progress, so full details will be added once things start to stabilize a bit and we're actually using this for tests.
This requires go 1.18 due to using type parameters.
Documentation
¶
Overview ¶
Package testy is a Go test running framework.
We couldn't find a framework that addressed our API acceptance testing desires, so we're making one ourselves. This project is still a rapid work in progress, so full details will be added once things start to stabilize a bit and we're actually using this for tests.
Index ¶
- func AddEchoRoutes(router *echo.Group)
- func AfterPackage(f Tester) any
- func AfterTest(f Tester) any
- func BeforePackage(f Tester) any
- func BeforeTest(f Tester) any
- func RunAsTest(t *testing.T)
- func Test(name string, tester Tester) any
- func TestEach[V any](t TestingT, values []V, tester func(TestingT, V))
- type Level
- type Msg
- type Result
- type TestResult
- type Tester
- type TestingT
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func AddEchoRoutes ¶
AddEchoRoutes adds routes to an Echo router that can run tests and retrieve tests results.
func AfterPackage ¶
AfterPackage registers a function to be run once after all tests in the given package have finished. A package may only have one AfterPackage function.
If AfterPackage panics, the reporting behavior depends on if RunAsTest or Run was called. If AfterPackage panics and BeforePackage had already panicked, then the panic for BeforePackage takes priority.
If RunAsTest was called, the panic will be reported as the top-level bootstrap test for the package. If Run was called, every test in the package will be marked as failed, and the panic's message will be appended to every test's own messages or the BeforePackage panic that replaced every test.
When run via Run, any logging output to the provided Tester will only be visible if AfterPackage panics or the Tester is marked as failed. When run via RunAsTest, the standard `go test` output rules apply. Notably, if a test fails, the output is not visible via Run but is via RunAsTest.
NOTE: Do not call TestingT.Fail, TestingT.FailNow, TestingT.Fatal, or TestingT.Fatalf in AfterPackage to report errors; always panic. It will work as expected in RunAsTest mode, but not Run mode. TODO parity here.
The return value may be discarded (and is always nil); it is provided to simplify writing test code, like so:
var _ = testy.AfterPackage(func(){})
func AfterTest ¶
AfterTest registers a function to be run once after every top level registered test in the given package has finished. It is not run after subtests created by `t.Run`. A package may only have one AfterTest function.
If AfterTest panics, the specific test that was just run will be marked as failed, and the panic's message will be appended to the test's own messages. Any subtests of the test will not be modified.
When run via Run, any logging output to the provided Tester will only be visible if AfterTest panics or the Tester is marked as failed. When run via RunAsTest, the standard `go test` output rules apply. Notably, if a test fails, the output is not visible via Run but is via RunAsTest.
NOTE: Do not call TestingT.Fail, TestingT.FailNow, TestingT.Fatal, or TestingT.Fatalf in AfterTest to report errors; always panic. It will work as expected in RunAsTest mode, but not Run mode. TODO parity here.
The return value may be discarded (and is always nil); it is provided to simplify writing test code, like so:
var _ = testy.AfterTest(func(){})
func BeforePackage ¶
BeforePackage registers a function to be run once before any tests in the given package are run. A package may only have one BeforePackage function.
If BeforePackage panics, no tests in the package will be run and the reporting behavior depends on if RunAsTest or Run was called. AfterPackage will still be run.
If RunAsTest was called, the panic will be reported as the top-level bootstrap test for the package. If Run was called, every registered test in the package will be marked as failed with the panic's message.
When run via Run, any logging output to the provided Tester will only be visible if BeforePackage panics or the Tester is marked as failed. When run via RunAsTest, the standard `go test` output rules apply. Notably, if a test fails, the output is not visible via Run but is via RunAsTest.
NOTE: Do not call TestingT.Fail, TestingT.FailNow, TestingT.Fatal, or TestingT.Fatalf in BeforePackage to report errors; always panic. It will work as expected in RunAsTest mode, but not Run mode. TODO parity here.
The return value may be discarded (and is always nil); it is provided to simplify writing test code, like so:
var _ = testy.BeforePackage(func(){})
func BeforeTest ¶
BeforeTest registers a function to be run before every top level registered test in the given package is run. It is not run before subtests created by `t.Run`. A package may only have one BeforeTest function.
If BeforeTest panics, the specific registered test that was about to be invoked will be marked as failed with the panic's message. AfterTest will still be run.
When run via Run, any logging output to the provided Tester will only be visible if BeforeTest panics or the Tester is marked as failed. When run via RunAsTest, the standard `go test` output rules apply. Notably, if a test fails, the output is not visible via Run but is via RunAsTest.
NOTE: Do not call TestingT.Fail, TestingT.FailNow, TestingT.Fatal, or TestingT.Fatalf in BeforeTest to report errors; always panic. It will work as expected in RunAsTest mode, but not Run mode. TODO parity here.
The return value may be discarded (and is always nil); it is provided to simplify writing test code, like so:
var _ = testy.BeforeTest(func(){})
func RunAsTest ¶
RunAsTest runs all registered tests under Go's testing framework. To run tests on a per-package basis, put a test file in each package containing a single test that calls this function. This is recommended so accurate per-package execution times are reported, as well as using the test cache. Do not import a test package into another test package as that will cause the tests in the second package to get executed with the first package. If code or resources need shared between test packages, put them in their own package which does not contain any test definitions.
Individual tests in a package may still be run using the standard -run test flag. See `go help testflag` for more information.
TODO: shuffle test execution order (see -shuffle in `go help testflag`)
func Test ¶
Test registers a new test to be run. Tests are run in lexicographical order within a package. TODO: support the -shuffle testflag. though maybe that only has to be in the runner?
The return value may be discarded (and is always nil); it is provided to simplify writing test code, like so:
var _ = testy.Test("my test", func(t testy.TestingT){})
Types ¶
type TestResult ¶
type TestResult struct {
Package string
Name string
Msgs []Msg
Result Result
Dur time.Duration
DurHuman string
Subtests []TestResult
}
func Run ¶
func Run() TestResult
Run runs all registered tests and returns result information about them.
TODO: ability to filter for specific packages and tests TODO: shuffle test execution order (see -shuffle in `go help testflag`) TODO: channel for results to support progressive result loading?
func (TestResult) FindFailingTests ¶ added in v0.1.0
func (tr TestResult) FindFailingTests() []TestResult
FindFailingTests finds the least deeply nested subtests that have sibling tests that passed. These subtests may be in different branches of subtests. This implies that this test failed; if it did not, then a nil slice is returned. If every subtest of test failed or if test has no subtests, then test itself is returned.
func (TestResult) SumTestStats ¶ added in v0.1.0
func (tr TestResult) SumTestStats() (total, passed, failed int)
SumTestStats returns the total number of leaf subtests, as well as the number of those that passed and failed.
type TestingT ¶
type TestingT interface {
Fail()
FailNow()
Fatal(args ...interface{})
Fatalf(format string, args ...interface{})
Errorf(format string, args ...interface{})
Helper()
Log(args ...interface{})
Logf(format string, args ...interface{})
Name() string
Run(string, Tester) bool
}
TestingT is a subset of testing.T that we have to implement for non-`go test` runs.
TODO flesh this out with more useful stuff from testing.T -- Parallel would be nice but tricky
Source Files
Directories