Documentation
¶
Overview ¶
TODO - frame2.Logf (f,v) - frame2.LogVerbosef (f,v) - frame2.LogLevelf (frame2.Level, f, v) - Shared namespaces - Redo Retry testing, other meta_test with Runner? - Example of generating Runner table from simpler table - Example using test-specific Runner (discouraged?) - Example using inter-test communication via references - Auto Doc() or require Doc - Retry report (logged): len, %, Batalha naval. ▂▃▄...█ if more than 40. Else, individual: ▌▐ █. - Implement all substep/substeps, validator/validators, etc options
Index ¶
- Constants
- Variables
- func ContextOrDefault(ctx context.Context) context.Context
- func Flag()
- func GetId() string
- func GetInt(name string, default_ int) int
- func GetShortId() string
- func IsVerboseCommandOutput() bool
- func OrSetLogger(x FrameLogger, logger *log.Logger)
- func SourceRoot() string
- type AlwaysDisruptor
- type Asserter
- type DefaultMonitor
- type DefaultRunDealer
- type Disruptor
- type DisruptorConfigurer
- type Execute
- type Executor
- type Expect
- type FinalizerHook
- type FrameLogger
- type Inspector
- type Log
- type Monitor
- type MonitorResult
- type Phase
- type PostMainSetupHook
- type Procedure
- type Retry
- type RetryFunction
- type RetryOptions
- type Run
- func (r *Run) AllowDisruptors(list []Disruptor)
- func (r *Run) CancelContext()
- func (r *Run) ChildWithT(t *testing.T, kind RunnerType) *Run
- func (r *Run) Finalize()
- func (r *Run) GetContext() context.Context
- func (r *Run) GetId() string
- func (r *Run) OrDefaultContext(ctx context.Context) context.Context
- func (r *Run) Report()
- func (r *Run) ReportChildren(indent int)
- type RunDealer
- type RunnerType
- type Step
- type Stepper
- type TearDowner
- type TransformFunc
- type ValidationResultHook
- type Validator
- type ValidatorConfig
Constants ¶
const ( // Define the upgrade strategy used by the Upgrade disruptor (possibly // other points as well?) // // Values: // // CREATION (default): order of skupper init // PUB_FIRST // PRV_FIRST // PUB_ONLY // PRV_ONLY // LEAVES_FIRST // LEAVES_ONLY // CORE_FIRST // CORE_ONLY // EDGES_FIRST // EDGES_ONLY // INTERIOR_FIRST // INTERIOR_ONLY // // Currently, only CREATION and CREATION:INVERSE are implemented // // For any of the options, if the value ends with :INVERSE, the order // is inverted. For example, ":INVERSE" or "CREATION:INVERSE" will // upgrade the lastly installed skupper site first; the first last. // // Valid values are of the string type TestUpgradeStrategy ENV_UPGRADE_STRATEGY = "SKUPPER_TEST_UPGRADE_STRATEGY" // A path to the Skupper binary to be used (the actual file, not just its parent directory) ENV_OLD_BIN = "SKUPPER_TEST_OLD_BIN" // The version that ENV_OLD_BIN refers to, such as 1.2 or 1.4.0-rc3 ENV_OLD_VERSION = "SKUPPER_TEST_OLD_VERSION" // The expected version of the skupper binary found on the PATH ENV_VERSION = "SKUPPER_TEST_VERSION" EnvOldConfigSyncImageEnvKey string = "SKUPPER_TEST_OLD_SKUPPER_CONFIG_SYNC_IMAGE" EnvOldConfigSyncPullPolicyEnvKey string = "SKUPPER_TEST_OLD_SKUPPER_CONFIG_SYNC_IMAGE_PULL_POLICY" EnvOldFlowCollectorImageEnvKey string = "SKUPPER_TEST_OLD_SKUPPER_FLOW_COLLECTOR_IMAGE" EnvOldFlowCollectorPullPolicyEnvKey string = "SKUPPER_TEST_OLD_SKUPPER_FLOW_COLLECTOR_IMAGE_PULL_POLICY" EnvOldPrometheusServerImageEnvKey string = "SKUPPER_TEST_OLD_PROMETHEUS_SERVER_IMAGE" EnvOldPrometheusServerPullPolicyEnvKey string = "SKUPPER_TEST_OLD_PROMETHEUS_SERVER_IMAGE_PULL_POLICY" EnvOldRouterImageEnvKey string = "SKUPPER_TEST_OLD_QDROUTERD_IMAGE" EnvOldRouterPullPolicyEnvKey string = "SKUPPER_TEST_OLD_QDROUTERD_IMAGE_PULL_POLICY" EnvOldServiceControllerImageEnvKey string = "SKUPPER_TEST_OLD_SKUPPER_SERVICE_CONTROLLER_IMAGE" EnvOldServiceControllerPullPolicyEnvKey string = "SKUPPER_TEST_OLD_SKUPPER_SERVICE_CONTROLLER_IMAGE_PULL_POLICY" EnvOldSkupperImageRegistryEnvKey string = "SKUPPER_TEST_OLD_SKUPPER_IMAGE_REGISTRY" EnvOldPrometheusImageRegistryEnvKey string = "SKUPPER_TEST_OLD_PROMETHEUS_IMAGE_REGISTRY" // Starting with 1.5 EnvOldOauthProxyImageEnvKey string = "SKUPPER_TEST_OLD_OAUTH_PROXY_IMAGE" EnvOldOauthProxyPullPolicyEnvKey string = "SKUPPER_TEST_OLD_OAUTH_PROXY_IMAGE_PULL_POLICY" EnvOldControllerPodmanImageEnvKey string = "SKUPPER_TEST_OLD_SKUPPER_CONTROLLER_PODMAN_IMAGE" EnvOldControllerPodmanPullPolicyEnvKey string = "SKUPPER_TEST_OLD_SKUPPER_CONTROLLER_PODMAN_IMAGE_PULL_POLICY" EnvOldOauthProxyRegistryEnvKey string = "SKUPPER_TEST_OLD_OAUTH_PROXY_IMAGE_REGISTRY" )
TODO: Move all skupper-specific variables to a skupper-specific file, on a skupper-specific package
const ( // If defined, both stdout and stderr of all issued skupper commands // will be shown on the test output, even if they did not fail ENV_CLI_VERBOSE_COMMANDS = "SKUPPER_TEST_CLI_VERBOSE_COMMANDS" )
const ( // This sets the 'Allow' parameter of the retry block for the final // validations, and needs to be an integer value. Any validations // marked as final will be retried this many times at the end of the // test. ENV_FINAL_RETRY = "SKUPPER_TEST_FINAL_RETRY" )
const EnvFrame2Verbose = "SKUPPER_TEST_FRAME2_VERBOSE"
Variables ¶
var EnvOldMap = map[string]string{ EnvOldConfigSyncImageEnvKey: "SKUPPER_CONFIG_SYNC_IMAGE", EnvOldConfigSyncPullPolicyEnvKey: "SKUPPER_CONFIG_SYNC_IMAGE_PULL_POLICY", EnvOldFlowCollectorImageEnvKey: "SKUPPER_FLOW_COLLECTOR_IMAGE", EnvOldFlowCollectorPullPolicyEnvKey: "SKUPPER_FLOW_COLLECTOR_IMAGE_PULL_POLICY", EnvOldPrometheusServerImageEnvKey: "PROMETHEUS_SERVER_IMAGE", EnvOldPrometheusServerPullPolicyEnvKey: "PROMETHEUS_SERVER_IMAGE_PULL_POLICY", EnvOldRouterImageEnvKey: "QDROUTERD_IMAGE", EnvOldRouterPullPolicyEnvKey: "QDROUTERD_IMAGE_PULL_POLICY", EnvOldServiceControllerImageEnvKey: "SKUPPER_SERVICE_CONTROLLER_IMAGE", EnvOldServiceControllerPullPolicyEnvKey: "SKUPPER_SERVICE_CONTROLLER_IMAGE_PULL_POLICY", EnvOldSkupperImageRegistryEnvKey: "SKUPPER_IMAGE_REGISTRY", EnvOldPrometheusImageRegistryEnvKey: "PROMETHEUS_IMAGE_REGISTRY", EnvOldOauthProxyImageEnvKey: "OAUTH_PROXY_IMAGE", EnvOldOauthProxyPullPolicyEnvKey: "OAUTH_PROXY_IMAGE_PULL_POLICY", EnvOldControllerPodmanImageEnvKey: "SKUPPER_CONTROLLER_PODMAN_IMAGE", EnvOldControllerPodmanPullPolicyEnvKey: "SKUPPER_CONTROLLER_PODMAN_IMAGE_PULL_POLICY", EnvOldOauthProxyRegistryEnvKey: "OAUTH_PROXY_IMAGE_REGISTRY", }
final
The map between the variables that indicate the image value for the old version, and the environment variable that actually needs to be set on the environment for that configuration to be effective. Perhaps it would be simpler to just s/SKUPPER_TEST_OLD//?
Functions ¶
func ContextOrDefault ¶
If the given context is not nil, return it.
Otherwise, return a default context.
For now, that's a brand new context.Background(), but that might change
func Flag ¶
func Flag()
Right now, this will only add the -H flag for showing the flag's help (flag.Usage())
func GetId ¶
func GetId() string
Returns the string form of the UUID that identifies this test execution
Every execution has a unique ID, that is logged at the very start of the execution.
func GetInt ¶
Returns the integer value of the named variable; returns the default value if not defined or empty. If there is a value and not an int, panic
TODO: once this file is moved to a package 'env', the function name 'env.GetInt' will make more sense
func GetShortId ¶
func GetShortId() string
Returns the small format of GetId(). It's the three first hex characters of the MD5 form of GetId()
func IsVerboseCommandOutput ¶
func IsVerboseCommandOutput() bool
func OrSetLogger ¶
func OrSetLogger(x FrameLogger, logger *log.Logger)
This checks that its input is a frame2.Logger, then calls OrSetLogger on it.
Its only objective is to keep the code concise (ie, remove all those type checks from the main code)
x should be a pointer to a FrameLogger instance
func SourceRoot ¶
func SourceRoot() string
Returns the root of the source directory
This assumes that this very file is located at the pkg directory, from the source root. Refactoring may require changes to the code
Types ¶
type AlwaysDisruptor ¶
type AlwaysDisruptor interface {
// This is just a marker; it does nothing
AlwaysDisruptor()
}
This is just a marker to indicate that the disruptor does not need to be listed on Run.AlwaysDisruptor on the test; just having it on the environment will suffice for it to take effect
type Asserter ¶
type Asserter struct {
// contains filtered or unexported fields
}
A helper to check a list of conditions and report them back as a block
func (*Asserter) Check ¶
Checks the condition and returns a new error if false.
The error is also saved on the list of errors in the Asserter, and the counters are updated
func (*Asserter) CheckError ¶
Updates the counters and error list, and return its input
type DefaultMonitor ¶
type DefaultMonitor struct {
// A list of validators and their names, which cannot conflict
// with those in ValidatorConfigs
Validators map[string]Validator
// The interval between Validator runs. For ValidatorConfigs, that's
// the default, if not specified there
Interval time.Duration
// Same as Validators, but allow for per-Validator configuration
ValidatorConfigs []ValidatorConfig
// TODO Path where the report should be put. If not set, TODO
Path string
Results map[string][]MonitorResult
Log
// contains filtered or unexported fields
}
func (*DefaultMonitor) Execute ¶
func (m *DefaultMonitor) Execute() error
This actually only sets up the monitor, internally. The actual execution of the monitor is on Monitor(r).
For any validators configured on this Monitor with an empty Logger configuration, the Monitor will replace it by a no-op Monitor (log.New(io.Discard, "[M]", 0))
func (*DefaultMonitor) Monitor ¶
func (m *DefaultMonitor) Monitor(runner *Run) error
func (*DefaultMonitor) Teardown ¶
func (m *DefaultMonitor) Teardown() Executor
type DefaultRunDealer ¶
type DefaultRunDealer struct {
// Perhaps make this private?
// For now it is public to break less things
Runner *Run
// TODO
// When implemented, should it really be here, or have another one
// for it?
// This needs to be public, so user can change it at will?
Ctx context.Context
}
DefaultRunDealer is a helper for building frames of more complex behavior
This struct should be embedded into frames that deal with the Context, or that execute other frames via a frame2.Phase.
The framework will automatically update their Runner.
This should be always embedded directly, and never as a reference type.
func (*DefaultRunDealer) GetRunner ¶
func (d *DefaultRunDealer) GetRunner() *Run
func (*DefaultRunDealer) SetRunner ¶
func (d *DefaultRunDealer) SetRunner(parent *Run, kind RunnerType)
type Disruptor ¶
type Disruptor interface {
DisruptorEnvValue() string
}
TODO Add a new method, Exercised() bool, that reports whether the disruptor was exercised at all, and causes the test to report Skipped if not.
EdgeOnPrivate, for example, should report as not exercised if all namespaces were Public on the test;
Those should probably also be checked for subfinal tests and reset after them; so subtests are checked as well
type DisruptorConfigurer ¶
type Expect ¶
type Expect struct {
StdOut []string
StdErr []string
StdOutRe []regexp.Regexp
StdErrRe []regexp.Regexp
StdOutReNot []regexp.Regexp
StdErrReNot []regexp.Regexp
}
A way to verify cli commands output
StdOut and StdErr take a slice of plain strings. It will expect that each string comes after the previous one. In other words, the search for the second string from StdOut starts where the match for the first one finished.
If you want to search for one static string instead, where there is nothing in between each segment, just use a single item with one big string
StdOutRe and StdErrRe take a slice of regular expressions. Those do not have the same restriction on one coming after the other. If you want that behavior with regexes, create a single regex with the two expressions you're looking for.
StdOutReNot and StdErrReNot behave like the previous ones, but ensure that the patters are not there in the checked string
type FinalizerHook ¶
type FinalizerHook interface {
// FinalizerHook will be executed at the end of the
// test, before all other finalizer tasks, such as the
// re-run of validators marked as final
PreFinalizerHook(runner *Run) error
// PostSubFinalizerHook will be executed after the
// final validators are run. It can be used, for example,
// to reset the disruptor, so it starts a new on the
// next sub test. On an upgrade disruptor, for example,
// the next cycle that has a subfinalizer needs to start
// with the old version again.
PostSubFinalizerHook(runner *Run) error
}
type FrameLogger ¶
type FrameLogger interface {
// passes its parameters to a log.Logger's Printf. Either
// the one returned by GetLogger, or the default logger if
// that call returns nil
Printf(format string, v ...any)
GetLogger() *log.Logger
SetLogger(logger *log.Logger)
// Sets the logger, but only if it is currently nil
OrSetLogger(logger *log.Logger)
}
type Inspector ¶
Disruptors that implement the Inspector interface will have its Inspect() function called before each step is executed.
The disruptor will then be able to analise whether that step is of interest for it or not, or even change the step's configuration
type Log ¶
Unconfigured, this simply forwards any Printf calls to the default logger.
If a log.Logger is set, use it instead.
func (*Log) OrSetLogger ¶
Sets the logger, but only if it is currently nil and the new value is not nil
type MonitorResult ¶
type Phase ¶
type Phase struct {
Name string
Doc string
Setup []Step
Teardown []Step
MainSteps []Step
Runner *Run
Log
DefaultRunDealer
// contains filtered or unexported fields
}
While the Run keeps context accross the test, the phases do the actual work, and they allow the test to be split in multiple pieces.
This allows to: - Put a phase on a loop - Access variables populated in previous phases
type PostMainSetupHook ¶
PostMainSetupHook will be executed right after the setup phase completes, before the main steps.
type Procedure ¶
type Procedure struct {
Fn func()
}
Calls a function with no args and no return values; use it to cancel contexts, for example
type Retry ¶
type Retry struct {
Fn RetryFunction // The thing to be retried
Options RetryOptions
}
func (Retry) ParallelRun ¶
Runs the retry in parallel; returns a function that will wait and return the results only when it finished (wait). TODO perhaps give it a context, too
type RetryFunction ¶
type RetryFunction func() (err error)
type RetryOptions ¶
type RetryOptions struct {
Allow int // for initial failures
Ignore int // initial successes
Ensure int // last n tries are successful. Minimum 1
Retries int // after Allow phase
Interval time.Duration // if not given, the default is 1s
Quiet bool // if true, no attempt logs
Min int // TODO Run as normal, but delay report until that number of tries have been done
Rate float32 // TODO: "Ensure" will not be 100%, but based on this rate. So, if Ensure is 100
KeepTrying bool
Ctx context.Context
Timeout time.Duration
}
Allow accounts for instabilities (for example, a service load balanced on two providers might return a mix of successes and failures while the two providers stabilize). The last success streak in this phase will count to Ensure.
Once past the Allow phase, any errors will cause a failure, unless there are Retries available
Even successes may require additional runs. There are two cases here:
- If Ensure is set, the test will keep trying on success until the required number of successes are met
- If Ignore was set, that number of successes will be ignored on the count to Ensure, possibly requiring additional runs until the Ensure target is met
These, however, do not count as Retries. i. e, Retries are only those additional runs that were caused by a failure.
The ignore counts from the first success in the last success streak from the Allow phase, or from the start of the retry phase (if no Allow configured or no success in that phase)
func (RetryOptions) IsEmpty ¶
func (r RetryOptions) IsEmpty() bool
Checks whether any fields on the struct have been set
func (RetryOptions) Max ¶
func (r RetryOptions) Max(other RetryOptions) (RetryOptions, context.CancelFunc)
Returns a new RetryOptions, whose values are the maximum between r and other.
The contexts are 'merged'; if any of them is cancelled, the merged context gets cancelled. Cancelling the merged context has no effect on the other contexts
type Run ¶
type Run struct {
T *testing.T
Doc string // TODO this is currently unused (ie, it's not printed)
RequiredDisruptors []Disruptor // TODO
// contains filtered or unexported fields
}
The Run (TODO rename?) keeps context accross the execution of the test. Each phase and each step has its runner, in a tree structure
func (*Run) AllowDisruptors ¶
List the disruptors that a test accepts, and initialize a disruptor if SKUPPER_TEST_DISRUPTOR set on the environment matches a disruptor from the list.
If any of the values on the environment variable does not match a value on the list, the test will be skipped in this run (ie, a disruptor test was requested, but the test does not allow for it).
If the environment variable is empty, this is a no-op.
Attention when calling with disruptors that use pointer reference methods: define them on the list as a reference to the struct. Otherwise, the pointer reference methods will not be part of the method set, and some interfaces may not match
func (*Run) CancelContext ¶
func (r *Run) CancelContext()
func (*Run) ChildWithT ¶
func (r *Run) ChildWithT(t *testing.T, kind RunnerType) *Run
TODO: make just Child(), which reuses the runner's own T
func (*Run) Finalize ¶
func (r *Run) Finalize()
Run steps that are still part of the test, but must be run at its very end, right before the tear down. Failures here will count as test failure
func (*Run) GetContext ¶
GetContext will always return a context. If not defined on the current level, check the parent. If not on the parent, create a new Background context.
When contexts are created on this method, they get scheduled for cancellation on T.Cleanup()
TODO contexts are not expected to change?
func (*Run) OrDefaultContext ¶
Return ctx if not nil. If nil, return the runner's default context
If the runner is not available (call on nil reference), return context.Background ¶
TODO review
func (*Run) Report ¶
func (r *Run) Report()
This will cause all active monitors to report their status on the logs.
It should generally be run as defer r.Report(), right after the Run creation
func (*Run) ReportChildren ¶
type RunDealer ¶
type RunDealer interface {
SetRunner(parent *Run, kind RunnerType)
GetRunner() *Run
}
TODO: add SetContext and GetContext, too?
type RunnerType ¶
type RunnerType int
Each step on a phase runs with its own Runner. These constants identify what type of work is being done by the Runner. The step IDs are derived from their Runner type
const ( RootRunner RunnerType = iota PhaseRunner ValidatorRunner ModifyRunner SetupRunner HookRunner SubTestRunner StepRunner TearDownRunner MonitorRunner )
type Step ¶
type Step struct {
Doc string
Name string
Level int
// Whether the step should always print logs
// Even if false, logs will be done if SKUPPER_TEST_FRAME2_VERBOSE
Verbose bool
PreValidator Validator
// TODO make this require a pointer, like validator does, to avoid
// Runner disconnects
Modify Executor
Validator Validator
Validators []Validator
ValidatorRetry RetryOptions
ValidatorFinal bool // final validators are re-run at the test's end
ValidatorSubFinal bool // and subfinal, at the end of subtests
Substep Stepper
Substeps []*Step
SubstepRetry RetryOptions
// A simple way to invert the meaning of the Validator. Validators
// are encouraged to provide more specific negative testing behaviors,
// but this serves for simpler testing. If set, it inverts the
// response from the call sent to Retry, so it can be used to wait
// until an error is returned (but there is no control on which kind
// of error that will be)
// TODO: change this by OnError: Fail, Skip, Ignore, Expect?
// TODO: add option to inspect error message for specific error; perhaps a function (error) bool,
// function(error)error or function(error)frame2.action (where action is fail, skip, etc)
ExpectError bool
// TODO: ExpectIs, ExpectAs; use errors.Is, errors.As against a list of expected errors?
SkipWhen bool
}
func (Step) GetValidators ¶
Returns a list where Step.Validator is the first item, followed by Step.Validators
func (*Step) IterFrames ¶
func (s *Step) IterFrames(transform TransformFunc) error
IterFrames will run transform() on each of its configured frames (Modify, Validator and Validators[]) and reassign the frame to the return of transform().
It allows disruptors to inspect Executors and Validators in a single go.
type TearDowner ¶
type TearDowner interface {
Teardown() Executor
}
type TransformFunc ¶
type ValidationResultHook ¶
type ValidationResultHook interface {
ValidationResultHook(runner *Run, step Step, err error) error
}
This hook looks into the validations as all retries have been exhausted, and it allows a disruptor to inspect the error, or transform it into another error, or even clear the error, by returning null.
The interface needs some work. Only err is really required; runner and step may be removed or replaced in the future
type Validator ¶
type Validator interface {
Validate() error
FrameLogger
}
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
frames
|
|
|
f2skupper1
Frames (Executor and Validator) for Skupper 1.x
|
Frames (Executor and Validator) for Skupper 1.x |
|
f2skupper1/tester
The files in this package contains two types of strucs:
|
The files in this package contains two types of strucs: |
|
test_unit
This package contains simple but full tests encapsulated into callable frame2 executors.
|
This package contains simple but full tests encapsulated into callable frame2 executors. |