Documentation
¶
Index ¶
- Constants
- func UtilsCalcPercStr(num, den int64) string
- type OPCTPluginSummary
- func (ps *OPCTPluginSummary) GetErrorCounters() *archive.ErrorCounter
- func (ps *OPCTPluginSummary) GetFailuresByFilterID(filterID string) ([]string, []string)
- func (ps *OPCTPluginSummary) GetPreviousFailuresByFilterID(filterID string) []string
- func (ps *OPCTPluginSummary) SetFailuresByFilterID(filterID string, failures []string, excluded []string)
- type PluginDefinition
- type SortedDict
- type SortedList
- type TestDocumentation
- type TestDocumentationItem
- type TestItem
- type TestTags
- type Tests
Constants ¶
const ( PluginNameOpenShiftUpgrade = "05-openshift-cluster-upgrade" PluginNameKubernetesConformance = "10-openshift-kube-conformance" PluginNameOpenShiftConformance = "20-openshift-conformance-validated" PluginNameConformanceReplay = "80-openshift-tests-replay" PluginNameArtifactsCollector = "99-openshift-artifacts-collector" // Old Plugin names (prior v0.2). It's used to keep compatibility PluginOldNameKubernetesConformance = "openshift-kube-conformance" PluginOldNameOpenShiftConformance = "openshift-conformance-validated" )
const ( // FilterNameSuiteOnly is the filter to remove failures of tests not included in the suite. FilterNameSuiteOnly = "suite-only" // FilterNameKF is the filter to exclude known failures from the OPCT CI. FilterNameKF = "known-failures" // FilterNameBaseline is the filter to exclude failures from the baseline archive (CLI arg). FilterNameBaseline = "baseline" // FilterNameFlaky is the filter to exclude flaky tests from the report based in Sippy API. FilterNameFlaky = "flaky" // FilterNameReplay is the filter to exclude failures which are passing the replay step. FilterNameReplay = "replay" // FilterNameFinalCopy is the last step in the filter pipeline to copy the final list of failures // to be used to compose the final report/data. FilterNameFinalCopy = "copy" )
Variables ¶
This section is empty.
Functions ¶
func UtilsCalcPercStr ¶
calcPercStr receives the numerator and denominator and return the numerator and percentage as string.
Types ¶
type OPCTPluginSummary ¶
type OPCTPluginSummary struct {
Name string
NameAlias string
Status string
Total int64
Passed int64
Failed int64
Timeout int64
Skipped int64
// DocumentationReference
Documentation *TestDocumentation
// Definition
Definition *PluginDefinition
// ErrorCounters is the map with details for each failure by regex expression.
ErrorCounters archive.ErrorCounter `json:"errorCounters,omitempty"`
// FailedItems is the map with details for each failure
Tests Tests
// FailedList is the list of tests failures on the original execution
FailedList []string
// FailedFiltered is the list of failures **after** filter(s) pipeline.
// Those tests must raise attention and alerts.
FailedFiltered []string
// Filter SuiteOnly:
// FailedFilter1 is the list of failures (A) included only in the original suite (B): A INTERSECTION B
// FailedFilterSuite []string
FailedFilter1 []string
FailedExcludedFilter1 []string
// Filter Baseline:
// FailedFilter2 is the list of failures (A) excluding the baseline(B): A EXCLUDE B
// FailedFilterBaseline []string
FailedFilter2 []string
FailedExcludedFilter2 []string
// Filter FlakeAPI:
// FailedFilter3 is the priority list of failures - not reporting as flake in OpenShift CI.
// FailedFilterPrio []string
FailedFilter3 []string
FailedExcludedFilter3 []string
// Filter BaselineAPI:
// FailedFilter4 is the list after excluding known failures from OPCT CI.
// This filter is similar BaseLine, but it's a list of failures collected from
// processed data (another OPCT execution) on OPCT CI after processed by OPCT report,
// exposed thorugh the OPCT API. This list is used to exclude known failures,
// to prevent false positives on the review pipeline.
// TODO(mtulio): deprecate Filter2 when Filter4 is accurated. Baseline results should
// not use Filter2.
FailedFilter4 []string
FailedExcludedFilter4 []string
// Filter KnownFailures:
// FailedFilter5 is the list of failures that are explicitly removed from pipeline.
// It should not be used to exclude failures from the report of e2e included in suite,
// but to remove known flake/failures that is not relevant to the pipeline.
// Example: '[sig-arch] External binary usage'
// Filter5KnownFailures []string
FailedFilter5 []string
FailedExcludedFilter5 []string
// Filter Replay:
// FailedFilter6 is the list of failures which also failed in the second shot: replay plugin/step.
FailedFilter6 []string
FailedExcludedFilter6 []string
}
OPCTPluginSummary handle plugin details
func (*OPCTPluginSummary) GetErrorCounters ¶
func (ps *OPCTPluginSummary) GetErrorCounters() *archive.ErrorCounter
func (*OPCTPluginSummary) GetFailuresByFilterID ¶
func (ps *OPCTPluginSummary) GetFailuresByFilterID(filterID string) ([]string, []string)
GetFailuresByFilterID returns the list of failures handlers by filter ID.
func (*OPCTPluginSummary) GetPreviousFailuresByFilterID ¶
func (ps *OPCTPluginSummary) GetPreviousFailuresByFilterID(filterID string) []string
GetPreviousFailuresByFilterID returns the list of failures from the previous plugin in the pipeline, by providing the current filter ID. TODO: move the filter logic to a dedicated structure using linked stack/list, allowing each plugin having a dynamic list of filters, instead of forcing the same pipeline across all plugins.
func (*OPCTPluginSummary) SetFailuresByFilterID ¶
func (ps *OPCTPluginSummary) SetFailuresByFilterID(filterID string, failures []string, excluded []string)
SetFailuresByFilterID stes the list of failures handlers by filter ID.
type PluginDefinition ¶
type SortedDict ¶
SortedDict stores and sorts the key/value map to be ranked by value.
type SortedList ¶
type SortedList []SortedDict
SortedList stores the list of key/value map, implementing interfaces to sort/rank a map strings with integers as values.
func (SortedList) Len ¶
func (p SortedList) Len() int
func (SortedList) Less ¶
func (p SortedList) Less(i, j int) bool
func (SortedList) Swap ¶
func (p SortedList) Swap(i, j int)
type TestDocumentation ¶
type TestDocumentation struct {
// UserBaseURL is a the User Facing base URL for the documentation.
UserBaseURL *string
// SourceBaseURL is the raw URL to be indexed.
SourceBaseURL *string
// Raw stores the data extracted from SourceBaseURL.
Raw *string
// Tests is the map indexed by test name, with URL fragment (page references) as a value.
// Example: for the e2e test '[sig-machinery] run instance', the following map will be created:
// map['[sig-machinery] run instance']='#sig-machinery--run-instance'
Tests map[string]*TestDocumentationItem
}
TestDocumentation is the struct that holds the test documentation. The struct is used to store the documentation URL, the raw data, and the tests indexed by name. The test documentation is discovered by name, and the URL fragment is used to mount the URL for the test documentation.
func NewTestDocumentation ¶
func NewTestDocumentation(user, source string) *TestDocumentation
func (*TestDocumentation) BuildIndex ¶
func (d *TestDocumentation) BuildIndex() error
BuildIndex reads the raw Document, discoverying the test name, and the URL fragments. The parser is based in the Kubernetes Conformance documentation: https://github.com/cncf/k8s-conformance/blob/master/docs/KubeConformance-1.27.md
func (*TestDocumentation) Load ¶
func (d *TestDocumentation) Load() error
Load documentation from Suite and save it to further query
type TestDocumentationItem ¶
type TestDocumentationItem struct {
Title string
Name string
// URLFragment stores the discovered fragment parsed by the Documentation page,
// indexed by test name, used to mount the Documentation URL for failed tests.
URLFragment string
}
TestDocumentationItem refers to items documented by
type TestItem ¶
type TestItem struct {
// Name is the name of the e2e test. It is hidden from JSON as Tests is a map, and
// the key can be used.
Name string `json:"-"`
// ID is the unique identifier of the test within the execution.
ID string `json:"id"`
// Status store the test result. Valid values: passed, skipped, failed.
Status string `json:"status"`
// State represents the state of the test. It can be any status value or filter name.
State string `json:"state,omitempty"`
// Failure contains the failure reason extracted from JUnit field 'item.detials.failure'.
Failure string `json:"-"`
// SystemOut contains the entire test stdout extracted from JUnit field 'item.detials.system-out'.
SystemOut string `json:"-"`
// Offset is the offset of failure from the plugin result file.
Offset int `json:"-"`
// Flaky contains the flake information from OpenShift CI - scraped from Sippy API.
Flake *sippy.SippyTestsResponse `json:"flake,omitempty"`
// ErrorCounters errors indexed by common error key.
ErrorCounters archive.ErrorCounter `json:"errorCounters,omitempty"`
// Reference for documentation.
Documentation string `json:"documentation"`
}
TestItem represents a single test unit holding attributes for the processor pipeline.
func (*TestItem) LookupDocumentation ¶
func (pi *TestItem) LookupDocumentation(d *TestDocumentation)
LookupDocumentation extracts from the test name the expected part (removing '[Conformance]') to link to the Documentation URL refereced by the Kubernetes Conformance markdown available at https://github.com/cncf/k8s-conformance/blob/master/docs/KubeConformance-<version>.md . The test documentation (TestDocumentation) should be indexed prior calling the LookupDocumentation.
func (*TestItem) UpdateErrorCounter ¶
func (pi *TestItem) UpdateErrorCounter()
UpdateErrorCounter reads the failures and stdout looking for error patterns from a specific test, accumulating the ErrorCounters structure.
type TestTags ¶
TestTags stores the test tags map with it's counter. The test tag is the work extracted from the first bracket from a test name. Example test name: '[sig-provider] test name' the 'sig-provider' is the tag.
func NewTestTags ¶
NewTestTags creates the TestTags populating the tag values and counters.
func NewTestTagsEmpty ¶
NewTestTagsEmpty creates the TestTags with a specific size, to be populated later.
func (TestTags) ShowSorted ¶
ShowSorted return an string with the rank of tags.