Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func WriteManifest ¶
WriteManifest writes m to path as YAML, sorted by script path for a stable, diffable file.
Types ¶
type Expect ¶
type Expect string
Expect is the semantic outcome the manifest records for a script: what TestConformance should see the next time it runs the suite.
const ( // ExpectPass means the script must semantically pass (exit code and // stdout oracle, if any, both match); a regression here fails the // blocking-lane gate. ExpectPass Expect = "pass" // ExpectFail means the script is known to fail right now; it needs a // Reason and never gates the build, but a script that starts passing // is reported as newly passing so it can be promoted. ExpectFail Expect = "fail" // ExpectSkip means the script is expected to be skipped this run (an // unmet prerequisite, a disabled lane, or missing tooling); it needs a // Reason for the same reasons as ExpectFail. ExpectSkip Expect = "skip" )
type Lane ¶
type Lane string
Lane groups scripts that need the same test environment and the same gating policy.
const ( // LaneBlocking scripts need only the plain HTTP server on :8000. This // is the lane whose pass rate matters for CI gating (Phase 5). LaneBlocking Lane = "blocking" // LaneExtended scripts additionally need TLS servers, the Unix-socket // server, an IPv6 listener, or the local proxy. LaneExtended Lane = "extended" // LaneNetwork scripts talk to hosts outside localhost/127.0.0.1/::1; // skipped unless SONDE_CONFORMANCE_NETWORK=1. LaneNetwork Lane = "network" // LaneTiming scripts are known to be timing-sensitive (sleeps, // retries, rate limits) and flaky under load; run, but reported // separately from LaneBlocking. LaneTiming Lane = "timing" // LaneUnsupported scripts need tooling the harness does not provide // (e.g. the upstream formatter binary). Not run. LaneUnsupported Lane = "unsupported" )
type Manifest ¶
type Manifest map[string]ManifestEntry
Manifest maps a script path (relative to testdata/conformance/hurl, forward-slash separated, e.g. "tests_ok/hello/hello.sh") to its recorded expectation. It is committed at internal/conformance/manifest.yaml — never inside the vendored testdata/conformance/hurl tree.
func LoadManifest ¶
LoadManifest reads and validates the manifest at path. A missing file is treated as an empty manifest (every script implicitly unclassified) rather than an error, so a fresh checkout can bootstrap one with `make conformance-update`.
type ManifestEntry ¶
type ManifestEntry struct {
Lane Lane `yaml:"lane"`
Expect Expect `yaml:"expect"`
// Reason explains why the script is not expected to pass (required for
// ExpectFail and ExpectSkip; e.g. "reports: Phase 5 in progress",
// "digest unsupported", "network"). Left empty for ExpectPass.
Reason string `yaml:"reason,omitempty"`
}
ManifestEntry is the recorded expectation for one script.
type Script ¶
type Script struct {
// Path is relative to the vendored hurl root (testdata/conformance/hurl),
// forward-slash separated, e.g. "tests_ok/add_header/add_header.sh".
Path string
Lane Lane
// Reason explains the lane, mainly useful for LaneUnsupported and
// LaneNetwork.
Reason string
// NeedsProxy is set when the script's classification depends on the
// local squid proxy (port 3128); the runner skips it at run time if
// squid could not be started, without changing its static lane.
NeedsProxy bool
}
Script is one discovered, classified conformance test entry point.
func DiscoverScripts ¶
DiscoverScripts walks the vendored directories under root and classifies every *.sh entry point it finds.
type ScriptResult ¶
type ScriptResult struct {
Path string `json:"path"`
Lane Lane `json:"lane"`
Skipped bool `json:"skipped"`
SkipReason string `json:"skip_reason,omitempty"`
ExpectedExit int `json:"expected_exit"`
ActualExit int `json:"actual_exit"`
SemanticPass bool `json:"semantic_pass"`
FullPass bool `json:"full_pass"`
MismatchInfo string `json:"mismatch,omitempty"`
Duration time.Duration `json:"duration_ns"`
}
ScriptResult is one script's outcome, serialized into results.json.