Documentation
¶
Index ¶
Constants ¶
const ( TreeHurl = "hurl" TreeHurlfmt = "hurlfmt" )
Tree names: the first segment of every manifest key, and the directory under the conformance root that holds the vendored upstream tree.
Variables ¶
This section is empty.
Functions ¶
func WriteManifest ¶
WriteManifest writes m to path as YAML under the standard header, sorted by script path for a stable, diffable file.
Types ¶
type Expect ¶
type Expect string
Expect is the semantic outcome the manifest records for a script: what TestConformance should see the next time it runs the suite.
const ( // ExpectPass means the script must semantically pass (exit code and // stdout oracle, if any, both match); a regression here fails the // blocking-lane gate. ExpectPass Expect = "pass" // ExpectFail means the script is known to fail right now; it needs a // Reason and never gates the build, but a script that starts passing // is reported as newly passing so it can be promoted. ExpectFail Expect = "fail" // ExpectSkip means the script is expected to be skipped this run (an // unmet prerequisite, a disabled lane, or missing tooling); it needs a // Reason for the same reasons as ExpectFail. ExpectSkip Expect = "skip" )
type Lane ¶
type Lane string
Lane groups scripts that need the same test environment and the same gating policy.
const ( // LaneBlocking scripts need only the plain HTTP server on :8000. This // is the lane whose pass rate matters for CI gating (Phase 5). LaneBlocking Lane = "blocking" // LaneExtended scripts additionally need TLS servers, the Unix-socket // server, an IPv6 listener, or the local proxy. LaneExtended Lane = "extended" // LaneNetwork scripts talk to hosts outside localhost/127.0.0.1/::1; // skipped unless SONDE_CONFORMANCE_NETWORK=1. LaneNetwork Lane = "network" // LaneTiming scripts are known to be timing-sensitive (sleeps, // retries, rate limits) and flaky under load; run, but reported // separately from LaneBlocking. LaneTiming Lane = "timing" // LaneUnsupported scripts need tooling the harness does not provide. // Not run. LaneUnsupported Lane = "unsupported" // LaneHurlfmt holds the formatter tree: its scripts plus one synthetic // entry per export fixture comparison (lint/json/html). Gating. LaneHurlfmt Lane = "hurlfmt" // LanePTY scripts need a terminal on stdout and stderr; they run // through the upstream two-pty runner (integration/term.py). Gating. LanePTY Lane = "pty" )
type Manifest ¶
type Manifest map[string]ManifestEntry
Manifest maps a script key (the tree name, then the path inside that tree, forward-slash separated, e.g. "hurl/tests_ok/hello/hello.sh"; see Script.Path) to its recorded expectation. It is committed at internal/conformance/manifest.yaml (manifest-next.yaml for the pinned upstream snapshot) — never inside a vendored tree.
func LoadManifest ¶
LoadManifest reads and validates the manifest at path. A missing file is treated as an empty manifest (every script implicitly unclassified) rather than an error, so a fresh checkout can bootstrap one with `make conformance-update`.
type ManifestEntry ¶
type ManifestEntry struct {
Lane Lane `yaml:"lane"`
Expect Expect `yaml:"expect"`
// Reason explains why the script is not expected to pass (required for
// ExpectFail and ExpectSkip; e.g. "reports: Phase 5 in progress",
// "digest unsupported", "network"). Left empty for ExpectPass.
Reason string `yaml:"reason,omitempty"`
}
ManifestEntry is the recorded expectation for one script.
type Script ¶
type Script struct {
// Path is the manifest key: the tree name, then the path inside that
// tree, forward-slash separated, e.g.
// "hurl/tests_ok/add_header/add_header.sh". An export fixture
// comparison appends "#lint", "#json" or "#html" to its .hurl path.
// DiscoverScripts alone (one tree) leaves the tree prefix off.
Path string
// Tree is TreeHurl or TreeHurlfmt; Rel is the script (or, for an export
// comparison, the .hurl input) relative to that tree's root.
Tree string
Rel string
// Export is the hurlfmt --out format an export fixture comparison
// checks ("hurl", "json" or "html"); empty for a script.
Export string
Lane Lane
// Reason explains the lane, mainly useful for LaneUnsupported and
// LaneNetwork.
Reason string
// NeedsProxy is set when the script's classification depends on the
// local squid proxy (port 3128); the runner skips it at run time if
// squid could not be started, without changing its static lane.
NeedsProxy bool
}
Script is one discovered, classified conformance test entry point.
func Discover ¶ added in v1.5.0
Discover finds every entry point under a conformance root (the directory holding the hurl/ and hurlfmt/ trees) and keys it by tree.
func DiscoverHurlfmt ¶ added in v1.5.0
DiscoverHurlfmt lists the formatter tree's entry points, keyed with the "hurlfmt/" prefix: the tests_ok (recursive) and tests_failed scripts, and one comparison per tests_export fixture and format whose expected file exists, as integration/hurlfmt/integration.py does upstream.
func DiscoverScripts ¶
DiscoverScripts walks the vendored directories under root (the hurl tree) and classifies every *.sh entry point it finds. Paths are relative to root, without the tree prefix Discover adds.
type ScriptResult ¶
type ScriptResult struct {
Path string `json:"path"`
Lane Lane `json:"lane"`
Skipped bool `json:"skipped"`
SkipReason string `json:"skip_reason,omitempty"`
ExpectedExit int `json:"expected_exit"`
ActualExit int `json:"actual_exit"`
SemanticPass bool `json:"semantic_pass"`
FullPass bool `json:"full_pass"`
MismatchInfo string `json:"mismatch,omitempty"`
Duration time.Duration `json:"duration_ns"`
}
ScriptResult is one script's outcome, serialized into results.json.