Documentation
¶
Overview ¶
Package conformance is the public conformance test suite for IaC providers. Each provider plugin (DO, AWS, GCP, Azure, …) imports this package from its own *_test.go and calls conformance.Run(t, Config) to exercise the spec-mandated scenarios that every provider MUST satisfy.
The package is import-only and pure-test: it does not provision real resources unless the caller passes Config{LiveCloud: true} and the scenario itself is RequiresCloud=true. This split lets non-cloud scenarios run on every PR while cloud-touching ones gate on a smoke workflow.
Scaffold per W-7 T7.1; individual scenarios are added in T7.2-T7.12.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Run ¶
Run is the public entry point provider plugins call from a *_test.go. It iterates the registered scenarios, applies the SmokeOnly / LiveCloud / SkipScenarios filters, and invokes each via t.Run so individual scenarios show up as discrete subtests in CI output.
Plan-spec deviation note: §T7.1 sketched the skip path as `t.Skipf(...); continue` on the outer t. That snippet would Goexit the dispatcher entirely on the first skip-match. The intended behavior — per-scenario skipped subtests — requires wrapping the Skipf in t.Run, which is what we do.
Diff-cache isolation: scenarios that exercise platform.ComputePlan with cacheable resources (non-empty ProviderID) call platform.SetDiffCacheForTest to install a fresh no-op cache for the scenario subtest. SetDiffCacheForTest Stores into platform's atomic.Pointer directly, so it works regardless of whether sync.Once has already sealed the cache backend earlier in the test process — every Run gets independent cache state. The earlier best-effort t.Setenv("WFCTL_DIFFCACHE", "disabled") workaround is retired in favour of this architecturally-correct helper.
Types ¶
type Config ¶
type Config struct {
// Provider returns a fresh interfaces.IaCProvider for each scenario.
// Required.
Provider func() interfaces.IaCProvider
// SkipScenarios maps scenario name → reason. The dispatcher emits a
// t.Skipf-flagged subtest for each entry instead of executing the
// body, preserving CI visibility into intentional skips.
SkipScenarios map[string]string
// SmokeOnly limits the run to scenarios with Smoke=true. Used by the
// per-PR smoke gate.
SmokeOnly bool
// LiveCloud opts in to scenarios that provision real cloud resources
// (RequiresCloud=true). Default false keeps test-suites hermetic.
LiveCloud bool
}
Config drives a conformance Run.
type Scenario ¶
type Scenario struct {
// Name is the t.Run subtest name and the SkipScenarios map key.
// Convention: "Scenario_<CamelCaseDescription>".
Name string
// Smoke marks scenarios that the per-PR smoke gate runs against an
// active provider's real cloud. Non-smoke scenarios run only when a
// caller opts in (e.g. nightly full-suite).
Smoke bool
// RequiresCloud marks scenarios that touch real cloud APIs. The
// dispatcher skips these unless Config.LiveCloud is true.
RequiresCloud bool
// Run is the scenario body. It receives the live *testing.T (a
// subtest) and the caller's Config so it can obtain a fresh provider
// via cfg.Provider().
Run func(t *testing.T, cfg Config)
}
Scenario is a single conformance test case. Each scenario lives in its own file (scenario_<name>.go) and registers itself via register() in an init() block.
Source Files
¶
- scenario_cross_resource_constraint.go
- scenario_delete_action.go
- scenario_grpc_roundtrip.go
- scenario_infra_output_cross_module.go
- scenario_needs_replace.go
- scenario_outputs_consistency.go
- scenario_outputs_refresh.go
- scenario_plan_stale.go
- scenario_protected_replace_with_override.go
- scenario_protected_replace_without_override.go
- scenario_replace_cascade_preserves_dependents.go
- scenario_upsert_on_already_exists.go
- scenarios.go