conformance

package
v0.21.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 5, 2026 License: MIT Imports: 14 Imported by: 0

Documentation

Overview

Package conformance is the public conformance test suite for IaC providers. Each provider plugin (DO, AWS, GCP, Azure, …) imports this package from its own *_test.go and calls conformance.Run(t, Config) to exercise the spec-mandated scenarios that every provider MUST satisfy.

The package is import-only and pure-test: it does not provision real resources unless the caller passes Config{LiveCloud: true} and the scenario itself is RequiresCloud=true. This split lets non-cloud scenarios run on every PR while cloud-touching ones gate on a smoke workflow.

Scaffold per W-7 T7.1; individual scenarios are added in T7.2-T7.12.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Run

func Run(t *testing.T, cfg Config)

Run is the public entry point provider plugins call from a *_test.go. It iterates the registered scenarios, applies the SmokeOnly / LiveCloud / SkipScenarios filters, and invokes each via t.Run so individual scenarios show up as discrete subtests in CI output.

Plan-spec deviation note: §T7.1 sketched the skip path as `t.Skipf(...); continue` on the outer t. That snippet would Goexit the dispatcher entirely on the first skip-match. The intended behavior — per-scenario skipped subtests — requires wrapping the Skipf in t.Run, which is what we do.

Diff-cache isolation: scenarios that exercise platform.ComputePlan with cacheable resources (non-empty ProviderID) call platform.SetDiffCacheForTest to install a fresh no-op cache for the scenario subtest. SetDiffCacheForTest Stores into platform's atomic.Pointer directly, so it works regardless of whether sync.Once has already sealed the cache backend earlier in the test process — every Run gets independent cache state. The earlier best-effort t.Setenv("WFCTL_DIFFCACHE", "disabled") workaround is retired in favour of this architecturally-correct helper.

Types

type Config

type Config struct {
	// Provider returns a fresh interfaces.IaCProvider for each scenario.
	// Required.
	Provider func() interfaces.IaCProvider

	// SkipScenarios maps scenario name → reason. The dispatcher emits a
	// t.Skipf-flagged subtest for each entry instead of executing the
	// body, preserving CI visibility into intentional skips.
	SkipScenarios map[string]string

	// SmokeOnly limits the run to scenarios with Smoke=true. Used by the
	// per-PR smoke gate.
	SmokeOnly bool

	// LiveCloud opts in to scenarios that provision real cloud resources
	// (RequiresCloud=true). Default false keeps test-suites hermetic.
	LiveCloud bool
}

Config drives a conformance Run.

type Scenario

type Scenario struct {
	// Name is the t.Run subtest name and the SkipScenarios map key.
	// Convention: "Scenario_<CamelCaseDescription>".
	Name string

	// Smoke marks scenarios that the per-PR smoke gate runs against an
	// active provider's real cloud. Non-smoke scenarios run only when a
	// caller opts in (e.g. nightly full-suite).
	Smoke bool

	// RequiresCloud marks scenarios that touch real cloud APIs. The
	// dispatcher skips these unless Config.LiveCloud is true.
	RequiresCloud bool

	// Run is the scenario body. It receives the live *testing.T (a
	// subtest) and the caller's Config so it can obtain a fresh provider
	// via cfg.Provider().
	Run func(t *testing.T, cfg Config)
}

Scenario is a single conformance test case. Each scenario lives in its own file (scenario_<name>.go) and registers itself via register() in an init() block.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL