e2e

package
v0.5.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 14, 2026 License: Apache-2.0 Imports: 45 Imported by: 0

README

E2E testing

$ source .ate-dev-env.sh
$ go test -v ./internal/e2e/suites/... -args --e2e

Principles

  • Keep it simple -- use go test for the harness.
  • e2e tests live under internal/e2e/suites/<suite>
  • Each suite should implement TestMain using e2e.RunTestMain()
    • e2e tests will be skipped for ordinary unit tests unless the --e2e flag is set e.g. go test ./internal/e2e/suites/... -args --e2e
  • Helper libraries live under internal/e2e
  • Setup and Teardown are on a per-component basis and the component's author's responsibility.

Preconditions

The e2e tests assume you have a cluster set up with Agent Substrate installed, for example via hack/install-ate.sh --deploy-ate-system --credential-provider='{"name":"k8s.io"}' or hack/install-ate-kind.sh --deploy-ate-system --credential-provider='{"name":"k8s.io"}'. The egress credential injection suite needs that bundled provider, which is what CI installs on the envoy lane; see internal/e2e/suites/egresscredinject.

Sandbox classes

The suites are runtime-agnostic: the same tests run against gVisor and against the micro-VM (kata + cloud-hypervisor) sandbox class. E2E_SANDBOX_CLASS selects which, by repointing every fixture at its variant --- see e2e.CounterFixture, e2e.EgressFixture and e2e.RenderFixtureManifest in sandbox.go. Unset means gVisor.

# gVisor (the default), against the demos install-ate-kind.sh deploys
$ hack/run-e2e-kind.sh -v -args --no-color

# micro-VM, against the counter-microvm and egress-microvm demos
$ E2E_SANDBOX_CLASS=microvm hack/run-e2e-kind.sh -v -args --no-color

The micro-VM lane needs its fixtures installed first, which also needs a node with /dev/kvm (hack/create-kind-cluster.sh detects one and labels the node):

$ hack/run-microvm-demo-kind.sh                        # counter-microvm + assets
$ hack/install-ate-kind.sh --deploy-demo-egress-microvm # egress-microvm

A handful of knobs override the class defaults, mostly for a cluster that installs the fixtures elsewhere: E2E_SUBSTRATE_TEMPLATE_ATESPACE / E2E_SUBSTRATE_TEMPLATE_NAME / E2E_SUBSTRATE_POOL_NAMESPACE / E2E_SUBSTRATE_POOL_NAME point the counter fixture somewhere else, and E2E_TEMPLATE_READY_TIMEOUT replaces the golden-snapshot budget (90s on gVisor, 10m on micro-VM, where the golden is a cloud-hypervisor cold boot plus a checkpoint).

Settings

The installer reads this setting in addition to those in docs/operator-install.md, through the same flag, environment variable, and configuration document channels. It sets the worker thread count (--concurrency) of the Envoy container in atenet-egress; 0 leaves the manifest unchanged. The nighthawk egress benchmark sets it to 1 to measure single-CPU Envoy performance.

Key Environment variable Flag Default Applies to
e2e.envoyConcurrency E2E_ATE_ENVOY_CONCURRENCY --e2e-envoy-concurrency 0 (global)

After a failure

A suite deletes the namespaces it created only when it passed. A failed run keeps them, because the failure is usually explained inside a worker pod (the ateom logs, and for a micro-VM worker the guest's console tail), and deleting the namespace takes those pods with it:

$ kubectl logs -n <kept-namespace> <worker-pod>

Nothing reclaims them afterwards, and each namespace holds a WorkerPool's worth of running pods, so clean up once you are done reading:

$ hack/cleanup-e2e.sh   # deletes every namespace labeled ate.dev/e2e

Creating a new test suite

Copy testmain_test.go from internal/e2e/suites/example into your new suite. It will look like this:

func run(m *testing.M) int {
	Setup()
	defer Teardown()
	// return allows the deferred Teardown to run.
	return e2e.RunTestMain(m)
}

func TestMain(m *testing.M) { os.Exit(run(m)) }

This will handle the standard flags and checks for running an e2e test suite.

Documentation

Index

Constants

View Source 
const (
	// CredentialSecretsNamespace holds the fixture Secret and is the only
	// namespace the credinject fixture's authorization policy allows the probe
	// atespaces to resolve.
	CredentialSecretsNamespace = "ate-e2e-credinject-secrets"

	// CredentialInjectionURI resolves to CredentialInjectionToken through the
	// k8s-credential-provider, for the atespaces the fixture policy allows.
	CredentialInjectionURI = "ate-secret://k8s.io/default/" + CredentialSecretsNamespace + "/api-token/token"

	// CredentialInjectionToken is the fixture Secret's value, what an injected
	// header carries after any prefix.
	CredentialInjectionToken = "e2e-cred-inject-token"
)
View Source 
const AtenetDataplaneEnv = "E2E_ATENET_DATAPLANE"

AtenetDataplaneEnv selects the dataplane exercised by an e2e lane.

View Source 
const (
	// EgressTrustBundleObjectName is the reconciler-owned ClusterTrustBundle.
	EgressTrustBundleObjectName = "egress-mitm.ate.dev:mitm:primary-bundle"
)

Constants of atecontroller's EgressMITMTrustReconciler (#946): the CA pool Secret it watches (the key is what `kubectl-ate admin make-ca-pool` writes) and the ClusterTrustBundle it derives from that pool — the backing object of the allowlisted "egress-mitm.ate.dev" bundle the probe fixture projects.

Suites provision the POOL and let the real reconciler publish the bundle, exercising the whole chain (pool -> reconciler -> bundle -> projection). Writing the bundle directly is not an option: the reconciler watches it and reverts or deletes hand-written contents.

View Source 
const LifecycleEventMetric = "substrate_actor_state_changes"

LifecycleEventMetric is the count connector's view of the actor lifecycle events. Reading it rather than a log store keeps this check on the metrics harness: kind has no place to query log records.

View Source 
const NamespaceLabel = "ate.dev/e2e"

NamespaceLabel marks the namespaces the suites create, so leftovers from a failed run (which are kept deliberately — see RetainNamespaces) can be found and deleted later: `kubectl delete ns -l ate.dev/e2e`, or hack/cleanup-e2e.sh.

View Source 
const ProbeName = "probe"

ProbeName is the name of the probe fixture's WorkerPool and ActorTemplate, inside the atespace (and matching k8s namespace) DeployProbe returns.

View Source 
const ResourcePrefixEnv = "E2E_RESOURCE_PREFIX"

ResourcePrefixEnv is the prefix the install under test puts on substrate's resource names. A packaging layer that renames resources prefixes them, and the harness addresses several of those resources by name.

View Source 
const SandboxClassMicroVM = "microvm"

SandboxClassMicroVM is the kata + cloud-hypervisor runtime, spelled as the WorkerPool spec.sandboxClasses[].name and ActorTemplate sandboxClass spell it.

View Source 
const SystemNamespaceEnv = "E2E_SYSTEM_NAMESPACE"

SystemNamespaceEnv names the namespace the substrate control plane under test was installed into. It mirrors the ATE_NAMESPACE hack/install-ate.sh installed with, and the namespace the deployment was installed into.

Variables

View Source 
var (
	RunE2E       bool
	KubeConfig   string
	KubeContext  string
	StorageClass string
)
View Source 
var (
	NoColor bool
)

noColor() disables colors for tests running in environments without colored output.

View Source 
var PlatformMetricPrefixes = []string{
	"ate_workerpool_workers",
	"ate_workerpool_desired_workers",
	"ate_workerpool_ready_workers",
	"ate_actor_crashes",
	"ate_actor_lifecycle_operation_duration",
	"ate_scheduler_assignment_duration",
	"ate_actor_restore_duration",
	"ate_actor_checkpoint_duration",
	"atelet_snapshot_size",
	"atenet_router_route_duration",
}

PlatformMetricPrefixes are the Prometheus metric-name prefixes (OTLP dots mapped to underscores) the substrate platform must emit. The Collector's prometheus exporter appends unit and type suffixes (e.g. _seconds_bucket, _bytes_count), so matching is by prefix. This slice grows as each metric slice lands and as more components are wired to push to the collector.

Functions

func AddEgressCA added in v0.4.0 

func AddEgressCA(t *testing.T, ctx context.Context, clients *Clients, id string) string

AddEgressCA adds a fresh CA, named id, to the egress CA pool, waits for the reconciler to publish the derived bundle, and returns that bundle: what a trustBundle projection must then deliver, in any order (see SameCertificates). The new CA is trusted but never signs, so every actor keeps trusting the leaves the gateway mints, and suites doing TLS through the gateway can run alongside. The pool is put back as found when the test ends.

func CheckEnv 

func CheckEnv(keys ...string) (map[string]string, error)

CheckEnv checks the list of env vars exist and returns their value. If any env var is not set, it returns an error.

func CleanupNamespaces 

func CleanupNamespaces()

CleanupNamespaces deletes all registered namespaces using the K8s API. This should be called at the end of RunTestMain.

A backstop rather than the main path: tests release their own namespaces as they finish, so this only picks up ones whose cleanup never ran.

func CollectorHasService added in v0.1.0 

func CollectorHasService(scrape string, services ...string) bool

CollectorHasService reports whether any named service has pushed telemetry to the collector. Its prometheus exporter maps each pushed resource's service.name onto the job label, so a service that has exported at least one data point shows up there. Note the exporter never emits a service_name label, and a resource whose instruments have recorded nothing yet produces no series at all.

func Colorf 

func Colorf(format string, a ...any) string

Colorf renders a formatted string with color directives:

"Example text: <green>this is green</green>".

Unterminated directives will be treated as literals.

func ConfigureCredentialProvider added in v0.4.0 

func ConfigureCredentialProvider(t *testing.T)

ConfigureCredentialProvider points the install's k8s-credential-provider at the credinject fixture: the Secret behind CredentialInjectionURI and the authorization policy that lets the probe atespaces resolve it. The provider Deployment is restarted after the policy ConfigMap is applied because it reads the policy once at startup. When the test passes the fixture is removed, the shipped policy put back, and the provider restarted again so it drops the fixture's grants.

A failed test keeps the fixture so the provider's logs can be read against it; the next run re-applies it.

The provider itself and the gateway's side of the connection — the credential-provider configuration — are the install's (hack/install-ate.sh --credential-provider='{"name":"k8s.io"}'), not something this helper can retrofit: the rollout wait below fails on a cluster installed without them.

func CreateSubstrateCounterTemplate added in v0.1.0 

func CreateSubstrateCounterTemplate(ctx context.Context, t *testing.T, clients *Clients, namespace string, opts SubstrateTemplateOptions) *ateapipb.ActorTemplate

CreateSubstrateCounterTemplate creates a per-test WorkerPool CRD plus a substrate ActorTemplate copying the resolved runtime from the substrate counter demo for the sandbox class under test.

func CreateSubstrateTemplateFrom added in v0.1.0 

func CreateSubstrateTemplateFrom(ctx context.Context, t *testing.T, clients *Clients, namespace string, src SubstrateFixture, opts SubstrateTemplateOptions) *ateapipb.ActorTemplate

CreateSubstrateTemplateFrom creates a per-test WorkerPool CRD plus a substrate ActorTemplate copying the resolved runtime (sandbox config, ateom image, container images, sandbox size) from the installed fixture src. It registers cleanup of the template (which does not ride the k8s namespace GC the CRD templates did) and blocks until the golden snapshot exists.

func DeployProbe added in v0.1.0 

func DeployProbe(t *testing.T, bucket, name string, opts ...ProbeOption) (string, *ateapipb.ActorTemplate)

DeployProbe builds the probe fixture image and installs the fixture for the sandbox class under test, removing it when the test ends. name distinguishes the caller (by convention its suite name): each suite gets its own copy of the fixture, so no suite's cleanup can delete the fixture out from under another running concurrently. It returns the fixture's atespace (which also names the k8s namespace holding the pool) and the created ActorTemplate, already golden-snapshotted.

func DeploySubstrateFixture added in v0.1.0 

func DeploySubstrateFixture(t *testing.T, ctx context.Context, clients *Clients, manifests SubstrateFixtureManifests, bucket, name string, trustBundles []string) (string, []*ateapipb.ActorTemplate)

DeploySubstrateFixture installs a fixture for the sandbox class under test: it ko-applies the pool manifest, creates the fixture's atespace and ActorTemplates through the ate API (ko-resolving the templates' ko:// image references first), and blocks until every template's golden snapshot exists. name distinguishes the caller (by convention its suite name): each suite gets its own copy of the fixture, so no suite's cleanup can delete it out from under another running concurrently.

Everything is removed when the test ends. The substrate resources need explicit cleanup — unlike the CRD templates they replaced, they do not ride the k8s namespace GC — and a template leaked by an interrupted earlier run is cleared before creating its replacement, since templates are immutable.

trustBundles names the trust bundles a template with a ${TEMPLATE_TRUST_BUNDLE} placeholder projects; nil projects none.

Returns the fixture's atespace (the same string that names the k8s namespace holding the pool) and the created templates.

func DescribePodState added in v0.1.0 

func DescribePodState(pod *corev1.Pod) string

DescribePodState summarizes why a pod is not ready yet, one clause per container, for a timeout message.

func EgressAllowAll added in v0.2.0 

func EgressAllowAll() []*ateapipb.EgressRule

EgressAllowAll is what a test that is not about egress policy gives its actor, since the gateway denies an actor with no policy at all: every name and address, as cleartext HTTP on any port and as intercepted HTTPS on 443. TLS forwarded unread is not decided by the gateway yet, and a passthrough rule on every port would tie with the http rule, which the API rejects.

func EgressAllowHTTP added in v0.3.0 

func EgressAllowHTTP(patterns ...string) *ateapipb.EgressRule

EgressAllowHTTP is a rule that lets an actor send cleartext HTTP to the hosts matching patterns (exact names, or "*." plus a name for one leftmost label) on port 80.

func EgressAllowHTTPS added in v0.3.0 

func EgressAllowHTTPS(patterns ...string) *ateapipb.EgressRule

EgressAllowHTTPS is a rule that lets an actor send HTTPS, intercepted by the gateway, to the hosts matching patterns on port 443.

func EgressAllowPassthrough added in v0.4.0 

func EgressAllowPassthrough(patterns ...string) *ateapipb.EgressRule

EgressAllowPassthrough is a rule that lets an actor send TLS to the hosts matching patterns on any port, relayed by the gateway without decryption.

func EgressAllowPassthroughOnPorts added in v0.4.0 

func EgressAllowPassthroughOnPorts(ports []int32, patterns ...string) *ateapipb.EgressRule

EgressAllowPassthroughOnPorts is EgressAllowPassthrough confined to the given ports.

func EgressInjectHeader added in v0.3.0 

func EgressInjectHeader(header, prefix, credentialURI string, patterns ...string) *ateapipb.EgressRule

EgressInjectHeader is an https rule (see EgressAllowHTTPS) that also carries a replace_headers effect: on a match, the gateway resolves credentialURI through its credential provider and replaces header with prefix plus the credential.

func EgressInjectHeaderHTTP added in v0.3.0 

func EgressInjectHeaderHTTP(header, prefix, credentialURI string, patterns ...string) *ateapipb.EgressRule

EgressInjectHeaderHTTP is an http rule (see EgressAllowHTTP) carrying the same credential replacement effect as EgressInjectHeader.

func EnsureEgressPolicy added in v0.2.0 

func EnsureEgressPolicy(t *testing.T, ctx context.Context, clients *Clients, actor *ateapipb.ObjectRef, rules ...*ateapipb.EgressRule)

EnsureEgressPolicy gives actor an EgressPolicy with exactly rules, replacing any it had. Deleting the actor deletes the policy, so there is no cleanup.

func EnsureEgressTrustBundle added in v0.1.0 

func EnsureEgressTrustBundle(t *testing.T, ctx context.Context, clients *Clients)

EnsureEgressTrustBundle makes sure the egress trust bundle exists, then waits until the reconciler-published bundle is non-empty. It provisions a pool only when there is none and never replaces one it finds: the pool is cluster-wide, and the egress gateway mounts the one the install created. A suite that needs to know the bundle's exact contents (the identity suite's assertions and live refresh) uses AddEgressCA.

func FColorf 

func FColorf(f io.Writer, format string, a ...any) error

FColorf writes a formatted string with color directives to the given writer.

func FColorfln 

func FColorfln(f io.Writer, format string, a ...any) error

FColorfln writes a formatted string with color directives to the given writer, followed by a newline.

func FindRepoRoot 

func FindRepoRoot() (string, error)

FindRepoRoot traverses directories upward starting from the current working directory to locate the repository root containing the go.mod file.

func FixtureName added in v0.1.0 

func FixtureName(base string) string

FixtureName suffixes a fixture's name for the sandbox class under test, so the gVisor and micro-VM lanes never share one. That matters most for the namespaces a suite creates and deletes itself: the two lanes run one after the other, and a namespace still Terminating from the previous one would fail the next one's apply. ${FIXTURE_SUFFIX} does the same job inside the fixture manifests.

func IsMicroVM added in v0.1.0 

func IsMicroVM() bool

IsMicroVM reports whether the suites are pointed at the micro-VM fixtures. Assertions that only hold for one runtime gate on this.

func LifecycleEventCounts added in v0.2.0 

func LifecycleEventCounts(scrape string) map[string]float64

LifecycleEventCounts returns the count the collector holds for each ate.actor.state. The counter is cumulative and the collector outlives any one test, so compare two reads rather than asserting a state is merely present: a stale count from an earlier run would pass a presence check even with the exporter turned off.

One state can appear on several lines, one per emitting ateapi instance, so the counts are summed.

func MissingPlatformMetrics added in v0.1.0 

func MissingPlatformMetrics(scrape string, prefixes []string) []string

MissingPlatformMetrics returns the prefixes with no matching series in the Prometheus exposition text. A metric matches when its name equals a prefix or begins with prefix+"_"; the underscore boundary stops "ate_actor_restore" from matching an unrelated "ate_actor_restored".

func PreflightChecks 

func PreflightChecks() error

PreflightChecks checks that the test environment is ready for the test suite.

func RegisterSuiteCleanup added in v0.1.0 

func RegisterSuiteCleanup(fn func())

func RenderFixtureManifest added in v0.1.0 

func RenderFixtureManifest(t *testing.T, relPath, bucket, name string) string

RenderFixtureManifest renders the manifest template at relPath (repo-relative, under internal/e2e/fixtures) for the sandbox class under test, writes it into the test's temp dir and returns that path. Both an apply and a later delete can then consume the same file, with no shell involved.

name distinguishes the caller (by convention its suite name) and is appended to ${FIXTURE_SUFFIX}: suite packages run as concurrent processes, so each caller must get its own copy of a fixture or one suite's cleanup deletes it out from under another.

One template serves both sandbox classes so the two variants of a fixture cannot drift apart. See renderManifest for the placeholder kinds a template can carry.

func ResourceName added in v0.2.0 

func ResourceName(name string) string

ResourceName returns name as the install under test renders it.

func ResumeActorAwaitCapacity added in v0.1.0 

func ResumeActorAwaitCapacity(t *testing.T, ctx context.Context, clients *Clients, req *ateapipb.ResumeActorRequest) (*ateapipb.ResumeActorResponse, error)

ResumeActorAwaitCapacity resumes the actor, retrying while its worker pool is saturated. The control plane returns ResourceExhausted when no worker is free and expects callers to wait (the router's parking resumer retries the same condition); suites running concurrently against shared pools make that a normal state in e2e, not a failure. Any other error, or saturation outlasting the wait budget, is returned to the caller. Each retry is logged so a pass that had to wait stays visible in the test output.

TODO: Add runtime diagnostic output for unscheduled pods/actors. a Pending pod's PodScheduled condition names resources the scheduler cannot satisfy. Reporting this needs no knowledge of devices a sandbox class requires. Deferred because selecting the pods requires a standard label such as the ate.dev/worker-pool label, which currently exists as two unexported constants under cmd/*/internal with different names, plus literals in three binaries. This label should be unified under internal/ first, rather than adding another copy here.

func RetainNamespaces added in v0.1.0 

func RetainNamespaces()

RetainNamespaces leaves the registered namespaces in the cluster and reports them, instead of deleting them. Used when the suite failed: the namespaces' worker pods hold the ateom (and, for micro-VM workers, guest) logs a post-mortem needs, and they are deleted long before anyone can read them.

Passing tests have already released theirs, so what is left is the failures' — exactly what a post-mortem wants to look at.

func RunCmd 

func RunCmd(t *testing.T, name string, args ...string)

RunCmd executes the given command with arguments, piping stdout and stderr to standard outputs, and fails the test if the command returns an error.

func RunCmdOutput added in v0.1.0 

func RunCmdOutput(t *testing.T, env []string, name string, args ...string) []byte

RunCmdOutput executes the given command with custom environment variables appended to the current process environment, streaming stderr like RunCmd does, and returns the captured stdout. Fails the test if the command returns an error.

func RunCmdWithEnv 

func RunCmdWithEnv(t *testing.T, env []string, name string, args ...string)

RunCmdWithEnv executes the given command with custom environment variables appended to the current process environment, and fails the test if it returns an error.

func RunTestMain 

func RunTestMain(m *testing.M) int

RunTestMain should be used to run your e2e test suite.

func SameCertificates added in v0.4.0 

func SameCertificates(a, b string) bool

SameCertificates reports whether two bundles are the same PEM CERTIFICATE blocks and nothing else. Order is ignored: atelet shuffles a bundle's certificates every time it writes one.

func SandboxClass added in v0.1.0 

func SandboxClass() string

SandboxClass returns the sandbox class under test, "" for gVisor. CI sets E2E_SANDBOX_CLASS=microvm for the micro-VM lane and leaves it unset for the gVisor one, so a single knob repoints every suite's fixtures.

func ScrapeAgentGatewayRouterMetrics added in v0.2.0 

func ScrapeAgentGatewayRouterMetrics(ctx context.Context) (string, error)

ScrapeAgentGatewayRouterMetrics reads the AgentGateway router's native Prometheus stats endpoint. AgentGateway instruments are not OTLP exports.

func ScrapeCollectorMetrics added in v0.1.0 

func ScrapeCollectorMetrics(ctx context.Context) (string, error)

ScrapeCollectorMetrics port-forwards the kind stack's OTel Collector and reads its Prometheus exporter surface, returning the raw exposition text.

func ScrapeEgressEnvoyMetrics added in v0.5.0 

func ScrapeEgressEnvoyMetrics(ctx context.Context) (string, error)

Port-forward each ready atenet-egress pod's Envoy admin port and read its stats/prometheus endpoint, returning the concatenated Prometheus exposition text.

func ServiceDNSTrustBundle added in v0.5.0 

func ServiceDNSTrustBundle(t *testing.T, ctx context.Context, clients *Clients) string

ServiceDNSTrustBundle returns the service-DNS signer's roots as PEM, for a fetch that has to verify a ServiceDNSTLS server's own certificate rather than a leaf the egress gateway minted.

func StatesNotAdvanced added in v0.2.0 

func StatesNotAdvanced(before, after map[string]float64, states []string) []string

StatesNotAdvanced returns the states whose count did not rise between the two reads. An empty result means every state was emitted during the window.

func SystemNamespace added in v0.2.0 

func SystemNamespace() string

SystemNamespace returns the namespace the control plane under test runs in, falling back to the canonical install namespace.

func TargetInfoLabel added in v0.1.0 

func TargetInfoLabel(scrape, service, label string) string

TargetInfoLabel returns one resource attribute of a service, or "" when it is absent. The Prometheus exporter keeps resource attributes off the series and publishes them once per resource in target_info, keyed by job.

func TemplateReadyTimeout added in v0.1.0 

func TemplateReadyTimeout(t *testing.T) time.Duration

TemplateReadyTimeout is how long to wait for an ActorTemplate's golden snapshot. A micro-VM golden (a cloud-hypervisor cold boot plus checkpoint, on nested KVM in CI) takes several times what a gVisor one does, so the default follows the class under test. E2E_TEMPLATE_READY_TIMEOUT overrides it.

func TemplateRef added in v0.1.0 

func TemplateRef(at *ateapipb.ActorTemplate) *ateapipb.ObjectRef

TemplateRef builds the substrate template reference an Actor carries.

func WaitForPodReady added in v0.1.0 

func WaitForPodReady(t *testing.T, ctx context.Context, namespace, name string, timeout time.Duration)

WaitForPodReady blocks until the pod passes its readiness probe, and fails the test with the pod's last observed state if it does not within timeout.

A suite that skipped this and dialed straight away would race the readiness probe, and a fixture that is still pulling its image or crash-looping would then be reported as whatever the code under test does with a refused connection. Reporting the container's own waiting/terminated reason instead is the whole point of the poll: it is the difference between "ImagePullBackOff" and an unexplained timeout.

func WaitForServerEndpoint added in v0.5.0 

func WaitForServerEndpoint(t *testing.T, ctx context.Context, spec ServerPod)

WaitForServerEndpoint waits for publication of the current origin Pod as a ready Service backend. It does not perform the protocol exchange under test.

func WaitForSubstrateTemplateReady added in v0.1.0 

func WaitForSubstrateTemplateReady(ctx context.Context, t *testing.T, clients *Clients, atespace, name string)

WaitForSubstrateTemplateReady blocks until the substrate ActorTemplate's golden snapshot exists, returning immediately for a template that opted out of the golden snapshot. The timeout follows the sandbox class under test (see TemplateReadyTimeout).

Types

type AtenetDataplane added in v0.2.0 

type AtenetDataplane interface {
	NewParkingObserver(context.Context) (ParkingObserver, error)
	IsRetryableParkingBudgetExhaustion(status int, body string) bool
	ParkingBudgetStatus() int
	IsEgressPolicyDenied(status int, body string) bool
	PlatformMetricPrefixes([]string) []string
	RouteDurationSeen(context.Context, string) (bool, error)
	ScrapeMetrics(context.Context) (string, error)
	EgressConnectStats(before, after string) EgressConnectStats
}

AtenetDataplane captures the observable differences between the supported atenet dataplanes. Suites use these operations rather than branching on the selected implementation.

func CurrentAtenetDataplane added in v0.2.0 

func CurrentAtenetDataplane() AtenetDataplane

CurrentAtenetDataplane returns the implementation selected for this test process. Envoy remains the default to match installation defaults.

type Clients 

type Clients struct {
	K8s          *kubernetes.Clientset
	CRD          *apiextensionsclientset.Clientset
	SubstrateK8s *versioned.Clientset
	SubstrateAPI *ateclient.Client
}

func GetClients 

func GetClients() *Clients

GetClients returns the shared E2E clients. It panics if the clients have not been initialized.

func NewClients 

func NewClients(ctx context.Context) (*Clients, error)

func (*Clients) Close 

func (c *Clients) Close()

type ColorWriter 

type ColorWriter struct {
	W    io.Writer
	ANSI string
}

ColorWriter wraps an io.Writer and forces all writes to be colored with the given ANSI code. It respects noColor().

func (*ColorWriter) Write 

func (cw *ColorWriter) Write(p []byte) (n int, err error)

Write writes p to the underlying writer, wrapped in the ANSI color code and reset code.

type EgressConnectStats added in v0.5.0 

type EgressConnectStats struct {
	Connections int
	Requests    int
}

EgressConnectStats holds the delta of egress CONNECT connections and requests observed between two metric scrapes. A value of -1 indicates that the dataplane does not report the counter.

type Fixture added in v0.1.0 

type Fixture struct {
	Namespace string
	Name      string
	// DeployWith is the install flag or script that creates the fixture, so a
	// missing one reports how to fix it rather than just failing.
	DeployWith string
}

Fixture identifies an installed WorkerPool + ActorTemplate pair (both carry the same name) that suites either create Actors from directly or copy the resolved runtime — sandbox class, ateom image, container images — out of.

func EgressFixture added in v0.1.0 

func EgressFixture() Fixture

EgressFixture returns the egress demo for the sandbox class under test. Its actors trust the CA the egress gateway mints its leaves from.

type IndentWriter 

type IndentWriter struct {
	W   io.Writer
	Val string
	// contains filtered or unexported fields
}

IndentWriter wraps an io.Writer and indents each line of output.

func NewIndentWriter 

func NewIndentWriter(w io.Writer, indent string) *IndentWriter

NewIndentWriter creates a new IndentWriter.

func (*IndentWriter) Write 

func (iw *IndentWriter) Write(p []byte) (n int, err error)

Write writes p to the underlying writer, inserting the indent string at the start of each line.

type Namespace 

type Namespace struct {
	Name string
}

func CreateNamespace 

func CreateNamespace(t *testing.T) *Namespace

CreateNamespace creates a new namespace with a randomized name using the K8s API. The namespace is deleted when the test finishes, unless it failed — see the cleanup registered below.

type ParkingObserver added in v0.2.0 

type ParkingObserver interface {
	WaitForCount(context.Context, func(int) bool) (int, error)
	Close()
}

ParkingObserver waits for the dataplane's active parked-request gauge.

type ParkingStatusz added in v0.1.0 

type ParkingStatusz struct {
	Enabled   bool   `json:"enabled"`
	Active    int    `json:"active"`
	MaxParked int    `json:"max_parked"`
	MaxWait   string `json:"max_wait"`
}

ParkingStatusz mirrors the "parking" section of /statusz?format=json.

type ProbeOption added in v0.1.0 

type ProbeOption func(*probeConfig)

ProbeOption adjusts what DeployProbe installs.

func WithSystemRoots added in v0.5.0 

func WithSystemRoots() ProbeOption

WithSystemRoots adds the WebPKI roots atelet ships to the probe's projected trust file, as the egress demo's template combines them with the egress trust bundle.

func WithTrustBundle added in v0.1.0 

func WithTrustBundle() ProbeOption

WithTrustBundle projects the egress trust bundle into the probe's system-info volume, ensuring the cluster-scoped bundle exists first.

Only suites that test the bundle should ask for it. An actor whose template includes the bundle won't start if the bundle is missing, so a suite that just needs a probe shouldn't depend on it.

type RouterClient added in v0.1.0 

type RouterClient struct {
	// contains filtered or unexported fields
}

RouterClient sends HTTP requests to actors through the ingress atenet-router, the same way real traffic arrives (so the request is routed and, if needed, the actor is resumed). It port-forwards the router Service, mirroring the approach in internal/ateclient.

func NewRouterClient added in v0.1.0 

func NewRouterClient(ctx context.Context) (*RouterClient, error)

NewRouterClient establishes a port-forward to the ingress atenet-router. Call Close to tear it down.

func (*RouterClient) BaseURL added in v0.1.0 

func (c *RouterClient) BaseURL() string

BaseURL returns the local router port-forward address.

func (*RouterClient) Close added in v0.1.0 

func (c *RouterClient) Close()

Close stops the port-forward tunnel(s).

func (*RouterClient) Connect added in v0.1.0 

func (c *RouterClient) Connect(ctx context.Context, actorRef resources.ActorRef, port int) (net.Conn, error)

Connect opens a CONNECT tunnel through the router to port on actorRef, exercising atenet-router's arbitrary-port ingress support: the target port travels in the CONNECT authority (e.g. "my-actor.team-a...:9090"), the same way a real client reaches a port other than an actor's primary one. On a non-2xx response the returned error carries the status and body, mirroring atunnel.Client.DialContext's handling of the same failure mode on the egress side. The caller owns the returned connection and must Close it; the underlying port-forward is torn down by RouterClient.Close.

func (*RouterClient) ConnectWithHeaders added in v0.5.0 

func (c *RouterClient) ConnectWithHeaders(ctx context.Context, actorRef resources.ActorRef, port int, header http.Header) (net.Conn, error)

ConnectWithHeaders is like Connect but allows specifying custom headers on the outer CONNECT request (e.g. testing repeated or missing headers).

func (*RouterClient) Get added in v0.1.0 

func (c *RouterClient) Get(ctx context.Context, actorRef resources.ActorRef, path string) (*http.Response, error)

Get issues GET path to actor through the router. The caller must close the body.

func (*RouterClient) PostJSON added in v0.1.0 

func (c *RouterClient) PostJSON(ctx context.Context, actorRef resources.ActorRef, path string, body []byte) (*http.Response, error)

PostJSON issues a POST with a JSON body to an Actor through the router. The caller must close the response body.

type Server added in v0.1.0 

type Server struct {
	// Name is the Service's name, which is also the Pod's.
	Name string
	// Namespace is the namespace the server was deployed into, for a suite that
	// wants to port-forward to it or read its logs on failure.
	Namespace string
	// ClusterIP is the Service's address. Address dials it rather than a DNS
	// name: an IP keeps a caller inside a sandbox off that sandbox's resolver,
	// and makes the authority in a gateway's access log exactly what the test
	// deployed. A TLS client needs the name its certificate covers instead; see
	// SvcName.
	ClusterIP string
	Port      int
	// TLSPort is ServerPod.TLSPort: where the Service publishes HTTPS, or 0.
	TLSPort int
}

Server is a deployed ServerPod, as the address a caller dials it at.

func CreateServerService added in v0.5.0 

func CreateServerService(t *testing.T, ctx context.Context, spec ServerPod) Server

CreateServerService reserves the origin address before its Pod starts, so callers can issue a certificate for that IP. Namespace must already exist. DeployServerPod subsequently applies the same Service without replacing it.

func DeployServerPod added in v0.1.0 

func DeployServerPod(t *testing.T, ctx context.Context, spec ServerPod) Server

DeployServerPod builds spec's image, applies the shared server manifest, waits for readiness and returns the address to dial.

It registers no cleanup: everything the manifest creates is namespaced, so it goes with the namespace CreateNamespace made — and, on failure, is retained with it for `kubectl logs`.

func (Server) Address added in v0.1.0 

func (s Server) Address() string

Address is the host:port to dial the server at.

func (Server) SvcName added in v0.5.0 

func (s Server) SvcName() string

SvcName is the Service's <name>.<namespace>.svc, the one name a ServiceDNSTLS server's certificate covers. It resolves through the search path cluster DNS gives every pod, an actor's sandbox and the egress gateway included.

func (Server) TLSAddress added in v0.5.0 

func (s Server) TLSAddress() string

TLSAddress is the host:port to dial the server's HTTPS listener at.

type ServerPod added in v0.1.0 

type ServerPod struct {
	// Name names the Pod, its container and the Service alike, and is what
	// appears in kubectl output when a test fails.
	Name string
	// ImportPath is the server binary's package, as a ko:// reference. The
	// template's contract is that the binary takes --listen=:<port>, which is
	// how one manifest serves fixtures that share nothing else.
	ImportPath string
	// Args are passed to the binary ahead of --listen=:<port>, which the
	// template always appends. One image (internal/e2e/fixtures/testserver)
	// backs every server, so this is where a caller names the subcommand that
	// picks its behavior -- []string{"grpc"}, say.
	Args []string
	// Port is what the Service publishes, so an address a suite grafts into an
	// assertion — a CONNECT authority in a gateway's access log, say — is this
	// number.
	Port int
	// TargetPort is what the binary listens on, defaulting to Port. Set it to
	// publish a port the container -- uid 65532, every capability dropped --
	// cannot bind, such as 80; the Service maps Port down to it.
	TargetPort int
	// TLSPort, when set, publishes the HTTPS listener testserver's http and
	// websocket subcommands open on container port 8443 when given a
	// certificate, beside the cleartext one on Port. The readiness probe then
	// checks HTTPS, the listener that needs the certificate.
	TLSPort int
	// Namespace deploys into an existing namespace instead of a fresh one, for
	// a suite that has to populate that namespace first: credentials the pod
	// mounts have to exist before it is scheduled, and DeployServerPod cannot
	// hand back a namespace it has not created yet.
	Namespace string
	// GRPCProbe asks kubelet to probe with the gRPC health protocol instead of
	// an HTTP GET. A gRPC server answers an HTTP request with a protocol error,
	// so a server speaking grpc must set this and register the health service.
	GRPCProbe bool
	// HealthPath is the HTTP readiness path, defaulting to /healthz. Ignored
	// when GRPCProbe is set.
	HealthPath string
	// Volumes and VolumeMounts carry whatever credentials the server needs.
	// Typed, rather than more YAML in the template, so the Secret names here
	// sit beside the code that creates them instead of drifting from it.
	Volumes      []corev1.Volume
	VolumeMounts []corev1.VolumeMount
}

ServerPod describes a plain server to stand up beside the code under test: the origin an Actor's egress lands on, or the probe that dials a gateway. Everything those have in common — the pod shape, the Service, the security context — lives in the shared template, so a suite only names what differs.

func ServiceDNSTLS added in v0.5.0 

func ServiceDNSTLS(spec ServerPod) ServerPod

ServiceDNSTLS returns spec serving HTTPS as well as HTTP, with a certificate from the cluster's service-DNS signer that covers only Server.SvcName. Unless spec sets them, HTTP is on port 80 and HTTPS on 443. The server must be testserver's http or websocket subcommand.

type StatuszClient added in v0.1.0 

type StatuszClient struct {
	// contains filtered or unexported fields
}

StatuszClient reads the atenet router's /statusz page over a port-forward, for suites that assert on router-internal state (e.g. the request-parking gauge) without going through the metrics pipeline.

func NewStatuszClient added in v0.1.0 

func NewStatuszClient(ctx context.Context) (*StatuszClient, error)

NewStatuszClient establishes a port-forward to the router's status port. Call Close to tear it down.

func (*StatuszClient) Close added in v0.1.0 

func (c *StatuszClient) Close()

Close tears down the port-forward.

func (*StatuszClient) Parking added in v0.1.0 

func (c *StatuszClient) Parking(ctx context.Context) (*ParkingStatusz, error)

Parking fetches the current request-parking snapshot.

func (*StatuszClient) WaitForCount added in v0.2.0 

func (c *StatuszClient) WaitForCount(ctx context.Context, cond func(int) bool) (int, error)

WaitForCount polls the request-parking gauge until cond holds.

type SubstrateFixture added in v0.1.0 

type SubstrateFixture struct {
	// Atespace and Name locate the ActorTemplate for GetActorTemplate.
	Atespace string
	Name     string
	// PoolNamespace and PoolName locate the WorkerPool CRD.
	PoolNamespace string
	PoolName      string
	// DeployWith is the install flag or script that creates the fixture, so a
	// missing one reports how to fix it rather than just failing.
	DeployWith string
}

SubstrateFixture identifies an installed substrate ActorTemplate (the proto resource created through the ate API, not the CRD) plus the CRD WorkerPool backing it. Suites copy the resolved runtime — container images, sandbox config, sandbox size — out of the template, and the ateom image and sandbox class out of the pool.

func SubstrateCounterFixture added in v0.1.0 

func SubstrateCounterFixture() SubstrateFixture

SubstrateCounterFixture returns the substrate-resource counter demo for the sandbox class under test. E2E_SUBSTRATE_TEMPLATE_ATESPACE / E2E_SUBSTRATE_TEMPLATE_NAME / E2E_SUBSTRATE_POOL_NAMESPACE / E2E_SUBSTRATE_POOL_NAME override it, for a cluster that installs the fixture somewhere else.

type SubstrateFixtureManifests added in v0.1.0 

type SubstrateFixtureManifests struct {
	// Pool declares the k8s side: a Namespace plus the WorkerPool CRD.
	Pool string
	// Template declares one or more protojson-shaped ActorTemplate
	// documents, separated by ---.
	Template string
}

SubstrateFixtureManifests names the two manifest templates a substrate fixture is built from (both repo-relative, under internal/e2e/fixtures).

type SubstrateTemplateOptions added in v0.1.0 

type SubstrateTemplateOptions struct {
	// Atespace and Name locate the new template. The atespace is created if
	// missing; Name must be unique within it (atespaces are shared across a
	// suite's tests, unlike the k8s namespaces the CRD templates lived in).
	Atespace string
	Name     string
	// PoolName and PoolReplicas shape the WorkerPool CRD created in the test's
	// k8s namespace.
	PoolName     string
	PoolReplicas int32
	// Labels tie the template's workerSelector to the pool, keeping this
	// pool's workers invisible to other namespaces' actors.
	Labels map[string]string
	// SnapshotConfig for the new template; nil copies the source's.
	SnapshotConfig *ateapipb.SnapshotConfig
	// Modify, when set, edits the template before it is created.
	Modify func(*ateapipb.ActorTemplate)
}

SubstrateTemplateOptions shapes CreateSubstrateTemplateFrom.

Source Files

View all Source files

Directories

Path Synopsis
fixtures
probe command
Command probe is a minimal introspection actor used by the e2e suites.
 Command probe is a minimal introspection actor used by the e2e suites.
testserver command
Command testserver is the one binary behind every plain helper pod the egress e2e suites stand up.
 Command testserver is the one binary behind every plain helper pod the egress e2e suites stand up.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.