e2e/

KubeSolo e2e smoke harness

A small Bash harness that boots KubeSolo as a container (the artifact we ship), waits for the cluster to become ready, and runs smoke assertions. It is driven by kubesoloctl so it exercises the same install path users do.

Scripts

Manifest deployment tiers (manifests.sh + manifests/)

Each tier applies its manifests, gates on readiness with kubectl wait/rollout status, asserts behaviour, then deletes its namespace(s). Set E2E_SKIP_MANIFESTS=1 to skip the manifest tiers. Authentication still runs.

Running locally

Requires Docker, kubectl, curl, Go, and a kubesoloctl binary. Privileged containers must be allowed (KubeSolo needs --privileged for cgroup delegation, mount propagation and iptables). Go builds the authentication issuer in test/e2e/oidc; that binary runs on the host, so the build uses GOHOSTOS and GOHOSTARCH.

# Build kubesoloctl and run against a locally built image:
make build-kubesoloctl
IMAGE=portainer/kubesolo:dev KUBESOLOCTL=./dist/kubesoloctl test/e2e/run.sh

# A local engine image can be passed by id (no registry fetch):
# IMAGE=sha256:<docker image id> KUBESOLOCTL=./dist/kubesoloctl test/e2e/run.sh

# …or via the Makefile target:
make test-e2e E2E_IMAGE=portainer/kubesolo:dev

Note: KubeSolo is Linux-only. On macOS it can only run inside a Linux VM/container, so the harness is intended for Linux hosts and Linux CI runners.

run.sh creates a temporary directory, bind-mounts it read-only at /etc/kubesolo/e2e-auth, boots the cluster without a JWT authentication file, runs smoke and the manifest tiers, then authentication.sh. That script serves a local issuer on the container network gateway, writes an authentication file mapping usernames to k8s-<sub>, points kubernetes.apiServer.authenticationConfigFile at /etc/kubesolo/e2e-auth/authentication.yaml, and restarts the same container once. Before that restart an issuer token must get 401; after it, the token must map to k8s-<sub> and a token for another audience must get 401. It then replaces the file with a panda-<sub> mapping, then with one whose username expression does not compile, and finally with the k8s-<sub> mapping again. The invalid file must raise kube-apiserver's apiserver_authentication_config_controller_automatic_reloads_total{status="failure"} while panda-<sub> keeps working, and none of these replacements may restart the container. The EXIT trap uninstalls the instance and deletes the temporary directory even when the fixture build fails.

RELOAD_TIMEOUT is how long authentication waits for each username mapping (180 seconds by default).

Tunables (environment)

Why this leans on kubesoloctl (the seam)

kubesoloctl install --run-mode=container already does everything the harness would otherwise hand-roll, which is why there is no raw docker run here:

• Pulls the image itself (ImagePull), so no separate docker pull step.
• Runs a privileged container with cgroupns=host, a persistent data volume, and publishes 6443/tcp on a random 127.0.0.1 host port.
• Waits for the API server and merges a host-reachable kubeconfig (server rewritten to https://127.0.0.1:<port>, context kubernetes-admin@<container name>) into ~/.kube/config.

Because the published port is chosen at install, do not assume 6443:6443 — always go through the merged kubeconfig (the kc helper resolves the context). kubesoloctl kubeconfig view returns the in-container kubeconfig (wrong host port) and is not suitable for host access.

Scope

Smoke suite, manifest deployment tiers, and the container authentication checks in run.sh. Performance/footprint measurement and a scheduled conformance/density run are layered on separately (Phase 6).