gantry

Gantry deployment artifacts

This directory carries the operator-facing pieces needed to roll out the gantry agent as a Kubernetes DaemonSet.

Files

These are Go templates (*.yaml.tmpl); the only templated value is the install namespace, which defaults to unbounded-system. Render them with make gantry-manifests (override with GANTRY_NAMESPACE=<ns> or the unified UNBOUNDED_NAMESPACE=<ns>), which writes plain manifests into deploy/gantry/rendered/.

The container image is built from images/gantry/Containerfile via make image-gantry-local (or make image-gantry-push to push).

Apply order

# Render the templates into deploy/gantry/rendered/ first (defaults to the
# unbounded-system namespace; override with UNBOUNDED_NAMESPACE / GANTRY_NAMESPACE).
make gantry-manifests

kubectl apply -f deploy/gantry/rendered/serviceaccount.yaml
kubectl apply -f deploy/gantry/rendered/configmap.yaml
# Operator: for any PRIVATE upstream registry, edit
# rendered/examples/registry-secret.example.yaml (rename it, fill in real
# username:password values keyed by registry `name:`) and apply,
# AND uncomment the matching `credentials_path:` line in
# configmap.yaml. The default ConfigMap ships credentials-free so
# the agent starts cleanly against public registries without any
# Secret being applied - origin.New eagerly reads every
# credentials_path at startup, so an unmatched path would
# crashloop the pod.
kubectl apply -f deploy/gantry/rendered/examples/registry-secret.example.yaml   # private registries only
kubectl apply -f deploy/gantry/rendered/daemonset.yaml
# rendered/examples/networkpolicy.yaml is a hardening overlay; do NOT
# apply it as part of the initial install. See "Hardening overlays"
# below for the workflow.

Building the image locally

# Single-arch into local container engine:
make image-gantry-local

# Build and push to $(CONTAINER_REGISTRY):
make image-gantry-push CONTAINER_REGISTRY=ghcr.io/your-org

# Explicit tag:
make image-gantry-local VERSION=v0.6.0

Per-node containerd setup

Nodes provisioned by unbounded-agent already have containerd configured to read /etc/containerd/certs.d and carry the managed default Gantry mirror entry in /etc/containerd/certs.d/_default/hosts.toml. On those nodes, install the Gantry DaemonSet normally; the mirror activates when the pod starts listening on 127.0.0.1:5000.

Use node-config.yaml only for standalone installs or non-agent-managed nodes that still need the default Gantry mirror entry written onto the node.

For each upstream registry the cluster pulls from, drop a hosts.toml at:

/etc/containerd/certs.d/<registry-host>/hosts.toml

derived from hosts.toml.template (substitute ${REGISTRY_SERVER} with the registry's https://... URL). containerd reloads certs.d on its own; no restart needed.

What to verify after rollout

See docs/detailed-design.md §7.6 for the full metric catalogue.

Hardening overlays

deploy/examples/ carries optional hardening manifests that are intentionally not part of the default kubectl apply workflow. Every overlay there contains at least one site-specific value (CIDR, endpoint, label) that no shipped manifest can guess correctly across arbitrary clusters, so applying them unedited will fail the cluster into a state that is hard to debug.

Production guidance: the default install leaves the mirror listener (5000) and transfer listener (5001) reachable from other pods on the cluster network at <podIP>:port. The hostIP: 127.0.0.1 binding on the DaemonSet's hostPort only restricts host-side reach; the listener inside the pod is still 0.0.0.0. Production installs should adopt examples/networkpolicy.yaml.tmpl (or an equivalent NetworkPolicy in their own overlay) to close that pod-network gap. The overlay is shipped as an example rather than a default because its allow-list depends on the cluster's node-CIDR range, which is site-specific (see the workflow below).

examples/networkpolicy.yaml

Locks transfer (5001), libp2p (4001), mirror (5000), and metrics (9095) to the minimum traffic each port needs. Holds the manifest shape required by §7.5 but defers four CIDR choices to the operator - apiserver endpoint, kubelet probe source, mirror DNAT source, registry egress. See the long "OPERATOR ACTION REQUIRED" block at the top of the file and the Production caveats table below.

Workflow:

1. Roll out the DaemonSet without the overlay and verify kubectl -n unbounded-system rollout status ds/gantry, p2p_cache_hit_total, and a successful workload pull.
2. Copy the overlay into your own repository (or a Kustomize / Helm chart), edit every ipBlock marked "OPERATOR ACTION REQUIRED", and review against your CNI's hostPort SNAT behaviour (kubectl get nodes -o yaml | grep -A2 podCIDR, etc.).
3. Apply with kubectl apply -f your-overlay/networkpolicy.yaml. Watch /readyz and any in-flight mirror pulls for at least one full image pull cycle - a wrong CIDR will surface as dht routing table empty (no peer libp2p traffic) or as containerd connection refused on 5000 (wrong mirror source CIDR), not as a NetworkPolicy validation error.
4. Roll back with kubectl delete networkpolicy -n unbounded-system gantry-agent if anything regresses.

Future hardening overlays (Pod Security Standards, dedicated PriorityClass, alternative hostNetwork: true topology) will live in the same directory and follow the same "deferred to operator, not in default install" rule.

Production caveats

A few configuration knobs that need operator attention before going to production:

HEAD semantics on cache miss

GET /v2/<repo>/blobs/<digest> on a cache miss warms the cache as a side effect; HEAD on the same URL does NOT. This is intentional (see the comment block in internal/mirror/mirror.go at the HEAD return after writeBlobHeaders) - caching a multi-GB blob just because a client asked for its size would defeat the bandwidth amplification fix Gantry exists to provide. A subsequent GET for the same digest follows the cache-miss path normally and warms the cache then.

If your client emits HEAD-then-GET patterns where you'd prefer to amortize the origin metadata round-trip, raise the issue upstream (containerd's puller, BuildKit's resolver, etc.) - those clients generally have a one-shot resolve-and-pull mode that skips the HEAD entirely.