Documentation
¶
Overview ¶
Package config is the single source of truth for every operator-tunable knob the Gantry agent exposes.
The design docs enumerate configuration in many places (the design doc cache, the design doc direct-origin-fallback / DHT health, the design doc origin-failure circuit breaker, the design doc hosts.toml / upstream registries, the design doc open questions). This package collects them into a single Config struct with field-level documentation pointing at the design-doc citation.
Sources, in increasing precedence:
1. Built-in defaults from NewDefault. 2. YAML file at --config=PATH (optional). 3. Environment variables prefixed GANTRY_. 4. Command-line flags.
Later sources win. Validate runs after all sources have been merged and returns an aggregate of every problem found, not just the first.
Index ¶
Constants ¶
const ( // StorageModeContainerd routes reads/writes through the local // containerd content store (plan). This is the only // accepted storage mode; the legacy hostPath "gantry-cache" mode // was removed in plan . StorageModeContainerd = "containerd" )
Recognised StorageMode values.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Config ¶
type Config struct {
// MirrorListen is the loopback address for containerd's mirror endpoint
// (the design doc). MUST be loopback-only unless
// MirrorBindAllowNonLoopback is true (operator opt-in: e.g. when the
// pod is reached via a hostPort with hostIP=127.0.0.1 DNAT'ing into
// the pod network - see deploy/gantry/daemonset.yaml).
MirrorListen string `yaml:"mirror_listen"`
// MirrorBindAllowNonLoopback relaxes the loopback-only validation on
// MirrorListen. Set to true ONLY when the deployment guarantees the
// mirror is unreachable from off-node by some other mechanism - the
// shipped DaemonSet uses hostPort with hostIP=127.0.0.1, which DNATs
// the node's loopback into the pod so containerd reaches the mirror
// over 127.0.0.1 even though the pod itself binds 0.0.0.0. Equivalent
// alternatives: a host-level firewall blocking the mirror port on
// non-loopback interfaces, or running the agent in the host network
// namespace bound to 127.0.0.1. The default (false) is the safe value
// for any non-Kubernetes deployment. See README.md "Security model"
// for the full opt-in checklist.
MirrorBindAllowNonLoopback bool `yaml:"mirror_bind_allow_non_loopback"`
// TransferListen is the peer-facing HTTP/2 endpoint (the design doc). The bind is
// typically 0.0.0.0; cluster-internal isolation comes from
// NetworkPolicy + the `Gantry-Mirrored: 1` request-header gate +
// digestpipe integrity verification, not from the bind address. See
// README.md "Security model" for the full threat model - in
// particular, the endpoint is plaintext h2c by design and assumes
// the cluster network is trusted. Operators that need on-the-wire
// confidentiality should layer a service mesh (Istio / Linkerd /
// Cilium mTLS) above Gantry.
TransferListen string `yaml:"transfer_listen"`
// MetricsListen is the Prometheus scrape endpoint and /readyz /livez
// kubelet-probe target (the design doc). Default is 0.0.0.0:9095 because both
// Prometheus (off-node) and the kubelet (off-pod, node-IP source)
// need to reach it - a loopback default would silently break the
// DaemonSet's readiness gate. Access control belongs in NetworkPolicy
// and pod ports, not in the bind address.
MetricsListen string `yaml:"metrics_listen"`
// Libp2pListen is the multiaddr(s) the libp2p host advertises (the design doc).
// Empty means "use libp2p defaults" and pick at random.
Libp2pListen []string `yaml:"libp2p_listen"`
// Libp2pIdentityPath is the on-disk path of the persisted libp2p key
// (the design doc). Lost identity is not catastrophic; old DHT records age out.
Libp2pIdentityPath string `yaml:"libp2p_identity_path"`
// Libp2pBootstrapPeers is an optional list of static multiaddrs to seed
// the libp2p host's connection set on startup. In production these are
// usually discovered via the K8s informer (the design doc) so this field defaults
// to empty; tests and small clusters can use it directly.
Libp2pBootstrapPeers []string `yaml:"libp2p_bootstrap_peers"`
// NodeName is the Kubernetes node this agent runs on. Sourced via the
// Downward API (env spec.nodeName) into GANTRY_NODE_NAME. Used as the
// stable HRW NodeID and as the join key against the Node informer for
// zone resolution.
NodeName string `yaml:"node_name"`
// PodName is the Kubernetes pod name of this agent. Sourced via the
// Downward API (env metadata.name) into GANTRY_POD_NAME. Used to
// self-patch pod annotations with the libp2p peer.ID and transfer
// addr so other agents can discover this peer (the design doc, the design doc).
PodName string `yaml:"pod_name"`
// PodIP is the agent's routable Pod IP. Sourced via the Downward
// API (env status.podIP) into GANTRY_POD_IP. Used to rewrite
// 0.0.0.0 wildcard listen addresses into dialable advertised
// addresses when self-announcing on Pod annotations (the design doc): a peer
// publishing 0.0.0.0:5001 is otherwise unreachable from other
// pods, defeating libp2p bootstrap on first-cluster boot. Empty
// when running outside Kubernetes; self-announce then publishes
// only non-wildcard listen addresses.
PodIP string `yaml:"pod_ip"`
// MembersNamespace restricts the Pod informer to a single namespace.
// Empty means cluster-wide list/watch - useful for read-only
// scenarios, but production deployments MUST set this. The
// self-announce write path (members.AnnounceSelf) patches the
// agent's own pod via Pods(namespace).Patch and refuses to run
// when namespace == "" because the apiserver does not infer a
// pod's home namespace from the pod name alone. Without a
// namespace the agent's three peer-coordination annotations
// (gantry.io/peer-id, gantry.io/p2p-addrs, gantry.io/transfer-addr)
// are never published, so peers cannot translate this agent's
// node name into a dialable libp2p peer ID - every inbound
// Coord.PleasePull / PullIntentQuery 503s silently.
//
// The shipped DaemonSet wires GANTRY_MEMBERS_NAMESPACE via the
// Downward API (`fieldRef: metadata.namespace`) so operators
// following deploy/gantry/daemonset.yaml satisfy this for free. Hand-
// rolled envFrom that misses it is the failure mode this
// validation catches at startup rather than at first
// Coord.PleasePull miss.
MembersNamespace string `yaml:"members_namespace"`
// MembersLabelSelector is the K8s label selector that identifies Gantry
// DaemonSet pods. Used to find peer agents (the design doc). Default matches the
// canonical app.kubernetes.io label.
MembersLabelSelector string `yaml:"members_label_selector"`
// MembersKubeconfig is an optional path to a kubeconfig file. Empty
// means in-cluster service-account discovery (the production path).
MembersKubeconfig string `yaml:"members_kubeconfig"`
// MembersSyncTimeout is how long the agent waits for the initial
// pod and node informer list-and-watch to complete at startup.
// In production mode a timeout is fatal (it surfaces broken RBAC /
// API egress early rather than silently degrading). Raise this on
// clusters with a slow API server or during large-scale simultaneous
// DaemonSet rollouts where the apiserver is under elevated load.
// Zero means "use the built-in default of 30s".
MembersSyncTimeout time.Duration `yaml:"members_sync_timeout"`
// StorageMode selects which backend the agent uses as the read/write
// content store. Only "containerd" is supported - the legacy
// "gantry-cache" hostPath backend was removed. The field is
// retained so existing ConfigMaps that still set it to "containerd"
// continue to parse, and so a clear migration error surfaces for
// any operator still on "gantry-cache". It is not exposed as a
// CLI flag or env var because there is no other valid value to
// select.
//
// Default is "containerd". Exposed via the
// gantry_storage_mode_info metric for observability.
StorageMode string `yaml:"storage_mode"`
// LegacyDeprecated captures YAML fields that used to live on
// Config but are no longer consumed by any code path. They are
// kept here purely so existing ConfigMaps that still set
// `cache_dir`, `cache_budget_bytes`, etc. continue to parse
// (LoadYAML uses KnownFields=true). New deployments should not
// set them. A future major version will remove the struct.
LegacyDeprecated LegacyDeprecatedConfig `yaml:",inline"`
// ContainerdSocket is the path to the containerd gRPC API socket
// that the cdsub subsystem dials to discover locally-cached images
// and announce them on the DHT (the design doc image-event -> Provide loop),
// that the transfer endpoint reads from on cache miss to serve
// peers without a re-download, and that the puller writes into on
// background origin pulls (storage_mode=containerd is the only
// supported mode; see).
//
// REQUIRED. Validate rejects an empty value when
// storage_mode=containerd (the only accepted storage_mode), which
// is enforced at startup. The default deploy manifests set it to
// "/run/containerd/containerd.sock"; operators on non-default
// socket paths override via `containerd_socket` in the ConfigMap
// or GANTRY_CONTAINERD_SOCKET in the environment. The agent will
// fail to start (and readyz will stay 503) if the socket cannot
// be reached.
ContainerdSocket string `yaml:"containerd_socket"`
// ContainerdNamespace is the containerd namespace cdsub watches.
// Kubelet uses "k8s.io" for pod containers - the default. Set to
// "moby" for Docker-managed images, or any custom namespace.
ContainerdNamespace string `yaml:"containerd_namespace"`
// ContainerdLeaseTTL is the lifetime of containerd content-store
// leases that Gantry attaches when ingesting blobs. Containerd's
// garbage collector will not delete content while at least one
// lease references it, so the TTL governs how long a Gantry-pulled
// blob is protected before the pod it was pulled for is
// responsible for keeping it alive via its own image reference.
// Plan range: 30m–120m. Default 60m.
ContainerdLeaseTTL time.Duration `yaml:"containerd_lease_ttl"`
// ContainerdLeaseCleanupInterval is the period at which the agent
// scans containerd's lease catalogue and deletes expired
// Gantry-owned leases. Defaults to ContainerdLeaseTTL/2 if zero.
ContainerdLeaseCleanupInterval time.Duration `yaml:"containerd_lease_cleanup_interval"`
// UpstreamRegistries enumerates every OCI registry the agent mirrors
// (the design doc, the design doc). The agent rejects requests whose ?ns= does not match
// one of these once more than one is configured.
UpstreamRegistries []UpstreamRegistry `yaml:"upstream_registries"`
// HRWK is the top-K size for HRW probe (the step 3 default 3; the design doc
// open question).
HRWK int `yaml:"hrw_k"`
// HRWTopologyScope selects "cluster" (HRW over all nodes) or "zone"
// (HRW within the requester's zone) - the design doc / the design doc open question.
HRWTopologyScope string `yaml:"hrw_topology_scope"`
// ZoneLabelKey is the Kubernetes node label that identifies the zone
// when HRWTopologyScope == "zone". Default
// `topology.kubernetes.io/zone` (the design doc).
ZoneLabelKey string `yaml:"zone_label_key"`
// CoordPeerAuthzEnforce flips coord peer authorization from
// observe-only (record the unauthorized-peer metric, still serve) to
// enforce (reject inbound coord requests whose libp2p peer ID is not
// in the membership view). Default false: ship observe-only first,
// verify peer-id annotations are visible for every ready Gantry pod,
// size p2p_coord_unauthorized_peer_total across a full rollout, then
// flip to true once it stays at zero.
CoordPeerAuthzEnforce bool `yaml:"coord_peer_authz_enforce"`
// CoordMaxDigestsPerRequest caps a single please_pull batch. The default
// 256 is intentionally far above normal manifest child counts while staying
// well below the 1 MiB coord envelope budget.
CoordMaxDigestsPerRequest int `yaml:"coord_max_digests_per_request"`
// CoordMaxConcurrentPulls caps background origin pulls started by inbound
// please_pull. Each pull holds an HTTP response body, a containerd writer,
// goroutine state, and a lease, so this protects the node from fan-out.
CoordMaxConcurrentPulls int `yaml:"coord_max_concurrent_pulls"`
// NF5JitterBase is the base delay in the direct-origin-fallback jitter window
// `[0, base * ln(N))` (the design doc default 3 s).
NF5JitterBase time.Duration `yaml:"nf5_jitter_base"`
// NF5JitterCap is a hard ceiling on the computed jitter window.
// Zero means no cap (original behaviour). Set this to bound worst-case
// cold-start latency on large clusters: at N=300 the uncapped window is
// ~17s; a cap of 10s limits the maximum additional delay imposed by NF5
// regardless of cluster size. The configured NF5JitterBase still
// controls the shape of the distribution up to the cap.
NF5JitterCap time.Duration `yaml:"nf5_jitter_cap"`
// NF5PerNodeRateLimit is the per-node direct-origin fallback rate
// (token bucket, fallbacks/minute; the design doc default 2).
NF5PerNodeRateLimit int `yaml:"nf5_per_node_rate_limit"`
// BootstrapWindow is the time after startup during which DHT-empty is
// not trusted as cold-start evidence (the design doc default 30 s).
BootstrapWindow time.Duration `yaml:"bootstrap_window"`
// BootstrapRoutingTablePct is the routing-table-size threshold that
// supersedes BootstrapWindow once met (the design doc default 25%).
BootstrapRoutingTablePct int `yaml:"bootstrap_routing_table_pct"`
// TopKExpansionFactorDegraded is the multiplier applied to HRWK when
// expanding top-K under Degraded health (the step 5 / the design doc default 2).
TopKExpansionFactorDegraded int `yaml:"topk_expansion_factor_degraded"`
OriginFailureCooldownInitial time.Duration `yaml:"origin_failure_cooldown_initial"`
OriginFailureCooldownMax time.Duration `yaml:"origin_failure_cooldown_max"`
OriginFailureCooldownMultiplier int `yaml:"origin_failure_cooldown_multiplier"`
OriginFailureHonorWindowCap time.Duration `yaml:"origin_failure_honor_window_cap"`
// OriginFailureClassesTrustedClusterWide controls which the design doc failure
// classes are propagated cluster-wide as 5xx-immediate (default
// {auth, not_found, rate_limited}; `transient` is honored locally
// only).
OriginFailureClassesTrustedClusterWide []string `yaml:"origin_failure_classes_trusted_cluster_wide"`
// LogLevel is one of "debug", "info", "warn", "error".
LogLevel string `yaml:"log_level"`
// LogFormat is "json" (production) or "text" (development).
LogFormat string `yaml:"log_format"`
}
Config is the typed configuration surface.
Every field carries a yaml/json tag matching the file/env name and a comment citing the design-doc section it derives from. Defaults are set by NewDefault; see Validate for hard correctness constraints.
func Load ¶
func Load(args []string, env func(string) string, configPath string) (*Config, *flag.FlagSet, error)
Load is the convenience composition of NewDefault -> LoadYAML(file) -> LoadEnv -> BindFlags -> fs.Parse(args). It returns the fully-merged Config and the FlagSet for callers that want to print -help text. Validate is the caller's responsibility - Load does not call it so callers can inspect partial configs (e.g., for `gantry version`).
func NewDefault ¶
func NewDefault() *Config
NewDefault returns a Config populated with the design-doc defaults. All fields are set; Validate against this MUST pass.
func (*Config) BindFlags ¶
BindFlags registers command-line flags on fs that overlay c. Call after LoadYAML / LoadEnv but before fs.Parse so flags win.
func (*Config) LoadEnv ¶
LoadEnv overlays environment variables of the form GANTRY_<UPPER_SNAKE>. Returns a multi-error of any parse failures encountered.
Only scalar fields are overlaid here; list fields (UpstreamRegistries, Libp2pListen, OriginFailureClassesTrustedClusterWide) are file-only by design - env vars are an awkward shape for them.
func (*Config) LoadYAML ¶
LoadYAML overlays a YAML document onto c. Unknown fields are an error so typos in config files don't silently no-op.
func (*Config) Redacted ¶
Redacted returns a copy of c suitable for logging. Currently, credentials are referenced only by path, so nothing requires actual redaction; the method exists so future secret-bearing fields have one obvious place to be sanitized.
func (*Config) ResolveUpstream ¶
func (c *Config) ResolveUpstream(ns string) (UpstreamRegistry, bool)
ResolveUpstream returns the UpstreamRegistry whose Name (or NSAlias) equals ns. Returns false if ns does not match any configured registry.
type LegacyDeprecatedConfig ¶
type LegacyDeprecatedConfig struct {
CacheDir string `yaml:"cache_dir,omitempty"`
CacheBudgetBytes int64 `yaml:"cache_budget_bytes,omitempty"`
CacheForcedEvictionHeadroomPct int `yaml:"cache_forced_eviction_headroom_pct,omitempty"`
EvictionProviderCountThreshold int `yaml:"eviction_provider_count_threshold,omitempty"`
}
LegacyDeprecatedConfig captures YAML field names that used to live on Config but no longer have any effect. They are accepted here purely so existing ConfigMaps that still set them parse without error under KnownFields=true. None of these fields are exposed via CLI flags or environment variables - setting them via env/flag is not supported, only YAML round-trips for back-compat. A future major version will remove this struct entirely.
Removal trail : - cache_dir / cache_budget_bytes / cache_forced_eviction_headroom_pct / eviction_provider_count_threshold: the hostPath cache backend was deleted; containerd's own GC owns blob lifetime now.
type UpstreamRegistry ¶
type UpstreamRegistry struct {
// Name is the canonical identifier (used as the ?ns= value from
// containerd and as the lookup key for credentials).
Name string `yaml:"name"`
// Endpoint is the HTTPS URL of the registry, e.g.
// "https://registry.example.com".
Endpoint string `yaml:"endpoint"`
// CredentialsPath is a hostPath-mounted file containing the registry
// credentials. Format: "username:password" (or "_json_key:<json>" for
// the well-known GCR pattern). Empty means anonymous pulls.
CredentialsPath string `yaml:"credentials_path"`
// NSAlias lets containerd's ?ns= use a different name than Name.
// Empty means ?ns= must equal Name.
NSAlias string `yaml:"ns_alias"`
}
UpstreamRegistry describes one OCI registry the agent mirrors.