Documentation
¶
Overview ¶
Package k3s is the config tier of a Kubernetes cluster SC stands up itself, on machines described by a `vm` resource (pkg/clouds/vm). It is the cheapest Kubernetes SC can offer in a cloud that has no managed Kubernetes at all — which is most of the Russian market — and it is deliberately a thin thing: everything above the kubeconfig (`kubernetes-cloudrun`, the Caddy edge, the helm operators) already exists and is reused unchanged.
Design: forge/docs/design/2026-10-13/sc-russia-market-readiness/k3s-all-clouds.md, roadmap item sc-k3s-cluster-resource (slice K.0).
Index ¶
- Constants
- Variables
- func KubeconfigURL(endpoint string) string
- func ReadK3sCloudrunTemplateConfig(config *api.Config) (api.Config, error)
- func ReadK3sClusterConfig(config *api.Config) (api.Config, error)
- func ToK3sCloudrunConfig(tpl any, composeCfg compose.Config, stackCfg *api.StackConfigCompose) (any, error)
- type K3sCloudrunInput
- type K3sCloudrunTemplate
- type K3sClusterConfig
- func (r *K3sClusterConfig) CredentialsValue() string
- func (r *K3sClusterConfig) DependsOnResources() []api.ParentResourceDependency
- func (r *K3sClusterConfig) EffectiveInstallScriptURL() string
- func (r *K3sClusterConfig) EffectiveServers() int
- func (r *K3sClusterConfig) EffectiveVersion() string
- func (r *K3sClusterConfig) ProjectIdValue() string
- func (r *K3sClusterConfig) ProviderType() string
- func (r *K3sClusterConfig) Validate() error
Constants ¶
const ( NodePortRangeStart = 30000 NodePortRangeEnd = 32767 )
The NodePort range a Kubernetes apiserver allocates `type: NodePort` services from, and the range k3s uses because SC never passes `--service-node-port-range`. Both ends inclusive.
They live here rather than in the firewall code that needs them because they are a property of the control plane, not of a cloud: `pkg/clouds/vm` imports this package for the `k3s` entry of a vm resource's `ingressPorts` preset table. That import direction is the reason this package must keep importing nothing from `pkg/clouds/vm` — the name of a vm resource (K3sClusterConfig.VmResource) is all it has ever needed, and widening it to the type would close a cycle.
const ( // FlannelVxlanPort carries every packet between pods on different nodes. UDP. FlannelVxlanPort = 8472 // KubeletPort serves metrics and `kubectl logs`/`exec`/`port-forward` — which // is also why it must never be public: it reaches into any pod on the node. KubeletPort = 10250 // EtcdClientPort and EtcdPeerPort are the embedded etcd of a multi-server // cluster. Contiguous, and used as a range. EtcdClientPort = 2379 EtcdPeerPort = 2380 )
The ports k3s nodes use to talk to EACH OTHER. Node-to-node only — see vm.PortRange.Internal for why the distinction is per-port and not per-machine.
Taken from k3s's own "Inbound Rules for K3s Nodes" and narrowed to the backends SC actually runs: flannel's default VXLAN backend (so no 51820/51821, which are the wireguard-native ones), and embedded etcd, which is what `--cluster-init` gives an HA control plane.
const APIPort = 6443
APIPort is the port the k3s API server listens on. Not configurable: it is baked into the kubeconfig k3s itself writes, into `--tls-san` handling, and into every firewall note in the design doc. A flag here would have to be threaded through all three to mean anything.
const DefaultInstallScriptURL = "https://get.k3s.io"
DefaultInstallScriptURL is Rancher's installer. It resolves the binary from GitHub releases, and both hosts were measured reachable from inside Russia on 2026-10-17 at 6–8 MB/s (roadmap item sc-ru-market-readiness, slice B0) — which is why this is the default rather than a bundled binary. InstallScriptURL exists for the networks where that stops being true.
const DefaultVersion = "v1.36.5+k3s1"
DefaultVersion is the k3s release SC installs when a resource does not pin one.
A literal version and NOT a channel ("stable", "latest"), deliberately, and the same reasoning refuses a channel in Validate: a channel is resolved by the install script at install time, so the same SC config provisioned twice a month apart stands up two different control planes, and the "is the installed version the wanted one?" guard that keeps a re-provision from restarting the API server cannot be evaluated at all. Bump this by hand, from `curl -s https://update.k3s.io/v1-release/channels` (the `stable` entry), and only together with a smoke — a control-plane upgrade is not a dependency bump.
const ProviderType = "k3s"
ProviderType is the discriminator K3sClusterConfig reports from ProviderType(), the key pkg/clouds/pulumi/k3s registers its Provider under, and the directory generated schemas land in (docs/schemas/k3s).
It is its OWN provider type rather than the vm resource's "vm", and that is not a stylistic choice: provision.go's initProvider looks the provider up by the config's ProviderType() (pkg/clouds/pulumi/provision.go:311), so a k3s-cluster config that reported "vm" would be handed to the vm provider, which casts its input to *vm.VmConfig and would refuse it with a type-assertion error.
const ResourceTypeK3sCluster = "k3s-cluster"
ResourceTypeK3sCluster is the `type:` a parent stack's resource declares.
const TemplateTypeK3sCloudrun = "k3s-cloudrun"
TemplateTypeK3sCloudrun deploys a cloud-compose stack onto a k3s-cluster resource of the parent stack.
It exists for one reason, and it is not a difference in what gets deployed: the existing `kubernetes-cloudrun` template takes its kubeconfig as a LITERAL string in the template config (k8s.KubernetesConfig.Kubeconfig, surfaced through CredentialsValue() so provision.go's initProvider can build the provider from it). A k3s cluster's kubeconfig does not exist until the cluster has been provisioned, so it can only arrive as a parent-stack output — the same way gke-autopilot's does. Hence a template that names a cluster resource instead of carrying credentials.
Everything below the kubeconfig is reused unchanged: the compose→containers mapping is k8s.ToKubernetesRunConfig, and the deploy is kubernetes.KubeRunWithProvider.
Variables ¶
var DisableableComponents = []string{
"traefik",
"servicelb",
"local-storage",
"metrics-server",
}
The k3s packaged components SC is willing to turn off. k3s accepts `--disable` for each of these; anything else is refused rather than passed through, because a misspelled component name leaves the component RUNNING and k3s says nothing — the operator believes traefik is gone while it holds port 80.
Functions ¶
func KubeconfigURL ¶
KubeconfigURL is the `server:` URL the exported kubeconfig must carry.
func ToK3sCloudrunConfig ¶
func ToK3sCloudrunConfig(tpl any, composeCfg compose.Config, stackCfg *api.StackConfigCompose) (any, error)
ToK3sCloudrunConfig converts a cloud-compose client stack for the k3s path.
It delegates the entire mapping to k8s.ToKubernetesRunConfig against a synthesised k8s.CloudrunTemplate. That is not laziness: ConvertComposeToContainers, the cloudExtras handling (probes, VPA, topology spread, security context) and ingress container detection are a few hundred lines that are identical for every Kubernetes flavour, and a second copy would drift silently — the symptom being a cloudExtras field that works on GKE and is ignored on k3s.
Types ¶
type K3sCloudrunInput ¶
type K3sCloudrunInput struct {
k8s.KubeRunInput `json:",inline" yaml:",inline"`
// K3sClusterResource is carried through from the template because the
// provisioning side needs it to spell the parent's export names, and the
// embedded KubeRunInput has nowhere to put it.
K3sClusterResource string `json:"k3sClusterResource" yaml:"k3sClusterResource"`
}
K3sCloudrunInput is what a converted client stack carries into provisioning. The embedded k8s.KubeRunInput is the point: it is byte-for-byte what kubernetes.KubeRunWithProvider consumes, so the deploy below the kubeconfig is the existing code path and not a second copy of it.
func (*K3sCloudrunInput) CredentialsValue ¶
func (i *K3sCloudrunInput) CredentialsValue() string
func (*K3sCloudrunInput) ProjectIdValue ¶
func (i *K3sCloudrunInput) ProjectIdValue() string
func (*K3sCloudrunInput) ProviderType ¶
func (i *K3sCloudrunInput) ProviderType() string
ProviderType/ProjectIdValue/CredentialsValue are re-declared on the input rather than inherited: the embedded k8s.KubeRunInput promotes k8s.KubernetesConfig's versions, which report "kubernetes" and hand back an empty kubeconfig as credentials. See K3sCloudrunTemplate.ProviderType for what that would cost.
type K3sCloudrunTemplate ¶
type K3sCloudrunTemplate struct {
api.Credentials `json:",inline" yaml:",inline"`
// K3sClusterResource is the name of the k3s-cluster resource in the same parent
// stack whose kubeconfig this template deploys through. Same role as
// gke-autopilot's `gkeClusterResource`.
K3sClusterResource string `json:"k3sClusterResource" yaml:"k3sClusterResource"`
// The registry images are pushed to and pulled from. Required, and there is no
// fallback: unlike the vm-compose path, which can ship an image over SSH with
// `imageDelivery: ssh-tar`, a kubelet can only pull. A k3s stand with no reachable
// registry is a stand that cannot deploy — which on the RU-market track is a
// deliberate thing to surface at config time rather than at image-pull time.
docker.RegistryCredentials `json:",inline" yaml:",inline"`
CaddyResource *string `json:"caddyResource,omitempty" yaml:"caddyResource,omitempty"` // name of the caddy resource in the parent stack
// TlsConfig carries `clientTls` / `originTls` and the deprecated `useSSL`,
// inline — the same vocabulary as the kubernetes-cloudrun template, because
// this template builds one.
k8s.TlsConfig `json:",inline" yaml:",inline"`
}
K3sCloudrunTemplate is the parent-stack template for deploying onto a k3s cluster.
Deliberately NOT embedding k8s.CloudrunTemplate: that would inherit a `kubeconfig` field which must always be empty here, and a config field that must never be set is a field someone will eventually set. The provisioning side builds a k8s.KubeRunInput from these values instead.
func (*K3sCloudrunTemplate) CredentialsValue ¶
func (t *K3sCloudrunTemplate) CredentialsValue() string
CredentialsValue returns an empty JSON object rather than "": api.ConvertAuth json.Unmarshals this, and "" is not valid JSON. There are no credentials to carry — reaching the cluster needs the parent's kubeconfig output, and pushing images needs the registry fields above.
func (*K3sCloudrunTemplate) ProjectIdValue ¶
func (t *K3sCloudrunTemplate) ProjectIdValue() string
func (*K3sCloudrunTemplate) ProviderType ¶
func (t *K3sCloudrunTemplate) ProviderType() string
ProviderType reports k3s for the same reason K3sClusterConfig does, and with a sharper consequence here: provision.go's initProvider builds a provider from this value on the DEPLOY path too (deploy.go:193 → getProvisionParams → initProvider, against the converted client-stack config). Reporting "kubernetes" would send it to kubernetes.Provider, which calls CredentialsValue() and builds a real pulumi-kubernetes provider out of an empty kubeconfig — and pulumi-kubernetes with no kubeconfig falls back to the AMBIENT one. On a laptop with a live ~/.kube/config that is a provider pointed at whatever cluster the operator last used.
So: k3s, whose provider is pulumi-command's and is simply unused on this path. K3sCloudrun overrides params.Provider with the real Kubernetes provider it builds from the parent's kubeconfig export.
func (*K3sCloudrunTemplate) Validate ¶
func (t *K3sCloudrunTemplate) Validate() error
type K3sClusterConfig ¶
type K3sClusterConfig struct {
// VmResource is the name of the `type: vm` resource, in the same parent stack and
// the same environment, whose hosts become this cluster's nodes.
VmResource string `json:"vmResource" yaml:"vmResource"`
// Version pins the k3s release. Defaults to DefaultVersion; a channel name is
// refused (see DefaultVersion).
Version string `json:"version,omitempty" yaml:"version,omitempty"`
// Endpoint is the host or IP that the EXPORTED kubeconfig points at, and the
// first `--tls-san`.
//
// It defaults to the node's own address, which is correct for the single-node
// stand K.0 builds. Set it when the API server is reached through a name or a
// load balancer rather than directly — an endpoint that is not a SAN on the
// serving certificate fails every request with an x509 hostname error, which is
// why this field and the SAN list are the same field by default.
Endpoint string `json:"endpoint,omitempty" yaml:"endpoint,omitempty"`
// Servers is how many of the vm resource's hosts run the control plane. The rest
// are agents.
//
// Default 1 (see EffectiveServers): one server and N-1 agents, which is the
// topology worth reaching for first — it schedules pods across the whole fleet
// and survives the loss of any agent, and it is the only shape a 2-machine pool
// can have. Its control plane is a single point of failure: while host 0 is down
// the API server is unreachable and nothing can be deployed, though pods already
// running on the agents keep serving.
//
// Set it to 3 (or 5) to run embedded etcd instead, which survives the loss of a
// control-plane node. It must be ODD: etcd needs a majority of its members to
// elect a leader, so 2 servers tolerate zero failures — strictly worse than 1,
// because either machine going away takes the API server down with it. Refused
// rather than rounded.
//
// Ignored on a byo pool that names its own `hosts[].role`, where the operator
// chose which machine runs what; a value disagreeing with that choice is refused
// rather than silently losing (pkg/clouds/pulumi/k3s/cluster.go).
Servers int `json:"servers,omitempty" yaml:"servers,omitempty"`
// TlsSans are additional names to put on the API server certificate, beyond
// Endpoint and the node address. Spend them freely: adding a SAN later means
// re-issuing the certificate and restarting the API server, while an unused SAN
// costs nothing (k3s-all-clouds.md §4).
TlsSans []string `json:"tlsSans,omitempty" yaml:"tlsSans,omitempty"`
// Disable overrides which packaged components are turned off. Leave it unset:
// the default is derived from the host (see pkg/clouds/pulumi/k3s/cluster.go),
// because whether klipper-lb can have ports 80 and 443 depends on whether the
// machine's own compose ingress already holds them.
Disable []string `json:"disable,omitempty" yaml:"disable,omitempty"`
// ExtraServerArgs is passed verbatim to the k3s server. The flags SC manages
// itself are refused (managedServerArgs).
ExtraServerArgs []string `json:"extraServerArgs,omitempty" yaml:"extraServerArgs,omitempty"`
// InstallScriptURL overrides the installer location for a network that cannot
// reach Rancher's (see DefaultInstallScriptURL).
InstallScriptURL string `json:"installScriptUrl,omitempty" yaml:"installScriptUrl,omitempty"`
api.Credentials `json:",inline" yaml:",inline"`
}
K3sClusterConfig is the resource config of `type: k3s-cluster`.
It describes the CLUSTER only. The machines it runs on are described by a `vm` resource in the same parent stack and referenced by name through VmResource — the same split as `gcp-gke-autopilot` referencing its own resources, and the reason the k3s path inherits cloud-init, SSH readiness, key handling and every future cloud driver from pkg/clouds/vm without a line of its own.
func (*K3sClusterConfig) CredentialsValue ¶
func (r *K3sClusterConfig) CredentialsValue() string
CredentialsValue returns the resolved `${auth:...}` blob verbatim, "{}" when none was given. A k3s cluster authenticates to nothing: it rides the SSH key of the vm resource it names, and that key is never copied into this config.
func (*K3sClusterConfig) DependsOnResources ¶
func (r *K3sClusterConfig) DependsOnResources() []api.ParentResourceDependency
DependsOnResources tells provision.go that this resource must be provisioned AFTER the vm resource it names, and makes the vm resource's output available as params.ResourceOutputs[VmResource] (pkg/clouds/pulumi/provision.go:145-172).
This is the whole mechanism by which a cluster learns its nodes' addresses and readiness gates without a StackReference back into the stack currently being provisioned — which is not merely slow, it is a cycle.
func (*K3sClusterConfig) EffectiveInstallScriptURL ¶
func (r *K3sClusterConfig) EffectiveInstallScriptURL() string
func (*K3sClusterConfig) EffectiveServers ¶
func (r *K3sClusterConfig) EffectiveServers() int
EffectiveServers is the control-plane node count, defaulting to 1.
The one place the default lives, so the provisioning tier never re-derives it: a resource that disagreed with itself about which hosts are servers would join a machine to a cluster as an agent on one apply and as an etcd member on the next, and the second of those is not reversible by changing the number back.
func (*K3sClusterConfig) EffectiveVersion ¶
func (r *K3sClusterConfig) EffectiveVersion() string
func (*K3sClusterConfig) ProjectIdValue ¶
func (r *K3sClusterConfig) ProjectIdValue() string
ProjectIdValue has no meaning for a cluster SC installs over SSH: there is no cloud project behind it. The k8s-style "n/a" keeps `${resource:...projectId}` from interpolating to something that looks real (same choice as vm.VmConfig).
func (*K3sClusterConfig) ProviderType ¶
func (r *K3sClusterConfig) ProviderType() string
func (*K3sClusterConfig) Validate ¶
func (r *K3sClusterConfig) Validate() error
Validate enforces the same rule as the rest of the VM path: either support a field or refuse it. It is called from the provisioning tier rather than from ReadK3sClusterConfig, because the config readers are also driven with an empty config by cmd/schema-gen, and a reader that rejects an empty config drops the whole resource out of the generated schema.