kubeaid-bootstrap-script

KubeAid Bootstrap Script

The KubeAid Bootstrap Script is used to bootstrap Kubernetes clusters using Cluster API and KubeAid.

Currently it only supports bootstrapping self-managed clusters in AWS.

Official Guides

• Bootstrapping a self-managed cluster in AWS

Developer Guide (AWS edition)

Make sure, you've Docker installed and running in your system.

Run make build-image-dev to build the KubeAid Bootstrap Script container image (development version).

Then run make run-container-dev to run the container.

Use make exec-container-dev to execute into the container.

Once you're inside the container, use make generate-sample-config-aws-dev to generate a sample config file at ./outputs/kubeaid-bootstrap-script.config.yaml, targetting the AWS cloud provider. Adjust the config file according to your needs.

Export your AWS credentials as environment variables like such :

export AWS_REGION=""
export AWS_ACCESS_KEY_ID=""
export AWS_SECRET_ACCESS_KEY=""
export AWS_SESSION_TOKEN=""

Then run make bootstrap-cluster-dev-aws to bootstrap the cluster!

[!NOTE] If the clusterawsadm bootstrap iam create-cloudformation-stack command errors out with this message :

 the IAM CloudFormation Stack create / update failed and it's currently in a `ROLLBACK_COMPLETE` state

then that means maybe there are pre-existing IAM resources with overlapping name. Then first delete them manually from the AWS Console and then retry running the script. Filter the IAM roles and policies in the corresponding region with the keyword : cluster / clusterapi.

If cluster provisioning gets stuck, then debug by :

• checking logs of ClusterAPI related pod.

• SSHing into the control-plane node. You can view cloud-init output logs stored at /var/log/cloud-init-output.log.

If you want to delete the provisioned cluster, then execute : make delete-provisioned-cluster-dev-aws.

Developer Guide (Running locally)

• Spin up the gitea containe using the docker compose file added in ./e2e/compose/docker-compose.yaml

cd ./e2e/compose
docker compose up -d

• create the general.yaml and secrets.yaml config files in ./outputs/configs/local

touch ./outputs/configs/local/general.yaml
touch ./outputs/configs/local/secrets.yaml

• add the below configs in ./outputs/configs/local/general.yaml and ./outputs/configs/local/secrets.yaml respectively for the bootstrap script to clone the repos and spin up k3d.

forkURLs:
  kubeaid: https://enableitdk-gitea:3001/test/KubeAid
  kubeaidConfig: https://enableitdk-gitea:3001/test/kubeaid-config

cluster:
  name: kubeaid-demo-local
  k8sVersion: v1.31.0
  kubeaidVersion: 10.0.0 # update this accordingly

cloud:
  local: {}

git:
  username: test
  password: password
  caCertPath: /home/ananth/go/src/gitea.obmondo.com/kubeaid-bootstrap-script/certs/custom-rootCA.pem # change this to match your local path

NOTE - The current gitea compose file in ./e2e/compose/ uses custom CA certs added in ./certs. In case you don't want to use the customCA for your local gitea, update the compose file accordingly and keep caCertPath in secrets.yaml empty.

• run the below command to add enableitdk-gitea in your local /etc/hosts

echo "127.0.0.1 enableitdk-gitea" >> /etc/hosts

• Install the necessary pre-requisites

sudo chmod 777 ./scripts/install-prerequisites.sh
./scripts/install-prerequisites.sh

• Now run the script locally

make bootstrap-cluster-local-dev

TODOs

• Check Git URL if SSH agent is used.
• Validation for sshagentauth (should not accept https url).
• Support using HTTPS for ArgoCD apps.
• Support scale to / from zero for the node-groups.

  Currently, I have added extra ClusterRole and ClusterRoleBinding in the KubeAid cluster-autoscaler Helm chart to support this feature. But I have also opened an issue in the kubernetes-sigs/autoscaler repository regarding this : Allow adding extra rules to the Role / ClusterRole template of the Cluster AutoScaler Helm chart.

• recover cluster command

REFERENCES

• Server-Side Apply

• The definitive guide to building Golang based CLI

• AWS S3 Sync Command – Guide with Examples

• How KubeAid backs up Sealed Secrets using a CRONJob : https://github.com/Obmondo/kubeaid/blob/master/argocd-helm-charts/sealed-secrets/templates/configmap.yaml

• Key Management

• Secret Rotation

• Kubernetes Backups, Upgrades, Migrations - with Velero

• Failover

• Auditing

• Kube API server args

• Using IAM roles in management cluster instead of AWS credentials

• KubeadmControlPlane CRD

• How can you call a helm 'helper' template from a subchart with the correct context?

• IRSA for non EKS Clusters | PlatformCon 2023

• Logging in Go with Slog: The Ultimate Guide

• Velero plugins for Microsoft Azure : Use Azure AD Workload Identity

• Azure Workload Identity authentication with AD fails

• Native Routing on Kubernetes with BGP

• Play with Cilium native routing in Kind cluster

• Cilium BGP Control Plane

• Golden config for golangci-lint

• ncdmv's flake.nix file

• Uploading a SARIF file to GitHub

• What is CA bundle?