kubeaid-bootstrap-script

KubeAid Bootstrap Script

The KubeAid Bootstrap Script is used to bootstrap Kubernetes clusters using Cluster API and KubeAid.

Currently it only supports bootstrapping self-managed clusters in AWS.

Official Guides

• Bootstrapping a self-managed cluster in AWS

Developer Guide (AWS edition)

Make sure, you've Docker installed and running in your system.

Run make build-image-dev to build the KubeAid Bootstrap Script container image (development version).

Then run make run-container-dev to run the container.

Use make exec-container-dev to execute into the container.

Once you're inside the container, use make generate-sample-config-aws-dev to generate a sample config file at ./outputs/kubeaid-bootstrap-script.config.yaml, targetting the AWS cloud provider. Adjust the config file according to your needs.

Export your AWS credentials as environment variables like such :

export AWS_REGION=""
export AWS_ACCESS_KEY_ID=""
export AWS_SECRET_ACCESS_KEY=""
export AWS_SESSION_TOKEN=""

Then run make bootstrap-cluster-dev-aws to bootstrap the cluster!

[!NOTE] If the clusterawsadm bootstrap iam create-cloudformation-stack command errors out with this message :

 the IAM CloudFormation Stack create / update failed and it's currently in a `ROLLBACK_COMPLETE` state

then that means maybe there are pre-existing IAM resources with overlapping name. Then first delete them manually from the AWS Console and then retry running the script. Filter the IAM roles and policies in the corresponding region with the keyword : cluster / clusterapi.

If cluster provisioning gets stuck, then debug by :

• checking logs of ClusterAPI related pod.

• SSHing into the control-plane node. You can view cloud-init output logs stored at /var/log/cloud-init-output.log.

If you want to delete the provisioned cluster, then execute : make delete-provisioned-cluster-dev-aws.

TODOs

• Check Git URL if SSH agent is used.
• Validation for sshagentauth (should not accept https url).
• --debug flag to print command execution outputs.
• Support adding multiple SSH keys via config file.
• Support using HTTPS for ArgoCD apps.
• Support enabling Audit Logging.
• Switch to IAM Role from (temporary) credentials after cluster bootstrap.
• ETCD metrics enabled.
• Support scale to / from zero for the node-groups.

  Currently, I have added extra ClusterRole and ClusterRoleBinding in the KubeAid cluster-autoscaler Helm chart to support this feature. But I have also opened an issue in the kubernetes-sigs/autoscaler repository regarding this : Allow adding extra rules to the Role / ClusterRole template of the Cluster AutoScaler Helm chart.

• In case of AWS, pick up AWS credentials from ~/.aws/credentials (if present).
• recover cluster command

REFERENCES

• Server-Side Apply

• The definitive guide to building Golang based CLI

• AWS S3 Sync Command – Guide with Examples

• How KubeAid backs up Sealed Secrets using a CRONJob : https://github.com/Obmondo/kubeaid/blob/master/argocd-helm-charts/sealed-secrets/templates/configmap.yaml

• Key Management

• Secret Rotation

• Kubernetes Backups, Upgrades, Migrations - with Velero

• Failover

• Auditing

• Kube API server args

• Using IAM roles in management cluster instead of AWS credentials

• KubeadmControlPlane CRD

• How can you call a helm 'helper' template from a subchart with the correct context?

• IRSA for non EKS Clusters | PlatformCon 2023

• Logging in Go with Slog: The Ultimate Guide