README
¶
WARNING: Pre-alpha. Do not use this in production. Only for testing.
aws-k8s-tester
https://github.com/kubernetes/enhancements/blob/master/keps/provider-aws/20181126-aws-k8s-tester.md
aws-k8s-tester is a set of utilities and libraries for testing Kubernetes on AWS.
- Uses AWS CloudFormation for resource creation
- Supports automatic rollback and resource deletion
- Flexible add-on support via environmental variables
- Extensible as a Go package;
eks.Tester.Upto create EKS
Install
https://github.com/aws/aws-k8s-tester/releases
aws-k8s-tester ec2
Make sure AWS credential is located in your machine:
# confirm credential is valid
aws sts get-caller-identity --query Arn --output text
See https://github.com/aws/aws-k8s-tester/blob/master/ec2config/README.md for more.
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text);
echo ${ACCOUNT_ID}
cd /tmp
rm -f /tmp/${USER}-test-ec2*
AWS_K8S_TESTER_EC2_ON_FAILURE_DELETE=true \
AWS_K8S_TESTER_EC2_NAME=${USER}-test-ec2 \
AWS_K8S_TESTER_EC2_REGION=us-west-2 \
AWS_K8S_TESTER_EC2_S3_BUCKET_NAME=aws-k8s-tester-ec2-s3-bucket \
AWS_K8S_TESTER_EC2_S3_BUCKET_CREATE=false \
AWS_K8S_TESTER_EC2_REMOTE_ACCESS_KEY_CREATE=true \
AWS_K8S_TESTER_EC2_REMOTE_ACCESS_KEY_NAME=aws-k8s-tester-ec2-key \
AWS_K8S_TESTER_EC2_ASGS_FETCH_LOGS=true \
AWS_K8S_TESTER_EC2_ASGS={\"${USER}-test-ec2-al2-cpu\":{\"name\":\"${USER}-test-ec2-al2-cpu\",\"remote-access-user-name\":\"ec2-user\",\"ami-type\":\"AL2_x86_64\",\"image-id-ssm-parameter\":\"/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2\",\"asg-min-size\":1,\"asg-max-size\":1,\"asg-desired-capacity\":1,\"instance-types\":[\"c5.xlarge\"],\"volume-size\":40},\"${USER}-test-ec2-bottlerocket\":{\"name\":\"${USER}-test-ec2-bottlerocket\",\"remote-access-user-name\":\"ec2-user\",\"ami-type\":\"BOTTLEROCKET_x86_64\",\"image-id-ssm-parameter\":\"/aws/service/bottlerocket/aws-k8s-1.15/x86_64/latest/image_id\",\"ssm-document-name\":\"${USER}InstallBottleRocket\",\"ssm-document-create\":true,\"ssm-document-commands\":\"enable-admin-container\",\"ssm-document-execution-timeout-seconds\":3600,\"asg-min-size\":1,\"asg-max-size\":1,\"asg-desired-capacity\":1,\"instance-types\":[\"c5.xlarge\"],\"volume-size\":40}} \
AWS_K8S_TESTER_EC2_ROLE_CREATE=false \
AWS_K8S_TESTER_EC2_ROLE_ARN=arn:aws:iam::${ACCOUNT_ID}:role/aws-k8s-tester-ec2-role \
AWS_K8S_TESTER_EC2_VPC_CREATE=false \
AWS_K8S_TESTER_EC2_VPC_ID=vpc-00219f2d3063b6d9c \
aws-k8s-tester ec2 create config -p /tmp/${USER}-test-ec2.yaml && cat /tmp/${USER}-test-ec2.yaml
# Or just run
aws-k8s-tester ec2 create config -p /tmp/${USER}-test-ec2.yaml
# to write initial configuration with default values
cd /tmp
aws-k8s-tester ec2 create cluster -p /tmp/${USER}-test-ec2.yaml
cd /tmp
aws-k8s-tester ec2 delete cluster -p /tmp/${USER}-test-ec2.yaml
aws-k8s-tester eks
Make sure AWS credential is located in your machine:
# confirm credential is valid
aws sts get-caller-identity --query Arn --output text
See https://github.com/aws/aws-k8s-tester/blob/master/eksconfig/README.md for more.
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text);
CLUSTER_ARN=arn:aws:eks:us-west-2:${ACCOUNT_ID}:cluster/${USER}-test-eks
echo ${CLUSTER_ARN}
cd /tmp
rm -rf /tmp/${USER}-test-eks*
AWS_K8S_TESTER_EKS_NAME=${USER}-test-eks \
AWS_K8S_TESTER_EKS_REGION=us-west-2 \
AWS_K8S_TESTER_EKS_S3_BUCKET_NAME=aws-k8s-tester-eks-s3-bucket \
AWS_K8S_TESTER_EKS_S3_BUCKET_CREATE=false \
AWS_K8S_TESTER_EKS_REMOTE_ACCESS_KEY_CREATE=false \
AWS_K8S_TESTER_EKS_REMOTE_ACCESS_KEY_NAME=aws-k8s-tester-ec2-key \
AWS_K8S_TESTER_EKS_REMOTE_ACCESS_PRIVATE_KEY_PATH=./aws-k8s-tester-ec2-key.pem \
AWS_K8S_TESTER_EKS_COMMAND_AFTER_CREATE_CLUSTER="aws eks describe-cluster --name ${USER}-test-eks" \
AWS_K8S_TESTER_EKS_COMMAND_AFTER_CREATE_ADD_ONS="aws eks describe-cluster --name ${USER}-test-eks" \
AWS_K8S_TESTER_EKS_PARAMETERS_ENCRYPTION_CMK_CREATE=true \
AWS_K8S_TESTER_EKS_PARAMETERS_ROLE_CREATE=false \
AWS_K8S_TESTER_EKS_PARAMETERS_ROLE_ARN=arn:aws:iam::${ACCOUNT_ID}:role/aws-k8s-tester-eks-role \
AWS_K8S_TESTER_EKS_PARAMETERS_VERSION=1.15 \
AWS_K8S_TESTER_EKS_PARAMETERS_VPC_CREATE=false \
AWS_K8S_TESTER_EKS_PARAMETERS_VPC_ID=vpc-0fd47acb73ace2208 \
AWS_K8S_TESTER_EKS_ADD_ON_NODE_GROUPS_ENABLE=true \
AWS_K8S_TESTER_EKS_ADD_ON_NODE_GROUPS_ROLE_CREATE=false \
AWS_K8S_TESTER_EKS_ADD_ON_NODE_GROUPS_ROLE_ARN=arn:aws:iam::${ACCOUNT_ID}:role/aws-k8s-tester-eks-role \
AWS_K8S_TESTER_EKS_ADD_ON_NODE_GROUPS_ASGS={\"${USER}-test-eks-ng-al2-cpu\":{\"name\":\"${USER}-test-eks-ng-al2-cpu\",\"remote-access-user-name\":\"ec2-user\",\"ami-type\":\"AL2_x86_64\",\"image-id-ssm-parameter\":\"/aws/service/eks/optimized-ami/1.15/amazon-linux-2/recommended/image_id\",\"asg-min-size\":1,\"asg-max-size\":1,\"asg-desired-capacity\":1,\"instance-types\":[\"c5.xlarge\"],\"volume-size\":40},\"${USER}-test-eks-ng-bottlerocket\":{\"name\":\"${USER}-test-eks-ng-bottlerocket\",\"remote-access-user-name\":\"ec2-user\",\"ami-type\":\"BOTTLEROCKET_x86_64\",\"image-id-ssm-parameter\":\"/aws/service/bottlerocket/aws-k8s-1.15/x86_64/latest/image_id\",\"ssm-document-name\":\"${USER}InstallBottleRocket\",\"ssm-document-create\":true,\"ssm-document-commands\":\"enable-admin-container\",\"ssm-document-execution-timeout-seconds\":3600,\"asg-min-size\":1,\"asg-max-size\":1,\"asg-desired-capacity\":1,\"instance-types\":[\"c5.xlarge\"],\"volume-size\":40}} \
AWS_K8S_TESTER_EKS_ADD_ON_MANAGED_NODE_GROUPS_ENABLE=true \
AWS_K8S_TESTER_EKS_ADD_ON_MANAGED_NODE_GROUPS_ROLE_CREATE=false \
AWS_K8S_TESTER_EKS_ADD_ON_MANAGED_NODE_GROUPS_ROLE_ARN=arn:aws:iam::${ACCOUNT_ID}:role/aws-k8s-tester-eks-role \
AWS_K8S_TESTER_EKS_ADD_ON_MANAGED_NODE_GROUPS_MNGS={\"${USER}-test-eks-mng-al2-cpu\":{\"name\":\"${USER}-test-eks-mng-al2-cpu\",\"remote-access-user-name\":\"ec2-user\",\"ami-type\":\"AL2_x86_64\",\"asg-min-size\":1,\"asg-max-size\":1,\"asg-desired-capacity\":1,\"instance-types\":[\"c5.xlarge\"],\"volume-size\":40}} \
AWS_K8S_TESTER_EKS_ADD_ON_NLB_HELLO_WORLD_ENABLE=true \
AWS_K8S_TESTER_EKS_ADD_ON_ALB_2048_ENABLE=true \
AWS_K8S_TESTER_EKS_ADD_ON_JOB_ECHO_ENABLE=true \
AWS_K8S_TESTER_EKS_ADD_ON_JOB_PI_ENABLE=true \
AWS_K8S_TESTER_EKS_ADD_ON_CRON_JOB_ENABLE=true \
aws-k8s-tester eks create config -p /tmp/${USER}-test-eks.yaml && cat /tmp/${USER}-test-eks.yaml
# Or just run
aws-k8s-tester eks create config -p /tmp/${USER}-test-eks.yaml
# to write initial configuration with default values
cd /tmp
aws-k8s-tester eks create cluster -p /tmp/${USER}-test-eks.yaml
cd /tmp
aws-k8s-tester eks delete cluster -p /tmp/${USER}-test-eks.yaml
This will create an EKS cluster with a worker node (takes about 20 minutes).
Once cluster is created, check cluster state using AWS CLI:
aws eks describe-cluster \
--name ${USER}-test-eks \
--query cluster.status
"ACTIVE"
Cluster states are persisted on disk and S3 bucket.
EKS tester uses this file to record status.
cat /tmp/config.yaml
# or
less +FG /tmp/config.yaml
Directories
¶
| Path | Synopsis |
|---|---|
|
client
module
|
|
|
cmd
|
|
|
aws-k8s-tester
command
aws-k8s-tester is a set of AWS test commands.
|
aws-k8s-tester is a set of AWS test commands. |
|
aws-k8s-tester/ec2
Package ec2 implements EC2 related commands.
|
Package ec2 implements EC2 related commands. |
|
aws-k8s-tester/eks
Package eks implements EKS related commands.
|
Package eks implements EKS related commands. |
|
aws-k8s-tester/version
Package version implements version command.
|
Package version implements version command. |
|
e2e
|
|
|
tester
module
|
|
|
e2e2
module
|
|
|
Package ec2 implements testing utilities using EC2.
|
Package ec2 implements testing utilities using EC2. |
|
Package ec2config defines EC2 configuration.
|
Package ec2config defines EC2 configuration. |
|
gen
command
gen generates ec2config documentation.
|
gen generates ec2config documentation. |
|
Package eks implements EKS cluster operations.
|
Package eks implements EKS cluster operations. |
|
alb
Package alb implements ALB plugin that installs 2048.
|
Package alb implements ALB plugin that installs 2048. |
|
appmesh
Package appmesh implements App Mesh add-on.
|
Package appmesh implements App Mesh add-on. |
|
configmaps
Package configmaps implements tester for ConfigMap.
|
Package configmaps implements tester for ConfigMap. |
|
cronjobs
Package cronjobs creates CronJob objects in Kubernetes.
|
Package cronjobs creates CronJob objects in Kubernetes. |
|
csrs
Package csrs implements tester for CertificateSigningRequest.
|
Package csrs implements tester for CertificateSigningRequest. |
|
fargate
Package fargate implements tester for Fargate.
|
Package fargate implements tester for Fargate. |
|
gpu
Package gpu implements GPU plugin.
|
Package gpu implements GPU plugin. |
|
irsa
Package irsa implements tester for IAM Roles for Service Accounts (IRSA).
|
Package irsa implements tester for IAM Roles for Service Accounts (IRSA). |
|
jobs-echo
Package jobsecho creates Job objects in Kubernetes.
|
Package jobsecho creates Job objects in Kubernetes. |
|
jobs-pi
Package jobspi creates example Job objects in Kubernetes.
|
Package jobspi creates example Job objects in Kubernetes. |
|
mng
Package mng defines AWS EKS Managed Node Group configuration.
|
Package mng defines AWS EKS Managed Node Group configuration. |
|
ng
Package ng implements EKS worker nodes with a custom AMI.
|
Package ng implements EKS worker nodes with a custom AMI. |
|
nlb
Package nlb implements NLB plugin.
|
Package nlb implements NLB plugin. |
|
secrets
Package secrets implements Secrets plugin.
|
Package secrets implements Secrets plugin. |
|
Package eksconfig defines EKS test configuration.
|
Package eksconfig defines EKS test configuration. |
|
gen
command
gen generates eksconfig documentation.
|
gen generates eksconfig documentation. |
|
k8s-tester
module
|
|
|
aqua
module
|
|
|
armory
module
|
|
|
cloudwatch-agent
module
|
|
|
clusterloader
module
|
|
|
cmd/k8s-tester
module
|
|
|
cmd/readme-gen
module
|
|
|
cni
module
|
|
|
configmaps
module
|
|
|
conformance
module
|
|
|
csi-ebs
module
|
|
|
csi-efs
module
|
|
|
csrs
module
|
|
|
epsagon
module
|
|
|
falco
module
|
|
|
falcon
module
|
|
|
fluent-bit
module
|
|
|
helm
module
|
|
|
jobs-echo
module
|
|
|
jobs-pi
module
|
|
|
kubecost
module
|
|
|
kubernetes-dashboard
module
|
|
|
metrics-server
module
|
|
|
nlb-guestbook
module
|
|
|
nlb-hello-world
module
|
|
|
php-apache
module
|
|
|
secrets
module
|
|
|
splunk
module
|
|
|
stress
module
|
|
|
sysdig
module
|
|
|
tester
module
|
|
|
vault
module
|
|
|
wordpress
module
|
|
|
kubetest2
module
|
|
|
pkg
|
|
|
aws
Package aws implements wrappers for AWS API calls.
|
Package aws implements wrappers for AWS API calls. |
|
aws/cloudformation
Package cloudformation implements common CloudFormation utilities.
|
Package cloudformation implements common CloudFormation utilities. |
|
aws/ec2
Package ec2 implements various AWS EC2 operations.
|
Package ec2 implements various AWS EC2 operations. |
|
aws/ec2/instance-types
command
instance-types auto-generates EC2 instance types from AWS API.
|
instance-types auto-generates EC2 instance types from AWS API. |
|
aws/ec2/metadata
Package metadata queries the EC2 metadata.
|
Package metadata queries the EC2 metadata. |
|
aws/elb
Package elb implements ELB utilities.
|
Package elb implements ELB utilities. |
|
aws/iam
Package iam implements various IAM components.
|
Package iam implements various IAM components. |
|
ctxhandler
Package ctxhandler implements context handler.
|
Package ctxhandler implements context handler. |
|
fileutil
Package fileutil implements file utilities.
|
Package fileutil implements file utilities. |
|
github
Package github implements github utilities.
|
Package github implements github utilities. |
|
httputil
Package httputil implements various HTTP operations.
|
Package httputil implements various HTTP operations. |
|
k8s-client
Package k8sclient implements various k8s utils.
|
Package k8sclient implements various k8s utils. |
|
logutil
Package logutil implements various log utilities.
|
Package logutil implements various log utilities. |
|
Package ssh implements various SSH commands.
|
Package ssh implements various SSH commands. |
|
utils
module
|
|
|
Package version defines aws-k8s-tester version.
|
Package version defines aws-k8s-tester version. |
Click to show internal directories.
Click to hide internal directories.