cluster-operator

StorageOS cluster-operator

The StorageOS Cluster Operator deploys and configures a StorageOS cluster on Kubernetes.

For quick installation of cluster operator, use the cluster operator helm chart.

Pre-requisites

• Kubernetes 1.9+
• Kubernetes must be configured to allow (configured by default in 1.10+):
  • Privileged mode containers (enabled by default)
  • Feature gate: MountPropagation=true. This can be done by appending --feature-gates MountPropagation=true to the kube-apiserver and kubelet services.

Refer to the StorageOS prerequisites docs for more information.

Setup/Development:

1. Install operator-sdk v0.0.6. v0.0.7 has breaking changes. Use v0.0.6 for now.
2. Run operator-sdk generate k8s if there's a change in api type.
3. Build operator container with operator-sdk build storageos/cluster-operator:<tag>
4. Apply the manifests in deploy/ to install the operator
   • Apply rbac.yaml to grant all the permissions
   • Apply crd.yaml to define the custom resources.
   • Apply operator.yaml to install the operator. Change the container image in this file when installing a new operator.
   • Apply cr.yaml to create a StorageOSCluster custom resource.

NOTE: Installing StorageOS on Minikube is not currently supported due to missing kernel prerequisites.

For development, run the operator outside of the k8s cluster by running:

$ OPERATOR_NAME=cluster-operator operator-sdk up local

Pass kubeconfig:

OPERATOR_NAME=cluster-operator operator-sdk up local --kubeconfig=/path/to/kubeconfig

After creating a resource, query the resource:

$ kubectl get storageoscluster
NAME                READY     STATUS    AGE
example-storageos   3/3       Running   4m

Inspect a StorageOSCluster Resource

Get all the details about the cluster:

$ kubectl describe storageoscluster/example-storageos
Name:         example-storageos
Namespace:    default
Labels:       <none>
Annotations:  kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"storageos.com/v1alpha1","kind":"StorageOSCluster","metadata":{"annotations":{},"name":"example-storageos","namespace":"default"},"spec":{"...
API Version:  storageos.com/v1alpha1
Kind:         StorageOSCluster
Metadata:
  Creation Timestamp:  2018-07-21T12:57:11Z
  Generation:          1
  Resource Version:    10939030
  Self Link:           /apis/storageos.com/v1alpha1/namespaces/default/storageosclusters/example-storageos
  UID:                 955b24a4-8ce5-11e8-956a-1866da35eee2
Spec:
  Join:  test07
Status:
  Node Health Status:
  ...
  ...
  Nodes:
    test09
    test08
    test07
  Phase:  Running
  Ready:  3/3
Events:   <none>

StorageOSCluster Resource Configuration

Once the StorageOS operator is running, a StorageOS cluster can be deployed by creating a Cluster Configuration. The parameters specified in the configuration will define how StorageOS is deployed, the rest of the installation details are handled by the operator.

The following tables lists the configurable spec parameters of the StorageOSCluster custom resource and their default values.

TLS Support

To enable TLS, ensure that an ingress controller is installed in the cluster. Set ingress.enable and ingress.tls to true. Store the TLS cert and key as part of the storageos secret as:

apiVersion: v1
kind: Secret
metadata:
  name: "storageos-api"
...
...
data:
  # echo -n '<secret>' | base64
  ...
  ...
  # Add base64 encoded TLS cert and key.
  tls.crt:
  tls.key:

CSI

StorageOS also supports the Container Storage Interface (CSI) to communicate with Kubernetes. Only versions 1.10+ are supported. CSI ensures forward compatibility with future releases of Kubernetes, as vendor-specific drivers will soon be deprecated from Kubernetes. However, some functionalities are not supported yet.

To enable CSI, set csi.enable to true in the StorageOSCluster resource config.

apiVersion: "storageos.com/v1alpha1"
kind: "StorageOSCluster"
metadata:
  name: "example-storageos"
  namespace: "default"
spec:
  secretRefName: "storageos-api"
  secretRefNamespace: "default"
  csi:
    enable: true

CSI Credentials

To enable CSI Credentials, ensure that CSI is enabled by setting csi.enable to true. Based on the type of credentials to enable, set the csi fields to true:

apiVersion: "storageos.com/v1alpha1"
kind: "StorageOSCluster"
metadata:
  name: "example-storageos"
  namespace: "default"
spec:
  ...
  ...
  csi:
    enable: true
    enableProvisionCreds: true
    enableControllerPublishCreds: true
    enableNodePublishCreds: true
  ...

Specify the CSI credentials as part of the storageos secret object as:

apiVersion: v1
kind: Secret
metadata:
  name: "storageos-api"
...
...
data:
  # echo -n '<secret>' | base64
  ...
  ...
  csiProvisionUsername:
  csiProvisionPassword:
  csiControllerPublishUsername:
  csiControllerPublishPassword:
  csiNodePublishUsername:
  csiNodePublishPassword: