container-app-operator

container-app-operator

The container-app-operator is an operator that reconciles Capp CRs.

Capp (or ContainerApp) provides a higher-level abstraction for deploying containerized workload, making it easier for end-users to deploy workloads on Kubernetes without being knowledgeable in Kubernetes concepts, while adhering to the standards required by the infrastructure and platform teams without any extra burden on the users.

The operator uses open-source projects, such as Knative Serving, logging-operator and nfspvc-operator to create an abstraction for containerized workloads.

Run Container Service

The container-app-operator project can work as a standalone solution, but is mostly used together with the rcs-ocm-deployer project, which allows deploying Capp workloads in a multi-cluster set-up, using the OCM (Open Cluster Management) open-source project.

High Level Architecture

1. The capp controller reconciles the Capp CRs in the cluster and creates (if needed) a Knative Service (ksvc) CR, a DommainMapping CR, and Flow & Output CRs for every Capp.

2. The knative controller reconciles the ksvc CRs in the cluster and controls the lifecycle an autoscaler and pods relevant to the ksvc.

3. The logging-operator controller reconciles the Flow and Output CRs in the cluster and collects logs from the pods' stdout and sends them to a pre-existing Elasticsearch or Splunk index.

Feature Highlights

• Support for autoscaler (HPA or KPA) according to the chosen scaleMetric (concurrency, rps, cpu, memory) with default settings.
• Support for HTTP/HTTPS DomainMapping for accessing applications via Ingress/Route.
• Support for all Knative Serving configurations.
• Support for exporting logs to Elasticsearch and Splunk indexes.
• Support for changing the state of Capp from enabled (workload is in running state) to disabled (workload is not in running state).
• Support for external NFS storage connected to Capp by using volumeMounts.

Getting Started

Prerequisites

1. A Kubernetes cluster (you can use KinD).

2. Knative Serving installed on the cluster (you can use the quickstart)

3. Logging Operator installed on the cluster (you can use the Helm Chart)

Knative Serving and Logging Operator can also be installed by running:

$ make prereq

Deploying the controller

$ make deploy IMG=ghcr.io/dana-team/container-app-operator:<release>

Build your own image

$ make docker-build docker-push IMG=<registry>/container-app-operator:<tag>

Change target autoscaler default values

To change the target values a configMap with the name autoscale-default in the namespace capp-operator-system needs to be created.

The configMap should contain the scale metric types as keys and for the value the desired target values.

The configMap will affect the ksvc autoscale target value annotation autoscaling.knative.dev/target.

Example

kind: ConfigMap
apiVersion: v1
metadata:
  name: autoscale-defaults
  namespace: capp-operator-system
data:
  rps: "200"
  cpu: "80"
  memory: "70"
  concurrency: "10"

Enable Persistent Volume extension in Knative

In order to use volumeMounts in Capp, Knative Serving needs to be configured to support volumes. This is done by adding the following lines to the ConfigMap of name config-features in the Knative Serving namespace:

kubernetes.podspec-persistent-volume-claim: enabled
kubernetes.podspec-persistent-volume-write: enabled

It's possible to use the following one-liner:

$ kubectl patch --namespace knative-serving configmap/config-features --type merge --patch '{"data":{"kubernetes.podspec-persistent-volume-claim": "enabled", "kubernetes.podspec-persistent-volume-write": "enabled"}}'

Example Capp

apiVersion: rcs.dana.io/v1alpha1
kind: Capp
metadata:
  name: capp-sample
  namespace: capp-sample
spec:
  configurationSpec:
    template:
      spec:
        containers:
          - env:
              - name: APP_NAME
                value: capp-env-var
            image: 'quay.io/danateamorg/example-python-app:v1-flask'
            name: capp-sample
            volumeMounts:
              - name: testpvc
                mountPath: /data
        volumes:
          - name: testpvc
            persistentVolumeClaim:
              claimName: nfspvc
              readOnly: false
  routeSpec:
    hostname: capp.dev
    tlsEnabled: true
    tlsSecret: cappTlsSecretName
  volumesSpec:
    nfsVolumes:
      - server: test
        path: /test
        name: nfspvc
        capacity:
          storage: 200Gi
  logSpec:
    type: elastic
    host: 10.11.12.13
    index: main
    username: elastic
    passwordSecretName: es-elastic-user
    sslVerify: false
  scaleMetric: concurrency
  state: enabled