operator

odigos-operator

The Odigos Operator is another installation, management, and upgrade method for installing Odigos in a Kubernetes cluster.

Currently, the Operator is designed specifically for OpenShift users. It may not function as expected in other Kubernetes clusters.

Overview

The main function of the Operator is to install, manage, upgrade, and uninstall Odigos in a cluster.

The operator uses a CRD called operator.odigos.io/odigos to represent a single installation of Odigos. This CRD essentially contains all of the system config options available through the Odigos configmap.

When an Odigos CRD object is created or updated, the Operator resolves it in internal/controller/odigos_controller.go, using the same packages used by the CLI installer (ie, resource managers). The operator makes each installed resource have an OwnerReference pointing to that CRD object. It also adds a finalizer to the Odigos object to handle uninstalling.

When an Odigos object is deleted, the finalizer allows the operator to uninstall Odigos. Note that this means deleting the operator without first deleting the Odigos object may leave the Odigos install hanging on that finalizer.

Operator Framework SDK

The operator was set up using the Operator Framework operator-sdk for Go, and many of the Makefile commands provided are scaffolded by that tool. The operator-sdk tool and patterns should be used with this operator. This tool is essentially a wrapper around Kubebuilder with some added functionality (mainly around building and packaging), so it shares much of the support for Kubebuilder.

Manifests, CSV, and Bundle

Operator manifests are defined under the config/ directory as Kustomize files. Many of these were scaffolded by default with operator-sdk and are open for review.

When packaging the operator for distribution on OpenShift, two things are created: the ClusterServiceVersion (CSV) and Bundle.

The CSV is a CoreOS/Operator Framework concept that defines metadata about the operator, its dependent images, CRDs, manifests, etc. It is essentially a single YAML file that allows the Operator to be installed and managed by the Operator Lifecycle Manager (OLM).

The base CSV file is at config/manifests/bases/odigos-operator.clusterserviceversion.yaml. This file can be edited to update fixed metadata like description, keywords, maintainer info, etc.

Running make bundle uses the base CSV and all other manifest Kustomize files to generate the final CSV in bundle/manifests/odigos-operator.clusterserviceversion.yaml. This final CSV is what is distributed to OperatorHub on OpenShift (see DEVELOPMENT.md).

Versioning

Currently, the operator is pinned to the same version of Odigos. Meaning v1.0.205 of the Operator will only install v1.0.205 of Odigos. To upgrade Odigos through the Operator, you only need to update the Operator itself. When the new operator pod spins up, it will re-list any existing odigos objects and follow the same upgrade path as the CLI.

The version is pinned through a ConfigMap, which is generated in config/manager/kustomization.yaml with a configMapGenerator directive. This allows the version to be generated as part of the release flow through the Makefile. The generated ConfigMap is automatically referenced by the operator pod generated by Kustomize when following the release flow in DEVELOPMENT.md. This isn't ideal, and leads to a new ConfigMap object being added to the bundle with each release, but Kustomize doesn't seem to support simpler methods like environment variable templating.

Permissions

The Odigos Operator requires all RBAC permissions used by any Odigos components, because the Operator is installing these components and granting them their permissions. These are currently all granted as auto-generated ClusterRoles using //+kubebuilder markers in odigos_controller.go.

If new RBAC permissions are granted to a component, but not the operator, then the operator will fail to install Odigos due to lacking those permissions. To help prevent this, we have a CI check which fails if changes are made to any component RBAC without any changes to odigos_controller.go. This is a simple approach, and it's possible that RBAC changes to components don't need an update to the operator (for example, if the operator already has that permission, or if a permission was removed.)

If you are sure that the operator does not need an RBAC update, you can add the operator-rbac-approved label to your PR to bypass this check. The check is not required however.

Otherwise, to add new permissions to the operator, update the //+kubebuilder markers in odigos_controller.go and run USE_IMAGE_DIGESTS=true make generate manifests bundle from this directory. This will update the operator manifests and generate a new operator bundle (which may include other irrelevant generated changes such as image references and timestamps -- this is normal).

Why ClusterRoles?

In the interest of generating as much code as possible, we use kubebuilder markers to define operator permissions. These markers support generating namespaced Roles, but the namespace must be coded into the marker to do so. Since the operator can be installed in any namespace, this makes it currently infeasible to generate namespaced Roles. This is a TODO to be improved.

Building and development

See DEVELOPMENT.md