fdb-kubernetes-operator

Overview

This project provides an operator for managing FoundationDB clusters on Kubernetes.

Running the Operator

To run the operator in your environment, you need to install the controller and the CRDs: Note this will install the latest version from main. For a production setup you should refer to a specific tag.

kubectl apply -f https://raw.githubusercontent.com/FoundationDB/fdb-kubernetes-operator/main/config/crd/bases/apps.foundationdb.org_foundationdbclusters.yaml
kubectl apply -f https://raw.githubusercontent.com/FoundationDB/fdb-kubernetes-operator/main/config/crd/bases/apps.foundationdb.org_foundationdbbackups.yaml
kubectl apply -f https://raw.githubusercontent.com/FoundationDB/fdb-kubernetes-operator/main/config/crd/bases/apps.foundationdb.org_foundationdbrestores.yaml
kubectl apply -f https://raw.githubusercontent.com/foundationdb/fdb-kubernetes-operator/main/config/samples/deployment.yaml

At that point, you can set up a sample cluster:

kubectl apply -f https://raw.githubusercontent.com/foundationdb/fdb-kubernetes-operator/main/config/samples/cluster.yaml

You can see logs from the operator by running kubectl logs -f -l app=fdb-kubernetes-operator-controller-manager --container=manager. To determine whether the reconciliation has completed, you can run kubectl get foundationdbcluster test-cluster. This will show the latest generation of the spec and the last reconciled generation of the spec. Once reconciliation has completed, these values will be the same.

Once the reconciliation is complete, you can run kubectl exec -it test-cluster-log-1 -- fdbcli to open up a CLI on your cluster.

You can also browse the sample directory for more examples of different resource configurations.

For more information about using the operator, including detailed discussion of how to customize your deployments, see the user manual.

For more information on version compatibility, see our compatibility guide.

For more information on the fields you can define on the cluster resource, see the API documentation.

Local Development

Environment Set-up

1. Install Go on your machine, see the Getting Started guide for more information.
2. Install the required dependencies with make deps.
3. Install the foundationDB client package.

Running Locally

To get this controller running in a local Kubernetes cluster:

1. The assumption is that you have local Kubernetes cluster running. Depending on what solution you use some of the following steps might differ.
2. Clone this repository onto your local machine.
3. Run config/test-certs/generate_secrets.bash to set up a secret with self-signed test certs.
4. Run make rebuild-operator to install the operator. By default, the container image is built for the platform where this command is executed. To override the platform, for example, to build an amd64 image on Apple M1, you can set the BUILD_PLATFORM env variable BUILD_PLATFORM="linux/amd64" make rebuild-operator.
5. Run kubectl apply -k ./config/tests/base to create a new FoundationDB cluster with the operator.

Running locally with nerdctl

Instead of Docker you can also use nerdctl to build and push your images. In order to use a different image builder than docker you can use the env variable BUILDER:

# This will use nerdctl for building the image in the k8s.io namespace
export BUILDER='nerdctl -n k8s.io'

You can test your setup with SKIP_TEST=1 make container-build which will build the image locally. After the command successfully finished you can verify with nerdctl -n k8s.io images fdb-kubernetes-operator:latest that the image is available.

Known Limitations

1. Support for backups in the operator is still in development, and there are significant missing features.
2. The unified image is still experimental, and is not recommended outside of development environments.
3. Additional limitations can be found under Warnings.