unit-operator

Unit Operator

Unit Operator - Distributed Workload Operator for Kubernetes

Manage UnitSet, Unit, and GrpcCall resources with built-in high availability, scaling, and lifecycle management capabilities

✨ Features

• 🎯 UnitSet Management: Manages sets of distributed units with shared configuration
• 📦 Unit Lifecycle: Individual workload instance management with sidecar agents
• 📞 gRPC Operations: Manages gRPC-based operations between units
• 🛡️ High Availability: Built-in replication and failover mechanisms
• 📈 Scaling: Horizontal and vertical scaling capabilities
• 🔄 Lifecycle Management: Automated backup, recovery, and upgrades
• ⚙️ Configuration Management: Template-based configuration with shared configs
• 📊 Monitoring: Integrated metrics and health checks
• 🔐 Security: Certificate management and secure credential handling
• 🧩 Compose Operator Integration: Optional integration with Compose Operator for database replication/topologies (MySQL, Redis, PostgreSQL, ProxySQL)

🏗️ Architecture

The Unit Operator follows a two-layer architecture:

┌─────────────────────────────────────┐
│           UnitSet                  │
│      (Cluster Manager)             │
├─────────────────────────────────────┤
│  ┌─────────────┐  ┌─────────────┐  │
│  │   Unit-0    │  │   Unit-1    │  │
│  │ (Pod+Agent) │  │ (Pod+Agent) │  │
│  └─────────────┘  └─────────────┘  │
│                                     │
│  ┌─────────────┐  ┌─────────────┐  │
│  │   Unit-2    │  │    ...      │  │
│  │ (Pod+Agent) │  │             │  │
│  └─────────────┘  └─────────────┘  │
└─────────────────────────────────────┘

• 🎯 UnitSet: Manages a set of distributed units with shared configuration
• 📦 Unit: Individual workload instances with sidecar agents for advanced operations
• 📞 GrpcCall: Manages gRPC-based operations between units
• 🤖 Agent: Sidecar container providing unit-specific operations and configuration management

📋 Prerequisites

• ☸️ Kubernetes: v1.27+ or OpenShift v4.6+
• 🐹 Go: 1.24+ (for development)
• ⚓ Helm: 3.0+ (for deployment)

🚀 Quick Start

📦 Installation

1. Install using Helm:

# Add the Helm repository
helm repo add upm-charts https://upmio.github.io/helm-charts

# Install the operator
helm install unit-operator --namespace upm-system --create-namespace \
  --set crd.enabled=true \
  upm-charts/unit-operator

1. (Optional) Install Compose Operator (for database replication/topologies):

   • Repository and docs: https://github.com/upmio/compose-operator
   • Unit Operator runs independently; install Compose Operator only if you need MySQL/Redis/PostgreSQL/ProxySQL replication/cluster topologies.

2. Install CRDs manually (recommended for production):

kubectl apply -f config/crd/bases/

🐳 Example: Deploy UnitSet with gRPC Communication

# Create shared configuration
apiVersion: v1
kind: ConfigMap
metadata:
  name: unitset-config
  namespace: default
data:
  service_group_uid: "6fa3ca2a-0ffd-4ca7-8615-e2589f7dd413"
  mysql_ports: '[{"name": "grpc", "containerPort": "50051","protocol": "TCP"}]'

# Deploy UnitSet
apiVersion: upm.syntropycloud.io/v1alpha2
kind: UnitSet
metadata:
  name: example-unitset
  namespace: default
spec:
  type: mysql
  version: "1.0.0"
  units: 3
  sharedConfigName: unitset-config
  resources:
    limits:
      cpu: "1"
      memory: 2Gi
    requests:
      cpu: "500m"
      memory: 1Gi
  storages:
    - name: data
      mountPath: /DATA_MOUNT
      size: 10Gi
      storageClassName: standard
  env:
    - name: POD_NAMESPACE
      valueFrom:
        fieldRef:
          fieldPath: metadata.namespace
    - name: POD_NAME
      valueFrom:
        fieldRef:
          fieldPath: metadata.name

📞 Example: Create gRPC Call

apiVersion: upm.syntropycloud.io/v1alpha1
kind: GrpcCall
metadata:
  name: example-grpc-call
  namespace: default
spec:
  targetUnit: "example-unitset-0"
  type: mysql
  action: set-variable
  ttlSecondsAfterFinished: 600
  parameters:
    variables:
      ping: "true"

✅ Verify Deployment

# Check UnitSet status
kubectl get unitset example-unitset

# Check individual units
kubectl get units

# Check pod status
kubectl get pods -l app=example-unitset

⚙️ Configuration

🎯 Supported Resource Types

📞 gRPC Communication

The operator supports gRPC communication between units:

• Service discovery and registration
• Health checks and monitoring
• Configuration synchronization
• Cross-unit communication

💾 Storage Configuration

# Persistent storage
storages:
  - name: data
    mountPath: /DATA_MOUNT
    size: 10Gi
    storageClassName: standard

# Temporary storage
emptyDir:
  - name: temp
    mountPath: /TEMP_MOUNT
    size: 1Gi

🔄 Update Strategy

updateStrategy:
  type: RollingUpdate
  rollingUpdate:
    maxUnavailable: 1
    partition: 0

💻 Development

🛠️ Setup Development Environment

# Clone the repository
git clone https://github.com/upmio/unit-operator.git
cd unit-operator

# Install dependencies
go mod download

# Install required tools
make kustomize controller-gen envtest
# Optional (recommended for packaging/linting)
make operator-sdk golangci-lint

🏗️ Build and Run

# Build the operator
make build

# Run locally
make run

# Run tests
make test

# Run with coverage
make check-coverage

🔧 Code Generation

# Generate CRDs and manifests
make manifests

# Generate deepcopy methods
make generate

# Generate protobuf code
make pb-gen

📚 API Reference

The Unit Operator provides the following custom resources:

• 🎯 UnitSet: Manages a cluster of database instances
• 📦 Unit: Individual database instance
• 📞 GrpcCall: gRPC-based operations

🧩 Compose Operator Integration (optional)

Unit Operator orchestrates workloads around UnitSet/Unit (Pods, PVCs, Services, rolling upgrades, agent coordination). If you need database replication/topologies, install and use Compose Operator alongside.

• Use UnitSet for distributed layout, shared configuration, and lifecycle
• Use Compose Operator CRDs for database replication/cluster topologies, for example:
  • MysqlReplication: MySQL async/semi-sync/Group Replication
  • PostgresReplication: PostgreSQL streaming replication
  • RedisReplication / RedisCluster: Redis replication/cluster
  • ProxySQLSync: ProxySQL administration and synchronization
• Use GrpcCall for agent-driven operational tasks across units

Note: Unit Operator works independently; install Compose Operator only if you need these replication capabilities.

🔐 Compose Operator password encryption example (optional)

When using Compose Operator database CRDs, credentials must be AES-256-CTR encrypted and stored in Secrets. Example workflow below (commands from the Compose Operator docs; build the tool in its repository):

# Build the encryption tool in the compose-operator repository
go build -o aes-tool ./tool/

# Get the AES key generated by compose-operator (replace namespace/release accordingly)
AES_KEY=$(kubectl get secret compose-operator-aes-secret -n upm-system -o jsonpath='{.data.AES_SECRET_KEY}' | base64 -d)

# MySQL example
aes-tool -key "$AES_KEY" -plaintext "mysql_root_password" -username "mysql"
aes-tool -key "$AES_KEY" -plaintext "replication_password" -username "replication"
kubectl create secret generic mysql-credentials \
  --from-file=mysql=mysql.bin \
  --from-file=replication=replication.bin

# Redis example
aes-tool -key "$AES_KEY" -plaintext "redis_auth_password" -username "redis"
kubectl create secret generic redis-credentials \
  --from-file=redis=redis.bin

# PostgreSQL example
aes-tool -key "$AES_KEY" -plaintext "postgresql_admin_password" -username "postgresql"
aes-tool -key "$AES_KEY" -plaintext "replication_password" -username "replication"
kubectl create secret generic postgres-credentials \
  --from-file=postgresql=postgresql.bin \
  --from-file=replication=replication.bin

# ProxySQL example
aes-tool -key "$AES_KEY" -plaintext "proxysql_admin_password" -username "proxysql"
aes-tool -key "$AES_KEY" -plaintext "mysql_backend_password" -username "mysql"
kubectl create secret generic proxysql-credentials \
  --from-file=proxysql=proxysql.bin \
  --from-file=mysql=mysql.bin

Note: The capabilities and tooling above come from the Compose Operator project. Refer to its latest documentation: https://github.com/upmio/compose-operator.

📊 Monitoring

The operator exposes metrics on port 20154 by default:

# Access metrics
kubectl port-forward -n upm-system svc/unit-operator-metrics 20154:20154
curl http://localhost:20154/metrics

Monitor your database clusters with built-in metrics

🚨 Troubleshooting

⚠️ Common Issues

1. Pods stuck in Pending state

   • Check resource requests/limits
   • Verify storage class availability
   • Ensure sufficient cluster resources

2. Replication not working

   • Verify network connectivity between pods
   • Check credential secrets
   • Review replication configuration

3. Upgrade failures

   • Check operator logs for errors
   • Verify upgrade strategy configuration
   • Ensure sufficient disk space

🔍 Debug Commands

# Check operator logs
kubectl logs -n upm-system deployment/unit-operator

# Check UnitSet events
kubectl describe unitset <name>

# Check individual Unit status
kubectl describe unit <name>

# Check agent logs
kubectl logs <pod-name> -c agent

🤝 Contributing

We welcome contributions! Please see our CONTRIBUTING.md for details.

🔄 Development Workflow

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests for new functionality
5. Run linting and tests
6. Submit a pull request

🎨 Code Style

• Follow Go standard formatting
• Use make fmt and make vet before committing
• Ensure tests pass with make test
• Maintain test coverage above threshold

📄 License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

🆘 Support

• 📚 Documentation
• 🐛 Issues
• 💬 Discussions

🙏 Acknowledgments

• 🏗️ Kubebuilder - Kubernetes API development framework
• 🎛️ controller-runtime - Kubernetes controller framework
• 🐘 Zalando Postgres Operator - Inspiration for PostgreSQL management
• 🐬 Presslabs MySQL Operator - Inspiration for MySQL management