Skip to main content
< All Topics

Kubernetes Resource Operators

Posted
Updated
Author -Rajkumar Aute
Views77

Kubernetes Controllers vs. Operators: Moving from Automation to Intelligent Orchestration

Think of Kubernetes as a high-tech factory. Controllers are the automated assembly lines that do standard tasks, like making sure there are always five boxes on a shelf. Operators are the expert human supervisors you hire to handle complex machines (like a database). They don’t just count boxes; they know how to fix the machine if it leaks, back up the data, and tune the engine for better performance.

Imagine a Thermostat vs. a Smart Home Manager.

  • The Controller (Thermostat): You set it to 22°C. If the room gets cold, it turns on the heater. It only knows “Temperature.” It doesn’t care if the window is open or if you are on vacation.
  • The Operator (Smart Home Manager): It knows it’s winter, sees the window is open, sends you an alert, closes the smart blinds to save energy, and coordinates the heater and the AC to ensure the pipes don’t freeze. It has Context and Domain Knowledge.

A Controller handles the “standard” stuff, while an Operator handles the “custom, intelligent” stuff using specific domain knowledge.

Key Characteristics to Remember
  • Controllers look after standard Kubernetes resources (Pods, Deployments).
  • Operators look after Custom Resources (CRDs) and complex applications.
  • All Operators are Controllers, but not all Controllers are Operators.
  • Controllers = Native Kubernetes logic.
  • Operators = Native logic + Application-specific Knowledge (Domain logic).
FeatureStandard ControllerResource Operator
FocusKubernetes Native Objects (Pods, Services).Application Specific Logic (DBs, Queues).
KnowledgeGeneric (“I need 3 replicas”).Deep (“I need to upgrade the primary DB first”).
FlexibilityRigid structure.Highly customizable via code (Go/Ansible/Helm).
InstallationBuilt-in to K8s.Installed separately (via Helm or OLM).
ExampleReplicaSet ensuring 3 pods exist.Prometheus Operator managing monitoring config.

Architect’s Note: Why not just use a StatefulSet? A StatefulSet handles pod identity and ordered startup, but an Operator handles application state. For example, a StatefulSet can restart a crashed database pod, but an Operator knows how to force that pod to rejoin the database cluster, resync its data, or trigger a backup before a version upgrade.

The Operator

A Kubernetes Resource Operator is a method of packaging, deploying, and managing a Kubernetes application. It is an application-specific controller that extends the functionality of the Kubernetes API to create, configure, and manage instances of complex applications on behalf of a Kubernetes user.

It builds upon the basic Kubernetes resource and controller concepts but adds domain or application-specific knowledge to automate the entire lifecycle of the software it manages.

How it works:

  1. CRD (Custom Resource Definition): You define a new type of resource, say PostgreSQL.
  2. CR (Custom Resource): You create an instance of it: kind: PostgreSQL, name: my-db.
  3. Operator Controller: The Operator sees this new my-db resource. It reads the specs (version, storage size, password) and creates the necessary Pods, Services, and Persistent Volumes in the correct order.

https://operatorhub.io

DevSecOps Architect Level Notes

For the architects designing the platform, you need to look at the Operator Capability Levels:

  1. Basic Install: Just sets up the app (Like a Helm chart).
  2. Seamless Upgrades: Handles patch and minor version upgrades automatically.
  3. Full Lifecycle: Handles backup, failure recovery, and storage resizing.
  4. Deep Insights: Provides metrics, logs, and alerts natively.
  5. Auto Pilot: Horizontal/Vertical scaling and auto-tuning based on load.

Tools for Building Operators:

Key Components
  1. The CRD: The YAML schema that defines your app settings.
  2. The Controller: The binary running in a pod that enforces the logic.
  3. The Reconcile Loop: The function that runs constantly (Current State vs Desired State).
  4. RBAC Roles: The permissions the Operator needs to touch other resources.
Use Cases
  • Stateful Services: Databases (Postgres, MySQL), Caching (Redis), Message Queues (Kafka).
  • Observability: Prometheus, Grafana, ELK Stack (managing configs dynamically).
  • GitOps: ArgoCD Operator (managing the deployment tool itself).
Benefits
  • “SRE in a Box”: Encapsulates the wisdom of a Site Reliability Engineer into code.
  • Consistency: Eliminates “snowflake” configurations. Every database is deployed exactly the same way.
  • Self-Healing: Detects configuration drift and fixes it automatically.

Challenges:

  • Security Risks (The DevSecOps Warning): Operators are powerful. If an attacker compromises an Operator, they often get cluster-admin access or access to all secrets in a namespace.
    • Solution: Always enforce Least Privilege RBAC. Don’t give an Operator access to the whole cluster if it only needs one namespace.
  • Resource Consumption: Each Operator is a running Pod. If you install 50 Operators, that is 50 pods just for management, eating up CPU/RAM.
  • Complexity Overload: Writing a good Operator requires deep knowledge of Kubernetes internals (Events, Informers, Caches).

Common Issues:

1. Security Risks (The DevSecOps Warning)

Operators are powerful. If an attacker compromises an Operator, they often get broad access.

  • Solution: Always enforce Least Privilege RBAC. Avoid giving an Operator a ClusterRole (access to the whole cluster) if it only needs a Role (access to a single namespace).

2. Resource Consumption

Each Operator is a running Pod containing a binary that constantly polls the Kubernetes API server. Installing 50 Operators means 50 management pods eating up CPU, RAM, and generating API overhead.

3. Stuck Deletions (The Finalizer Trap)

If an Operator’s “Finalizer” logic is broken or the Operator pod dies, you cannot delete the custom resource. It stays in the Terminating state forever.

  • The Fix: Manually patch the resource to remove the finalizer:
Bash
kubectl patch <crd-type> <name> -p '{"metadata":{"finalizers":null}}' --type=merge

Lab: Deploy the Prometheus Operator (The “Hello World” of Ops)

Goal: We will not write code; we will use a Resource Operator to see its magic.

1: Install the Operator Bundle: We will use the kube-prometheus-stack which installs the Operator.

Bash
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
helm install my-monitoring prometheus-community/kube-prometheus-stack

2: Verify the CRDs: Check what new “Resources” your cluster understands now.

Bash
kubectl get crds
# You will see things like: alertmanagers.monitoring.coreos.com, prometheuses.monitoring.coreos.com

3: The Magic (Create a ServiceMonitor): Normally, to monitor a new app, you have to edit the Prometheus config file and restart it. Not anymore!

Just create a ServiceMonitor resource (a Custom Resource).

Bash
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: my-app-monitor
  labels:
    release: my-monitoring
spec:
  selector:
    matchLabels:
      app: my-app
  endpoints:
  - port: web

Observation: The Prometheus Operator will detect this new file and automatically reconfigure Prometheus to start scraping your app. No manual config edits needed!

Contents
Scroll to Top