Skip to main content
< All Topics

Kubernetes LimitRange

Posted
Updated
Author -Rajkumar Aute
Views83
Sizing Your Pods Just Right for Production Environments

If you don’t set strict rules, a single container can act like a “noisy neighbor,” eating up all the CPU and Memory on a node. This can crash your other critical applications and bring down your entire production environment.

While ResourceQuotas set limits for the entire Namespace (the whole team’s budget), LimitRanges are the specific, detailed rules for individual Pods and Containers (the individual player). They ensure every single container is sized perfectly not too small to fail, and not too big to waste expensive cloud resources. Best of all, if a developer forgets to set a CPU or Memory request in their YAML file, the LimitRange automatically steps in and applies a safe, default setting.

Think of a Kubernetes Namespace as a Shared Office Cafeteria.

  • ResourceQuota is the budget for the entire department (e.g., “The Engineering team gets 500 plates of food total per day”).
  • LimitRange is the strict rule for the individual plate you carry:
    • Min: You must take at least 1 spoon of rice (so you don’t starve and crash the system).
    • Max: You cannot take more than 2 full plates at a time (so you don’t leave others hungry).
    • Default: If you don’t say what you want at the counter, the staff automatically gives you a standard “Thali” (Standard Default Request/Limit).

Quick Reference
  • The Golden Rule: LimitRange governs the individual Pod/Container; ResourceQuota governs the entire Namespace.
  • The Safety Net: No limits defined by the developer? LimitRange fills in the blanks automatically.
  • The Goldilocks Fix: It ensures pods are never too big and never too small.
  • The Gatekeeper: If a Pod breaks the rules at the API door, it gets rejected immediately before it even reaches a worker node.
FeatureResourceQuotaLimitRange
ScopeAggregate (Entire Namespace)Individual (Single Pod, Container, or PVC)
Primary GoalPrevent one team from using the whole cluster.Prevent one Pod from using the whole node.
Auto-configurationNO (Just counts total usage and blocks).YES (Injects default CPU/Memory values).
Object Counts?YES (Can limit count of Pods, Services, PVCs).NO (Cannot limit the sheer number of objects).
Action on ViolationDenies creation if the total quota is exceeded.Denies creation immediately if out of bounds.
Real-world AnalogyTotal Bank Account BalanceATM Daily Withdrawal Limit

LimitRanges operate as a core resource management tool within Kubernetes. By defining a LimitRange object in a specific namespace, administrators can enforce minimum and maximum compute resource usage per Pod or Container, as well as storage request limits for PersistentVolumeClaims (PVCs).

Crucially, it acts as an automated safety mechanism. In rapid CI/CD pipelines, developers often push deployments without specifying resources.requests or resources.limits. A LimitRange intercepts these deployments and seamlessly injects predefined default values. This guarantees that every workload scheduled in the cluster has known boundaries, allowing the kube-scheduler to effectively pack nodes without risking Out-Of-Memory (OOM) kills or CPU throttling across the board.

To master LimitRanges, you must understand how Kubernetes measures resources:

  • CPU: Measured in “millicores” (m). 1000m equals 1 full CPU core. If you set a limit of 500m, the container gets half a core.
  • Memory: Measured in Mebibytes (Mi) or Gibibytes (Gi). 512Mi is roughly 512 Megabytes of RAM.
  • Requests: What the container needs to start (the scheduler uses this to find a node with enough free space).
  • Limits: The maximum hard boundary. If a container tries to use more memory than its limit, Kubernetes will terminate it with an OOMKilled error.

At an advanced architect level, LimitRanges are your first line of defense against Denial of Service (DoS) attacks from internal bad actors or misconfigured automated pipelines.

The LimitRange is implemented as an Admission Controller Plugin that runs natively inside the kube-apiserver. When an API request comes in to create a Pod, it goes through two distinct admission phases:

  1. Mutating Phase (MutatingAdmissionWebhook level): If the pod has no compute resources defined, the LimitRange mutates the incoming Pod spec payload to inject the default and defaultRequest values.
  2. Validating Phase (ValidatingAdmissionWebhook level): It evaluates the final resource numbers. If the requested values or the limits fall outside your defined Min, Max, or maxLimitRequestRatio bounds, the API server outright rejects the Pod creation, returning a 403 Forbidden error to the client.
DevSecOps Architect Level

For production-grade DevSecOps architectures, never allow a Namespace to be provisioned without an accompanying LimitRange and ResourceQuota.

Tool Integration Strategy: While LimitRange handles the numeric enforcement, you must combine it with Policy-as-Code engines to enforce which LimitRange profile is applied to which namespace.

  • Use Kyverno or OPA Gatekeeper to mandate that every Namespace creation dynamically triggers the creation of a baseline LimitRange.
  • Monitor OOMKilled events and CPU throttling caused by these limits using observability platforms like Datadog or Prometheus.
  • Security Context: By forcing strict CPU/Memory ceilings, you drastically reduce the blast radius if a container is compromised. A threat actor attempting a cryptojacking attack will immediately hit the CPU limit, triggering alerts without taking down the underlying EC2/VM node.
  • Strategy: Never allow a Namespace to exist without a LimitRange. This ensures that even if a developer is lazy, their pods are “capped” by the default limits.
  • Integration: Combine LimitRanges with OPA Gatekeeper or Kyverno. While LimitRange handles the numeric values, Kyverno can enforce which LimitRange profile is applied to which team.
  • Quality of Service (QoS) classes. When a LimitRange injects default limits and requests, it directly impacts the Pod’s QoS class:
    • Guaranteed: If the LimitRange forces Requests to exactly equal Limits.
    • Burstable: If the LimitRange sets Limits higher than Requests.
    • BestEffort: This is effectively prevented from happening if a LimitRange with defaults is active, as it will always inject some values, pulling the Pod out of the risky BestEffort tier.
Additional Details
  1. Key Components
    • type: Container: Rules applied individually to application containers, sidecars, and init containers.
    • type: Pod: Rules applied to the aggregate sum of all containers inside the Pod wrapper.
    • type: PersistentVolumeClaim: Rules applied to storage volumes.
    • maxLimitRequestRatio: Prevents massive “bursting” setups (e.g., stopping a developer from requesting 100m CPU but setting a limit of 4000m CPU).
  2. Key Characteristics
    • Namespace-scoped.
    • Proactive (acts during API admission).
    • Mutating (changes YAML dynamically).
    • Non-retroactive (does not affect already running pods).
  3. Use Case
    • Multi-tenant Clusters: When Team A and Team B share a cluster, LimitRanges prevent Team A from creating a “Monster Pod” that hogs an entire 64GB RAM node.
    • Development Environments: Developers often forget to add YAML resources. LimitRange fixes this by auto-injecting “small t-shirt size” resources.
  4. Benefits
    • Unmatched Stability: Prevents node starvation and cascading node failures.
    • Cost Control: Prevents accidental provisioning of massive, expensive containers in public clouds (AWS/GCP/Azure).
    • Standardization: Enforces a baseline for all applications deployed in the organization.
  5. Best Practices
    • Always define both default (limits) and defaultRequest.
    • Keep your maxLimitRequestRatio tight for Memory (e.g., 1 or 2) to prevent OOM issues, but slightly looser for CPU (e.g., 4) since CPU is a compressible resource.
    • Apply different LimitRanges per environment (e.g., generous limits in Production, strict tiny limits in Dev/Test).
  6. Technical Challenges
    • Init Containers: If you have heavy Init Containers (like database migration scripts), the Pod-level LimitRange might block them if they require more resources temporarily than the main application container.
  7. Limitations
    • No Retroactive Action: Applying a LimitRange today does not kill or resize existing Pods that violate the rule.
    • Node Capacity Ignorance: A LimitRange allows you to set a Max of 100GB RAM even if your biggest cluster node is only 64GB. It validates the number mathematically, not the physical cluster reality.
  8. Common Issues
    • Developers complaining that their deployments are instantly failing.
    • Pods getting stuck in Pending state if the injected defaults are larger than any available node’s free capacity.
  9. Problems and Solutions
    • Problem: “My Pod creation is failing with a Forbidden error.”
      • Solution: The Pod requested resources outside the Min/Max range. Run kubectl describe limitrange in your namespace and adjust your Pod spec accordingly.
    • Problem: “I didn’t set limits in my YAML, but my Pod has limits now.”
      • Solution: This is the Default feature working as intended! If you don’t want this specific value, you must explicitly define limits in your Pod.
    • Problem: “My Pods keep crashing with OOMKilled (Out of Memory) errors.”
      • Solution: The default memory limit injected by the LimitRange is likely too aggressive for your app (e.g., Java apps need more RAM). Specify a higher limit directly in your Pod YAML to override the default.

Always refer to the official source of truth:

Conclusion

Implementing LimitRanges is a non-negotiable step for any mature DevSecOps pipeline. They act as the automated guardrails that keep your Kubernetes clusters balanced, secure, and cost-effective. By understanding the mutating and validating phases, you ensure that no single application can accidentally or maliciously bring down your infrastructure. Keep learning, keep securing, and happy clustering!

Yaml file to create LimitRange

YAML

# ==============================================================================
# MASTER LIMITRANGE CONFIGURATION
# Purpose: Enforce strict resource governance on Containers, Pods, and Storage.
# ==============================================================================
apiVersion: v1
kind: LimitRange
metadata:
  name: master-governance-policy
  # CHANGE THIS: Apply this to your target namespace
  namespace: default 
spec:
  limits:

  # ----------------------------------------------------------------------------
  # SECTION 1: CONTAINER LEVEL RULES
  # Applies to every individual container (App, Sidecar, InitContainer)
  # ----------------------------------------------------------------------------
  - type: Container
    
    # 1. DEFAULT LIMIT (The Ceiling)
    # If a user forgets to specify 'limits', this value is injected automatically.
    # Why? To prevent a container from consuming infinite resources and crashing the node.
    default:
      cpu: "500m"       # 500 millicores (0.5 CPU)
      memory: "512Mi"   # 512 Mebibytes

    # 2. DEFAULT REQUEST (The Guarantee)
    # If a user forgets to specify 'requests', this value is injected automatically.
    # Why? ensures the scheduler reserves at least this much space for the container.
    defaultRequest:
      cpu: "100m"       # 100 millicores (0.1 CPU)
      memory: "128Mi"   # 128 Mebibytes

    # 3. MAX (The Hard Stop)
    # No container in this namespace can EVER be larger than this.
    # Why? Forces developers to split large monolithic apps into smaller microservices.
    max:
      cpu: "2"          # Max 2 Cores allowed per container
      memory: "2Gi"      # Max 2GB RAM allowed per container

    # 4. MIN (The Floor)
    # No container can request less than this.
    # Why? Prevents "spamming" the scheduler with tiny, useless containers.
    min:
      cpu: "10m"        # Minimum 10 millicores
      memory: "32Mi"    # Minimum 32MB RAM

    # 5. RATIO (The Burst Controller)
    # Calculation: Limit / Request <= Ratio
    # If Request is 1GB, Limit cannot be more than 2GB (because ratio is 2).
    # Why? Prevents massive "bursting" where a container reserves little but uses a lot.
    maxLimitRequestRatio:
      cpu: "4"          # Limit can be max 4x the Request
      memory: "2"        # Limit can be max 2x the Request

  # ----------------------------------------------------------------------------
  # SECTION 2: POD LEVEL RULES
  # Applies to the SUM of all containers inside a single Pod.
  # ----------------------------------------------------------------------------
  - type: Pod
    
    # 1. MAX TOTAL (The Group Limit)
    # The combined resources of all containers in the Pod cannot exceed this.
    # Why? Useful if you have many sidecars (Istio, Logging) and need to cap the whole group.
    max:
      cpu: "4"          # The whole Pod cannot use more than 4 Cores
      memory: "4Gi"      # The whole Pod cannot use more than 4GB RAM

    # 2. MIN TOTAL (The Group Floor)
    # The combined resources must be at least this much.
    min:
      cpu: "50m"
      memory: "64Mi"
    
    # 3. RATIO (Pod Level Bursting)
    maxLimitRequestRatio:
      cpu: "10"

  # ----------------------------------------------------------------------------
  # SECTION 3: STORAGE (PVC) LEVEL RULES
  # Applies to PersistentVolumeClaims requested by the Pods.
  # ----------------------------------------------------------------------------
  - type: PersistentVolumeClaim
    
    # 1. MAX STORAGE (The Disk Cap)
    # No single PVC can request more than this size.
    # Why? Prevents accidental requests for massive, expensive volumes (e.g., 10TB).
    max:
      storage: "50Gi"   # Max 50GB per volume

    # 2. MIN STORAGE (The Disk Floor)
    # No single PVC can be smaller than this.
    # Why? Some storage providers (like AWS EBS) have minimum size requirements or performance issues with tiny disks.
    min:
      storage: "1Gi"    # Min 1GB per volume

Contents
Scroll to Top