Kubernetes Resource Optimization: VPA/HPA Configuration, Resource Requests and Limits, Node Affinity, and Cost Optimization

Kubernetes clusters are notorious for over-provisioned resources and unexpected cloud bills. Studies show the average Kubernetes cluster wastes 50-70% of its allocated compute. This guide covers systematic resource optimization from correct requests and limits to dynamic autoscaling and intelligent node placement.

Resource Requests and Limits

Understanding the Difference

• Request: the amount Kubernetes reserves for scheduling; guaranteed to be available
• Limit: the maximum the container can use; exceeded CPU is throttled, exceeded memory causes OOM kill

resources:
  requests:
    memory: "256Mi"
    cpu: "250m"    # 0.25 vCPU
  limits:
    memory: "512Mi"
    cpu: "1000m"   # 1 vCPU

Finding the Right Values

Start by observing actual usage:

# Current usage for all pods in namespace
kubectl top pods -n production --sort-by=cpu

# Resource usage over time (requires metrics-server)
kubectl top pods -n production --containers

# Historical usage via PromQL

# Average CPU usage for a container over 7 days
avg_over_time(
  rate(container_cpu_usage_seconds_total{
    container="api",
    namespace="production"
  }[5m])[7d:]
)

# P95 memory usage
quantile_over_time(0.95,
  container_memory_working_set_bytes{
    container="api",
    namespace="production"
  }[7d]
)

Setting Requests from Historical Data

# Rule of thumb:
# CPU request = P50 usage * 1.2
# Memory request = P95 usage * 1.1
# CPU limit = P99 usage * 1.5 (or remove CPU limits to avoid throttling)
# Memory limit = P99 usage * 1.5 (OOM risk vs waste trade-off)

CPU Throttling Problem

Setting CPU limits too low causes throttling even when nodes have headroom:

# Check CPU throttling rate
kubectl exec -n production deploy/api -- cat /sys/fs/cgroup/cpu/cpu.stat

# PromQL: throttling rate
rate(container_cpu_cfs_throttled_seconds_total{container="api"}[5m])
/ rate(container_cpu_cfs_periods_total{container="api"}[5m])

Many teams remove CPU limits entirely for latency-sensitive workloads, relying on HPA to scale instead of throttling.

Horizontal Pod Autoscaler (HPA)

HPA scales the number of pod replicas based on metrics.

CPU-Based HPA

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api-hpa
  namespace: production
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api
  minReplicas: 2
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 60   # target 60% CPU utilization
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 60
      policies:
        - type: Pods
          value: 4
          periodSeconds: 60
    scaleDown:
      stabilizationWindowSeconds: 300   # wait 5min before scaling down
      policies:
        - type: Percent
          value: 10
          periodSeconds: 60

Custom Metrics HPA

Scale on request rate, queue depth, or any Prometheus metric:

metrics:
  - type: Pods
    pods:
      metric:
        name: http_requests_per_second
      target:
        type: AverageValue
        averageValue: "1000"   # target 1000 RPS per pod

  - type: External
    external:
      metric:
        name: sqs_queue_depth
        selector:
          matchLabels:
            queue: order-processing
      target:
        type: AverageValue
        averageValue: "50"    # 50 messages per pod

Install the Prometheus Adapter to expose Prometheus metrics to HPA:

# prometheus-adapter values.yaml
rules:
  custom:
    - seriesQuery: 'http_requests_total{namespace!="",pod!=""}'
      resources:
        overrides:
          namespace: { resource: "namespace" }
          pod: { resource: "pod" }
      name:
        matches: "^(.*)_total
quot;
        as: "${1}_per_second"
      metricsQuery: 'sum(rate(<<.Series>>{<<.LabelMatchers>>}[2m])) by (<<.GroupBy>>)'

Vertical Pod Autoscaler (VPA)

VPA automatically adjusts resource requests based on observed usage.

# Install VPA
kubectl apply -f https://github.com/kubernetes/autoscaler/releases/latest/download/vertical-pod-autoscaler.yaml

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: api-vpa
  namespace: production
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api
  updatePolicy:
    updateMode: "Auto"   # Auto, Recreate, Initial, or Off
  resourcePolicy:
    containerPolicies:
      - containerName: api
        minAllowed:
          cpu: 100m
          memory: 128Mi
        maxAllowed:
          cpu: 4
          memory: 4Gi
        controlledResources: ["cpu", "memory"]
        controlledValues: RequestsAndLimits

Important: VPA and HPA cannot both manage CPU/memory simultaneously. Use VPA for the application's resource sizing, HPA for replica count scaling. For workloads needing both, use VPA in recommendation-only mode (updateMode: "Off") and apply recommendations manually.

Node Affinity and Topology

Node Affinity

spec:
  affinity:
    nodeAffinity:
      # Hard requirement: must run on this node type
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
          - matchExpressions:
              - key: node.kubernetes.io/instance-type
                operator: In
                values:
                  - m5.xlarge
                  - m5.2xlarge

      # Soft preference: prefer nodes in us-east-1a
      preferredDuringSchedulingIgnoredDuringExecution:
        - weight: 100
          preference:
            matchExpressions:
              - key: topology.kubernetes.io/zone
                operator: In
                values:
                  - us-east-1a

Pod Anti-Affinity for High Availability

spec:
  affinity:
    podAntiAffinity:
      # Hard: no two replicas on same node
      requiredDuringSchedulingIgnoredDuringExecution:
        - labelSelector:
            matchLabels:
              app: api
          topologyKey: kubernetes.io/hostname

      # Soft: prefer spreading across AZs
      preferredDuringSchedulingIgnoredDuringExecution:
        - weight: 50
          podAffinityTerm:
            labelSelector:
              matchLabels:
                app: api
            topologyKey: topology.kubernetes.io/zone

Topology Spread Constraints (newer approach)

spec:
  topologySpreadConstraints:
    - maxSkew: 1
      topologyKey: topology.kubernetes.io/zone
      whenUnsatisfiable: DoNotSchedule
      labelSelector:
        matchLabels:
          app: api
    - maxSkew: 1
      topologyKey: kubernetes.io/hostname
      whenUnsatisfiable: ScheduleAnyway
      labelSelector:
        matchLabels:
          app: api

Spot/Preemptible Instances for Cost Reduction

Run stateless workloads on spot instances for 60-90% savings:

# Karpenter NodePool for spot instances
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: spot-workers
spec:
  template:
    metadata:
      labels:
        capacity-type: spot
    spec:
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot"]
        - key: kubernetes.io/arch
          operator: In
          values: ["amd64"]
        - key: node.kubernetes.io/instance-type
          operator: In
          values: ["m5.xlarge", "m5.2xlarge", "m4.xlarge", "m4.2xlarge"]
      nodeClassRef:
        apiVersion: karpenter.k8s.aws/v1
        kind: EC2NodeClass
        name: default
  disruption:
    consolidationPolicy: WhenUnderutilized
    consolidateAfter: 30s

Graceful Spot Termination

# Handle 2-minute termination notice
spec:
  terminationGracePeriodSeconds: 120
  containers:
    - name: api
      lifecycle:
        preStop:
          exec:
            command: ["/bin/sh", "-c", "sleep 10"]  # drain connections

Resource Quotas and LimitRanges

ResourceQuota per Namespace

apiVersion: v1
kind: ResourceQuota
metadata:
  name: production-quota
  namespace: production
spec:
  hard:
    requests.cpu: "100"
    requests.memory: "200Gi"
    limits.cpu: "200"
    limits.memory: "400Gi"
    pods: "200"
    services.loadbalancers: "5"
    persistentvolumeclaims: "20"

LimitRange for Default Requests

apiVersion: v1
kind: LimitRange
metadata:
  name: default-limits
  namespace: production
spec:
  limits:
    - type: Container
      default:          # default limit if not specified
        cpu: 500m
        memory: 256Mi
      defaultRequest:   # default request if not specified
        cpu: 100m
        memory: 128Mi
      max:              # maximum allowed
        cpu: 4
        memory: 8Gi
      min:              # minimum required
        cpu: 50m
        memory: 64Mi

Cost Monitoring with Kubecost

helm install kubecost cost-analyzer \
  --repo https://kubecost.github.io/cost-analyzer/ \
  --namespace kubecost \
  --create-namespace \
  --set kubecostToken="your-token"

Key Kubecost queries:

• Cost per namespace
• Cost per team label
• Idle/wasted resources
• Savings recommendations

Cluster Autoscaler vs Karpenter

Karpenter is recommended for new clusters due to faster scaling and better bin-packing.

Conclusion

Kubernetes resource optimization is a continuous process. Start by measuring actual usage and setting accurate requests. Add HPA to handle traffic spikes without over-provisioning. Use VPA recommendations to right-size containers. Spread workloads across zones with topology constraints. Run stateless workloads on spot instances for dramatic cost savings. With systematic optimization, teams routinely cut Kubernetes costs by 40-60% without sacrificing reliability.