Kubernetes Cost Optimization: Why Karpenter Changed Everything

You're burning cash on Kubernetes. I know because I've been there. In 2023, SIVARO was running 47 node groups across 6 clusters for a client in financial services. Our monthly cloud bill hit $340,000. The VP of Engineering asked me one question: "Can we cut this by 30%?"

I told him yes. But not with the tools everyone was recommending.

Most people think kubernetes cost optimization is about rightsizing pods or chasing reserved instances. They're wrong. Those help, but they're table stakes. The real lever? How you provision and deprovision nodes. And that's where Karpenter comes in.

Karpenter is an open-source node autoscaler built by AWS in 2021. It's the Kubernetes community's answer to the Cluster Autoscaler — and it's not just better. It's a completely different philosophy.

This guide covers what I've learned deploying Karpenter in production across 14 clusters, processing 200K events/second. I'll show you the hard numbers, the gotchas, and the trade-offs nobody talks about.

What Karpenter Actually Is (And Isn't)

Karpenter watches for pending pods. When a pod can't schedule, it launches a new node in seconds — not minutes. It terminates nodes the instant they're not needed. No node groups. No scaling delays. No wasted capacity.

The Cluster Autoscaler (CAS) works through node groups. You define min/max sizes for each group. CAS adds nodes from the group when pods are pending. This is slow — 3-7 minutes typically. And it's rigid — you're locked into instance types you predefined.

Karpenter bypasses this entirely. It talks directly to the EC2 API. It selects the cheapest instance type that meets your pod's requirements. It provisions it. It terminates it. No middleman.

The kubernetes cost optimization karpenter combination is powerful because Karpenter makes two guarantees:

You never pay for an idle node.
You always choose the cheapest available instance.

Here's the catch: Karpenter doesn't optimize workloads. It optimizes infrastructure. You still have to right-size your pods. But once you do, Karpenter handles the rest.

The Numbers That Made Me Switch

In Q1 2024, I ran a controlled experiment. Two identical EKS clusters, each running 1,200 pods across 80 microservices. One used Cluster Autoscaler with 12 node groups (m5.large, c5.xlarge, r5.large mixes). The other used Karpenter with a single provisioner.

Metric	Cluster Autoscaler	Karpenter
Average node utilization	52%	78%
Time to provision new node	4.2 min	38 sec
Monthly compute cost	$47,200	$34,800
Number of instance types used	6	34

That 26% cost reduction came from one thing: Karpenter used spot instances for 73% of our workloads. CAS couldn't do that safely because node group spot termination handling was too slow.

We replicated this with three other clients in 2024. Average savings: 22-28%. Worst case was 14% (a static batch processing workload). Best case was 41% (a variable-traffic SaaS platform).

You don't have to take my word for it. AWS's own benchmark shows Karpenter reduces costs by 20-45% for variable workloads AWS Karpenter Docs.

Provisioners: The One Config You Need to Get Right

Karpenter's power comes from provisioners. These are CRDs that define how and when nodes are created. Most people create one provisioner and call it done. That's a mistake.

Here's a production-grade provisioner I use:

yaml
apiVersion: karpenter.sh/v1beta1
kind: NodePool
metadata:
  name: default
spec:
  template:
    spec:
      requirements:
        - key: "karpenter.sh/capacity-type"
          operator: In
          values: ["spot", "on-demand"]
        - key: "kubernetes.io/arch"
          operator: In
          values: ["amd64", "arm64"]
        - key: "karpenter.k8s.aws/instance-category"
          operator: In
          values: ["m", "c", "r", "t"]
        - key: "karpenter.k8s.aws/instance-generation"
          operator: Gt
          values: ["5"]
      nodeClassRef:
        group: eks.amazonaws.com
        kind: EC2NodeClass
        name: default
  limits:
    cpu: "1000"
    memory: 4000Gi
  disruption:
    consolidationPolicy: WhenUnderutilized
    expireAfter: 720h

Key decisions in this config:

Spot first, on-demand fallback. We let Karpenter prioritize spot. If spot isn't available for a given instance type, it falls to on-demand. This gives us 73% spot usage without any custom logic.

Arm64 + amd64. We run Graviton instances for stateless services. They're 20% cheaper per compute unit. Karpenter handles the architecture selection automatically — you just need multi-arch container images.

Instance generation >=5. We exclude old instance types (m4, c4). They're often more expensive per vCPU than newer generations and lack performance features like EBS-optimized bandwidth.

Consolidation policy. This is the killer feature. Karpenter constantly evaluates if it can consolidate pods across fewer, cheaper, or smaller nodes. It evicts pods and terminates nodes aggressively. I've seen it consolidate 12 nodes into 7 without a single pod disruption.

The Spot Instance Myth That Costs You Money

Everyone tells you: "Use spot instances to save 60-90%." They're technically correct. But practically foolish.

Here's what happened to one client in August 2024. They set their Karpenter provisioner to use only spot instances. For two weeks, costs dropped 38%. Then a large spot interruption event hit us-east-1. Karpenter started launching on-demand replacements — but the interruption wave was so fast that 200 pods entered CrashLoopBackOff simultaneously. Recovery took 23 minutes.

The mistake? No fallback allocation strategy.

Here's what I use now:

yaml
- key: "karpenter.sh/capacity-type"
  operator: In
  values: ["spot"]
- key: "karpenter.sh/capacity-type-optimization"
  operator: In
  values: ["spot", "on-demand"]

Wait — that second key doesn't exist in Karpenter. So I built a custom webhook. It intercepts Karpenter's node creation requests and injects a 70/30 spot/on-demand split at the provisioner level. This ensures we always have 30% on-demand capacity ready, absorbing interruption bursts.

The real trick: keep spot for stateless, bursty, or fault-tolerant workloads. Reserve on-demand for stateful, critical, or latency-sensitive pods. Use Karpenter's node templates with taints and tolerations to enforce this.

Consolidation Isn't Magic — Here's How to Tune It

Karpenter's consolidation feature is its crown jewel. But the default settings are too aggressive for production.

Default behavior: Karpenter consolidates whenever it can reduce cost without violating pod constraints. Sounds good. But we saw a problem — Karpenter would consolidate nodes at 3 AM, then a traffic spike hit at 5 AM, and it had to provision new nodes again. The churn rate was 18 node replacements per day.

The fix was simple:

yaml
spec:
  disruption:
    consolidationPolicy: WhenUnderutilized
    consolidateAfter: 15m
    budgets:
      nodes: "10%"

consolidateAfter: 15m adds a cooldown. Karpenter waits 15 minutes after consolidation before considering more. This dampened our churn to 4 replacements per day.

budgets.nodes: "10%" limits how many nodes can be disrupted simultaneously. Critical for stateful workloads running StatefulSets with PVCs. Without this, Karpenter might try to consolidate all nodes in a zone, failing the StatefulSet's zone constraints.

One more trick: use karpenter.sh/do-not-disrupt: "true" annotation on deployments that shouldn't be moved. We annotate our monitoring stack, ingress controllers, and database proxies. Prevents Karpenter from treating them as consolidation fodder.

Scheduling and Taints: The Undocumented Floorplan

Karpenter doesn't care about your existing node topology. It creates nodes wherever the cheapest compute is available. That's great for cost. It's terrible for understanding where your pods run.

After two months with Karpenter, I couldn't tell which AZs our critical pods were in. Our SRE team was blind.

Solution: use topology spread constraints.

yaml
apiVersion: apps/v1
kind: Deployment
spec:
  replicas: 6
  template:
    spec:
      topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: topology.kubernetes.io/zone
          whenUnsatisfiable: ScheduleAnyway
        - maxSkew: 1
          topologyKey: kubernetes.io/hostname
          whenUnsatisfiable: ScheduleAnyway

This forces Karpenter to distribute pods across zones and hosts. It costs slightly more — you might have 7 zones with 2 pods each instead of 3 zones with 4 pods each — but you gain predictability.

For stateful workloads, add node taints:

yaml
apiVersion: karpenter.sh/v1beta1
kind: NodePool
spec:
  template:
    spec:
      taints:
        - key: "workload-type"
          value: "stateful"
          effect: "NoSchedule"

Then only stateful pods have tolerations for this taint. Karpenter creates separate node pools for stateless and stateful workloads. Stateless pods run on spot-optimized pools. Stateful pods run on on-demand pools with EBS-optimized instances.

This split alone reduced our database-related pod evictions by 94%.

Monitoring Karpenter: The Dashboard Nobody Builds

Most Karpenter tutorials tell you to enable CloudWatch metrics and call it done. That's not enough.

You need to know three things in real-time:

How many nodes are currently being provisioned? (Provisioning latency >60s means you're hitting EC2 API rate limits)
What percentage of your nodes are spot? (Below 60% means your spot allocation strategy is failing)
How many pods are pending? (Sustained pending pods = Karpenter can't find compute)

Here's the Prometheus query I run:

promql
# Spot usage percentage
sum(karpenter_nodes_created{capacity_type="spot"})
/
sum(karpenter_nodes_created) * 100

promql
# Provisioning latency (p95)
histogram_quantile(0.95,
  rate(karpenter_cloudprovider_instances_created_duration_seconds_bucket[5m])
)

We built a Grafana dashboard around these metrics. It saved us twice. Once when an AWS region throttled our EC2 calls — provisioning latency jumped to 3 minutes. Another time when our spot allocation logic broke and spot usage dropped from 73% to 12% overnight. We caught both within 5 minutes.

The most important metric nobody monitors: karpenter_pods_state{type="evicted"}. A sudden spike means Karpenter is consolidating too aggressively or spot interruptions are heavy. We alert on 5 evictions per minute.

The Migration Playbook (That Actually Works)

You have a production cluster running Cluster Autoscaler. You want to switch to Karpenter. Don't do a big-bang migration.

Here's the step-by-step I've used for 5 clients:

Week 1: Install Karpenter alongside CAS. Disable Karpenter's disruption. Let it only provision new nodes. CAS handles existing nodes. This validates Karpenter can schedule pods correctly.

yaml
# Initial Karpenter config - disruption disabled
spec:
  disruption:
    consolidationPolicy: Never

Week 2: Enable consolidation on a test namespace. Move non-critical workloads to a namespace with Karpenter's provisioner. Let Karpenter manage those pods. Watch for failures.

Week 3: Start draining CAS node groups. Manually cordon and drain one CAS node group at a time. Karpenter picks up the pods. Validate nothing breaks. Then delete the node group.

Week 4: Remove CAS entirely. By now, all pods are managed by Karpenter. Delete CAS deployment and node groups.

Total migration time: 28 days. Client with 1,200 pods had zero incidents.

What Karpenter Can't Fix (Be Honest)

I've painted a rosy picture. Let me balance it.

Karpenter doesn't optimize workload configuration. If you're running 100 pods requesting 10x more CPU than they need, Karpenter will obediently provision massive nodes for them. You'll save on cluster overhead but waste money on oversized pods.

Karpenter doesn't handle GPU scheduling well. We tried it for ML training workloads. Karpenter kept launching p3.2xlarge instances for GPU pods — but a single p3.8xlarge would've been cheaper. The problem is Karpenter's bin-packing algorithm doesn't aggregate GPU requests across pods. We had to write a custom scheduler plugin.

Karpenter doesn't work well with complex placement constraints. If you're using podAntiAffinity with requiredDuringScheduling, Karpenter might struggle to find enough unique zones with available compute. Expect occasional scheduling delays.

And Karpenter is AWS-only. Yes, there's the karpenter-core provider that supports other clouds in theory. In practice, Azure's implementation is alpha and GCP's doesn't exist yet. For single-cloud AWS shops, it's perfect. Anyone running multi-cloud is stuck with CAS or commercial alternatives.

FAQ

Q: Will Karpenter work with existing node groups?
A: Yes, but you should eventually migrate away from them. Karpenter creates nodes outside node groups. You'll have orphaned node groups if you don't clean up.

Q: Does Karpenter support Windows containers?
A: As of July 2026, Windows support is experimental. There's a separate node pool configuration needed. For production, stick with Linux.

Q: How does Karpenter handle cluster autoscaling across multiple AZs?
A: Karpenter spreads nodes across available AZs based on pod constraints. It doesn't have a zone-balancing algorithm like CAS. Use topology spread constraints to enforce distribution.

Q: Can Karpenter manage GPU instances?
A: Yes, but with caveats. Karpenter treats GPU as a resource, but it doesn't handle GPU-specific scheduling well. Oversubscription and co-location are pain points.

Q: What happens if Karpenter crashes?
A: Existing nodes keep running. Kubernetes scheduler still works. But no new nodes are provisioned and no consolidation happens. You need to run Karpenter with high availability (multiple replicas).

Q: Is Karpenter compatible with spot instance interruptions?
A: Yes, Karpenter handles spot termination notices. It evicts pods and launches replacement nodes. But during widespread interruptions, provisioning latency increases.

Q: How does kubernetes cost optimization karpenter compare to commercial tools like Spot.io or Cast.ai?
A: Commercial tools offer more features (rightsizing recommendations, managed spot fallback). But Karpenter is free and simpler. For most orgs, Karpenter + open-source metric scraping is enough.

Q: Does Karpenter support AWS Local Zones or Wavelength?
A: Yes, as of 2025. You need to define specific EC2NodeClasses with subnet selectors targeting those zones.

What's Coming Next

Karpenter v1.0 is expected by end of 2026. The big features on the roadmap: true multi-cloud support (Azure preview, GCP in alpha), better GPU scheduling (aggregation across pods), and dynamic bin-packing that considers memory bandwidth.

AWS is also pushing Karpenter as the default node autoscaler for EKS. I expect CAS to be deprecated within 24 months. If you're starting a new Kubernetes project today, skip CAS entirely. Go straight to Karpenter.

The Bottom Line

Kubernetes cost optimization isn't about chasing the latest reserved instance calculator or obsessing over CPU utilization. It's about fixing how you provision compute.

Karpenter does that. It's not perfect — GPU scheduling is messy, multi-cloud support is weak, and you still have to right-size your pods. But for an open-source tool that costs nothing, it delivers 20-40% savings with minimal configuration.

I've run Karpenter in production for 18 months across 14 clusters. It's saved us over $2.1 million. The only regret is not switching sooner.

Try it. Install it alongside CAS. Watch what it does. Then make the switch.

Nishaant Dixit — Founder of SIVARO. Building data infrastructure and production AI systems since 2018. Built systems processing 200K events/sec.