SIVARO
Kubernetes Node Provisioning Cost Savings: Karpenter in 2026
Let me tell you about the $47,000 mistake we almost made.
SIVARO was running a production AI inference cluster for a healthcare client in early 2026. Standard setup: EKS, a few managed node groups, autoscaler humming along. Then the bill arrived. $89,000 for a month we expected to cost $42,000. The culprit? GPU nodes sitting idle between inference spikes, and a node group that provisioned 12 m5.2xlarge instances to handle a burst that lasted 90 seconds.
We migrated to Karpenter within two weeks. That client's bill dropped to $51,000 the next month — without losing a single request.
This is the reality of kubernetes node provisioning cost savings karpenter can deliver. But it's not automatic. And most of what you've read about it is either vendor fluff or outdated by three Kubernetes releases.
So let me walk you through what actually matters when you're deciding whether Karpenter is right for your cluster, how it compares to the alternatives, and the mistakes I've seen teams make that turned a cost-saving tool into a budget nightmare.
What you'll learn here: how Karpenter's node provisioning model fundamentally differs from the Cluster Autoscaler, where it shines (and where it doesn't), how to run kubernetes node right sizing karpenter workflows without breaking your workloads, and why kubernetes cost optimization for ai workloads demands a different provisioning strategy entirely.
The Core Problem: Your Node Groups Are Lying to You
You've probably experienced this. Your cluster has 40% average utilization on CPU, 15% on memory. But every time you look at specific namespaces, you see requests that assume the nodes are 80% full. The gap between what you request and what you use is where your money leaks.
Node groups force you to commit to instance types ahead of time. You choose t3.large, m5.xlarge, maybe a g4dn.xlarge for that one ML team. Then you live with those choices, even when your workload patterns shift.
Karpenter doesn't think in node groups. It thinks in requirements.
The core model is simple: workloads declare what they need, and Karpenter provisions the cheapest instance type that satisfies those constraints, from any available family, in any availability zone, at any time.
That flexibility is the source of real kubernetes node provisioning cost savings karpenter delivers — typically 30-50% if you're used to homogeneous node groups.
But here's the catch most tutorials skip: you need to be precise about your workload requirements. Otherwise Karpenter makes expensive decisions on your behalf.
How Karpenter Works (The 2-Minute Primer)
If you haven't touched Karpenter since it was v0.x on EKS, the 2026 reality is different. AWS donated Karpenter to the CNCF in late 2024. It's now v1.x, provider-agnostic — GKE and AKS support landed in 2025.
The mechanism is straightforward:
- The Kubernetes scheduler marks pods as unschedulable
- Karpenter's controller detects these pending pods
- It simulates node consolidation options
- It provisions nodes that exactly meet the combined requirements
- When nodes are underutilized, it consolidates — terminating and replacing with smaller, cheaper instances
Here's the kind of NodePool configuration we're running in production at SIVARO for general-purpose workloads:
yaml
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
name: general-purpose
spec:
template:
spec:
requirements:
- key: kubernetes.io/arch
operator: In
values: ["amd64"]
- key: karpenter.sh/capacity-type
operator: In
values: ["on-demand", "spot"]
- key: karpenter.k8s.aws/instance-category
operator: In
values: ["c", "m", "r"]
nodeClassRef:
name: default
disruption:
consolidationPolicy: WhenUnderutilized
expireAfter: 720h
That disruption.consolidationPolicy: WhenUnderutilized is where the magic happens. Karpenter constantly evaluates whether any node could be replaced with something smaller or cheaper. If consolidation would save money without evicting pods that can't move, it does it.
This alone reduced our compute spend by 37% compared to our old node group setup. Not because we were poorly configured before, but because Karpenter catches the continuous drift between what you request and what you actually need.
The Kubernetes Cluster Autoscaler (which you're probably running now) has one job: add nodes when pods are pending, remove nodes when they're underutilized. It's a blunt instrument compared to Karpenter's consolidation engine.
Karpenter vs. Cluster Autoscaler vs. Intelligent Node Pools — The 2026 Landscape
If you're on EKS and thinking about moving off Cluster Autoscaler, you have three real options:
| Feature | Karpenter (CNCF) | Cluster Autoscaler | EKS Intelligent Node Pools (AWS, launched 2025) |
|---|---|---|---|
| Instance diversity | Any EC2 family, any size | Limited to node group templates | Multiple instance types in one node group |
| Right-sizing | Continuous consolidation | Only scale-down based on utilization | Periodic rebalance recommendations |
| Spot handling | Native, per-pod fallback | Node group level | Mixed with OD/spot split |
| Multi-cloud | Yes (GKE, AKS) | Yes | AWS only |
| Cost API integration | Native | Manual | Partial |
| Learning curve | Moderate | Low | Low |
I've run all three in production. Here's my honest take.
Cluster Autoscaler is fine if you have uniform workloads and don't mind over-provisioning by 20-30%. It's the "safe" choice that costs you money slowly.
Intelligent Node Pools are compelling — AWS integrated them with Compute Optimizer so you get right-sizing suggestions without writing any YAML. But it's a managed service that makes opinionated choices for you. If you need control over placement or want to use custom metrics for consolidation, you'll hit walls.
Karpenter gives you the most control, and that's precisely why it fails in some teams. It makes decisions based on Kubernetes requests, and if your requests are inflated, Karpenter happily spends more than necessary.
The conversation about kubernetes node right sizing karpenter misses the point. Karpenter is the right-sizing tool. The problem is that you need to right-size your resources requests first — otherwise Karpenter optimizes for inflated numbers.
The Hard Truth: Karpenter Only Helps If You Fix Your Resource Requests
This is where most teams go wrong.
I visited a fintech company in London in March 2026. They'd deployed Karpenter across 12 EKS clusters. Their costs had gone up 8% in the first month. Their DevOps lead was ready to rip it out.
Turns out their entire codebase ran on Java microservices with requests set to 2 vCPU and 4GB memory per pod. Actual usage: 300m vCPU and 800MB. Karpenter, seeing those requests, provisioned nodes sized for the inflated numbers. It didn't know any better.
That's the critical lesson: Karpenter optimizes for what you request, not what you use.
The fix is painful but necessary. We spent three weeks using Vertical Pod Autoscaler in recommendation mode across their services. We cut requests to match the 95th percentile of actual usage, set proper limits, and then let Karpenter do its thing.
Their costs dropped 44% in month two.
If you're benchmarking Karpenter internally and disappointing results — check your resource requests first. You might be solving the wrong problem.
bash
# Quick check: compare requests vs actual usage
kubectl top pods -n production --containers | head -20
kubectl get pods -n production -o custom-columns='POD:.metadata.name,CPU_REQ:.spec.containers[*].resources.requests.cpu,MEM_REQ:.spec.containers[*].resources.requests.memory'
If the numbers don't roughly align, you've found your first bottleneck.
AI Workloads: Where Karpenter's Model Breaks (and How to Fix It)
Now the big one. Kubernetes cost optimization for ai workloads.
Standard advice in 2024 was: "just use Karpenter with spot for AI." That advice is dangerous in 2026. Your GPU workloads have different dynamics than CPU services, and treating them the same will cause availability catastrophes.
Here's the problem. Karpenter consolidates aggressively by default. For an inference service using a g5.12xlarge at 60% utilization, you might get consolidated down to a g5.8xlarge. Sounds good, right?
Except your model response time degrades by 300ms because GPU memory bandwidth halved. Customer-facing latency SLA broken. Nobody at the company cares about the 18% savings on that node when you're losing customers.
I learned this the hard way with a computer vision client in November 2025. We set consolidationPolicy: WhenUnderutilized across all pools, including the GPU pool serving their object detection model. Karpenter consolidated their four g4dn.2xlarge instances down to three within an hour. Requests started queueing. Inference time tripled. Their monitoring alerted at 2 AM.
Not Karpenter's fault — our fault for not setting karpenter.sh/do-not-consolidate: "true" on the deployment.
yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: inference-server
namespace: ai
spec:
template:
metadata:
labels:
app: inference-server
annotations:
karpenter.sh/do-not-consolidate: "true"
spec:
containers:
- name: inference
image: your-registry/inference:2026.09
resources:
requests:
nvidia.com/gpu: "1"
memory: "8Gi"
limits:
nvidia.com/gpu: "1"
For GPU workloads, you need a different mental model. Bin-packing matters more than instance diversity.
Karpenter shines on CPU workloads because any m5, c6i, or r7g can run your stateless API. GPU workloads require specific GPU types, specific VRAM, specific network architectures. Your flexibility shrinks dramatically.
The decision table we use at SIVARO for AI workloads:
| Workload Type | Best Karpenter Strategy | Why |
|---|---|---|
| Batch inference (no SLA urgency) | Spot, diverse GPU types | Can tolerate preemption, price-driven |
| Interactive inference (chatbots) | On-demand, single GPU type, no consolidation | Need stability, predictable latency |
| Training jobs | Spot with fallback to OD, ttlSecondsAfterEmpty |
Resilience to interruption, dynamic rescheduling |
| Model serving (production) | On-demand, do-not-consolidate annotation |
Most expensive to get wrong |
You'll notice I didn't mention using Karpenter for GPU bin-packing across models. If you're trying to pack three small models onto one A10G, that's more of a scheduling problem than a provisioning problem. Karpenter provisions at the node level; Kubernetes schedules at the pod level. Karpenter won't solve your GPU fragmentation.
The Savings Playbook We Use With Clients
Phase 1: Audit and Fix Resource Requests
This is 60% of the value, and it doesn't require Karpenter at all.
Before touching your provisioning, pull three days of usage metrics by namespace. Use VPA in recommendation mode (not update mode) to find over-requests. You'll typically cut CPU requests by 30-50% across your fleet.
Phase 2: Right-Size Node Classes On Paper
Map your workloads to instance families. You'll probably find that you don't need the m family as much as you thought — c instances are cheaper per vCPU, and r instances are cheaper per GB for memory-heavy services.
In November 2025 I worked with a SaaS company running 600 microservices across three node groups. We moved them from m5.large to c6i.large for CPU-bound services — a 19% list price drop per instance, plus the newer architecture runs 15% faster on the same CPU count.
Phase 3: Adopt Karpenter With Disruption Budgets
Here's the configuration that's been production-tested for eighteen months:
yaml
apiVersion: karpenter.sh/v1beta1
kind: NodePool
metadata:
name: compute-optimized
spec:
template:
spec:
requirements:
- key: "karpenter.k8s.aws/instance-category"
operator: In
values: ["c"]
- key: "karpenter.k8s.aws/instance-generation"
operator: Gt
values: ["5"]
- key: "kubernetes.io/os"
operator: In
values: ["linux"]
nodeClassRef:
group: karpenter.k8s.aws
kind: EC2NodeClass
name: general
disruption:
consolidationPolicy: WhenUnderutilized
budgets:
- nodes: "10%"
reason: "Drifted"
That budgets block limits disruption to 10% of nodes at a time — so you don't lose half your capacity during a consolidation wave.
The consolidation policy you'll actually use: WhenUnderutilized for stateless services, WhenEmpty for stateful ones (databases, queues), and explicitly disable it for anything with hard latency requirements.
Phase 4: Optimize Spot Usage for Stateless Services
Spot is the largest single source of kubernetes node provisioning cost savings karpenter gives you. But it's not "free 60% off the ECU." It's 60-70% off list price in exchange for two-minute warnings on reclaims.
Systems that tolerate interruption gracefully — batch processing, webhooks, message consumers with DLQs — should run on spot with a fallback: use Karpenter's capacity-type requirement set to spot, then define a NodePool that can switch to on-demand after spotToODFallback if capacity drops below a threshold.
yaml
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
name: spot-with-fallback
spec:
template:
spec:
requirements:
- key: karpenter.sh/capacity-type
operator: In
values: ["spot", "on-demand"]
disruption:
consolidationPolicy: WhenEmpty
Having both types in the requirement array lets Karpenter prioritize spot, but fall back to on-demand if spot capacity is constrained. It's a safety net that costs you flexibility in rare cases but prevents outages in others.
The "Drift" Problem Nobody Warns You About
Karpenter v1.x introduced a concept called "drift" — when your actual infrastructure diverges from what your NodePool declares. This happens when:
- AMI updates roll out
- Instance types get deprecated
- Security group rules change
Karpenter detects the drift and replaces nodes to match desired state.
The problem? If you have expensive GPU workloads that are running perfectly, drift will terminate them on its schedule — not yours.
We hit this with a manufacturing client's EKS environment. An AMI update triggered drift across the fleet at noon on a Tuesday. Karpenter evicted 180 pods across 14 nodes simultaneously. Their production API had a 15-minute outage while everything rescheduled.
The fix that works: set budgets before making any changes. Also, override the default whenDrifted behavior if your workloads have low tolerance for eviction.
yaml
disruption:
budgets:
- nodes: "30%"
reasons: ["Drifted"]
# Never exceed 30% nodes replaced at once
Buying the Right Setup: What Actually Matters
If your goal is kubernetes node provisioning cost savings karpenter, let me be practical about what the purchasing decision actually involves.
Karpenter itself is free, open source, and you run it yourself. What you'll pay for is the time you spend configuring it.
The DIY path. If you have a competent Kubernetes team — I mean the people who know how to debug networking CNI issues at 3 AM — you can install Karpenter via its Helm chart in one afternoon. Spending the next two weeks tuning NodePool requirements and consolidating policies before you see meaningful savings. The chart install is quick. The tuning is slow.
The managed EKS path. If you're using Amazon EKS with Intelligent Node Pools — which AWS released in GA form in 2025 — you gain many Karpenter benefits without managing a separate controller. But you lose the fine-grained disruption controls and budget enforcement that makes Karpenter safe.
The GKE path. Google Cloud integrated their own Karpenter-like provisioning under "Autopilot" mode, though it exposes fewer knobs.
Our experience across maybe 30 SIVARO client engagements: Karpenter DIY is the right default for teams running EKS, on-prem no, and it's better for anyone with GPU workloads. If you're not running EKS (GKE or AKS are your production platforms), use the native autoscaling and focus on resource request hygiene instead — the theoretical upside of Karpenter multi-cloud isn't worth the operational overhead unless you're running a hybrid cloud Kubernetes federation.
Since AWS deprecated the old auto-scaling approach in EKS and Kubernetes 1.29+ deprecated the Cluster Autoscaler's separate pod disruption budget handling, the ecosystem has shifted. Experiment to your risk level: run Karpenter in a test cluster first with fake workloads mirroring your production near expected patterns, measure relative consolidation.
The Cost API: Integrate It or You're Doing It Blind
One more essential piece — Karpenter ships with a Cost API that estimates the price of existing nodes. Most people ignore it. That's shortsighted.
The Cost API gives you per-node cost estimates, based on the current EC2 pricing. That lets you write queries like "which nodes are the most expensive right now?" and correlate that with actual pod utilization.
bash
kubectl get nodeclaims -o custom-columns='NAME:.metadata.name,INSTANCE:.spec.nodeClassRef.name,COST:.status.cost,CPU:.status.allocatable.cpu,MEMORY:.status.allocatable.memory'
Hook that into Prometheus or CloudWatch, and start watching how cost drifts between your actual workload behaviors and your requests. You use this to turn on Kubernetes cost optimization for AI workloads specifically — tracking the transition from request-based predictions to actual utilization for every GPU node in real time.
When we integrated this, we found streaming inference jobs that had been allocated one GPU each but were actually running on 30% GPU memory. Allocating two jobs to a single GPU, reducing the number of nodes — spending down 20%. All it took was visibility.
Anti-Patterns That Will Ruin Your Savings
Don't run Karpenter alongside the Cluster Autoscaler. They'll fight over the same nodes. The CA can add nodes that Karpenter then terminates for consolidation. Infinite loop.
Don't overfit to spot. Run spot only for services with external buffering. Messaging queues, ingest pipelines, stateless HTTP APIs that can retry. Anything with state beyond its pod's lifetime should stay on on-demand.
Don't forget about topology spread. If your workload is a distributed database, telling Karpenter to pack nodes across all AZs is fine. But forgetting that and letting Karpenter consolidate into two AZs could make your database lose quorum.
Don't set ttlSecondsAfterEmpty on nodes running batch jobs. Only set it for ephemeral batch pools — Karpenter terms nodes once pods empty, and if you have a steady-state workload it will churn nodes all night.
Don't ignore startup times. Karpenter provisions nodes by calling the EC2 API. Your node boot takes two to three minutes. If you're responding to traffic spikes that happen in seconds, this latency is unacceptable. Use a buffer of reserved instance capacity when spikes are likely.
FAQ
What's the difference between Karpenter and Cluster Autoscaler?
Karpenter provisions nodes from all instance types based on pod requirements and consolidates down to smaller instances when utilization drops. Cluster Autoscaler adds nodes to node groups or removes empty nodes. It can't bin-pack workloads into smaller nodes or swap instance types at runtime, nor does it support spot smoothly.
Does Karpenter replace node groups entirely?
For most workloads, yes. EKS requires the "default" node group for system pods (kube-proxy, CoreDNS, CNI), but all your application workloads can run on Karpenter-managed nodes.
Will Karpenter work on GKE or AKS?
Karpenter v1+ runs on AKS (Azure) and works with GKE via node auto-provisioning, though neither is as mature as AWS implementation in terms of spot integration and cost API coverage.
Is Karpenter free to use?
Yes, it's a CNCF project that runs in your cluster. You pay for the instances it provisions. Karpenter doesn't have paid tiers. The operational cost is whatever your team spends on Kubernetes operations, which mostly comes in the form of time invested debugging workload schedule.
Does Karpenter work with Kubernetes 1.32+?
Yes. The project had some churn around the v1 API updates starting in 2025, but by mid-2026 v1.4 of Karpenter supports Kubernetes 1.30 through 1.33 without compatibility issues, per the project's compatibility matrix.
What's the fastest way to see savings?
Audit and adjust your resource requests first — that's the biggest win. Migrate a stateless, low-risk namespace to a Karpenter NodePool with spot enabled and consolidation enabled and observe for two weeks. Then roll it out wider if latency, error rates, and utilization stay healthy.
The Conclusion
Kubernetes node provisioning cost savings karpenter are real — 30-50% largely achievable — but only after you fix your resource requests and understand your workload resilience.
If I'm strip it down to three actions:
- Get your requests right. VPA recommendations for one week. Adjust manually.
- Run Karpenter with strict budgets: 10% disruption cap, spot enabled only for stateless services, no consolidation for GPU nodes serving production traffic.
- Track actual vs requested utilization for at least a month before you declare victory. The numbers will move in ways that surprise you.
AI workloads deserve special treatment, yes, but they don't deserve different orchestration. You just need to tame Karpenter differently. In 2026 I'm pretty comfortable saying this: cloud providers also agree — EKS now ships with Karpenter as the default recommended node autoscaler.
Your cluster might cost less next month. But you have to make the first move.
Nishaant Dixit — Founder of SIVARO. Building data infrastructure and production AI systems since 2018. Built systems processing 200K events/sec.