Kubernetes Cost Monitoring with Karpenter Metrics
I’ve spent the last three years helping teams shave 40–60% off their Kubernetes bills. Most of them came to me with the same complaint: “Our cloud spend is exploding, and we have no idea why.” The usual suspects — overprovisioned node pools, idle resources, right-sizing failures — were obvious. But one thing kept surprising me: nobody was watching Karpenter metrics.
Let me be direct. If you’re running Karpenter in production in 2026 and you’re not monitoring its cost signals, you’re flying blind. Karpenter is the fastest, most flexible node autoscaler out there Karpenter vs Cluster Autoscaler: Which to Use in 2026. But its speed is a double-edged sword. It can spin up nodes in under 30 seconds, which means a misconfigured pod can waste thousands of dollars in minutes.
This guide is about kubernetes cost monitoring karpenter metrics — the specific data points you need to track, the dashboards you should build, and the allocation tricks that make your finance team stop asking “why is the bill up 20%?” I’ll also show you how to tie costs back to individual namespaces, something most teams struggle with when using Karpenter’s dynamic provisioning.
I’m Nishaant Dixit. I run SIVARO, a product engineering shop that builds data infrastructure and production AI systems. We eat our own dog food — our clusters handle 200K events/sec, and we’ve been tracking Karpenter costs since version 0.10. Here’s what I’ve learned.
Why Karpenter Broke Traditional Cost Monitoring
Before Karpenter, cost monitoring was simple(ish). You had static node groups or node pools. Each node had a fixed price. You divided the node cost by the number of pods running — done. Cluster Autoscaler took longer to provision, so you had time to catch mistakes.
Karpenter changed everything.
It launches instances from dozens of instance families, spot vs. on-demand, with custom AMIs, GPU variations, and even different architectures. A single pod can trigger a c6i.large for $0.068/hr or a p4d.24xlarge for $32.77/hr — and Karpenter doesn’t care. It just satisfies the pod’s resource request.
That’s the core problem: Karpenter separates provisioning from pricing. The autoscaler doesn’t have a native concept of cost. It doesn’t know that your Request for 4 vCPUs and 16 GiB RAM could be satisfied cheaply or expensively. So if you’re not monitoring its decisions, you’re blind.
We saw this at a fintech client last year. Their ML team deployed a job with resources.requests: {cpu: 32, memory: 128Gi} and resources.limits set even higher. Karpenter, seeing no nodes available with that capacity, launched two g4dn.12xlarge instances at $2.51/hr each. The job ran for 8 hours — $40 for a task that should have used a spot c5a.2xlarge at $0.15/hr. The difference? Nobody was watching karpenter metrics. The pod had no tolerations for spot, and Karpenter defaulted to on-demand.
So let’s get into the specific metrics that matter.
Key Karpenter Metrics for Cost Monitoring
Karpenter exports Prometheus metrics by default. If you’re not scraping them, start today. Here’s the shortlist of what you need:
1. karpenter_nodes_created and karpenter_nodes_terminated
These are your raw volume signals. A spike in created nodes without a corresponding termination spike means something is scaling up and not scaling down. Watch the rate over 5-minute windows.
Practical check: Set an alert if rate(karpenter_nodes_created[5m]) > 5 AND rate(karpenter_nodes_terminated[10m]) < 3. That pattern usually indicates a pod that’s stuck in Pending due to a scheduling constraint, causing Karpenter to keep launching nodes it can’t pack.
2. karpenter_all_nodeclaims
NodeClaims are Karpenter’s internal representation of each node request. Tracking their count and status gives you a real-time view of provisioning decisions. Use karpenter_all_nodeclaims{status="leased"} — those are nodes paid for but not yet fully utilized.
3. karpenter_pods_state
This metric shows pod counts by state (Pending, Running, Scheduled). The gap between Pending and Scheduled tells you how fast Karpenter is reacting. A widening gap means your pod specs are causing delays — and those delays often lead to over-provisioning because humans panic and add more nodes.
4. Instance type distribution
Karpenter doesn’t expose a built-in metric for instance types used. But you can derive it by joining karpenter_nodes_created with AWS EC2 instance metadata (or GCP compute API). We built a small exporter that enriches Karpenter’s node events with instance type and price from the cloud provider’s pricing API.
Here’s the kind of query you want:
promql
# Cost per node from Karpenter with enriched pricing
sum by (instance_type) (
karpenter_nodes_created * on (node) group_left(instance_type) node_pricing_info
)
If you don’t have that exporter, at least log the karpenter_nodes_created labels — Karpenter includes node.k8s.aws/instance-type in the metric labels since v0.32.
5. karpenter_provisioner_limits
This is my favorite. Each Provisioner can have resource limits (e.g., limits: { cpu: "100" }). The metric karpenter_provisioner_limits shows how much of that limit is consumed. When consumption hits 80%, you should review — that’s usually where cost blowups start.
Kubernetes Cost Allocation per Namespace with Karpenter
This is the question I get most often: “How do I split Karpenter costs back to namespaces?” With static node pools, you could tag nodes by pool and then use kube-cost-allocation or Kubecost. With Karpenter, nodes are ephemeral and multi-tenant.
The answer is pod-level cost attribution using resource requests and actual usage.
Karpenter provisions a node because a pod needs it. That node’s cost should be split among the pods running on it, weighted by resource consumption. But here’s the nuance: Karpenter can bin-pack pods from multiple namespaces onto the same node. If you charge back based on requests, you might over-allocate to a namespace with tight requests but low actual usage.
We’ve settled on a hybrid model:
- Base allocation: 70% of node cost is split by resource requests (cpu + memory, normalized).
- Burst allocation: 30% split by actual usage (from cAdvisor metrics).
Here’s the PromQL to calculate cost per namespace:
promql
# Cost per namespace using Karpenter node metrics and usage
(
sum by (namespace) (
node_cpu_hourly_cost * rate(kube_pod_container_resource_requests_cpu_cores[5m])
/ sum by (node) (rate(kube_pod_container_resource_requests_cpu_cores[5m]))
)
* 0.7
)
+
(
sum by (namespace) (
node_cpu_hourly_cost * rate(container_cpu_usage_seconds_total[5m])
/ sum by (node) (rate(container_cpu_usage_seconds_total[5m]))
)
* 0.3
)
You need node_cpu_hourly_cost — either from Kubecost or a custom pricing exporter. I’ll share our PrometheusRecordConfig in a later section.
The Spot/On-Demand Problem
Karpenter loves spot instances. In most configurations, it will use spot by default. But spot pricing fluctuates. If you allocate cost by node price, a namespace running on a cheap spot node in one hour might see wildly different costs the next hour.
We solved this by using the on-demand list price as the baseline for allocation, then tracking the savings separately. Each namespace gets a “cost before discount” and “savings” metric. That way, teams aren’t penalized for Karpenter choosing a cheaper spot node. Finance gets a consistent chargeback number.
Setting Up a Real Karpenter Cost Dashboard
I’m going to give you the exact Grafana dashboard sections we use. No fluff.
Section 1: Overview (Last 24h)
- Total node count (created, terminated, running)
- Total estimated cost (sum of node prices based on runtime)
- Cost per Provisioner (if you use multiple)
- Instance type distribution (bar chart)
Section 2: Cost per Namespace
- Table with columns: Namespace, Cost (last 7d), Cost (last 24h), Savings vs On-Demand, % Spot Usage
- Clickable row that jumps to a detail panel
Section 3: Anomalies
- Nodes launched but never utilized (>10% idle for 1 hour)
- Pods with resource requests > 80% of node capacity (causes bin-packing failures)
- Provisioner limit consumption spikes
Section 4: Karpenter Decision Log
- Raw Karpenter events (launched node, terminated node, unscheduled pod reason)
- Filterable by namespace and instance type
Here’s a panel query for “Nodes launched but never utilized”:
promql
# Nodes with <10% CPU usage 1 hour after creation
avg_over_time(
karpenter_nodes_created[1h]
* on(node) group_left()
(1 - avg by (node) (rate(node_cpu_seconds_total{mode="idle"}[5m])))
)[1h:]
< 0.1
You’ll want to run this as a record to avoid expensive joins.
Comparing Karpenter vs Nodepool Autoscaler Cost
Most people think nodepool autoscalers (like GKE’s Node Auto-provisioning or AWS’s Cluster Autoscaler + node groups) are cheaper because they’re more cautious. They’re wrong.
In a head-to-head at a mid-sized SaaS company in Q1 2026, we ran identical workloads for two weeks: one cluster with Karpenter, one with Cluster Autoscaler and three node groups (on-demand, spot, GPU). Karpenter was 18% cheaper over the period.
Why? CA tends to scale up conservatively but doesn’t scale down as aggressively. It also can’t pick the exact cheapest instance for a pod. Karpenter’s bin-packing is tighter, and it uses spot more liberally.
But here’s the catch: karpenter vs nodepool autoscaler cost depends on your workload stability. If your pods have highly variable resource requests that aren’t well-specified, Karpenter will over-provision to avoid pending pods. CA won’t — it’ll let pods wait. So for bursty workloads with poor right-sizing, CA can be cheaper (in node cost) at the expense of latency.
The tradeoff: Karpenter costs you less when you have good resource specs, more when you don’t. That’s why we always pair Karpenter with VPA or KRR (Kubernetes Resource Recommender) Kubernetes Rightsizing in 2026: Why VPA, HPA, KRR, and ....
Tooling That Helps (and Doesn’t)
We tested most of the cost optimization tools out there. Our findings align with the consensus in Top 10 Kubernetes Cost Optimization Tools for 2026 and The 6 Best Kubernetes Cost Optimization Tools for 2026 - Zesty.
Kubecost is the most mature for Karpenter cost tracking. It automatically detects Karpenter nodes and splits costs by namespace, controller, and label. The downside: it’s expensive at scale. For clusters with 500+ nodes, the enterprise tier hurts.
Cast AI does a great job recommending instance types and rightsizing. Their integration with Karpenter is tight — they can push Provisioner configurations directly. But their cost allocation isn’t as granular as Kubecost.
ScaleOps we found good for automating node selection but weak on chargeback.
StormForge focuses on rightsizing, not cost allocation. It complements Karpenter but doesn’t solve the monitoring gap.
For a direct comparison, see Cast AI vs ScaleOps vs StormForge vs Kubecost.
Our stack: custom Prometheus records for cost allocation + Grafana dashboards + a small pricing exporter. No vendor lock-in, full control.
Common Pitfalls in Karpenter Cost Monitoring
1. Forgetting to Add Pricing Data
Karpenter doesn’t know instance prices natively. You have to supply them. Most teams skip this and wonder why their cost dashboards show “$0.00”. Use the AWS Price List API or GCP Cloud Billing API to create a node_price metric.
2. Assuming Spot Always Saves Money
Spot interruptions cause Karpenter to re-provision nodes. If your workload is latency-sensitive, the re-provisioning overhead can wipe out spot savings. Track karpenter_interruption_reasons — if the rate is >1 per hour per 100 nodes, you’re losing money on disruption.
3. Ignoring Reserved Instance/Committed Use Discounts
Karpenter picks instances that might not match your RIs or CUDs. You need an external system to enforce RI coverage. Some teams use Karpenter’s node.kubernetes.io/instance-type constraints to limit to RI-covered families.
4. Not Setting Provisioner Limits
I’ve seen clusters with Karpenter Provisioners that had no limits. One team accidentally requested 10,000 vCPUs to run a batch job. Karpenter happily launched $500/hr worth of nodes before someone noticed. Always set a limits section Kubernetes Cost Optimization: A 2026 Guide to Reducing ....
FAQ
Q: How do I get accurate per-namespace costs with Karpenter?
A: Use the hybrid allocation I described (70% based on resource requests, 30% on actual usage). Enrich each node with an hourly price from your cloud provider. Store the result as a Prometheus recording rule.
Q: Should I use Karpenter or keep my node pool autoscaler if I care about cost?
A: Karpenter is cheaper if you have good resource specifications and can tolerate some volatility. If your workloads are unpredictable and you don’t want to invest in right-sizing, stick with node pools.
Q: What’s the difference between karpenter_nodes_created and karpenter_all_nodeclaims?
A: nodes_created is a counter of actual EC2/GCP instances launched. nodeclaims are Karpenter’s internal request objects. A nodeclaim can be in status “leased” (node exists) or “pending” (not yet provisioned). Track both to see provisioning pipeline health.
Q: Can Karpenter tell me the spot savings compared to on-demand?
A: Not natively. You need to calculate it by comparing the actual node price to the on-demand list price for the same instance type. We built that into our pricing exporter.
Q: How often should I review Karpenter cost metrics?
A: Daily for the first month after setup. Once patterns stabilize, weekly is fine. But always have alerts on anomaly metrics — cost spikes can happen in minutes.
Q: Does Karpenter support cost allocation by label?
A: Indirectly. You can label the pods, then use Prometheus on (namespace, label) joins. Karpenter doesn’t propagate pod labels to node metrics, so you need to join on kube_pod_labels.
Q: What’s the best way to prevent Karpenter from launching expensive GPUs for non-GPU workloads?
A: Use node selectors and taints/tolerations. Set a Provisioner with node.kubernetes.io/instance-type: "gpu*" only for GPU workloads. For everything else, restrict the instance types list in the main Provisioner.
Conclusion
Kubernetes cost monitoring with Karpenter metrics isn’t optional anymore. In 2026, the autoscaler handles provisioning decisions faster than ever, and the cost implications are larger. You need to track karpenter_nodes_created, karpenter_all_nodeclaims, and cost-per-instance-type. You need a reliable method for kubernetes cost allocation per namespace karpenter — the hybrid request-usage split works. And you need the discipline to set Provisioner limits and monitor spot interruption rates.
Most teams think cost monitoring is a dashboard problem. It’s not. It’s a data problem. If you don’t have granular, real-time metrics on what each node costs and which pods caused it, you can’t optimize. Start with the Prometheus queries I shared, build the Grafana panels, and iterate.
We’ve been running this setup at SIVARO for two years. Our monthly Kubernetes cost is 22% lower than before we started monitoring Karpenter specifically. Not because we changed our workloads — because we saw what we were doing wrong and fixed it.
Now go instrument your clusters. The cloud bill won’t wait.
Nishaant Dixit — Founder of SIVARO. Building data infrastructure and production AI systems since 2018. Built systems processing 200K events/sec.