SIVARO
How to Reduce Cloud Infrastructure Costs in 2026
Two weeks ago I sat in a boardroom in Austin watching a CFO scroll through a Datadog bill. $412,000 for August. The CTO next to me kept saying "but our traffic only doubled." That's the moment I realized most teams don't have a cost problem — they have an architecture problem dressed up as a pricing problem. I've spent the last eight years building data infrastructure and production AI systems at SIVARO, and I've watched how to reduce cloud infrastructure costs go from a quarterly cleanup exercise to a weekly engineering discipline. This guide compares your real options — reserved instances, spot fleets, serverless, containers, self-hosted GPUs — with numbers, trade-offs, and the version of the truth the vendors won't tell you.
The Billing Problem Most Teams Misdiagnose
Here's what nobody says out loud: your cloud bill is a symptom, not a disease.
At first I thought cost overruns were a tagging discipline problem. Tag everything, build chargeback dashboards, shame the teams. Turns out tagging just shows you where the money went — it doesn't stop it from leaving. The actual fix lives one layer down in architecture decisions you made eighteen months ago and never revisited.
The three biggest silent cost killers I see in 2026:
Idle compute at the wrong granularity. A Kubernetes cluster sized for Black Friday running at 12% utilization on a Tuesday. You're paying for peak capacity 8760 hours a year.
Egress you didn't model. Cross-AZ traffic, NAT gateway charges, and model inference traffic between regions. AWS NAT Gateway alone runs $0.045/GB processed. I've seen a single misconfigured service burn $18K/month just talking to itself across availability zones.
Storage tier rot. S3 Standard buckets holding logs from 2023. That data costs $0.023/GB/month. Glacier Deep Archive is $0.00099/GB/month. Twenty-three times cheaper. Nobody moves it because nobody owns it.
The instinct is to call the vendor and negotiate. Don't. Negotiation gets you 5-8%. Architecture gets you 60%.
What Is the Most Cost Efficient Architecture for LLM Inference?
Let me be blunt: most teams running LLM inference in the cloud are burning 4-10x more money than necessary.
I've deployed inference stacks for four companies since early 2024. The pattern repeats. Somebody wires up an OpenAI or Anthropic API call, it works beautifully for the prototype, and then production traffic hits and the invoice explodes.
Here's the real decision tree.
Managed API (OpenAI, Anthropic, Bedrock). Zero ops. Pay per token. For low-volume, spiky, or prototype workloads, this wins. Full stop. A company I worked with in February 2026 was spending $340/month on GPT-4 class inference for a support bot. Building infrastructure for that would cost more than the API bill.
Serverless GPU (Modal, RunPod Serverless, Replicate). Pay per second of GPU use. Cold starts hurt. Fine for bursty batch jobs, terrible for interactive latency-sensitive chat.
Reserved GPU instances (AWS p5, GCP A3, Lambda Labs). Fixed monthly cost, you own the utilization problem. Great if you can keep utilization above 60%. Disaster below 30%.
Self-hosted with vLLM or SGLang on spot instances. This is where the real savings live for steady-state workloads. We ran a Llama 3.3 70B setup on spot H100s for a client in May 2026 and cut their inference bill from $47K/month to $9,800/month.
The math that matters:
python
# Rough monthly cost comparison for 1B tokens/month of 70B-class inference
managed_api = 1_000_000_000 * 0.0000008 # ~$0.80 per 1M tokens blended
# = $800
reserved_h100_aws = 8 * 3200 # 8x p5.48xlarge reserved, 1yr
# = $25,600
spot_h100_vllm = 8 * 3200 * 0.35 * 0.75 # spot discount + 75% util
# = $6,720
serverless_gpu = 1_000_000_000 / 2500 * 0.0005 # tokens/sec * rate
# = $200... until you hit cold starts and retry storms
The "cheapest" number depends entirely on your utilization curve. If you can't keep GPUs above 50% utilized, self-hosting loses. That's the honest trade-off nobody posts about on LinkedIn.
Serverless vs Container Architecture: The Real Cost Conversation
Everyone frames this as a developer experience question. Wrong frame. Frame it as a cost-of-idleness question.
What is serverless architecture vs container architecture? Serverless means you pay per invocation and the platform handles scaling to zero. Containers mean you pay for provisioned capacity whether it's working or not.
The break-even is roughly 15-20% utilization. Below that, serverless wins. Above that, containers win, and they win big.
Real numbers from a March 2026 migration we did:
| Workload | Serverless (Lambda) | Containers (EKS) |
|---|---|---|
| API gateway, 2M req/mo, avg 200ms | $340 | $890 (min cluster) |
| Same API, 40M req/mo | $6,200 | $1,100 |
| Overnight batch ETL | $180 | $890 |
| Always-on WebSocket service | Disqualified | $890 |
See the pattern? Serverless dominates spiky and low-volume. Containers dominate steady-state and high-throughput. The mistake is picking one religion and applying it everywhere.
Another angle — cold starts have a cost. Not just latency. Every Lambda cold start means you're paying for Init duration. If your Node function takes 2.3 seconds to boot and you're doing 5M invocations/month, that's real money. We've seen teams shave 30% off Lambda bills just by trimming the deployment bundle.
yaml
# Bad: monolith zip with dev dependencies
# Good: tree-shaken, esbuild-bundled, layer-isolated
# Lambda init time: 2300ms -> 340ms after bundling
# Cost impact at 5M invocations/month: ~$1,200 saved
The hybrid play most mature teams land on: containers for the steady 80%, serverless for the spiky 20%. Route by endpoint, not by philosophy.
Reserved, Spot, and Savings Plans — What Actually Moves the Needle
I'm going to say something controversial: for most teams, Savings Plans are a trap.
Not always. But often. Here's why.
Savings Plans lock you into a dollar-per-hour commitment for 1-3 years. If your architecture changes — and it will, because AI workloads are reshaping infrastructure faster than any prior shift — you're paying for capacity you don't need.
The tier list, from my experience:
Spot instances. 60-90% discount. Interruption risk. Perfect for stateless workers, batch jobs, CI runners, and (importantly) LLM inference with proper request draining. This is the single highest-leverage lever available.
Reserved Instances, 1-year, no upfront. 30-40% discount. Boring but reliable for the 20% of your fleet that's genuinely stable.
Savings Plans. 20-30% discount with flexibility. Only makes sense if you have high confidence in your 3-year compute shape. In 2026, precious few teams do.
On-demand. You're paying 100% for optionality. Fine for spiky, expensive for steady.
The number that surprised me: spot adoption is still under 15% at most mid-market companies. People are scared of interruptions. They shouldn't be. With proper drain handling and multi-AZ diversification, we keep spot interruption rates under 0.5% on most workloads.
bash
# Karpenter spot config — diversify across instance families
apiVersion: karpenter.sh/v1beta1
kind: NodePool
spec:
template:
spec:
requirements:
- key: karpenter.sh/capacity-type
operator: In
values: ["spot", "on-demand"]
- key: node.kubernetes.io/instance-type
operator: In
values: ["m5.2xlarge", "m6i.2xlarge", "m6a.2xlarge", "m7i.2xlarge"]
disruption:
consolidationPolicy: WhenUnderutilized
expireAfter: 720h
That consolidation policy alone has cut Kubernetes bills 40-55% for clients who left the default settings.
The Storage and Egress Line Items Nobody Audits
Compute gets 70% of the attention. Storage and egress are where 30% of savings hide quietly.
S3 lifecycle policies. Turn them on. Set them today. Logs to Infrequent Access after 30 days, Glacier Instant Retrieval after 90, Deep Archive after 365. For a 500TB log bucket, this is a $9K/month swing.
RDS snapshots. Copy them to cold storage. Delete old automated snapshots. I've seen 4-year-old snapshots costing $2,400/month for a database that no longer exists.
Cross-AZ traffic. This is the sneaky one. Kubernetes by default spreads pods across AZs. If your service mesh routes every request between zones, you get $0.01/GB in each direction. A chatty microservices setup with 40TB/month of cross-AZ traffic costs $800/month for literally nothing.
Fix: zone-aware routing. Pin client and server to the same AZ where possible. Topology-aware hints in Kubernetes handle most of this.
Egress to the internet. $0.09/GB from AWS. If you're serving media or large payloads, put CloudFront in front. CloudFront egress is $0.085/GB but includes 1TB free tier monthly, and you get caching. For a media-heavy workload, that's a 40-70% reduction.
The one that actually shocked me: NAT Gateway costs. $0.045/GB processed + $0.045/hour per gateway. Teams with 100TB/month outbound through NAT are paying $4,500 just in processing fees. VPC endpoints for S3 and DynamoDB eliminate that for the two highest-volume services.
The Architecture Review That Actually Saves Money
Here's my honest take on how to reduce cloud infrastructure costs at the organizational level: quarterly architecture reviews beat monthly cost reviews.
Monthly cost reviews turn into blame sessions. Nobody wants to be the team that shows up on the leaderboard, so they quietly provision under and the service suffers.
Quarterly architecture reviews ask different questions:
- Is this service still on the right compute primitive for its current shape?
- What's our actual p50 and p95 utilization?
- What's blocking us from spot?
- Where are we paying for capacity that exists only because of a 2023 assumption?
We ran this for a Series C fintech in January 2026. Three questions on the second one — "what's blocking spot" — turned into $180K/year of savings in a single afternoon of reconfiguring node pools.
The pattern I've learned: cost reduction is not a project, it's a muscle. Build the muscle once and it stays. Try to do it as a one-time cost-cutting exercise and it grows back within six months.
Vendor-Specific Trade-offs Nobody Talks About
AWS is the most expensive on list price and has the deepest discounting. If you're over $500K/year, negotiate hard. AWS will move 20-30% on committed spend.
GCP has the best sustained-use discount program and the cleanest pricing for Kubernetes. GKE Autopilot removes the node management tax entirely.
Azure is the cheapest for Windows workloads and has aggressive licensing deals if you already have Microsoft agreements. For Linux-only shops, it's usually third place.
Cloudflare Workers and R2 are the disruptors. R2 has zero egress fees. For bandwidth-heavy workloads, moving from S3 to R2 has been a straight 60-80% savings on the storage and delivery line for two clients I've worked with. Workers compete seriously with Lambda for the edge use case.
Fly.io and Railway are worth mentioning for small teams. Not for scale, but for the "I have 4 engineers and I don't want to think about infrastructure" case. Monthly bills of $200-800 replacing what would cost $2-4K on a managed cloud with equivalent operational overhead.
The boring truth: the cheapest cloud is usually the one you already know how to operate. Migration costs, retraining time, and incident risk eat the savings in year one. Only migrate for a 3x-plus cost delta on a workload you understand.
FAQ
How much of my cloud bill is typically reducible?
From the audits I've run, 30-55%. The low end is teams already disciplined. The high end is teams who've never done an architecture review. Compute rightsizing and storage tiering get you to 25%. Spot adoption gets you to 45%. Architecture changes get you the rest.
Is it worth moving from AWS to a cheaper provider?
Usually no, unless your workload is bandwidth-heavy (then Cloudflare R2) or you're under 20 engineers (then a PaaS). Migration projects I've overseen take 4-9 months and rarely produce savings in year one.
What's the fastest single win?
Kill idle resources. Every AWS account has 15-30% of its spend on unattached EBS volumes, unused load balancers, orphaned RDS instances, and developer environments that haven't been touched in six weeks. Auto-shutdown scripts on dev environments alone typically save $2-8K/month for a mid-size team.
Does serverless actually save money?
Sometimes. For low-volume, spiky, or ephemeral workloads, yes — dramatically. For steady-state high-throughput APIs, containers are 3-5x cheaper. The honest answer is "it depends on your utilization curve," and anyone who tells you otherwise is selling something.
How do I handle GPU costs for AI workloads?
Three levers: batch aggressively, use spot for anything non-interactive, and match model size to task. A 7B model fine-tuned on your data often beats a 70B general model for your specific task at 1/10th the cost. Most teams over-provision model size out of FOMO.
What's the single biggest mistake you see?
Treating the cloud bill as a finance problem. It's an engineering problem. CFOs can't fix architecture. Finance dashboards just make you feel bad. Put an engineer with product context on cost ownership and it changes overnight.
When should I migrate off a managed LLM API to self-hosted?
When you're above roughly $8-12K/month of inference spend AND you have some baseline utilization floor. Below that, the operational overhead eats the savings. Above it, self-hosting with vLLM or SGLang on reserved or spot GPUs typically lands 60-75% cheaper.
Does multi-cloud reduce costs?
No. Multi-cloud increases costs 15-40% in my experience. The negotiation leverage story is mostly a myth. Only go multi-cloud for regulatory reasons or genuinely differentiated services.
The Uncomfortable Conclusion
The best way to reduce cloud infrastructure costs isn't a tool, a vendor, or a negotiation. It's a discipline of asking hard questions about your architecture every quarter, and being willing to rip out decisions you made when your business looked different.
I've watched teams save $2M a year by changing three node pool configurations. I've watched teams lose $400K by chasing a "cheaper cloud" that turned out to be more expensive once you added operational cost.
The pattern that works: measure utilization, challenge assumptions, use spot for anything interruptible, match compute to workload shape, and review architecture quarterly. Everything else is a detail.
If you're burning more than $50K/month and don't have a clear picture of why, that's the actual emergency. Not the number. The not knowing.
Nishaant Dixit — Founder of SIVARO. Building data infrastructure and production AI systems since 2018. Built systems processing 200K events/sec.