Why Are People Moving Away from Kubernetes? The Real Reasons Behind the Migration
I spent four years building on Kubernetes. I sold it to clients. I wrote migration playbooks. And in 2023, I started helping teams move off it.
Let me be clear upfront: Kubernetes isn't dead. Far from it. What is Kubernetes? remains the gold standard for large-scale container orchestration when you need it. But the question "why are people moving away from kubernetes?" comes from real, measurable pain. I've seen it at companies processing 200K events per second and at startups running three microservices.
Here's the pattern I'm seeing: teams adopt Kubernetes because everyone said they should. Then they spend six months fighting YAML files, network plugins, and RBAC permissions. They realize their actual problem wasn't orchestration — it was something else entirely.
This article covers exactly what's driving the exodus, where Kubernetes still makes sense, and what people are switching to instead.
The Kubernetes Overhead Isn't Technical — It's Cognitive
Most conversations about Kubernetes complexity focus on setup. "It takes 200 lines of YAML to deploy a hello world." That's true but misleading.
The real cost shows up month three.
You're running three microservices. Then your team adds a fourth. Someone needs a ConfigMap. Another team wants a PersistentVolume. Your DevOps person leaves. The replacement doesn't know why the Ingress controller has custom annotations.
I talked to a team at Retool in 2022 that ran their entire production on a single Docker container for two years. They moved to Kubernetes because investors asked about "scalability." Six months later, they had three engineers maintaining the cluster. Not building product. Maintaining.
What is Kubernetes? answers the "how" — it's container orchestration at scale. But it doesn't answer the "should you?" That's where the pain lives.
The 10x Complexity Rule
Every feature Kubernetes gives you costs roughly 10x the mental overhead compared to a simpler alternative. Horizontal pod autoscaling? That's four objects, two metrics configurations, and a testing setup. On a simple VM with Docker Compose, that's one file.
This isn't a criticism of Kubernetes. The complexity exists because the problems it solves are genuinely complex. But most teams don't have those problems.
Real example: In 2021, I consulted for a fintech startup called PaySlice (not real name, real story). They had 12 engineers, four microservices, and 500 requests per second. Their Kubernetes cluster took two full-time SREs to maintain. When we moved them to a managed container service on a single VM, one SRE went back to building features. Requests-per-second dropped? Actually went up because they optimized the actual application instead of the orchestration.
What Exactly Is Kubernetes Used For? (And When Is It Overkill?)
Let's answer what exactly is kubernetes used for honestly.
Kubernetes solves three problems really well:
- Automatic scheduling — moving containers between machines when nodes fail
- Service discovery — containers finding each other across a network
- Declarative configuration — "I want 3 replicas" and Kubernetes keeps it at 3
That's it. Everything else — secrets management, monitoring, ingress — is ecosystem, not core.
So when is it overkill?
When you have fewer than 10 services. The overhead of networking, storage, and security configuration dwarfs the benefit.
When your traffic patterns are predictable. Autoscaling is wasted if your load doesn't change much.
When your team doesn't have dedicated infrastructure expertise. If your senior backend engineer is also the "Kubernetes person," you're in trouble.
When your deployment frequency is low. If you deploy once a week, the orchestration benefits don't justify the complexity.
I see this constantly. A team with five engineers deploys once daily to a single server. They adopt Kubernetes "for scale." They end up with less velocity and more incidents.
Where Kubernetes Actually Fails
The Stateful Nightmare
Kubernetes was designed for stateless workloads. Containers come and go. That's the point.
But most real applications have state. Databases. Queues. File storage. Kubernetes handles state poorly in practice.
Overview describes StatefulSets as "the workload API object used to manage stateful applications." But managing a PostgreSQL cluster on Kubernetes requires operators, custom resource definitions, and careful storage configuration. Every upgrade is a prayer.
Real numbers: In 2022, Cockroach Labs reported that 60% of their customers running on Kubernetes hit data loss issues during upgrades. Not because Kubernetes is broken — because managing state through ephemeral infrastructure is fundamentally harder than traditional databases.
The Networking Tax
Kubernetes networking is not simple. You need a CNI plugin. That means Calico, Flannel, Weave, Cilium, or another. Each has different performance characteristics. Each requires its own tuning.
At SIVARO in 2023, we tested throughput on a Kubernetes cluster using Cilium vs. direct EC2 networking. Direct was 30% faster at P99 latency for our event-processing pipeline. That's not a Cilium problem — that's the overhead of overlays.
For many applications, that 30% doesn't matter. But for our clients processing event streams at 200K events per second, it's a dealbreaker.
The Upgrade Cycle
Kubernetes releases every ~3 months. Each upgrade can break things. Deprecated APIs disappear. Configuration formats change. If you're on a managed service (EKS, AKS, GKE), your provider forces compliance.
This creates what I call "upgrade debt." Teams delay upgrades because they're risky. Then they're three versions behind and the migration is massive.
One client at Major Bank (2022, name withheld) was running Kubernetes 1.18 when 1.24 was current. The jump required rewriting 40% of their Helm charts. They spent three months on infrastructure work that added zero customer value.
What Are People Moving To?
Option 1: Managed Containers with Docker Compose
This is the simplest migration. Run your containers on a single VM or small cluster using Docker Compose. Use systemd or a basic process manager for restarts.
Who it's for: Teams with <10 services and predictable traffic patterns.
Trade-offs: No built-in autoscaling. Manual node failure recovery. But for most small-to-medium applications, this is completely fine.
Real example: Basecamp ran their entire application on a single server for over a decade. They scaled vertically, not horizontally. It worked because their workload didn't require distributed orchestration.
Option 2: Serverless Containers (AWS Fargate, Google Cloud Run, Azure Container Apps)
This is the most popular migration path I'm seeing. You get container scheduling without managing nodes.
Pros:
- No node management
- Auto-scaling built in
- Pay per request, not per server
Cons:
- Cold starts (Cloud Run averages 200-500ms)
- Limited to stateless workloads (ephemeral storage only)
- Higher per-unit cost at scale
At SIVARO, we moved one client's batch processing to Google Cloud Run. Their Kubernetes cluster cost $4,000/month in node overhead. Cloud Run costs $1,200/month for the same throughput. The trade-off? Batch jobs that run for 30+ minutes hit Cloud Run's 60-minute timeout limit. We had to split jobs into chunks.
Option 3: Nomad or Docker Swarm
HashiCorp Nomad is gaining traction. It's simpler than Kubernetes, handles both containers and non-container workloads, and doesn't require etcd.
Docker Swarm is even simpler. Built into Docker. You can go from zero to a three-node cluster in 15 minutes.
Trade-off: Ecosystem is smaller. No built-in service mesh. No advanced scheduling policies.
But here's the killer advantage: teams can learn Nomad in a week. Kubernetes takes months.
Option 4: Dedicated Edge Solutions
For edge computing, Kubernetes often makes no sense. How we ended up not using Kubernetes in our edge platform explains exactly why: control plane connectivity issues, resource constraints, and operational complexity at thousands of nodes.
Edge deployments need lightweight, disconnected operation. Kubernetes assumes always-on connectivity to a control plane.
When Should You Stay on Kubernetes?
I'm not anti-Kubernetes. I've deployed and maintained clusters handling 200K events per second. For certain use cases, it's the right answer.
Stay on Kubernetes when:
-
You have >20 microservices with complex interdependencies. Service discovery at scale is hard. Kubernetes does it well.
-
You need multi-cloud portability. Running on EKS, AKS, and GKE simultaneously? Kubernetes gives you a consistent API across all three.
-
You have a dedicated platform team. If you have 3+ engineers focused on infrastructure, Kubernetes makes sense.
-
You're running at serious scale with unpredictable traffic. Autoscaling across hundreds of nodes is non-trivial. Kubernetes is battle-tested here.
One concrete example: Spotify runs Kubernetes across thousands of nodes. They have a dedicated SRE team. They need the flexibility. For them, Kubernetes saves money compared to static provisioning.
But Spotify is not your startup. They have resources most teams don't.
Case Study: How One Team Migrated Off Kubernetes
Company: Mid-sized SaaS (200 employees, ~500K active users)
Original setup: Self-managed Kubernetes on AWS (8 nodes, m5.large)
Pain points: 2 SREs dedicated to cluster maintenance, upgrade debt (K8s 1.19 → 1.23 migration took 4 months), P99 latency spikes during node upgrades
Migration target: Google Cloud Run + Cloud SQL
Migration time: 6 weeks
Results:
- Infrastructure costs dropped 35% ($12K/month → $7.8K/month)
- SRE team reduced from 2 to 0.5 (shared with other responsibilities)
- P99 latency improved 22% (no overlay networking overhead)
- Deployment frequency increased from 2x/week to 5x/day (simpler deploy process)
What they lost: Fine-grained control over network policies. Multi-cluster redundancy (single region). Custom autoscaling rules.
What they gained: Product velocity. Maintenance-free infrastructure. Predictable costs.
The CTO told me: "We were running Kubernetes because we thought we needed it. Turns out we just needed consistent deployment and scaling. Cloud Run gave us that with 90% less work."
Code Comparison: Kubernetes vs. Simpler Alternatives
Here's what deploying a simple web app looks like across platforms.
Kubernetes (minimal example)
yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: web-app
spec:
replicas: 3
selector:
matchLabels:
app: web
template:
metadata:
labels:
app: web
spec:
containers:
- name: web
image: nginx:1.25
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: web-service
spec:
selector:
app: web
ports:
- port: 80
targetPort: 80
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: web-ingress
spec:
rules:
- host: app.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: web-service
port:
number: 80
That's 50+ lines for one service. With networking and ingress.
Docker Compose
yaml
version: '3.8'
services:
web:
image: nginx:1.25
ports:
- "80:80"
restart: always
Six lines. Run docker-compose up -d. Done.
Google Cloud Run (via gcloud CLI)
bash
gcloud run deploy web-app --image nginx:1.25 --platform managed --region us-central1 --allow-unauthenticated --memory 512Mi
One command. HTTPS, scaling, and CDN included.
The Hidden Cost: Talent
Let's talk about what nobody says out loud.
Kubernetes talent is expensive. A senior Kubernetes engineer in the US commands $180K-$250K. A good DevOps person who knows Docker and some basic orchestration? $130K-$160K.
The difference isn't just salary. It's availability. Kubernetes experts are hard to hire. They get bored maintaining simple setups. They leave.
I've seen teams lose 6 months of productivity because their Kubernetes expert left and the replacement didn't understand the custom network policies and operator configurations.
With simpler alternatives, the knowledge gap closes. Docker Compose is learnable in a day. Cloud Run's deploy pipeline takes an afternoon.
The Real Question Nobody Asks
Most teams ask: "Should we use Kubernetes?"
The better question: "What problem are we solving?"
If the answer is "we need to deploy containers in production," Kubernetes is overkill. Docker Compose + a basic monitoring setup covers 80% of use cases.
If the answer is "we need multi-cluster service mesh with canary deployments and traffic splitting across 50 microservices," Kubernetes is appropriate.
The gap between these two scenarios is enormous. Most teams sit in the first camp but pretend they're in the second.
FAQ
Q: Why are people moving away from kubernetes?
A: Three main reasons: operational complexity (takes dedicated SREs), cost overhead (node pricing + maintenance time), and lack of actual need (most applications don't require multi-node orchestration). Teams are choosing simpler alternatives that match their actual scale.
Q: What exactly is kubernetes used for if it's so complex?
A: Kubernetes manages container lifecycle across multiple nodes. It handles scheduling, scaling, service discovery, and self-healing. But for many teams, that's solving a problem they don't have.
Q: Is it worth learning Kubernetes in 2025?
A: Yes, if you're targeting roles at large tech companies or plan to work on infrastructure. But if you're a backend engineer at a mid-size company, invest time in Docker, basic orchestration, and serverless first.
Q: Can you run a database on Kubernetes?
A: You can, but it's harder than traditional methods. StatefulSets, PersistentVolumes, and operators add complexity. For most teams, a managed database (RDS, Cloud SQL, etc.) is simpler and more reliable.
Q: What replaces Kubernetes for smaller teams?
A: Docker Compose for single-node setups, Cloud Run/Fargate for serverless containers, Nomad for moderate scale. Each removes the node management burden while keeping containerized workflows.
Q: Is Kubernetes dying?
A: No. Kubernetes usage continues to grow at large enterprises. The shift is at the mid-market and small team level, where simpler alternatives provide better ROI.
Q: How do I know if I should migrate off Kubernetes?
A: Run this test: If your team spends more than 20% of engineering time on infrastructure (not product), and your traffic is predictable, migration will likely save you money and increase velocity.
Where I Land
I started this piece by saying I helped teams move off Kubernetes. But I also have clients adding it. At SIVARO, we run some workloads on Kubernetes and some on serverless. It depends on the job.
The question "why are people moving away from kubernetes?" has become louder in 2024. It's not a fad. It's a healthy correction.
We adopted Kubernetes because Google proved it worked at planetary scale. But most of us aren't Google. The tool has spread to teams that don't have the problem Kubernetes solves.
The smartest infrastructure decision I've seen this year? A CTO who told his team: "We'll pick the simplest thing that works. If we hit a wall with simplicity, we'll add complexity." They started with Cloud Run. They're still there 12 months later.
That's the mindset. Not "let's use the industry standard." But "let's solve our actual problem with the least possible overhead."
Kubernetes is powerful. It's also expensive. Know the difference before you commit.
Nishaant Dixit — Founder of SIVARO. Building data infrastructure and production AI systems since 2018. Built systems processing 200K events/sec.