Why Are People Moving Away From Kubernetes?

I spent three years helping a fintech company run Kubernetes in production. By year four, we were migrating off it. Not because we couldn't make it work — but because the cost of keeping it working was eating our engineering budget alive.

Here's the awkward truth nobody at KubeCon will tell you: Kubernetes solves a problem most teams don't actually have yet. And for the teams that do have that problem, Kubernetes often creates five new ones.

Let me explain what I mean.

First — a quick refresher. What is Kubernetes? It's an orchestration platform for containerized applications. It schedules workloads, manages networking, handles scaling. At its core, what exactly is kubernetes used for? Running containers at scale across multiple machines. Kubernetes gives you deployment automation, service discovery, load balancing, self-healing.

That sounds great. Until you've been paged at 3 AM because a kubelet upgrade broke your CSI driver.

I'm Nishaant Dixit, founder of SIVARO. We build data infrastructure and production AI systems. We've deployed Kubernetes for clients. We've also helped teams migrate off Kubernetes. This article is about why that second group is growing.

The Complexity Tax Is Real

Let's start with the obvious one. Kubernetes is complicated. Not "a bit tricky" complicated. "Your team needs dedicated headcount just to keep it running" complicated.

Think about what a production-grade Kubernetes cluster actually needs:

A CNI plugin (Calico, Cilium, whatever)
An ingress controller
A service mesh (Istio, Linkerd)
Cert-manager
External DNS
A monitoring stack (Prometheus + Grafana + Alertmanager)
Logging (Loki, Elastic, or similar)
A storage provisioner
Secrets management (Vault, External Secrets Operator)
A GitOps tool (ArgoCD, Flux)

That's before you deploy your actual application.

I worked with a Series B company in 2023. They had 12 engineers. Three were dedicated to "Kubernetes operations." That's 25% of their engineering team. For infrastructure. Not features. Not product. Just keeping the cluster alive.

Why are people moving away from kubernetes? Because that math doesn't work for most businesses.

The "It's Just YAML" Lie

People say "Kubernetes is just declarative config." Sure. And a 747 is just a metal tube.

The reality: a typical microservice deployment involves 5-8 YAML files. Each has 40-60 lines. Each interacts with the others in ways you only discover during a production incident.

yaml
# A minimal deployment — and it's already complex
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-service
  namespace: production
spec:
  replicas: 3
  selector:
    matchLabels:
      app: api-service
  template:
    metadata:
      labels:
        app: api-service
    spec:
      containers:
      - name: api
        image: myapp/api:v2.1.3
        ports:
        - containerPort: 8080
        resources:
          requests:
            memory: "256Mi"
            cpu: "100m"
          limits:
            memory: "512Mi"
            cpu: "200m"
        env:
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: db-secret
              key: url
        livenessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 10
          periodSeconds: 5

This is one deployment, one service, one secret. A real system has 40 of these. Plus Helm charts. Plus Kustomize overlays. Plus whatever abstraction your platform team built on top.

The abstraction layers stack. Each one is a place where things break.

The Cost Surprise

Everybody talks about Kubernetes being free. The CNCF project is open source. Yes.

But nobody talks about the cost of running Kubernetes.

In 2024, we analyzed a client's monthly AWS bill. They were running three EKS clusters (dev, staging, prod). The Kubernetes-specific costs:

EKS control plane: $0.10/hour × 3 clusters × 730 hours = $219/month
Worker nodes (only counting overhead from system pods): 8 m5.large instances = $800/month
Load balancers for ingress: 3 ALBs at ~$25 each = $75/month
EBS volumes for etcd backups and logging: $100/month
NAT gateway for private subnets: $55/month

Total: ~$1,250/month just for Kubernetes infrastructure overhead. Not for their application. For the platform running the application.

And that's a modest setup. Add a service mesh and that number doubles.

A team at HashiCorp told me in 2022 their internal analysis showed each Kubernetes cluster cost about $3,000/month in operational overhead before a single application pod ran.

What does kubernetes actually do? It abstracts infrastructure. But abstraction doesn't come free. You're paying — in compute, in complexity, in cognitive load.

When Simple Solutions Beat Orchestration

Here's the contrarian take: most applications don't need Kubernetes.

I've seen this pattern at five different companies:

Start with a monolith on a VM or a single server. Works fine.
Grow. Decide to "modernize." Move to microservices + Kubernetes.
Everything gets slower. Deployments break. Developers are unhappy.
Realize that the monolith was handling 10,000 requests/second just fine.
Migrate back to something simpler. Or keep Kubernetes but only for the one service that actually needs it.

A healthcare startup I advised in 2023 had this exact arc. They moved a Rails monolith to 14 microservices on Kubernetes. Deploy time went from 2 minutes to 45 minutes. Developer velocity dropped 60%. They reverted in six months.

Their CTO told me: "We didn't need orchestration. We needed a better deployment pipeline for our monolith."

What Actually Needs Kubernetes?

Genuinely: large-scale, multi-team organizations running hundreds of microservices. Companies like Google, Spotify, Netflix. Teams where you need to isolate workloads, enforce resource quotas, and manage deployments across dozens of teams.

For everyone else? There's a simpler option.

I run several production systems on plain VMs with Docker Compose and systemd. They handle 50,000 requests/second with 99.99% uptime. No Kubernetes. No service mesh. No pain.

yaml
# A deployment with Docker Compose — handles most use cases
version: '3.8'
services:
  api:
    image: myapp/api:latest
    ports:
      - "8080:8080"
    environment:
      - DATABASE_URL=${DATABASE_URL}
      - REDIS_URL=${REDIS_URL}
    deploy:
      replicas: 3
      resources:
        limits:
          cpus: '0.50'
          memory: 512M
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
      interval: 30s
      retries: 3
  
  worker:
    image: myapp/worker:latest
    deploy:
      replicas: 2
    environment:
      - DATABASE_URL=${DATABASE_URL}
  
  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro

This handles 99% of what most teams need. Zero Kubernetes overhead.

The Skill Gap Problem

Here's a number that shocked me: in a 2023 survey by the CNCF, 67% of organizations said lack of Kubernetes expertise was their biggest barrier to adoption.

Translation: teams adopt Kubernetes, realize their engineers can't operate it, and either hire specialists (expensive) or deal with constant outages (painful).

I've seen the resume spike. "Kubernetes Administrator" roles paying $180k+ are common. But those people are rare. And they often burn out — because managing Kubernetes is a thankless job.

Why are people moving away from kubernetes? Because they can't hire the people to run it.

When You Don't Have a Platform Team

Kubernetes assumes you have a platform team. People who build things for other developers to use. Who maintain the cluster. Who upgrade control planes at 2 AM.

Most startups don't have this. Most mid-size companies don't either. They have backend engineers who "also handle DevOps."

Those engineers will tell you: Kubernetes is the thing they hate most.

Why? Because debugging Kubernetes issues requires deep knowledge of:

Linux kernel networking (CNI, iptables, eBPF)
Distributed systems theory (etcd consensus, leader election)
Container runtime internals (containerd, CRI-O)
Storage abstraction (CSI, PV/PVC lifecycle)
Identity and access management (RBAC, OIDC)

That's five specializations. One person rarely has three of them.

What Are People Moving To?

The alternatives aren't "worse" or "less sophisticated." They're just different trade-offs.

Nomad by HashiCorp

HashiCorp's Nomad is simpler. Single binary. Minimal configuration. Supports both containers and non-containerized workloads.

A client in 2022 moved from EKS to Nomad. Their infrastructure team went from 3 people to 1.5 (one full-time, one half-time). Deployment times improved. Developer happiness went up.

AWS ECS / Google Cloud Run

Fully managed container platforms. No control plane to manage. No upgrades. No etcd.

A SaaS company I consulted for moved 40 microservices from Kubernetes to AWS ECS Fargate. Their monthly AWS bill dropped 35%. Not because ECS is cheaper — because they didn't need three extra nodes for system workloads anymore.

Bare Metal / VMs with Systemd

This sounds primitive. It works incredibly well.

How we ended up not using Kubernetes in our edge platform — this article by Avassa describes exactly this pattern. They needed edge deployments where Kubernetes was overkill. They used simple orchestration with Docker Compose and custom tooling.

I do similar things at SIVARO. For AI inference workloads, we often run on bare metal with systemd units managing containerized services. It's fast. It's reliable. It takes one engineer to maintain.

yaml
# A systemd service unit for containerized AI inference
[Unit]
Description=AI Inference Service
After=docker.service
Requires=docker.service

[Service]
Restart=always
RestartSec=10
ExecStartPre=/usr/bin/docker pull myregistry/inference:latest
ExecStart=/usr/bin/docker run   --rm   --name inference   --gpus all   -p 8080:8080   -v /models:/models:ro   -e MODEL_PATH=/models/llama-3.1-8b   myregistry/inference:latest
ExecStop=/usr/bin/docker stop inference
ExecStopPost=/usr/bin/docker rm inference

[Install]
WantedBy=multi-user.target

One file. One service. Handles restarts, upgrades, everything. No YAML spaghetti.

The Kubernetes-For-Everything Trap

Most people think Kubernetes is the default choice. They're wrong.

The CNCF Landscape has over 2,000 projects. Visit any cloud provider's console and Kubernetes is presented as the natural way to run containers. Every conference talk assumes you're on Kubernetes.

But that's vendor interest masquerading as best practice.

What exactly is kubernetes used for in practice? A lot of teams use it because:

Everyone else does
Job postings require it
They think it's "production-grade"

None of these are good reasons.

I've walked into companies where:

10-person engineering team runs their 3 microservices on Kubernetes
Two engineers maintain the cluster (that's 20% of the team)
Their actual application code is 5,000 lines of Python
Their Kubernetes YAML is 3,000 lines

The math is insane. But nobody questions it because "that's how modern companies run."

The Maintenance Burden Never Ends

Kubernetes isn't "set and forget." It's "tend like a garden that's actively trying to kill you."

Every 3-4 months, a new Kubernetes version comes out. You need to upgrade. Or you lose support. Or you miss a security patch.

Upgrading the control plane
Updating the node images
Testing all add-ons against new versions
Validating custom controllers and operators
Updating Helm charts that depend on specific API versions

Each upgrade is a project. Each project takes 2-4 weeks for a moderate cluster.

I worked with a company in 2023 that was running Kubernetes 1.21 in production. The current version was 1.28. They were behind by 7 versions. They couldn't upgrade without breaking their custom operator. That operator's author had left the company. They had to rewrite it.

That's the hidden cost. Kubernetes moves fast. If you don't keep up, you're stuck.

When Kubernetes Makes Sense

I'm not anti-Kubernetes. I'm anti-Kubernetes-for-everything.

Kubernetes makes sense when:

You have multiple teams deploying independently
You need to enforce resource isolation across teams
You're running at massive scale (hundreds of services)
You have a dedicated platform team (3+ people)
Your application actually needs dynamic scaling

If you have 3 out of 5, Kubernetes might be right.

If you have 1 out of 5? Look elsewhere.

FAQ

Is Kubernetes dying?

No. Kubernetes is not dying. It's maturing. But the hype cycle is ending. Teams are becoming more selective about when to use it. The growth rate of new Kubernetes clusters has slowed since 2023. More teams are adopting it for specific use cases rather than as a default choice.

What is the easiest alternative to Kubernetes?

AWS ECS Fargate or Google Cloud Run for cloud-based workloads. Docker Swarm or Nomad for self-hosted. Plain VMs with Docker Compose for smaller applications. The right choice depends on your team size and workload complexity.

Can you run Kubernetes without a dedicated team?

Technically, yes. Practically, no. Not for production workloads. The maintenance burden — upgrades, security patches, incident response — requires dedicated attention. I've seen teams try. They either burn out or experience significant outages.

What does kubernetes actually do that simpler tools don't?

Kubernetes provides advanced scheduling, automatic scaling, service discovery, rolling updates, and resource management across a cluster. Most teams only use 20% of these features. The other 80% adds complexity without benefit.

Why are people moving away from kubernetes in edge computing?

Edge deployments have constraints that Kubernetes doesn't handle well: limited resources, unreliable networks, infrequent updates. How we ended up not using Kubernetes in our edge platform explains this perfectly. Simpler tools work better when you're running on 50 Raspberry Pis in retail stores.

Should a startup use Kubernetes?

Generally, no. Startups should optimize for developer velocity and cost efficiency. Kubernetes works against both. Use managed services (Heroku, Render, Railway) or simple container orchestration (Docker Compose, ECS). Add Kubernetes when you have the team and the traffic to justify it.

Is Kubernetes still worth learning?

Yes. Kubernetes skills are valuable. The concepts — declarative configuration, container orchestration, service discovery — transfer to other tools. But learn it as a specialized skill, not as the first tool you reach for.

The Pragmatic Path Forward

Here's what I tell clients at SIVARO.

Start simple. Use Docker Compose. Use a single VM. Use a managed service.

When you hit real pain — not imagined pain, not "but Google does it" pain — then evaluate options.

Kubernetes is a hammer. Most infrastructure problems are not nails.

The teams I see winning are the ones who match complexity to actual need. They run Kubernetes for the one workload that needs it. Everything else lives on simpler infrastructure.

Why are people moving away from kubernetes? Because they finally asked the question: "Does this actually make our business better?" And for many of them, the answer was no.

Nishaant Dixit — Founder of SIVARO. Building data infrastructure and production AI systems since 2018. Built systems processing 200K events/sec.