SIVARO
GPU Admission Control Policy Kubernetes: The Missing Manual for AI Infrastructure
The GPU panic of 2024 was real. Every CTO I talked to in Bangalore and San Francisco was hoarding A100s like canned goods before a hurricane. Now it's 2026, and the hardware finally caught up — H200s are everywhere, B200s are hitting data centers, and the cloud providers are practically begging you to spend. But here's the uncomfortable truth: your GPU scheduling is still terrible.
I've spent the last two years at SIVARO building data infrastructure for companies running production inference and fine-tuning workloads. I've watched teams buy $3 million worth of hardware and then underutilize it by 60% because they treated GPU scheduling like it was CPU scheduling with extra steps. It's not. And the single most underrated tool in your arsenal is the Kubernetes admission control policy.
Not the default one. Not the basic LimitRange you copy-pasted from a blog in 2023. The real one. The one that makes your cluster behave like a well-oiled machine instead of a free-for-all.
Let me show you what that looks like.
What Is a GPU Admission Control Policy in Kubernetes?
You already know the basics. Kubernetes admission controllers intercept API requests — the things you kubectl apply — before the objects get persisted. They validate, mutate, and sometimes reject. The GPU admission control policy is a specific set of these controllers that govern how GPU resources get requested, allocated, and enforced.
But that's the textbook definition. Here's what it actually means in practice.
When someone in your organization submits a Pod spec that requests nvidia.com/gpu: 1, the admission controller has a say. It can say "yes, that's fine" (the default). It can say "no, the policy says you get 0.5 GPUs max for batch jobs" (that's a custom admission policy). It can say "I'm going to rewrite your request to use MIG instances because that's what this namespace allows" (that's a mutating webhook).
The default behavior in Kubernetes is a free-for-all. Anyone who can create a Pod can request any number of GPUs, and there's no enforcement of sharing, fairness, or allocation boundaries. That was fine when GPUs were a novelty. It's not fine when GPU spend is your second-largest OpEx line item.
The GPU admission control policy in Kubernetes is what separates "we have GPUs" from "we manage GPUs."
Let me show you the practical layers of this.
Layer 1: The Basics — LimitRanges and ResourceQuotas
Start here. I know it's boring. I know you want the fancy webhooks. But I've seen too many teams skip this and then wonder why their custom policy is a mess.
A LimitRange sets defaults for Pods in a namespace. If someone submits a Pod without specifying GPU limits, the admission controller fills in the blank. If someone tries to request 8 GPUs when the namespace max is 4, the admission controller rejects it outright.
yaml
apiVersion: v1
kind: LimitRange
metadata:
name: gpu-constraints
namespace: inference-prod
spec:
limits:
- type: Pod
max:
nvidia.com/gpu: "4"
min:
nvidia.com/gpu: "1"
- type: Container
default:
nvidia.com/gpu: "1"
defaultRequest:
nvidia.com/gpu: "1"
max:
nvidia.com/gpu: "4"
That's the guardrail. Now the ResourceQuota — this is what enforces fairness in GPU scheduling multi-tenant clusters. Without it, one tenant can hog all the GPUs and your entire cluster becomes a hostage situation.
yaml
apiVersion: v1
kind: ResourceQuota
metadata:
name: gpu-quota
namespace: inference-prod
spec:
hard:
nvidia.com/gpu: "16"
This is the admission control policy in its most primitive form. It works. It doesn't require external dependencies. But it also doesn't solve the real problems — which are fragmentation, utilization, and priority inversion.
Layer 2: The Real Problems Start Here
Let me tell you about a company I worked with in 2025. They had a 64-GPU A100 cluster running production inference for a fintech product. The symptom was simple: latency spikes during peak hours. The diagnosis was not.
Turns out, their teams were requesting GPUs in whole units — always nvidia.com/gpu: 1 even for tiny models that needed maybe 2GB of the 80GB available on an A100. The scheduler was placing these requests on separate physical GPUs, which meant 16 inference pods were spread across 16 physical devices, each using 3% of the memory and 5% of the compute.
The fix wasn't more GPUs. It was GPU sharing.
And that's where the admission control policy gets interesting, because you need to intercept requests and decide: does this workload qualify for a fractional GPU, a MIG slice, or a full device?
Layer 3: MIG and Sharing via Mutating Admission Webhooks
NVIDIA's MIG (Multi-Instance GPU) technology lets you partition a physical GPU into multiple isolated instances. It's not sharing in the fuzzy sense — the memory and compute are hard-partitioned. For inference workloads, this is the difference between using 20% of your cluster and 80%.
But here's the thing: Kubernetes doesn't natively know about MIG. The default device plugin can expose it, but the admission control layer is where you encode the policy: who gets MIG slices, what sizes, and under what conditions.
I built a mutating webhook at SIVARO that does exactly this. It intercepts every Pod creation request, examines the container's resource requirements, and rewrites the GPU request based on policy. Here's the heart of it:
python
# Simplified logic from our mutating admission webhook
def mutate_gpu_request(pod, policy_config):
for container in pod.spec.containers:
gpu_request = container.resources.requests.get("nvidia.com/gpu")
gpu_limit = container.resources.limits.get("nvidia.com/gpu")
if not gpu_request:
continue # No GPU requested, no mutation needed
# Policy: Models with requested memory < 4GB get MIG 1g.5gb slices
if gpu_request == 1 and container_memory_request(container) < "4Gi":
container.resources.requests["nvidia.com/mig-1g.5gb"] = 1
container.resources.limits["nvidia.com/mig-1g.5gb"] = 1
del container.resources.requests["nvidia.com/gpu"]
del container.resources.limits["nvidia.com/gpu"]
return pod
The policy is the interesting part. It's not "request what you want" — it's "request what you need, and the system gives you exactly that."
The key is that the webhook is a mutating admission controller. It rewrites the request before the scheduler sees it. The user thinks they're getting a full GPU. They're getting a slice. And their latency doesn't change because the model was small enough anyway.
This is, in my opinion, the best gpu scheduling policy for inference clusters — because inference workloads are mostly memory-bound and latency-sensitive, not compute-bound. A full GPU is overkill for 90% of production inference models. The admission policy should reflect that.
Layer 4: Fairness Without the Drama
Here's where it gets contentious. I'm going to take a position: shared GPU pools with admission control beat dedicated GPU assignments for most inference workloads.
Most people think the opposite. They think, "I'll give the model team their own 8 GPUs, the recommenders their own 4, and nobody fights." That's the static allocation approach. It's simple. It's also a utilization disaster.
A team with a spiky workload holds 8 GPUs that sit idle 80% of the time. Another team with a steady workload can't get resources because the quota says no. You're paying for idle silicon.
The fix is an admission control policy that enforces fairness dynamically. Not static quotas — dynamic ones that account for actual usage patterns.
I've seen this work well with a validating admission webhook that checks cluster-wide GPU utilization before allowing a Pod to request a non-shared GPU. The logic:
- If cluster GPU utilization is below 70%, allow full-GPU requests.
- If above 70%, check if the workload specifies a model size that fits a MIG slice.
- If it does, reject the full-GPU request and the user retries with a slice (or the mutating webhook rewrites it automatically).
This way, you get fairness in gpu scheduling multi-tenant clusters without the administrative overhead of static partitions. The admission policy is the traffic cop, redirecting less demanding workloads to shared resources and preserving full GPUs for the workloads that genuinely need them.
I'm not saying this is easy. It requires you to know your workloads. You need model size annotations, latency requirements, and an understanding of what "good enough" looks like for different services. But that's the job. If you're running a multi-tenant GPU cluster and not doing this, you're leaving money on the table.
Layer 5: Priority and Preemption — The Missing Piece
The last time I wrote about admission policies, someone from a hyper-scaler told me, "You're missing preemption." They were right.
A GPU admission control policy that doesn't consider Pod priority is like a bouncer who lets in anyone based on height but doesn't check the VIP list. When a batch training job requests 8 GPUs and an inference service with a 99.99% SLA needs them, the admission policy should know the difference.
Here's what I mean. The QueueSort and Priority admission plugins in Kubernetes handle ordering. But for GPUs, you often want to evict running Pods (batch workloads) to make room for critical ones (interactive inference serving).
That's the PriorityClass + PodDisruptionBudget combination. Define a PriorityClass for production inference, another for batch, and configure your admission policy to never let a batch job get accepted if it would starve a critical service.
yaml
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
name: production-inference
value: 1000
globalDefault: false
description: "Production inference workloads — never preempt these."
---
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
name: batch-training
value: 100
globalDefault: false
description: "Batch training jobs — preemptible."
The admission controller's job is to validate that a new Pod's PriorityClass is compatible with the namespace's policy. You don't want a batch job landing in a production-inference-only namespace. And you do want a validating webhook that checks whether accepting a high-priority Pod would exceed the namespace's GPU quota — considering not just requested but currently in-use resources.
This is where the gpu admission control policy kubernetes becomes a living thing. Not a config file. A system.
Layer 6: The Dynamic Resource Allocation (DRA) Shift
By 2026, the industry is moving toward Dynamic Resource Allocation as the standard for GPU scheduling. It replaces the classic nvidia.com/gpu with a resource.k8s.io API that allows for much richer semantics — resource requirements that include memory, bandwidth, and topology.
This changes the admission control game. Instead of a simple integer, you're validating claims:
yaml
apiVersion: resource.k8s.io/v1beta1
kind: ResourceClaim
metadata:
name: inference-gpu
namespace: fintech-inference
spec:
devices:
requests:
- requestClassName: nvidia-gpu-mig
selectors:
- capacity:
nvidia.com/memory: "5Gi"
nvidia.com/compute: "10"
The admission controller now validates the claim request against the available DeviceClasses, checks if the requesting namespace's quota allows for the claimed resources, and then either approves or rejects.
Honestly, this is better governance. The old way treated GPUs like apples — "give me one." The DRA model treats them like complex instruments — "give me one with this memory, this compute profile, this network bandwidth."
The transition isn't trivial. If you're on a pre-1.30 cluster, you're not getting DRA. But if you're building new infrastructure in 2026, start here. The admission policy you write for DRA is your long-term investment.
The Hard Lessons
Let me be honest about what I've learned the hard way, so you don't repeat my mistakes.
Most people think this is a technical problem. It's not. It's an organizational problem.
The hardest part of implementing a gpu admission control policy kubernetes isn't writing the YAML. It's getting the teams to classify their workloads. "It's production critical" — everyone says that. Nobody wants to admit their model retraining job can wait until tomorrow.
I learned this with a healthcare client in 2025. Their ML team swore every workload was latency-critical and needed full GPUs. After three weeks of negotiating and actually looking at logs, we found that 70% of their GPU usage was batch inference with no real time requirement. The admission policy we wrote mandated fractional GPU allocation for those jobs. Their infrastructure bill dropped 45% in the first quarter.
The technical side — the webhook, the mutation logic, the validation rules — took two days. The organizational alignment took three months.
The second lesson is about over-enforcement.
I've seen teams implement such strict admission policies that no legitimate Pod can ever get scheduled. They set minimum GPU sizes too high, quotas too tight, and validation rules too aggressive. The cluster ends up at 30% utilization because the policy is more restrictive than the actual demand.
Your admission policy should be a loose-fitting jacket — it guides behavior without constraining movement. Start permissive. Add restrictions as you see violations. Don't try to solve every edge case on day one.
The third lesson is about observability.
You can't enforce policies you can't debug. Every rejection from your admission webhook should be logged with enough context to diagnose: who requested, what they requested, why it was denied, what they should have requested instead.
At SIVARO, we built a rejection dashboard that shows denied requests in real time. The first month of any deployment, this dashboard is the most important tool in the org. It shows you where your policy is too strict (rejections from the same team for the same reason repeatedly), too lenient (no rejections at all — you're enforcing nothing), or just wrong (rejection patterns that don't match your intent).
Implementation Checklist
If you're building this today, here's the order of operations:
-
Inventory your workloads. Run [
kubectl describe nodes | grep nvidia.com/gpu] and see who's actually requesting what. You'll be surprised. -
Set baseline quotas. Start with
ResourceQuotaandLimitRange. Get the fundamentals in place before you write custom webhooks. -
Enable MIG in your devices. This requires the NVIDIA device plugin with MIG support. Test on a small pool first.
-
Deploy a mutating webhook that rewrites single-GPU requests for small-memory workloads to MIG slices. Start with 50% of the cluster, not 100%.
-
Deploy a validating webhook for priority and preemption rules. Make sure your critical services are protected.
-
Build the rejection dashboard. You can't measure improvement without data.
-
Iterate monthly, not quarterly. Your workloads change. Your policy should too.
The Costs and Trade-offs
Nothing here is free. The mutating webhook adds latency to Pod creation — usually 10-50ms per request if implemented correctly. The validating webhooks also add overhead. If you're creating hundreds of pods per second (you're not), this might matter. For most teams, the admission control latency is imperceptible.
What does matter is the CPU cost of running these webhooks. A high-traffic cluster can hit the webhook with thousands of requests per minute. Make sure your webhook service autoscales like any other HTTP service. I've seen admission webhooks become the bottleneck in clusters that push 100+ pod creations per minute.
Also, the operational complexity is real. A misconfigured webhook can brick your cluster — literally reject every Pod creation. This is why you need a failurePolicy: Ignore during the rollout phase, not fail-closed from day one. I'll say that again: start with failurePolicy: Ignore. Only flip to Fail after you've verified your webhook in production.
FAQ
What is the default GPU admission behavior in Kubernetes?
By default, with the built-in NVIDIA device plugin, any Pod can request nvidia.com/gpu as a resource. The scheduler will place it on a node with available capacity. There's no admission-level policy — no fairness, no sharing, no priority. It's a first-come-first-served free-for-all with no management overhead.
Can I use a GPU admission control policy with any device plugin?
Not exactly. The device plugin needs to expose the resource types that your admission policy references. If you're using MIG slices, your plugin must expose nvidia.com/mig-* resources. If you're using DRA (device plugin v1), your plugin must support the claim API.
What's the difference between a mutating and validating admission webhook?
A mutating webhook rewrites the Pod spec (e.g., changes 1 GPU to a MIG slice). A validating webhook either allows or rejects a request without modifying it. You use both. Mutation for intelligent defaults. Validation for enforcement.
How do admission policies affect GPU scheduling in multi-tenant clusters?
They're the primary mechanism for enforcing fairness in gpu scheduling multi-tenant clusters. Without admission policies, a single team can request (and hold) all available GPUs, starving every other tenant. Admission policies enforce quotas, priority, and sharing rules at the API layer.
Is a GPU admission control policy the same as a scheduler?
No. The scheduler decides where to place a Pod. The admission controller decides whether to accept the request at all. They work together: admission enforces the rules, scheduler optimizes placement within those rules.
What's the best gpu scheduling policy for inference clusters?
It depends on your workload, but I'd argue for a policy that: (1) enforces namespace quotas, (2) mutates requests to MIG slices for models under 8GB memory footprint, (3) prioritizes interactive inference over batch training via priority classes, and (4) uses DRA for richer resource selection. That combo gives you utilization and performance without sacrificing stability.
How much overhead does a mutating webhook add to pod creation?
Approximately 5-50ms per request, depending on your webhook implementation, network latency, and whether it's a simple mutation or a complex external API call. That's negligible for most clusters.
What happens when the admission webhook fails?
If configured with failurePolicy: Ignore, the Pod request bypasses the webhook and proceeds with defaults. If failurePolicy: Fail, the Pod creation is rejected. For GPU admission policies, I'd start with Ignore and only switch to Fail after the webhook has been stable in production for a few weeks.
Final Thought
GPU admission control policies in Kubernetes are the difference between owning GPUs and managing them. The hardware is expensive. The software enforcement of how you use them is what determines whether your infrastructure investment pays off or burns.
I've watched too many teams throw money at GPU clusters and then lose it to misallocation, idle capacity, and scheduling chaos.
The admission policy is your first line of defense. Set it up right, and everything else becomes easier. Actually, that's a lie — everything else is still hard. But with a proper admission control policy, the "everything else" becomes manageable. That's the best you can ask for in this business.
Nishaant Dixit — Founder of SIVARO. Building data infrastructure and production AI systems since 2018. Built systems processing 200K events/sec.