Kubernetes Is Getting Rebuilt Around Idle AI Agents

Nov 12, 2026 516 views

Kubernetes wasn't built for AI agents, and the mismatch is starting to show. The problems came up repeatedly at KubeCon + CloudNativeCon North America 2026 this week, covering everything from how much infrastructure agents consume to how the control plane holds up under their load. Fixing them will take real re-engineering, and the first results probably won't reach mainstream distributions for about a year. If you run Kubernetes in production, that's a timeline worth planning around, because the workarounds you rely on today may not survive a steady rise in agent traffic.

The idle agent problem

Agents spend a lot of time waiting. They pause for a model response, an external service, or a human to approve something. That's not a bug so much as a property of how agents work, since an agent loop often sits between steps for far longer than a typical web request would. Today, each active agent gets its own pod, and that pod keeps its memory and CPU allocated the whole time it sits there doing nothing.

Multiply that across a large deployment and the cost adds up quickly. From the scheduler's point of view, an idle pod takes up the same room as a busy one, and anyone who has run a cluster knows how reserved-but-unused capacity tends to get noticed. The control plane also ends up tracking millions of persistent objects, a volume it was never designed to handle. (And this is the part most people overlook: the bookkeeping load can become a bigger problem than the compute bill.)

Google's answer is an open source framework called Agent Substrate, which it contributed to the Cloud Native Computing Foundation (CNCF). It separates an agent's stateful session and identity from the compute running it. When an agent goes idle, the framework snapshots its state and stores it, freeing the worker for other tasks. When the agent needs to run again, Google says the session can be restored in under 500 milliseconds, on whichever worker is available.

That's a practical shift, and a more interesting one than it sounds. For years, the working assumption has been that a workload owns its node resources for as long as it runs. Decoupling the session from the compute flips that, so the agent becomes a resumable state rather than a running process. My take is that this is the right abstraction, though the sub-half-second restore figure is a vendor claim until someone measures it against their own workloads.

The control plane has its own limits

Freeing up workers doesn't solve everything. Mickey Boxell, senior principal product manager for cloud native services at Oracle, pointed out that agents make API calls to the Kubernetes control plane at a rapidly growing rate. As that traffic climbs, control plane performance becomes a serious concern, and it needs work of its own. Every one of those calls lands on the same control plane that serves human operators and CI pipelines, so agent traffic competes with everything else. That's why this is a scaling question, not just a missing feature.

Alex Zenla, CTO of hypervisor provider Edera, sees reason for optimism. Kubernetes has been extended before to handle workloads it was never designed for, most notably stateful applications on a platform built around stateless ones. Here's the thing: Zenla's comparison is a useful one, because stateful workloads were once considered a poor fit for Kubernetes too. Those problems were solved by extending the platform rather than abandoning it. That said, an analogy isn't a plan, and agent traffic differs in how often and how unpredictably it spikes.

Securing agents on shared infrastructure

Agents also need protection, and the security question is less about whether agents are trusted and more about what happens when they aren't. An agent that writes and runs code, or calls tools on a user's behalf, can be steered by the inputs it receives in ways a conventional service usually can't. Here's the thing: the two main contributions here target different risks, and they're easy to conflate. They shouldn't be.

Google has contributed gVisor to the CNCF. It isolates code inside a lightweight userspace application kernel that can run as a container, which keeps untrusted code away from the host. Sandboxes of this kind typically trade some performance for a smaller attack surface, because the guest talks to a userspace kernel rather than directly to the host's. That addresses the question of what an agent's code is allowed to touch at the system level.

NVIDIA took a different approach. It contributed OpenShell, an open source runtime for deploying AI agents securely. OpenShell enforces policies on what an agent can access and what actions it can take. Where gVisor asks what code can touch, OpenShell asks what the agent is allowed to do with it. Isolation and policy enforcement are complementary layers, and a production deployment will probably need both. Policy is only as good as the people writing it, though, and writing good agent policy is a skill most platform teams are still building.

What comes next

Nobody has a firm timeline for when these pieces will show up in the clusters IT teams run in their data centers. The reason is mundane. These projects are early, and the path from an open source framework to a supported distribution runs through vendor integration, testing, and upgrade tooling. The need is growing, though, and some teams are already evaluating alternatives rather than waiting.

My read is that the Kubernetes running agentic workloads in a few years will look quite different from today's clusters. Expect the idle-agent problem to be solved first, since the cost savings are immediate and easy to measure. Control plane scaling and agent-specific security policies will likely follow as the pieces mature. I'd bet on that sequence, but I wouldn't bet the roadmap on it.

What this means if you're building agents

If you're building or operating agents today, the practical takeaway is to treat the current tooling as a starting point. Watch how your agents behave between steps, because that's where your cost and your control plane load are quietly accumulating. Measure before you believe a restore-time number, and test under realistic concurrency rather than a demo load.

What this means for you depends on your role. Platform engineers should start asking which parts of their stack assume a workload runs continuously. Security teams should separate the question of isolation from the question of policy, since the tools answering each are different and neither covers the other.

Mostly theory, for now. But the mismatch described at KubeCon is real, and the teams that map it early will spend less time firefighting once agent traffic becomes the norm.

Source: Mike Vizard · cloudnativenow.com

Comments

Sign in to comment.
No comments yet. Be the first to comment.

Related Articles

Initiatives to Re-Engineer Kubernetes for the Agentic AI ...