The new Kubernetes production workload: AI inference
Insights from the conference floor at the largest cloud native event in North America
I'm at KubeCon + CloudNativeCon North America in Salt Lake City. The opening keynotes on Tuesday morning made it clear: AI inference is the new cloud native production workload.
Over the years, Kubernetes has evolved from managing pods to managing workloads. However, Kubernetes wasn't built with the idea of running agents.
Now, the focus is on solving problems around constrained resources (how can we get access to the resources required to scale?) and control and security guardrails.
To respond, as one keynote put it, "Some problems can only be solved in the open."
This is where the Cloud Native Computing Foundation (CNCF) is really growing, with 350,000 project contributors, up 17% from last year. The CNCF estimates there are over 21 million cloud native developers worldwide.
Countless open source projects are emerging to solve these problems, like Agent Substrate, OpenShell, KubeRay, agentgateway and plenty of others.
The scale is wild: an AI agent generates 450% more network traffic than a human doing the same task, according to Cisco.
And security is just as pressing, since, as NVIDIA put it, we can't simply ask agents to follow the rules; the infrastructure must enforce them.
This will be my focus at the event: seeing how the Kubernetes ecosystem is evolving to handle AI workloads! The keynotes got straight to the heart of it right away.
Excited to work the floor and learn more this week!
Other Blog Posts
