By Most AI infrastructure conversations start and end with GPUs. AMD just argued that for agentic workloads, that framing is backwards. At this week’s Open Compute Project Global Summit, the company unveiled Helion — an open rack blueprint built for AI agents that routes model calls between expensive cloud APIs and local open-source alternatives running on-prem. The headline claim: a 43% reduction in token costs and response times up to 3x faster for workloads shifted to local inference.
Why Agents Are a CPU Problem
Token generation is only a fraction of what an AI agent actually does. Tool calls, database queries, state management, security checks, context retrieval, API coordination — all of that runs on CPU. Research into production agent deployments shows CPUs handle 60–90% of total wall-clock time per request, with GPU inference consuming just 10–40%. A single tool call adds 200–800ms of CPU-bound latency per agent step.
The traditional data center ratio of one CPU for every four to eight GPUs creates a structural bottleneck when those CPUs are doing the majority of the work. AMD’s argument with Helion is that agent infrastructure needs to separate concerns: dedicated CPU capacity for orchestration and execution, GPU capacity for inference, each scalable independently based on actual demand.
Two Configurations
Helion ships in two configurations. The hyperscale option is CPU-only: up to 128 sixth-generation AMD EPYC 9006 “Venice” processors per rack, built on TSMC’s 2nm process, delivering up to 32,768 Zen 6 cores alongside 102 Tbps of networking bandwidth. This targets large operators who already run separate GPU inference infrastructure and need to scale agent execution independently.
The enterprise configuration adds up to 16 AMD Instinct MI350P GPU nodes to the EPYC base, combining execution and inference in a single rack. Smaller teams that want everything in one place get that option without sacrificing the CPU scaling story.
The Model Router: Where the Cost Savings Come From
The more interesting piece is the integrated software model router. It sits between the agent and the model layer, analyzing each request and dispatching it to the cheapest appropriate backend: cloud-hosted frontier models for complex reasoning, or open-source models running locally inside the rack for routine work. AMD’s Dan McNamara cited internal testing showing a 43% reduction in token costs alongside response times up to 3x faster for shifted workloads.
The math holds on paper. At current rates — Claude Sonnet 5 runs $3 per million input tokens and $15 per million output tokens, and agentic tasks generate 5–30x more tokens than standard chatbots — a pipeline that shifts a meaningful share of calls to local inference eliminates real dollar costs at scale. For a deployment running millions of agent tasks per day, 43% is not a rounding error.
The caveat: AMD has not detailed how the router integrates with production agent frameworks like LangGraph, CrewAI, or AutoGen. NAND Research flagged this gap — the router’s value depends heavily on integration with identity systems and cost governance tooling that enterprise teams already have in place. No independent benchmarks have been published. The 43% figure is AMD’s own measurement.
Open Blueprint vs. NVIDIA’s Integrated Stack
Helion is built on Open Compute Project standards end to end: ORv3 rack dimensions, DC-MHS modular host specs, HPRv2 power management, OCP NIC 3.0, and Caliptra security. AMD plans to contribute the full specification to the OCP community. Eight partners — Dell, HPE, Lenovo, Supermicro, Foxconn, QCT, Wiwynn, and Cisco — are building production systems targeting November 2026 availability.
This directly contrasts with NVIDIA’s Vera Rubin approach, which ties CPU, GPU, networking, and software into a single coordinated AI factory. AMD is betting that enterprise buyers prefer modularity and vendor interoperability over NVIDIA’s tightly integrated stack. Historically that bet has worked when AMD can execute on the software side, not just the hardware spec. That’s the open question here.
What AMD Still Needs to Prove
NVIDIA already has Vera CPU Rack in market with named production customers and published agent-specific performance claims. Helion is a blueprint with partner commitments, not shipping systems. The model router needs documented framework integrations before it drives procurement decisions rather than marketing conversations.
Developers evaluating Helion-based systems should profile their agent workflows before committing: measure where execution time actually concentrates, verify the CPU-to-GPU split for your specific workloads, and check the router’s governance capabilities against your compliance requirements. The November 2026 availability window from Dell, HPE, and Lenovo will be the first real test of whether this architecture delivers on its promise.
