By Apple ML Research published a benchmark yesterday that answers a question the industry had been circling around but hadn’t formally tested: can AI coding agents optimize the software that serves AI models? Across 37 tasks on real production engines, the answer is yes — by 1.68x on average. Claude Sonnet 5.5 topped the leaderboard of 12 agents, beating both Claude Opus 5.5 and OpenAI’s GPT-6 Astra. If you run vLLM or SGLang in production, this benchmark is worth reading carefully.
What RSI-Inference Bench Actually Tests
RSI-Inference Bench (Recursive Self-Improving Inference Benchmark) from Apple ML Research covers three categories of inference engineering work:
- Day-0 Model Support — 10 tasks asking the agent to serve a newly released model, scored pass/fail.
- Inference Optimization — 20 tasks requiring the agent to increase throughput on vLLM.
- Bug Fix — 7 tasks on SGLang asking the agent to find and fix a performance regression.
Each agent works offline against a pinned engine commit, with 1 to 8 real GPUs, for up to 8 hours. An independent verifier then rebuilds the modified engine, serves a model, and measures throughput. Any modification that fails correctness checks scores 1.00x — no improvement. The benchmark cannot be gamed by breaking things to make them appear faster.
This design makes RSI-Inference Bench the most production-realistic agent benchmark for inference engineers published to date. vLLM and SGLang are the dominant open-source serving engines in production — not toy codebases. An 8-hour window per task approximates a focused human engineering sprint.
The Leaderboard — and the Surprise
Sonnet 5.5 topped the board at Elo 2022, ahead of Opus 5.5 at Elo 1889, Fable 5.1 at 1696, GPT-6.1 Sol at 1680, and GPT-6 Astra at 1667. On Day-0 Model Support, Sonnet 5.5 scored 26.7% pass@1 and fully passed 5 of 10 tasks at least once.
The surprising result isn’t that Anthropic dominated — it’s which Anthropic model won. On most coding benchmarks, Opus 5.5 leads Sonnet 5.5 by 2 to 3 percentage points. Here, the gap flips by over 100 Elo points. The likely explanation is how the benchmark rewards agent behavior: 8 hours, offline, iterating against a real system. That environment favors agents that move quickly, try more approaches, and recover from failed attempts efficiently. Sonnet 5.5 is 30 percent faster and 30 percent cheaper than Opus 5.5 per task — and in an 8-hour window, that means more iterations.
This isn’t a knock on Opus 5.5. It’s a signal that inference optimization as a task has its own cognitive profile, and the right model for it isn’t necessarily the one that scores highest on general software engineering benchmarks.
What 1.68x Means for Your Infrastructure Bill
A 1.68x average throughput speedup means serving the same token volume takes roughly 40 percent less GPU time. For teams running production inference at meaningful scale:
- A 0k/month inference bill drops to roughly 0k with agent-generated optimizations.
- A team hitting GPU capacity can serve 68 percent more traffic without adding hardware.
These are benchmark numbers, not production guarantees — but they are measured on real engines under real conditions. The gap between benchmark and production is narrower here than for most AI coding benchmarks. See the full AI coding agent leaderboard for context on how these models compare across other evaluation dimensions.
AI Agents Eating Their Own Stack
RSI-Inference Bench arrives as part of a broader 2026 pattern: AI agents being directed at the infrastructure that runs AI. AlphaEvolve has demonstrated agents improving neural architecture search. OpenAI Codex has been applied to training pipeline optimization. Now Apple has formally benchmarked agents against inference serving — the most cost-sensitive layer of the AI production stack.
The implication for engineering teams is direct: inference optimization is no longer a human-only domain, and the gains are measurable, not speculative. Teams spending meaningful money on GPU inference should be running these experiments now.
What to Do Now
If you run vLLM or SGLang in production, the practical step is clear: test an agent-driven optimization pass against a staging environment. Use Sonnet 5.5 given its RSI-Inference Bench performance. Start with the throughput optimization category — 20 tasks, measurable output, no ambiguity about whether the result worked. The benchmark code and leaderboard are public from Apple ML Research as of October 16.
