By ByteBot
OpenAI revealed this week at Hot Chips 2026 that it used its own language models to write the hardware description code for Jalapeño, its first custom inference ASIC. Co-developed with Broadcom, the chip went from RTL design to tape-out in nine months, with a team averaging fewer than 100 engineers — a pace that would embarrass most established chip programs. The result is a chip delivering 13.4 petaflops of MXFP4 compute at 700W TDP, with performance benchmarks that beat Nvidia’s GB300 on single-token prediction latency by 3.6x.
How LLMs Actually Wrote Hardware Code
The key to OpenAI’s accelerated design workflow was Google’s open-source XLS high-level synthesis toolchain. Instead of writing raw Verilog — a hardware description language that current LLMs handle poorly — engineers coded in DSLX (a Rust-inspired domain-specific language) and C++. XLS then converted those to Verilog automatically. LLMs perform significantly better on “software-looking things,” as OpenAI’s Chris Leary put it in the IEEE Spectrum report on the project, and XLS kept the team in that comfortable territory throughout front-end design.
The efficiency gains were not just anecdotal. Compared with an optimized human baseline, OpenAI’s LLMs achieved a 56% improvement on the BF16 multiplier design, 21% on the FP4 dot-product block, and a 10% reduction in matrix multiplication unit area. After first silicon arrived in May 2026, a Codex-generated kernel for DeepSeek R1’s Multi-Head Latent Attention went from 0.31% of theoretical hardware ceiling to 88.94% in roughly 40 hours, with no manual intervention. An experienced chip engineer on Hacker News summarized it plainly: “I am awed at how the world has changed.”
Related: NVIDIA CUDA Rust: Two Tracks, One Already in Production
The OpenAI Jalapeño Chip Specs and How It Stacks Up
Jalapeño is not a research prototype. Each chip runs on TSMC’s N3P process with 232 GB of HBM4 memory and 15.4 TB/s of bandwidth. At 700W TDP, it consumes significantly less power than Nvidia’s Vera Rubin (900–1,150W), which matters when you’re running 128-chip racks. According to SemiAnalysis, Jalapeño “smokes every other chip” on tokens-per-megawatt — a metric that directly maps to inference cost. DeepSeek R1 runs at over 700 tokens per second per user at concurrency 1; Kimi-K2.5 achieves throughput roughly 9x better than the next competitor.
The caveat worth noting: those benchmarks use single-token prediction only. Nvidia Rubin’s real-world advantage includes speculative decoding, which can deliver 3–5x cost reduction on top of raw hardware throughput. Head-to-head comparisons that ignore speculative decoding may flatter Jalapeño. Still, on latency alone, the 3.6x advantage over GB300 is meaningful for interactive AI workloads where time-to-first-token matters most.
What This Means for the Nvidia Moat
Meta has been working on its MTIA chip program for years without commercially competitive results. Microsoft’s custom silicon efforts have similarly stalled. OpenAI, with roughly 100 engineers and 16 months from team formation to first silicon, taped out a chip that beats the current Nvidia standard on key metrics. The differentiator is the LLM-assisted design workflow. Richard Ho, who led the effort, said models are “giving superpowers to our engineers” — engineers who can stay in high-level languages while LLMs handle the mechanical translation to hardware.
More striking: the LLMs OpenAI used improved throughout the project. The team started with o3 precursors and ended with precursors to GPT-6 Astra. Those newer models can write Verilog directly, skipping the XLS translation layer entirely. This creates a recursive loop — better AI helps design better inference chips, which run better AI. The immediate industry implication, per SemiAnalysis: “The CUDA moat is potentially dead given how fast OpenAI can bring up new models.” Any well-funded AI lab following this playbook can now realistically produce competitive custom silicon.
The actual bottleneck is no longer design expertise. Multiple engineers familiar with the industry pointed to foundry access — TSMC allocation slots — as the constraint limiting how many labs can follow OpenAI’s path. Hacker News commenters with chip backgrounds were blunt about it: design democratization without manufacturing capacity narrows the gap considerably, but does not close it.
Key Takeaways
- OpenAI taped out a competitive custom inference ASIC in nine months using fewer than 100 engineers — made possible by LLMs writing hardware description code via the XLS/DSLX workflow
- Measured gains over human baselines are real: 10% matrix unit area reduction, 56% BF16 multiplier improvement, kernel MFU from 0.31% to 88.94% in 40 hours
- Jalapeño beats Nvidia GB300 on latency (3.6x) and tokens-per-watt, though fair comparisons must account for speculative decoding advantages Nvidia employs
- The CUDA software moat faces a new threat: not from a chip manufacturer, but from an AI lab that used AI to build better AI hardware
- Foundry access remains the binding constraint — LLM-assisted design lowers the engineering bar but does not solve TSMC allocation
