DiffusionGemma: Google’s 4x Faster Text Diffusion Model

Google DeepMind shipped DiffusionGemma on June 10, 2026 — a 26B open-weight model that generates text in a fundamentally different way than every LLM you have used before. Instead of predicting one token at a time from left to right, it starts with noise and denoises 256 tokens simultaneously. On an NVIDIA H100, that translates to roughly 1,008 tokens per second — around 4-5x faster than comparable autoregressive models. The catch is real: this is not Gemma 4 quality at 4x speed. It is a different model built on different tradeoffs, and the right question is whether those tradeoffs work for your use case.

How Discrete Diffusion Works

Every LLM you have used before generates text autoregressively: predict one token, append it, repeat. This is sequential by design and creates a fundamental bottleneck — each forward pass accesses the full model weights to produce a single token, making the process memory-bandwidth-bound at low batch sizes.

DiffusionGemma borrows from image generation. At inference time, it fills a canvas of 256 token positions with noise, then runs up to 48 denoising steps. Each step proposes values for all 256 positions simultaneously. Positions where the model is confident get committed; uncertain positions get re-noised and reconsidered in the next step. The process continues until the canvas is stable enough to output.

The critical architectural difference is bidirectional attention. Standard autoregressive transformers use causal attention — token 5 can only attend to tokens 1-4, never forward. DiffusionGemma uses full bidirectional attention across all 256 positions simultaneously. This means it can revise a token it already proposed if later context makes a better choice obvious. Autoregressive models cannot do this: once a token is generated, it is committed. This is also why code infilling is a genuine strength — seeing both the prefix and suffix simultaneously before committing to any token is exactly what that task requires.

The Speed and Quality Numbers

The speed claims hold up. With vLLM and FP8 quantization, DiffusionGemma reaches 1,008 tokens per second on an H100 and 1,288 tokens per second on an H200. For context, typical 27B autoregressive models on H100 generate roughly 200-250 tokens per second. This is not a quantization trick — it is architectural.

Quality is where the tradeoffs are visible. Google’s own documentation is direct about this: DiffusionGemma is experimental and is not positioned as a drop-in replacement for Gemma 4 27B. On benchmarks, it scores 77.6% on MMLU Pro and 73.2% on GPQA Diamond — both below the standard Gemma 4 model. Complex multi-step reasoning, math, and science tasks show the largest gaps. Instruction following is also slightly less precise on complex prompts.

The gap narrows on tasks where bidirectional attention is an actual advantage: code infilling, content reformatting, bulk summarization, and inline editing. For those workloads, the quality difference becomes less material and the speed difference becomes very material.

Hardware Requirements and How to Run It

The model is Apache 2.0 licensed with weights on Hugging Face, Kaggle, and Vertex AI. Two main deployment paths:

Hardware minimum: 18GB VRAM with NVFP4 quantization. The RTX 4090 (24GB) is the entry-level consumer option. The RTX 4080’s 16GB is not enough. Despite only 3.8B parameters being active per forward pass, all 26B expert weights load into VRAM at startup — the MoE sparsity does not reduce your memory requirement. AMD ROCm and Apple Silicon are not supported at launch; NVFP4 is NVIDIA-specific.

For development and exploration, install via Hugging Face Transformers:

pip install "transformers>=5.11.0" accelerate

For production serving, vLLM is the recommended stack — and DiffusionGemma is the first diffusion LLM to receive native vLLM support. The vLLM team built new ModelState abstractions specifically for the non-autoregressive serving path. Continuous batching, memory efficiency, and the OpenAI-compatible API all work out of the box:

vllm serve google/diffusiongemma-26B-A4B-it --dtype float16

Then query it with the standard OpenAI client:

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="dummy")
response = client.chat.completions.create(
    model="google/diffusiongemma-26B-A4B-it",
    messages=[{"role": "user", "content": "Your prompt here"}]
)
print(response.choices[0].message.content)

When to Use DiffusionGemma (and When Not To)

The honest framework is not “is DiffusionGemma better than Gemma 4?” — on quality, it mostly is not. The right question is whether your use case values speed enough to accept a quality tradeoff.

Use DiffusionGemma when:

• You are building real-time coding assistants where sub-second latency defines the product experience
• You need a fast local chatbot without cloud API latency
• You are running content pipelines — bulk summarization, reformatting, document extraction — where throughput matters more than precision
• Your task is code infilling or inline editing (bidirectional attention is a genuine advantage)
• You are building agentic loops that need fast iteration across many tool calls

Stick with a standard autoregressive model when:

• You need the highest-quality output for complex reasoning, math, or science tasks
• Precise instruction following is critical
• Your users expect token-by-token streaming (DiffusionGemma outputs arrive in 256-token blocks)
• You are on Apple Silicon or AMD GPUs

Why This Architecture Matters

DiffusionGemma is significant not because it beats Gemma 4 — it does not — but because it is the first production-ready demonstration that text diffusion works at scale, ships with a permissive license, and integrates with the standard inference stack. The vLLM team building new ModelState abstractions to support it is a meaningful infrastructure investment in this architecture’s future.

The model is labeled experimental for good reason. But “experimental” with working code, a production vLLM path, Apache 2.0 weights, and a clear use case envelope is the kind of experimental worth running. If you build speed-sensitive AI applications, the setup time is an afternoon. The official developer documentation and NVIDIA’s deployment guide cover the rest.