By Cloudflare added Moondream 3.1 to Workers AI on July 8, and the latency numbers are worth paying attention to: detect tasks complete in roughly 160ms, captions in 480ms. Those are end-to-end times, network round-trip included. That crosses the threshold where vision inference can run synchronously inside a request handler — not in an async pipeline you bolt on afterward.
What Moondream 3.1 Is (And Why It’s Fast)
Moondream 3.1 is a 9-billion-parameter vision language model built on a mixture-of-experts (MoE) architecture. The key: only 2 billion of those parameters activate per forward pass. Sixty-four total experts, eight active at a time. The model carries the representational capacity of a 9B system but runs at the compute cost of a 2B dense model.
That’s why it fits at the edge. Cloudflare’s Workers AI GPUs have real memory constraints — a 9B dense model doesn’t slot in cleanly, but a model that activates 2B per token does. And because edge GPUs are distributed across 290 cities, the network hop to the model is short for most users worldwide.
Four Task Modes, Real Numbers
The model ships with four built-in task modes — not prompt workarounds, but first-class outputs with distinct API parameters and their own output schemas:
| Task | What It Does | Latency (p50) |
|---|---|---|
query | Open-ended Q&A about an image | ~770ms |
caption | Short, normal, or long descriptions | ~480ms |
point | Returns coordinates for a named object | ~145ms |
detect | Returns bounding boxes for a named object | ~160ms |
Point and detect are the sleepers. Under 200ms for grounded spatial output makes UI automation, agent workflows, and live object overlays viable in a request path — a class of workloads that previously required async queues and server-side processing.
Code: Image Moderation in a Workers Request Handler
Here’s a concrete use case: check an uploaded image for inappropriate content before storing it. The check runs in the request handler — no async queue, no webhook, no second round-trip to a separate service.
export interface Env {
AI: Ai;
}
export default {
async fetch(request: Request, env: Env): Promise<Response> {
const formData = await request.formData();
const file = formData.get("image") as File;
const buffer = await file.arrayBuffer();
const result = await env.AI.run(
"@cf/moondream/moondream3.1-9B-A2B",
{
image: [...new Uint8Array(buffer)],
task: "query",
question: "Does this image contain inappropriate content? Answer yes or no.",
reasoning: false,
}
);
if (result.answer?.toLowerCase().includes("yes")) {
return new Response("Upload rejected", { status: 400 });
}
return new Response("Upload accepted");
},
} satisfies ExportedHandler<Env>;The model ID is @cf/moondream/moondream3.1-9B-A2B. The AI binding is configured in your wrangler.toml — no separate API key required beyond your Cloudflare account.
What This Actually Unlocks
The sub-second threshold isn’t just faster — it’s architecturally different. When vision inference takes 1.5–2 seconds via a server-side API, you work around it with async pipelines and queues. When it takes 500ms at the edge, you run it inline. That changes what you build:
- Image moderation before storage: Reject bad uploads in the request handler before they reach your storage bucket.
- Agent screenshot inspection: An AI agent captures a screenshot, runs it through Moondream, and decides the next action — all within a single agent turn.
- Document field extraction: Pull structured data from an uploaded form during the HTTP request itself, not in a background job.
- Live object overlays: Locate an object in a video frame in time to drive a real-time UI element.
Pricing and Honest Caveats
Moondream 3.1 on Workers AI costs $0.30 per million input tokens and $1.00 per million output tokens. Cloudflare’s free tier includes 10,000 Neurons per day — enough for experimentation at no cost. Compare that to GPT-4o Vision at roughly $5 per million input tokens: for structured, latency-sensitive tasks, the cost difference is significant.
That said, Moondream 3.1 is not a replacement for frontier vision models. Complex visual reasoning, multi-image analysis, and nuanced instruction-following still favor GPT-4o Vision or Claude’s vision capabilities. The practical split: use Moondream 3.1 for structured tasks where speed and cost matter, and keep frontier models for tasks requiring deeper inference. Most production pipelines will use both.
Getting Started
The model is live on Workers AI now. Add the AI binding to your wrangler.toml, reference the model ID in your worker, and you’re running vision inference at the edge with no additional infrastructure. The official Cloudflare changelog announcement covers the release details. Moondream.ai documents the upstream model and on-device deployment options via their Photon runtime for embedded or air-gapped use cases. Review Workers AI pricing before scaling to production volumes.
