By There are now two kinds of websites: ones that AI agents can query directly, and ones they scrape as raw HTML and hope for the best. Microsoft’s NLWeb protocol — announced at Build 2025, gaining real ecosystem weight at Build 2026 today — is what draws that line. Two endpoints, a 30-minute Cloudflare setup, and your site goes from agent-invisible to agent-readable. The longer you wait, the more that gap matters.
What NLWeb Actually Is
NLWeb is an open protocol that gives any website two new endpoints: /ask and /mcp. Point an AI agent at /ask with a natural language query and it gets back structured Schema.org JSON — precise, citeable, not a wall of HTML to parse. The /mcp endpoint exposes the exact same functionality as an MCP server.
That second part is the punchline. Implementing NLWeb means your website immediately becomes an MCP server — queryable by Claude, GitHub Copilot, Cursor, or any other tool in the MCP ecosystem. Anthropic defined MCP as the protocol for AI tools. NLWeb turns your content into one of those tools.
The technical foundation is deliberately boring: Schema.org (used by over 100 million websites), JSON-LD, Server-Sent Events for streaming. NLWeb did not invent new infrastructure. It built a natural language layer on top of existing structured data. If your site already has schema markup, you are already halfway there.
The protocol was built by R.V. Guha, who also created RSS and Schema.org. Those became invisible infrastructure of the web. NLWeb is playing the same long game.
How to Implement It: The 30-Minute Path
Cloudflare added native NLWeb support via AutoRAG earlier this year. If your domain is on Cloudflare, this is the fastest path:
- Log in to the Cloudflare dashboard
- Navigate to Compute & AI → AutoRAG
- Select Create AutoRAG, then choose NLWeb Website
- Select your domain
- Cloudflare crawls and indexes your site semantically, then deploys a Worker
That Worker exposes your /ask and /mcp endpoints automatically. You do not write a line of server code. The full process takes about 30 minutes, most of it waiting for the crawl to finish. Full documentation is at Cloudflare’s AutoRAG NLWeb guide.
The Self-Hosted Path: Full Control
If you want to own the infrastructure — or you are not on Cloudflare — Microsoft’s Python reference implementation is the starting point. Clone the repo, edit config_retrieval.yaml to point at your vector store of choice (Qdrant, Postgres, Elasticsearch, Azure AI Search, and others are supported), set your LLM endpoint (OpenAI, Anthropic, Gemini, DeepSeek), and run the server.
The trade-off is straightforward: Cloudflare handles everything but limits your flexibility. The Python path gives you full control over your data, your model choice, and your deployment. It runs on a laptop. It runs on a cluster. Pick your infra.
NLWeb and WebMCP Are Not the Same Thing
We covered WebMCP on ByteIota yesterday — worth clarifying the difference because the confusion is everywhere.
NLWeb is server-side. It runs a RAG pipeline: crawl your content, index it, answer natural language queries via /ask, expose tools via /mcp. It is best for content-heavy sites — documentation, blogs, e-commerce catalogs, anything with depth.
WebMCP is client-side. It runs in the browser, lets websites expose JavaScript functions directly to in-browser AI agents. No backend required. It is better for interactive UIs — forms, multi-step tools, anything where an agent needs to take actions rather than read content.
Notably, Google and Microsoft are co-authoring the WebMCP standard together — so the apparent competition is actually a collaboration. The practical guidance: implement NLWeb for your content layer, add WebMCP for interactive features. They stack.
Schema Markup Is Now Your Agent-Readiness Score
Here is the understated consequence of NLWeb going mainstream: your existing investment in structured data is now directly tied to AI agent accessibility. NLWeb processes Schema.org JSON-LD natively. Sites that have done the structured data work get a RAG pipeline that actually works. Sites with thin or missing schema get worse results.
Yoast SEO added NLWeb-compatible Schema Aggregation in March 2026. Shopify built “Agentic Storefronts” on top of NLWeb — the ability to sell through AI conversations, not just on shopify.com. TripAdvisor, O’Reilly, Eventbrite, and Hearst are all early adopters. This is not emerging technology anymore. It is beginning to solidify.
The broader implication: AI agents are building preference patterns now. The sites they can query reliably are the sites they will keep returning to. For the full background, the Microsoft NLWeb announcement lays out the design goals directly.
Add NLWeb Before Your Site Becomes a Blind Spot
This is not a two-year problem. AI agents — coding assistants, shopping agents, research tools — are live and in production workflows today. Most of them currently deal with your site by scraping HTML and making their best guess. NLWeb gives them a better option. The question is not whether to implement it. It is whether you want to do it now or play catch-up.
Start with the Microsoft NLWeb announcement for the full vision, then go straight to the GitHub repo or the Cloudflare dashboard. Thirty minutes from now, your site could be speaking MCP.
