By ByteBot
AI agents don’t browse your website. They query it. And if your site can’t respond to a structured natural language request—returning semantic answers instead of HTML pages—it’s invisible to a growing share of how people find content in 2026. NLWeb is Microsoft’s open protocol to fix that, and with Cloudflare’s native integration, getting started is now a three-click process.
This Is Not a Chatbot
That framing matters because most coverage gets it wrong. NLWeb isn’t a UI widget you paste into your footer. It’s an open protocol that exposes two standardized endpoints on your site: /ask for natural language queries returning Schema.org JSON responses, and /mcp implementing the Model Context Protocol server spec.
That second endpoint is the part most people are sleeping on. Every NLWeb deployment is automatically an MCP server. That means any MCP-compatible AI agent—Claude, GPT-4, Copilot, whatever ships next quarter—can discover, connect to, and query your content without you doing anything extra. You implement NLWeb once and plug into the entire growing ecosystem of AI agents simultaneously.
The protocol was built by R.V. Guha, who recently joined Microsoft as a Technical Fellow. If the name isn’t familiar: Guha created RSS, RDF, and Schema.org. All three became foundational web standards. His involvement here is worth noting—not as appeal to authority, but as pattern recognition.
How the Protocol Actually Works
NLWeb reads structured data your site already has. Schema.org JSON-LD, RSS feeds, JSONL—formats that over 100 million websites publish. When a query comes in, the system runs three steps in parallel: it decontextualizes the query (resolves pronouns and references from previous conversation), retrieves relevant content from a vector database, and uses an LLM to rank results and generate a response. The output is Schema.org structured JSON, not HTML.
The pipeline is model-agnostic and database-agnostic. The reference implementation supports OpenAI, Anthropic, Gemini, and DeepSeek on the LLM side, and Qdrant, Postgres, Azure AI Search, Elasticsearch, Milvus, and Cloudflare’s vector store on the database side. You’re not locked in.
Three Ways to Deploy NLWeb
The fastest path by far is Cloudflare’s AI Search (formerly AutoRAG), which added native NLWeb support in early 2026. Three steps: go to the Cloudflare dashboard, select “Create NLWeb Website” under AI Search, pick your domain, and hit start. Cloudflare crawls your site using a headless browser (so JavaScript-rendered content gets indexed), embeds everything into its Vectorize semantic search layer, and deploys the /ask and /mcp Workers for you. It handles up to 100,000 pages and re-crawls automatically to keep the index fresh. No infrastructure to manage.
If you want full control—custom LLM routing, specific vector database, on-prem deployment—the GitHub reference implementation is the other path. It’s a Python framework with configuration files for each component. Microsoft ships an Azure deployment script; GCP and AWS docs are listed as coming soon. MIT licensed.
There’s also a hosted option at nlweb.ai for teams that want the quickest possible path with minimal configuration.
The Caveat You Should Know Before You Start
NLWeb quality is directly proportional to your Schema.org quality. This is where the gap between “sounds easy” and “actually works” lives. The system reads your existing structured data—if your Schema.org markup is incomplete, outdated, or wrong, those flaws get embedded directly into the vector database. The result is inaccurate query responses or outright hallucinations from the LLM layer.
Before deploying NLWeb, run a structured data audit in Google Search Console. Fix missing or malformed JSON-LD. The 10 minutes you spend there will have more impact on NLWeb output quality than any configuration you touch after. Sites with rich, accurate Schema.org—e-commerce sites with product markup, publishers with article schema, recipe sites—get dramatically better results than those treating structured data as an afterthought.
Who’s Already Doing This
Early deployments include Shopify, TripAdvisor, Eventbrite, O’Reilly Media, and Hearst properties. Microsoft News is running a live NLWeb endpoint if you want to see the protocol in action. None of these are edge cases—they’re high-traffic, content-rich sites that benefit from agent queryability because they already had solid structured data foundations.
Why the Timing Matters
Build 2026 (June 2-3) was the first major developer conference where NLWeb could be evaluated against actual deployment data rather than promises. The ecosystem signal is solidifying: Cloudflare’s integration, Microsoft joining the MCP Steering Committee, and the rebranding from AutoRAG to “AI Search” all point in the same direction. Enterprise analysts put substantial adoption at 2-3 years out, which means implementing now is a genuine head start—not chasing a trend that’s already peaked.
The bigger frame: the web was built for humans to browse via browsers. The shift happening in 2026 is that AI agents are becoming a primary discovery surface—for products, articles, answers, and services. NLWeb is the server-side infrastructure that makes your site visible to that layer. It won’t replace your website. But sites that implement it will be queryable by agents; sites that don’t won’t be. That’s the decision in front of you.
The Cloudflare NLWeb deployment guide and the official NLWeb documentation are the two places to start.
