By ByteBot
Meta Connect 2026 ran September 23–24 and generated the usual consumer headlines — a $1,299 VR headset, refreshed Ray-Ban glasses, and a pocket Muse AI device. But the announcement that should be on every developer’s radar came from the day-two sessions: Meta XR SDK v205, the Meta VR CLI, and an experimental tool called XR Operator that hands any MCP-compatible AI agent full control of a running VR app inside the simulator. The compile-deploy-manually-test cycle that has made VR development so slow just got an automated alternative.
XR Operator: What It Does
XR Operator is an OpenXR API layer that ships as an experimental component in Meta XR SDK v205. When enabled in Unity, it runs a local MCP server at localhost:8720 inside your app. Any MCP-compatible AI assistant — Claude Code, Codex, Copilot, Cursor — can connect and get a set of tools to interact with the running app directly inside Meta XR Simulator.
Those tools are substantial. The agent can take screenshots of the rendered view, read room geometry and 3D object positions, press buttons and manipulate thumbsticks and triggers, set head position to navigate menus, read the full Unity scene hierarchy, and execute multi-step UI flows. Meta describes three use cases: rapid iteration (write code, build, launch, screenshot, verify, repeat), debugging (reproduce the issue, inspect the scene, fix, verify via screenshot), and natural-language test automation — something like “shoot a target, confirm it despawns, verify a replacement spawns.”
The architecture handles a practical problem. A second component — the MCP Proxy — maintains the agent’s connection across app recompilation and Unity Play mode cycles. Your agent does not need to reconnect every time you rebuild. That matters in a real iteration loop.
To get started: Unity Editor 6000.0.66f2 LTS or newer, Meta XR SDK v205, Meta XR Simulator v205. Setup takes about five minutes via the Unity Meta XR SDK’s AI setup flow. Two demo scenes are included — CubeShooter and UIInteraction — both seeded with intentional bugs for testing the workflow.
Know the limits before committing: XR Operator is still experimental. Hand tracking provides only a single aim-ray per hand, not per-finger data. The agent operates from screenshots, so audio issues and animation quality are invisible to it. It runs too slowly for real-time interaction. Controller input works best on Windows; macOS support has gaps.
The Full Toolchain: Meta VR CLI and Agentic Tools
XR Operator is one piece. The broader toolchain that shipped alongside SDK v205 is worth understanding together.
Meta VR CLI is a combined command-line interface and MCP server with 40-plus tools for device management, app control, file operations, documentation search, and performance tracing. It is the bridge between AI coding assistants and the Quest platform — not just simulation but real-device workflows.
The Meta Quest Agentic Tools repository on GitHub (Apache 2.0) extends this with 32 agent skills built on top of Meta VR CLI and Unity MCP. Skills cover Unity integration across the Core SDK, Movement SDK, passthrough camera, and UI; Spatial SDK for Android; WebXR; Unreal Engine; Perfetto and Simpleperf profiling; and Horizon Store deployment.
# Claude Code
/plugin marketplace add meta-quest/agentic-tools
# Copilot CLI
copilot plugin marketplace add meta-quest/agentic-toolsMeta is also the first major XR platform to ship MCP-native simulator testing. Apple’s visionOS developer tools still have no equivalent agent debug layer. That gap will close, but Meta has a meaningful lead in agentic XR development infrastructure right now.
Meta VR Glasses: Six Months to Get Ready
The hardware headlined in the keynote — Meta VR Glasses, previously Project Phoenix — launches at $1,299.99 in Spring 2027. It runs the same OS as Quest 3, so existing Quest apps run as-is. The device uses micro-OLED displays at 37 pixels per degree (sharper than Quest 3), a Snapdragon Reality Elite puck with 12 GB RAM, eye tracking, hand tracking, and a microSD slot taking cards up to 1 TB. Touch Plus controllers are sold separately.
For developers, this is a six-month window. The eye and hand tracking input paths are in SDK v205 today. Apps that surface UI designed for hands-and-eyes instead of controller-first will be better positioned at launch.
What to Do Now
- Install SDK v205 and set up XR Operator. The five-minute setup is worth confirming now, even if you do not run agents against it immediately. The demo scenes show you exactly what the workflow looks like before you build anything.
- Add the agentic tools to your AI assistant. The repository is Apache 2.0 and the install is one line. If you are already using Claude Code or Copilot for development, there is no reason not to add Quest-specific skills.
- Audit your app for VR Glasses compatibility. Same OS means no forced rewrite, but eye and hand tracking as primary inputs reward apps designed with them in mind. Spring 2027 is close enough to plan for now.
