Exo: Run AI Clusters on Phones, Laptops, Smartwatches

While cloud providers race to build billion-dollar AI data centers, an open-source project trending on GitHub is proving you can run AI clusters using the devices already in your pocket. Exo enables distributed AI inference across phones, laptops, desktops, and even smartwatches—no expensive GPU servers required. Last updated December 27, 2025, the project is making waves by achieving datacenter-class performance on consumer hardware.

What Is Exo?

Exo is an open-source distributed AI inference framework that lets developers pool everyday devices into a single AI cluster. Unlike traditional distributed systems that rely on master-worker architecture with a central coordinator, exo uses true peer-to-peer connections. Devices discover each other automatically via UDP broadcast every 2.5 seconds and connect directly—no configuration required.

This architectural choice eliminates single points of failure and makes the system more resilient. As long as a device is connected somewhere in the network, it can contribute to running models. Each device provides an API and dashboard at localhost:52415 for cluster interaction. The project hit GitHub trending on December 22, 2025, as part of a broader wave of democratized AI infrastructure tools.

Datacenter Performance on Consumer Devices

The breakthrough is RDMA (Remote Direct Memory Access) over Thunderbolt 5, enabled by macOS 26.2 released December 12, 2025. This reduces latency from 300 microseconds down to 5-9 microseconds—a 99% improvement that matches the performance of datacenter InfiniBand connections. By bypassing traditional networking stacks and enabling direct memory-to-memory transfers, RDMA unlocks speeds previously impossible on consumer hardware.

Real-world benchmarks validate the claims. Developer Jeff Geerling tested a 4-node Mac Studio cluster and achieved 1.5TB of combined VRAM running at 80Gb/s with sub-10 microsecond latency. On the Qwen3 235B model, a single device runs at 19.5 tokens per second. Scale to four nodes with RDMA, and exo delivers 31.9 tokens per second. Compare that to llama.cpp, which actually gets slower with more nodes: 20.4 tokens per second on one device dropping to 15.2 on four.

For massive models, the gap widens further. DeepSeek V3.1 with 671 billion parameters runs at 32.5 tokens per second on a four-node exo cluster versus just 14.6 tokens per second with llama.cpp over TCP. The Kimi K2 Thinking model with 1 trillion parameters reaches 28.3 tokens per second in the same configuration. These numbers prove you don’t need a data center to achieve data center performance.

Cost Democratization for Indie Developers

Cloud GPU costs are prohibitive. AWS charges over $4 per hour for a single A100 GPU. Budget alternatives like RunPod and Lambda Labs offer better rates at $0.27 to $2.99 per hour, but that’s still ongoing costs that add up fast for indie developers and small teams.

Exo flips the model: $0 per hour, forever. Use the devices you already own—that old MacBook, your phone, even a smartwatch. Pool them into a cluster and run models that would otherwise require renting cloud infrastructure. No VC funding needed. No IT budget required. Just existing hardware and an internet connection.

This isn’t just about saving money. It’s about privacy and control. Your data never leaves your devices. No third-party API calls. No cloud vendor lock-in. For developers building privacy-conscious applications or needing to meet GDPR and HIPAA requirements, self-hosted inference isn’t optional—it’s mandatory. Exo makes it accessible.

Production Reality Check

Before you decommission your cloud GPUs, understand what exo is and isn’t. An October 2025 technical analysis identified critical gaps in security, fault tolerance, and operational tooling. The framework is classified as suitable for research and development, not production-ready for mission-critical workloads.

That’s not a weakness—it’s honesty. Exo excels at prototyping, experimentation, and learning distributed AI systems. It’s perfect for indie developers testing ideas before committing to cloud costs. It’s ideal for students and researchers exploring AI without institutional budgets. It’s a development tool, not a replacement for enterprise infrastructure.

Hardware requirements matter too. RDMA performance requires macOS 26.2 or later and Thunderbolt 5-equipped Macs (M4 Pro, M4 Max, or M3 Ultra). Linux support exists but runs on CPU with GPU support still under development. The barrier to entry is low, but optimal performance needs recent hardware.

The Bigger Shift Toward Decentralization

Exo isn’t an isolated curiosity. It’s evidence of a fundamental shift in how AI infrastructure is evolving. Industry analysis describes decentralized infrastructure as “the new standard for AI workloads” that demand speed, scale, and security. AI is moving beyond hyperscale cloud zones—running on streets, in factories, inside hospitals, and at the edge of networks.

The drivers are clear: low latency requirements for real-time decisions, massive bandwidth costs when moving data to centralized clouds, regulatory compliance demanding data stay on-premises, and high availability concerns about single points of failure. Centralized cloud AI solves some problems. Distributed, self-hosted approaches solve different ones.

Exo challenges a core assumption: that high-performance AI requires corporate infrastructure. It’s the same democratization narrative playing out across development tools in 2025—from TypeScript surpassing Python as GitHub’s most-used language to frameworks making sophisticated technology accessible by default. Developer empowerment isn’t a buzzword. It’s projects like exo proving alternatives exist.

Try It Yourself

The project is open source and available on GitHub at exo-explore/exo. Requirements are Python 3.12.0 or higher and compatible devices on the same network. Installation involves cloning the repository, building the dashboard, and running the exo command. From there, automatic discovery handles the rest.

For deeper technical implementation details, Leif Markthaler’s Medium deep dive on distributed AI inference covers architecture, inference layers, and optimization strategies. Jeff Geerling’s 1.5TB VRAM benchmarking post provides real-world validation and performance data.

This is AI infrastructure for the 99%. Not because it’s worse than cloud alternatives, but because it’s accessible without corporate budgets. You don’t need a data center to run AI. You just need exo and the devices you already own.