By ByteBot
AI agents already drive browsers, terminals, and desktop apps. The last holdout has been the iPhone — locked behind Apple’s XCTest signing wall, where every automation tool required Appium, a provisioning profile, and a tolerance for taps that silently land in the wrong place. iphone-use, an open-source tool released October 6, changes that. It gives agents a 21-tool MCP server, honest action feedback, and a screen reader that sends structured text — not screenshots — keeping token costs low. It picked up 2,500 GitHub stars in its first week.
The Problem With iOS Automation
Android agents have had it easy for years. A single adb command gives an agent direct device control — no signing, no provisioning profiles, no Xcode. iOS is different. Apple gates all on-device input injection through XCTest, a signed test runner that must be compiled and installed through Xcode. Every automation tool from Appium onward is a wrapper around this constraint — which means they all share the same failure modes: silent taps that land in empty space, no confirmation that an action succeeded, and expensive screenshot-based feedback for every step.
For AI agents, silent failures are expensive. An agent that taps a button and assumes success will continue down the wrong branch until something more obvious breaks — burning context and tokens on a task that went wrong three steps ago.
What iphone-use Does Differently
iphone-use ships a 21-tool MCP server alongside an HTTP API and CLI. The design decision that sets it apart: every action returns an explicit outcome state. Actions come back as applied, not sent, or unknown. Agents get something they can branch on — retry, escalate, or recover — instead of a silent non-answer.
The tool set covers the full iPhone interaction surface: tap by coordinate, element, or visible label; type text; scroll; read the screen; list interactive elements; press hardware keys; and record and replay complete flows. Setup is a one-line addition to .mcp.json:
{
"mcpServers": {
"iphone-use": {
"command": "npx",
"args": ["iphone-use"]
}
}
}Claude Code, Codex, Cursor, and any other MCP client picks it up automatically from there.
Screen as Text, Not Screenshots
The token efficiency story is worth highlighting. iphone-use reads the iPhone’s accessibility tree and sends it to the model as compact structured text. A typical screen compresses from 8.7 KB of raw accessibility JSON to 694 bytes. Screenshots are captured after each action but held back unless the agent specifically requests one. For multi-step tasks on dense screens, the savings compound quickly.
The native XCTest runner (replacing WebDriverAgent in recent versions) also speeds things up: label taps settle in 1.7–2.1 seconds, compared to 4.2 seconds through the older WebDriverAgent path.
Replayable Flows: The Feature That Matters for Real Use
Once an agent completes a task, iphone-use captures the sequence as a replayable flow. Running it again costs zero model tokens — the flow executes directly. This changes the economics of repetitive mobile workflows. Tasks like checking a notification, confirming a delivery, or logging a health metric get defined once and run indefinitely without re-invoking the model. It is not glamorous, but it is the feature that will make this useful in production rather than just impressive in demos.
Who This Is For
The tool was built with a specific use case in mind: apps that have no public API. That means Chinese super-apps — WeChat, Taobao, Meituan, Xiaohongshu — where the entire interaction surface lives inside the app and no REST endpoint exists. It also works for mobile QA engineers who want agent-driven test automation, and for developers building personal workflows that span a mobile app and a desktop process.
The Product Hunt launch drew strong interest from both Western and Chinese developer communities; the repository ships documentation in both English and Chinese.
Setup and Caveats
The Mac requirement is non-negotiable — this is a constraint from Apple’s architecture, not a design choice by the iphone-use team. You need macOS, Xcode, a USB cable for initial pairing, and a free Apple Developer account for signing. After pairing, Wi-Fi operation is supported. The tool is at v0.17.x and actively developed; see this writeup on the challenges of driving a real iPhone from Claude Code for a realistic view of what the setup actually involves.
Multiple iPhones are supported simultaneously — each gets its own daemon — which is useful for parallel testing or multi-device workflows.
iOS automation has lagged Android for years because of Apple’s signing requirements. iphone-use does not eliminate the complexity, but it wraps it in a shape that MCP-aware agents can actually use. That is the gap that has been missing.
