Handsy AI is an open-source AI agent that controls a stock iPhone through a roughly $7 ESP32 board—no jailbreak required—by watching the screen via AirPlay and tapping back over USB.
About
Handsy AI is an open-source project that lets an AI agent operate a real, unmodified iPhone. Instead of relying on jailbreaks, private APIs, or developer mode, it uses a small ESP32 microcontroller board as the agent's physical interface with the phone. This solves a common problem for iOS automation: Apple's platform is largely closed to programmatic control, so agents that could freely operate Android devices have had no comparable path on a stock iPhone.
The project was created by Henry Yuan and is distributed under an MIT license, with the firmware's AirPlay receiver component licensed under GPL-3.0. The setup works in three parts. The ESP32 board hosts its own Wi-Fi network and appears in iOS Screen Mirroring, receiving the phone's AirPlay stream. It forwards that stream to the ESP32 Agent app over Bluetooth, and the app sends each screenshot to a vision model of the user's choosing, which plans the next action. The board, plugged into the phone's USB port, presents itself as an ordinary mouse and keyboard, so taps, drags, and swipes work through iOS AssistiveTouch.
Key Features
Pricing & Fees
- Pricing: The software is open source and free to build from source; the required hardware costs roughly $7 for the board.
- Supported devices: Any iPhone on iOS 18 (iPhone XS or newer). USB-C models connect directly; Lightning models require a USB camera adapter.
- Installation: A browser-based firmware installer is provided on the official site, along with a Simplified Chinese version and an AI-assisted installation guide.
- Extra hardware needed: A short USB-C to USB-C data cable connects the board to the phone.
Who It's For
Handsy AI suits hobbyists and tinkerers who enjoy combining cheap microcontrollers with AI, developers researching on-device agents and embodied control, and anyone interested in automating iOS tasks without jailbreaking. It also appeals to hardware enthusiasts already working with ESP32 boards, and to experimenters who want to run agents against a real phone screen rather than a simulator. Typical uses include UI automation experiments, accessibility research, and demonstrations of vision-driven agents completing multi-step tasks such as setting a repeating alarm.
FAQ
What is Handsy AI?Handsy AI is an open-source AI agent system that controls a stock iPhone using a roughly $7 ESP32-S3 board, which watches the screen through AirPlay and taps back over USB.
Does Handsy AI require a jailbreak?No. It works on an unmodified iPhone running iOS 18, and a computer is needed only once to flash the board's firmware.
How much does Handsy AI cost?The software is free and open source; the main cost is the hardware, with the XIAO ESP32S3 board priced around $7.
Which iPhones and boards are supported?Any iPhone XS or newer on iOS 18 is supported, and any ESP32-S3 board with native USB and PSRAM can replace the XIAO reference board.
Does Handsy AI send data to the cloud?The app sends screenshots to whichever vision model the user chooses, and the creator has demonstrated a fully local setup with no cloud APIs, so inference can stay on the local network.
Related Tutorials
View all →
Related Tools
View all →
OmniRoute - Free AI Gateway for Multi-Provider LLMs Free, open-source AI router with auto-fallback. 236 providers, one endpoint, 95 MCP tools, 17 routing strategies, A2A protocol, auto-combo engine, semantic cache, memory & skills. Deploy anywhere. 
Dograh - Voice Assistant Workflow Builder Open Source Voice Assistant Workflow Builder. 
OpenHands Open-source, model-agnostic platform for building and deploying cloud coding agents. Automate real engineering work securely with full transparency and control. 
Agent-Native Apps Framework Build agentic apps where AI agents and UI share the same database and state. Open source framework with ready-to-fork templates. 
LangBot - AI Chat Assistant LangBot is an AI chat assistant that connects to QQ and other platforms, providing intelligent responses and automation.