Ghost in the Droid

Ghost in the Droid

Summon a ghost into your phone.

It sees the screen. It taps the buttons. It never sleeps.

Website · Docs · Skill Hub · Releases

---

▶ Watch full HD with sound on YouTube · Download mp4

Nine agents. Nine real devices. One ghost. (Click to watch.)

---

The pitch, in one breath

Every AI agent can think. Almost none can touch a phone.

Ghost is the body. It gives any LLM agent a real Android phone or iPhone: it reads the screen, taps, swipes, and types through a clean tool surface, and it scales from one device on your desk to a whole phone farm. Point Claude Code, Codex, Antigravity, Cursor, your LangChain app, or a model running *inside the phone itself* at it, and your agent grows hands.

Bring your own brain. Keep the same body. MIT, forever.

---

Mix and match: the Ghost matrix

Ghost 1.3 is the only Android + iOS agent framework where platform, brain, and driver are all swappable. Pick one from each column. They all compose.

The Docker + KVM emulator pool is Android-only; iOS uses Apple's own simulator and Xcode signing, which are macOS-native.

The body stays the same 62 MCP tools no matter what you plug in. Every new model release is a free upgrade to your phone agent.

---

See it in action

Nine demos, one per feature, each recorded on real devices. Click any clip for the full mp4.

MCP clients: point your agent at a phone

Claude Code : type in the TUI, watch it drive the phone.

Codex : OpenAI's CLI operating a real phone over MCP.

Antigravity : Google's agent, same 62-tool body.

Interfaces: CLI, framework, browser

Ghost CLI : ghost "check reddit" --device asus .

LangChain : Ghost's tools in a framework agent.

Web chat : chat in your browser, it drives a real phone.

On-device and iOS: the phone as body and brain

On-device Android : model runs in the app, airplane mode.

On-device iPhone : your iPhone talks to itself.

iPhone, real app : Ghost does the Duolingo lesson.

---

What It Does

Open-source Python framework for controlling Android and iOS devices from one agent harness. Android runs through ADB. iOS runs through Appium XCUITest and WebDriverAgent, with real iPhones as the target path and simulators for development and CI.

Define skills for any app, run them from the dashboard or API, scale across a phone farm.

The ghost taps what you'd tap

• Android control through ADB: tap, swipe, type, clipboard, shell, intents, and stealth variants
• iOS control through WebDriverAgent: screenshot, accessibility tree, tap, swipe, type, app launch, clipboard, browser actions
• Live phone screen streaming: Android MJPEG/WebRTC, iOS WDA MJPEG with screenshot fallback
• Interactive touch-to-tap on the streamed screen
• Multi-device phone farm with per-device job queues

Forge reusable skills for any app

• YAML-based UI element definitions per app
• Platform-specific selectors with elements.yaml for Android and elements_ios.yaml for iOS
• Python action classes with precondition checks
• Multi-step workflows that chain actions together
• Built-in skills for TikTok and Play Store
• iOS browser/news demo skill and smoke-level TikTok iOS workflows
• Skill Hub: browse, search, and install skills from the community registry
• Install from CLI: android-agent skill install tiktok

Teach the ghost new tricks

• BFS-based auto app explorer: discovers every screen and transition
• LLM-assisted Skill Creator: chat with AI while viewing the live device stream
• The AI identifies UI elements and generates action/workflow code

Scale the haunting

• Multi-device phone farm with per-device job queues
• Bot runner: queue, schedule, and monitor automation jobs
• Per-device integration tests with Android screen recording or iOS WDA MJPEG recording

Platform Support

---

Why it is built this way

Skills cost $0. Thinking costs tokens. An agent should not pay an LLM to tap a button it has tapped a thousand times. Known workflows compile to skills: deterministic YAML + Python recipes that replay in seconds with zero LLM calls. Use AI for the unknown task, replay for the known one.

The phone is the easiest sandbox there is. Give the ghost an old Android with its own SIM and its own accounts, physically separate from your life. You already know how to set up a phone.

Zero-app by default. A fresh install touches nothing on the Android device: screen reads go through uiautomator, actions through adb shell input, no root, no accessibility service. Want it faster? The optional Portal companion app gives roughly 30x quicker UI reads. Want it private? On-device mode runs the whole loop inside the phone, nothing leaves it. All opt-in.

Local-first, cloud-optional. The server and dashboard run on your machine. With a local or on-device brain, your screenshots never leave your network. Pick a cloud brain and prompts go to that provider, same as any tool. Your call, always.

---

Requirements

• Python 3.10+
• Android: Android phone with USB debugging enabled and ADB on PATH (adb devices should list your phone)
• iOS: macOS with Xcode, Appium 2 + XCUITest driver, and a trusted iPhone or booted simulator
• Node.js 18+ (for the frontend dev server)

---

iOS Support

Ghost drives iPhones through Appium/WebDriverAgent with the same tool surface: ios: device refs route tap/swipe/type/screenshot to WDA, and iOS-aware browser primitives (open_url, read_news, extract_visible_text) cover web tasks. Android-only tools (shell, launch_intent, Portal overlay) return a clear platform error instead of failing silently.

iOS is opt-in: enable it with GITD_ENABLE_IOS=1 (or ios_platform_enabled=true in .env). Wireless drive rides your Tailscale tailnet, so treat the tailnet as the trust boundary. See docs/SETUP_IOS.md for Appium/WDA setup, and on-device on iPhone to run the model on the iPhone itself.

---

Quick Start

Zero-install with uvx (or pipx):

uvx ghost-in-the-droid doctor   # green/red preflight: python, adb on PATH, devices, ports, LLM keys

uvx ghost-in-the-droid login # sign in with your Claude subscription, no API key needed uvx ghost-in-the-droid up # server + dashboard at http://localhost:5055 (API docs at /docs)

doctor prints a checklist with fix hints instead of a stack trace when something is missing (e.g. adb not on PATH). Prefer pipx? pipx install ghost-in-the-droid gives you the `ghost-