Phone Harness

A harness that puts a real phone at the end of an agent’s tool calls. The agent writes a short Python script — open_app, tap, tap_text, type_text, screenshot, ocr — and the harness runs it against the phone, then hands the screen back so the agent can check what happened. Two backends share one set of helpers. On an iPhone the Mac’s iPhone Mirroring window is the phone: the harness captures that window, reads text out of it with Apple’s Vision framework and returns tap-ready coordinates, and posts HID-level mouse and keyboard events for taps, swipes and typing. On Android, adb is the transport — screencap for the picture, the phone’s accessibility tree for text, input for the hands — over USB or Wi-Fi, with no window involved. Nothing is installed on the phone and there is no jailbreak, and when there is no device on the desk a hosted service rents Android phones, and iPhones, that answer to the same helpers.

Who built itThe repository is effectively one person: the GitHub account ShawnPana wrote 97 of its 98 commits, 66 of them from an address at ucsd.edu, and the contributor list in the recon report carries that one name. The README says the project is free and maintained in the author’s own time, and a funding file and a sponsor section ask for support. 52 of the 98 commits carry a co-author trailer and 48 of those name a Claude model — Claude Fable 5, Claude Fable 5.1 or Claude Opus 5 — so most commits arrive with a model credited beside the author. The remaining commit comes from an unlinked account, ayataro48, which GitHub does not tie to a user.

1. 01

   Seven weeks, three releases, one backend nobody planned for

   The repository was created on 2026-08-07, and the first commit landed the same evening: “phone-harness v0: iPhone control via iPhone Mirroring”. The report counts 98 commits — 82 in August and 16 in September — with the newest dated 2026-09-20, while the API records the last push on 2026-09-27. Six tags exist (0.1, 0.1.0, 0.1.1, 0.2.0, 0.2.1, 0.3.0) and three of them are full releases, and the release titles are the roadmap in three lines: 0.1.0 on 2026-08-17, 0.2.0 — Android the next day, 0.3.0 — Cloud (Android) on 2026-09-20. So the iPhone path, which is the oldest code in the tree, was joined by an Android backend within eleven days and by a hosted one about a month later. Around that sit 3,120 stars, 321 forks and 16 watchers on a 445 KB Python package under the MIT licence, with topics that read agent, ai, automation and developer-tools, and with 55 open issues and pull requests as the API counts them.

2. 02

   Two ways to reach a phone, one vocabulary

   The README gives one paragraph to each platform, and the file sizes say where the work went: src/phone_harness/android.py is 42 KB, the largest file in the repository, followed by cloud.py at 34 KB and mirror.py at 29 KB, against ocr.py at 2.1 KB. On Android the transport is adb — screencap produces the picture, the phone’s accessibility tree is the text source, input is the hands — over USB or Wi-Fi. On iPhone the phone is a Mac window, so the harness captures that window, OCRs it through Apple’s Vision framework for text with tap-ready coordinates, and posts HID-level events for taps, swipes and typing. A script arrives on standard input with the helpers already imported; the README opens with the shape of one, where the agent calls find_text("Weather"), gets (400, 468), taps there, reads the screen and reports that the forecast is up. What the two backends share is a contract, not a coincidence: tests/test_backend_contract.py holds them to it, and phone-harness config set platform ios|android chooses a default rather than a second vocabulary. Setup is a prompt pasted into an agent: clone into ~/.phone-harness, read install.md, put phone-harness on the PATH, register it as a skill and walk through onboarding.md.

3. 03

   The permission belongs to whichever app launched it

   On macOS the harness needs Accessibility and Screen Recording, and PR 104 explains why the old advice was useless: “When the harness is launched from an agent app (Cursor, VS Code, Grok Bot, …), TCC attributes the grant to that host app, not the terminal”, so a terminal that already holds both permissions does not help, and Accessibility never surfaced a dialog. The proposed doctor requests each missing grant, names the responsible app, tells the user to quit and reopen it, waits up to 45 seconds “only when stdin is a terminal” and returns immediately under an agent; --doctor ios --fix opens the exact Settings pane. Android asks for developer options and an adb approval, and the README draws the line plainly: “Connecting the phone is always the user’s job”. The hosted side writes its rules down: onboarding says “Never sign them up or click Join for them; never spend their credit beyond the proof”, grants 500 cents of credit on signup, calls the first 100 minutes free, caps a session at 30 minutes through --minutes, and PR 82 makes cloud start --temp refuse to reuse a saved phone, which could otherwise “operate on persistent data”. Content is the other boundary: telemetry.py ships in the repository, and PR 100 removes script contents, console output and raw errors from the usage events while saying it “does not implement the suggested first-run notice or content opt-in”.

4. 04

   A month of outside pull requests, nearly all of them open

   Thirty issues and pull requests appear in the report, numbered 78 to 108 with 90 absent, and few of them belong to the author. DivyamTalwar sends a long run of hardening work on cloud.py, android.py and helpers.py, each change reproduced against the named commit 88b3642ac4733c47daf12e0f2837bc789eda63c6, and each explicit about what it does not claim — the pagination fix is labelled lower-frequency defensive hardening, not a fault seen in the service. KingAmo files three issues that read like a second audit: the test suite opens eight real browser tabs on a developer’s machine, --doctor ios crashes on exactly the capture failure the previous check exists to report, and ocr() never sets recognition languages, so a Chinese phone comes back as Latin gibberish. The one comment on those thirty threads is a correction: a host language tag with a region was assumed to be rejected, was tested, and ["zh-Hans-CN"] returned 22 boxes correctly. Elsewhere the reports are shorter — henrikra on background mode freezing the phone’s view when the Mirroring window is covered, D-E-A-G on a pyobjc call that breaks OCR on macOS 27 — and two contributors arrive with code: Sam780214, verifying a fix on Arch Linux against a Huawei NOH-AN00, and dishanest, whose replacement for his own earlier pull request types on the iPhone without taking the Mac’s focus.

5. 05

   What the author says it cannot do

   The README closes with four limits, written as limits: unlocking the iPhone pauses mirroring and a PIN-locked Android needs the user; “OCR sees text, not icons”, so an unlabeled control needs a screenshot and a vision model; there is no multi-touch, no camera or Face ID flow, and DRM video is black. The same instinct shows in the documentation: SKILL.md was restructured from 311 lines to 272 with “nothing dropped”, so a test agent no longer read about 150 lines of iPhone Mirroring caveats before reaching the cloud section; the new order runs which phone, then a method for all of them, then cloud, Android and iPhone. Onboarding was reordered twice in one weekend: the cloud offer moved from before Verify and Demo to step 6 after it, since “a new user was pitched a second phone before seeing their own work”, and three strings that assumed a waitlist were rewritten when Clerk’s sign-up mode flipped to Open. The compiler-shaped failures are in the history too: PR 83 bundles four fixes, one commit each — a script passed as the CLI’s single argument printing the usage banner, an Android host without pyobjc dying in screen_info() with ModuleNotFoundError: No module named "Quartz", and emoji or CJK output raising on a cp936 or cp949 console. The macOS-only import that broke Android on Linux and Windows is the subject of PR 108, still open, which reads dimensions from the header.