UI Debugger MCP

UI Debugger MCP

An MCP server that debugs UIs autonomously — so the AI that wrote your app can also test it, without a human clicking through every flow.

The problem

AI coding agents (Claude, etc.) are great at writing code. They're bad at knowing if the UI actually works. For backend code there are unit and integration tests. For UI, a human still has to open the app, log in, click around, and report what's broken. That human-in-the-loop is slow, boring, and the main bottleneck when an entire product is built by AI.

The idea

Eliminate the human from the UI-debug loop with an MCP server.

• A smart agent (Claude Code, Cursor, …) finishes a PR and wants to verify the UI.
• It hands a story to this server: *"on web, log in and do X, Y, Z — tell me if it breaks."*
• A small fast agent runs inside this server (via the Vercel AI SDK). It drives the browser or desktop, watches console + network, takes screenshots.
• It reports structured findings back: pass/fail, what broke, evidence.
• The smart agent fixes the code and asks again. Loop until the UI works.

Unlike playwright-mcp — where the smart model issues every single click itself — here the smart model stays high-level and delegates the whole clicking loop to the small agent.

How it's different from playwright-mcp

Architecture — the three actors

Picture a boss, a fast blind driver, and a describer with eyes:

   ┌─────────────┐   MCP conversation    ┌──────────────────────────────────────┐

│ smart agent │ start_debug ───────▶ │ UI Debugger MCP server │ │ (Claude) │ send_message (live) │ │ │ │ ◀─────── get_findings │ ┌────────────┐ ┌────────────┐ │ │ sets goals │ │ │ fast guy │ look│ vision guy │ │ │ fixes code │ │ │ (driver) │────▶│ (eyes) │ │ │ loops │ │ │ deepseek │◀────│ qwen3-vl │ │ └─────────────┘ │ │ text·blind │ desc│ image │ │ ▲ │ └─────┬──────┘ └────────────┘ │ │ "works + looks nice" │ observe / act (SQL-like) │ │ findings + screenshots │ │ shared adapter contract │ └──────────────────────────────│─────────┼─────────────────────────────│ └─────────┼─────────────────────────────┘ ▼ ┌──────────────┬──────────────┬──────────────┐ │ web (CDP) │ desktop │ android │ │ browser │ X11/Wayland │ ADB │ └──────────────┴──────────────┴──────────────┘

• smart agent — the boss (Claude/caller). Sends a goal, reads findings, fixes the code, loops. Stays high-level — never clicks.
• fast guy — the driver. Fast, cheap, text-only and blind. Runs the click loop on structure (DOM / a11y tree / view hierarchy). Default: deepseek.
• vision guy — the eyes. Multimodal. The driver calls look to ask *"does this look right? is the button centred?"* and gets a description back. Default: qwen. Spent only when visual judgment is needed.

One goal: **the UI works *and* looks nice.** Full design in docs/idea/.

Every run keeps its screenshots and stitches them into a short captioned replay video — Claude attaches it to the PR so a reviewer sees the flow working in ~10 seconds (docs/idea/workspace.md).

Targets

One project can expose several debug targets. A large app can have all three:

The desktop and android adapters read structure only — there is no automatic vision/OCR fallback: the adapter never switches to pixels. A custom-drawn surface (canvas, game, custom widget) exposes no nodes, so it is unreachable there; Chromium/Electron need --force-renderer-accessibility. The only vision is the agent-level look the driver calls when it needs a visual judgment.

Three adapters, one shared contract. Each runs managed (server launches the target) or attach (connect to a running one via cdpUrl / adbSerial). Linux first. iOS is out of scope on Linux (macOS-only tooling).

A managed android run boots the emulator windowless — -no-window with -gpu swiftshader_indirect -no-audio, because -no-window alone still opens a host GL context and dies on a box with no display. Set "headless": false on the target to watch it in a window; append anything else with emulatorArgs.

Setup

Install like any local MCP server — one entry in your .mcp.json:

{

"mcpServers": { "ui-debugger": { "command": "npx", "args": ["-y", "@developerz.ai/ui-debugger-mcp@latest"], "env": { "OPENAI_API_KEY": "sk-...", "OPENAI_BASE_URL": "https://openrouter.ai/api/v1" } } } }

Wiring more than one repo? .mcp.json is usually committed, so don't paste the key into each one — export it once in your shell and reference it, which keeps every repo's entry identical and secret-free:

"env": {

"OPENAI_API_KEY": "${UI_DEBUGGER_API_KEY}", "OPENAI_BASE_URL": "${UI_DEBUGGER_BASE_URL:-https://openrouter.ai/api/v1}" }

It's also published in the official MCP Registry as io.github.developerz-ai/ui-debugger-mcp — any client that browses the registry (instead of a hand-written .mcp.json entry) can find and install it by that name.

Then add a per-project config describing the app to debug (models, targets, urls) — .dz/ui-debugger/ui-debugger-mcp.json if your repo keeps agent config under .dz/ (checked first), else the root .ui-debugger-mcp.json (legacy). The fastest way is the init command:

npx @developerz.ai/ui-debugger-mcp@latest init   # in your project root

`ui-debugger-mcp init` scaffolds a project for debugging (described in docs/idea/config.md):

• creates the workspace dir ./tmp/ui-debugger-mcp/
• writes a starter config (default deepseek/qwen models, a web target stub) if one doesn't already exist — at .dz/ui-debugger/ui-debugger-mcp.json when the repo already has a .dz/ dir, else at the root .ui-debugger-mcp.json
• adds tmp/ to .gitignore
• prints the .mcp.json snippet to paste (it never writes your API key)

Config files:

• .mcp.json → how to launch the server (command + secret key). Gitignored.
• .dz/ui-debugger/ui-debugger-mcp.json (or root .ui-debugger-mcp.json) → how to debug this app (models, targets). Committed. When both exist the .dz/ copy wins and the tool prints a notice naming the ignored root file.

Every key of both files is documented in docs/idea/config.md; every tool's exact input/output shape is in docs/idea/mcp-tools.md.

The server reads the current directory to pick the project session — open it in your repo and it debugs that repo.

Quickstart

# 1. Scaffold the project (run once in your app's root)

npx @developerz.ai/ui-debugger-mcp@latest init

This creates ./tmp/ui-debugger-mcp/, writes a starter config (.dz/… or root), and prints the .mcp.json snippet to paste.

// 2. Paste into your project's .mcp.json (add your API key)

{ "mcpServers": { "ui-debugger": { "command": "npx", "args": ["-y", "@developerz.ai/ui-debugger-mcp@latest"], "env": { "OPENAI_API_KEY": "sk-...", "OPENAI_BASE_URL": "https://openrouter.ai/api/v1" } } } }

// 3. Edit the project config (.dz/ui-debugger/ui-debugger-mcp.json or root

// .ui-debugger-mcp.json) — set your app's URL { "targets": { "web": { "adapter": "browser", "url": "http://localhost:3000" } } }

// 4. In Claude Code (or any MCP client):

start_debug { target: "web", goal: "log in and add item 3 to the cart", url: "http://localhost:3000" }

// 5. Poll until done: get_findings { session_id: "...", wait: 30000 }

// 6. Read bugs[] + visual[] + summary. Fix code, repeat.

Using it

It's a conversation, not a remote control — five fat tools, not one-per-click:

A run is always time-capped: start_debug's timeout (seconds) overrides the default 300s, so a session can never hang forever — it auto-ends and frees the profile lock when the cap fires.

Logging in — named personas

Almost every interesting screen is behind a login, and re-explaining the login flow in every goal string costs driver steps before the real goal starts. Say it once, in the target:

"targets": {

"dashboard": { "adapter": "browser", "url": "http://localhost:5173", "auth": { "admin": { "path": "/login", "fields": { "email": "admin@dev.local", "password": "admin" }, "submit": "Sign in" }, "user": { "path": "/login", "fields": { "email": "user@dev.local", "password": "user" }, "submit": "Sign in" } } } }

start_debug { target: "dashboard", as: "admin", goal: "open Audit and check the table renders" }

The run signs in out-of-band, before the driver's first step: it costs zero driver steps, and the credentials never enter the model's context — the prompt only learns *"you are already signed in as admin"*. Values are redacted out of every log the run writes; describe reports persona names only. A typo'd as fails loud listing the valid names, and a login that does not take fails the run rather than handing back a session that will report every page behind it as empty. Use dev/seed credentials — this file is committed. Details in docs/idea/config.md.

Telling it what's expected — notes

A freshly-migrated app with no seed data renders empty states everywhere, and the driver reports *"the table is empty"* as the run's headline bug. It is right, and it is useless. Say what's expected once, in the target:

"targets": {

"dashboard": { "adapter": "browser", "url": "http