Claude KVM

Claude KVM

Remote Access, Artificial Intelligence

claude-kvm.ai

Claude KVM is an MCP tool that controls remote desktop environments over VNC. It consists of a thin JS proxy layer (MCP server) and a platform-native Swift VNC daemon running on your macOS system.

[!TIP]

[Phantom-WG](https://www.phantom.tc) could be a great alternative for you. Isolate your VNC server within your own network while enjoying self-hosted VPN performance with the extra privacy features you gain along the way.

Live Test Runs

• Integration Test
• Mac Integration Test
• Mac Calculator Test
• Mac Scientific Calculator Test
• Mac Safari Browsing Test
• Mac Drag & Drop Test
• Mac Chess Test
• Mac Chess Direct Test
• Mac Phantom-WG Install Test

[!NOTE]

Tests are conducted transparently on GitHub Actions — each step is visible in the CI environment. At the end of every test, whether the integration passes or fails, you'll find screenshots of each step the agent took during the session, along with an .mp4 video recording that captures the entire session. By reviewing these recordings and screenshots, you can observe how the agent progressed through each stage, how long the task took, and what decisions were made based on the system prompt. You can use these examples as a reference when crafting your own system prompts or instructions for the MCP server in your own environment.

[!WARNING]

Artifacts attached to these runs may have expired due to GitHub's artifact retention policy. Persistent copies are prepared via the Persist Artifacts workflow and can always be accessed by run ID from the artifacts/ directory on the press-kit branch.

Architecture

graph TB

subgraph MCP["MCP Client (Claude)"] AI["Claude"] end

subgraph Proxy["claude-kvm · MCP Proxy (stdio)"] direction TB Server["MCP Server index.js "] Tools["Tool Definitions tools/index.js "] Server --> Tools end

subgraph Daemon["claude-kvm-daemon · Native VNC Client (stdin/stdout)"] direction TB CMD["Command Handler PC Dispatch "] Scale["Display Scaling Scaled ↔ Native "]

subgraph Screen["Screen"] Capture["Frame Capture PNG · Crop · Diff "] OCR["OCR Detection Apple Vision "] end

subgraph InputGroup["Input"] Mouse["Mouse Click · Drag · Move · Scroll "] KB["Keyboard Tap · Combo · Type · Paste "] end

VNC["VNC Bridge LibVNCClient 0.9.15 "]

CMD --> Scale Scale --> Capture Scale --> Mouse Scale --> KB Capture -.->|"framebuffer"| VNC Mouse -->|"pointer events"| VNC KB -->|"key events"| VNC end

subgraph Target["Target Machine"] VNC_Server["VNC Server :5900 "] Desktop["Desktop Environment"] VNC_Server --> Desktop end

AI <-->|"stdio JSON-RPC"| Server Server <-->|"stdin/stdout PC (NDJSON)"| CMD VNC <-->|"RFB Protocol TCP :5900"| VNC_Server

classDef proxy fill:#1a1a2e,stroke:#16213e,color:#e5e5e5 classDef daemon fill:#0f3460,stroke:#533483,color:#e5e5e5 classDef target fill:#1a1a2e,stroke:#e94560,color:#e5e5e5

class Server,Tools proxy class CMD,Scale,VNC,Capture,Mouse,KB daemon class VNC_Server,Desktop target

Layers

PC (Procedure Call) Protocol

Communication between the proxy and daemon uses the PC protocol over NDJSON:

Request:      {"method":"<name>","params":{...},"id":<int|string>}

Response: {"result":{...},"id": } Error: {"error":{"code": ,"message":"..."},"id": } Notification: {"method":" ","params":{...}}

Coordinate Scaling

The VNC server's native resolution is scaled down to fit within --max-dimension (default: 1280px). Claude works more consistently with scaled coordinates — the daemon handles the conversion in the background:

Native:  4220 x 2568  (VNC server framebuffer)

Scaled: 1280 x 779 (what Claude sees and targets)

mouse_click(640, 400) → VNC receives (2110, 1284)

Screen Strategy

Claude minimizes token cost with a progressive verification approach:

diff_check       →  changeDetected: true/false     ~5ms    (text only, no image)

detect_elements → OCR text + bounding boxes ~50ms (text only, no image) cursor_crop → crop around cursor ~50ms (small image) screenshot → full screen capture ~200ms (full image)

detect_elements uses Apple Vision framework for on-device OCR. Returns text content with bounding box coordinates in scaled space — enables precise click targeting without consuming vision tokens.

---

Installation

Requirements

• macOS (Apple Silicon / aarch64)
• Node.js (LTS)

Daemon

brew tap ARAS-Workspace/tap

brew install claude-kvm-daemon

[!NOTE]

claude-kvm-daemon is compiled and code-signed via CI (GitHub Actions). The build output is packaged in two formats: a .tar.gz archive for Homebrew distribution and a .dmg disk image for notarization. The DMG is submitted to Apple servers for notarization within the same workflow — the process can be tracked from CI logs. The notarized DMG is available as a CI Artifact; the archived .tar.gz is also published as a release on the repository. Homebrew installation tracks this release.

- Release · Source Code

- LibVNC Build · LibVNC Branch

- Homebrew Tap

MCP Configuration

Create a .mcp.json file in your project directory:

{

"mcpServers": { "claude-kvm": { "command": "npx", "args": ["-y", "claude-kvm"], "env": { "VNC_HOST": "192.168.1.100", "VNC_PORT": "5900", "VNC_USERNAME": "user", "VNC_PASSWORD": "pass", "CLAUDE_KVM_DAEMON_PATH": "/opt/homebrew/bin/claude-kvm-daemon", "CLAUDE_KVM_DAEMON_PARAMETERS": "-v" } } } }

[!NOTE]

The tool is end-to-end tested via CI — Claude executes tasks over VNC while an independent vision model observes and verifies the results. See the Integration Test for live workflow runs, system prompts, and demo recordings.

Configuration

MCP Proxy (ENV)

Daemon Parameters (CLI)

Additional arguments passed to the daemon via CLAUDE_KVM_DAEMON_PARAMETERS:

"CLAUDE_KVM_DAEMON_PARAMETERS": "--max-dimension 800 -v"

Runtime Configuration (PC)

All timing and display parameters are configurable at runtime via the configure method. Use get_timing to inspect current values.

Set timing:

{"method":"configure","params":{"click_hold_ms":80,"key_hold_ms":50}}

{"result":{"detail":"OK — changed: click_hold_ms, key_hold_ms"}}

Change display scaling:

{"method":"configure","params":{"max_dimension":960}}

{"result":{"detail":"OK — changed: max_dimension","scaledWidth":960,"scaledHeight":584}}

Reset to defaults:

{"method":"configure","params":{"reset":true}}

{"result":{"detail":"OK — reset to defaults","timing":{"click_hold_ms":50,"combo_mod_ms":10,"cursor_crop_radius":150,"double_click_gap_ms":50,"drag_min_steps":10,"drag_pixels_per_step":20,"drag_position_ms":30,"drag_press_ms":50,"drag_settle_ms":30,"drag_step_ms":5,"hover_settle_ms":400,"key_hold_ms":30,"max_dimension":1280,"paste_settle_ms":30,"scroll_press_ms":10,"scroll_tick_ms":20,"type_inter_key_ms":20,"type_key_ms":20,"type_shift_ms":10},"scaledWidth":1280,"scaledHeight":779}}

Get current values:

{"method":"get_timing"}

{"result":{"timing":{"click_hold_ms":80,"combo_mod_ms":10,"cursor_crop_radius":150,"double_click_gap_ms":50,"drag_min_steps":10,"drag_pixels_per_step":20,"drag_position_ms":30,"drag_press_ms":50,"drag_settle_ms":30,"drag_step_ms":5,"hover_settle_ms":400,"key_hold_ms":50,"max_dimension":1280,"paste_settle_ms":30,"scroll_press_ms":10,"scroll_tick_ms":20,"type_inter_key_ms":20,"type_key_ms":20,"type_shift_ms":10},"scaledWidth":1280,"scaledHeight":779}}