Kvm Pilot

kvm-pilot

Smart hands for your AI agents. A write-capable, multi-plane (KVM + BMC + SSH) MCP server for controlling physical machines — gated, verified, audited.

kvm-pilot lets an agent drive a headless box through POST, firmware, the bootloader, and an OS install with no agent on the target: it works at the pixel level through an IP-KVM (PiKVM, the GL.iNet GLKVM fork GL-RM1 / GL-RM1PE, BliKVM), at the structured-state level through a BMC (Redfish on iDRAC/iLO/OpenBMC, IPMI on BMCs that predate Redfish), at the firmware level through Intel AMT/vPro (a BIOS/POST/GRUB screenshot + power + SOL on a business laptop an HDMI-capture KVM can't see boot on), and over SSH once an OS is up. A pluggable vision subsystem reads a KVM screenshot and tells you what boot phase the machine is in — bios_menu, grub_menu, installer_progress, login_prompt, crash_screen, and so on — and a safety layer gates every destructive operation behind operator opt-ins and per-call approvals.

Vision runs on Claude or any local OpenAI-compatible VLM (LM Studio, Ollama, vLLM, llama.cpp). Point it at a model on your own GPU and the screenshots never leave your network and cost nothing per frame.

How it works

kvm-pilot runs a see → decide → act loop, and the screen is its only sensor: it pulls a screenshot from the KVM, a vision model classifies the boot phase, and kvm-pilot acts back through the KVM's keyboard and power. Because it works at the pixel level, there is no agent on the target — the same loop drives POST, firmware, the bootloader, and an OS install.

A real, unedited run against a GLKVM on the home fleet — an honest preflight, a headless snapshot (watch it wake the on-demand encoder), a gated dry-run power-cycle, and the boot console the agent actually saw:

Quickstart

One install gives you the whole product — the `kvm-pilot` CLI, the `kvm-pilot-mcp` MCP server, and the bundled Claude skill — nothing to clone.

pip install kvm-pilot                          # CLI + skill + MCP server + WebSocket events

pip install "kvm-pilot[totp]" # + 2FA / TOTP support (pyotp) kvm-pilot install-skill # optional: put the bundled skill where Claude Code loads it

Driving a KVM from an AI agent (MCP)

claude mcp add kvm-pilot -s user \

-e KVM_PILOT_PROFILE=mykvm -e KVM_PILOT_MCP_READ_ONLY=1 -- \ kvm-pilot-mcp

KVM_PILOT_MCP_READ_ONLY=1 is the recommended first rung of the trust ladder — the agent can see everything and touch nothing until your hardware is verified. The Getting started guide covers credentials, Claude Desktop JSON config, sample prompts, and climbing the ladder. The server is published to the official MCP registry as `io.github.DustinTrap/kvm-pilot`, so registry-aware hosts can discover and install it by name. Agents: the repo root carries an llms.txt doc map.

Scripting from Python

from kvm_pilot import KVMClient

from kvm_pilot.vision import ScreenAnalyzer, make_backend

kvm = KVMClient("192.168.8.1", "admin", "secret")

Classify the current screen with Claude (model auto-resolved at runtime)

analyzer = ScreenAnalyzer(kvm, make_backend("anthropic")) print(analyzer.classify().phase)

Or run entirely on a local VLM — nothing leaves your network

local = make_backend("local", base_url="http://127.0.0.1:1234/v1", model="qwen2.5-vl-7b") analyzer = ScreenAnalyzer(kvm, local)

Block until the box reaches the GRUB menu, then pick the first entry

analyzer.wait_for_state("grub_menu", timeout=120) kvm.press_key("Enter")

For the latest unreleased tree:

pip install "kvm-pilot[totp] @ git+https://github.com/DustinTrap/kvm-pilot"

CLI

kvm-pilot info     --host 192.168.8.1 --user admin --ask-passwd   # prompt (no echo)

kvm-pilot capabilities --profile homelab # what this driver supports kvm-pilot snapshot screen.jpg --profile homelab kvm-pilot --timeout 60 power-cycle --profile homelab --dry-run # log, don't send kvm-pilot eject --profile homelab # detach virtual media kvm-pilot events --profile homelab --count 5 # stream device events kvm-pilot watch grub_menu --profile homelab \ --backend local --vision-url http://127.0.0.1:1234/v1 --vision-model qwen2.5-vl-7b

The CLI prompts for confirmation before any destructive action (power, virtual media — including uploads — keyboard/mouse injection, GPIO). Use --yes to skip prompts in automation, or --dry-run to log intended actions without sending them — dry-run short-circuits *before* the prompt, so it never blocks waiting for input. --timeout (HTTP per-request timeout) is a global flag and goes *before* the subcommand; watch keeps its own --timeout for the vision wait deadline.

Profiles like homelab live in ~/.config/kvm-pilot/config.toml. See docs/cli.md for the full command table (every subcommand, the capability it needs, and its gating), and docs/configuration.md for the config-file format, every KVM_PILOT_* environment variable, and the precedence between flags, env, and profiles.

GLKVM setup note: on GL.iNet firmware the PiKVM REST API is **disabled by

default** and a firmware upgrade can re-disable it — enable it first, per

Every /api/* call returns 404.

The tool surface, by plane

The same capability protocols span three actuation planes, so one agent workflow can mix pixels, structured BMC state, and shell access — with every destructive effect gated per class:

A machine with no KVM and no BMC at all is itself a first-class target: driver = "ssh" plus `ssh_host` (the machine's own address, #248) stands kvm-pilot on the recovery ladder's in-band rung — every device capability honestly absent, and the healthcheck saying plainly that there is no out-of-band recovery path there.

The canonical per-tool reference — annotations, effect gates, approval lifecycle — is the MCP server README; the CLI covers the full surface in docs/cli.md.

Status & maturity

Status: 1.0, general availability — on the strength of the run ledger,

not beyond it. Three device types have run live (GL-RM1PE, Dell R710 over

iDRAC6/IPMI, Dell Latitude 5411 over Intel AMT); three of the seven device

drivers (redfish, pikvm, blikvm) have only ever run against emulators.

See the

Hardware-Compatibility list

for what has actually been exercised — and please add to it. (The exact version lives in the

CHANGELOG.)

The core paths have graduated

from mocked-only to live-verified: a fleet of GL-RM1PE units has exercised

snapshot/healthcheck/logs/power_state/virtual_media/info across

two firmware lines — on V1.9.1 the run ledger that ships in the wheel puts

info/logs/healthcheck/power_state at ga, snapshot at rc, and

virtual_media at beta — derived from real runs, never hand-edited — and a Dell iDRAC6 has exercised the IPMI driver live end-to-end

(power, boot-device, sensors, event log, SOL serial console). The paths that

can hurt are hardened: transports never re-fire a destructive request, MCP

approvals are signed single-use receipts with an audit trail, and every

destructive effect — power, HID, media, boot-config, appliance, SSH,

external writes — has its own operator opt-in gate. Recent betas added

remote boot-device control (Redfish, IPMI, and in-band efibootmgr),

Wake-on-LAN, an IPMI driver for BMCs that predate Redfish, a serial (SOL)

console, mouse auto-calibration, and headless native-resolution GLKVM

snapshots; kvm-pilot test-report turns contributing evidence into one

command, and the firmware registry feeds itself (firmware-check auto-files

registry updates).

Now we need your hardware. PiKVM, BliKVM, other GLKVM models, and

Redfish BMCs (iDRAC/iLO/OpenBMC) are the combos the matrix needs most —

success *or* failure, a

hardware report

takes two minutes and the hourly ingest does the rest. Anything the

Hardware-Compatibility list

doesn't show as exercised is still unverified: the 1.x line keeps the public

API stable, but a rating only moves when a live run moves it; note the remote

firmware-flash no-op on GL-RM1PE

([#94](https://github.com/DustinTrap/kvm-pilot/issue