Clickcast

🎞️ clickcast — Playwright browser automation, screenshots, and GIF/MP4/WebP recording for AI agents

Give AI agents visual + structured feedback about live web UIs — and give humans deterministic demo reels while you're at it.

What's new (v0.4.4) — npx-installable npm wrappers (clickcast + clickcast-mcp) on top of the v0.3.0 highlights: live agent control via an MCP server, pixel-level clickcast diff, accessibility semantics fused with the pixel-grid overlay, an official CI GitHub Action, Homebrew/apt packaging, and a self-healing first run. See CHANGELOG.md for the full notes.

See [`docs/ONE_PAGE_NAVIGATION_ORDER_TIPS.md`](docs/ONE_PAGE_NAVIGATION_ORDER_TIPS.md) for the nine principles behind why this reel reads as legibly as it does — and the scenario template you can copy for your own reels.

clickcast drives a real browser through a website and hands back two things:

1. A watchable reel — GIF / MP4 / WebP / raw frames.
2. A machine-readable JSON sidecar — every step's selector, timings, per-step frame paths, discovered elements, and post-action page state (title, URL, console errors, failed requests). Versioned. See docs/feedback-schema.md.

Point it at a URL and it will *auto-discover* the interactive elements and build a tour for you, or hand it a small YAML scenario for a scripted, repeatable walkthrough.

---

Install

One line, ready to run immediately after:

pip install clickcast && clickcast install --with-deps chromium && clickcast doctor

Broken down — pip install clickcast (requires Python ≥ 3.10) gets you the CLI; clickcast install downloads Chromium (~one-time, ~180MB — kept out of the pip package itself since it's versioned independently and every project doesn't need every engine); --with-deps also pulls the system libraries Chromium needs (Linux only, may prompt for sudo); clickcast doctor confirms everything above actually worked.

Forgot the second step? Any command that needs a browser (auto, run, shot, elements, mcp) detects a missing engine itself and offers to install it right then instead of failing — say yes once and it retries automatically. That prompt only fires in an interactive terminal; CI/scripted runs fail fast with the exact fix command instead of hanging on stdin.

Homebrew (macOS/Linux)

brew install --build-from-source ./Formula/clickcast.rb   # works today, clone this repo first

brew install AlexKay28/clickcast/clickcast # tap not bootstrapped yet -- see docs/packaging/homebrew.md

apt (Debian/Ubuntu, .deb)

bash scripts/build_deb.sh 0.2.9 && sudo dpkg -i dist-deb/clickcast_0.2.9_amd64.deb   # works today, clone this repo first

sudo apt install clickcast # hosted repo not bootstrapped yet -- see docs/packaging/apt.md

Both native packages skip the Chromium download (~180MB, versioned independently of clickcast) and never install a second ffmpeg (clickcast already bundles one via imageio[ffmpeg]) -- run clickcast install --with-deps chromium once after either install path. Full design rationale, what's live today vs. what needs one-time bootstrapping, and the exact bootstrap steps: docs/packaging/homebrew.md, docs/packaging/apt.md.

npm (clickcast + clickcast-mcp)

Two npm packages -- thin Node wrappers whose postinstall provisions an isolated Python venv and pip-installs the pinned PyPI clickcast, since there's no way to ship clickcast's Playwright/Pillow/ffmpeg runtime as pure JS. Built specifically because the MCP ecosystem's install pattern is npx , not pip install:

npx -y clickcast-mcp                 # MCP entry point -- execs `clickcast mcp`

npx clickcast --version # general CLI wrapper

Both are published on the npm registry. To run against your working copy instead of the published version:

cd npm/clickcast-mcp && npm pack && npm install -g ./clickcast-mcp-*.tgz && clickcast-mcp --help

cd npm/clickcast && npm pack && npm install -g ./clickcast-*.tgz && clickcast --version

Full design rationale (including why the shared provisioning code is a vendored copy rather than a file: dependency), what's live today vs. what needs bootstrapping, and the exact bootstrap steps: docs/packaging/npm.md.

---

For AI agents — copy this prompt into your chat

Paste the block below into your coding agent (Claude Code, Cursor, Copilot Chat, Codex, etc.) to teach it clickcast in one message. The agent will install the tool, verify the environment, generate a visual + machine-readable report for your project, and know how to gate CI on the results.

You are going to use `clickcast` (https://pypi.org/project/clickcast/) to visually verify

this project's frontend. It produces a watchable GIF reel + a JSON sidecar with per-step selectors, timings, page state, error codes, and a graph of the app's pages. Follow this exact sequence:

1. INSTALL pip install clickcast clickcast install --with-deps chromium # first time only, ~180 MB clickcast doctor # bail out if this fails

1. LOAD FULL CAPABILITIES Run clickcast skill --json and read the output. It lists every subcommand, flag, docs URL, and sidecar schema pointer. Prefer this over guessing.

1. RUN A QUICK TOUR OF THE APP If we have a running URL (dev server / staging / production): clickcast auto --for-humans --emit-events --out tour.gif If we have a specific flow to verify, write a YAML scenario and run: clickcast run tour.yml --emit-events --out tour.gif --for-humans gives a legible reel for the user to watch; --emit-events prints a machine-readable JSONL line you can parse.

1. READ THE SIDECAR (JSON at .json, schema v4) For each step, gate on the structured fields — NOT regex over prose:

• status : "ok" | "failed" | "skipped"
• error_code : "timeout" | "locator_missing" | "cross_origin" | "navigation_error" | "selector_ambiguous" | "other"
• skip_reason : "optional_no_reaction" | "pre_action_failed" | "element_vanished" | "cross_origin_bounce"
• page_state : title, url_after, console_errors, page_errors, network_failed The top-level graph block gives page nodes + navigation edges you can use to reason about the app's shape, not just the sequence you ran.

1. WATCH STDERR FOR ADVISORIES clickcast prints ⚠ — see lines for known anti- patterns (nav-heavy tour, click without DOM reaction, very short reel, cross-origin bounce, incoherent cursor styling). Each has a stable kebab-case id you can dedupe or gate on.

1. FOR CI REGRESSION GATES clickcast assertions .json --baseline golden.json diffs the run against a committed baseline; nonzero exit on drift. Byte-identical across runs (timestamps, frame paths, and URL query strings excluded).

Reference docs (all in-repo, load lazily as needed):

• Sidecar shape: https://github.com/AlexKay28/clickcast/blob/main/docs/feedback-schema.md
• Human-legible reel authoring: https://github.com/AlexKay28/clickcast/blob/main/docs/ONE_PAGE_NAVIGATION_ORDER_TIPS.md
• Agent integration: https://github.com/AlexKay28/clickcast/blob/main/docs/ai-integration.md

If something is unclear, run clickcast --help before asking me.

Once the agent has this, ask it something concrete like *"run clickcast auto against http://localhost:3000 and tell me which clicks had DOM reactions"* — it now has everything it needs.

---

Live agent control (MCP)

Everything above is batch mode: record a whole tour, then read back a GIF + sidecar. clickcast mcp is the live counterpart — an MCP server that drives one action at a time (goto/click/type/scroll/...) and hands back clickcast's richer per-call payload (annotated frame, page_state, grid coordinates, an enumerated error_code) instead of a bare screenshot, so an agent can react before deciding the next step.

pip install 'clickcast[mcp]'

{

"mcpServers": { "clickcast": { "command": "clickcast", "args": ["mcp"] } } }

Reach for mcp when an agent needs to explore and react live; reach for auto/run for a repeatable, one-shot artifact (CI, docs, release notes). Full tool reference, client config for Claude Code / Claude Desktop, and the schema doc: docs/mcp.md · docs/mcp-tool-schema.md.

---

First run — 30 seconds

clickcast auto https://example.com --out tour.gif

Produces two files:

• tour.gif — the reel
• tour.gif.json — the AI-consumable sidecar (schema_version: 4, spec at docs/feedback-schema.md)

For a walkthrough of how an LLM agent consumes both, see docs/ai-integration.md.

---

Three modes

All three are deterministic, headless-by-default, and CI-friendly.

---

Commands

auto

Discover interactive elements and record a click-tour.