AInotate

Annotated screenshots for AI agents: the agent shows you the button instead of describing where it is.

Website · Get started · Privacy

Ask an agent how to post a link on Hacker News and you usually get directions: the orange bar at the top, the last link after *jobs*. With AInotate it sends the picture above instead and says "click *submit*, where the blue arrow points".

It captures a web page, a window or a screenshot you already have, finds the elements you mean, draws numbered steps, arrows, boxes and labels where they hide the least, blacks out emails and tokens, and looks at the result before sending it. It works in Claude Code, Cowork, Claude Desktop, Codex, Gemini CLI, Cursor and any other MCP client, and from the command line or Python.

We built it at DontPayFull because our own agents kept answering "where is it?" with a paragraph. A picture with a number on the right button settles the question in a second, in a chat, a ticket or a guide.

What it is for

• "Where is it?" Settings, menus, buttons that hide behind icons. The agent answers with the screen itself, the control boxed and numbered.
• "Walk me through it." One image per step, in the order to follow; ainotate guide turns them into a Markdown, HTML or PDF guide.
• "Your turn." A permission switch, a 2FA code, a payment or consent screen: the agent does not click for you. It shows exactly what to press and says what the click does.
• Bug reports and tickets. What is wrong, outlined in red, what is right in green, emails and tokens blacked out before anyone sees them.
• Docs and changelogs. Screenshots that point at the thing the text talks about, re-shot from the same spec when the UI changes; before and after plates for a change.

Set it up in your agent

1. Install, one of:

Then ainotate doctor checks the machine and prints the exact fix for anything missing, including the one command that downloads Chromium for web capture. To update later: brew upgrade ainotate, pipx upgrade ainotate or uv tool upgrade ainotate; the plugin with /plugin marketplace update ainotate; the Desktop extension by opening the newer .mcpb from Releases.

2. Connect it to the agent you use:

{"mcpServers": {"ainotate": {"command": "ainotate", "args": ["mcp"]}}}

AInotate runs on your own computer: Cowork reaches it through the Claude desktop app, so keep the app open while a task uses it. The plugin runs AInotate with uvx, pinned to its release, so it needs uv.

The skill teaches an agent when and how to annotate: pick a source, find exact positions, annotate, read the image to verify, deliver. The plugin brings it along; without the plugin it ships with the package:

ainotate install-skill             # Claude Code and Codex (~/.claude/skills, ~/.agents/skills)

ainotate install-skill --zip ~/Desktop # a ZIP for Claude Desktop, Cowork, claude.ai:

Customize > Skills > Upload a skill

npx skills add dontpayfull/AInotate # any agent the skills CLI knows (Cursor, Windsurf, ...)

Config paths for every OS, permissions and the tool list: skills/ainotate/references/mcp.md.

3. Ask. "Show me where to switch Wikipedia to dark mode." The agent captures the page, annotates it, checks it and sends the image. With the skill it also does this on its own whenever an answer depends on where something is on the screen: a "how do I" question, a visual bug, a before/after of a change.

Gallery

Every image below was made in one shoot call against a public site: open the page, find the elements, place the labels, draw, frame. The specs are in docs/images/specs.

Four arrow styles : skitch (default: straight when the way is clear, a gentle bend when it is long and diagonal, a curve around text), curved , straight , line

Phone preset : mobile viewport, touch and user agent

Labels that stay off the content

A label dropped at a fixed spot next to its target usually lands on the text the reader needs. AInotate places every label itself:

• It maps the page content first (text, icons, lines, images) and tries hundreds of spots per label. Empty space near the target wins; a spot over text is used only when the frame has no free room.
• Arrows are priced too: a long arrow, or one that crosses text, another label or another mark, loses to a shorter, cleaner one. When the straight path crosses text, the arrow curves around it.
• All labels are planned together, so the first one cannot take the only clean spot a later one needs.
• Step badges sit on the corner of their box that covers the least.

The image is never enlarged to make room. If a label still has to cover something, AInotate says so in a warning, and label_at pins a label exactly where you want it.

From the command line

Save this as hn.json:

{"url": "https://news.ycombinator.com",

"frame": {"chrome": "browser"}, "marks": [ {"type": "step", "n": 1, "target": {"text": "new", "exact": true}, "label": "Newest"}, {"type": "step", "n": 2, "target": {"text": "login"}, "label": "Sign in"}]}

ainotate shoot hn.json --draft    # temp file; drop --draft to save

The output path is printed on stdout. Images are saved to ~/Pictures/AInotate unless you configure another folder. ainotate --help lists every command (capture, annotate, locate by OCR, grid, zoom, window capture with UI element targets, elements, guide, compare, animate, copy, install-skill); the full spec is in skills/ainotate/references/spec.md. Exit codes: 0 ok, 1 cannot save, 2 invalid spec, 3 cannot draw, 4 capture failed or an optional dependency is missing, 5 ambiguous text target, 6 target not found.

Python. The same pipeline is importable:

from ainotate.shoot import shoot

res = shoot({"url": "https://en.wikipedia.org/wiki/Screenshot", "marks": [{"type": "box", "target": {"text": "View history"}, "label": "Past edits"}]}, draft=True) print(res.paths[0], res.redactions)

Features

• Marks: numbered step, box, arrow, click ripple, keys (keycaps), magnify (loupe), highlight, spotlight, text, solid redact, and decorative blur / pixelate.
• Placement that reads well: