AI Visibility Checker

aivis — can AI assistants actually read your store?

One command. No install, no dependencies, no account, nothing uploaded.

curl -sO https://raw.githubusercontent.com/krisdiallo/ecom-agent/main/aivis.py

python3 aivis.py yourstore.com

aivis 1.0.0 — can AI assistants read brooklinen.com?

1. robots.txt — are you blocking the crawlers that recommend you? [ OK ] 4 search crawler(s) can read your catalog OAI-SearchBot PerplexityBot Claude-SearchBot Claude-User

1. Product page — can a crawler read your facts? [FAIL] Structured data is injected by JavaScript — crawlers never see it [FAIL] Your contradicts your own og:title title: Classic Cotton Sheet Set | Brooklinen og:title: Super-Plush 4-Piece Bath Towel Set [WARN] Only 0 concrete measurement(s) in the readable text

Summary: 2 critical 2 to review 1 passed

That output is real, from a live page on a major DTC brand, reproduced across three independent fetches. Prefer a browser? Same checks, paste-based: [the web version](https://krisdiallo.github.io/ecom-agent/ai-visibility.html).

One page is an anecdote. Sample your catalogue to see whether a problem is systemic:

python3 aivis.py yourstore.com --pages 10

[FAIL] Structured data injected by JavaScript (7/10 pages)

[WARN] Fewer than 5 concrete measurements (9/10 pages) [ OK ] 1/10 pages clean on every check

---

What this is, in the words you'd search for it

This is a free GEO tool — generative engine optimization, also called LLM SEO or answer engine optimization: making sure an AI assistant can read your store well enough to cite or recommend it. It is the ecommerce-specific version of the broader AI search optimization question, focused on the two things that decide whether an answer engine can use your content at all:

1. Your robots.txt — are you disallowing the crawlers that feed answers (OAI-SearchBot, PerplexityBot, Claude-SearchBot)? Most "block AI" advice gets this wrong by conflating them with training crawlers.
2. Your product page's structured data and facts in raw HTML — what a crawler that does not run JavaScript actually receives, not what dev tools show you.

If you searched for an AI visibility checker or a robots.txt checker for an ecommerce store, this does both, with primary-source data on 165 AI crawlers rather than generic advice.

---

Or give it to your AI

An MCP server, so Claude Code, Claude Desktop, Cursor or any MCP client can run the check itself — "is my store readable by AI assistants?" answered in the conversation you're already in.

Nothing to download, no paths to edit — paste this into your MCP config:

{ "mcpServers": {

"aivis": { "command": "uvx", "args": ["--from", "git+https://github.com/krisdiallo/ecom-agent", "aivis-mcp"] } } }

Also listed in the [official MCP registry](https://registry.modelcontextprotocol.io) as io.github.krisdiallo/aivis, so any client that reads the registry can find it, and there is a one-file aivis_mcp.py in this repo if you would rather run it directly.

Stdio JSON-RPC, zero dependencies — the whole tool is standard library, which is why it can be fetched and run straight from git. It imports aivis.py rather than shelling out, so the CLI, web tool, study, GitHub Action and MCP server all share one verified implementation.

Or run it in CI

Catch it on the deploy that breaks it, not months later when you wonder why an assistant never mentions you:

- uses: krisdiallo/ecom-agent@v1

with: site: yourstore.com fail-on: critical # critical | warning | never

Writes a findings table to the job summary, exposes critical / warnings / passed / report as step outputs, and fails the build on real problems. This repo runs it against its own site on every push — a tool that tells you to check your AI visibility should be checking its own, in public.

--json works standalone too, if you'd rather wire it into something else:

python3 aivis.py yourstore.com --json --fail-on warning

The one thing most advice gets backwards

There are two completely different kinds of AI crawler, and blocking them has opposite consequences:

Blocking `GPTBot` does not remove you from ChatGPT's recommendations. OAI-SearchBot is the token that does. The "block AI scrapers" wave conflated these, and a lot of sites opted out of training thinking they were protecting something else.

Three more that circulating robots.txt snippets routinely get wrong:

• `Amazonbot` vs `Amzn-SearchBot`. Amazon splits the roles. Amazonbot "may be used to train Amazon AI models"; Amzn-SearchBot is what makes you "eligible to appear in search experiences such as Alexa" and "does not crawl content for generative AI model training." Most guides name only the first. Block the wrong one and you lose Alexa for nothing.
• `Applebot-Extended` does not crawl anything. It is purely an opt-out signal. Apple states pages that disallow it "can still be included in search results."
• `facebookexternalhit` is not an AI crawler. It renders your link previews. It gets swept into "block AI bots" lists, which then silently breaks how your links look when shared.

Blocking AI is two decisions, not one

ai-robots-txt/ai.robots.txt (~4k stars) is the ecosystem's default answer to "block AI bots". Its generated robots.txt disallows all 166 known AI user-agents. Audited with this tool:

$ python3 -c "import aivis,urllib.request as u; \

print(aivis.audit_robots(u.urlopen('https://raw.githubusercontent.com/\ ai-robots-txt/ai.robots.txt/main/robots.txt').read().decode()))"

visible_to_ai_search: False blocked_search : OAI-SearchBot, PerplexityBot, Claude-SearchBot, Claude-User, Amzn-SearchBot, Amzn-User, Applebot

That is deliberate on their part, and correct for their audience. Their FAQ states the position plainly: these crawlers are *"extractive, confer no benefit to the creators of data they're ingesting."* A publisher who does not want ChatGPT answering questions from their articles without a click wants exactly this file. We are not describing a bug, and an earlier version of this section wrongly implied one.

The point is that blocking AI is two decisions, not one, and a single list collapses them:

• *"Don't train on me."* Costs you nothing in recommendations.
• *"Don't answer questions using me."* Removes you from ChatGPT search, Perplexity, Claude, Alexa, Siri and Spotlight.

A publisher usually wants both. An ecommerce store usually wants the first and not the second — being absent from AI answers is lost demand, not protected work. If that is you, this generates the other file:

python3 aivis.py --training-optout >> robots.txt

Blocks all 8 training crawlers, leaves all 7 search crawlers allowed. Verified in CI by auditing our own generated output.

Use it as a library

If you're building a GEO tool, a crawler, or a shopping agent, you probably want the primitive rather than the report:

from aivis import classify_crawler, audit_robots

classify_crawler("GPTBot")["blocking_effect"] # 'opts_out_of_training_only' classify_crawler("OAI-SearchBot")["blocking_effect"] # 'removes_from_ai_answers' classify_crawler("SomeRandomBot") # None — it won't guess

r = audit_robots(open("robots.txt").read()) r["visible_to_ai_search"] # False if any AI *search* crawler is blocked r["blocked_search"] # the ones that actually cost you answers r["blocked_training"] # blocking these costs nothing; listed separately on purpose

audit_robots applies real group precedence — a crawler obeys its own group and ignores User-agent: * when it has one — and classifies paths so a stock Shopify file, with its ~45 default Disallow rules, doesn't read as broken.

Every classification is sourced to the vendor's own documentation (crawlers.json carries the quotes and dates), and research/test_api.py is a contract test in CI so the values can't drift. Zero dependencies: pip install git+https://github.com/krisdiallo/ecom-agent.

agent-commerce.json — which stores an AI agent can actually buy from

[`agent-commerce.json`](agent-commerce.json) — 70 storefronts probed, 49 expose a live endpoint an agent can call to search a catalogue, build a cart and complete a checkout, bypassing HTML entirely.

The result worth the file: all 49 exposed an identical 13-tool surface on one UCP version. Zero variation across 49 independent brands.

That means agent-commerce capability is currently a property of the platform, not a merchant choice. A store has it because its platform switched it on. Two consequences:

• a merchant cannot meaningfully "optimise" this axis today beyond choosing a platform — which is the opposite of how this is usually sold
• an agent developer can assume a uniform tool surface across these stores rather than negotiating capabilities per merchant

# every store in the sample an agent can transact with

curl -s https://raw.githubusercontent.com/krisdiallo/ecom-agent/main/agent-commerce.json \ | python3 -c "import json,sys;[print(h['host']) for h in json.load(sys.stdin)['hosts'] if h['agent_commerce']]"

A live endpoint does not mean any agent can buy. Probing further: tools/list answers anonymously, but every tool *call* is refused without a published UCP agent profile URI (invalid_profile_url), verified on three independent stores. So these stores are transactable by identified agents, not anonymous ones — capability enumeration is open, transacting is not. That is an accountability layer worth knowing about, and our first phrasing was too strong.

We did not create an agent profile to test past it. Manufacturing an identity in order to transact on someone else's storefront is not something we will do for a cleaner dataset.

Probe was read-only throughout: tools/list only. It never created a cart or started a checkout.

crawlers.json — the data behind all of this

[`crawlers.json`](crawlers.json) is the machine-readable registry the tools are built on: 21 tokens, each with its vendor, purpose, what blocking it actually does, whether it honours robots.txt, the vendor's own words, a source URL, and the date it was checked.

# every token whose blocking removes you from AI answers

curl -s https://raw.githubusercontent.com/krisdiallo/ecom-agent/main/crawlers.json \ | python3 -c "import json,sys;[print(c['token']) for c in json.load(sys.stdin)['crawlers'] \ if c['blocking_effect']=='removes_from_ai_answers']"

Three entries (CCBot, Bytespider, Amazonbot's siblings aside) could not be tied to a first-party quote; those carry an explicit verification flag rather than being dressed up as sourced. research/test_consistency.py fails the build if the CLI and the registry ever disagree — which is how the Amazon misclassification above was caught before publication.

MIT. Use it in your own tool; a link back is welcome but not required.

crawler-consequences.json — the whole ecosystem, and where the data runs out

crawlers.json is deep but narrow: 21 tokens, each with a vendor quote. The ecosystem's list of *which* AI user-agents exist, ai-robots-txt/ai.robots.txt (MIT), is the re