Robots.txt & AI Crawler Checker
robots.txt tester (RFC 9309), AI crawler access (GPTBot, ClaudeBot...) and sitemap check.
やりたいことや使っているアプリで検索し、カテゴリで絞り込めます。
利用スコア順:GitHubスター、週あたりのダウンロード数、最近のリリース、エンドポイントの応答。よく使われている=一番良い、ではありません。
カードの見方:↓ 週あたりのダウンロードi★ GitHubスターi3つの柱:レビュー · チェック · 利用状況i更新=最新バージョンをレジストリに公開した日i
robots.txt tester (RFC 9309), AI crawler access (GPTBot, ClaudeBot...) and sitemap check.
Read any public web page as clean Markdown for LLMs, with its metadata. Paid per call, x402.
Publish, share and measure interactive HTML presentations in your Lumdeck workspace.
Free human usability tests for agent-built websites. Submit tasks; get written findings as JSON.
Screenshot any URL, HTML or Markdown in a real browser, and read web pages as Markdown.
How websites treat AI agents: bot walls, JS needs, robots rules, llms.txt. Look up before you fetch.
Free HTML/CSV conversion, JSON validation, text stats and time APIs. Optional agent interviews.
Free AI readiness scan for any website. Paid reports for agents are coming soon.
AI website builder: build, publish and edit websites from Claude, ChatGPT or Claude Code.
Build and test an AI front desk for a service business from its website, before signing up.
Check if a website will block your AI agent and find its llms.txt, MCP or OpenAPI door.
Search hand-picked websites for plumbers, electricians, roofers and other trades, with screenshots.
SEO, broken links, Lighthouse, sitemaps, Open Graph tags and SSL expiry for any list of URLs.
AI-agent readiness of business websites: scans, city indexes, assistants' answers, direct links.
Check one web page for broken links, redirect chains hop by hop and mixed content. Free, no key.
Minecraft builds and skins made by your AI agent, live in your browser tab. Export .schem.
Free, no sign-in: checks a website's readiness for AI search and lists what it found.
Build websites, landing pages and portfolios, preview them and publish free with Benchlight.
Paid website audits for agents: flat JSON, HTML report, deep audit. USDC via x402.
Check if a website is down, explain browser errors and score agent readability (MCP, WebMCP).
Check Swedish limited companies (aktiebolag): registry data, their website and the evidence for it.
Search Accent's public website and get non-binding budgetary estimates from published plan prices.
Monitor web pages for changes: create monitors, list detected changes and read diffs.
Automated accessibility check (axe-core, Lighthouse) of one public web page; fetch saved results.