Robots.txt & AI Crawler Checker
robots.txt tester (RFC 9309), AI crawler access (GPTBot, ClaudeBot...) and sitemap check.
하고 싶은 일이나 쓰는 앱으로 검색한 다음, 카테고리로 걸러 보세요.
사용 점수 순: GitHub 스타, 주간 다운로드, 최근 출시 여부, 엔드포인트 응답. 많이 쓴다고 가장 좋은 건 아니에요.
카드 보는 법:↓ 주간 다운로드i★ GitHub 스타i세 기둥: 리뷰 · 점검 · 사용 정도i업데이트 = 최신 버전을 레지스트리에 공개한 날i
robots.txt tester (RFC 9309), AI crawler access (GPTBot, ClaudeBot...) and sitemap check.
Read any public web page as clean Markdown for LLMs, with its metadata. Paid per call, x402.
Publish, share and measure interactive HTML presentations in your Lumdeck workspace.
Free human usability tests for agent-built websites. Submit tasks; get written findings as JSON.
Screenshot any URL, HTML or Markdown in a real browser, and read web pages as Markdown.
How websites treat AI agents: bot walls, JS needs, robots rules, llms.txt. Look up before you fetch.
Free HTML/CSV conversion, JSON validation, text stats and time APIs. Optional agent interviews.
Free AI readiness scan for any website. Paid reports for agents are coming soon.
AI website builder: build, publish and edit websites from Claude, ChatGPT or Claude Code.
Build and test an AI front desk for a service business from its website, before signing up.
Check if a website will block your AI agent and find its llms.txt, MCP or OpenAPI door.
Search hand-picked websites for plumbers, electricians, roofers and other trades, with screenshots.
SEO, broken links, Lighthouse, sitemaps, Open Graph tags and SSL expiry for any list of URLs.
AI-agent readiness of business websites: scans, city indexes, assistants' answers, direct links.
Check one web page for broken links, redirect chains hop by hop and mixed content. Free, no key.
Minecraft builds and skins made by your AI agent, live in your browser tab. Export .schem.
Free, no sign-in: checks a website's readiness for AI search and lists what it found.
Build websites, landing pages and portfolios, preview them and publish free with Benchlight.
Paid website audits for agents: flat JSON, HTML report, deep audit. USDC via x402.
Check if a website is down, explain browser errors and score agent readability (MCP, WebMCP).
Check Swedish limited companies (aktiebolag): registry data, their website and the evidence for it.
Search Accent's public website and get non-binding budgetary estimates from published plan prices.
Monitor web pages for changes: create monitors, list detected changes and read diffs.
Automated accessibility check (axe-core, Lighthouse) of one public web page; fetch saved results.