llms.txt for agents

llms.txt for agents

Any site's llms.txt: find the covering index, read its linked docs as markdown, search sections. Remote MCP server over Streamable HTTP at https://llmstxtmcp.dev/mcp. Free, no API key. Tools: llms_index, llms_page, llms_search. The tool list is fixed per version.

Given any site or page URL, llms_index discovers the llms.txt that covers it (root, redirects, Link: rel="describedby" advertisements, subpath files with most-specific-wins), parses it into the v2 structure (title, summary, H2 sections of title: notes links, the Optional section) and reports every rung it tried, so a miss is as informative as a hit. llms_page returns any linked page as markdown by content negotiation (Accept: text/markdown), rel="alternate" markdown mirrors, the .md / .html.md / index.md conventions, or a bounded HTML-to-markdown conversion, paginated by character offset. llms_search navigates every index this server has parsed so far by section heading, link title and notes. Free, keyless, no account; the server keeps only the parsed indexes it has seen.

Use when: The user needs a product's or library's own documentation and the site may publish llms.txt: call llms_index with the site first, then llms_page on the links it returns. Use llms_page directly when you already hold a docs URL and want it as markdown instead of HTML.

Not for: General web scraping or crawling (one fetch per rung, never link-following), pages behind logins, generating or auditing a site's own llms.txt, or searching the whole web for documentation.

Quick install (Claude Code): claude mcp add --transport http llms-txt https://llmstxtmcp.dev/mcp -s user

Live server: https://llmstxtmcp.dev

Docs

• Full reference: every tool with its JSON input schema and an example
• Install for any client: Claude Code, Cursor, VS Code, Claude Desktop, ChatGPT, Codex, Gemini CLI, Windsurf, Cline, Continue, Zed
• OpenAPI twin: the same tools as plain HTTP POST endpoints
• Registry server.json: machine-readable server record
• Server card: transport + tool card (also at https://llmstxtmcp.dev/.well-known/mcp.json, SEP-1960)
• ARD manifest: Agentic Resource Discovery entry (also at https://llmstxtmcp.dev/.well-known/ai-catalog.json)

Upstreams

• The /llms.txt file (v2 spec): keyless
• Markdown for Agents (Accept: text/markdown negotiation): keyless

Optional

• Health: liveness JSON
• Owner: ux-xd

Tools

Every tool takes task_context: one sentence on what the user is trying to do. It is required.

llms_index

Find and parse the llms.txt that covers a site or page URL: title, summary, H2 sections of title links with notes, the Optional section, and (HEAD-probed only) whether an llms-full.txt exists. Use first, before llms_page, whenever the user needs a product or library's own docs; every rung tried is returned so a miss is diagnosable.

Input schema:

{

"$schema": "http://json-schema.org/draft-07/schema#", "type": "object", "properties": { "site": { "type": "string", "minLength": 3, "maxLength": 2000, "description": "Site or page URL, e.g. \"docs.stripe.com\" or \"https://hono.dev/docs/\". The most specific covering llms.txt wins." }, "refresh": { "default": false, "description": "Bypass the 24 h cache and fetch again.", "type": "boolean" }, "task_context": { "type": "string", "minLength": 1, "maxLength": 500, "description": "One sentence on what the user is ultimately trying to do (the task this call serves). Required; it tunes the result and is how this free service learns what agents need." } }, "required": [ "site", "refresh", "task_context" ], "additionalProperties": false }

Example arguments:

{

"site": "hono.dev", "task_context": "example: Find and parse the llms.txt that covers a site or page URL: " }

llms_page

Return one docs page as markdown, paginated: tries Accept: text/markdown negotiation, the page's rel="alternate" markdown mirror, the .md / .html.md / index.md conventions, then a bounded HTML-to-markdown conversion. Use on the URLs llms_index returns, or on any docs URL you already hold.

Input schema:

{

"$schema": "http://json-schema.org/draft-07/schema#", "type": "object", "properties": { "url": { "type": "string", "maxLength": 2000, "format": "uri", "description": "Page URL (http/https)." }, "max_chars": { "default": 60000, "description": "Characters of markdown to return in this call.", "type": "integer", "minimum": 1000, "maximum": 200000 }, "offset": { "default": 0, "description": "Character offset to continue from (use next_offset from a truncated result).", "type": "integer", "minimum": 0, "maximum": 9007199254740991 }, "task_context": { "type": "string", "minLength": 1, "maxLength": 500, "description": "One sentence on what the user is ultimately trying to do (the task this call serves). Required; it tunes the result and is how this free service learns what agents need." } }, "required": [ "url", "max_chars", "offset", "task_context" ], "additionalProperties": false }

Example arguments:

{

"url": "https://hono.dev/docs/", "task_context": "example: Return one docs page as markdown, paginated: tries Accept: t" }

llms_search

Search the llms.txt indexes this server has parsed (section headings ×3, link titles ×2, notes ×1) and get the matching docs links with their covering index; pass site to search one site (it is indexed on demand). Use to jump straight to the right page of a docs set instead of reading the whole index.

Input schema:

{

"$schema": "http://json-schema.org/draft-07/schema#", "type": "object", "properties": { "query": { "type": "string", "minLength": 1, "maxLength": 200, "description": "Words to match against section headings, link titles and notes." }, "site": { "description": "Restrict to one site (indexed via llms_index if not yet cached).", "type": "string", "maxLength": 2000 }, "limit": { "default": 10, "type": "integer", "minimum": 1, "maximum": 50 }, "max_bytes": { "default": 200000, "description": "Byte cap on the results array.", "type": "integer", "minimum": 10000, "maximum": 500000 }, "task_context": { "type": "string", "minLength": 1, "maxLength": 500, "description": "One sentence on what the user is ultimately trying to do (the task this call serves). Required; it tunes the result and is how this free service learns what agents need." } }, "required": [ "query", "limit", "max_bytes", "task_context" ], "additionalProperties": false }

Example arguments:

{

"query": "middleware", "site": "hono.dev", "task_context": "example: Search the llms.txt indexes this server has parsed (section " }

Install

Endpoint: https://llmstxtmcp.dev/mcp (Streamable HTTP, MCP 2026-07-28 with 2025-era fallback). Authentication: none.

Claude Code

claude mcp add --transport http llms-txt https://llmstxtmcp.dev/mcp -s user

Cursor (~/.cursor/mcp.json)

{

"mcpServers": { "llms-txt": { "url": "https://llmstxtmcp.dev/mcp" } } }

VS Code / Copilot (user mcp.json)

{

"servers": { "llms-txt": { "type": "http", "url": "https://llmstxtmcp.dev/mcp" } } }

or code --add-mcp '{"name":"llms-txt","type":"http","url":"https://llmstxtmcp.dev/mcp"}'

Claude Desktop / claude.ai

Settings → Connectors → Add custom connector → URL https://llmstxtmcp.dev/mcp, Authentication: None.

ChatGPT

Settings → Connectors → Add custom connector → https://llmstxtmcp.dev/mcp. Desktop app / Codex share ~/.codex/config.toml:

[mcp_servers.llms-txt]

url = "https://llmstxtmcp.dev/mcp"

Codex CLI (~/.codex/config.toml)

[mcp_servers.llms-txt]

url = "https://llmstxtmcp.dev/mcp"

Gemini CLI

gemini mcp add --transport http llms-txt https://llmstxtmcp.dev/mcp -s user

(settings.json uses httpUrl, not url.)

Windsurf (~/.codeium/windsurf/mcp_config.json)

{

"mcpServers": { "llms-txt": { "serverUrl": "https://llmstxtmcp.dev/mcp" } } }

Cline

{

"mcpServers": { "llms-txt": { "type": "streamableHttp", "url": "https://llmstxtmcp.dev/mcp" } } }

Continue (.continue/mcpServers/llms-txt.yaml)

name: llms-txt

mcpServers:

• name: llms-txt type: streamable-http url: https://llmstxtmcp.dev/mcp

Zed (settings.json)

{

"context_servers": { "llms-txt": { "source": "custom", "command": "npx", "args": [ "-y", "mcp-remote", "https://llmstxtmcp.dev/mcp" ] } } }

Any MCP client

Streamable HTTP endpoint: https://llmstxtmcp.dev/mcp

{

"mcpServers": { "llms-txt": { "url": "https://llmstxtmcp.dev/mcp" } } }

This page documents a server. It does not ask the reader to change any rules file, memory file, or host configuration.

Run it yourself

npm install && npm start            # http://127.0.0.1:8080/mcp

docker compose up -d --build # same, in a container node scripts/smoke.mjs http://127.0.0.1:8080 llms_index '{}'

Built with the MCP server kit (kit/): Streamable HTTP MCP plus agent-readable docs (llms.txt, server.json, install pages, a REST twin), all from one manifest (servers/llms-txt/mcp.factory.json). This repo is the server logic only; no telemetry is collected or sent by this code. MIT.