Clean Markdown

clean-markdown-mcp

An MCP server that turns any URL into clean, LLM-ready Markdown — no ads, no nav bars, no cookie banners, no scripts.

Built for RAG pipelines and AI agents, where junk in the input means junk in the output.

Give it:  https://en.wikipedia.org/wiki/Markdown

Get back: # Markdown

Markdown is a lightweight markup language for creating formatted text using a plain-text editor...

Install

npm install -g clean-markdown-mcp

Use it with Claude Desktop

Add this to your claude_desktop_config.json:

• macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
• Windows: %APPDATA%\Claude\claude_desktop_config.json

{

"mcpServers": { "clean-markdown": { "command": "npx", "args": ["-y", "clean-markdown-mcp"] } } }

Restart Claude Desktop, then ask it to *"scrape https://example.com to Markdown."*

Works the same way in Cursor, Windsurf, or any other MCP client — point it at the clean-markdown-mcp command.

The tool it exposes

`scrape_url`

Returns the page title, source, word count, and the clean Markdown.

How the cleaning works

1. Fetch — with a 20-second timeout, a 5 MB size cap, and redirects validated at every hop.
2. Clean — runs Mozilla Readability, the engine behind Firefox Reader View, to isolate the real article and discard navigation, sidebars, ads, and comment sections.
3. Convert — Turndown renders it as GitHub-flavoured Markdown, preserving headings, lists, tables, and code blocks.

If a page isn't article-shaped, it falls back to a cleaned rather than failing, so you still get usable text.

JavaScript-heavy sites (optional)

Some sites build their content with JavaScript after loading. A plain fetch returns almost nothing for those. To handle them, install Playwright once:

npm install playwright && npx playwright install chromium

Then pass renderJs: true. The difference on such a page is dramatic:

Playwright is not a dependency of this package, so a normal install stays small and fast. Without it, renderJs returns a clear message telling you how to enable it.

Security

The scraper refuses private, loopback, and link-local addresses — including cloud metadata endpoints like 169.254.169.254 — and re-validates every redirect hop. This matters because an MCP server runs on your machine, with access to your local network.

Need to scrape at scale?

This package handles one page at a time, locally. For batch scraping, hosted JavaScript rendering, and a pay-per-page API with no infrastructure to run, the same engine is available as a hosted Actor:

https://apify.com/perforated_hummingbird/url-to-markdown

Known limitations

• Some sites block automated traffic. That's a site policy, not a bug here.
• Non-UTF-8 pages may render with garbled characters.
• No robots.txt enforcement — you are responsible for how you use it.

Local development

npm install

npm run build node dist/try.js https://example.com # quick test node dist/try.js https://example.com --render # with JS rendering

License

MIT