FrankenSurf

Frankensurf

Access 96% of the web, cheaply and accurately.

A free, self-hosted alternative to Firecrawl: one container, the same API, and every page checked before your agent reads it.

Website · Docs · Quick start · Benchmark · Why I built it

---

Most scrapers count a 200 as a success, so your agent ends up reading challenge pages, empty app shells and menus, and answering anyway. Frankensurf reads each page with the cheapest tool that works, checks it's the real page, and climbs to stealth browsers (or paid APIs, if you allow them) only when a site pushes back. It speaks Firecrawl's API, so anything with a Firecrawl URL setting can point at it.

Gets past walls 41 web tools behind one call: plain HTTP, Jina Reader, browsers, free stealth browsers, then paid APIs only if you allow them. Every page checked A challenge page, a login wall or a search page whose results never loaded counts as a failure, with a receipt saying why. Drop-in for Firecrawl Same API, one Docker container. Check it on your own URLs first with frankensurf compare .

On 150 sites picked blind: 96% read right with paid fallbacks against 85% for Firecrawl's cloud on its own, and 88% with free tools only. On a harder blind set of 148 less-known sites: 88.5% against 70.3%, and 81.8% free. Against a self-hosted Firecrawl on the same sites, 126 read against 89 (details, losses included).

Quick start

As a Firecrawl-compatible server (one container, free tools inside, no account needed):

docker run -d -p 3002:3002 --shm-size=1g -e FRANKENSURF_API_KEY=pick-a-key ghcr.io/yail259/frankensurf

Then point your tool's Firecrawl URL at http://localhost:3002 with that key (which tools, and how).

As a Python package and CLI (Python 3.12+, on Linux, macOS, or Windows through WSL):

pip install "frankensurf[mcp]"

frankensurf install # every free tool: browsers, stealth browsers, local search, Crawl4AI frankensurf read "https://www.walmart.com/search?q=air+fryer" --explain

The read climbs until it gets the real page:

https://www.walmart.com/search?q=air+fryer

✗ http CAPTCHA 0.4s ✗ local CAPTCHA 1.5s ✓ camoufox got the page 15.6s

air fryer - Walmart.com 24,192 characters via camoufox, complete (107 results), free

frankensurf install works like playwright install: name just the tools you want (frankensurf install search chromium), add --with-deps for the browsers' system libraries on Linux, --list to see what's installed where, and frankensurf uninstall all to remove it. No Docker needed. Each release is installed from PyPI on fresh Ubuntu and macOS machines and checked.

Use it

From an agent, as an MCP server (setup for each client):

claude mcp add frankensurf -- frankensurf-mcp

From Python:

import frankensurf as fs

page = fs.read("https://example.com/pricing") if page.ok: print(page.title, page.text[:500]) else: print("couldn't read it:", page.failure) # {"code": "CAPTCHA", "message": ...}

pages = fs.batch(urls, links=True) # paced per site, in order hits = fs.search("playwright python tutorial", recency="month") page = await fs.aread(url) # aread, abatch, asearch inside async code

For a long-running service, keep one Runtime open: async with Runtime() as web: await web.read(url).

Map and crawl: fs.map(url) lists a site's URLs; fs.crawl(url, limit=50) reads a section of it as markdown.

Agent mode (pip install "frankensurf[agent]", any LiteLLM model): fs.ask("question") searches, reads and answers with sources, and says what it couldn't read; fs.extract(urls, schema=...) and fs.summarize(url). Agent mode.

In Docker, one install for your whole network: docker run -p 3002:3002 -e FRANKENSURF_API_KEY=... ghcr.io/yail259/frankensurf (Docker).

From any tool that takes a Firecrawl URL (GPT Researcher, Hermes Agent, OpenClaw, Open WebUI, Dify, LibreChat, n8n, LangChain, LlamaIndex and others): frankensurf serve, then point it at http://127.0.0.1:3002 (setup).

From the command line:

frankensurf read https://example.com --markdown

frankensurf search playwright python tutorial --recency week frankensurf watch https://news.ycombinator.com/ --link-pattern 'item\?id=\d+'

In Claude Code as a plugin (the MCP server plus the skill that teaches Claude to use it):

/plugin marketplace add yail259/frankensurf

/plugin install frankensurf@frankensurf

From LangChain, the OpenAI Agents SDK or CrewAI (pip install "frankensurf[langchain]", and so on):

from frankensurf.agent_tools import langchain_tools   # or openai_agents_tools, crewai_tools

tools = langchain_tools() # frankensurf_read, frankensurf_search

You get raw page data, not answers: text or markdown, JSON-LD, the page's own JSON, images, and the receipt. Your agent decides what the page means. If it isn't what it wanted, it asks again with retry_of= and Frankensurf skips every tool it already tried.

Benchmark

Cost per 1,000 correct pages: Frankensurf's free tools $0 in fees, about $0.01 of a $5/month server (87–88% right); with paid fallbacks $0.22 (96%); Firecrawl on its own $0.71 on its Scale plan, $0.98 on Standard (85%). How that's worked out.

scripts/toolbench.py runs each tool on its own, then Frankensurf, on the same pages with the same content checks. The newest set: 148 less-known sites in 15 categories, picked before any was read, every tool on its own, paid ones included (28 October 2026, rows and corrections in benchmarks/2026-10-28-heldout9/).

• Frankensurf read 28 sites Firecrawl missed; Firecrawl read 1 it missed.
• Every tool's reads put together come to 84.5%; Frankensurf reads more than that, through retries and rescues.
• Its slowest tenth takes 33.0 s against Firecrawl's 10.9 s.
• This set is harder than the last: every tool read 10 to 20 points less. A rerun the same evening gave 87.2% and 79.7%.

The set before (150 sites, 11 October 2026, rows in benchmarks/2026-10-11/):

• Frankensurf read 17 sites Firecrawl missed; Firecrawl read none it missed.
• With no paid tools at all it is level with the best paid tool on its own (88.0% against 86.0% and 84.7%; at 150 sites that gap is within the noise).
• Its slowest reads are slower: the slowest tenth takes 25.7 s against Firecrawl's 10.6 s, when a read climbs several tools.
• Not a one-off: 95.3% vs 85.3% on the set the day before; 89.5% vs 80.3% and 88.5% vs 74.1% on earlier fresh sets.

Every tool on the 11 October set

Medians are over pages read; cost is per 1,000 pages read right. ZenRows and Scrapfly are left out (their plans ran out of credits). From a fresh cloud VM, whose data-centre address sites wall more often: free tools 82.0%, lightning mode 84.0% at a 1.2 s median. See Benchmarks.

Against self-hosted Firecrawl

Firecrawl is open source too, so the fair question for anyone running their own is how the Firecrawl you'd self-host does. Its docs say self-hosted instances don't get Fire-engine, the anti-bot layer of its cloud. Same 150 sites, 23 October 2026, both servers in Docker on one machine, each site read by both with the first reader alternating site by site:

• Frankensurf read 38 sites the self-hosted Firecrawl did