Intercept

intercept-mcp

Give your AI the ability to read the web. One command, no API keys required.

Without it, your AI hits a URL and gets a 403, a wall, or a wall of raw HTML. With intercept, it almost always gets the content — clean markdown, ready to use.

Handles tweets, YouTube videos (with transcripts when available), arXiv papers, PDFs, Wikipedia articles, and GitHub repos. If the first strategy fails, it tries up to 14 more before giving up.

Works with any MCP client: Claude Code, Claude Desktop, Codex, Cursor, Windsurf, Cline, and more.

Install

Claude Code

claude mcp add intercept -s user -- npx -y intercept-mcp

Codex

codex mcp add intercept -- npx -y intercept-mcp

Cursor

Settings → MCP → Add Server:

{

"mcpServers": { "intercept": { "command": "npx", "args": ["-y", "intercept-mcp"] } } }

Windsurf

Settings → MCP → Add Server → same JSON config as above.

Claude Desktop

Add to your claude_desktop_config.json:

{

"mcpServers": { "intercept": { "command": "npx", "args": ["-y", "intercept-mcp"] } } }

Other MCP clients

Any client that supports stdio MCP servers can run npx -y intercept-mcp.

No API keys needed for the fetch tool.

How it works

URLs are processed in four stages:

1. Site-specific handlers

Known URL patterns are routed to dedicated handlers before the fallback pipeline:

The GitHub API endpoints work unauthenticated (60 requests/hour). Set GITHUB_TOKEN to raise the limit.

2. Shared cache (agentsweb.org)

After specialized handlers, public requests check agentsweb.org for a complete result within the requested freshness window. Partial, stale, mismatched, and blocked responses are rejected. Cache latency depends on the network and storage location.

Complete public live retrievals can contribute refresh hints. agentsweb verifies a page through its own retrieval pipeline before publishing it. Confirmations do not reset retrieval time. Authenticated, signed, and GitHub-token requests never use the public cache.

Opt out entirely with INTERCEPT_SHARED_CACHE=false, or use read-only mode (consume but never contribute) with INTERCEPT_CACHE_READ_ONLY=true.

agentsweb.org API

agentsweb.org also exposes standalone endpoints for direct use:

• `/web?q=` — search the web
• `/research?q=` — search + fetch + cache in one call
• `/fetch?url=` — fetch on demand, auto-cached

See agentsweb.org/docs for full API documentation.

3. Fallback pipeline

If no handler matches (or the handler returns nothing), the URL enters the multi-tier pipeline:

A successful hosted result stops a slower provider after a 200 ms comparison window. Repeated provider failures trigger a short cooldown. Direct-host health is tracked per host. Historical snapshots never outrun live sources; noCache disables archives unless allowArchive is explicitly true.

4. Caching

Results are cached in-memory with TTL (60 min for successes, 5 min for failures). Max 250 entries with LRU eviction. Failures are cached by retrieval policy, so a shallow failed attempt does not prevent a later broader retry. Duplicate in-flight requests share work while retaining independent cancellation. All three knobs are configurable via INTERCEPT_CACHE_TTL_MS, INTERCEPT_CACHE_FAILURE_TTL_MS, and INTERCEPT_CACHE_SIZE.

Tools

fetch

Fetch a URL and return its content as clean markdown.

• url (string, required) — URL to fetch
• maxTier (number, optional, 1-5) — Stop at this tier for speed-sensitive cases
• maxLength (number, optional, default 50000) — Maximum characters to return
• startIndex (number, optional, default 0) — Character offset for paginating long content
• noCache (boolean, optional) — Skip session and shared caches and fetch live
• timeoutMs — Overall deadline, including downloads (default 25000; research 35000)
• maxAgeSeconds — Maximum cache age (default 3600; 0 forces retrieval)
• allowArchive — Permit labeled historical fallbacks (disabled by noCache unless explicitly enabled)

Long pages are truncated at maxLength with a notice telling the agent which startIndex continues the content. Structured output reports source, quality, contentLength, truncated, nextStartIndex, and cacheAgeSeconds so agents can branch on them programmatically. It also reports completeness, sourceKind, retrievedAt, verifiedAt when available, canonicalUrl, contentHash, and title. Research includes verbatim relevant passages with offsets and citation URLs; set passagesOnly to return those excerpts instead of full documents.

Direct image URLs (.png, .jpg, .gif, .webp, up to 5 MB) are returned as an MCP image block instead of text, so the agent's own vision model can read charts, diagrams, screenshots, and scanned documents. The structured output reports source: "image", mimeType, and bytes.

fetch_batch

Fetch up to 10 URLs in parallel, each through the same handler/fallback chain.

• urls (string[], required, 1-10) — URLs to fetch
• maxTier, noCache — as in fetch
• maxLength (number, optional, default 20000) — Per-URL character budget

research

Search the web and fetch the top results in one call — replaces a search followed by several fetches.

• query (string, required) — Search query
• count (number, optional, 1-5, default 3) — Results to fetch
• maxLength (number, optional, default 20000) — Per-result character budget
• site (string, optional) — Restrict to a domain
• freshness (string, optional) — day, week, month, or year

search

Search the web and return results.

• query (string, required) — Search query
• count (number, optional, 1-20, default 5) — Number of results
• site (string, optional) — Restrict results to a domain
• freshness (string, optional) — day, week, month, or year
• page (number, optional, 1-10) — Results page for pagination

Uses Brave Search API if BRAVE_API_KEY is set, then SearXNG if SEARXNG_URL is set, then DuckDuckGo as an unreliable last resort. freshness and page are ignored by the DuckDuckGo fallback.

extract

Extract specific values from a page as JSON instead of markdown prose — for when you need particular data, not the whole page. Honors per-domain auth and proxies.

• url (string, required) — The URL to extract from
• selectors (object, optional) — Map of field name → CSS selector. Each value is either a selector string (returns the first match's text) or { selector, attr?, all? } — attr extracts an attribute (e.g. href), all: true returns every match as an array.
• tables (boolean, optional) — Convert every HTML table to an array of row objects (defaults to true when no selectors are given).

{

"url": "https://shop.example.com/item", "selectors": { "title": "h1", "price": ".price", "images": { "selector": "img.gallery", "attr": "src", "all": true } } }

Returns the extracted fields and/or tables as structured output.

Resources

intercept://session/recent

Markdown list of URLs fetched and cached in this session, most recent first. Re-fetching any of them is instant.

Prompts

research-topic

Search for a topic and fetch the top results for a multi-source summary.

• topic (string) — The topic to research
• depth (string, default "3") — Number of top results to fetch

extract-article

Fetch a URL and extract the key points from the content.

• url (string) — The URL to fetch and summarize

Environment variables

Search: Has a DuckDuckGo fallback but it's rate-limited and unreliable. For production use, self-host SearXNG and set SEARXNG_URL (see below), or get a Brave Search API key.

Fetch: Works without any keys. Set CF_API_TOKEN + CF_ACCOUNT_ID to enable Cloudflare Browser Run (formerly Browser Rendering) for JavaScript-heavy pages (SPAs, React sites).

Stealth fetch (USE_STEALTH_FETCH)

Use at your own risk. When enabled, t