Pulse Fetch

Pulse Fetch MCP Server

Haven't heard about MCP yet? The easiest way to keep up-to-date is to read our weekly newsletter at PulseMCP.

---

This is an MCP (Model Context Protocol) Server that pulls specific resources from the open internet into context, designed for agent-building frameworks and MCP clients that lack built-in fetch capabilities.

Pulse Fetch is purpose-built for extracting clean, structured content from web pages while minimizing token usage and providing reliable access to protected content through advanced anti-bot bypassing capabilities.

This project is built and maintained by PulseMCP.

Table of Contents

• Highlights
• Capabilities
• Usage Tips
• Examples
• Setup
• Prerequisites
• Environment Variables
• Claude Desktop
• Local Setup
• Remote Setup
• Development
• Scraping Strategy Configuration
• Extract Feature

Highlights

Clean content extraction: Strips out HTML noise using Mozilla's Readability algorithm to minimize token usage during MCP Tool calls.

Intelligent caching: Automatically caches scraped content as MCP Resources. Subsequent requests for the same URL return cached content instantly without network calls, dramatically improving performance.

Anti-bot bypass: Integrates with Firecrawl and BrightData APIs to reliably work around anti-scraping technology.

Smart strategy selection: Automatically learns and applies the best scraping method for specific URL patterns, improving performance over time.

LLM-optimized: Offers MCP Prompts and descriptive Tool design for better LLM interaction reliability.

Flexible formats: Supports multiple output formats including clean markdown, HTML, screenshots, and structured data extraction.

Intelligent extraction: Extract specific information using natural language queries powered by LLMs.

Capabilities

This server is built and tested on macOS with Claude Desktop. It should work with other MCP clients as well.

Usage Tips

• The scrape tool handles all web content extraction needs and automatically bypasses anti-bot protection when necessary
• Result handling modes: Control how scraped content is handled with the resultHandling parameter:
• saveAndReturn (default): Saves content as MCP Resource AND returns it - best for reuse
• returnOnly: Returns content without saving - use when you only need content once
• saveOnly: Saves content as MCP Resource, returns only resource link - efficient for large content
• Automatic caching: Previously scraped URLs are cached by default. The tool returns cached content instantly on repeat requests
• Use forceRescrape: true to bypass the cache and get fresh content when you know the page has changed
• Use maxChars and startIndex parameters to handle large content that exceeds token limits
• Configure the timeout parameter (default 60s) for slow-loading sites
• Use the extract parameter with natural language queries to extract specific information from pages (requires LLM configuration)

Examples

Basic Web Content Extraction

User: "Get the main content from this article: https://example.com/article"

Assistant: I'll extract the content from that article for you.

[Uses scrape tool with onlyMainContent: true]

I've extracted the main article content. The article is titled "Example Article Title" and contains approximately 2,500 words discussing...

Extracting Structured Data

User: "Extract the product information from this e-commerce page: https://shop.example.com/product/123"

Assistant: I'll extract the structured product data from that page.

[Uses scrape tool with format: 'extract' and appropriate schema]

I've extracted the product information:

• Product Name: Example Product
• Price: $99.99
• Rating: 4.5/5 stars
• Description: High-quality example product...

Handling Protected Content

User: "This page is blocking me with CAPTCHA. Can you get the content from https://protected.example.com/data"

Assistant: I'll extract the content from that protected page for you.

[Uses scrape tool with automatic anti-bot bypass]

I successfully bypassed the protection and extracted the content from the page using BrightData's Web Unlocker capabilities.

Intelligent Caching

User: "Get the content from https://example.com/article again"

Assistant: I'll retrieve that content for you.

[Uses scrape tool - automatically returns cached content]

I've retrieved the content from cache (originally scraped 2 hours ago). The article contains...

User: "Actually, I think that article was updated. Can you get the latest version?" Assistant: I'll fetch a fresh copy of the article for you.

[Uses scrape tool with forceRescrape: true]

I've fetched the latest version of the article. I can see it was indeed updated with new information about...

Result Handling Options

User: "I need to archive this entire article: https://example.com/long-article"

Assistant: I'll save that article as a resource for you.

[Uses scrape tool with resultHandling: 'saveOnly']

I've successfully archived the article. It's now saved as an MCP Resource that you can access later.

User: "Can you extract just the summary from https://example.com/report without saving it?" Assistant: I'll extract just the summary without saving the full content.

[Uses scrape tool with resultHandling: 'returnOnly' and extract parameter]

Here's the summary I extracted from the report: [summary content]

Why Choose Pulse Fetch?

If our Design Principles resonate with you, you should consider using our server.

The official reference implementation of fetch is the closest alternative. However:

• fetch has no mechanisms for bypassing anti-scraping technology, meaning attempts may randomly fail. We integrate with third-party services for reliable access.
• fetch is maintained by volunteers, so bugs or edge cases are less likely to be addressed quickly.
• Pulse Fetch caches responses as Resources, allowing easy inspection and re-use of Tool call outcomes.
• Pulse Fetch has more descriptive Tool design that more reliably triggers and completes desired tasks.

Most other alternatives fall short on one or more vectors:

• They are not purpose-built for pulling specific internet resources into context
• They may be multi-page crawlers, search engines, or tied to full REST APIs that confuse LLMs
• They do not make maximal use of MCP concepts like Resources and Prompts

Setup

Prerequisites

• Node.js (recommended: use the version specified in package.json)
• Claude Desktop application (for local setup)
• Optional: Firecrawl API key for enhanced scraping capabilities
• Optional: BrightData bearer token for web unlocking features

Environment Variables

Core Configuration

LLM Configuration for Extract Feature

The extract feature provides an alternative to MCP's native sampling capability for clients that don't support it. When configured, it enables intelligent information extraction from scraped content using LLMs. If neither LLM configuration nor MCP sampling is available, the extract parameter will not be shown in the tool.

Default Models:

• Anthropic: claude-sonnet-4-20250514 (Claude Sonnet 4 - latest and most capable)
• OpenAI: gpt-4.1-mini (GPT-4.1 Mini - latest and most capable)
• OpenAI-compatible: Provider-specific (must be specified)

Claude Desktop

Local Setup

You'll need Node.js installed on your machine to run the local version.

macOS: ~/Library/Application Support/Claude/claude_desktop_config.json

Windows: %APPDATA%\Claude\claude_desktop_config.json

Add this configuration to your Claude Desktop config file:

Minimal configuration (uses native fetch only):

{

"mcpServers": { "pulse-fetch": { "command": "npx", "args": ["-y", "@pulsemcp/pulse-fetch"] } } }

Full configuration (with all optional environment variables):

{

"mcpServers": { "pulse-fetch": { "command": "npx", "args": ["-y", "@pulsemcp/pulse-fetch"], "env": { "FIRECRAWL_API_KEY": "your-firecrawl-api-key", "BRIGHTDATA_API_KEY": "your-brightdata-bearer-token", "STRATEGY_CONFIG_PATH": "/path/to/your/scraping-strategies.md", "OPTIMIZE_FOR": "cost", "MCP_RESOURCE_STORAGE": "filesystem", "MCP_RESOURCE_FILESYSTEM_ROOT": "/path/to/resource/storage" } } } }

To set up the local version:

1. Clone or download the repository
2. Navigate to the local directory: cd pulse-fetch/local
3. Install dependencies: npm install
4. Build the project: npm run build
5. Update your Claude Desktop config with the correct