Prototype · Preview: every server in the official registry, synced every 30 minutes. Sign in with Google to write reviews.Prototype · updated every 30 min
PopNx MCPMCP servers for people who use AI · explained in plain words

MTScraper

by mazprotechversion 2.1.1i★ 0iRuns on your computerRepository ↗
Registry nameio.github.mazprotech/mtscraper

How to install

What it does

Free local crawler: any site, JavaScript pages and linked PDFs, into clean text.

  • Connects Runs on your computer
  • Account or key No key needed
  • Category Web & browsers

Use together with

Skills that teach the AI this kind of work; they work alongside this one

timesfm

TimesFM (Time Series Foundation Model) is a pretrained time-series foundation model developed by Google Research for time-series forecasting.

by google-research★ 35,6291 skills

Skill · How-to guide

Shadowbroker

Open-source intelligence for the global theater. Track everything from the corporate/private jets of the wealthy, and spy satellites, to seismic events in one unified interface. Hook an AI agent up to have it parse through data and find previously unseen correlations. The knowledge is available to…

by BigBodyCobain★ 11,6021 skills

Skill · How-to guide

google-maps-scraper

scrape data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place

by gosom★ 8,2091 skills

Skill · How-to guide

Why it ranks here

Lists are ordered by a usage score (out of 100) built from these parts

Popularity0/80GitHub stars or weekly downloads: the larger counts 60, the smaller 20
Freshness12/12Full marks for a release or code change within 30 days
Still answering0/8The cloud endpoint answered at the last check
Usage score12/100

Quality: Passed 3/3 basic checks (shown, not used for ordering) · Momentum: No data yet (Used only on the Trending tab)

Install

Pick the AI app you use, then copy the settings.

Let your AI install it with one sentence

Please install MTScraper. Follow https://mcp.popnx.com/i/mazprotech-mtscraper.md, show me the commands and wait for my OK before running them.

Paste it into an AI that can work on your computer (such as Claude Code or Cursor). It reads our steps, shows you the commands and waits for your OK. No key, no sign-up. · See the full text your AI reads

Passed 3/3 checks · Runs on your computer · ✓ Commands checked Nov 15, 2026

First: which AI do you use?

Pick your app and the steps appear below.

Works with Claude, with some setup first

Install Node.js on your computer first (one time) · Only the Claude desktop app (Windows or Mac), not the web or phone

Let AI install it

Copy this message into a Claude chat. The AI checks your computer, tells you which file it will change, and waits for your OK every time.

Please install the MCP server MTScraper for Claude on this computer, in this order: 1) Check Node.js with node -v. If it is missing, tell me to install it from https://nodejs.org/en/download; do not install it yourself. 2) Use only this config (already checked by PopNx MCP): { "mcpServers": { "mtscraper": { "command": "npx", "args": [ "-y", "mtscraper", "--mcp" ] } } } 3) Tell me the full path of the file you will change, or the command you will run, and wait for my OK. 4) No key is needed. 5) Do what we agreed, then check that the MTScraper tools really show up.

This message has no key in it. If a key is needed, the AI will ask you to paste it yourself into the file or the app’s field.

  1. Install Node.js (if you don’t have it)

    Already installed? Check first · Open Terminal, type this command and press Enter

    node -v
    • A version number such as v26.13.1 = installed, skip this step
    • "command not found" (on Windows: "is not recognized") = not installed yet, or installed but Terminal needs to be closed and reopened
    How to open Terminal

    Mac: press ⌘ + Space, type Terminal, press Enter

    Windows: open Start, type Windows PowerShell, then Open

    Not installed: Open the official download page, pick the LTS version and the click-through installer: .pkg on Mac · .msi on Windows. Double-click and follow it to the end. No curl command needed

    Open the Node.js download page ↗

    When it’s done, close Terminal, open it again and run the same command. You should see a version number: node -v

    Install once; it works for everything that needs Node.js

  2. Open the Claude desktop app’s settings file

    SettingsDeveloperEdit Config

  3. Paste these settings and save the file

    {
      "mcpServers": {
        "mtscraper": {
          "command": "npx",
          "args": [
            "-y",
            "mtscraper",
            "--mcp"
          ]
        }
      }
    }
  4. Quit Claude and open it again

  5. Try it

    Use MTScraper and tell me what you can do for me, then show me one thing

    ✓ If Claude says it is using MTScraper or shows a MTScraper tool in the answer, you are connected

Menu names checked against Anthropic’s help page on Nov 11, 2026 · your screen may differ slightly by app version

ChatGPT cannot connect MTScraper yet

MTScraper runs on your computer, but ChatGPT can only connect servers that run on the web

What you can do instead

A · Use an app that can connect MTScraper

  • Install Node.js on your computer first (one time) · Only the Claude desktop app (Windows or Mac), not the web or phone
  • Install Node.js on your computer first (one time)
  • Install Node.js on your computer first (one time)
  • Install Node.js on your computer first (one time)

B · Others in Web & browsers that work with ChatGPT

Picked from the most used in the same category, only ones that connect by address with no key

Works with Claude Code, with some setup first

Install Node.js on your computer first (one time)

Let AI install it

Copy this message into a Claude Code chat. The AI checks your computer, tells you which file it will change, and waits for your OK every time.

Please install the MCP server MTScraper for Claude Code on this computer, in this order: 1) Check Node.js with node -v. If it is missing, tell me to install it from https://nodejs.org/en/download; do not install it yourself. 2) Use only this config (already checked by PopNx MCP): claude mcp add --scope user mtscraper -- npx -y mtscraper --mcp 3) Tell me the full path of the file you will change, or the command you will run, and wait for my OK. 4) No key is needed. 5) Do what we agreed, then check that the MTScraper tools really show up.

This message has no key in it. If a key is needed, the AI will ask you to paste it yourself into the file or the app’s field.

  1. Install Node.js (if you don’t have it)

    Already installed? Check first · Open Terminal, type this command and press Enter

    node -v
    • A version number such as v26.13.1 = installed, skip this step
    • "command not found" (on Windows: "is not recognized") = not installed yet, or installed but Terminal needs to be closed and reopened
    How to open Terminal

    Mac: press ⌘ + Space, type Terminal, press Enter

    Windows: open Start, type Windows PowerShell, then Open

    Not installed: Open the official download page, pick the LTS version and the click-through installer: .pkg on Mac · .msi on Windows. Double-click and follow it to the end. No curl command needed

    Open the Node.js download page ↗

    When it’s done, close Terminal, open it again and run the same command. You should see a version number: node -v

    Install once; it works for everything that needs Node.js

  2. Open Terminal

    Mac: press Command+Space and type Terminal · Windows: open PowerShell

  3. Paste this command and press Enter

    claude mcp add --scope user mtscraper -- npx -y mtscraper --mcp
  4. Type /mcp in Claude Code

    You should see MTScraper listed as connected

  5. Try it

    Use MTScraper and tell me what you can do for me, then show me one thing

    ✓ If Claude Code says it is using MTScraper or shows a MTScraper tool in the answer, you are connected

Menu names checked against Anthropic’s help page on Nov 11, 2026 · your screen may differ slightly by app version

Works with Cursor, with some setup first

Install Node.js on your computer first (one time)

Let AI install it

Copy this message into a Cursor chat. The AI checks your computer, tells you which file it will change, and waits for your OK every time.

Please install the MCP server MTScraper for Cursor on this computer, in this order: 1) Check Node.js with node -v. If it is missing, tell me to install it from https://nodejs.org/en/download; do not install it yourself. 2) Use only this config (already checked by PopNx MCP): { "mcpServers": { "mtscraper": { "command": "npx", "args": [ "-y", "mtscraper", "--mcp" ] } } } 3) Tell me the full path of the file you will change, or the command you will run, and wait for my OK. 4) No key is needed. 5) Do what we agreed, then check that the MTScraper tools really show up.

This message has no key in it. If a key is needed, the AI will ask you to paste it yourself into the file or the app’s field.

  1. Install Node.js (if you don’t have it)

    Already installed? Check first · Open Terminal, type this command and press Enter

    node -v
    • A version number such as v26.13.1 = installed, skip this step
    • "command not found" (on Windows: "is not recognized") = not installed yet, or installed but Terminal needs to be closed and reopened
    How to open Terminal

    Mac: press ⌘ + Space, type Terminal, press Enter

    Windows: open Start, type Windows PowerShell, then Open

    Not installed: Open the official download page, pick the LTS version and the click-through installer: .pkg on Mac · .msi on Windows. Double-click and follow it to the end. No curl command needed

    Open the Node.js download page ↗

    When it’s done, close Terminal, open it again and run the same command. You should see a version number: node -v

    Install once; it works for everything that needs Node.js

  2. Click this button to open Cursor

    Open in Cursor

    Cursor asks you to confirm; click Install

  3. Open the AI chat in Cursor in Agent mode

  4. Try it

    Use MTScraper and tell me what you can do for me, then show me one thing

    ✓ If Cursor says it is using MTScraper or shows a MTScraper tool in the answer, you are connected

Menu names checked against Cursor’s help page on Nov 11, 2026 · your screen may differ slightly by app version

Works with VS Code, with some setup first

Install Node.js on your computer first (one time)

Let AI install it

Copy this message into a VS Code chat. The AI checks your computer, tells you which file it will change, and waits for your OK every time.

Please install the MCP server MTScraper for VS Code on this computer, in this order: 1) Check Node.js with node -v. If it is missing, tell me to install it from https://nodejs.org/en/download; do not install it yourself. 2) Use only this config (already checked by PopNx MCP): { "servers": { "mtscraper": { "type": "stdio", "command": "npx", "args": [ "-y", "mtscraper", "--mcp" ] } } } 3) Tell me the full path of the file you will change, or the command you will run, and wait for my OK. 4) No key is needed. 5) Do what we agreed, then check that the MTScraper tools really show up.

This message has no key in it. If a key is needed, the AI will ask you to paste it yourself into the file or the app’s field.

  1. Install Node.js (if you don’t have it)

    Already installed? Check first · Open Terminal, type this command and press Enter

    node -v
    • A version number such as v26.13.1 = installed, skip this step
    • "command not found" (on Windows: "is not recognized") = not installed yet, or installed but Terminal needs to be closed and reopened
    How to open Terminal

    Mac: press ⌘ + Space, type Terminal, press Enter

    Windows: open Start, type Windows PowerShell, then Open

    Not installed: Open the official download page, pick the LTS version and the click-through installer: .pkg on Mac · .msi on Windows. Double-click and follow it to the end. No curl command needed

    Open the Node.js download page ↗

    When it’s done, close Terminal, open it again and run the same command. You should see a version number: node -v

    Install once; it works for everything that needs Node.js

  2. Click this button to open VS Code

    Open in VS Code

    VS Code asks you to confirm; click Install

  3. Open the AI chat in VS Code in Agent mode

  4. Try it

    Use MTScraper and tell me what you can do for me, then show me one thing

    ✓ If VS Code says it is using MTScraper or shows a MTScraper tool in the answer, you are connected

Menu names checked against VS Code’s help page on Nov 11, 2026 · your screen may differ slightly by app version

Other apps that use MCP, such as Windsurf, Zed or Gemini CLI, each add servers differently. See how in the app list.

See every app that uses MCP →

JSON settings are under “For developers” below

Stuck? Ask people who use MTScraper💬 Discussion →
For developers: Terminal commands and JSON settings

Installed on your computer

Claude Code

claude mcp add --scope user mtscraper -- npx -y mtscraper --mcp

Claude Desktop (claude_desktop_config.json)

{
  "mcpServers": {
    "mtscraper": {
      "command": "npx",
      "args": [
        "-y",
        "mtscraper",
        "--mcp"
      ]
    }
  }
}

Cursor (~/.cursor/mcp.json)

{
  "mcpServers": {
    "mtscraper": {
      "command": "npx",
      "args": [
        "-y",
        "mtscraper",
        "--mcp"
      ]
    }
  }
}

VS Code (mcp.json)

{
  "servers": {
    "mtscraper": {
      "type": "stdio",
      "command": "npx",
      "args": [
        "-y",
        "mtscraper",
        "--mcp"
      ]
    }
  }
}

These settings are generated from the maker’s registry entry. If the README says otherwise, follow the README.

At a glance

Downloads per weeki
Not on npm/PyPI
Connection
Runs on your computer
Latest versioni
2.1.1
Published to registryi
Nov 15, 2026
Code last changedi
Nov 15, 2026
Maker
mazprotech
Packages
npm: mtscraper
Registry status
Active

Which apps can use it

Based on how this server connects and on each app’s own docs, checked Nov 10, 2026.

Run on your computer in

ClaudeGooseLM StudioMsty StudioCherry StudioCursorVS Code + GitHub CopilotVisual StudioWindsurf (Devin Desktop)ZedKiroClineContinueJetBrains AI AssistantClaude CodeCodex CLIGemini CLIOpenCodenodecodeWarpLibreChatRaycast

All apps, with pros and cons →

Related guides

What it can do

Runs on your computer; the tool list appears in your AI app after you install it.

Basic checksi

We check information anyone can see; we have not read the code. Checks that do not apply are left out of the count. · Checked Nov 19, 2026

  • ✓ Pass

    Maker matches the repo

    The registry name was verified with the GitHub account mazprotech, which owns the repo

  • ✓ Pass

    Still maintained

    Code last changed 4 days ago

  • ✓ Pass

    Has a licence

    Uses MIT

  • – Not applicable

    Endpoint responds

    This one runs on your computer; there is no internet endpoint

  • – Not applicable

    Sign-in with OAuth

    Runs on your computer and uses local settings instead

  • … Not checked yet

    No hidden instructions in tool descriptions

    Runs on your computer; we can only read tool lists from internet endpoints

Published vulnerabilities (OSV.dev)

No entries on OSV.dev for mtscraper@2.1.1

Checks only this package version, not every dependency · checked Nov 16, 2026

What we check → · JSON

Makers: ask for a re-check or report wrong info
For makers: add a badge to your README

If you made this, paste this code into your README. The badge links back to this page.

[![Featured on PopNx MCP](https://mcp.popnx.com/badge/mazprotech-mtscraper.svg)](https://mcp.popnx.com/en/servers/mazprotech-mtscraper)

Discussion

Ask, share a tip or report that it does not work. Sorted by what people marked helpful.

No posts yet. Be the first to ask or share.

Posts here are written by users, not the PopNx team.

Sign in to post

User reviews

No reviews yet

Sign in to write a review

Compared with others in Web & browsersi

  • More stars than 0% of the other 1613 servers in this category
  • Published to the registry more recently than 88% of the other 1613 servers in this category
  • Basic checks equal to or better than 99% of the other 1613 servers in this category

Compared with every active server in this category in the registry. Stars count only when the publisher owns the repo.

Common questions

Is MTScraper safe?

It passed 3 of 3 applicable basic checks. We have not read the code. Before adding a key or token, see what access the server asks for and use a key with only the permissions you need.

What keys or codes does it need?

The registry does not list a required key, though some servers ask you to sign in on first use.

Does it run on my computer or in the cloud?

It runs on your computer. Your AI app starts it when needed, and data does not pass through the maker’s servers (unless it calls an online service itself).

Is it still maintained?

The GitHub repo last changed 4 days ago, and the latest version (2.1.1) reached the registry 4 days ago.

Which apps can use it?

Apps that support MCP, such as Claude Desktop, Claude Code, Cursor and VS Code. You also need Node.js on your computer.

What can I use instead?

In the same category: Browser Use, changedetection.io (official), Apify MCP Server, Pomerado, Actionbook

Similar alternatives

Same category, other makers, commands already checked. Official ones first, then by stars.

Reading a card:↓ Downloads per weeki★ GitHub starsiThree pillars: reviews · checks · usageiUpdated = latest version published to the registryi

Browser Use

Lets AI do web tasks for you, such as filling forms, searching and extracting data, in the maker’s cloud browsers or on your computer.

by browser-useWidely used↓ 1.7M/wk★ 121.4KUpdated 29 days ago

Computer or cloud✓ service owner · browser-use.com

Apify MCP Server

Extract data from any website with thousands of scrapers, crawlers, and automations on Apify Store ⚡

by apifyWidely used★ 16.9KUpdated 2 days ago

Runs in the cloud✓ service owner · apify.com

Pomerado

Connect your agents to any website. The cheaper, faster and more reliable alternative to browser use

by PomeradoGrowing★ 1.8KUpdated yesterday

Runs in the cloud✓ service owner · pomerado.ai

Actionbook

Let your AI agent operate any website through a real, logged-in browser session.

by actionbookGrowing★ 1.6KUpdated 31 days ago

Runs in the cloud✓ service owner · actionbook.app
Compared with similar ones
This one
MTScraper
Browser Usechangedetection.io (official)Apify MCP ServerPomeradoActionbook
Runs whereInstalls on your computerCloud or install on your computerRuns in the cloud, nothing to installRuns in the cloud, nothing to installRuns in the cloud, nothing to installRuns in the cloud, nothing to install
Needs a keyNoYesNoYesNoNo
Add with + in ClaudeNoYesNoNoNoNo
Basic checks3/35/53/34/44/44/4
LicenseMITMITApache-2.0MITMITApache-2.0
PriceNo cost listed in the registryNo cost listed in the registryNo cost listed in the registryNo cost listed in the registryNo cost listed in the registryNo cost listed in the registry

README from the maker

Open the README

Copied from the maker’s repo as written. PopNx MCP has not reviewed or endorsed it. HTML, images and scripts are removed. Links lead off this site.

MTScraper

Free website crawler for AI coding agents, by MazTech.

Ask your agent to *"scrape example.com"*. MTScraper crawls the site, follows its internal links, and saves each page's clean main text (no menus or footers) to JSONL or CSV. Linked PDFs are read too, text and tables. Then your agent works with the data for you.

Free, open source, no credits, no page caps, no API key, runs on your PC.

  • Clean text: main content only, extracted with Mozilla Readability
  • Markdown when you want it: headings, lists, links and tables kept (--format markdown)
  • One page now: read hands back a single page's content without saving anything
  • Look before you crawl: map lists a site's pages from its sitemap without downloading them
  • Linked PDFs: text and tables extracted by default (--no-pdf to skip)
  • JavaScript sites: optional headless Chromium rendering (--js)
  • One-page apps: clicks through tabs and buttons and saves every view (--click)
  • Resumable: stop a big crawl and pick it up later (--resume)
  • Polite: obeys robots.txt and stays on the target domain
  • One record format for everything: url, title, view, type, text, tables
  • Node.js only, nothing to install: runs through npx

Why MTScraper

  • Runs on your machine. Pages and PDFs are never uploaded anywhere.
  • No account, no API key, no per-page credits. A crawl of 5 pages or 5,000 costs the same: nothing.
  • One record format for HTML pages and PDFs, so the data is ready for a database, a spreadsheet or an embedding index without cleanup.
  • Works inside the agents people already use. Claude Code, Codex, Gemini CLI, Cursor and Copilot pick it up as a skill or an MCP server.
  • Polite by default: robots.txt obeyed, four requests at a time, stays on the target domain.
  • A few hundred lines of plain JavaScript you can read in one sitting.

How it works

  1. The start URLs go into a Crawlee crawler: robots.txt is checked, links on the same domain are followed, failed requests are retried, and at most four requests run at once.
  2. Each HTML page goes through Mozilla Readability, which keeps the main content and drops menus, footers and ads.
  3. Each linked PDF is downloaded and read with pdf.js. Lines are rebuilt from the text positions, and lines whose cells line up become tables.
  4. Every page becomes one record with the same six fields, appended to a JSONL or CSV file as it arrives.
  5. With --js, pages are rendered in headless Chromium first. With --click, each matching tab or button is clicked and every new view is saved as its own record.
  6. With --format markdown, the main content is turned into Markdown instead of plain text, and PDF tables become Markdown tables.
  7. read is the same crawler limited to one page, with the result handed back instead of written to a file. map reads the site's sitemap (found through robots.txt, or at /sitemap.xml), or else the links on the start page, and downloads nothing else.
  8. With --mcp, all of this is exposed as three MCP tools, crawl, read and map, over stdio.

Requirements

  • Node.js 20 or newer
  • An AI agent that can run skills or MCP servers (see Install), or just a terminal
  • Chromium, once, only for --js, --click and buttons: npx -y github:mazprotech/mtscraper setup

Install / where it works

MTScraper reaches your agent in two ways, and both run the same tool through npx -y github:mazprotech/mtscraper:

  • Skill: a folder (skills/mtscraper) with instructions the agent reads. The agent runs the CLI itself.
  • MCP server: your app starts npx -y github:mazprotech/mtscraper --mcp and gets three tools: crawl, read and map.

To copy the skill folder, clone the repo once and copy skills/mtscraper into the skills folder listed for your tool below (~ is your home folder; in PowerShell use $HOME).

git clone https://github.com/mazprotech/mtscraper.git

mkdir -p ~/.claude/skills && cp -r mtscraper/skills/mtscraper ~/.claude/skills/ # example: Claude Code

```powershell

git clone https://github.com/mazprotech/mtscraper.git New-Item -ItemType Directory -Force "$HOME\.claude\skills" | Out-Null # example: Claude Code Copy-Item -Recurse -Force mtscraper\skills\mtscraper "$HOME\.claude\skills\"

Every MCP client needs the same values: command npx, arguments -y github:mazprotech/mtscraper --mcp. In the clients that use an mcpServers JSON file it looks like this:

{

"mcpServers": { "mtscraper": { "command": "npx", "args": ["-y", "github:mazprotech/mtscraper", "--mcp"] } } }

Claude Code

Plugin (recommended). Inside Claude Code, run:

/plugin marketplace add mazprotech/mtscraper

/plugin install mtscraper@maztech

Or copy the skill folder to `~/.claude/skills/` (all projects) or `.claude/skills/` (one project). Restart Claude Code, or run `/reload-plugins` after a plugin install.
Claude Desktop

MCP only. Open Settings, Developer, Edit Config, or edit the file directly:

  • macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
  • Windows: %APPDATA%\Claude\claude_desktop_config.json

Add the mcpServers block above (merge it with any servers you already have), then fully quit and restart Claude Desktop.

OpenAI Codex (the "ChatGPT" option)

"Works with ChatGPT" here means Codex: the Codex CLI, the Codex IDE extension and the ChatGPT desktop app. The ChatGPT chat app in the browser or on your phone cannot run local tools, so it cannot use MTScraper.

  • Skill: copy the folder to ~/.agents/skills/ (all projects) or .agents/skills/ (one repo).
  • MCP: codex mcp add mtscraper -- npx -y github:mazprotech/mtscraper --mcp. This writes ~/.codex/config.toml, which the CLI, IDE extension and desktop app share. Or add it by hand:
  [mcp_servers.mtscraper]

command = "npx" args = ["-y", "github:mazprotech/mtscraper", "--mcp"]

Gemini CLI
  • Skill: copy the folder to ~/.gemini/skills/ (or ~/.agents/skills/), or to .gemini/skills/ (or .agents/skills/) in a project.
  • MCP: add the mcpServers block above to ~/.gemini/settings.json (all projects) or .gemini/settings.json (one project).
Cursor
  • Skill: copy the folder to ~/.cursor/skills/ (or ~/.agents/skills/), or to .cursor/skills/ (or .agents/skills/) in a project.
  • MCP: add the mcpServers block above to ~/.cursor/mcp.json (all projects) or .cursor/mcp.json (one project).
GitHub Copilot CLI
  • Skill: copy the folder to ~/.copilot/skills (or ~/.agents/skills), or to .github/skills, .claude/skills or .agents/skills in a repo.
  • MCP: copilot mcp add mtscraper -- npx -y github:mazprotech/mtscraper --mcp. This writes ~/.copilot/mcp-config.json.
VS Code
  • Skill: copy the folder to ~/.copilot/skills/, ~/.claude/skills/ or ~/.agents/skills/, or to .github/skills/, .claude/skills/ or .agents/skills/ in a project.
  • MCP: create .vscode/mcp.json in your workspace. VS Code uses servers instead of mcpServers:
  {

"servers": { "mtscraper": { "type": "stdio", "command": "npx", "args": ["-y", "github:mazprotech/mtscraper", "--mcp"] } } }

Windsurf

See Windsurf's docs for where skills and MCP servers are configured. Use the command and arguments above for the MCP server.

Any other MCP client

Register a stdio server with command npx and arguments -y github:mazprotech/mtscraper --mcp, in whatever format that client uses. See your client's docs.

Restart the app after installing so it picks up the skill or server.

Use

Just ask your agent in plain language:

  • *"Scrape the docs at example.com/docs, max 100 pages"*
  • *"Pull every blog post from site.com into a CSV"*
  • *"Crawl this competitor's site and summarize their services"*
  • *"Get the price tables out of the PDFs on example.com/downloads"*
  • *"Read this page and summarize it: example.com/pricing"*
  • *"How many pages does example.com have? List them first"*

With the skill, crawl results are saved to a scraped/ folder in your current project. With the MCP server:

ToolInputsGives back
crawlurl, max_pages (default 50), follow, js, click, pdf, format (text default, or markdown)The records inline (up to about 200,000 characters) and the path of a JSONL file that holds everything
readurl, format (markdown default, or text), jsOne page's title, address and clean content, in the reply. No file is saved
mapurl, max_pages (default 50)The list of the site's page addresses, where it came from (sitemap or links), and whether the list was cut at max_pages

All three obey robots.txt. read says so when robots.txt refuses a page.

Run it yourself

No agent needed:

npx -y github:mazprotech/mtscraper https://example.com --max-pages 200

npx -y github:mazprotech/mtscraper https://example.com -o pages.csv npx -y github:mazprotech/mtscraper https://spa.example.com --js npx -y github:mazprotech/mtscraper https://example.com/docs --format markdown npx -y github:mazprotech/mtscraper read https://example.com/pricing npx -y github:mazprotech/mtscraper map https://example.com -n 200

read prints one page's content (Markdown unless --format text) and saves nothing. map prints one page address per line, and a summary such as found 120 pages (from sitemap).

One-page app with tabs (needs Chromium, see Requirements). List the clickable elements first to pick a selector, then click only the view-switching ones:

npx -y github:mazprotech/mtscraper buttons https://app.example.com

npx -y github:mazprotech/mtscraper https://app.example.com --click "[role=tab]" --max-pages 1 -o views.jsonl

Options

mtscraper <url> [url...] [options]

mtscraper read print one page's clean content; saves nothing mtscraper map list a site's pages without downloading them mtscraper buttons list clickable elements, to pick a --click selector mtscraper setup install the Chromium build that --js, --click and buttons need mtscraper --mcp run as an MCP server over stdio (tools: crawl, read, map)

OptionMeaning
-o, --out Output .jsonl or .csv (default scraped/ .jsonl; appends if the file exists)
-n, --max-pages Stop after N pages (default 1000, 0 = no cap). PDFs count as pages. Also caps how many pages map lists
--format text or markdown (keeps headings, lists, links and tables). Default text; read defaults to markdown
-f, --follow Only follow links whose URL contains this text, e.g. /blog/ (in Git Bash on Windows, run with MSYS_NO_PATHCONV=1 so /blog/ is not rewritten as a path)
--jsRender pages in headless Chromium for JavaScript sites (5-10x slower)
--click Click each matching button or tab and save every view (implies --js)
--wait Wait after each click for the view to update (default 1500)
--no-pdfSkip linked PDFs (by default their text and tables are extracted too)
--resume Resumable: re-run the same command with the same name after Ctrl+C
-q, --quietOnly print the summary
-h, --helpShow the help

When the run ends it prints a summary line: done: N records -> file (finished), or max_pages reached, plus the number of failed pages if any.

Examples

  • [

Only the beginning is shown; read the rest in the repo