LionScraper MCP + CLI + HTTP API (Python)

LionScraper MCP + CLI + API service (Python)

简体中文

• Website: lionscraper.com
• PyPI: project lionscraper

What is this?

LionScraper is a browser extension that can collect lists, articles, links, images, and more from web pages. This PyPI package provides the companion bridge to that extension in three ways:

• MCP (lionscraper-mcp): connect your AI app (e.g. Cursor) so the model can call scraping tools in chat.
• CLI (lionscraper): run daemon, scrape, and ping from a terminal over the same local HTTP/WebSocket port as the extension.
• HTTP API: call the same tools over loopback JSON HTTP (/v1/...) when the daemon is running—useful for scripts, services, or any HTTP client without MCP or the CLI front-end.

The real scraping logic runs in the extension; this package connects and forwards.

Before you start

1. Browser: Chrome or Edge (follow what the extension actually supports).
2. LionScraper extension: Install and enable it from your browser’s store (the listing title may vary by storefront).

• Chrome: Chrome Web Store — LionScraper
• Microsoft Edge: Edge Add-ons — LionScraper

1. Python: 3.10 or newer on your machine. If you do not have it yet, download an installer from the Python website and follow the prompts (check the option to add Python to PATH on Windows if the installer offers it).
2. For MCP: an AI app that supports MCP (e.g. Cursor, Trae).
3. For the HTTP API: same browser, extension, and daemon as the CLI; use curl, fetch, or any HTTP client against http://127.0.0.1:$PORT (see HTTP API (local REST) below).

This package uses aiohttp for outbound HTTP and WebSocket probes to the daemon. When Chrome/Edge are not detected and the extension is not connected, ping / scrape* can use a server-side HTTP fallback (no in-page JS).

Install (pip)

This package is published on PyPI as [lionscraper](https://pypi.org/project/lionscraper/).

pip install -U lionscraper

You may use a virtual environment (recommended) or pip install -U --user lionscraper if you prefer not to install into the system interpreter.

You get two commands; together they support three integration styles (MCP, CLI, HTTP API):

If lionscraper-mcp is not on your PATH after install, use the full path to the script (e.g. under your venv’s Scripts/ or bin/), or the `python -m lionscraper` form below in MCP (still requires a normal pip install -U lionscraper).

---

MCP (AI applications)

Add MCP in your AI app

Examples assume `lionscraper-mcp` is on your PATH (UIs differ). In MCP JSON, every `env` value is a string.

Minimal config (omit env for built-in defaults; `PORT` defaults to 13808 and must match the extension bridge port):

{

"mcpServers": { "lionscraper": { "command": "lionscraper-mcp" } } }

Full `env` example (drop keys you do not need; empty strings behave like “unset” for most of these):

{

"mcpServers": { "lionscraper": { "command": "lionscraper-mcp", "env": { "PORT": "13808", "TIMEOUT": "120000", "LANG": "en-US", "TOKEN": "", "DAEMON": "" } } } }

• `PORT`: HTTP + WebSocket listen port; default 13808; must match the extension bridge port.
• `TIMEOUT`: Milliseconds to wait for a previous instance to release the port before forcing takeover; default 120000; `0` means force quickly.
• `LANG`: Tool descriptions and stderr log language (`en-US`, `zh-CN`, or POSIX forms like en_US.UTF-8).
• `TOKEN`: Bearer token shared with the daemon; empty means no Authorization header.
• `DAEMON`: Only `0` disables auto-spawning lionscraper daemon from thin MCP; empty or other values match omitting the key (auto-start allowed).

Restart MCP or the app so the config applies.

If lionscraper-mcp is not on PATH

After `pip install -U lionscraper`, you can run the same routing with `python -m lionscraper`: no arguments after the module name selects thin MCP over stdio; any extra argument (including --debug) uses the CLI instead. Example:

{

"mcpServers": { "lionscraper": { "command": "python", "args": ["-m", "lionscraper"] } } }

Use the same `python` executable you used to install the package (or python3 on some systems).

Match the port in the browser extension

1. Open LionScraper settings / options.
2. Set bridge port to the same value as `PORT` (e.g. 13808).
3. If needed, use Reconnect in the extension, or reload the extension / restart the browser.

Day-to-day use (MCP)

1. Keep the extension enabled and target pages open as required.
2. Ask in natural language, e.g. “Check if LionScraper is connected” or “Scrape lists / article / emails / phones / links / images from this page.”
3. If you see not connected or timeout, retry a connection check and confirm PORT matches.

MCP tools (summary)

The server registers tools that mirror extension capabilities. Names and shapes are what your MCP client shows to the model.

Parameters: A full JSON schema for every field belongs in the tool definitions your client displays and in release-accurate docs; duplicating it here goes stale quickly. Use the tool list / descriptions in the AI app when calling MCP.

MCP Resources / Prompts

The thin MCP process (lionscraper-mcp) exposes Resources and Prompts in addition to Tools:

• Resources: static Markdown at stable URIs, e.g. lionscraper://guide/connection (PORT alignment, ping troubleshooting), lionscraper://guide/when-to-use-tools (prefer LionScraper over WebFetch/curl/wget by scenario), lionscraper://guide/cli (terminal CLI), lionscraper://reference/tools, lionscraper://reference/common-params. Clients list/read them into context; they are served inside the stdio process and do not require the daemon HTTP path (works even if the extension is offline). For the loopback HTTP control plane (/v1/...), see HTTP API (local REST) below (not an MCP resource URI).
• Prompts: workflow templates (e.g. ping-then-scrape, multi-URL, scrape_article, prefer_lionscraper_scraping, extension troubleshooting). Clients list/get prompts; UI varies by host (Cursor, Trae, …).

Copy follows `LANG` (e.g. zh-CN), same as tool metadata.

---

CLI (terminal)

The `lionscraper` command is the terminal front-end to the same stack as MCP: `lionscraper daemon` listens on `PORT` (default 13808) for HTTP (used by the CLI and by the thin lionscraper-mcp process) and WebSocket (used by the extension). Set `PORT` (and optional `TOKEN`) to match the extension bridge port and any MCP config. Use the CLI for scripts, CI, or quick one-off runs without opening an AI chat.

The CLI talks to the daemon HTTP API on http://127.0.0.1:$PORT (default port 13808, same as the extension). If you do not pass --api-url, a local daemon is auto-started when possible when you run scrape or ping.

Common commands:

lionscraper --help

lionscraper daemon # keep running; HTTP + WebSocket on PORT lionscraper stop # stop daemon on configured PORT lionscraper ping lionscraper scrape -u https://www.example.com lionscraper scrape --method article -u https://www.example.com

Shorthand: lionscraper -u https://www.example.com → same as scrape

`--method` selects which tool the daemon runs (default scrape): scrape, article, emails, phones, urls, images. Repeat `-u` / `--url` to pass several URLs in one run.

Set `PORT` (and optional `TOKEN`) in the environment so the CLI matches the extension and MCP. Use `--api-url http://127.0.0.1:PORT` if the daemon is not on the default base URL. Run `lionscraper --help` for every flag the binary accepts.

Scrape: parameters and richer examples

Below flags are forwarded to the extension (except `--bridge-timeout-ms`, which only caps how long the CLI waits on the bridge). Actual behavior still depends on the extension version.

Output and connection

Timing and load

Lazy-loaded / infinite scroll (maps to waitForScroll in the tool payload)

Locale

Extra payload

Method-specific filters (examples; see --help for the full set)

• Emails (--method emails): --email-domain, --email-keyword, --email-limit
• Phones (--method phones): --phone-type, --phone-area-code, --phone-keyword, --phone-limit
• URLs (--method urls): --url-domain, --url-keyword, --url-pattern, --url-limit
• Images (--method images): --img-min-width, --img-min-height, --img-format, --img-keyword, --img-limit

Optional browser automation hints (when the extension supports them): --auto-launch-browser, --no-auto-launch-browser, --post-launch-wait-ms.

---

Example A — List / table scrape with pagination, delays, and scroll assist

lionscraper scrape \

-u https://www.example.com/items \ --max-pages 5 \ --delay 800 \ --timeout-ms 90000 \ --bridge-timeout-ms 180000 \ --wait-scroll-speed 400 \ --wait-scroll-interval 350 \