Atlas Web Compiler & Scraper

@atlascompiler/mcp

The official Model Context Protocol (MCP) server for Atlas.

Connects AI coding assistants, agents, and LLMs (Cursor, Claude Desktop, Windsurf, ChatGPT, MCP Inspector) directly to Atlas's zero-LLM web compilation pipeline. Provides deterministic, clean markdown extraction, site topology discovery, batch compilation, and bounded crawling without LLM hallucinations or context bloat.

---

Features

• 0-LLM Deterministic Compilation: Content is parsed and structured purely through the Atlas web compiler engine—no hallucinated summaries, no injected noise, and no missing data.
• Context-Window Protection: Automatic Table of Contents (TOC), section splitting, token budgeting, and explicit truncation banners protect LLM context limits.
• Modern Protocol Standard: Implements the 2026-07-28 protocol era with native prompt caching hints (cacheHints: { ttlMs, cacheScope }) and asynchronous task tracking via io.modelcontextprotocol/tasks.
• Dual Transports:
• Stdio CLI Bridge: Run locally with npx @atlascompiler/mcp or configure directly into desktop IDEs.
• Edge Streamable HTTP: Native Cloudflare Worker endpoint mounted at POST /mcp for remote clients and web agents.
• Enterprise-Grade Security:
• Rejects CLI credential flags (--api-key, -k) with exit code 1 to eliminate secret exposure in ps aux and process inspection.
• Strict tenant isolation (WHERE workspace_id = ?1 AND id = ?2) on all atlas:// resources.
• Hostile Origin blocking prevents browser-based cross-origin drive-by attacks.
• All outbound network requests are delegated to atlas-egress for zero-trust SSRF protection.

---

Quick Start

Get your API key

Sign in to the Atlas Dashboard to generate your free API key (atlas_...).

Interactive setup wizard (Recommended)

Run the 1-click interactive setup wizard to validate your API key and automatically configure your desktop editors (Claude Desktop, Cursor, VS Code, Windsurf):

# Interactive mode (prompts for scope: Global, Project, or Both):

npx -y @atlascompiler/mcp init

Or preselect scope directly via CLI flags:

npx -y @atlascompiler/mcp init --global # User-level (Claude, ~/.cursor, Windsurf) npx -y @atlascompiler/mcp init --project # Repo-level (.cursor/mcp.json, .vscode/mcp.json) npx -y @atlascompiler/mcp init --both # Both user-level and project-level npx -y @atlascompiler/mcp init --skip-editors # Save credentials only

The wizard:

• Prompts for your Atlas API key (atlas_...).
• Stores it securely in ~/.atlas/credentials.json (POSIX 0600 permissions).
• Detects installed editors and merges the Atlas MCP server definition non-destructively without hardcoding sensitive tokens into editor configuration files.
• Distinguishes standard clients (mcpServers) from VS Code workspace format (servers).
• Optionally installs the canonical Atlas Agent Skill for local or global agent discovery.

Run Stdio server directly

Once configured via init, you can run the MCP server with zero flags or environment variables:

npx -y @atlascompiler/mcp

Headless & CI/CD environments

In automated pipelines, containerized environments, or ephemeral workers, supply credentials via the ATLAS_API_KEY environment variable:

# macOS / Linux

export ATLAS_API_KEY="atlas_your_api_key_here"

Windows PowerShell

$env:ATLAS_API_KEY = "atlas_your_api_key_here"

npx -y @atlascompiler/mcp

Security Note: Passing API keys via CLI flags (--api-key or -k) is strictly prohibited and blocked by the CLI to avoid process table inspection leaks. Always use init or the ATLAS_API_KEY environment variable.

---

CLI Commands

---

Agent Skill Installation

Atlas publishes an official Agent Skill for AI coding agents (Claude Code, Cursor, Antigravity, Windsurf) providing progressive disclosure playbooks for web compilation, scraping, mapping, crawling, and batch extraction.

Option 1: Via @atlascompiler/mcp CLI (Fastest)

# In your current project:

npx -y @atlascompiler/mcp skill

Or globally for all projects:

npx -y @atlascompiler/mcp skill --global

Option 2: Via Skills Ecosystem CLI (npx skills)

# Via domain discovery (RFC 8615):

npx skills add atlas-compiler.com

Or via GitHub repository:

npx skills add redcliffe0003/Atlas

Option 3: Direct Download / Curl

curl -fsSL https://atlas-compiler.com/skill.md -o .agents/skills/atlas/SKILL.md

---

Client Setup Configurations

You can configure your editor in two ways:

1. Interactive Setup Wizard (Recommended): Run npx -y @atlascompiler/mcp init to store credentials securely in ~/.atlas/credentials.json (POSIX 0600). Editor config files remain clean with no tokens checked into git.
2. Direct Manual Configuration: Add the server block below with your ATLAS_API_KEY directly in your editor's MCP config file.

Cursor IDE

Add the server to your Cursor MCP settings (~/.cursor/mcp.json for global, or .cursor/mcp.json in your repository):

{

"mcpServers": { "atlas": { "command": "npx", "args": ["-y", "@atlascompiler/mcp"], "env": { "ATLAS_API_KEY": "atlas_your_api_key_here" } } } }

*(If you already ran `atlas-mcp init`, the `"env"` block can be omitted.)*

VS Code

Add the server to your project's .vscode/mcp.json:

{

"servers": { "atlas": { "command": "npx", "args": ["-y", "@atlascompiler/mcp"], "env": { "ATLAS_API_KEY": "atlas_your_api_key_here" } } } }

Note: VS Code expects the root key "servers", whereas Cursor, Claude Desktop, and Windsurf use "mcpServers". The atlas-mcp init wizard automatically applies the correct format for each editor target.

Claude Desktop

Add the server to your Claude Desktop configuration:

• macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
• Windows: %APPDATA%\Claude\claude_desktop_config.json
• Linux: ~/.config/Claude/claude_desktop_config.json

{

"mcpServers": { "atlas": { "command": "npx", "args": ["-y", "@atlascompiler/mcp"], "env": { "ATLAS_API_KEY": "atlas_your_api_key_here" } } } }

Windsurf / Codeium

Add the server to ~/.codeium/windsurf/mcp_config.json:

{

"mcpServers": { "atlas": { "command": "npx", "args": ["-y", "@atlascompiler/mcp"], "env": { "ATLAS_API_KEY": "atlas_your_api_key_here" } } } }

Remote / Hosted Streamable HTTP (POST /mcp)

For remote agents or the official MCP Inspector:

• Server URL: https://api.atlas-compiler.com/mcp
• Transport: Streamable HTTP
• HTTP Headers:
• Authorization: Bearer atlas_YOUR_API_KEY_HERE
• Content-Type: application/json
• Accept: application/json
• OAuth Discovery: Supports RFC 8414 protected resource discovery at GET /.well-known/oauth-protected-resource.

---

Available MCP Tools

Atlas exposes 5 focused tools for AI coding assistants and agent workflows:

| # | Tool | Description | Best For | | :-: | :--- | :--- | :--- | | 1 | atlas_compile | Deep 12-pass AST compilation into high-density Markdown | Complex docs, API references, tables, and code blocks | | 2 | atlas_scrape | Fast single-page content and link extraction | Articles, blog posts, news, and raw text | | 3 | atlas_map | Domain topology spider and sitemap explorer | Discovering reachable URLs without downloading page bodies | | 4 | atlas_batch_compile | Concurrent parallel compilation (up to 20 URLs) | Multi-page comparisons and batch data ingestion | | 5 | atlas_crawl | Bounded recursive website crawl with path glob filtering | Section-specific site crawls (e.g. /docs/**) |

---

atlas_compile

Compile any web page into a structured, high-density markdown document optimized for LLM reasoning.

{

url: string; // Required: Absolute HTTP/HTTPS URL format?: 'markdown' | 'semantic' | 'markdown+links'; // 'semantic' includes AST graph; 'markdown+links' appends links max_output_chars?: number; // Maximum characters to return in the tool response (default: 30000) force_fresh?: boolean; // Bypass edge cache and force fresh compile from origin (default: false) }

Example Agent Invocation:

*"Compile the Stripe payment intents documentation at https://docs.stripe.com/api/payment_intents and show me the API parameters."*

---

atlas_scrape

Fast, lightweight single-page extraction when deep compilation isn't required.

{

url: string; // Required: Absolute HTTP/HTTPS URL formats?: Array<'markdown' | 'links' | 'text'>; // Formats to extract (default: ['markdown']) max_output_chars?: number; // Capping threshold (default: 30000) timeout_ms?: number; // Maximum fetch timeout in milliseconds (default: 15000) }

Example Agent Invocation:

*"Scrape the main text and links from https://atlas-compiler.com."*

---

atlas_map

Discover site topology, sitemaps, and reachable URLs on a target domain without downloading page bodies.

{

url: string; // Required: Root URL or domain (e.g. 'https://docs.github.com') limit?: number; // Maximum URLs to return (1-500, default: 100) include_subdomains?: boolean; // Follow subdomains (default: false) sitemap?: 'include' | 'skip' | 'only'; // Sitemap inspection strategy (default: 'include') }

Example Agent Invocation:

*"Map the documentation pages on https://docs.github.com so we know what guides are available."*

---

atlas_batch_compile

Compile multiple pages in parallel (up to 20 URLs) with per-document success/failure aggregation.

{

urls: string[]; // Required: Array of 1 to 20 URLs format?: 'markdown' | 'semantic' | 'markdown+links'; // Content format (default: 'markdown') max_output_chars_per_page?: number; // Character cap per document (default: 15000) }

Example Agent Invocation:

*"Compile these 3 release note pages simultaneously: ['https://site.com/v1', 'https://site.com/v2', 'https://site.com/v3']."*

---

atlas_crawl

Bounded crawl of a website starting from a seed URL with link discovery, depth limits, and path filtering.

{

url: string; // Required: Seed URL to begin crawling max_pages?: number; // Maximum pages to process (1-50, default: 10) max_depth?: num