methods-mcp

methods-mcp

Lightweight, on-demand MCP server for structured methods extraction + reproducibility heuristics on academic papers. Built for the Worldwide AI Science Fellowship build challenge.

⚠️ Status: alpha (0.1.x). The tool surface and output shapes may shift between minor versions. Pin to an exact version in production. Bug reports very welcome via GitHub Issues.

Quick demo

$ uvx --from methods-mcp methods-mcp --version

methods-mcp 0.1.6

In a Claude Code session:

/mcp add methods-mcp methods-mcp

Run methods_repro_review on https://arxiv.org/abs/2509.06917

→ tool: methods_repro_review({"input_str":"https://arxiv.org/abs/2509.06917"})

Returns a MethodsReproReview object. Read narrative first — it explains

everything else in plain English, so no tool-learning is required:

{ "status": "ok", "narrative": "Resolved the paper: 'Paper2Agent' by Miao et al. (arxiv 2509.06917, " "2025-09-08). Extracted 11 methods steps at moderate self-reported " "confidence (0.72) — the procedure is clearly described but hyperparameters " "and software versions are absent. Detected the associated code repository " "https://github.com/jmiao24/Paper2Agent from an inline link in the paper " "text (detection confidence 0.94). The repo scored 0.90/1.00 on the " "reproducibility heuristic — verdict: likely reproducible. Present signals: " "substantive README, dependencies file, notebooks, figure-plotting script, " "recent activity, permissive license. Missing: data/fixtures directory. " "Suggested entrypoint: python make_figures.py.", "metadata": { ... }, # PaperMetadata "methods": { ... }, # MethodsStructured (null if extraction failed) "code_repo": { ... }, # CodeRepo (null only if input unresolvable) "repro_assessment": { ... }, # ReproAssessment (null if no repo detected) "errors": [] # [{step, error_type, message, hint}] on partial }

methods-mcp is a small, sharply-scoped Model Context Protocol server. It gives any AI agent (Claude Code, Claude Desktop, your Agent SDK script, etc.) eight tools that turn an academic paper URL into:

• canonical metadata,
• best-effort full text + section split,
• a Pydantic-validated structured methods object (steps / reagents / equipment / analyses),
• the paper's associated code repository (best-effort discovery),
• a no-execution-required reproducibility verdict for that repo, and
• a multi-mode summary.

The wedge: heavyweight pipelines like Paper2Agent (Stanford) take 30 minutes to hours to digest a paper into agent-ready tools. methods-mcp is the agent-callable, on-demand complement — every tool returns in seconds, no clone, no execution.

Install

uv add methods-mcp

or, install globally:

uv tool install methods-mcp

or, classic pip:

pip install methods-mcp

API keys

For best performance, set both:

export ANTHROPIC_API_KEY=sk-ant-...

export GITHUB_TOKEN=ghp_... # optional but recommended

Neither key is logged or persisted — they're sent only to api.anthropic.com and api.github.com respectively. See SECURITY.md.

Use it from Claude Code

/mcp add methods-mcp methods-mcp

Then in any Claude Code chat:

Take https://arxiv.org/abs/2509.06917 and run methods_repro_review. Summarise what the paper does, the methods steps, and how reproducible the repo looks.

Use it from the Claude Agent SDK

from claude_agent_sdk import ClaudeAgentOptions, ClaudeSDKClient

options = ClaudeAgentOptions( mcp_servers={ "methods-mcp": { "type": "stdio", "command": "methods-mcp", "args": [], } }, allowed_tools=["mcpmethods-mcpmethods_repro_review"], )

async with ClaudeSDKClient(options=options) as client: await client.query( "Run methods_repro_review on https://arxiv.org/abs/2509.06917 " "and tell me whether the repo looks reproducible." ) async for msg in client.receive_response(): print(msg)

Tools

All tools return Pydantic v2 models (validated, JSON-serialisable). See src/methods_mcp/schemas.py for the full type surface.

Design notes

• `extract_methods` uses Anthropic tool-use to coerce the model into emitting an instance of the `MethodsStructured` Pydantic schema. On validation failure we send one repair message with the validation error and try again before raising.
• `assess_repo_reproducibility` does not clone or execute anything. It scores the repo from publicly-readable GitHub metadata + the recursive tree listing. This is the deliberate wedge against batch tools that try to actually rerun the paper.
• `fetch_paper_text` prefers ar5iv HTML over PDF parsing for arXiv papers. Falls back to pypdf for non-arXiv inputs.
• The default model is `claude-sonnet-4-6`. Override via METHODS_MCP_MODEL env var or per-call model= arg.
• `methods_repro_review` returns a self-describing response. Every call sets a top-level status ("ok" / "partial" / "empty") and a narrative string that summarises everything retrieved in plain English — including every numeric score in context. A reader who reads only narrative + status gets the full picture without needing to learn the sub-object shapes. Sub-objects can be null when unavailable (e.g. repro_assessment: null on a paper with no detected repo — status stays "ok" because "no repo" isn't a failure). Failed sub-steps contribute a structured entry to errors with {step, error_type, message, hint}, where hint is an actionable plain-English suggestion for recognised patterns (missing API keys, rate-limits, 404s, timeouts, etc.) and null otherwise.

Scores & verdicts explained

Tool outputs contain three numeric fields that look similar but mean very different things. They are triage signals for an agent deciding whether a paper is worth digging into, not calibrated claims about correctness.

Verdict buckets (repro_assessment.verdict) are thresholds on overall_score:

Enum values you'll see in outputs:

• code_repo.detection_method: paper-text | abstract-link | papers-with-code | metadata | none
• metadata.source: arxiv | biorxiv | doi | url | unknown

Security & limitations

What this server *actually* does when you install and run it:

• Network calls only to: export.arxiv.org, ar5iv.labs.arxiv.org, arxiv.org (PDFs), api.github.com, paperswithcode.com, api.anthropic.com. No telemetry, no analytics, no phone-home.
• Reads ANTHROPIC_API_KEY (required for LLM tools) and optionally GITHUB_TOKEN from environment variables. These are sent only to Anthropic / GitHub respectively. Never logged, never persisted to disk.
• Writes nothing to your filesystem. No cache directories, no downloaded PDFs, no temp files.
• Executes no user-supplied code. No eval, exec, subprocess, pickle.loads, or shell-outs. The reproducibility tool deliberately *does not* clone or run repositories — it scores from the GitHub REST API only.

Limitations to be aware of:

• Adversarial papers may produce misleading structured output. The extract_methods tool sends paper text to Claude. A paper containing prompt-injection content could yield wrong (but schema-valid) structured methods. Treat the output as a research aid, not ground truth.
• The reproducibility verdict is a heuristic, not a proof. A high score means the repo *