Neuronscope

NeuronScope

Ask a language model "why did you say that" and get back the actual attention heads and neurons responsible, as JSON, from the command line or from an agent over MCP.

Install

pip install neuronscope-cli

That gets you the neuronscope command. To install from source instead (for development or to track main):

git clone https://github.com/RudrenduPaul/NeuronScope

cd NeuronScope pip install -e .

[!NOTE]

The first run of any command downloads the requested model from the HuggingFace Hub

(gpt2 is about 500MB) and prints two lines to stderr that are expected, not errors: a

CPU-fallback notice if you don't have a CUDA GPU, and an unauthenticated-HF-Hub

rate-limit notice. Neither one means anything broke.

Quickstart

neuronscope trace gpt2 "The capital of France is Paris. The capital of Japan is" --top-k 5

Real output from this exact command (stderr trimmed to the two expected warnings mentioned above):

Prompt: The capital of France is Paris. The capital of Japan is

Predicted next token: ' Tokyo' Top attention heads (by direct logit attribution) ┏━━━━━━━┳━━━━━━┳━━━━━━━━━━━━━━━━━━━┓ ┃ Layer ┃ Head ┃ Logit attribution ┃ ┡━━━━━━━╇━━━━━━╇━━━━━━━━━━━━━━━━━━━┩ │ 9 │ 8 │ 4.0679 │ │ 8 │ 11 │ 2.9028 │ │ 10 │ 7 │ -1.4782 │ │ 8 │ 10 │ -1.3999 │ │ 10 │ 0 │ 1.1424 │ └───────┴──────┴───────────────────┘ Top MLP neurons (by activation magnitude) ┏━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┓ ┃ Layer ┃ Neuron ┃ Activation ┃ ┡━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━┩ │ 10 │ 97 │ 7.8394 │ │ 11 │ 611 │ 4.6954 │ │ 11 │ 2997 │ 4.6468 │ │ 10 │ 1793 │ 4.5443 │ │ 9 │ 1460 │ 4.4196 │ └───────┴────────┴────────────┘

gpt2 predicts Tokyo correctly, and head L9H8 is the single biggest contributor to that prediction. Add --json to get the machine-readable version of the same result:

neuronscope trace gpt2 "The capital of France is Paris. The capital of Japan is" --top-k 3 --json

{

"schema_version": 1, "operation": "trace", "model": { "requested_name": "gpt2", "resolved_name": "gpt2", "backend": "transformer_lens", "device": "cpu", "n_layers": 12, "n_heads": 12, "d_model": 768, "d_mlp": 3072 }, "prompt": "The capital of France is Paris. The capital of Japan is", "predicted_token": " Tokyo", "predicted_token_id": 11790, "top_neurons": [ { "layer": 10, "neuron_index": 97, "activation": 7.839381217956543 }, { "layer": 11, "neuron_index": 611, "activation": 4.695372581481934 }, { "layer": 11, "neuron_index": 2997, "activation": 4.646785736083984 } ], "top_heads": [ { "layer": 9, "head_index": 8, "logit_attribution": 4.067923545837402 }, { "layer": 8, "head_index": 11, "logit_attribution": 2.9028172492980957 }, { "layer": 10, "head_index": 7, "logit_attribution": -1.4781968593597412 } ] }

What it does

NeuronScope is a CLI and MCP server built on top of TransformerLens. TransformerLens does the actual model loading, hooking, and activation math; NeuronScope adds a stable CLI, a versioned JSON schema, and an MCP server around it, so a script or an agent can ask "which components drove this output" without writing TransformerLens code directly.

• `trace`: runs a prompt through the model and ranks attention heads by direct logit attribution to the predicted token, and MLP neurons by activation magnitude at the final prompt position.
• `activations`: dumps shape, mean, std, min/max, and the max-activating sequence position for every layer's residual stream, MLP neuron activations, and attention pattern.
• `patch`: zero-ablates one component (resid_pre, resid_mid, resid_post, attn_out, mlp_out, or mlp_post) at a given layer and reports how the predicted token and its logit changed.
• `circuit`: a best-effort automated circuit sketch. Ranks candidate heads/neurons by logit attribution, then measures each one's individual causal effect via single-component ablation. This is not full path-patching with clean/corrupted prompt pairs and does not capture interaction effects between components. The --json output says so explicitly in its method field.
• Every command supports --json for a schema_version-stamped document instead of a table, and the same four operations are exposed as MCP tools returning the identical shape via .model_dump(), so a CLI call and an MCP tool call produce the same document for the same input.
• Model support is whatever transformer_lens.HookedTransformer.from_pretrained supports. Installing neuronscope-cli today pulls TransformerLens 3.6.0, which supports 249 pretrained checkpoints and aliases (OFFICIAL_MODEL_NAMES), covering GPT-2, Pythia, Llama, Gemma, Qwen, and more. Small models like gpt2 run comfortably on CPU.

NeuronScope does not replace TransformerLens, nnsight, SAELens, Anthropic's circuit-tracer, or Neuronpedia. It wraps TransformerLens for one narrower job: fast, scriptable, agent-callable component tracing on a single prompt. It leaves deeper mechanistic work (SAE training, transcoder-based circuit graphs, hosted feature browsing) to those tools.

CLI reference

Every command takes MODEL (any name HookedTransformer.from_pretrained accepts, for example gpt2 or EleutherAI/pythia-70m) and PROMPT as positional arguments.

Global: neuronscope --version, neuronscope --help. Exit codes: 0 success, 1 a runtime error (prompt too long for the model's context window, --layer out of range, etc.), 2 a Click usage error (bad flags), 3 an unsupported model name.

MCP Server

NeuronScope ships a Model Context Protocol server so an AI agent (Claude, Cursor, or any MCP-compatible client) can trace, inspect, ablate, and sketch circuits directly, without a human invoking the CLI by hand.

Install the extra:

pip install "neuronscope-cli[mcp]"

Add it to your MCP client's config (for Claude Desktop, claude_desktop_config.json):

{

"mcpServers": { "neuronscope": { "command": "uvx", "args": ["--from", "neuronscope-cli", "neuronscope-mcp"] } } }

The server exposes four tools, trace, activations, patch, and circuit, each returning the identical pydantic-model-shaped JSON the CLI's --json flag prints, via .model_dump(), so an agent calling this server and a script calling the CLI get the same document for the same input. A real trace call and its response:

trace(model="gpt2", prompt="The capital of France is Paris. The capital of Japan is", top_k=3)

{ "schema_version": 1, "operation": "trace", "predicted_token": " Tokyo", "predicted_token_id": 11790, "top_neurons": [ { "layer": 10, "neuron_index": 97, "activation": 7.839381217956543 } ], "top_heads": [ { "layer": 9, "head_index": 8, "logit_attribution": 4.067923545837402 } ] }

Errors never raise across the tool boundary: every handler catches its exceptions and returns a structured ErrorResponse dict instead, so a calling agent always gets a parseable result.

[!WARNING]

NeuronScope caps model size (2B parameters by default, NEURONSCOPE_MAX_MODEL_PARAMS)

and how many models can load at once (1 by default, NEURONSCOPE_MAX_CONCURRENT_LOADS),

but puts no timeout on model loading or forward passes. If you expose this MCP server

somewhere an untrusted agent can call it, still put a resource limit around the process

(a cgroup, ulimit, or a container memory/CPU cap) as defense in depth rather than

relying on these in-process caps alone.

Transport is stdio, so there is nothing to host: the MCP client spawns the server as a local subprocess. Source: neuronscope/mcp_server.py.

• Claude Code reads this from a project-level .mcp.json in your repo root, or you can add it with claude mcp add neuronscope -- neuronscope-mcp.
• Claude Desktop reads this from its claude_desktop_config.json (~/Library/Application Support/Claude/claude_desktop_config.json on macOS, %APPDATA%\Claude\claude_desktop_config.json on Windows), under the same "mcpServers" key.
• The neuronscope mcp-server CLI subcommand still works as a local, non-uvx alternative that runs the same server over stdio from an existing install.

How it compares

All five of these are real, actively maintained projects doing different jobs. This table compares CLI/JSON-agent-output surface and model coverage, not depth of interpretability research, where TransformerLens, nnsight, SAELens, circuit-tracer, and Neuronpedia are all more mature than NeuronScope. Star counts, release info, and last-push dates below were pulled from each project's GitHub API on 2026-08-03 and will drift over time; check the repos directly for current numbers.

| TransformerLens | 3,750 | v3.6.0 released 2026-07-28, pushed 2026-08-03