Modelwatch
Overview
Description from the maker (original English)
Continuous behavioral drift monitoring for LLM apps — catches silent provider model updates.
PopNx MCP’s three pillars
These three are never merged into one score: opinions, measurements and popularity are different things.
- User voiceiNo reviews yetReviews open together with accounts in a later step
- Our basic checksi2 of 2 passedChecked from public information; code not scanned yet
- UsageiVery new↓ 60/wk · ★ 0
What you can do with it
We have not written a plain-language summary for this one yet. Below are the facts we read from the registry.
- Connection: Runs on your computer
- Transport: stdio
- Category: Search & AI memory
- You need to fill in: MODELWATCH_API_KEY
Install
Pick the AI app you use, then copy the settings.
Before you install
- Basic checks: 2 of 2 passed
- Runs on your computer with your user’s permissions (files, network, apps).
- You fill in yourself:
MODELWATCH_API_KEY,MODELWATCH_API_BASE
Values marked secret are keys to your account. Never share them or paste them into a chat.
You need
Which app do you use?
Pick your app above. We remember it, so next time the button says “Install in …”.
Open Claude Desktop → Settings → Developer → Edit Config, and paste this into claude_desktop_config.json
{
"mcpServers": {
"modelwatch": {
"command": "npx",
"args": [
"-y",
"modelwatch-mcp"
],
"env": {
"MODELWATCH_API_KEY": "your-value",
"MODELWATCH_API_BASE": "your-value"
}
}
}
}Run this in your terminal
claude mcp add modelwatch --env MODELWATCH_API_KEY=your-value --env MODELWATCH_API_BASE=your-value -- npx -y modelwatch-mcpFill in the empty values before running it.
Or copy the settings yourself
Paste into .cursor/mcp.json in your project, or ~/.cursor/mcp.json
{
"mcpServers": {
"modelwatch": {
"command": "npx",
"args": [
"-y",
"modelwatch-mcp"
],
"env": {
"MODELWATCH_API_KEY": "your-value",
"MODELWATCH_API_BASE": "your-value"
}
}
}
}Your browser asks to open the app, then the app shows the settings and asks you to confirm. Key values are left empty for you to fill in.
Or copy the settings yourself
Paste into .vscode/mcp.json in your project
{
"servers": {
"modelwatch": {
"type": "stdio",
"command": "npx",
"args": [
"-y",
"modelwatch-mcp"
],
"env": {
"MODELWATCH_API_KEY": "your-value",
"MODELWATCH_API_BASE": "your-value"
}
}
}
}Your browser asks to open the app, then the app shows the settings and asks you to confirm. Key values are left empty for you to fill in.
This one runs on your computer, and ChatGPT can only connect to servers on the internet. Pick another app.
Your choice stays on this device in a cookie; no account needed.
Values you fill in
- API key from https://modelwatch.app (free tier: 5 specs, 500 runs/mo).
MODELWATCH_API_BASEenvOverride the API base URL for self-hosted deployments.
These settings are generated from the maker’s registry entry. If the README says otherwise, follow the README.
Which apps can use it
Based on how this server connects and on each app’s own docs, checked Nov 10, 2026.
Run on your computer in
ClaudeGooseLM StudioMsty StudioCherry StudioCursorVS Code + GitHub CopilotVisual StudioWindsurf (Devin Desktop)ZedKiroClineContinueJetBrains AI AssistantClaude CodeCodex CLIGemini CLIOpenCodenodecodeWarpLibreChatRaycast
Related guides
What it can do
Runs on your computer; the tool list appears in your AI app after you install it.
Basic checksi
We check information anyone can see; we have not read the code. Checks that do not apply are left out of the count. · Checked Nov 11, 2026
- ✓ Pass
Maker matches the repo
The registry name was verified with the GitHub account bch1212, which owns the repo
- ✓ Pass
Still maintained
Code last changed yesterday
- … Not checked yet
Has a licence
We have not read this repo from GitHub yet
- – Not applicable
Endpoint responds
This one runs on your computer; there is no internet endpoint
- – Not applicable
Sign-in with OAuth
Runs on your computer and uses local settings instead
- … Not checked yet
No hidden instructions in tool descriptions
Runs on your computer; we can only read tool lists from internet endpoints
Compared with others in Search & AI memoryi
- More stars than 0% of the other 5867 servers in this category
- Published to the registry more recently than 53% of the other 5867 servers in this category
- Basic checks equal to or better than 100% of the other 5867 servers in this category
Compared with every active server in this category in the registry. Stars count only when the publisher owns the repo.
Common questions
Is Modelwatch safe?
It passed 2 of 2 applicable basic checks. We have not read the code. Before adding a key or token, see what access the server asks for and use a key with only the permissions you need.
What keys or codes does it need?
You need to fill in: MODELWATCH_API_KEY
Does it run on my computer or in the cloud?
It runs on your computer. Your AI app starts it when needed, and data does not pass through the maker’s servers (unless it calls an online service itself).
Is it still maintained?
The GitHub repo last changed yesterday, and the latest version (0.1.3) reached the registry 34 days ago.
Which apps can use it?
Apps that support MCP, such as Claude Desktop, Claude Code, Cursor and VS Code. You also need Node.js on your computer.
What can I use instead?
In the same category: Ruflo, Context7, Metabase
Alternatives
Servers in the same category from other makers, sorted by stars.
Reading a card:↓ Downloads per weeki★ GitHub starsiThree pillars: reviews · checks · usageiUpdated = latest version published to the registryi
README from the maker
Open the README
Copied from the maker’s repo as written. PopNx MCP has not reviewed or endorsed it. HTML, images and scripts are removed. Links lead off this site.
modelwatch-mcp
MCP server for ModelWatch — continuous behavioral drift monitoring for LLM-powered applications.
Lets Claude Desktop / Claude Code (or any MCP-compatible client) create drift specs, run them, and surface drift events without leaving the chat.
Install
npm install -g modelwatch-mcpConfigure (Claude Desktop)
Add to ~/Library/Application Support/Claude/claude_desktop_config.json:
{"mcpServers": { "modelwatch": { "command": "npx", "args": ["-y", "modelwatch-mcp"], "env": { "MODELWATCH_API_KEY": "mw_PASTE_FROM_EMAIL" } } } }
Get a free key at https://modelwatch.app/#signup (5 specs, 500 runs/mo, no card). Then restart Claude Desktop.
The example configuration stores a plaintext key in a local config file. Restrict that file to your user account, do not commit it, and rotate the key if it is exposed. The key never belongs in a command-line argument.
Tools
| Tool | What it does |
|---|---|
list_endpoints | Show monitored LLM endpoints |
create_endpoint | Add an endpoint (provider + model) |
list_specs | Show behavioral specs |
create_spec | Add a spec (prompt + schedule + optional semantic similarity threshold) |
run_spec | Run a spec on demand; baseline if first run |
reset_baseline | Re-baseline after an intentional model swap |
get_drift_events | Recent drift events across the workspace |
get_spec_history | Run history for one spec |
get_health | Per-spec health status, last drift score and last run time |
Example session
You: Create a refusal canary against the OpenAI endpoint, asking about
chest pain. Run it now and tell me the baseline.
Claude calls list_endpoints → create_spec → run_spec and reports the score. The same spec then runs daily forever and emails you when drift is detected.
Environment variables
| Var | Required | Default |
|---|---|---|
MODELWATCH_API_KEY | Yes | — |
MODELWATCH_API_BASE | No | https://api.modelwatch.app |
Self-hosted? Point MODELWATCH_API_BASE at an HTTPS origin. HTTP is accepted only for loopback development; paths, query strings, URL credentials and redirects are rejected before a key is sent.
Source
https://github.com/bch1212/modelwatch — MIT licensed.