Prototype · Preview: every server in the official registry, synced every 30 minutes. Sign in with Google to write reviews.Prototype · updated every 30 min
PopNx MCPMCP servers for people who use AI · explained in plain words

MCP As A Judge

by OtherVibesversion 0.3.20i★ 17iRuns on your computerRepository ↗
Registry nameio.github.OtherVibes/mcp-as-a-judge

How to install

What it does

MCP as a Judge: a behavioral MCP that strengthens AI coding assistants via explicit LLM evaluations

  • Connects Runs on your computer
  • Account or key No key needed
  • Category Search & AI memory

Use together with

Skills that teach the AI this kind of work; they work alongside this one

timesfm

TimesFM (Time Series Foundation Model) is a pretrained time-series foundation model developed by Google Research for time-series forecasting.

by google-research★ 35,6381 skills

Skill · How-to guide

Shadowbroker

Open-source intelligence for the global theater. Track everything from the corporate/private jets of the wealthy, and spy satellites, to seismic events in one unified interface. Hook an AI agent up to have it parse through data and find previously unseen correlations. The knowledge is available to…

by BigBodyCobain★ 11,6051 skills

Skill · How-to guide

google-maps-scraper

scrape data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place

by gosom★ 8,2361 skills

Skill · How-to guide

Why it ranks here

Lists are ordered by a usage score (out of 100) built from these parts

Popularity15/80GitHub stars or weekly downloads: the larger counts 60, the smaller 20
Freshness3/12Full marks for a release or code change within 30 days
Still answering0/8The cloud endpoint answered at the last check
Usage score18/100

Quality: Passed 2/3 basic checks (shown, not used for ordering) · Momentum: No data yet (Used only on the Trending tab)

Install

Pick the AI app you use, then copy the settings.

Let your AI install it with one sentence

Please install MCP As A Judge. Follow https://mcp.popnx.com/i/othervibes-mcp-as-a-judge.md, show me the commands and wait for my OK before running them.

Paste it into an AI that can work on your computer (such as Claude Code or Cursor). It reads our steps, shows you the commands and waits for your OK. No key, no sign-up. · See the full text your AI reads

Passed 2/3 checks · Runs on your computer · ✓ Commands checked Nov 13, 2026

First: which AI do you use?

Pick your app and the steps appear below.

Works with Claude, with some setup first

Install Python (uv) on your computer first (one time) · Only the Claude desktop app (Windows or Mac), not the web or phone

Let AI install it

Copy this message into a Claude chat. The AI checks your computer, tells you which file it will change, and waits for your OK every time.

Please install the MCP server MCP As A Judge for Claude on this computer, in this order: 1) Check uv with uv --version. If it is missing, tell me to install it from https://docs.astral.sh/uv/getting-started/installation/; do not install it yourself. 2) Use only this config (already checked by PopNx MCP): { "mcpServers": { "as-a-judge": { "command": "uvx", "args": [ "mcp-as-a-judge" ] } } } 3) Tell me the full path of the file you will change, or the command you will run, and wait for my OK. 4) No key is needed. 5) Do what we agreed, then check that the MCP As A Judge tools really show up.

This message has no key in it. If a key is needed, the AI will ask you to paste it yourself into the file or the app’s field.

  1. Install uv (if you don’t have it)

    Already installed? Check first · Open Terminal, type this command and press Enter

    uv --version
    • A version number such as uv … = installed, skip this step
    • "command not found" (on Windows: "is not recognized") = not installed yet, or installed but Terminal needs to be closed and reopened
    How to open Terminal

    Mac: press ⌘ + Space, type Terminal, press Enter

    Windows: open Start, type Windows PowerShell, then Open

    Not installed: uv has no click-through installer. Copy the one-line command for your system (macOS or Windows) from uv’s official page into Terminal · no separate Python install needed; uv downloads it when required

    Open the uv download page ↗

    When it’s done, close Terminal, open it again and run the same command. You should see a version number: uv --version

    Install once; it works for everything that needs uv

  2. Open the Claude desktop app’s settings file

    SettingsDeveloperEdit Config

  3. Paste these settings and save the file

    {
      "mcpServers": {
        "as-a-judge": {
          "command": "uvx",
          "args": [
            "mcp-as-a-judge"
          ]
        }
      }
    }
  4. Quit Claude and open it again

  5. Try it

    Use MCP As A Judge and tell me what you can do for me, then show me one thing

    ✓ If Claude says it is using MCP As A Judge or shows a MCP As A Judge tool in the answer, you are connected

Menu names checked against Anthropic’s help page on Nov 11, 2026 · your screen may differ slightly by app version

ChatGPT cannot connect MCP As A Judge yet

MCP As A Judge runs on your computer, but ChatGPT can only connect servers that run on the web

What you can do instead

A · Use an app that can connect MCP As A Judge

  • Install Python (uv) on your computer first (one time) · Only the Claude desktop app (Windows or Mac), not the web or phone
  • Install Python (uv) on your computer first (one time)
  • Install Python (uv) on your computer first (one time)
  • Install Python (uv) on your computer first (one time)

B · Others in Search & AI memory that work with ChatGPT

Picked from the most used in the same category, only ones that connect by address with no key

Works with Claude Code, with some setup first

Install Python (uv) on your computer first (one time)

Let AI install it

Copy this message into a Claude Code chat. The AI checks your computer, tells you which file it will change, and waits for your OK every time.

Please install the MCP server MCP As A Judge for Claude Code on this computer, in this order: 1) Check uv with uv --version. If it is missing, tell me to install it from https://docs.astral.sh/uv/getting-started/installation/; do not install it yourself. 2) Use only this config (already checked by PopNx MCP): claude mcp add --scope user as-a-judge -- uvx mcp-as-a-judge 3) Tell me the full path of the file you will change, or the command you will run, and wait for my OK. 4) No key is needed. 5) Do what we agreed, then check that the MCP As A Judge tools really show up.

This message has no key in it. If a key is needed, the AI will ask you to paste it yourself into the file or the app’s field.

  1. Install uv (if you don’t have it)

    Already installed? Check first · Open Terminal, type this command and press Enter

    uv --version
    • A version number such as uv … = installed, skip this step
    • "command not found" (on Windows: "is not recognized") = not installed yet, or installed but Terminal needs to be closed and reopened
    How to open Terminal

    Mac: press ⌘ + Space, type Terminal, press Enter

    Windows: open Start, type Windows PowerShell, then Open

    Not installed: uv has no click-through installer. Copy the one-line command for your system (macOS or Windows) from uv’s official page into Terminal · no separate Python install needed; uv downloads it when required

    Open the uv download page ↗

    When it’s done, close Terminal, open it again and run the same command. You should see a version number: uv --version

    Install once; it works for everything that needs uv

  2. Open Terminal

    Mac: press Command+Space and type Terminal · Windows: open PowerShell

  3. Paste this command and press Enter

    claude mcp add --scope user as-a-judge -- uvx mcp-as-a-judge
  4. Type /mcp in Claude Code

    You should see MCP As A Judge listed as connected

  5. Try it

    Use MCP As A Judge and tell me what you can do for me, then show me one thing

    ✓ If Claude Code says it is using MCP As A Judge or shows a MCP As A Judge tool in the answer, you are connected

Menu names checked against Anthropic’s help page on Nov 11, 2026 · your screen may differ slightly by app version

Works with Cursor, with some setup first

Install Python (uv) on your computer first (one time)

Let AI install it

Copy this message into a Cursor chat. The AI checks your computer, tells you which file it will change, and waits for your OK every time.

Please install the MCP server MCP As A Judge for Cursor on this computer, in this order: 1) Check uv with uv --version. If it is missing, tell me to install it from https://docs.astral.sh/uv/getting-started/installation/; do not install it yourself. 2) Use only this config (already checked by PopNx MCP): { "mcpServers": { "as-a-judge": { "command": "uvx", "args": [ "mcp-as-a-judge" ] } } } 3) Tell me the full path of the file you will change, or the command you will run, and wait for my OK. 4) No key is needed. 5) Do what we agreed, then check that the MCP As A Judge tools really show up.

This message has no key in it. If a key is needed, the AI will ask you to paste it yourself into the file or the app’s field.

  1. Install uv (if you don’t have it)

    Already installed? Check first · Open Terminal, type this command and press Enter

    uv --version
    • A version number such as uv … = installed, skip this step
    • "command not found" (on Windows: "is not recognized") = not installed yet, or installed but Terminal needs to be closed and reopened
    How to open Terminal

    Mac: press ⌘ + Space, type Terminal, press Enter

    Windows: open Start, type Windows PowerShell, then Open

    Not installed: uv has no click-through installer. Copy the one-line command for your system (macOS or Windows) from uv’s official page into Terminal · no separate Python install needed; uv downloads it when required

    Open the uv download page ↗

    When it’s done, close Terminal, open it again and run the same command. You should see a version number: uv --version

    Install once; it works for everything that needs uv

  2. Click this button to open Cursor

    Open in Cursor

    Cursor asks you to confirm; click Install

  3. Open the AI chat in Cursor in Agent mode

  4. Try it

    Use MCP As A Judge and tell me what you can do for me, then show me one thing

    ✓ If Cursor says it is using MCP As A Judge or shows a MCP As A Judge tool in the answer, you are connected

Menu names checked against Cursor’s help page on Nov 11, 2026 · your screen may differ slightly by app version

Works with VS Code, with some setup first

Install Python (uv) on your computer first (one time)

Let AI install it

Copy this message into a VS Code chat. The AI checks your computer, tells you which file it will change, and waits for your OK every time.

Please install the MCP server MCP As A Judge for VS Code on this computer, in this order: 1) Check uv with uv --version. If it is missing, tell me to install it from https://docs.astral.sh/uv/getting-started/installation/; do not install it yourself. 2) Use only this config (already checked by PopNx MCP): { "servers": { "as-a-judge": { "type": "stdio", "command": "uvx", "args": [ "mcp-as-a-judge" ] } } } 3) Tell me the full path of the file you will change, or the command you will run, and wait for my OK. 4) No key is needed. 5) Do what we agreed, then check that the MCP As A Judge tools really show up.

This message has no key in it. If a key is needed, the AI will ask you to paste it yourself into the file or the app’s field.

  1. Install uv (if you don’t have it)

    Already installed? Check first · Open Terminal, type this command and press Enter

    uv --version
    • A version number such as uv … = installed, skip this step
    • "command not found" (on Windows: "is not recognized") = not installed yet, or installed but Terminal needs to be closed and reopened
    How to open Terminal

    Mac: press ⌘ + Space, type Terminal, press Enter

    Windows: open Start, type Windows PowerShell, then Open

    Not installed: uv has no click-through installer. Copy the one-line command for your system (macOS or Windows) from uv’s official page into Terminal · no separate Python install needed; uv downloads it when required

    Open the uv download page ↗

    When it’s done, close Terminal, open it again and run the same command. You should see a version number: uv --version

    Install once; it works for everything that needs uv

  2. Click this button to open VS Code

    Open in VS Code

    VS Code asks you to confirm; click Install

  3. Open the AI chat in VS Code in Agent mode

  4. Try it

    Use MCP As A Judge and tell me what you can do for me, then show me one thing

    ✓ If VS Code says it is using MCP As A Judge or shows a MCP As A Judge tool in the answer, you are connected

Menu names checked against VS Code’s help page on Nov 11, 2026 · your screen may differ slightly by app version

Other apps that use MCP, such as Windsurf, Zed or Gemini CLI, each add servers differently. See how in the app list.

See every app that uses MCP →

JSON settings are under “For developers” below

Stuck? Ask people who use MCP As A Judge💬 Discussion →
For developers: Terminal commands and JSON settings

Installed on your computer

Claude Code

claude mcp add --scope user as-a-judge -- uvx mcp-as-a-judge

Claude Desktop (claude_desktop_config.json)

{
  "mcpServers": {
    "as-a-judge": {
      "command": "uvx",
      "args": [
        "mcp-as-a-judge"
      ]
    }
  }
}

Cursor (~/.cursor/mcp.json)

{
  "mcpServers": {
    "as-a-judge": {
      "command": "uvx",
      "args": [
        "mcp-as-a-judge"
      ]
    }
  }
}

VS Code (mcp.json)

{
  "servers": {
    "as-a-judge": {
      "type": "stdio",
      "command": "uvx",
      "args": [
        "mcp-as-a-judge"
      ]
    }
  }
}

These settings are generated from the maker’s registry entry. If the README says otherwise, follow the README.

At a glance

Downloads per weeki
70/wk
Connection
Runs on your computer
Latest versioni
0.3.20
Published to registryi
Sep 20, 2025
Code last changedi
Dec 15, 2025
Maker
OtherVibes
Packages
pypi: mcp-as-a-judge
Registry status
Active

Which apps can use it

Based on how this server connects and on each app’s own docs, checked Nov 10, 2026.

Run on your computer in

ClaudeGooseLM StudioMsty StudioCherry StudioCursorVS Code + GitHub CopilotVisual StudioWindsurf (Devin Desktop)ZedKiroClineContinueJetBrains AI AssistantClaude CodeCodex CLIGemini CLIOpenCodenodecodeWarpLibreChatRaycast

All apps, with pros and cons →

Related guides

What it can do

Runs on your computer; the tool list appears in your AI app after you install it.

Basic checksi

We check information anyone can see; we have not read the code. Checks that do not apply are left out of the count. · Checked Nov 20, 2026

  • ✓ Pass

    Maker matches the repo

    The registry name was verified with the GitHub account OtherVibes, which owns the repo

  • ! Caution

    Still maintained

    Code last changed 11 months ago, more than 6 months ago

  • ✓ Pass

    Has a licence

    Uses MIT

  • – Not applicable

    Endpoint responds

    This one runs on your computer; there is no internet endpoint

  • – Not applicable

    Sign-in with OAuth

    Runs on your computer and uses local settings instead

  • … Not checked yet

    No hidden instructions in tool descriptions

    Runs on your computer; we can only read tool lists from internet endpoints

Published vulnerabilities (OSV.dev)

No entries on OSV.dev for mcp-as-a-judge@0.3.20

Checks only this package version, not every dependency · checked Nov 15, 2026

What we check → · JSON

Makers: ask for a re-check or report wrong info
For makers: add a badge to your README

If you made this, paste this code into your README. The badge links back to this page.

[![Featured on PopNx MCP](https://mcp.popnx.com/badge/othervibes-mcp-as-a-judge.svg)](https://mcp.popnx.com/en/servers/othervibes-mcp-as-a-judge)

Discussion

Ask, share a tip or report that it does not work. Sorted by what people marked helpful.

No posts yet. Be the first to ask or share.

Posts here are written by users, not the PopNx team.

Sign in to post

User reviews

No reviews yet

Sign in to write a review

Compared with others in Search & AI memoryi

  • More stars than 94% of the other 6447 servers in this category
  • Published to the registry more recently than 0% of the other 6447 servers in this category
  • Basic checks equal to or better than 50% of the other 6447 servers in this category

Compared with every active server in this category in the registry. Stars count only when the publisher owns the repo.

Common questions

Is MCP As A Judge safe?

It passed 2 of 3 applicable basic checks. We have not read the code. Before adding a key or token, see what access the server asks for and use a key with only the permissions you need.

What keys or codes does it need?

The registry does not list a required key, though some servers ask you to sign in on first use.

Does it run on my computer or in the cloud?

It runs on your computer. Your AI app starts it when needed, and data does not pass through the maker’s servers (unless it calls an online service itself).

Is it still maintained?

The GitHub repo last changed 11 months ago, and the latest version (0.3.20) reached the registry 14 months ago.

Which apps can use it?

Apps that support MCP, such as Claude Desktop, Claude Code, Cursor and VS Code. You also need Python (uv) on your computer.

What can I use instead?

In the same category: OpenSEO, Exa, Microsoft Learn MCP, Docs, COTAL actions (cotal.ai)

Similar alternatives

Same category, other makers, commands already checked. Official ones first, then by stars.

Reading a card:↓ Downloads per weeki★ GitHub starsiThree pillars: reviews · checks · usageiUpdated = latest version published to the registryi

OpenSEO

Keyword research, site audits, rank tracking and AI visibility for your AI agent.

by every-appWidely used★ 28.3KUpdated 6 days ago

Runs in the cloud✓ everyapp.com

Exa

Connect AI agents to Exa for web search, content fetching, and multi-step research.

by exa-labsWidely used★ 5.2KUpdated 2 months ago

Runs in the cloud✓ service owner · exa.aiTried by the PopNx team

Microsoft Learn MCP

Official Microsoft Learn MCP Server – real-time, trusted docs & code samples for AI and LLMs.

by MicrosoftDocsWidely used★ 2KUpdated 12 months ago

Runs in the cloud✓ service owner · microsoft.comTried by the PopNx team

Docs

Search and read Empryo's documentation. Read-only, no auth, no local access.

by proxysoulGrowing★ 1.6KUpdated 2 months ago

Runs in the cloud✓ empryo.com
Compared with similar ones
This one
MCP As A Judge
OpenSEOExaMicrosoft Learn MCPDocsCOTAL actions (cotal.ai)
Runs whereInstalls on your computerRuns in the cloud, nothing to installRuns in the cloud, nothing to installRuns in the cloud, nothing to installRuns in the cloud, nothing to installRuns in the cloud, nothing to install
Needs a keyNoNoNoNoNoNo
Add with + in ClaudeNoYesYesYesNoNo
Basic checks2/35/52/44/44/44/4
LicenseMITMITMITCC-BY-4.0NOASSERTIONApache-2.0
PriceNo cost listed in the registryNo cost listed in the registryNo cost listed in the registryNo cost listed in the registryNo cost listed in the registryNo cost listed in the registry

README from the maker

Open the README

Copied from the maker’s repo as written. PopNx MCP has not reviewed or endorsed it. HTML, images and scripts are removed. Links lead off this site.

MCP as a Judge ⚖️

mcp-name: io.github.OtherVibes/mcp-as-a-judge

MCP as a Judge acts as a validation layer between AI coding assistants and LLMs, helping ensure safer and higher-quality code.

MCP as a Judge is a behavioral MCP that strengthens AI coding assistants by requiring explicit LLM evaluations for:

  • Research, system design, and planning
  • Code changes, testing, and task-completion verification

It enforces evidence-based research, reuse over reinvention, and human-in-the-loop decisions.

If your IDE has rules/agents (Copilot, Cursor, Claude Code), keep using them—this Judge adds enforceable approval gates on plan, code diffs, and tests.

Key problems with AI coding assistants and LLMs

  • Treat LLM output as ground truth; skip research and use outdated information
  • Reinvent the wheel instead of reusing libraries and existing code
  • Cut corners: code below engineering standards and weak tests
  • Make unilateral decisions when requirements are ambiguous or plans change
  • Security blind spots: missing input validation, injection risks/attack vectors, least‑privilege violations, and weak defensive programming

Vibe coding doesn’t have to be frustrating

What it enforces
  • Evidence‑based research and reuse (best practices, libraries, existing code)
  • Plan‑first delivery aligned to user requirements
  • Human‑in‑the‑loop decisions for ambiguity and blockers
  • Quality gates on code and tests (security, performance, maintainability)
Key capabilities
  • Intelligent code evaluation via MCP sampling; enforces software‑engineering standards and flags security/performance/maintainability risks
  • Comprehensive plan/design review: validates architecture, research depth, requirements fit, and implementation approach
  • User‑driven decisions via MCP elicitation: clarifies requirements, resolves obstacles, and keeps choices transparent
  • Security validation in system design and code changes
Tools and how they help
ToolWhat it solves
set_coding_taskCreates/updates task metadata; classifies task_size; returns next-step workflow guidance
get_current_coding_taskRecovers the latest task_id and metadata to resume work safely
judge_coding_planValidates plan/design; requires library selection and internal reuse maps; flags risks
judge_code_changeReviews unified Git diffs for correctness, reuse, security, and code quality
judge_testing_implementationValidates tests using real runner output and optional coverage
judge_coding_task_completionFinal gate ensuring plan, code, and tests approvals before completion
raise_missing_requirementsElicits missing details and decisions to unblock progress
raise_obstacleEngages the user on trade‑offs, constraints, and enforced changes

🚀 Quick Start

Requirements & Recommendations
MCP Client Prerequisites

MCP as a Judge is heavily dependent on MCP Sampling and MCP Elicitation features for its core functionality:

  • [MCP Sampling](https://modelcontextprotocol.io/docs/learn/client-concepts#sampling) - Required for AI-powered code evaluation and judgment
  • [MCP Elicitation](https://modelcontextprotocol.io/docs/learn/client-concepts#elicitation) - Required for interactive user decision prompts
System Prerequisites
  • Docker Desktop / Python 3.13+ - Required for running the MCP server
Supported AI Assistants
AI AssistantPlatformMCP SupportStatusNotes
GitHub CopilotVisual Studio Code✅ FullRecommendedComplete MCP integration with sampling and elicitation

| Claude Code | - | ⚠️ Partial | Requires LLM API key | Sampling Support feature request Elicitation Support feature request | | Cursor | - | ⚠️ Partial | Requires LLM API key | MCP support available, but sampling/elicitation limited | | Augment | - | ⚠️ Partial | Requires LLM API key | MCP support available, but sampling/elicitation limited | | Qodo | - | ⚠️ Partial | Requires LLM API key | MCP support available, but sampling/elicitation limited |

✅ Recommended setup: GitHub Copilot + VS Code — full MCP sampling; no API key needed.

⚠️ Critical: For assistants without full MCP sampling (Cursor, Claude Code, Augment, Qodo), you MUST set LLM_API_KEY. Without it, the server cannot evaluate plans or code. See LLM API Configuration.

💡 Tip: Prefer large context models (≥ 1M tokens) for better analysis and judgments.

If the MCP server isn’t auto‑used

For troubleshooting, visit the FAQs section.

🔧 MCP Configuration

Configure MCP as a Judge in your MCP-enabled client:

Method 1: Using Docker (Recommended)
One‑click install for VS Code (MCP)

Notes:

  • VS Code controls the sampling model; select it via “MCP: List Servers → mcp-as-a-judge → Configure Model Access”.
  1. Configure MCP Settings:

Add this to your MCP client configuration file:

   {

"command": "docker", "args": ["run", "--rm", "-i", "--pull=always", "ghcr.io/othervibes/mcp-as-a-judge:latest"], "env": { "LLM_API_KEY": "your-openai-api-key-here", "LLM_MODEL_NAME": "gpt-4o-mini" } }

📝 Configuration Options (All Optional):

  • LLM_API_KEY: Optional for GitHub Copilot + VS Code (has built-in MCP sampling)
  • LLM_MODEL_NAME: Optional custom model (see Supported LLM Providers for defaults)
  • The --pull=always flag ensures you always get the latest version automatically

Then manually update when needed:

   # Pull the latest version

docker pull ghcr.io/othervibes/mcp-as-a-judge:latest

Method 2: Using uv
  1. Install the package:
   uv tool install mcp-as-a-judge
  1. Configure MCP Settings:

The MCP server may be automatically detected by your MCP‑enabled client.

📝 Notes:

  • No additional configuration needed for GitHub Copilot + VS Code (has built-in MCP sampling)
  • LLM_API_KEY is optional and can be set via environment variable if needed
  1. To update to the latest version:
   # Update MCP as a Judge to the latest version

uv tool upgrade mcp-as-a-judge

### Select a sampling model in VS Code
  • Open Command Palette (Cmd/Ctrl+Shift+P) → “MCP: List Servers”
  • Select the configured server “mcp-as-a-judge”
  • Choose “Configure Model Access”
  • Check your preferred model(s) to enable sampling

🔑 LLM API Configuration (Optional)

For AI assistants without full MCP sampling support you can configure an LLM API key as a fallback. This ensures MCP as a Judge works even when the client doesn't support MCP sampling.

  • Set LLM_API_KEY (unified key). Vendor is auto-detected; optionally set LLM_MODEL_NAME to override the default.
Supported LLM Providers
RankProviderAPI Key FormatDefault ModelNotes
1OpenAIsk-...gpt-4.1Fast and reliable model optimized for speed
2Anthropicsk-ant-...claude-sonnet-4-20250514High-performance with exceptional reasoning
3GoogleAIza...gemini-2.5-proMost advanced model with built-in thinking
4Azure OpenAI[a-f0-9]{32}gpt-4.1Same as OpenAI but via Azure
5AWS BedrockAWS credentialsanthropic.claude-sonnet-4-20250514-v1:0Aligned with Anthropic
6Vertex AIService Account JSONgemini-2.5-proEnterprise Gemini via Google Cloud
7Groqgsk_...deepseek-r1Best reasoning model with speed advantage
8OpenRoutersk-or-...deepseek/deepseek-r1Best reasoning model available
9xAIxai-...grok-code-fast-1Latest coding-focused model (Aug 2025)
10Mistral[a-f0-9]{64}pixtral-largeMost advanced model (124B params)
Client-Specific Setup
Cursor
  1. Open Cursor Settings:
  • Go to File → Preferences → Cursor Settings
  • Navigate to the MCP tab
  • Click + Add to add a new MCP server
  1. Add MCP Server Configuration:
   {

"command": "uv", "args": ["tool", "run", "mcp-as-a-judge"], "env": { "LLM_API_KEY": "your-openai-api-key-here", "LLM_MODEL_NAME": "gpt-4.1" } }

📝 Configuration Options:

  • LLM_API_KEY: Required for Cursor (limited MCP sampling)
  • LLM_MODEL_NAME: Optional custom model (see Supported LLM Providers for defaults)
Claude Code
  1. Add MCP Server via CLI:
   # Set environment variables first (optional model override)

export LLM_API_KEY="your_api_key_here" export LLM_MODEL_NAME="claude-3-5-haiku" # Optional: faster/cheaper model

Add MCP server

claude mcp add mcp-as-a-judge -- uv tool run mcp-as-a-judge

  1. Alternative: Manual Configuration:
  • Create or edit ~/.config/claude-code/mcp_servers.json
   {

"command": "uv", "args": ["tool", "run", "mcp-as-a-judge"], "env": { "LLM_API_KEY": "your-anthropic-api-key-here", "LLM_MODEL_NAME": "claude-3-5-haiku" } }

📝 Configuration Options:

  • LLM_API_KEY: Required for Claude Code (limited MCP sampling)
  • LLM_MODEL_NAME: Optional custom model (see Supported LLM Providers for defaults)
Other MCP Clients

For other MCP-compatible clients, use the standard MCP server configuration:

{

"command": "uv", "args": ["tool", "run", "mcp-as-a-judge"], "env": { "LLM_API_KEY": "your-openai-api-key-here", "LLM_MODEL_NAME": "gpt-5" } }

📝 Configuration Options:

  • LLM_API_KEY: Required for most MCP clients (except GitHub Copilot + VS Code)
  • LLM_MODEL_NAME: Optional custom model (see Supported LLM Providers for defaults)

🔒 Privacy & Flexible AI Integration

🔑 MCP Sampling (Preferred) + LLM API Key Fallback

Primary Mode: MCP Sampling

  • All judgments are performed using MCP Sampling capability
  • No need to confi

Only the beginning is shown; read the rest in the repo