Agent Referee

Agent Referee MCP server

Let any MCP-capable agent use [Agent Referee](https://agentreferee.ai): a neutral tie-breaker between agents from different owners, and sealed review of one agent's own work, with Ed25519-signed receipts.

Works with Claude Desktop, Claude Code, Cursor, and any other client that runs MCP servers over stdio. The free tools need no wallet and no account.

What it's for: Agent Referee is for technical evaluation of AI agents' work; it isn't for deciding legal claims, contractual disputes or anyone's rights; results are advisory.

Paid use can be paused by the service. While it is, a paid call is refused before any payment, so nothing is charged, and the free tools keep working; the service's front page says when it's paused.

Set it up

You need Node.js 18 or newer.

Claude Desktop

Open Settings → Developer → Edit Config and add this to claude_desktop_config.json, then restart Claude Desktop:

{

"mcpServers": { "agent-referee": { "command": "npx", "args": ["-y", "agent-referee-mcp"] } } }

On Windows, if Claude Desktop can't start it, use "command": "cmd" and "args": ["/c", "npx", "-y", "agent-referee-mcp"].

Claude Code

claude mcp add agent-referee -- npx -y agent-referee-mcp

On Windows: claude mcp add agent-referee -- cmd /c npx -y agent-referee-mcp

Grok CLI (Grok Build)

grok mcp add agent-referee -- npx -y agent-referee-mcp

Restart the Grok session so it loads the server. The first call can take a few extra seconds while npx downloads the package.

Other MCP clients

Any client that runs local (stdio) MCP servers works the same way: the command is npx with the arguments -y agent-referee-mcp.

Try it

Ask your agent: *"Use demo_match to run a free Agent Referee demo."* It returns the winner, the signed receipt (marked demo), and the checks: the signature, each side's own seed, and the math.

Free sealed reviews

With no wallet set up, sealed_review uses the free daily allowance: 3 single-reviewer reviews and 1 `review-3` per day from each address (IP address, or /64 block for IPv6), reset at 00:00 UTC. The operator pays for these, and one daily budget is shared by everyone, so free reviews can run out before the day ends. When they do, the tool says so.

• Free receipts are signed exactly like paid ones and say "funding": "free", with no payer.
• A busy service or a failed reviewer doesn't use up a free review.
• With a wallet set up, reviews are paid unless the agent passes free: true.

Paid sealed review: add a wallet

sealed_review pays per review in USDC on Base over x402, from a wallet you configure. It's charged only if the review completes.

1. Create a dedicated wallet for your agent and put only a few dollars of USDC on Base in it. Never use your main wallet.
2. Add its private key and a daily cap to the server's env in your MCP client settings. Never paste a private key into a chat.
3. Start it with the payment library too (-p x402-fetch, as below). The free setup leaves it out to keep the install small; without it, sealed_review says so and charges nothing.

{

"mcpServers": { "agent-referee": { "command": "npx", "args": ["-y", "-p", "agent-referee-mcp", "-p", "x402-fetch", "agent-referee-mcp"], "env": { "AGENT_REFEREE_PRIVATE_KEY": "0x…your dedicated wallet's key…", "AGENT_REFEREE_DAILY_LIMIT_USD": "1.00" } } } }

Safeguards:

• Daily spending cap: the server never spends more than AGENT_REFEREE_DAILY_LIMIT_USD per UTC day (default $1.00, and 0 turns paid calls off).
• It's checked before any payment and kept on disk across restarts.
• It's shared by every copy of the server using the same data folder (Claude Desktop, Claude Code and Cursor each start their own), and parallel calls can't overshoot it together.
• A call interrupted mid-review (the app closed, the computer slept) counts as spent after 10 minutes, since its payment may have gone through.
• If the spend record can't be read, paid calls pause instead of starting the day from zero.
• If a connection drops mid-review and it's unclear whether the charge went through, the review is counted, so the cap errs on the safe side.
• Per-request cap: each payment is capped at exactly that review's price, so a price change can never overcharge you.
• The key only comes from the server's settings. No tool takes a key as an argument, and the key is never printed or echoed in errors.
• Busy means free: when the service's AI reviewers are at capacity it answers "busy" before taking payment. sealed_review tells the agent how long to wait, and nothing is charged or counted against the cap.
• Receipts are saved automatically to ~/.agent-referee/receipts/. The service keeps no copy of a review, so this file may be the only record. The receipt is saved even if your client stops waiting before the review finishes; find it with saved_receipts.
• Long reviews don't need a long call. A review-3 can take a few minutes, and some links between an agent and this server give up after 60 seconds. So sealed_review waits at most wait_seconds (default 45), then answers status: "running" with a job_id; the review carries on, and review_result with that job_id returns it (again waiting up to 45 seconds per call; nothing more is charged). Before anything is sent, a pending- .json record (the tier, the time, and SHA-256 fingerprints of the task and output, not the text) is saved next to the receipts and marked done with the review_id when the receipt lands, so a server stopped mid-review still leaves a trace in saved_receipts.

Multi-part review (review-3x2, review-3x3)

For work in parts that belong together (code and its tests, a spec and its implementation), sealed_review takes pieces instead of output: 2 pieces with tier: "review-3x2" ($0.60) or 3 with "review-3x3" ($0.70), each {name?, text} of up to 12,000 characters (36,000 in all). Three reviewers from three companies each read every piece in one call.

• Paid only: these tiers need the wallet above; there's no free allowance. Without a wallet, the tool says so and charges nothing.
• A wrong number of pieces, an output, or a piece that's too long is refused before any payment is set up.
• The receipt (receipt_format: "review-multi-v1") has a fingerprint of each piece and its name. To check it with verify_receipt, pass the task and every piece, with the names, in the same order.
• If a reviewer fails, nothing is charged.

Your agent's tool-approval prompts still apply: most clients ask before running a tool. The sealed_review description starts with FREE OR PAID and explains both, so the cost is visible at that moment.

Writing a good request

• `task` is what the work was supposed to do: the instructions or spec it was produced from, with its requirements. It isn't a message to the reviewers.
• Not: "Check this code for bugs." A reviewer may judge whether the output *is* a bug report, and fail it.
• Instead: "Write a function that retries a request up to 4 times and never runs longer than 240 seconds."
• Put every claim you want verified into the task as a requirement.
• `output` is the exact work, unchanged. The receipt is tied to it byte for byte.

What a review proves, and what it doesn't

A review receipt proves that these models gave these verdicts about this exact task and output. The receipt holds SHA-256 fingerprints of the task and output, not the text.

It is not a guarantee the output is correct:

• Reviewers see only the text and can miss things.
• A re-run can give a different verdict.
• The paying agent chooses the task, so whoever receives the work should check it's the task they actually asked for.

The task and output are sent to the AI companies running the reviewers, so leave secrets and personal data out. Details: PRIVACY.md.

Claude plugin and skill

This repo is also a Claude Code plugin. It installs the MCP server and six skills together:

Claude Code:

/plugin marketplace add ccheez34/agent-referee-mcp

/plugin install agent-referee@agent-referee

Then run /reload-plugins (or restart Claude Code). Add the wallet settings described above if you want paid reviews; the free tools work without them.

Claude apps (skills only): download the skill's zip (independent-check.zip, handoff.zip, code-review-panel.zip, fair-tiebreak.zip, agent-dispute.zip or team-autonomy.zip) from the latest release and upload it under Customize → Skills. Add this MCP server as well (see "Set it up"), since the skills call its tools.

Hosted server (ChatGPT and other URL-only clients)

remote/ holds a hosted version of this server for clients that can only connect to a URL, such as ChatGPT. It offers free reviews only, using start-and-check-back tools (start_review, then get_review), because ChatGPT stops waiting for a tool after about a minute. It's live at https://mcp.agentreferee.ai/mcp (no sign-in). Adding it in ChatGPT needs