Elliot

Independent verification for code written by AI agents.

Elliot checks what your agent claims, and what your README promises, against the code. Every finding comes with proof.

Quick start · What it found · How it works · Commands · Configuration · FAQ

Quick start

In the repository you want to check:

npx -y github:EmanueleCell4/elliot-mcp init

Restart your agent, then type:

/elliot-audit

A minute or three later the report is in elliot-report/ - / at the root of the repo: report.html, report.md and claims.json. There is nothing to open, no dashboard and no account.

Requirements: Node 20 or newer, git, and Claude Code, Cursor or Codex.

As a Claude Code plugin

Inside Claude Code:

/plugin marketplace add EmanueleCell4/elliot-mcp

/plugin install elliot@elliot-mcp

The first command registers Elliot's plugin marketplace; the second installs Elliot. After installation, run /elliot:audit in your project. This is an opt-in local tool and does not automatically post reports or install itself in other repositories.

That adds the Elliot tools, a /elliot:audit command, and a skill that makes Claude verify its own "done" claims before reporting them.

On every pull request (GitHub Action)

Elliot reads the pull request description, checks each claim against the code ("fixes X", "adds tests for Y", "no behaviour change"), and comments with the proof. If an agent wrote the PR, this is where its "done" gets checked.

# .github/workflows/elliot.yml

name: Elliot on: pull_request: types: [opened, edited, synchronize] permissions: contents: read pull-requests: write jobs: check: runs-on: ubuntu-latest steps:

• uses: actions/checkout@v4 with: fetch-depth: 0
• uses: EmanueleCell4/elliot-mcp@v0.7.3 with: mode: strict anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}

The comment looks like this:

It runs Claude Code with your API key. One check usually takes 1–2 minutes. Nothing is sent anywhere except Anthropic's API and GitHub.

mode: strict (shown above, available from v0.7.0) fails the check for findings, missing evidence, skipped audits and inconclusive results, so a merge can be blocked on it: add the job as a required status check in your branch protection rules. Drop mode to get the non-blocking report instead. A ready-to-copy workflow is in examples/workflows/elliot-strict.yml. Do not enable enforcement on forked PRs without reviewing credential exposure and runner isolation.

README badge

When every claim Elliot checks holds up, the report includes a badge you can paste into your README:

Why

Coding agents say "Done ✅" when the work isn't done. They delete failing tests, skip checks, describe security that doesn't exist, and write READMEs that promise what the code never does. CI stays green because tests check the code the agent wrote, not the promises the product makes.

I found this out the hard way. My own legal-tech app had a security page promising role-based access enforced in the database. The data layer bypassed it by design, and every permission check was a function that ignored its input. The agent had told me it was done.

The usual fix is to ask a second AI to review the first one. That gives you another opinion from the same kind of system that made the mistake. In the first automatic audit I ran, the agent doing the checking cited line numbers that don't exist in 4 of 17 submissions.

Elliot works the other way around: the agent proposes, Elliot verifies. Every quote must really be in the file, at that line, at that commit: the audit fingerprints the tree when it starts (tracked changes and untracked files included) and refuses evidence once anything changes, and symlinks that lead outside the repository are not evidence. Every "this doesn't exist" is searched by Elliot itself. A fresh reviewer then tries to prove each finding wrong before it reaches the report.

What it found

Elliot 0.3 ran unattended inside Claude Code on 9 popular open-source tools from the AI coding-agent ecosystem (October 2026). A separate reviewer that had not seen the audits then checked every finding against the code.

"Wrong" means a factual error: the code does not show what the finding says.

Some of what it caught:

• A tool whose README says "No Telemetry: no data collection or tracking" ships analytics that are on by default, with consent pre-set to "yes". (public issue)
• An app where logging in turns telemetry back on after the user switched it off, then identifies the user by email. (public issue)
• A security page that calls a script sandbox "read-only" while the code can give scripts file writes and MCP tools. Confirmed by the project's triage bot, with a docs fix sent upstream. (issue, PR)
• Docs saying an API key is stored encrypted while the code writes it as plain JSON.
• A popular agent tool that sends users' full AI prompts and responses to a third-party error tracker by default, under an option named "anonymousTelemetry". Reported privately to the maintainers; details will be published after a fix.

What it still misses: on the same 9 repos it found 6 of the 10 serious issues I had found by hand beforehand (a small, hand-picked set, so read it as a rough coverage figure, not a recall estimate). The most common miss is a missing authorization check (for example, a download endpoint that doesn't check who is asking). Elliot doesn't look for those yet; see the roadmap.

Benchmark: Claude with Elliot vs Claude alone

Eight widely used SDKs, 18 broken promises planted by flipping a value behind a documented default, two runs per condition, same model and instructions. A blind judge checked every finding that was not a planted bug.

Elliot raises about a third as many false alarms and finds slightly more planted bugs; Claude alone casts a wider net. When the two blind judges (Claude and Codex) disagreed, reading the code showed Codex had misstated the code 9 times: a second model is not a check unless something re-reads the source. Method, limits and all raw outputs: benchmark/planted-promises.

Benchmark: can anyone re-check what the agent says it verified?

56 real bug fixes (28 tasks, Opus and Haiku). Every fix was correct and no agent claimed "done" falsely. But in 19 of 56 runs (34%) the check the agent said it ran could not be re-run by anyone else: a deleted scratch script, a check described in prose, a path on the agent's own machine. The work was right; the evidence was gone. benchmark/done-claims

How it works

1. Risk sweep (Elliot). Before the agent reads anything, Elliot scans the docs for risky promises (telemetry, privacy, encryption, access control, sandboxes, "never", "by default") and the code for signals that often contradict them: analytics SDKs, permission bypasses such as --dangerously-skip-permissions, privileged database clients, secrets written to disk. The audit cannot finish until every one is addressed or skipped with a reason.
2. Read and propose (your agent). Your agent traces each claim into the code and proposes a verdict, quoting the docs and the code.
3. Verify the proof (Elliot). Each quote is matched against the file at the pinned commit, within two lines of where the agent said it is. Each "absent" claim is searched by Elliot with git grep. Citations that don't check out are rejected, and the agent has to fix them.
4. Materiality test. A FALSE or PARTIAL verdict must say what a user who trusts the docs would get wrong. Omissions, naming details and tiny delays become notes, not findings. Changelogs, roadmaps and internal developer docs are rejected as sources, because they are not promises to users.
5. Independent review. A fresh subagent that never saw the audit tries to prove every finding wrong, and confirms, downgrades or retracts it. Elliot will not write the report until every finding has been reviewed.
6. Report. HTML, Markdown and JSON in elliot-report/, plus a tamper-evident receipt for every check in .elliot/receipts.jsonl.

If you wrote code in the same session, the protocol tells the agent to run the audit in a fresh subagent, so the auditor is not the author.

Example report

The full report is in examples/lex-ai-report.html. It was produced by an earlier version, before the independent review step existed.

Commands

Set-up: init

npx -y github:EmanueleCell4/elliot-mcp init            # every agent it detects

npx -y github:EmanueleCell4/elliot-mcp init --claude # only Claude Code npx -y github:EmanueleCell4/elliot-mcp init --cursor # only Cursor npx -y github:EmanueleCell4/elliot-mcp init --codex # only Codex npx -y github:EmanueleCell4/elliot-mcp init --all # all three

init is idempotent: running it twice changes nothing, and it never overwrites a comman