Verdict QA

Verdict

CI · verdict on itself · eval: 8/8 seeded defects · pinned rules: 527/527 killed · PyPI: verdict-qa-mcp · OpenSSF Best Practices: passing · MIT license

Your test suite is green. Verdict found a defect that had lived 4,595 days.

Verdict is a QA agent that does not fix, does not flatter, and does not forget. It measures before it judges — the harness runs your gates, hashes every line a finding cites, re-runs the guarding test at the old commit and the new one — and it keeps a memory: every run is a delta against the last, findings age, regressions rank first, and the tester's own misses are published beside its hits. The contract it runs under is immutable and hashed into every verdict; what it learns lives beside the contract, dated and auditable, and never edits it. The number above is real: FilePerms in a 7k★ Python library could not revoke a permission bit since 2014-02-07, and every one of its 625 tests was green the day Verdict filed it — the run, and the misses, are in the ledger.

/plugin marketplace add ArtJack/verdict     # Claude Code

/plugin install verdict@verdict /verdict:run

```

npx skills add ArtJack/verdict # every other coding agent

Most AI "QA agents" are a paragraph of enthusiasm with a checklist. They audit your repo from scratch every time, re-report the same 20 findings until you stop reading, call flaky tests "failures", call stale tests "failures", and end with "LGTM! 🎉".

Verdict is a Claude Code plugin built the way QA is actually practiced:

• It remembers. A state file carries the baseline. Every finding gets a stable ID and an age; every run reports NEW / STILL_OPEN / RESOLVED / REGRESSED — regressions ranked first, always.
• A red test means something. Every failure is classified — REAL_DEFECT, STALE_EXPECTATION (which needs a citation proving the change was intended), BRITTLE_TEST, ENVIRONMENT, or FLAKY (confirmed by re-runs, quarantined with an expiry). The classification most likely to excuse a regression carries the highest evidence bar.
• A verdict you can defend. Every run ends in exactly one of pass | pass with risks | blocked | fail — an open Blocker forces fail, blocked is a legitimate outcome, and a pass always names what was *not* tested.
• It never fixes your code. There is no Edit tool, a hook confines its writes to the QA root, and a strict-mode Bash guard closes the shell's write channels — a tester that patches what it judges isn't independent. The guard is a heuristic, not a sandbox, and the README says so.
• It is tested, and it tests itself. A scored eval suite with the misses published, a hash-chained run history a copied or edited state fails, a track record the harness computes rather than the tester — and an audit of its own releases: 78 findings filed against itself in its first 15 runs — 59 fixed with the fix verified, one an accepted risk on the record, two still open, and 16 closed without proof, which its own ledger counts as unknown, not as wins. Every fixed harness rule is pinned as a mutant the suite must kill.

What a run looks like: a real delta report — Verdict on its own code, run 14.

Who pays for the model? You do, with the Claude subscription you already have: the plugin runs inside your own session, nothing routes through anyone else, and everything below the model — the state, the gate, the harness, the eval scorer — is stdlib Python (the optional MCP server adds the mcp SDK) that runs for free. Works on Python, TypeScript, Go, or anything with a test runner; the eval fixtures cover Python and TypeScript.

Read next: Install · What installs, and when it runs · Quickstart · Why another QA agent · The tested tester · CI gate · Accepting a risk · FAQ

Install

/plugin marketplace add ArtJack/verdict

/plugin install verdict@verdict

Any other coding agent — Cursor, Codex, OpenCode and the rest of the agent skills ecosystem — gets the same doctrine as five skills, and the same harness as a pip package:

npx skills add ArtJack/verdict        # release risk · verify a fix · flaky triage · root cause · spec review

pip install verdict-qa-mcp # verdict-facts, verdict-finalize, verdict-gate, verdict-accept, verdict-answer, verdict-replay, verdict-dupes

Listed on skills.sh.

The skills restate the contract for an agent that cannot run the verdict agent; the hooks that enforce the read-only guarantee exist only in Claude Code, so there the guarantee is the agent's own discipline plus the harness's refusals. AGENTS.md and llms.txt at the repository root are for agents that read before they act.

Python 3.9 or newer, whatever your python3 resolves to — the hooks and the fact harness are stdlib-only and are invoked by that name, which on a stock Mac is /usr/bin/python3 (3.9). The optional MCP server is a pip install and needs 3.10+, which is what requires-python in pyproject.toml refers to. The floor is tested: a module that would fail to import on 3.9 fails CI instead (tests/test_interpreter_floor.py) — because the failure it prevents was silent. The Bash guard once raised on import there while the write guard beside it kept working, so a strict session looked armed with half its controls missing.

What installs, and when it runs

Installing a plugin means letting its code run in your sessions, so here is exactly what this one does — measured from hooks/hooks.json, not summarised from memory. Six hook registrations; each starts a python3 (tens of milliseconds) when its event fires:

Every hook fails open: malformed input, missing files, or an exception mean exit 0 and silence — a broken hook must never brick a session. VERDICT_STRICT=1 is what arms the scope guards, and you set it only for dedicated QA sessions (headless, CI, the nightly); in ordinary interactive work the guards are no-ops.

Prefer not to install globally? Everything works per-repository: copy agents/verdict.md into /.claude/agents/ and the hooks/hooks.json entries into /.claude/settings.json, with ${CLAUDE_PLUGIN_ROOT} replaced by a checkout path. That is exactly how the eval harness provisions its scratch projects — eval/run_eval.py is the reference implementation.

Quickstart

/verdict:run                          # first run: profile + isolation rules + baseline

/verdict:run the payment retry change # every later run: a delta against the stored state

Every later run is a delta against the stored state. A repeat run returns something like:

VERDICT: fail

Scope: 2c67f47..b4e2943 (4 commits, 16 files) · run 4 (delta) Isolation check: pass (no .env present; no live service touched)

Findings — REGRESSED first: REGRESSED PRICER-F-002 Critical/P0 round_cents uses banker's rounding again (pricer.py:17) resolved 08-19, reintroduced by b4e2943 — this forces the verdict NEW PRICER-F-007 Major/P1 quarantine graveyard: test_listable_at_floor_exactly skipped 114 days with no expiry — it is the test that would catch F-001 STILL_OPEN PRICER-F-001 Critical/P0 age 6d is_listable rejects a price exactly at the floor FLAKY test_bulk_discount_applies — fails 3/6 runs with no code change; quarantined until 2026-09-07, excluded from this verdict, listed until re-evaluated

Delta gates: tests 213 → 213 · duration +0.4% · coverage on changed files: no decrease Release blockers: PRICER-F-002 (regressed), PRICER-F-001 Not tested: concurrency under parallel checkout — no harness present Fix order: 1) F-002 2) F-001 (unskip its test first, watch it fail red) 3) F-007 expiry Artifact: .qa/reports/2026-08-24-pricer-review.md

Why another QA agent

| AI-authored code | same