Test Genie

test-genie-mcp

Built for vibe coders: one command, get a prioritized list of what's actually broken about your project.

Self-healing test automation for iOS, Android, Flutter, React Native and Web apps — as an MCP server.

v3.1.1 — vibe-check + honest auto-fix. One MCP call, ~30 seconds: race conditions + security issues + memory leaks + logic errors + perf smells, prioritized. Stays on your machine, no telemetry. Pass autoFix: true for the small, safe mechanical fixes (weak-hash, simple Math.random assignment) — backup + syntax-validate + rollback-on-syntax-fail. For test-verified application of harder fixes, use v3.0.0's iterate-fix loop.

---

Vibe coders quickstart

You don't read the docs. You open the project, talk to Claude, and want a verdict. Here it is:

In Claude (with test-genie-mcp installed — setup):

/vibe-check /Users/me/my-app

Claude calls diagnose_project under the hood. ~30 seconds later you see:

# vibe-check report

• Project: /Users/me/my-app
• Platform: web
• Findings: 11 total — 4 critical, 4 high, 1 medium, 1 low
• Estimated fix time: ~85 min

Top 5 issues

1. [CRIT] Hardcoded AWS access key id found in source

• File: server.js:7
• Category: security / secret (CWE-798)
• Confidence: 95%
• Fix: Move the value to an env var, gitignore the config, rotate the leaked key.

2. [CRIT] SQL string built by concatenating user input

• File: server.js:21
• Category: security / injection (CWE-89)
• Fix: Use parameterized queries (db.query("... WHERE id = ?", [id])).

3. [HIGH] useState setter called after await without mount guard

• File: UserProfile.tsx:16
• Category: race-condition / react-setstate-after-await (CWE-362)
• Confidence: 78%
• Fix: Use AbortController and check signal.aborted before calling setters.

… (top 5 shown — full list at output: "detailed")

Next steps

1. Address the critical / high findings above.
2. Re-run diagnose_project after fixing to confirm convergence.
3. Use run_iterative_fix_loop for test-driven verification of each fix.

If any finding is autoFixable: true and is at high/critical severity, the diagnose_project call accepts autoFix: true to apply the mechanical replacement directly (with backup + syntax validation — see SAFETY.md for the exact guards). The v3.1.1 honest scope is narrow: weak hash (createHash('md5'|'sha1') → createHash('sha256')) and standalone Math.random() in security-sensitive files. For broader/structural fixes (race conditions, eval, exec injection) run run_iterative_fix_loop separately — it re-runs tests and auto-rolls-back on regression.

---

Why test-genie?

The bottleneck in mobile + cross-platform test automation isn't writing tests — it's the loop *between* a failing test and a passing test. test-genie closes that loop:

failing test → analyzer flags issue → fix proposed → dry-run + syntax check →

applied with backup → affected tests re-run → regression check → loop or stop

This full loop is the run_iterative_fix_loop tool. The diagnose_project autoFix: true path in v3.1.1 covers a strict subset — backup + dry-run + syntax-validate + apply, without re-running tests (so no test-regression rollback in that path). Use the right tool for the job — and see SAFETY.md for the exact guards on each.

Other tools (Detox, Maestro, Playwright, xcodebuild test) run tests. test-genie **runs tests *and* drives the fix until the bar is met or it can no longer make progress** — without you scrubbing through stack traces.

---

5-minute Quickstart

# 1. Install

npm install -g test-genie-mcp

2. Add to Claude Desktop config (~/.config/claude/claude_desktop_config.json)

{ "mcpServers": { "test-genie": { "command": "npx", "args": ["test-genie-mcp"], "env": { "TEST_GENIE_ALLOWED_ROOT": "/path/to/your/project" } } } }

3. Restart Claude Desktop. From a chat:

"Run the iterate-fix loop on /Users/me/my-rn-app with autoApply=false"

Expected output (truncated):

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Iterative fix loop f8b3… — PAUSED-FOR-CONFIRMATION ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Iterations completed: 1 Fixes applied: 0 Regressions rolled back: 0 Final tests: 7/10 passing (3 failing)

Pending confirmations (3):

• 71fbe…: Fix: useEffect missing cleanup for setInterval (confidence: 85)
• 92ad1…: Fix: Force-unwrap on possibly-undefined name (confidence: 85)
• …

Resume token: f8b3…

Re-call with autoApply: true (or resumeToken: "f8b3…") to actually patch the files.

---

Real use cases

The flows below describe the `run_iterative_fix_loop` path (v3.0

headline) — full detect → propose → dry-run → apply-with-backup →

re-run-tests → rollback-on-regression. The diagnose_project autoFix

path in v3.1.1 is the narrower mechanical-replacement-only path; see

SAFETY.md §4 for what that one actually touches.

1. React Native memory-leak self-healing

A team adds setInterval(...) in a useEffect and forgets cleanup. test-genie's detect_memory_leaks flags it, suggest_fixes proposes return () => clearInterval(id) (src/tools/fixing/suggestFixes.ts:169-179), the loop dry-runs the patch through the TS compiler, applies with backup, re-runs only the affected snapshot test, confirms 100% pass, stops. Before: 1 failing snapshot. After: 0 failing, 1 fix applied, 1 backup at .test-genie-backups/.

2. Flutter widget dispose() automation

AnimationController left undisposed. test-genie sees the missing dispose() override, generates a Dart @override dispose() { controller.dispose(); super.dispose(); } block (suggestFixes.ts:214-217), runs dart analyze on the patched file, applies, re-runs flutter test, converges.

3. iOS retain-cycle (closure capture)

self.timer = Timer.scheduledTimer(...) { _ in self.tick() } — rule-based detector flags closure self-capture, fixer rewrites to [weak self] _ in guard let self = self else { return }; self.tick() (suggestFixes.ts:239-242). If swiftc is on PATH the syntax check is real; otherwise test-genie reports "downgraded validation" so you know.

---

How the iterate-fix loop works

┌────────────────────┐

│ collect tests │ (run_scenario_test / supplied list) └─────────┬──────────┘ │ pass-rate ≥ threshold? ── yes ──▶ SUCCESS │ no ▼ ┌────────────────────┐ │ detect issues │ memory + logic analyzers └─────────┬──────────┘ │ ┌────────────────────┐ │ suggest fixes │ rule-based (default) → LLM (hybrid, optional) └─────────┬──────────┘ │ ┌────────────────────┐ │ dry-run + syntax │ TS compiler API / platform compiler / brace check └─────────┬──────────┘ │ ┌────────────────────┐ │ apply with backup │ per-file .test-genie-backups/ └─────────┬──────────┘ │ ┌────────────────────┐ │ re-run tests │ regression? yes → auto-rollback └─────────┬──────────┘ │ ▼ loop (≤ maxIterations, ≤ totalTimeout)

See [docs/ITERATE_FIX_LOOP.md](docs/ITERATE_FIX_LOOP.md) for a sequence diagram and the full safety-guard list.

---

Tools (23)

mode legend in [docs/SIMULATION_VS_REAL.md](docs/SIMULATION_VS_REAL.md).

Plus 4 resources (test-genie://iteration-logs, …/test-history/{path}, …/iteration-logs/{loopId}, …/applied-fixes/{path}) and 3 prompts (full-test-pipeline, diagnose-failure, vibe-check).

---

What vibe-check catches

Race conditions (detect_race_conditions / diagnose_project):

v3.1.1 honesty audit: useEffect-no-abort and forEach-await were

previously advertised as auto-fixable. They are not — wrapping with

AbortController or rewriting to Promise.all(arr.map(...)) changes

behavior we can't verify statically. They are now report-only. See

SAFETY.md.

Security (detect_security_issues / diagnose_project):

v3.1.1 honesty audit: .env/Math.random (general)/yaml.load were

previously advertised as auto-fixable. They were either too risky to

rewrite blindly or no strategy shipped — flipp