five46

five46

Say what should work, in plain English. An AI agent checks it against your real app or API, using your own LLM key, and hands you a real Playwright test.

You just built something and want to know whether it actually works, before writing a test. Give five46 a goal like *"log in and confirm the dashboard loads"* or *"create a user via POST, then confirm it via GET"*. An LLM (OpenAI, Anthropic, Gemini, Groq or AWS Bedrock, with your key) drives your real app or API one action at a time. It tells you honestly whether the goal was met, with a root-cause hypothesis when it wasn't. A successful run is saved as a standalone Playwright (or node:test) spec, so today's check becomes tomorrow's regression test, and running it needs no five46 or LLM.

Quick start

npm install -g five46

npx playwright install chromium # one-time browser download five46 config # pick a provider and paste a key (Gemini's is free) five46 doctor # confirms the key works five46 test http://localhost:3000 --goal "log in as demo@example.com and confirm the dashboard says Welcome back"

You get a pass or fail (and a matching exit code), a report with screenshots and, for failures, a root-cause hypothesis, plus a five46-agent-*.spec.ts you can commit and run with npx playwright test. Testing an API instead? Use five46 api --goal "...". It's read-only unless you pass --allow-writes.

Status: early and actively developed. Verified end-to-end against dozens of real-world sites and APIs.

Check every pull request

The five46 GitHub Action reads the Acceptance criteria section of a pull request's description and checks each item against the PR's preview deployment. It keeps one PR comment up to date with the verdict per criterion, why each failure failed, and a ready-to-commit spec for every criterion that passed.

on: deployment_status   # Vercel, Netlify and other preview deploys

jobs: five46: if: github.event.deployment_status.state == 'success' runs-on: ubuntu-latest permissions: { contents: read, pull-requests: write } steps:

• uses: sekharsdet/five46-action@v1 with: url: ${{ github.event.deployment_status.target_url }} api-key: ${{ secrets.FIVE46_API_KEY }}

How it compares

Already working in an AI coding tool and growing a Playwright suite there? Playwright's own agents are a great fit, especially the healer for keeping existing tests green. five46 is for getting a verdict: is this feature working right now, in CI or on a PR, for a browser flow or an API, with any LLM you already pay for. More detail is in Why five46.

Highlights

• Safe by default. API runs are read-only unless you unlock writes; destructive-looking clicks ("Delete account") are blocked unless allowed.
• Login credentials never reach the LLM. It sees placeholders, and the real values are filled in locally. Log in once, then reuse the session.
• Acceptance criteria in, verdicts out. --story splits a user story or ticket into independent scenarios and reports pass/fail per criterion.
• Specs you can trust. --verify-clean-session re-runs the spec cold; --verify mutates its assertions and requires them to fail.
• Works with any current key. If your provider retires a model, five46 finds a current one your key can use. You can also pin one with FIVE46_LLM_MODEL.
• Fits your tools. CI-friendly exit codes, --report JSON for dashboards, flaky-run detection (--repeat), video recording, and an MCP server for IDE assistants.

Docs

• Configuration: providers, getting a key, five46 doctor, choosing models
• CLI reference: every command and flag, login sessions, API testing, story mode, verification, reports
• MCP server: use five46 from an IDE-embedded AI assistant
• Security model: what leaves your machine, what five46 guards, and what's your call
• Why five46: design choices, the full feature list, comparisons
• CHANGELOG

Known limitations

• **No *closed* shadow DOM traversal.** Elements inside a mode: "closed" shadow root aren't visible to the agent's page snapshot (a closed shadow root is deliberately inaccessible to external tooling in general, not just five46). *Open* shadow roots, the overwhelming majority of real-world web components, work fine, as do and content.
• Chromium only for browser mode, with no Firefox or WebKit yet.
• A run itself is not deterministic. The same goal against the same page can take a different path next time. The generated spec is the frozen, repeatable artifact.

Contributing

Issues and pull requests are welcome. See CONTRIBUTING.md.

git clone https://github.com/sekharsdet/five46.git

cd five46 npm install npm run build node dist/cli.js test --goal "..." npm test

If five46 is useful to you, a ⭐ on GitHub helps other people find it.

License

MIT