Tewip

Tewip

Tewip (Uyghur *téwip*, تېۋىپ: healer, doctor) is the layer that proves software work - written by a person or by an agent.

Created and maintained by Mamuti Aishan (Hasan) (@hmamut39).

It finds why each failing test failed (a real bug, a flaky test, the environment, test data, or a changed selector), says whether a test could ever fail at all, breaks your code to find out whether the tests would notice, reads what a migration will do to a live database, and gives one verdict on whether a change is done. Every answer carries its evidence and names what decided it: a deterministic rule, the AI model, or a person's correction.

What makes it worth installing is what it refuses to say. It answers *"cannot say"* rather than guessing, it will not narrow a test run it cannot prove safe to narrow, and every report ends with what it did not check. Each number below was measured on strangers' real code, never on fixtures.

It works for every stack: Node.js (node:test, Jest, Vitest, Mocha), Python (pytest, unittest), Java and Kotlin (JUnit, TestNG), Go, .NET, Ruby, PHP and Rust, each checked against its real runner's report (the full list), plus Playwright in depth with traces, selector drift and verified fixes. It also reads the failure wording of the browser and component-test tools on top of those runners (Selenium, WebdriverIO, Cypress, Testing Library for React, Vue and Angular, Vue Test Utils, jest-dom and Jasmine), checked against more than 500 real messages from public projects.

Formerly *triage-agent*: the repository moved to hmamut39/tewip, and GitHub redirects the old address. See ROADMAP.md for the plan.

Status: built, and measured on other people's code. Tewip collects failures from any framework, diagnoses them with rules first and a model only for what rules cannot settle, and turns that into work for each role: a CI comment and merge gate, test plans and verified tests, a flake dossier, a PR-risk review, security scanning, tickets in Jira, Rally, GitHub Issues, Azure Boards or Linear, checks generated from a Figma design, specifications written for people, and - opt in, and only when proved - fixes to test or application code. The fix loops are built and measured, not planned: on real labelled CI failures every one of 17 locator suggestions was the locator the developers themselves wrote, a wait fix is kept only after three passes in a row, and a change to application code is kept only when the failing test passes and the whole suite still does. It runs live in GitHub CI and answers questions from AI assistants through MCP. On fresh real Gutenberg CI failures, with the pipeline frozen before labelling, accuracy is 93.4%; across six real datasets it ranges from 86.1% to 99.7%, and the caveats are in eval/README.md.

If your CI already writes a coverage report, point Tewip at it

Tewip decides what a test reaches by following imports, which is a fact rather than a name match - and which cannot see a test that posts a GraphQL query, hits an HTTP route or renders a controller. On a Django, Rails or Spring codebase that is most of the suite, so tewip done used to answer *"cannot say"* to the question that matters most.

A coverage report settles it, and CI almost certainly produces one already. Nothing to configure:

With one present, done stops guessing:

### Is the changed code tested?

Yes: the suite executes saleor/graphql/utils/sorting.py.

_Decided by coverage.xml, which records what ran rather than what imports what._

and it will say the opposite just as plainly - *"is in coverage.xml with no line executed, so the suite runs none of this"* is a not ready, not a shrug.

What a coverage report cannot do. It records which lines ran during the whole suite, not which test ran them, so it answers *"is this code tested"* and not *"which tests would notice a change here"*. tewip pick still narrows from imports and still refuses to narrow when it cannot show the rest are unaffected. A report from a different checkout is ignored rather than believed: if the files it names are not on disk, Tewip says nothing instead of reporting the whole change as untested.

Running with no data leaving the building

A hospital, a bank or a government department usually cannot send source code to a hosted API, and that is most of who this is built for. Tewip runs against a model on the same machine:

# once

curl -fsSL https://ollama.com/install.sh | sh # or ollama.com for Windows and macOS ollama pull qwen3:8b

then

export TEWIP_PROVIDER=ollama tewip done --diff main...HEAD

No key, no account, no network call off the machine. TEWIP_BASE_URL points the same path at LM Studio, vLLM, llama.cpp's server, or anything else that speaks the OpenAI shape.

What changes and what does not. The deterministic layers - vouch, schema, access, perf, standards - call no model at any time, with any provider, and are unaffected. Only the parts that need judgement use one: explaining a failure, writing a test, drafting a plan. A smaller local model is less accurate at those than a frontier one, and Tewip says which layer decided every answer, so the difference is visible rather than hidden.

A local model is also less likely to be *constrained* to the schema it was asked for: some backends accept the request and quietly do not enforce it. Tewip reads a reply that arrives inside a markdown fence or after a sentence of preamble, and still refuses one with no JSON in it at all.

What it costs

Most failures never reach a model. On 3,556 real CI failures from six public projects, the deterministic rules answered 75.1% on their own - and on the labelled ones they answered correctly 1,979 times out of 2,012 (98.4%). A rule-answered failure costs nothing, explains itself, and gives the same answer every time.

Only the remaining quarter is sent to a model, so triaging 1,000 real failures costs about $1.12 at gpt-5.4-mini's published price. That is measured, not estimated: three real uncached calls on held-out failures billed 3,624 prompt and 499 completion tokens each. A real week costs less, because a failure that repeats is answered from cache, and a failure someone has corrected never goes to the model again.

Tewip prints what every command costs, caps its own spend, and works with no key at all - the rules run on their own and everything they cannot settle is left as UNKNOWN rather than guessed.

Start here

npm i -g tewip-ai

cd your-project tewip

tewip on its own looks at your project and names the two or three commands that will tell you something, with your own paths in them — and says plainly what it cannot answer here. It reads the directory and stops: nothing is run, nothing is written, about a second.

Here is what Tewip can tell you about **this** project, in the order worth doing.

1. `tewip vouch` 282 test files here (282 JavaScript or TypeScript). This reads them and names the ones that cannot fail - no model, a second or two.

2. `tewip schema --diff main...HEAD` 165 migrations here (165 SQL). This says what a change would do to a live database before it runs.

What it will not do here

• No k6, JMeter or Lighthouse report was found. Tewip judges one you produce; it does not run load tests itself.

Everything below is the full list, for when you want it.

Use it in your project

Giving it to your team (or another company): [TEAM-GUIDE.md](TEAM-GUIDE.md) — what to install, what each role uses it for, and what it never does.

Putting it on a real repository: [QUICKSTART.md](QUICKSTART.md) — ten minutes, copy-paste, and what to check on the first failing run.

1. In GitHub CI (developers, SDETs, DevOps)

Tell Playwright to write a blob report in CI. It's a built-in reporter, so there's no triage-agent code in your config:

// playwright.config.ts

reporter: process.env.CI ? [['list'], ['blob']] : 'list', use: { trace: 'retain-on-failure' },

Then add one step after your tests:

permissions:

contents: write # only needed for autofix pull-requests: write issues: write actions: read # lets it read the failed job's log when no test report was written

steps:

... checkout, install, npx playwright test ...

• name: triage-agent if: always() uses: hmamut39/tewip@main with: openai-api-key: ${{ secrets.OPENAI_API_KEY }} # optional: without it only the rules run autofix: 'true' # optional: verified locator-fix PRs

Run the verdict verbs in CI (Phase 38, new) — one step, as many checks as you want:

- uses: hmamut39/tewip@main

with: mode: check checks: schema, access, vouch # or: done, standards, perf, or all gate: blocking # optional: fail the job on what a verb calls blocking

Every check runs even when an earlier one found something, so a pull request gets the whole picture in one push. The gate is deliberately narrower than the report: it fails only on what the verb itself calls blocking, because a gate that fires on advice teaches a team to pass continue-on-error. A check with nothing to look at says so and never fails the job — no migrations in the diff is not a pass.

Use `@main` or `v0.46.0` or later. Every tag before that shipped an action.yml whose

description contained an unquoted colon, which YAML reads as a nested mapping — the runner answered

*"Failed to load action.yml"* and the action never ran. It is fixed, and there is now a test over

the manifest so it cannot come back.

On every run, the action:

• Developers: posts one diagnosis comment on the pull request and keeps it updated. For each failure it says whether the change likely caused it, why, and what to do next.
• SDETs: writes the triage board into the job summary.
• DevOps: collects environment failures (server errors, connection resets, outages) in one GitHub issue, and each new run adds a single grouped comment to it.
• The tests in the pull request: on a PR it reads the test files the change touched and say