Dunstan

Dunstan

Verify an AI coding agent's pull request claims, with a verdict anyone can rerun. The agent says which files it changed, which tests ran and which issues it closed; Dunstan checks each claim against the pull request, its commits and its CI runs, and reports the verdict as a check run. Every verdict comes with a record: dunstan verify recomputes it offline and dunstan rerun re-reads the sources, so a reviewer or customer can check it themselves. Deterministic, no LLM.

Add it to a workflow; the agent's report goes in the pull request body (npx -y dunstan template --base main writes the block for you):

on:

pull_request: types: [opened, edited, synchronize, reopened, ready_for_review] permissions: contents: read pull-requests: read issues: read deployments: read checks: write actions: read jobs: handback: runs-on: ubuntu-latest steps:

• uses: BargLabs/dunstan@v0.1.16 # pin the full commit SHA in production with: report-source: pr-body

More in docs/action.md.

Dunstan holds a coding agent's completion report to the record.

An agent finishes and writes a handback: files changed, tests run, references closed, head commit, merged at. Dunstan reads a declared block in that handback and checks each claim against the repository's own record (pull request state, changed files, commits, CI runs) with deterministic checks. It writes a record anyone can re-run on their own machine and get the same verdict.

• No model decides pass or fail.
• Fail closed: a claim the record cannot answer is unverifiable, never a pass by absence.
• Evidence, not a gate: Dunstan returns a record; your branch protection decides what blocks.

The claim format, the block an agent declares and the record Dunstan writes, is an open specification: spec/claim-format.md, with JSON Schemas in spec/schema/ and examples in spec/examples/. A record is an in-toto Statement. How it reads in in-toto, SLSA and PER terms, and what does not carry over: docs/interop.md (informative; no export exists).

Use with Claude

Dunstan ships as a Claude Code plugin: three MCP tools and a skill that has Claude declare, check and report a pull request's handback before it says done (plugins/dunstan/).

Install. In Claude Code:

/plugin marketplace add BargLabs/dunstan

/plugin install dunstan@barglabs

The plugin runs the MCP server with npx -y dunstan@0.1.16 mcp, the dunstan npm package at that exact version: one self-contained file, no dependencies.

The token. When you enable the plugin, Claude Code asks for an optional GitHub token and keeps it in your system's secure credential store. Use a fine-grained personal access token with read-only permissions, limited to the repositories you check, with an expiry; the permissions are in docs/mcp.md. Leave it empty to read public repositories anonymously. Two kinds of claim need a token, even on a public repository: GitHub's GraphQL API refuses anonymous calls, so a closes reference is unverifiable, never fail; and GitHub answers an artifact download only with a token (401 "Requires authentication" anonymously, observed 2026-11-02), so a tests count is unverifiable (source_unreadable:artifact). The plugin never reads a token from your environment.

What it reads, and from where. The server runs on your machine and calls GitHub's API directly from it: the pull request, its changed files, commits, check runs, workflow runs and test artifacts, deployments, and the issues the block references. Every request is a read. Nothing is written to GitHub or to the repository, and nothing is sent to Barg Labs.

Try it. Three prompts on a public Barg Labs pull request:

1. Use Dunstan to check this handback for BargLabs/cejel#408: head commit 35d6201d00cf926f9af10df61bfec05b6624ee12, files changed .github/workflows/verify-published-windows-binary.yml and scripts/validate-distribution-metadata.mjs, merged at 2026-10-04T13:30:48Z. Every claim holds, so the verdict is pass.
2. Before I report BargLabs/cejel#408 as done, use Dunstan to list the claims in this report that my handback block does not declare: followed by the report and its dunstan-handback block. Dunstan lists each prose claim the block leaves out, such as a test count or a closes, with the field that would declare it. It is never a verdict.
3. Use Dunstan to check whether BargLabs/cejel#408 closes #407, with head commit 35d6201d00cf926f9af10df61bfec05b6624ee12 and the same two files. Without a token the closes row is unverifiable (source_unreadable:closing_references): the record could not settle it, which is not the same as false. With a token it is pass.

Support. GitHub issues or team@barglabs.ai. Privacy and terms.

The checker

pnpm install && pnpm build          # writes dist/dunstan.mjs and prints its SHA-256

Check a pull request against the report it names (exactly one report source).

GITHUB_TOKEN=... dist/dunstan.mjs check --repo --pr --report-pr-body --out record.json

or --report-file , or --report-comment-author (that author's latest comment)

dist/dunstan.mjs verify record.json # offline: recompute claims, verdict and digests dist/dunstan.mjs rerun record.json # online: re-read the sources, report evidence_changed

Exit codes: 0 pass, 1 fail, 2 unverifiable, 3 no record: a usage error, a pull request that cannot be read or that moved while it was checked and again when checked once more (pull_request_moved), or an internal error (verify: 0 verified, 1 not). --sign-key --signer writes the record as its JCS bytes with a detached SSH signature; verify --sig --allowed-signers checks it.

dist/dunstan.mjs mcp serves the same checker as an MCP tool, dunstan_check_handback, which an agent calls with its block before it says done, beside dunstan_suggest_declarations (below): docs/mcp.md.

checker.digest in a record is the SHA-256 of dist/dunstan.mjs, the single-file bundle that ran: the checker code, its dependencies and both schemas from spec/schema/, inlined at build time.

What Dunstan does not check

Dunstan checks the claims a handback declares against the record. It does not check that the handback covers what the issue asked for: an accurate report of the wrong work passes.

What to declare: docs/agents.md.

Survey: how much of a repository's handbacks can be checked

dunstan survey (DRAFT) runs dunstan check over a repository's recent closed pull requests and reports settleability: which claims the record could not settle, why, and what would let it settle them. It is read-only and needs no token to start.

dist/dunstan.mjs survey --repo <owner/name> --last 30 [--author <agent-login>] --out survey.json --summary survey.md

dist/dunstan.mjs verify survey.json # offline: every embedded record, the selection, the aggregate and the digest

It reports two rates, each with its numerator, denominator and Wilson 95% interval:

• the share of claims the record could not settle, (unverifiable + not record-checkable) / all claims, led by a table of the reasons, each with its remedy;
• the share of settled claims the record contradicts, contradicted / (supported + contradicted).

With --advisory it also runs the advisory prose reader and counts its notes in a table of their own. Advisory notes never enter either rate. A tests claim can be settled only with GITHUB_TOKEN set and a JUnit record that CI uploads: GitHub serves no artifact to an anonymous reader, and a test run CI never recorded, or a local run declared as one, is not record-checkable. The survey record says whether it read with a token, and an anonymous survey's summary says its cannot-settle share is higher for it. The survey measures what the record can settle. A survey record is unsigned: verify proves it consistent with itself and every embedded record valid, not that its listing is what GitHub returned, so to trust its completeness, run it again online. Details: docs/survey.md.

Declare before done

Checking a declared claim is exact. Finding the claims in prose is where readers fail: a hosted model judge clears far fewer false reports once it is handed the right paragraph, and Dunstan's own extractor missed the planted false claims it was tested on. Both fail at the reading, not at the checking. The agent that wrote the report is its best reader, and it is still there before handback. So Dunstan makes the block nearly free to write, then asks the author about the prose the block does not cover:

dist/dunstan.mjs template                         # the block for HEAD, filled from local git

dist/dunstan.mjs suggest --report-file report.md # prose claims the block does not declare

template fills headCommit and filesChanged (HEAD against its merge base with the remote's default branch, or --base). With --tests-ci it fills tests from the JUnit record CI published at HEAD, read as the gate reads it; with --tests-local it declares a local run as one (below). It never fills references or mergedAt, and when git cannot answer it prints no block. suggest runs the advisory extractor over the prose outside the fence and lists each claim it proposes that the block does not declare, to declare or reword. It is never a verdict, writes nothing to a record and always exits 0; the MCP server offers it as dunstan_suggest_declarations. The loop, an instruction block to copy into AGENTS.md or CLAUDE.md, and example output: docs/agents.md.

Make test claims checkable

Most claims about tests in a completion report ("all tests pass", "142 passed") cannot be checked from a repository's records today: the tests ran on the agent's machine, or CI ran them and published no counts. Dunstan settles a tests claim only against a JUnit file a CI run uploaded. Two changes make such claims checkable:

1. CI publishes the record. One upload step after the test command: an artifact named dunstan-junit holding junit.xml, uploaded even when the tests fail. Recipes for pytest, jest, vitest, Maven, Gradle and Go, with dunstan junit-merge for runners that write one file per test class: docs/ci-test-records.md.
2. The agent declares CI's counts. After CI has finished on its last push, dunstan template --tests-ci --tests-command --tests-workflow --tests-job reads the counts from that record, as the gate will, and declares them. A run made only locally is declared with --tests-local as {"kind": "local"}, which the gate reports unverifiable (local_run_not_in_record), never pass.

Reading the artifact needs a token: without one a tests row is unverifiable (source_unreadable:artifact). Each tests row the record cannot settle is printed with a remedy: line saying what to do.

The GitHub Action

One step in a workflow, one required check in branch protection:

permissions: { contents: read, pull-requests: read, issues: read, deployments: read, checks: write, actions: read }

steps:

• uses: BargLabs/dunstan@ with: report-source: pr-body # or comment: , or file:

It writes a dunstan check run on the head commit whose summary is the claims table, the record as the artifact dunstan-record, and the same table in the job summary. Only pass concludes `succes