falsepass

falsepass

English | 中文

Probe RL environment verifiers for false passes (a known-bad completion is rewarded) and false fails (a known-good completion is not).

Website and docs: https://super-eric.github.io/falsepass/ · write-up: Your reward function is an assertion

In one public environment, echoing the question back earned full reward on every task: each question listed all of its candidate answers, and the grader only checked that the right one appeared somewhere in the reply, so a policy could collect full reward by listing everything. falsepass finds this kind of hole before training does.

An RL environment is a task dataset plus a verifier that turns a model's output into reward. falsepass treats that verifier as the system under test: it calls the environment's own reward code on completions whose correctness is known in advance and reports every disagreement. No model, no rollouts, no GPU.

It works with four frameworks and with plain reward functions, published or still in a local folder:

Install

Needs uv on the PATH (every audit runs in its own throw-away virtualenv) and Python 3.11 or newer.

uvx falsepass demo              # try it, nothing to install

uv tool install falsepass # or keep falsepass on the PATH pip install falsepass # or into an existing environment uvx --from git+https://github.com/Super-Eric/falsepass falsepass --version # the unreleased main branch

Quick start

falsepass demo                            # audits a bundled toy environment with one planted defect (about a minute)

falsepass init owner/environment # a Hub environment: package, version and verifiers pin read from the Hub falsepass init ./my-environment # or your own, from its folder (pyproject.toml); edits are picked up on every run falsepass audit my-environment # probe it; results in audits/my-environment/summary.md falsepass results my-environment --json # headline numbers and failing probes, for scripts and agents

demo writes the toy environment to ./falsepass-demo and audits it. Its grader pays when the reference appears anywhere in the answer, and the summary shows that one line from several sides: the static scan points at it, and the probes that list several answers (P5), retract the answer (P9) or deny it inside the answer slot (P15) are paid. A condensed excerpt of the real output (full file: examples/toy-env/summary.md):

- False-pass ceiling: 1 of 1 (highest reward given to a known-bad completion; via P5 shotgun/all_in_one_slot)

• Lazy-policy reward: 0 (best average reward of one fixed strategy that never uses the reference answer; via P3 empty/empty)
• False-fail rate: 0.108 (93 known-good completions)

| Probe | Variant | Expect | Runs | Failed | Score min / mean / max | | P5 shotgun | all_in_one_slot | zero | 10 | 10 | 1 / 1 / 1 | | P9 oversized | correct_then_retracted | zero | 10 | 10 | 1 / 1 / 1 | | P15 hedged_in_slot | denied | zero | 10 | 10 | 1 / 1 / 1 |

Static hint: falsepass_toy_env.py:26 reference_substring_match return 1.0 if answer in reply else 0.0 confirmed: P9 failed 10/20, P5 failed 10/30

Reward functions (TRL GRPOTrainer style)

falsepass init rewards.py --dataset data.jsonl                       # every function in the file that takes `completions`

falsepass init rewards.py --dataset hf:owner/name --split test \ --func accuracy_reward --func format_reward --weight 1 --weight 0.2 # which functions, with the trainer's reward_weights falsepass init trl.rewards --framework reward --dataset hf:trl-lib/DeepMath-103K --func accuracy_reward --with math-verify falsepass init train.py --dataset call:get_dataset --map to_prompt # a loader and a row mapper defined in the script itself falsepass audit rewards

Many graders are not packaged as an environment: a training script defines functions such as accuracy_reward(completions, solution, **kwargs) and hands them to GRPOTrainer with a dataset. falsepass calls each function the way the trainer does, one completion at a time, with every dataset column as a keyword argument (prompts, completions, completion_ids, trainer_state, log_extra, log_metric too), and sums the results with their weights, a None counting as nothing. A completion is a one-message list when the dataset's prompts are conversations or a function reads completion0, else text. The reference column is the one a function names (solution), else answer, solution, ground_truth, ... (--answer-column to choose); a worked solution is reduced to its last \boxed{} or what follows #### for the probes, while the functions always receive the column as it is. The dataset is a .jsonl/.json/.csv/.parquet file, hf:owner/name (streamed through datasets), or call:function in the file. The file's third-party imports are installed for the audit (--with adds, --without drops). examples/reward-toy/ has one planted defect.

OpenEnv

falsepass init hf:owner/space                          # a Space on the Hugging Face Hub: cloned next to the audit

falsepass init ./my_env # or a local environment folder (openenv.yaml, server/app.py) falsepass init hf:owner/space --answers refs.json # with reference answers: every probe runs falsepass audit space

An OpenEnv environment computes its reward in step(), so falsepass drives the environment class the way a session on its server does: reset() to a task, then step() with the reply as the action, in-process, with the Space's own requirements installed and no Docker. (The server's HTTP /reset and /step routes build a fresh environment per request, so they cannot grade a reset followed by a step.) Tasks: audits/ /tasks.json lists the keyword sets reset() is called with (most environments pick a task through task_id, task_name, difficulty or a seed; each set may carry an answer), else the Task API when the environment implements it, else repeated reset() calls, one task per distinct prompt; an environment that draws a task at random is reset again until the recorded one comes up. Without a reference answer only the answer-free probes run. A plain reply goes into the action's free-text field; other required fields (a decision enum, a flag) are filled with a constant, and O1 plays every value of such a decision field (a fixed strategy, reported as the lazy-policy reward) while O2 moves a self-reported field (confidence, score) between its minimum and maximum. A reply that is a JSON object naming the action's fields is the action itself: an adapter, or an answers file of such objects, fills several fields that way. examples/openenv-toy/ has one planted defect to try.

Inspect AI and HUD

falsepass init inspect_evals/gsm8k                     # a task from inspect_evals (pinned from PyPI)

falsepass init my_eval.py --task hard_split # a task file; --task picks one of several falsepass init ./my-hud-env # a HUD environment folder (env.py + tasks.py) falsepass init ./my-hud-env --answers refs.json # with reference answers: every probe runs

Inspect: the scripted model replaces only the call to generate(); prompt templates, system messages and answer parsing (multiple_choice()) are the task's own code. A solver or scorer that calls a model itself (self-critique, model_graded_qa) is flagged on every row instead of being scored. Tasks that need a sandbox are not supported.

HUD: a task template keeps its expected answer inside the grader, so without --answers only answer-free probes run: an empty reply, the prompt echoed back, and replies that list every integer from 0 to 1000, every letter, yes/no words or the prompt's words (A1; it also runs, with every other probe, when answers are given). A grader that pays those is a substring test in practice. The answers file is JSON: an object from task slug to answer ({"add-2-3": "5"}) or a list in task order (["5", "12"]). To add answers to an existing audit, rerun init with --force. examples/inspect-toy/ and examples/hud-toy/ each have one planted defect to try.

For environment authors: check before you publish

Run falsepass init ./your-env && falsepass audit your-env locally, or add the GitHub Action to the environment's repository so every push is checked:

# .github/workflows/falsepass.yml

on: [push, pull_request] jobs: falsepass: runs-on: ubuntu-latest steps:

• uses: actions/checkout@v5
• uses: Super-Eric/falsepass@main with: path: . # folder with the environment's pyproject.toml fail-on: "ceiling>0.1 errors>0" # fail when a known-bad answer earns over 0.1, or scoring raises

The step summary shows the probe table. Gate names: ceiling (highest reward any known-bad completion got), lazy (best average reward of one fixed strategy that never uses the answer), false_fail (share of correct answers refused), errors. Datasets that need a login or an API key at load time cannot be audited in CI without that secret.

For coding agents

• MCP server. falsepass mcp serves the commands as tools (falsepass_scan, _init, _audit, _audit_status, _results, _completion, _reach, _repro, _draft, _upstream, _verify_fix, _discover) over stdio. Audits run in the background, so a client's tool-call timeout never cuts one off; failures come back as tool errors with the reason. Any MCP client starts it with uvx --from "falsepass[mcp]" falsepass mcp; in Claude Code, for example: claude mcp add falsepass -- uvx --from "falsepass[mcp]" falsepass mcp.
• Skill. [skills/falsepass/SKILL.md](https://github.com/Super-Eric/falsepass/blob/main/skills/falsepass