Honestcode

HonestCode

The deterministic verification layer for AI coding agents.

Catch invented APIs, undefined symbols, wrong calls, and other code hallucinations before your agent moves on.

---

The problem

AI coding agents hallucinate. They invent function names, fabricate library APIs, and call methods that do not exist.

HonestCode is the layer that checks what the agent just wrote against the repository it is supposed to be grounded in.

Agent

↓ writes code ↓ HonestCode ↓ evidence ↓ Agent fixes ↓ verified code

No LLM calls. No network. Pure AST + symbol resolution.

---

Demo

The agent was asked to *"add authentication using the existing UserClient."* It generated:

client.refresh_token()

The real project only has:

class UserClient:

def refresh(self): ... def refresh_access_token(self): ...

HonestCode catches it deterministically:

✗ verification failed — 1 issue in app/login_broken.py

[invented_api] app/login_broken.py:9 UserClient.refresh_token() does not exist. did you mean: refresh_access_token()? available methods:

• refresh()
• refresh_access_token() confidence: deterministic · action: revise

Run it yourself:

cd demos/invented-api

python run.py

---

Why not just use Ruff / Pyright?

HonestCode is not a replacement — it is a verification layer that answers a specific question: *did the agent's code actually come from this repository?*

The key difference is repository grounding. Ruff checks the file. Pyright types the call. HonestCode checks whether the call resolves to a real symbol that exists in the codebase the agent is editing. (HonestCode does not do full type inference — its structural checks catch mismatched arities, iterating None, and calling constants, not general type errors.)

What a call site is checked against depends on how the receiver's type is known: an annotation (client: UserClient), a constructor assignment (client = UserClient()), self/cls, and an inline constructor (UserClient().refresh()) are all resolved. A receiver produced by a factory function (make_client().refresh()) is not judged — function signatures carry no return type in the surface model, and guessing there would trade a missed finding for a false one.

---

Install

One line — no clone, no virtualenv:

claude mcp add honestcode -- uvx honestcode-mcp

uvx fetches the server into an isolated environment on first use; install uv first if you do not have it. From a checkout (to hack on the code, or if you prefer not to use uv):

git clone https://github.com/Fengrru/honestcode

cd honestcode pip install -e . claude mcp add honestcode -- honestcode-mcp

Cursor / VS Code / Claude Desktop: the honestcode install command probes those clients' config files and registers the server for you:

honestcode install            # all detected clients

honestcode install --client cursor --dry-run # preview without writing

Requires Python >= 3.10. 100% local.

---

30-second setup with Claude Code

# Add HonestCode as an MCP server

claude mcp add honestcode -- honestcode-mcp

Or add it manually to your Claude Code config (~/.claude.json on macOS/Linux, %USERPROFILE%\.claude.json on Windows):

{

"mcpServers": { "honestcode": { "command": "honestcode-mcp", "args": [] } } }

Restart Claude Code. You now have two verification tools available:

• scan_file(path) — legacy tool, returns issues plus the new evidence shape.
• verify_file(path) — agent-facing tool, returns status, findings with evidence, and a human-readable text summary.

Try prompting Claude:

Add a login endpoint using the existing UserClient, then run verify_file on the

file you just wrote and fix anything it reports.

HonestCode auto-discovers the project root from the file path, indexes the codebase, and returns grounded evidence.

Making it automatic (no prompt discipline required)

MCP tools only run when the model chooses to call them. To make verification unconditional, wire the hook in scripts/claude_hook.py into Claude Code's PostToolUse event — every .py file the agent writes is verified, and a finding blocks the edit with the evidence fed straight back into the conversation:

{

"hooks": { "PostToolUse": [ { "matcher": "Write|Edit", "hooks": [ { "type": "command", "command": "python /absolute/path/to/honestcode/scripts/claude_hook.py" } ] } ] } }

Put that in ~/.claude/settings.json (all projects) or .claude/settings.json (one project). The hook never blocks on its own failures: unreadable input, non-Python files, and verification crashes all exit silently, so it can never turn into a wall that blocks every edit.

---

Real example

# auth/client.py

class UserClient: def refresh_access_token(self): ...

auth/login.py

from auth.client import UserClient

def login(client: UserClient): client.refresh_token() # invented API

>>> from honestcode.mcp.tools import verify_file

>> verify_file("auth/login.py")

{ "status": "fail", "file": "auth/login.py", "findings": [ { "line": 7, "kind": "invented_api", "symbol": "refresh_token", "owner": "UserClient", "message": "UserClient.refresh_token() does not exist.", "evidence": { "available_methods": ["refresh", "refresh_access_token"], "did_you_mean": "refresh_access_token" }, "confidence": "deterministic", "action": "revise" } ], "text": "✗ verification failed — 1 issue in auth/login.py\n ..." }

---

Agent output protocol

verify_file returns evidence an agent can act on directly, not just an error message:

{

"status": "fail", "file": "auth/client.py", "line": 42, "kind": "invented_api", "symbol": "refresh_token", "owner": "UserClient", "message": "UserClient.refresh_token() does not exist.", "evidence": { "available_methods": ["refresh", "refresh_access_token"] }, "confidence": "deterministic", "action": "revise" }

---

How it works

1. Auto-index the project (or reuse the cached symbol index).
2. Parse the target file with the standard-library ast module.
3. Resolve every call site to a concrete symbol:

• infer the receiver type from annotations, constructors, and imports;
• reconstruct the class's member surface from the repository AST;
• mark the surface as *unknown* when a base class cannot be resolved or the class defines __getattr__.

1. Emit findings with kind, owner, evidence, confidence, and action.

The loop is deterministic and auditable.

Scope and limitations

The verifier stays silent wherever it cannot *prove* absence — that is the feature, but it means detection has known boundaries:

• Decorated classes are not judged for invented members. Any class decorator (including @dataclass) may add or rewrite members, so invented method calls on such classes are not reported. Undecorated classes are fully checked.
• Standard-library member calls are not checked. from typing import TypeVarr (an invented *import*) is caught, but os.path.joinn() (an invented *call* on a real module) is not — third-party and project members are the covered surface.
• Factory-function receivers are not judged. make_client().refresh() is skipped; annotations, constructor assignments, self/cls, and inline constructors (UserClient().refresh()) are resolved.
• `undefined_symbol` carries `confidence: high`, not `deterministic`. Only member-surface claims (invented APIs on a fully enumerated class) are provable from the repository; name absence rests on the syntactic binding model and keeps the lower confidence. Variables reassigned across types in one scope are treated as unknown and never judged.
• Graph tools are anchored to the last indexed project. scan_file / verify_file resolve the project root per file (including monorepo sub-projects); find_dead_code, explore_call_graph, search_code and friends operate on the project most recently passed to index_project.
• `check_api` imports the base package on demand — but only when it is genuinely installed (or stdlib). A bare file dropped into the workspace is never imported, so agent-authored code cannot be executed server-side through this tool.

---

MCP tools

By default scan_file and verify_file are exposed. Set HONESTCODE_TOOLS=all to enable the full set, including:

Dependency manifests (requirements.txt, pyproject.toml) are declared automatically on the first scan, so imports from a dependency that is not installed in the MCP server's interpreter are trusted rather than reported as invented. Set HONESTCODE_AUTO_DEPS=0 to disable that.

See the old tool reference below for the complete list.

---

Crash-safe side effects

verify_file checks code before it runs. guards/ covers what happens after: an agent that retries a tool call can charge a card twice, and no durable execution engine prevents it — Temporal and Restate both guarantee deterministic replay of *orchestration* and leave exactly-once external effects to the application.

from honestcode.guards import EffectStore, SideEffectGuard, durable_scope

guard = SideEffectGuard(session_id="run-42", store_path=".agent/effects.db")

outcome = guard.run( "charge", target="cus_123", payload={"cents": 4200}, execute=lambda: charge_card("cus_123", 4200), approve_irreversible=True, scope=durable_scope("order-4711", "charge", "cus_123"), )

outcome.executed # False on the retry — a receipt, not a second charge outcome.effect_id # stable across a restart, because the key is on disk

The interesting part is what scope does. Pass it and the effect's identity comes from the scope rather than the session — which is the difference between a retry being recognised and a customer being charged twice, because a restart *is* a new session id. Pass nothing and the guard stays session-scoped.

Refusing is the default direction: an unrecognised action is treated as irreversible rather than idempotent, and irreversible effects need explicit approval. A run that dies between the call and the record leaves the effect executing, and the next attempt reports the outcome as unknown instead of guessing — which is what stops the retry. Scoped effects are recovered with guard.recover(scope=durable_scope(...)), and guard.effects(scope=...) / guard.summary(scope=...) report them; the records live under a namespace d