Hone

Hone

Keep your edge.

AI made producing work cheap. Hone measures the thing that's now scarce: **whether the *human* actually understands the work they shipped with AI.**

Every AI eval scores the model. Hone scores you — your AI-Q.

What it feels like · Quickstart · AI-Q · Why · Whitepaper

---

The 30-second version

Every AI eval tool answers one question: was the model's output good? (accuracy, hallucination, toxicity, latency, cost.)

None of them answer the question that actually decides whether AI-assisted work is safe to ship: does the human who produced it understand it well enough to defend it?

Hone is an open toolkit for measuring exactly that. Give it a piece of AI-assisted work; it interviews the human — a few sharp questions, one at a time, the way a good mentor would — then returns their AI-Q: a score over the three Ds of working with AI. Discernment (can you tell when the AI is wrong? — six auditable dimensions, the spine of the score), Delegation (did you brief it, choose the task, and steer it well?), and Design (are you re-architecting how you work? — read over time, never faked from one review).

It's bring-your-own-model (any LLM), Apache-2.0, and drops into a coding agent, a CI pipeline, or your own app in minutes. The methodology is an open spec — the AI-Q — the open standard — so the score means the same thing everywhere it's computed.

What a review feels like

You:  Run hone on the pricing analysis we just built.

Sol: Happy to take a look. This is a strategy memo recommending a Pro-tier price increase — real money riding on it, so I'll ask five questions. Just between us, and "I don't know" is a perfectly useful answer.

First: walk me through the recommendation in your own words — why $49, and not $39 or $59?

...three more, one at a time — including: what did you give the AI to work with, and what did you have to correct in its first answer? ...

Last one: before I show you anything — honestly, how confident are you in this work, 0 to 100?

You: 85.

Sol: Here's my read.

Pro-tier pricing recommendation · AI-Q 58 — Mostly owns it (discernment 57 · delegation 60) You can explain the mechanism, but you took the churn projection on faith — and that's the load-bearing number.

Work on this first: the 4% churn assumption came straight from the AI with no source. If churn runs 8%, the revenue case inverts.

[ what a rigorous review surfaced vs. what you caught ] [ before-you-ship checklist · the toughest objection you'll face ] [ worth remembering · 3 short notes on the skills you missed ]

No grades for the work. No "AI detector." A measure of *your grip on it* — and coaching to close the gap.

Quickstart

In your coding agent (the best way) — review work where it lives

When an agent built the thing — a repo, an analysis, a doc set — don't paste it anywhere. Hone runs inside the agent, which already has the work loaded:

Claude Desktop — one click: download hone.mcpb from the latest release and open it (Settings → Extensions). No config files, no terminal.

Claude Code — add the MCP server (protocol + deterministic scoring):

git clone https://github.com/derekchoyai/hone && cd hone/mcp && npm install && npm run build

claude mcp add hone -- node "$(pwd)/dist/index.js"

Or load the Claude skill — zero infrastructure. Then just say:

*"Run hone on what we built today."*

The agent reads the work it already has, interviews you one question at a time, commits your confidence before the reveal, and computes your AI-Q deterministically. With your okay it remembers the scores — never the work — in a local file, so the next review pushes on your actual weak spots ("your verification has been the soft dimension — let's start there"). Nothing leaves your machine.

Gate AI-generated code in CI (GitHub Action)

# .github/workflows/judgment.yml

• uses: derekchoyai/hone/integrations/github-action@v0 with: domain: coding paths: "src/**/*.ts" model: openai # or anthropic env: OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}

On a PR touching AI-generated code, the author answers a few questions and the work gets a Judgment Score. Below your threshold → the check explains the gaps. See integrations/github-action.

Build it into your own app (TypeScript SDK)

npm install hone-sdk

import { analyzeWork, reviewJudgment, type ModelFn } from "hone-sdk";

// Bring your own model — Hone never sees your API keys. const model: ModelFn = async ({ system, user }) => callYourLLM(system, user);

// 1. Decompose the work + generate an interview tailored to it. const { workMap, questions } = await analyzeWork({ work: aiGeneratedArtifact, domain: "coding", model, });

// 2. Collect the human's answers (in YOUR ui), then score. const result = await reviewJudgment({ work: aiGeneratedArtifact, domain: "coding", answers, // [{ question, answer }] model, });

console.log(result.composite, result.band); // 72 "Mostly owns it" ← the AI-Q console.log(result.discernment, result.delegation); // sub-scores (delegation only when assessed) console.log(result.gaps, result.verifyBeforeShip);

result conforms to spec/score.schema.json.

No install at all — run the prompts by hand

Paste a prompt into any chat model with your work. See examples/ for fully worked reviews with scores.

AI-Q — the score

A transparent 0–100 score derived from a rubric you can read and change — never vibes, never model arithmetic. AI-Q measures the three Ds of working with AI, weighted by how much they protect you (Discernment 0.6 · Delegation 0.3 · Design 0.1, renormalized over what was actually assessed):

D1 · Discernment — *judgment about the AI's output* (the spine)

Can you tell when the AI is wrong, lazy, or hallucinating? Six auditable dimensions, always assessable from the work in front of the reviewer:

D2 · Delegation — *judgment about your input to the AI*

Did the brief carry what the task needed? Was this the right thing to hand to AI at all? Did you steer between the first output and the final one? Three facets — brief quality, task selection, iteration control — scored *only* when the interview actually surfaced the brief (the evidence gate). No evidence → reported as not assessed, never guessed, never zero.

D3 · Design — *judgment about your system of work*

Are you re-architecting how you work around AI — or just using it where it lands? One artifact can't show a system, so Design is never scored from a single review. It's read qualitatively from your profile over time (domain breadth, delegation trend, whether discernment holds as you delegate more); a numeric Design score needs cross-user baselines and stays on the roadmap.

Agentic work, too. When the thing you made *is* a system — an agent loop, an automation, a multi-agent setup — the same three Ds re-point to the loop: is the "done" check real or just "looks done" (Discernment), was a loop even the right call vs. a one-shot prompt and is the blast radius bounded (Delegation). Hone measures judgment over how you *run* AI, not just one-shot answers. See spec/aiq.md.

Bands: 80–100 Owns it · 55–79 Mostly owns it · 30–54 Riding the AI · 0–29 Black box.

Scoring is deterministic — every sub-score and the composite are computed by the same code everywhere (hone-sdk), so a 72 in CI means the same thing as a 72 in your agent. A six-dimension review's AI-Q equals its Discernment sub-score, so every v0.1 Judgment Score is already a valid AI-Q. The full normative definition is spec/aiq.md; two extended Discernment dimensions — *Counterargument* and *Curiosity* — are optional and defined in spec/judgment-dimensions.md.

A score *of* judgment has to itself be defensible. Each dimension is scored independently against published 1–5 anchors (not a holistic vibe), with explicit guards against the self-preference and fluency biases an LLM judge is prone to, and the implementation leads with the band, not a false-precision integer — with an ensemble option for high-stakes reviews. The full rules are in spec/aiq.md → Scoring rigor.

The interview methodology is built on the evidence, not vibes either: premortems (Klein), consider-the-opposite and reference-class forecasting (Tetlock/GJP), calibration against stated confidence (Kahneman–Klein), and confidence-before-reveal — because subjective confidence is not an accuracy signal.

Why AI evaluation isn't enough

Current AI evaluation asks: *was the model correct?* Hone asks: *does the human understand?*

As AI collapses the cost of producing work, the scarce skill becomes recognizing good work — knowing what to trust, what to verify, and what you're prepared to defend with your name on it. A 2025 CHI study of 319 knowledge workers (Lee et al., Carnegie Mellon & Microsoft Research) found confidence in AI associated with *less* self-reported critical thinking (β = −0.69, p < 0.001), while confidence in one's *own* ability was associated with *more* (β = +0.26). That gap — between what the work needed and what the human caught — is the judgment gap, and nothing in the eval stack measures it.

The un-gameable layer. Automated metrics get gamed: in 2026, a single automated agent was shown to break every major agent benchmark via reward-hacking — near-perfect scores without solving the task. The one thing that can't be optimized away by the model is whether the *human* can stand behind the result. That's what Hone measures, and why it scores the person rather than the output.

Full argument: The AI Judgment Gap — Why AI Evaluation Is Not Enough.

How it works (the architecture bet)

The host does the reading; Hone does the judging. When your agent already has the repo/docs loaded, re-ingesting them into a review service duplicates the hardest infrastructure in the industry — and leaks your work. So Hone inverts it: methodology flows *in* (rubrics, interview protocol, question altitude), judgment metadata flows *out* (six scores, a band). The work itself never transits. Privacy isn't a policy here; it's the architecture.

┌─ your agent / CI / app ──────────────────────────┐

│ has the work loaded · runs the interview │ │