defluff

defluff

*The deterministic slop check for AI-generated prose. Point it at a changelog, a doc, or an agent's own output and get back the filler phrases to cut — plus a CI exit code and a pinnable score, identical on every run. No model, no API key.*

Every flagged span carries no information, so cutting them loses nothing. Clean text, same tool, passes straight through:

What makes defluff worth installing over a one-off grep is the engine around the list: bring your own phrases, per-project overlays, and an MCP server your agents pick up with no wiring.

---

Install

pip install defluff

Or on macOS/Linux via Homebrew:

brew install ahmedak/defluff/defluff

That's it. No model download. No API key. Runs anywhere Python does.

---

Quick start

# Lint a file — exit 1 on slop, 0 when clean

defluff lint essay.md

Pipe text

cat draft.md | defluff lint

Get a bare score for scripts (0.0 – 1.0)

defluff score essay.md

Machine-readable JSON for downstream tooling

defluff lint essay.md --json

---

MCP server

Exposes three tools so any MCP-aware agent can self-check prose without bespoke wiring — including its own draft, before returning it.

Zero-install via uvx (recommended) — pulls the package and the mcp extra on first run:

{

"mcpServers": { "defluff": { "command": "uvx", "args": ["--from", "defluff[mcp]", "defluff-mcp"] } } }

Or install it and run the entry point directly:

pip install "defluff[mcp]"

defluff-mcp

{

"mcpServers": { "defluff": { "command": "defluff-mcp" } } }

Published to the MCP Registry as io.github.ahmedak/defluff (see server.json).

mcp-name: io.github.ahmedak/defluff

---

Common use cases

• Agent self-correction — call slop_detect on a draft and revise the flagged phrases before returning it. Zero wiring, one session, no second model in the loop.
• CI gate on generated content — fail the build when an AI-drafted changelog or doc ships full of "furthermore" and "robust." Deterministic + exit codes + a pinnable lexicon is what an LLM-judge gate can't give you.
• Writing assistant feedback — highlight the exact phrases an editor would cut, instead of a vague "this sounds AI."
• **A reward *component* for fine-tuning** — see reward loops for caveats; on its own it's gameable.

---

Why defluff?

Every "AI detector" tries to classify *whether* text was AI-generated — a hard, unreliable problem. defluff asks a different question: does this text contain removable filler? That's deterministic, and it's true whether a human or an LLM wrote "at the end of the day."

proselint is the closest prior art — deterministic, no model — but emits yes/no warnings rather than a tunable density score, and isn't built around a list you swap, overlay, or pin.

A grep over a word list gives you raw hits. defluff gives you what you'd otherwise have to build around that list:

• An MCP server — agents self-check with zero wiring.
• Markdown- and code-aware — strips code fences, inline code, and URLs first.
• Whole-word matching — "foster" won't fire inside "fostering".
• A normalized score instead of a hit count — filler *density*, so one threshold works on a tweet or a 5,000-word doc.
• Overlap handling — "at the end of the day" overlapping "end of the day" counts each word once (longest-match-wins).
• Exit codes, JSON, and char-offset spans — drop into CI, pre-commit, and editor tooling.
• A pinnable lexicon hash — prove the ruler didn't move between runs.

Bring your own slop

defluff lint draft.md --lexicon team-slop.md

# team-slop.md

• circle back
• low-hanging fruit
• boil the ocean

paradigm shift

One phrase per line (# comments and -/* markers ignored). These layer on top of the built-in defaults and report under a neutral custom category. For real categories and per-phrase weights, use a .json list (see Lexicon overlays).

Ready-made domain packs

defluff lint post.md --pack marketing-growth

defluff lint post.md --pack marketing-growth,ai-llm # stack several

List them with defluff packs. High-false-positive terms (e.g. pivot, detox) ship commented-out so they're inert until you opt in. See the packs README.

Batteries-included defaults

~130 curated patterns across five weighted categories, case-insensitive, whole-word matched:

Rhetorical patterns (beyond the list)

Some AI tells are sentence *shapes*, not fixed phrases — the antithesis: it's not X, it's Y, not just a list but a runtime. A regex pattern layer catches these under a rhetoric category:

Each match counts as one unit toward the score regardless of length. Turning on fragment mode changes the lexicon hash.

defluff lint draft.md                  # compound antithesis caught by default

defluff lint draft.md --pack rhetoric # also catch the punchy "X, not Y" fragment defluff lint draft.md --category rhetoric # gate CI on antithesis alone

---

Python API

import defluff

report = defluff.detect("It is worth noting that we should leverage synergies.") print(report.slop_score) # 0.0 – 1.0 print(report.spans) # flagged phrase locations + categories

score = defluff.score(text) # bare float clean = defluff.is_slop(text) # bool at default threshold

lex = defluff.load_lexicon() # pin for reproducible runs score = defluff.score(text, lexicon=lex)

SlopReport fields:

---

Lexicon overlays and versioning

The bundled lexicon is the baseline; layer on top without editing the package:

# "synergy" is slop on this machine

defluff lexicon add "synergy" --category corporate --scope user

"leverage" is fine in this repo (finance context) — commit this with the repo

defluff lexicon rm "leverage" --scope project git add .defluff/ignore.json && git commit -m "allow 'leverage' in finance context"

• User overlay (~/.config/defluff/) — machine-wide, not committed
• Project overlay (.defluff/ at git root) — per-repo, commit it for your whole team

Writes are atomic and cross-process locked; a corrupt overlay is warned and skipped — detect() never crashes.

Every resolved lexicon (bundled + overlays + packs) carries a short content hash, printed on every run (lexicon: 2cc05ba84457) and exposed as SlopReport.lexicon_version. Pass lexicon=defluff.load_lexicon() once and every call scores against the same ruler — pinnable for CI baselines and RL rewards, and auditable since the hash changes if and only if the resolved entry set changes. Each release ships a dated lexicon with a changelog (CHANGELOG.md); ai-vocab is expected to turn over release to release, cliche/hedge/corporate/transition are near-stable.

---

Reading the output and setting the threshold

1. The spans — the exact filler phrases, with category. Deterministic, reliable. Act on these.
2. The score — slop_score = flagged words ÷ total, weighted by category (ai-vocab counts a little more, transition a little less). 0.20 ≈ "a fifth of this text is listed filler." Usually 0.0–1.0; can edge slightly above on text that's almost nothing but slop. Instead of being a quality quantfier, its just a tripwire that drives the CI exit code.

Pick a threshold based on how filler-dense the text is *allowed* to be:

The default 0.08 is provisional — hand-chosen, not yet calibrated on a labeled corpus (hence the [threshold provisional] tag). For a hard CI gate, set your own threshold and suppress your domain