Ghostchars

ghostchars

Find and remove invisible Unicode characters and machine-writing punctuation, in text and in documents. Everything runs on your machine: no network, no telemetry, no account, and zero runtime dependencies.

npx ghostchars clean -i notes.md          # clean a file in place, atomically

cat draft.md | npx ghostchars inspect # what is hiding: line:col, codepoint, name npx ghostchars check --bar text # the gate: exit 1 if the tree is dirty npx ghostchars report draft.md # habits that make a draft read as machine written npx ghostchars clean -i essay.docx # documents too: .docx, .odt, .html

Needs Node 22 or newer. The whole package is a launcher and three bundles: the CLI, the document reader, and the MCP server. Only the first is loaded unless you name a document or start the server.

Commands

clean reads stdin or the files you name, writes the cleaned text to stdout, or rewrites each file in place with -i. With --check it writes nothing and exits 1 if anything would change.

inspect lists every suspicious character with its line, column, codepoint, Unicode name and the flag that would act on it. It changes nothing and exits 1 when it finds something.

report measures the habits that make a draft read as machine written: sentence length uniformity, stock transitions, tell words, dash density, tricolons, hedging, missing specifics. It is not a detector and emits no probability.

check holds a whole tree to a bar and prints every file that would change, with exact character positions. This is the command for CI and for a pre-commit hook. Configure surfaces and pins in ghostchars.json.

mcp runs a Model Context Protocol server on stdin and stdout, so an agent can call clean_text, inspect_text, style_report and check_paths as tools instead of shelling out.

help prints the commands, help prints one command's flags, and help --json prints the whole surface as one machine-readable document. version prints the version.

Flags

The thirteen cleaning switches, their spellings and their meanings are exactly the Python reference CLI's, because it is the same engine underneath.

Default cleaning is context aware. It strips a zero-width joiner sitting between two Latin letters and keeps the one inside a family emoji or a Persian word. It strips free tag characters, which is how text is smuggled into a message, and keeps a complete subdivision flag. It strips a left-to-right mark in English text and keeps one next to Hebrew. -a turns that judgement off.

An option can only be turned on. There is no way to turn one off, because the reference CLI has none either.

The three bars

A bar is a named set of options, so a person or an agent picks one word instead of thirteen booleans.

Resolution order is fixed, so the result is always predictable:

1. start from every option off,
2. apply --bar,
3. apply --max if given,
4. apply each explicit flag, which can only turn an option on.

So --bar text --collapse is -s --punct --collapse, and --bar max --punct is still just max.

max folds look-alike Cyrillic and Greek letters and applies NFKC, which rewrites letters. Do not run it over text you did not write. It is not offered as a gate bar at all: see check below.

Documents

clean and inspect recognise a document by its extension and handle it structurally.

--as text forces plain-text treatment of any path, which is what the Python reference CLI does with an .html file.

Without -i, a cleaned document is written to .clean beside the original, or to the path you give with -o.

inspect on a document prints the character report over the concatenated document text, then what hides around that text: hidden runs, white-on-white and tiny runs, tracked changes, comments, the count of distinct rsid editing-session ids, metadata fields, macros, media. --metadata blanks the authoring fields and drops custom properties; --rsids removes the editing-session fingerprint.

Hidden runs, comments and tracked changes are reported and never removed. Deleting them changes the document, and that stays a human call.

Out of scope here: PDF, images, .pptx, .xlsx, .rtf, .doc, .epub and .pages. check does not open documents at all.

check

ghostchars check [--bar default|text] [option flags]

[--config ] [--no-config] [--git|--no-git] [--max-bytes N] [--limit N] [--fix] [--json] [--verbose] [paths...]

--bar accepts exactly default and text. There is deliberately no max gate bar: max turns on confusables and NFKC, which target scripts and symbols rather than typography, and a gate that condemns legitimate Cyrillic, Greek and emoji content gets deleted by the people who installed it. The default bar is default, for the same reason: typography is a house style and has to be opted into.

With no path arguments, check finds files with git ls-files, so it honours the same ignore rules your repository already has and still sees files that are new and not yet added. Outside a repository, or with --no-git, it walks the tree instead and skips node_modules, build output, virtual environments and every directory whose name starts with a dot, except the ones coding agents read instructions from (.cursor, .github, .claude, .agents, .windsurf, .devin, .gemini, .codex, .clinerules, .continue, .kiro, .roo). --git makes a git that cannot be used a hard error, whether or not you named paths, so CI cannot silently switch strategy between one container and another. Path arguments are always walked rather than looked up in git, so --git asserts that git was available and reports "discovery": "walk". Results are sorted by path, so the output never depends on directory order. A directory the walk cannot open is an error on stderr and exit 2, never a silent omission.

A file is skipped, silently and without failing the run, when its extension is a known binary one, when it is larger than --max-bytes (2 MiB by default), when one of its first 8192 bytes is NUL, or when it is not valid UTF-8. A file you name on the command line is different: you asserted it is text, so both content checks stop applying to it, the strict decode decides, and a decode failure is an error that exits 2.

check finds 's own ghostchars.json even when you run it from somewhere else: with no config beside the working directory, the search moves up from each path you named, stopping at that path's repository root. Two paths governed by two different config files is an error, because one run has one root and a pin is written relative to it. --verbose prints which config was used.

ghostchars.json

Put it at the root of the tree, or point at it with --config.

{

"version": 1, "surfaces": [ { "id": "invisibles", "bar": "default", "include": ["**/*"], "exclude": ["vendor/"] }, { "id": "prose", "bar": "text", "include": ["/*.md", "src/messages/*.ts"], "decodeEscapes": true, "pins": { "docs/glyphs.md": 12 } } ], "maxBytes": 2097152 }

A surface names a bar (or an options list, one of the two, never both), the globs it covers, and its pins. A file that matches no surface is not checked at all. An exclusion wins over everything, including a path you named explicitly. An unknown key is an error that lists the known keys, so a typo in pins is never silently ignored.

The glob subset is ** for any run of path segments, * for any run inside one segment, and ? for one character. No braces, no character classes.

decodeEscapes resolves \uXXXX and \u{XXXX} to the characters they render before the file is checked. Turn it on for a source file whose strings a person reads, and leave it off everywhere else: a table of codepoints is full of escapes that match characters rather than ship them.

Pins

A pin says a file changes by exactly N characters on purpose. It is enforced in both directions.

That last row is the one people forget. It catches a pin that went stale when somebody quietly fixed the file.

Output

docs/post.md: the text bar would change this file:

docs/post.md:1:26 U+2014 EM DASH Ghostchars is a text tool it finds what a reader cannot see. docs/post.md:2:43 U+200B ZERO WIDTH SPACE There is a zero width space in this one.

ghostchars: 1 file would change, 3 scanned

The two placeholders stand for the real characters. This file is held to the text bar itself, so it names a codepoint where it would otherwise print one.

One path:line:col U+XXXX NAME line per offending character, and the source line printed once per distinct line number. Exit 1. When everything passes, one line: ghostchars: 3 files pass the text bar. --verbose also names the files that passed.

In CI

npx -y ghostchars check --bar text

That is the whole step. Never pass --fix in CI. --fix cleans the failing files in place and then re-checks, which turns a red build green without anyone reading what changed. It also skips pinned files and names them, because fixing one would inv