refereekit

refereekit

refereekit helps you referee a paper with a language model. It checks every quotation, page, equation and figure that an answer or a draft cites against the submitted PDF. It then drafts a referee report and an editor letter in your style, from claims that checked out. You edit them. The verdict is yours.

You need a terminal and uv. Two commands start a review:

uv tool install refereekit

refx chat paper.pdf

In the chat, /model access sets your model. For anthropic, bedrock and vertex it names one more install line. /tutorial runs a practice review that needs no key. Your terminal is the private view.

You can also review in pi, Claude Code, Codex or Claude Desktop, or in the ChatGPT desktop app as a preview. Paste the setup prompt into your agent, which sets refereekit up. refereekit never gives the paper to the agent's model. What you type in the agent's chat does reach it, so keep the paper's title out. You read the paper, and type your judgment, in a private view.

Confidentiality first. A manuscript under review goes only to a model backend, after you attest what its account keeps. It never enters this repository. See Confidentiality.

Documentation: start with Before you start, then pick a reading path from the documentation index.

Citation

If you use refereekit in your research, please cite the following paper:

@inproceedings{yen2026models,

title={Models Quote, Code Verifies, Humans Judge: Lessons for AI-Assisted Peer Review}, author={Tzu-Chi Yen}, booktitle={NeurIPS 2026 Workshop on AI-Native Academia}, year={2026} }

License

MIT. See LICENSE.