paper-preflight

**Check every reference of a paper against real scholarly records before you submit.

No LLM guessing, no false accusations.**

English · 简体中文 · Quick start · User guide · Releases · Feedback

Language models invent references, and copy-pasted BibTeX carries wrong years, wrong authors and dead DOIs. paper-preflight reads your paper (LaTeX, Word, Markdown, Typst, a PDF or a plain reference list) and asks Crossref, dblp, arXiv, DataCite, PubMed and OpenAlex about every cited work, with Semantic Scholar too if you have a free key, Open Library for books without a DOI, and GitHub, PyPI, CRAN, Hugging Face and OpenML for software, models and datasets:

• Does it exist?
• Does it match what you wrote?
• Has it been retracted?
• Has the preprint you cite been published since?

Every finding is backed by a record, or by every source that could know the work answering "no". When it cannot tell, it says "cannot determine" and why, instead of guessing. No language model takes part in a verdict.

Status: 1.0. On three weeks of real arXiv papers held out for the release: 0.79 false

alarms per 100 references, against 409 real problems found (accuracy).

Rule IDs, the JSON report, settings, commands and the Python API are kept through 1.x

(what is stable). False positives are the bugs we most want to hear

about: please open an issue.

Quick start

With uv nothing needs installing:

uvx paper-preflight check path/to/paper

path/to/paper is the project's folder, its main .tex file or a .bib file. To install it for good (Python 3.11 or later):

pip install paper-preflight

The repository's demo paper cites eleven works, several of them wrong on purpose. The animation above is a real run of paper-preflight check examples/demo-paper against the live sources. In it, the correct NeurIPS paper is verified through dblp although Crossref only holds fake copies of it, the book without an identifier is confirmed by Open Library, and the Chinese book, whose script is not supported yet, is "cannot determine", not "not found".

The same run as text

$ paper-preflight check examples/demo-paper

paper-preflight 1.0.0 · main.tex · 12 entries, 12 cited keys

error CIT001 main.tex:31 Citation key 'nonexistent2023' is not defined in any bibliography file (1 use(s)). error REF001 refs.bib:43 The doi of 'devlin2019bert' (10.1109/cvpr.2016.90) resolves to a different work in Crossref: "Deep Residual Learning for Image Recognition" (He et al., 2016). error REF003 refs.bib:66 'lindqvist2024quantum' was not found in Crossref, dblp and Semantic Scholar, and every source responded. Check that the work exists and that its title is correct. error REF004 refs.bib:73 'wakefield1998ileal' has been retracted (reported by Crossref, OpenAlex). Cite it only if the text discusses the retraction. error CIT002 refs.bib:128 Entry key 'kingma2015adam' is already defined at line 47; BibTeX ignores this one. warning REF015 refs.bib:31 'he2015residual' cites a preprint that has been published in CVPR (2016), DOI 10.1109/cvpr.2016.90. Cite the published version and keep the eprint field. warning CIT004 refs.bib:37 Entries 'devlin2019bert' and 'he2016deep' look like the same work (same DOI). warning REF013 refs.bib:51 'kingma2015adam' gives the year 2016, but dblp records 2014, 2015. warning REF017 refs.bib:112 The doi of 'tacl2019example' contains LaTeX escapes: '10.1162/tacl\_a\_00276'. Write it as: 10.1162/tacl_a_00276 info REF005 refs.bib:73 'wakefield1998ileal' has a published correction (reported by Crossref). info REF090 refs.bib:95 'zhou2016ml' could not be verified: non-Latin titles are not supported yet; grey literature without an identifier (book, report, software, web page). info CIT003 refs.bib:116 Entry 'lecun1998gradient' is never cited.

References: 7 verified · 1 metadata mismatch · 1 identifier conflict · 1 not found · 1 cannot determine 5 error(s) · 4 warning(s) · 3 info

What it reads

What each input can and cannot tell, in detail: docs/inputs.md.

What it checks

Every reference gets one verdict:

The findings say what is wrong:

Every rule has a page: what it checks, when it can be wrong, and what to do. paper-preflight explain REF003 prints it in the terminal.

How accurate is it?

Four measurements, all against the live sources:

Precision comes first: a reference is called fabricated only on positive evidence, and an unanswered or ambiguous lookup is reported as "cannot determine", never as "not found". The evaluation harness and every run's summary are in evals/.

Real papers

Each week is 20 arXiv papers (cs, stat, q-bio, quant-ph and astro-ph) chosen mechanically. For 1.0, three weeks never used before were committed together, before the changes they measure, and each was run once on the same code. The gate was at most 1.5 false alarms per 100 references in each week and at most 1.0 on average. Above them are the more recent weeks that stayed under 1.0, each run once with the version of its time; every week measured, those over 1.0 too, is in evals/README.md.