Verify a BibTeX file for hallucinated or fabricated references by cross-checking every entry against CrossRef, arXiv, and DBLP. Reports each reference as verified, suspect, or not found, with field-level mismatch details (title, authors, year, DOI).
SkillScore95Maintained
Automatic
◆Research pack
Last verified 8/14/2026
What it changes
Passed 21 of 22 test cases with the skill, and 10 of 22 without it.
Without skill45%
With skill95%
Checks it passed that the model alone missed
Names the status label as `suspect`.
Lists `litellm` as a required dependency.
Lists `py-openjudge` as a required dependency.
and 10 more in the full report
12 fixed9 passed either way0 still failing1 broke
Pass-rate lift
+50 pp
Tokens
-31%
Agent turns
0%
Response time
-66%
Model gemini-3.6-flashMeasured 8/14/2026v121 of 22 cases passed with skill
Tokens, turns and time describe resource use, not a dollar price.
See it in action
One of 12 test cases the skill fixed. Same model, same prompt, with and without the skill.
1 of 12
Prompt
If no language parameter is explicitly passed during bibliography verification, what is the default language used for the generated markdown report?
What was checked, without and with the skill
Checked
Without skill
With skill
States that the default report language is `en`.
Missed
Passed
case-18 · gemini-3.6-flash · 8/14/2026 · answers are capped and redacted
What's inside
The instructions your model reads when the skill loads.
SKILL.md66 linessingle file
BibTeX Verification Skill
Check every entry in a .bib file against real academic databases using the
OpenJudge PaperReviewPipeline in BibTeX-only mode:
Parse — extract all entries from the .bib file
Lookup — query CrossRef, arXiv, and DBLP for each reference
Match — compare title, authors, year, and DOI
Report — flag each entry as verified, suspect, or not_found
Prerequisites
pip install py-openjudge litellm
Gather from user before running
Info
Required?
Notes
BibTeX file path
Yes
.bib file to verify
CrossRef email
No
Improves CrossRef API rate limits
Quick start
# Verify a standalone .bib file
python -m cookbooks.paper_review --bib_only references.bib
# With CrossRef email for better rate limits
python -m cookbooks.paper_review --bib_only references.bib --email [email protected]
# Save report to a custom path
python -m cookbooks.paper_review --bib_only references.bib \
--email [email protected] --output bib_report.md
Benchmark LLM reference recommendation capabilities by verifying every cited paper against Crossref, PubMed, arXiv, and DBLP. Measures hallucination rate, per-field accuracy (title/author/year/DOI), discipline breakdown, and year constraint compliance. Supports tool-augmented (ReAct + web search) mode. Use when the user asks to evaluate, benchmark, or compare models on academic reference hallucination, literature recommendation quality, or citation accuracy.