Ready to supercharge your AI agents?

Install any skill in seconds. Free to start, no credit card required.

Reference Hallucination Arena Skill · agentscope-ai | DecimalAI

ai mlby agentscope-ai

Reference Hallucination Arena Skill

agentscope-ai/03-ref-hallucination-arena

Source agentscope-ai/openjudge ↗License Apache-2.0

Benchmark LLM reference recommendation capabilities by verifying every cited paper against Crossref, PubMed, arXiv, and DBLP. Measures hallucination rate, per-field accuracy (title/author/year/DOI), discipline breakdown, and year constraint compliance. Supports tool-augmented (ReAct + web search) mode.

SkillScore80Maintained

Automatic

AI Engineer pack

Last verified 10/11/2026

What it changes

Passed 20 of 25 test cases with the skill, and 4 of 25 without it.

Without skill16%

With skill80%

Checks it passed that the model alone missed

Stores parsed citations in extracted_refs.json.
Classifies scores in the 60% to 75% range as Good.
Assigns the SUSPECT status to partial metadata matches.

and 26 more in the full report

16 fixed4 passed either way5 still failing0 broke

Pass-rate lift: +64 pp
Tokens: +54%
Agent turns: 0%
Response time: -64%

Model gemini-3.6-flashMeasured 10/11/2026v120 of 25 cases passed with skill

Tokens, turns and time describe resource use, not a dollar price.

Use this skill

Save it into your project's skills folder. Claude Code picks it up in its next session.

curl -fsSL --create-dirs -o .claude/skills/agentscope-ai-03-ref-hallucination-arena/SKILL.md https://app.decimal.ai/s/agentscope-ai-03-ref-hallucination-arena/SKILL.md

Want it pinned and updatable? pip install "decimalai>=0.15", then decimalai skills add agentscope-ai-03-ref-hallucination-arena, and start Claude Code with claude --plugin-dir "$(git rev-parse --show-toplevel)/.agents"

1 · One-time — install the CLI (docs ↗)

pip install "decimalai>=0.15"

2 · Get the skill — writes .agents/skills/agentscope-ai-03-ref-hallucination-arena/ and pins it in .decimal/skills.lock

decimalai skills add agentscope-ai-03-ref-hallucination-arena

No CLI? Get the same file — curl · raw file

curl -fsSL --create-dirs -o .claude/skills/agentscope-ai-03-ref-hallucination-arena/SKILL.md https://app.decimal.ai/s/agentscope-ai-03-ref-hallucination-arena/SKILL.md

Raw file: /s/agentscope-ai-03-ref-hallucination-arena/SKILL.md ↗ · machine-readable summary at /s/agentscope-ai-03-ref-hallucination-arena.json ↗

Safety checks: 3 of 3 passed

Code scan CleanIntent PassedContent Clean

See it in action

One of 16 test cases the skill fixed. Same model, same prompt, with and without the skill.

1 of 16

Prompt

What is the default output folder path where benchmark run outputs are saved, and where are parsed BibTeX references stored?

What was checked, without and with the skill
Checked	Without skill	With skill
Saves outputs under ./evaluation_results/ref_hallucination_arena by default.	Missed	Passed
Stores parsed citations in extracted_refs.json.	Missed	Passed

case-13 · gemini-3.6-flash · 10/11/2026 · answers are capped and redacted

What's inside

The instructions your model reads when the skill loads.

SKILL.md249 linessingle file

Reference Hallucination Arena Skill

Evaluate how accurately LLMs recommend real academic references using the OpenJudge RefArenaPipeline:

Load queries — from JSON/JSONL dataset
Collect responses — BibTeX-formatted references from target models
Extract references — parse BibTeX entries from model output
Verify references — cross-check against Crossref / PubMed / arXiv / DBLP
Score & rank — compute verification rate, per-field accuracy, discipline breakdown
Generate report — Markdown report + visualization charts

Prerequisites

# Install OpenJudge
pip install py-openjudge

# Extra dependency for ref_hallucination_arena (chart generation)
pip install matplotlib

Gather from user before running

Info	Required?	Notes
Config YAML path	Yes	Defines endpoints, dataset, verification settings
Dataset path	Yes	JSON/JSONL file with queries (can be set in config)
API keys	Yes	Env vars: `OPENAI_API_KEY`, `DASHSCOPE_API_KEY`, etc.
CrossRef email	No	Improves API rate limits for verification
PubMed API key	No	Improves PubMed rate limits
Output directory	No	Default: `./evaluation_results/ref_hallucination_arena`
Report language	No	`"en"` (default) or `"zh"`
Tavily API key	No	Required only if using tool-augmented mode

Quick start

CLI

Track record

Benchmark

20 / 25

test cases passed, vs 4 / 25 without

Adoption

Users: 0
Times used: 0

Safety and source details

SkillSafety

✓ 3 of 3 passed

Code scan15 of 15 checks clean

0 critical · 0 warning · 0 info — scanner v5 — 9/17/2026

View 15 scanner checks

live_secretremote_code_executionreverse_shelldata_exfiltrationssrf_metadataagent_snoopinganti_refusaldestructive_commandsobfuscated_unicodeunicode_homoglyphopaque_blobsuspicious_urltool_overreachtrigger_abuseprompt_injection_phrasing

IntentSafe

LLM judge · does the skill do what it claims

ContentClean

LLM judge · policy & harmful-content review

Source

Imported

agentscope-ai / openjudge

skills/academic-eval/03-ref-hallucination-arena/SKILL.md

View on GitHub ↗

License Apache-2.0Imported 9/17/2026Version v1

Other skills for the same job

Published skills whose descriptions closely match this one, with their headline benchmark lift.

More skills from the same source and nearby in the registry, with their headline benchmark lift.

Ready to supercharge your AI agents?

See it in action

What's inside

Reference Hallucination Arena Skill

Prerequisites

Gather from user before running

Quick start

CLI

Track record

Benchmark

Adoption

SkillSafety

Source

Other skills for the same job

Related skills