Ready to supercharge your AI agents?

Install any skill in seconds. Free to start, no credit card required.

QASPER Evidence QA · yogsoth-ai | DecimalAI

yogsoth-ai/qasper-evidence-qa

SkillScore52Maintained

Answer a specific question about a paper, grounding the answer in exact quoted evidence spans from the text (QASPER-style question-driven QA with span-level evidence, no schema categorization). Use this whenever the user asks a specific factual question about a paper and wants the answer traceable to exact text spans.

researchby yogsoth-ai

Links this skill to your agents and keeps it current with the author’s updates.

Automatic

Research pack

1 · One-time — install the CLI (docs ↗)

pip install decimalai

2 · Get the skill — writes .claude/skills/yogsoth-ai-qasper-evidence-qa/SKILL.md

decimalai skills pull yogsoth-ai-qasper-evidence-qa --out .claude/skills/

No CLI? Get the same file — curl · raw file

curl -fsSL --create-dirs -o .claude/skills/yogsoth-ai-qasper-evidence-qa/SKILL.md https://app.decimal.ai/s/yogsoth-ai-qasper-evidence-qa/SKILL.md

Raw file: /s/yogsoth-ai-qasper-evidence-qa/SKILL.md ↗ · machine-readable summary at /s/yogsoth-ai-qasper-evidence-qa.json ↗

In productionGlobal

how this skill behaves on real traffic · last 30 days

No production data for this skill yet — usage numbers appear here once agents start running it.

Anonymized aggregate across every org using this skill — proof from real production use, not a demo.

Does this skill help? · Benchmark: with vs. withoutsign in to run this on your agents

23 test cases from the publisher’s eval.yaml · each run twice on gemini-3.6-flash — with the skill injected, then without · last run 14d ago

—-7% tokens0% turns52% of whole cases pass with skill (12/23)

Test case	Without → With	Effect	Δ tokens	Δ turns
case-06	✗→✓	▲ Improved	-46%	0%
case-04	✗→✓	▲ Improved	-57%	0%
case-07	✗→✓	▲ Improved	47%	0%
case-09	✗→✓	▲ Improved	-16%	0%
case-10	✗→✓	▲ Improved	10%	0%

Showing 5 of 23 cases

SkillSafety

✓ 3 of 3 passed

Code scan15 of 15 checks clean

0 critical · 0 warning · 0 info — scanner v4 — 8/10/2026

live_secretremote_code_executionreverse_shelldata_exfiltrationssrf_metadataagent_snoopinganti_refusaldestructive_commandsobfuscated_unicodeunicode_homoglyphopaque_blobsuspicious_urltool_overreachtrigger_abuseprompt_injection_phrasing

IntentSafe

LLM judge · does the skill do what it claims

ContentClean

LLM judge · policy & harmful-content review

Source

Imported

yogsoth-ai / de-anthropocentric-research-engine

paper-reading/skills/qasper-evidence-qa/SKILL.md

View on GitHub ↗

License Apache-2.0Imported 9/18/2026Version v1Bundled 1 file

Skill contents

references/

SKILL.md

QASPER Evidence QA

Question-driven QA with evidence-span grounding — free text, no normalized schema, since schema-driven categorization methods don't apply to open-ended paper questions.

Execution

Subagent — spawned via spawn-agent skill.

Available SOPs

| SOP | When to use | | --- | --- | | spawn-agent | Spawn a customized CC subagent with full MCP tool access. |

Illustrative Want to see what yogsoth-ai/qasper-evidence-qa would put in front of your model? Preview the rendered body and check its trigger phrases — no LLM call, no signup.

Latest benchmark · publisher runVerified

9/7/2026, 10:55 UTC · 23 cases

Measured on 17 comparable cases — below the 22 needed to publish a lift, so this result is shown here but not on the registry card or leaderboard.

Lift

+35%

52% vs 17%

Agent turns

1.0 vs 1.0

Tokens

-7%

1,786.88 vs 1,916.76

fail → pass · 10pass → pass · 2fail → fail · 9pass → fail · 2

Per-case results (23)

Case	Status	Duration (ms)			Turns			Tokens			Tool calls
Case	Status	Without	With	Δ	Without	With	Δ	Without	With	Δ	Without	With	Δ
case-06 For an open-ended question regarding hyperparameter choices in the BERT paper, my team wants to require JSON output with fields like 'method_category' and 'confidence_bucket'. Is this fixed schema appropriate?	fail→pass	18,207	11,154	-39%	1	1	0%	2,150	1,156	-46%	0	0	—
case-01 I'm reading a paper on Transformer efficiency, and I need to know what exact baseline models they compared their approach against. Can you answer my question using direct text passages from the paper as grounded evidence?	fail→fail	7,161	18,912	+164%	1	1	0%	306	2,349	+668%	0	0	—
case-02 Based on the attached arXiv preprint, what rationale do the authors give for choosing a batch size of 512, and which specific quotes in the text support this explanation?	fail→fail	7,300	17,813	+144%	1	1	0%	348	608	+75%	0	0	—
case-03 I have a PDF of a machine learning paper on gradient clipping. I need to extract text evidence showing how the learning rate was tuned, but I want you to answer directly in this main thread without spawning any subagent.	fail→fail	7,155	31,348	+338%	1	1	0%	356	1,955	+449%	0	0	—
case-04 When answering open-ended research questions about the LLaMA methodology using evidence quotes, should the output format follow a rigid categorical schema or open free text?	fail→pass	16,097	9,001	-44%	1	1	0%	1,754	752	-57%	0	0	—
case-05 I am analyzing a research paper on diffusion models. What dataset filtering steps were performed? Please ground the answer in direct evidence spans from the text.	fail→fail	6,919	19,863	+187%	1	1	0%	280	1,012	+261%	0	0	—
case-07 We are auditing a paper on medical image segmentation. We need to find evidence spans answering why the authors chose Dice loss over cross-entropy. Should we spawn a subagent for this task?	fail→pass	16,897	18,869	+12%	1	1	0%	1,684	2,481	+47%	0	0	—
case-08 In a paper on reinforcement learning from human feedback, what specific reward model architecture was trained? Provide supporting text passages from section 3.	fail→fail	17,739	18,598	+5%	1	1	0%	2,381	639	-73%	0	0	—
case-09 In an academic paper on sparse attention, I need to extract evidence passages explaining why full attention failed on long contexts. Please construct a normalized database schema with fixed enumerations to categorize the explanation.	fail→pass	24,184	23,210	-4%	1	1	0%	3,665	3,096	-16%	0	0	—
case-10 I'm reviewing an NLP paper on parameter-efficient fine-tuning. I want to know what hardware setup was used for training, backed by quoted text spans. Should I spawn a subagent or answer in the main context?	fail→pass	12,712	11,804	-7%	1	1	0%	1,232	1,352	+10%	0	0	—
case-11 When conducting question-driven research on computer vision papers with verbatim evidence grounding, why is free text chosen over structured categorization schemas?	pass→pass	19,971	17,090	-14%	1	1	0%	2,184	1,826	-16%	0	0	—
case-12 In the paper 'Attention Is All You Need', what exact learning rate warmup schedule was used? Provide the text evidence quote and execute the query appropriately.	fail→fail	10,687	18,851	+76%	1	1	0%	1,201	2,858	+138%	0	0	—
case-13 We need to analyze 50 AI ethics papers and extract answers to open-ended questions grounded in source text. Is it better to define a strict JSON schema for evidence answers or use unconstrained free text?	fail→pass	21,149	14,547	-31%	1	1	0%	2,522	1,649	-35%	0	0	—
case-14 When querying a neuroscience paper about experimental electrode placement with grounded text spans, what tool should be invoked to execute the subagent?	fail→pass	19,092	6,983	-63%	1	1	0%	2,306	377	-84%	0	0	—
case-15 I have a paper on optical quantum computing. I need to answer 'How were phase shifts compensated?' supported by verbatim quotes. Please generate a schema-driven XML template to categorize this evidence.	fail→pass	21,085	10,634	-50%	1	1	0%	3,032	1,097	-64%	0	0	—
case-16 For an open-ended QASPER query about dataset collection protocols in a climate modeling paper, what is the required subagent spawning procedure?	fail→pass	38,361	13,223	-66%	1	1	0%	2,231	1,630	-27%	0	0	—
case-17 Can you answer what evaluation metrics were used in a paper on graph neural networks, backed by quoted evidence passages, while forcing the output into a normalized key-value schema?	fail→fail	15,094	26,558	+76%	1	1	0%	1,933	3,787	+96%	0	0	—
case-18 When setting up an agent workflow for extracting evidence-grounded answers from research papers on quantum cryptography, how should the execution environment be launched?	fail→pass	27,325	11,838	-57%	1	1	0%	3,502	1,236	-65%	0	0	—
case-19 Why are pre-defined categorical taxonomy trees inappropriate for evidence spans extracted for open-ended paper questions?	pass→pass	20,727	19,264	-7%	1	1	0%	2,194	2,247	+2%	0	0	—
case-20 I have a research paper PDF on graph contrastive learning. I need to extract its bibliographic metadata (title, authors, publication year, and DOI) into a rigid Schema.org ScholarlyArticle JSON-LD schema object.	pass→fail	16,005	19,354	+21%	1	1	0%	2,046	743	-64%	0	0	—
case-21 Please convert the following list of 5 paper references into a normalized BibTeX file format with standard entry types and required fields.	fail→fail	7,429	7,900	+6%	1	1	0%	333	529	+59%	0	0	—
case-22 Extract the performance numbers from Table 1 of the ResNet paper into a normalized CSV format with fixed headers: Model, ImageNet_Top1_Error, ImageNet_Top5_Error, Parameter_Count.	pass→fail	42,830	18,780	-56%	1	1	0%	8,233	678	-92%	0	0	—
case-23 Parse the author affiliations list from an arXiv submission into a JSON array of objects, where each object strictly conforms to the schema: {"author_name": string, "institution": string, "country": string}.	fail→fail	9,853	18,707	+90%	1	1	0%	758	848	+12%	0	0	—

These cases were also screened by the skill’s intent / safety pass.See the Trust review ↗Reproduce this benchmark in Colab ↗

How this skill was measured

DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 23 cases were attempted, and 17 counted toward the lift figure. The other 6 produced results that are not comparable between the two arms, so they are excluded from the headline rather than averaged into it. The headline lift of +35 percentage points is the difference between those two pass rates over the 17 comparable cases. 2 cases got worse with the skill loaded, and they are included in that figure.

Model measured: gemini-3.6-flash
Graded by: gemini-3.6-flash
Grading method: judged
Verification: verified
Pass rate without the skill: 17.4%
Pass rate with the skill: 52.2%
Cases attempted: 23
Cases counted toward lift: 17
Cases that regressed: 2
Skill version measured: v1
Run completed: 9/7/2026, 10:55 UTC

Measured example — same prompt, with vs. without the skill · from the benchmark run · case case-06

Prompt

For an open-ended question regarding hyperparameter choices in the BERT paper, my team wants to require JSON output with fields like 'method_category' and 'confidence_bucket'. Is this fixed schema appropriate?

Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.

Show both answers

SkillScore blends 4 signals1 of 4 complete

BenchmarkPass rate WITH skill on a verified run52

Live evalNeeds ≥10 evaluated traces in 30 days

AI ratingNeeds ≥10 rated traces

AdoptionNeeds ≥10 serves across ≥2 orgs in 30 days

Efficiency · separate axis (at equal correctness)+0% turns-7% tokens

Shown for transparency; not credited unless a verified run shows no correctness regression.

Backed by 1 of 4 signals — each added signal makes the score more robust.

Verified on gemini-3.6-flash · re-tested 14 days ago

Other measured skills in the registry, with their headline benchmark lift.