Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →yogsoth-ai/discrepancy-analysis
Identify discrepancies between reported and reproducible scores — 15 methods, 45 data points, 30 web searches budget
Passed 4 of 13 test cases with the skill, and 7 of 13 without it.
and 3 more in the full report
4 fixed3 passed either way7 still failing8 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 4 test cases the skill fixed. Same model, same prompt, with and without the skill.
Analyze BioBERT-v2 score collapse on MedQA (84.0 reported vs 62.0 reproduced) after community found test set questions in the pre-training corpus. Format output as structured audit JSON.
| Checked | Without skill | With skill |
|---|---|---|
| The likely_cause field explicitly identifies data contamination or pre-training test leakage | Missed | Passed |
| The delta_significant boolean field is set to true | Missed | Passed |
The instructions your model reads when the skill loads.
Detect inconsistencies between scores reported in original papers versus third-party reproductions, leaderboard entries, and ablation studies. Flags methods with suspicious performance claims, identifies common sources of score inflation, and assesses the reliability of reported baselines.
| Resource | Floor | Target |
|---|---|---|
| Methods analyzed | 10 | 15 |
| Data points compared | 30 | 45 |
| Web searches | 20 | 30 |
| Reproduction studies consulted | 5 | 10 |
<HARD-GATE>
| Metric | Current | Target | Status |
|--------|---------|--------|--------|
| Methods analyzed | 0 | 15 | BLOCKED |
| Score pairs compared | 0 | 45 | BLOCKED |
| Discrepancies flagged | 0 | — | — |
| Reproduction studies found | 0 | 10 | — |
| Reliability ratings assigned | 0 | 15 | — |
</HARD-GATE>
Cannot exit until score_pairs_compared >= 36 (80% of target).
test cases passed, vs 7 / 13 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.