Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →Compare and standardize experimental conditions across papers
.claude/skills/yogsoth-ai-condition-normalization/SKILL.md| Test case | Without → With | Effect | Δ tokens | Δ turns |
|---|---|---|---|---|
| case-01 | ✗→✓ | ▲ Improved | -4% | 0% |
| case-03 | ✗→✓ | ▲ Improved | -9% | 0% |
| case-07 | ✗→✓ | ▲ Improved | 654% | 0% |
| case-08 | ✗→✓ | ▲ Improved | 16% | 0% |
| case-10 | ✗→✓ | ▲ Improved | 86% | 0% |
Build a structured understanding of how experimental conditions vary across papers, then define normalization schemes that enable fair method-to-method comparison. Addresses the fundamental problem that papers evaluate under different settings, making raw score comparison misleading.
For each method-paper pair, extract all evaluation conditions:
Yield: Condition vectors per method-paper pair.
Build a matrix showing which conditions differ across methods:
Yield: Condition difference matrix with impact annotations.
Define rules for adjusting scores to common conditions:
Yield: Normalization rule set with validity bounds.
Apply normalization to produce fair comparison subsets:
Yield: Fair comparison tables with methodology notes.
| Metric | Floor | |--------|-------| | Condition dimensions cataloged | 5 | | Methods with full condition vectors | 10 | | Normalization rules defined | 3 | | Fair comparison sets produced | 2 |
<!-- BEGIN available-tables (generated) -->
Optional, no fixed order; the final leaf is always a sop.
| SOP | When to use | | --- | --- | | compute-normalization | Normalize results by compute budget (Pareto analysis) | | condition-cataloging | Record evaluation conditions (data splits, hyperparams, hardware, seeds) from a paper | | performance-table-assembly | Assemble unified comparison table with confidence interval annotations |
<!-- END available-tables (generated) -->
| Case | Status | Duration (ms) | Turns | Tokens | Tool calls | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Without | With | Δ | Without | With | Δ | Without | With | Δ | Without | With | Δ | ||
case-01 | fail→pass | 33,005 | 27,814 | -16% | 1 | 1 | 0% | 6,212 | 5,987 | -4% | 0 | 0 | — |
case-02 | fail→fail | 36,426 | 33,269 | -9% | 1 | 1 | 0% | 6,217 | 6,790 | +9% | 0 | 0 | — |
case-03 | fail→pass | 35,303 | 27,323 | -23% | 1 | 1 | 0% | 6,208 | 5,642 | -9% | 0 | 0 | — |
case-04 | pass→fail | 15,964 | 28,094 | +76% | 1 | 1 | 0% | 2,974 | 6,758 | +127% | 0 | 0 | — |
case-05 | pass→pass | 18,703 | 17,621 | -6% | 1 | 1 | 0% | 2,999 | 3,315 | +11% | 0 | 0 | — |
case-06 | pass→pass | 31,184 | 36,796 | +18% | 1 | 1 | 0% | 4,790 | 6,761 | +41% | 0 | 0 | — |
case-07 | fail→pass | 3,297 | 17,713 | +437% | 1 | 1 | 0% | 515 | 3,883 | +654% | 0 | 0 | — |
case-08 | fail→pass | 20,486 | 22,454 | +10% | 1 | 1 | 0% | 3,692 | 4,293 | +16% | 0 | 0 | — |
case-09 | fail→fail | 29,885 | 28,458 | -5% | 1 | 1 | 0% | 6,175 | 5,463 | -12% | 0 | 0 | — |
case-10 | fail→pass | 21,023 | 30,999 | +47% | 1 | 1 | 0% | 3,125 | 5,797 | +86% | 0 | 0 | — |
case-11 | fail→fail | 15,170 | 30,161 | +99% | 1 | 1 | 0% | 3,171 | 6,748 | +113% | 0 | 0 | — |
case-12 | fail→pass | 16,573 | 28,476 | +72% | 1 | 1 | 0% | 2,976 | 5,550 | +86% | 0 | 0 | — |
case-13 | fail→pass | 33,067 | 25,960 | -21% | 1 | 1 | 0% | 1,535 | 5,207 | +239% | 0 | 0 | — |
case-14 | fail→fail | 25,076 | 26,375 | +5% | 1 | 1 | 0% | 4,580 | 5,337 | +17% | 0 | 0 | — |
case-15 | fail→pass | 16,122 | 22,397 | +39% | 1 | 1 | 0% | 3,145 | 5,037 | +60% | 0 | 0 | — |
case-16 | fail→pass | 28,587 | 22,532 | -21% | 1 | 1 | 0% | 5,403 | 4,742 | -12% | 0 | 0 | — |
case-17 | fail→pass | 30,156 | 31,929 | +6% | 1 | 1 | 0% | 5,640 | 6,601 | +17% | 0 | 0 | — |
case-18 | pass→pass | 16,681 | 17,387 | +4% | 1 | 1 | 0% | 3,468 | 4,016 | +16% | 0 | 0 | — |
case-19 | fail→fail | 6,284 | 17,327 | +176% | 1 | 1 | 0% | 1,014 | 3,460 | +241% | 0 | 0 | — |
case-20 | fail→fail | 9,104 | 18,501 | +103% | 1 | 1 | 0% | 1,602 | 3,925 | +145% | 0 | 0 | — |
case-21 | pass→pass | 20,198 | 32,873 | +63% | 1 | 1 | 0% | 3,360 | 6,747 | +101% | 0 | 0 | — |
case-22 | fail→pass | 24,249 | 20,561 | -15% | 1 | 1 | 0% | 4,208 | 4,371 | +4% | 0 | 0 | — |
DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 22 cases were attempted, and 21 counted toward the lift figure. The other 1 produced results that are not comparable between the two arms, so they are excluded from the headline rather than averaged into it. The headline lift of +45 percentage points is the difference between those two pass rates over the 21 comparable cases. 1 case got worse with the skill loaded, and it is included in that figure.
Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.
Other measured skills in the registry, with their headline benchmark lift.