Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →notque/skill-eval
Evaluate skills: trigger testing, A/B benchmarks, structure validation, head-to-head bake-offs.
Passed 16 of 20 test cases with the skill, and 6 of 20 without it.
and 8 more in the full report
11 fixed7 passed either way4 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 11 test cases the skill fixed. Same model, same prompt, with and without the skill.
After running a blind comparison between two skill outputs, which subagent file provides instructions for analyzing why the winning output surpassed the loser?
| Checked | Without skill | With skill |
|---|---|---|
| References agents/analyzer.md | Missed | Passed |
The instructions your model reads when the skill loads.
Measure and improve skill quality through empirical testing — because structure doesn't guarantee behavior, and measurement beats assumption. Also covers head-to-head bake-offs of two peer implementations of the same artifact (Mode F).
| Signal | Load These Files | Why |
|---|---|---|
| reading or writing eval artifacts: evals.json, grading.json, metrics.json, history.json | schemas.md | Loads detailed guidance from schemas.md. |
| improving a skill via variant generation and blind A/B promotion | self-improve-loop.md | Loads detailed guidance from self-improve-loop.md. |
| "bake-off", "head-to-head", "compare implementations", "grade two versions", "which persona skill is better" | bake-off-methodology.md | Loads the bake-off rubric, anti-rationalization gate, fold-filter, and worked persona example. |
Step 1: Identify the skill
# Validate skill structure first
python3 -m scripts.skill_eval.quick_validate <path/to/skill>
This checks: SKILL.md exists, valid frontmatter, required fields (name, description), kebab-case naming, description under 1024 chars, no angle brackets.
Step 2: Choose evaluation mode based on user intent
| Intent | Mode | Script |
|---|---|---|
| "Test if description triggers correctly" | Trigger eval | run_eval.py |
| "Optimize/improve the description through autoresearch" | Route to agent-comparison | optimize_loop.py |
| "Compare skill vs no-skill output" | Output benchmark | Manual + aggregate_benchmark.py |
| "Validate skill structure" | Quick validate | quick_validate.py |
| "Self-improve skill" / "optimize skill" / "improve skill with A/B" | Self-improvement loop | references/self-improve-loop.md |
| "Bake-off" / "head-to-head grade these two" / "compare X vs Y implementation" | Head-to-head bake-off | references/bake-off-methodology.md |
GATE: Skill path confirmed, mode selected.
test cases passed, vs 6 / 20 without
Published skills whose descriptions closely match this one, with their headline benchmark lift.
More skills from the same source and nearby in the registry, with their headline benchmark lift.