Compute and report task-correct held-out metrics for a trained medical-imaging model — segmentation (Dice plus a boundary metric such as HD95 or NSD, per structure), classification (AUROC plus AUPRC and sensitivity/specificity with bootstrap CIs at the deployment prevalence), detection (FROC or mAP with a stated IoU criterion), interactive/promptable segmentation (the interaction-count, convergence, and per-case-time axes a static Dice omits), or generative/synthesis image evaluation (similarity
NewMaintained
On-demandOn-demand: invoked explicitly — never occupies your agent's context until called.
◆AI Engineer pack
Last verified 8/24/2026
What it changes
Passed 16 of 20 test cases with the skill, and 16 of 20 without it.
Without skill82%
With skill82%
Checks it passed that the model alone missed
Names check_metric_reporting.py with task and strict arguments.
Identifies metric gate error flags such as ACCURACY_ONLY or DETECTION_METRIC_MISSING.
States that actual prediction files and ground truth data are missing and halts rather than inventing synthetic metric results.
3 fixed15 passed either way1 still failing3 broke
Pass-rate lift
0 pp
Tokens
+32%
Agent turns
0%
Response time
-32%
Model gemini-3.6-flashMeasured 8/24/2026v1 · latest is v218 of 22 cases passed with skill · 20 comparable
Tokens, turns and time describe resource use, not a dollar price.
This benchmark measured an earlier version of the skill.
A newer attempt on gemini-3.6-flash on 10/3/2026 did not produce a publishable comparison (not comparable). The snapshot above is the previously measured run.
See it in action
One of 3 test cases the skill fixed. Same model, same prompt, with and without the skill.
1 of 3
Prompt
We finished evaluating our organ segmentation model and saved our findings in a summary markdown report. What automated CLI script and flags should be executed to deterministically verify metric selection compliance?
What was checked, without and with the skill
Checked
Without skill
With skill
Names check_metric_reporting.py with task and strict arguments.
Missed
Passed
case-12 · gemini-3.6-flash · 8/24/2026 · answers are capped and redacted
What's inside
The instructions your model reads when the skill loads.
SKILL.md3 linessingle file
/model-evaluation was renamed to /model-assessment in v6.0; this redirect is removed in v7.
Run /model-assessment now, passing the same arguments the user gave /model-evaluation.
Start at /model-assessment Part B, Phases 8–9 (task-correct held-out metrics and the metric-reporting gate).
Turn a folder of research PDFs into an Obsidian knowledge vault — consistently formatted literature notes with frontmatter, PDF embed links, and cross-referenced atomic concept notes. Use whenever the user wants PDFs converted to Obsidian notes, a batch of papers summarized into a common template, a research "second brain" built or extended, or concepts extracted across accumulated notes — even if they never say "Obsidian". Pairs with /lit-sync, which owns the same vault folders from the Zotero/