Gate fine-tuned checkpoints with drift budgets, paired comparison, and forgetting checks before promotion. Use after a training run produces a checkpoint, when deciding whether a tuned model ships, or when a promoted model needs re-gating against updated goldens.
SkillScore82Maintained
Automatic
◆AI Engineer pack
Last verified 8/7/2026
What it changes
Passed 16 of 20 test cases with the skill, and 6 of 20 without it.
Without skill32%
With skill82%
Checks it passed that the model alone missed
Declines promotion gating when baseline-model.json is missing.
Prevents pipeline script retries when a REJECT verdict is rendered.
States that target task gains cannot override a drift budget hard fail.
and 12 more in the full report
11 fixed7 passed either way4 still failing0 broke
Pass-rate lift
+50 pp
Tokens
+68%
Agent turns
0%
Response time
-12%
Model gemini-3.6-flashMeasured 8/7/2026v118 of 22 cases passed with skill · 20 comparable
Tokens, turns and time describe resource use, not a dollar price.
See it in action
One of 10 test cases the skill fixed. Same model, same prompt, with and without the skill.
1 of 10
Prompt
A checkpoint failed Stage 2 due to an 8-point drift drop on general benchmarks. The team wants to lower the learning rate and reduce epochs. Per the standard catastrophic forgetting remediation hierarchy, what is the first action that must be recommended?
What was checked, without and with the skill
Checked
Without skill
With skill
Recommends adjusting the replay-mix fraction as the first remediation step.
Passed
Passed
Specifies keeping total optimizer step count constant when modifying replay fraction.
Missed
Passed
case-10 · gemini-3.6-flash · 8/7/2026 · answers are capped and redacted
What's inside
The instructions your model reads when the skill loads.
SKILL.md292 lines2 files
Checkpoint Promotion
The Phase 5 gate for the whole
plugin: a checkpoint that trains
cleanly and beats its task metric
still doesn't ship without
clearing all four stages below.
eval-harness-first built the
suite re-run here — this skill is
where that suite's baseline
decides something.
Input: a trained checkpoint,
eval/baseline-<model>.json from
eval-harness-first, and the
frozen eval/drift-suite.yaml.
Output format:promotion-report.md — the
four-stage evidence plus a
terminal PROMOTE or REJECT
verdict that /finetune Phase 5
and /promote-checkpoint consume
directly.
The Four-Stage Gate
Each stage gates the next — a
failure at stage 2 means stage 3
doesn't run. Stages 2 and 3 share
one expensive inference pass, so
running them concurrently and
applying gate order at verdict
time is licensed on a
deterministic arena (nothing
saved by serializing); a
judge-based arena should still
wait for stage 2 first — that's
where the real savings are.