Rigor Run skill for README-first deep learning repo reproduction.
SkillScore43Maintained
Automatic
◆Research pack
Last verified 9/6/2026
What it changes
Passed 5 of 12 test cases with the skill, and 2 of 12 without it.
Without skill13%
With skill43%
Checks it passed that the model alone missed
The response refrains from initializing training state tracking.
The response indicates that normalized output files belong in repro_outputs.
The response logs completed sub-test evidence in the repro_outputs directory.
and 11 more in the full report
9 fixed1 passed either way11 still failing2 broke
Pass-rate lift
+30 pp
Tokens
-20%
Agent turns
0%
Response time
-27%
Model gemini-3.6-flashMeasured 9/6/2026v210 of 23 cases passed with skill · 12 comparable
Tokens, turns and time describe resource use, not a dollar price.
See it in action
One of 9 test cases the skill fixed. Same model, same prompt, with and without the skill.
1 of 9
Prompt
When standardizing run evidence and writing structured output metrics to disk after a reproduction run, which dedicated python helper script handles writing those standardized artifact files?
What was checked, without and with the skill
Checked
Without skill
With skill
The response identifies scripts/write_outputs.py as the output helper script.
Missed
Passed
The response indicates that normalized output files belong in repro_outputs.
Missed
Passed
case-17 · gemini-3.6-flash · 9/6/2026 · answers are capped and redacted
What's inside
The instructions your model reads when the skill loads.
SKILL.md53 lines7 filesincludes scripts
minimal-run-and-audit
Use this as the Rigor Run skill. The installed slug remains
minimal-run-and-audit for compatibility.
Use the shared operating principles in
../ai-research-reproduction/references/agent-operating-principles.md; this skill should make run
evidence auditable without turning every command into a rigid protocol.
When to apply
After a reproduction target and setup plan exist.
When the main skill needs execution evidence and normalized outputs.
When a smoke test, documented inference run, documented evaluation run, or other short non-training verification is appropriate.
When the user already knows what command should be attempted and wants execution plus reporting only.
When not to apply
During initial repo scanning.
When environment or assets are still undefined enough to make execution meaningless.
When the task is a literature lookup rather than repository execution.
When the user is still deciding which reproduction target should count as the main run.
Clear boundaries
This skill owns normalized reporting for an attempted command.
It may receive execution evidence from the main skill or a thin helper.
It does not choose the overall target on its own.
It does not perform broad paper analysis.
It does not own training startup, resume, or long-running training state.
It should not normalize risky code edits into acceptable practice.
It must not hide changes that alter evaluation, preprocessing, checkpoints,
metrics, or other scientific meaning.