Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →Rigor Reproduce compatible skill slug for README-first deep learning repository reproduction.
Passed 10 of 19 test cases with the skill, and 3 of 19 without it.
and 11 more in the full report
9 fixed3 passed either way9 still failing1 broke
Tokens, turns and time describe resource use, not a dollar price.
This benchmark measured an earlier version of the skill.
A newer attempt on gemini-3.6-flash on 10/2/2026 did not produce a publishable comparison (not comparable). The snapshot above is the previously measured run.
One of 9 test cases the skill fixed. Same model, same prompt, with and without the skill.
Before running scripts in `./bert-fine-tuning`, we need a read-only check of Python module availability without altering the environment. Which entrypoint script in the reproduction framework should be executed?
| Checked | Without skill | With skill |
|---|---|---|
| Invokes scripts/doctor.py for read-only module verification. | Missed | Passed |
| Passes --repo or --require-module arguments to check repository requirements. | Missed | Passed |
The instructions your model reads when the skill loads.
Guide README-first deep learning reproduction toward the smallest trustworthy run with auditable evidence. Preserve documented meaning; record assumptions, deviations and blockers instead of changing semantics to manufacture success. Load specialized references only for a concrete uncertainty.
For a routine bounded run, keep the control path short:
scripts/orchestrate_repro.py --repo <repo> --plan-only --agent-output with any explicit user timeout bound (--timeout or --train-timeout) already supplied; review command_candidates, the selected cmd-XX, side-effect contract, selection fingerprint, and returned reviewed_run_args. With no --output-dir, later evidence goes to <repo>/repro_outputs regardless of caller cwd.--run-selected --command-id <cmd-XX> --plan-fingerprint <fingerprint> --agent-output plus requested timeout/metric/source-adjacent options. Preserve an explicit user command-timeout bound instead of silently making it stricter on a routine trusted run. --timeout limits the target command; do not wrap the whole orchestrator in an equal or shorter external timeout, because it still needs time to terminate children and write terminal evidence. A changed command set fails closed; setup/download commands are never target candidates.--verify-output --agent-output; inspect detailed evidence files only when verification fails or the result is partial/blocked.For a host with short tool-call deadlines, rerun planning with --include-agent-handoff and follow references/agent-job.md; otherwise keep the direct path above. Job completion is not task acceptance, and uncertain state is never a reason for automatic replay.
Do not inspect orchestrate_repro.py, annotate_readme.py, _bundled/, writers, or runtime internals on a normal success path. Inspect implementation only for a concrete blocker, unexpected side effect, bundle-integrity failure, or unresolved safety question. Use scripts/doctor.py for first-use environment/install diagnostics. Executed commands keep full lifecycle/log evidence under repro_outputs/_runtime/<run_id>/.
Use this skill for repository-grounded, multi-phase trusted reproduction where the goal is a small reproducible target. Do not use it for paper summaries, generic setup, isolated scanning, standalone commands, open-ended research design, or explicitly authorized candidate exploration.
Choose the smallest target that can honestly demonstrate repository-grounded reproduction:
test cases passed, vs 3 / 19 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.