Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →brycewang-stanford/atc-reproducibility
Use when building the reproducibility story for an ATC (ACM SIGOPS Annual Technical Conference, formerly USENIX ATC) systems paper — pinning testbed and software environments, providing a turnkey path from the artifact to the headline numbers, and preparing an anonymized-but-runnable review package ahead of the Available/Functional/Reproduced badges.
Passed 20 of 22 test cases with the skill, and 19 of 22 without it.
1 fixed19 passed either way2 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
A test case the skill fixed. Same model, same prompt, with and without the skill.
Our evaluation scripts run without errors on our development workstations. Is this sufficient to guarantee that evaluators will not experience Functional failures?
| Checked | Without skill | With skill |
|---|---|---|
| Recommends testing the repository from a fresh checkout on a clean machine. | Missed | Passed |
The instructions your model reads when the skill loads.
Build the reproducibility story alongside the system, not at the deadline. ATC has an active
artifact culture inherited from USENIX: reviewers expect a runnable, anonymized artifact at review
time, and after acceptance an Artifact Evaluation Committee awards Available / Functional /
Reproduced badges (see atc-artifact-evaluation). The through-line is that a systems result other
people can re-run is worth more than one they must take on faith — and systems provenance cannot be
reconstructed after the fact.
Record these at collection time; none can be recovered at the deadline:
[Hardware] CPU/NIC/SSD models, core/memory counts, firmware/BIOS where it matters
[OS/kernel] kernel version, distro, relevant sysctl/tuning, hugepages/NUMA settings
[Toolchain] compiler, library, and runtime versions; build flags
[Workload] trace source + extraction date, generator version + seeds, request mix
[Method] warm-up window, measurement duration, run count, aggregation method
[Code] commit SHAs for your system and every baseline; patches applied
The single most valuable artifact property is that an evaluator can regenerate your paper's main figures and tables:
./run_fig3.sh) that does setup, run,
and plot.requirements, Nix) over
"install these 30 packages by hand."test cases passed, vs 19 / 22 without
Published skills whose descriptions closely match this one, with their headline benchmark lift.
More skills from the same source and nearby in the registry, with their headline benchmark lift.