Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →brycewang-stanford/cav-reproducibility
Use when strengthening CAV (Computer Aided Verification) reproducibility, covering benchmark provenance (SV-COMP/SMT-COMP/HWMCC/VNN-COMP set revisions), pinned tool and baseline versions, resource limits and hardware, seeds for randomized/portfolio solvers, checkable proof witnesses/certificates for soundness claims, and consistency between the paper's tables and the artifact.
Passed 18 of 22 test cases with the skill, and 12 of 22 without it.
and 6 more in the full report
9 fixed9 passed either way1 still failing3 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 9 test cases the skill fixed. Same model, same prompt, with and without the skill.
Our paper Table 2 lists an average run time of 14.2 seconds for the R3 division, but our artifact log files show average run times of 28.6 seconds because the logs were generated on an older 4-core machine before we upgraded to a 64-core server. What fix is required?
| Checked | Without skill | With skill |
|---|---|---|
| Identifies discrepancy between paper tables and artifact execution logs. | Passed | Passed |
| Requires regenerating PDF table numbers from actual logged artifact runs. | Missed | Passed |
The instructions your model reads when the skill loads.
Use this before submission and again before camera-ready. In computer-aided verification, reproducibility is not a courtesy — a benchmark result is only meaningful relative to a fixed benchmark set, pinned tool versions, and a stated resource budget, and a soundness claim is only credible if it ships a checkable witness. The goal is that a competent reader could rerun your evaluation and re-check your correctness claims and reach your conclusions.
| Claim in the paper | Weak answer | CAV-ready answer |
|---|---|---|
| "Faster than solver X" | "X was slower in our tests" | X vA.B, its documented config, same time/memory limit, same hardware; per-instance data in the artifact |
| "Solves N hard instances" | "on standard benchmarks" | The named division of set , the exact instance list, the fetch/pin in the artifact |
| "Our result is sound/UNSAT" | asserted | The unsat proof + a bundled independent checker that accepts it |
| "Randomized search finds it" | one lucky run | Fixed seed(s), number of runs, variance reported |
| "Scales to large designs" | "large" | The size metric and the largest instance run, with the timeout that bounds it |
[Benchmarks] pin the set + revision (SV-COMP/SMT-COMP/HWMCC/VNN-COMP subset); archive the instance
list and the fetch script, not just "the standard benchmarks"
[Tools] record exact versions (yours and every baseline), build flags, and the commit/tag
[Limits] state per-instance wall-clock and memory limits, core count, and the hardware/CPU
test cases passed, vs 12 / 22 without
Published skills whose descriptions closely match this one, with their headline benchmark lift.
More skills from the same source and nearby in the registry, with their headline benchmark lift.