Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →brycewang-stanford/chi-reproducibility
Use when strengthening research transparency for an ACM CHI paper — protocols, instruments, codebooks, analysis scripts, preregistration, and data availability under human-subjects constraints — so methods survive the ADR-Method screening and others can actually build on the work.
Passed 20 of 23 test cases with the skill, and 15 of 23 without it.
and 4 more in the full report
7 fixed13 passed either way1 still failing2 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 7 test cases the skill fixed. Same model, same prompt, with and without the skill.
What is the primary risk during initial screening if a CHI submission completely conceals its analysis code, measurement details, and protocol materials without justification?
| Checked | Without skill | With skill |
|---|---|---|
| The response identifies research opacity as a ground for desk rejection under ADR-Method screening. | Missed | Passed |
The instructions your model reads when the skill loads.
Reproducibility at CHI is not "same script, same numbers." Human-subjects research reproduces at the level of protocol and analysis: could a competent lab run your study again, and could a skeptic re-derive your findings from your materials? CHI's screening now names "research transparency" explicitly inside the ADR-Method assisted desk-reject ground, so opacity is a pre-review rejection risk. The working principle for data: as open as consent allows, as documented as possible where it does not.
| Layer | What must be true | Typical artifacts |
|---|---|---|
| Protocol | Another lab could run the study | Task descriptions, scripts read to participants, stimuli, apparatus specs, recruitment text, screening criteria, compensation |
| Analysis | A skeptic could re-derive results from your data | Analysis code, codebook + coding decisions, exclusion rules, model specifications, software versions |
| Data | Shared where consent permits; described honestly where not | De-identified quantitative data, aggregate tables, transcript excerpts, or a documented reason why not |
The protocol layer is the cheapest and the most neglected: your consent scripts,
questionnaires, and interview guides already exist — publishing them in the
supplement costs an afternoon and answers half of the methods questions reviewers
would otherwise raise (chi-supplementary).
chi-submission).Qualitative work cannot ship a replication button; it can ship an audit trail:
test cases passed, vs 15 / 23 without
Published skills whose descriptions closely match this one, with their headline benchmark lift.
More skills from the same source and nearby in the registry, with their headline benchmark lift.