Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →brycewang-stanford/chi-experiments
Use when designing or auditing the studies behind an ACM CHI paper — matching evidence shape to contribution type, powering quantitative experiments, making qualitative work rigorous and auditable, reporting participants and ethics properly, and avoiding the ADR-Data and ADR-Method screening grounds.
Passed 22 of 22 test cases with the skill, and 17 of 22 without it.
and 3 more in the full report
5 fixed17 passed either way0 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 5 test cases the skill fixed. Same model, same prompt, with and without the skill.
I am preparing a CHI manuscript evaluating an adaptive UI technique against a static baseline in a within-subjects lab study with 12 participants where we reported p-values and brief post-task feedback. Please audit my study design and check for potential desk-reject risks. I need a structured response showing my contribution category, an inventory of the evidence collected, any unsupported claim dead ends, quantitative rigor status (power calculation, effect sizes/CIs, analysis provenance), qualitative rigor status (named method, quote balance), ethical reporting status, and an assessment of desk-rejection exposure for data and method including the primary vulnerability.
| Checked | Without skill | With skill |
|---|---|---|
| Identifies ADR-Data exposure as high due to the small sample size of 12 for a quantitative comparison. | Passed | Passed |
| Uses the exact line prefix format bracket [Contribution type] at the beginning of the summary output. | Missed | Passed |
The instructions your model reads when the skill loads.
"Experiments" at CHI means human evidence: controlled lab studies, field deployments, interview and diary studies, surveys, log analyses, and mixtures of these. Two of the four assisted desk-reject rubric grounds CHI now screens with — ADR-Data (grossly insufficient data for the claims) and ADR-Method (grossly insufficient methodological detail or transparency) — are study-design judgments made before full review. Evidence design is therefore survival, not polish.
| Claim shape | Evidence that convinces CHI reviewers | Chronic mismatch seen in reviews |
|---|---|---|
| "Technique X outperforms Y" | Controlled comparison, counterbalanced, powered, effect sizes | Underpowered n=12 with p-values only |
| "Users experience/need Z" | Interviews or diary study to saturation, systematic analysis | Cherry-picked quotes, no analysis method stated |
| "System S is usable/useful in practice" | Field deployment with real tasks over time | One-hour lab walkthrough of a demo |
| "Population P interacts differently" | Sampling strategy that can reach P, comparative design | Convenience sample of students standing in for P |
| "Design guideline G holds" | Multiple probes/instantiations, triangulated methods | Single prototype, single context, universal claim |
| "Measure M captures construct C" | Validation study: reliability, convergent validity | New questionnaire used, never validated |
Mixed methods are a CHI signature: a quantitative result explains that, the paired qualitative strand explains why. If you run both, integrate them in the analysis — a qualitative section bolted after the ANOVA reads as decoration.
# a priori sample size for a within-subjects comparison (paired t-test)
from statsmodels.stats.power import TTestPower
n = TTestPower().solve_power(effect_size=0.5, alpha=0.05, power=0.8,
alternative="two-sided")
print(round(n)) # ≈ 34 participants for d=0.5 — n=12 detects only d≈0.88
test cases passed, vs 17 / 22 without
Published skills whose descriptions closely match this one, with their headline benchmark lift.
More skills from the same source and nearby in the registry, with their headline benchmark lift.