Use when designing the experiments for a Cognitive Psychology (Elsevier) manuscript so they tightly control confounds, discriminate competing models, and have adequate power across a multi-experiment program. Hardens stimulus construction, counterbalancing, design logic, and sample-size justification; it does not write analysis or modeling code.
SkillScore77Maintained
Automatic
◆Research pack
Last verified 9/1/2026
What it changes
Passed 17 of 22 test cases with the skill, and 13 of 22 without it.
Without skill59%
With skill77%
Checks it passed that the model alone missed
The response includes the tag 【Confound control】.
The response recommends estimation using rdrobust.
The response requires reporting matched stimulus pools.
and 7 more in the full report
6 fixed11 passed either way3 still failing2 broke
Pass-rate lift
+18 pp
Tokens
+30%
Agent turns
0%
Response time
-5%
Model gemini-3.6-flashMeasured 9/1/2026v117 of 22 cases passed with skill
Tokens, turns and time describe resource use, not a dollar price.
See it in action
One of 6 test cases the skill fixed. Same model, same prompt, with and without the skill.
1 of 6
Prompt
A journal reviewer questioned whether my 3-study behavioral priming protocol actually differentiates race models from drift-diffusion models, claiming Study 2 is a near-duplicate and the interaction lacks power. Please review my revised design. Format your response to check model discrimination capability, confound controls, power/trial justification for the critical parameter, preregistration lockdown, an explanation of each experiment's incremental inference, and the logical follow-up step.
What was checked, without and with the skill
Checked
Without skill
With skill
The response includes the tag 【Confound control】.
Missed
Passed
The response includes the tag 【Next】cogpsych-data-analysis.
Missed
Passed
case-02 · gemini-3.6-flash · 9/1/2026 · answers are capped and redacted
What's inside
The instructions your model reads when the skill loads.
SKILL.md114 linessingle file
Experiment Design (cogpsych-study-design)
Cognitive Psychology expects tightly controlled cognitive experiments whose design is engineered to
discriminate models, organized as a multi-experiment program in which each experiment adds
inference. The craft is in stimulus construction, counterbalancing, confound control, and powering the
critical contrast — not just the main effect. Co-design the experiments with the model
(cogpsych-theory-and-hypotheses).
When to trigger
Designing an experiment or a multi-experiment series
Constructing stimuli, item sets, and counterbalancing schemes
A reviewer questioned confounds, power, design logic, or whether the design discriminates the models
Justifying sample size for the critical contrast (often an interaction)
Design standards
Design for discrimination. Build the design so the data produce the signature that separates
the models (e.g., a manipulation that the rival accounts predict to diverge). A design that both
models predict equally well wastes the experiment.
Control researcher and stimulus degrees of freedom. Counterbalance condition/item assignment;
control low-level confounds (frequency, length, familiarity, response mapping); randomize order; use
attention/manipulation checks. Document the full stimulus pool, not a curated subset.
Power the critical contrast. Justify N (and trials per cell) for the discriminating effect —
often an interaction or a model parameter — not the easy main effect. State the assumed effect size
and its source. Trials-per-participant is part of power for within-subjects designs.
Multi-experiment logic. Say what each experiment adds: rules out a confound, extends scope,
replicates the critical pattern, or tests a further model prediction. Avoid near-duplicate runs.
Validity. Argue construct validity (does the task measure the process the model is about) and the
generality of the claim across the stimulus space and population.
Powering the critical contrast — worked example (illustrative)
For the recognition-memory program, power the z-ROC shape contrast, not just overall accuracy.
Critical contrast: the diagnostic difference in z-ROC curvature between
UVSD and DPSD predictions.
Within-subjects: trials per participant drive ROC precision — target enough
old/new trials per confidence bin to estimate the slope reliably