Use when defending the research design of an American Educational Research Journal (AERJ) manuscript — quantitative (multilevel, IRT, quasi-experimental, RCT), qualitative (case study, ethnography, interview), or mixed methods. AERJ judges each tradition on its own terms against the AERA reporting standards. Strengthens the design; it does not write code.
NewMaintained
Automatic
◆Research pack
Last verified 8/15/2026
What it changes
Passed 17 of 23 test cases with the skill, and 18 of 23 without it.
Without skill78%
With skill74%
Checks it passed that the model alone missed
Recommends using the Romano-Wolf stepdown procedure for multi-outcome family-wise error control.
1 fixed16 passed either way4 still failing2 broke
Pass-rate lift
-4 pp
Tokens
+38%
Agent turns
0%
Response time
-18%
Model gemini-3.6-flashMeasured 8/15/2026v117 of 23 cases passed with skill
Tokens, turns and time describe resource use, not a dollar price.
See it in action
A test case the skill fixed. Same model, same prompt, with and without the skill.
Prompt
In our randomized trial of a social-emotional learning intervention, we are testing impacts across 18 distinct behavioral and academic subscales, reporting any result with p < 0.05 as statistically significant. What adjustment method should we apply to control family-wise error rates across these multiple endpoints?
What was checked, without and with the skill
Checked
Without skill
With skill
Recommends using the Romano-Wolf stepdown procedure for multi-outcome family-wise error control.
Missed
Passed
case-07 · gemini-3.6-flash · 8/15/2026 · answers are capped and redacted
What's inside
The instructions your model reads when the skill loads.
SKILL.md114 linessingle file
Research Design (aerj-research-design)
AERJ accepts many methodologies but is demanding about each. The design must credibly connect the
framework (aerj-theory-and-framework) to evidence and meet the relevant AERA reporting standards.
This skill is mode-aware: name the dominant education-research lens and defend it against the strongest
alternative explanation.
When to trigger
Specifying sampling, measurement, identification, case selection, or an integration plan
A reviewer questioned causal claims, generalizability, trustworthiness, or measurement validity
Preparing a pre-analysis plan / preregistration for a prospective design
Justifying how the design addresses the rival account from aerj-literature-positioning
Quantitative (the field's common designs)
Nesting is the default. Students in classrooms in schools — use multilevel/HLM models;
specify levels, random effects, and cluster-correct inference. Report the design effect / ICC.
Measurement. Tie constructs to validated instruments; report reliability and, where relevant,
IRT/factor evidence. Validity is a design issue, not an afterthought.
Causal claims need a credible design: RCT (with power/MDE, attrition, fidelity), or
quasi-experimental (DID/event study with modern estimators, RD, IV, matching) — defend identifying
assumptions, don't assert them. Map to What Works Clearinghouse-style expectations when claiming effects.
Large-scale assessment data require plausible values and replicate/survey weights.
Qualitative (judged on its own terms)
Case/site/participant selection justified by design logic (typical, extreme, theoretical
sampling), not convenience. Say what the case is a case of.
Data and analysis: how data were generated, how coding/interpretation proceeded, how themes
were warranted by evidence (hand off to aerj-data-analysis).
Mixed methods
State the design type (convergent, explanatory-sequential, exploratory-sequential, embedded) and
the rationale for mixing — what integration buys you that one strand cannot.
Plan the point and method of integration (e.g., joint displays); avoid two papers stapled together.
The adjudication test (AERJ-specific)
For the single strongest rival explanation, write one sentence: *"If the rival were true rather than
Use when defending the research design of an American Journal of Political Science (AJPS) manuscript — causal identification for observational work, experimental and survey-experimental design, formal-empirical linkage, or case-based inference. AJPS reviewers are quantitatively demanding, so identification must license the claim being made. Strengthens the design; it does not write code.