Use when planning or reporting the analysis for an American Educational Research Journal (AERJ) manuscript — multilevel/HLM and growth models, IRT/measurement, quasi-experimental estimation, or qualitative coding and thematic analysis. Analysis must meet the AERA reporting standards (warrant + transparency). Strengthens analysis reporting; it does not run models for you.
SkillScore82Maintained
Automatic
◆Research pack
Last verified 8/15/2026
What it changes
Passed 18 of 22 test cases with the skill, and 13 of 22 without it.
Without skill59%
With skill82%
Checks it passed that the model alone missed
The response requires addressing negative cases or disconfirming evidence.
The response requires demonstrating robustness to alternative specifications.
The response specifically recommends using the Romano-Wolf step-down procedure.
and 2 more in the full report
5 fixed13 passed either way4 still failing0 broke
Pass-rate lift
+23 pp
Tokens
+30%
Agent turns
0%
Response time
-32%
Model gemini-3.6-flashMeasured 8/15/2026v118 of 22 cases passed with skill
Tokens, turns and time describe resource use, not a dollar price.
See it in action
One of 5 test cases the skill fixed. Same model, same prompt, with and without the skill.
1 of 5
Prompt
We are running automated analysis replication checks on an existing regression output handle in Stata or R. Which command or tool re-fits off one handle to list missing diagnostic checks and suggest specific fix functions?
What was checked, without and with the skill
Checked
Without skill
With skill
The response identifies audit_result(result_id) as the execution command to list missing checks.
Missed
Passed
case-18 · gemini-3.6-flash · 8/15/2026 · answers are capped and redacted
What's inside
The instructions your model reads when the skill loads.
SKILL.md112 linessingle file
Data Analysis (aerj-data-analysis)
AERJ analyses must be warranted (adequate evidence for the claim) and transparent (explicit
logic of inquiry), per the AERA reporting standards. Whatever the method, report enough that a reader
can judge — and a replicator could reproduce — the result.
When to trigger
Specifying the analytic strategy or writing the results section
A reviewer questioned model specification, uncertainty, or coding rigor
Reporting effect sizes, fit, robustness, or qualitative warrant
Reconciling quantitative and qualitative results in a mixed-methods paper
Quantitative analysis norms
Respect nesting. Multilevel/HLM (or cluster-robust) inference for students-in-schools data;
report ICC, level-specific predictors, and random effects. Center predictors deliberately (group- vs
grand-mean) and say which.
Report effect sizes and uncertainty, not just p-values: standardized effects, confidence
intervals, and practical significance for education stakes.
Measurement. Report reliability and validity evidence; for scales, factor/IRT results; handle
measurement error rather than ignoring it.
Missing data. State the mechanism assumption and method (multiple imputation / FIML), not
listwise-by-default. Report attrition for longitudinal/experimental data.
Multiplicity. Adjust or pre-specify when testing many outcomes/subgroups.
Large-scale assessment data. Use plausible values and replicate/survey weights correctly.
Robustness. Show the result survives reasonable alternative specifications.
Qualitative analysis norms
Make the analytic process explicit: how codes/themes were developed, who coded, how disagreements
were resolved, and how interpretations were warranted by data.
Evidence the claims: quotations/excerpts tied to themes; negative cases acknowledged; saturation
or sufficiency addressed where relevant.
Reflexivity: how the researcher's position shaped generation and interpretation.
Mixed-methods integration
Report how the strands were integrated (joint displays, meta-inferences) and what the integration
revealed that neither strand alone could. Do not report two disconnected analyses.
Use when planning or auditing the analysis of an American Journal of Sociology (AJS) manuscript so the evidence credibly supports the theoretical claim. Covers analysis norms, uncertainty, robustness, and triangulation across quantitative, comparative-historical, and ethnographic work. Improves the analysis chain; it does not fabricate results.