Use when the project collects primary data or runs a field, lab, or survey experiment, before the intervention begins — write the pre-analysis plan, size the sample from a power calculation, and register with the AEA RCT Registry. Apply after the design is chosen in aer-identification and before any outcome data are seen.
SkillScore91Maintained
Automatic
◆Research pack
Last verified 8/8/2026
What it changes
Passed 20 of 22 test cases with the skill, and 17 of 22 without it.
Without skill77%
With skill91%
Checks it passed that the model alone missed
The response states that pre-registration should not be performed for realized secondary observational data.
The response advises focusing on robustness checks and specification sensitivity rather than pre-registration.
The response directs the user to observational robustness methods rather than drafting an experimental pre-analysis plan.
and 2 more in the full report
3 fixed17 passed either way2 still failing0 broke
Pass-rate lift
+14 pp
Tokens
+42%
Agent turns
0%
Response time
-23%
Model gemini-3.6-flashMeasured 8/8/2026v120 of 22 cases passed with skill
Tokens, turns and time describe resource use, not a dollar price.
See it in action
One of 3 test cases the skill fixed. Same model, same prompt, with and without the skill.
1 of 3
Prompt
We are evaluating the impact of the 2014 Medicaid expansion on county-level mortality using existing CMS administrative data from 2008 to 2019. We plan to write a pre-analysis plan to post on an open science platform before running our difference-in-differences regressions. Should we register a pre-analysis plan for this project, and how should we structure it?
What was checked, without and with the skill
Checked
Without skill
With skill
The response states that pre-registration should not be performed for realized secondary observational data.
Missed
Passed
The response directs the user to observational robustness methods rather than drafting an experimental pre-analysis plan.
Missed
Passed
case-04 · gemini-3.6-flash · 8/8/2026 · answers are capped and redacted
What's inside
The instructions your model reads when the skill loads.
SKILL.md149 lines2 files
AER Pre-Registration
Overview
For experimental and prospective-data AER-track work, credibility is bought
before the data exist. This skill writes the pre-analysis plan (PAP), sizes
the sample from a power calculation, and registers the study. Its job is to make
the eventual results un-p-hackable: a referee who sees a public PAP timestamp
predating data collection cannot accuse you of specification search.
Registration is not optional at AEA journals for RCTs. If the design is
observational, skip to aer-robustness; there is nothing to pre-register.
When to Use
The paper runs a randomized controlled trial (field, lab, or survey experiment)
The project collects primary data whose analysis should be pre-committed
A referee or editor asks for the trial registration number or the PAP
Reviewers suspect the reported specification was chosen after seeing outcomes
You must justify the sample size, or explain why an effect was "not detected"
Register-or-Not Decision
Is treatment assigned by the researcher (randomization)?
├── Yes → register with the AEA RCT Registry BEFORE the intervention; write a PAP
├── No, but you collect new primary data → post a PAP for the pre-committed analysis
└── No, secondary/observational data already realized → do NOT pre-register
(a PAP written after the outcomes exist is theater); go to aer-robustness
Pre-Analysis Plan Contents
Pre-specify, in order, and timestamp before unblinding:
Hypotheses — each stated as a signed, testable prediction, not a topic:
H1: the cash transfer raises household consumption at endline by at
least 0.15 SD, estimated by OLS of log consumption on treatment with
strata fixed effects, clustered at the village level.