Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →brycewang-stanford/aer-referee-sim
Use when a complete draft exists and needs an adversarial internal review before submission — simulating the AER desk screen and three referee reports with calibrated severity, scoring the paper against the editorial rubric, and producing a prioritized revise list. Apply after aer-consistency passes and before aer-submission; rerun until the simulated verdict is at least major R&R.
Passed 14 of 19 test cases with the skill, and 4 of 19 without it.
and 10 more in the full report
13 fixed3 passed either way4 still failing2 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 13 test cases the skill fixed. Same model, same prompt, with and without the skill.
Referee 2 notes that `housing_subsidies.pdf` duplicates the main identification strategy and sample of a 2023 working paper without citing it. Which workflow skill should be assigned to fix this issue in the revision list?
| Checked | Without skill | With skill |
|---|---|---|
| Routes the novelty critique to aer-literature or aer-topic-selection. | Missed | Passed |
The instructions your model reads when the skill loads.
Most papers submitted to AER are rejected; the realistic acceptance rate is 6-8 percent, and a large share never reach referees. The cheapest referee report is the one generated before submission — but only if it is as harsh as the real one. The failure mode of self-review (human or AI) is leniency: reviewing the paper one hopes was written instead of the one on the page.
This skill runs the AER editorial process against the draft: a ten-minute desk screen, then three referee reports written from distinct, adversarial priors, then an editor's synthesis with a calibrated verdict and a prioritized revise list. The simulation has one rule that overrides all others:
The simulated reviewers' job is to reject the paper. Every comment must survive the question "would this withstand the authors' best rebuttal?" — but praise requires the same evidence as criticism.
aer-consistency reports all-passDo not use on a half-draft — the simulation will correctly report that the
paper is incomplete, which wastes the run. And do not let it replace
aer-consistency: typo-hunting referees are wasted referees.
Simulate the editor's first pass: ten minutes, first three pages, then the main tables, then the bibliography. The editor is deciding only one thing — is this worth three referees' time?
test cases passed, vs 4 / 19 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.