---
name: brycewang-stanford/ajps-replication-and-verification
source: https://app.decimal.ai/s/brycewang-stanford-ajps-replication-and-verification@1/SKILL.md
source_sha256: 05539783b602
---

# Replication & Verification (ajps-replication-and-verification)

This is AJPS's signature differentiator. Unlike journals that check reproducibility in-house (or not at
all), **AJPS sends your deposited materials to an independent third party that actually re-runs the
code** and confirms it reproduces the numerical results reported in the **main text** — *before*
publication. If it does not reproduce, the article does not publish until it does. Build the package as
you analyze (`ajps-data-analysis`); it cannot be improvised at acceptance.

## When to trigger

- Building the replication / verification package
- A manuscript is accepted and you have received upload instructions (after final draft, during the
  technical check, before copyediting)
- Data cannot be fully shared (privacy, ethics, provider restriction) and you need a path
- A **qualitative** or **multi-method** paper whose evidence needs a documented verification route

## How AJPS verification actually works (official baseline checked 2026-06-20)

1. **Deposit to the AJPS Dataverse.** Materials go into a Dataset within the **AJPS Dataverse on the
   Harvard Dataverse Network** — not a personal site or generic cloud link. Other copies may exist
   elsewhere as long as everything needed is in the AJPS Dataverse Dataset.
2. **Independent third-party verification.** A verifier re-runs the code and confirms it reproduces the
   **numerical results in the main text**. The published article carries a statement: *"The Cornell
   Center for Social Sciences verified that the data and replication code submitted to the AJPS
   Dataverse replicates the numerical results reported in the main text of this article."* MPSA also
   identifies Cornell Center for Social Sciences as the contracted replication-and-verification provider.
3. **Qualitative path.** Qualitative / multi-method materials follow the AJPS verification guidelines
   and qualitative checklist: document collection procedures, instruments, analytic operations, source
   fragments or access restrictions, and any exemption route needed for human-subjects, copyright, or
   legal constraints.
4. **Timing.** It happens **after the final draft is submitted, during the technical check, before
   copyediting** — late enough that fixing an unscripted analysis under deadline is painful.

## Build-as-you-go checklist (so the re-run matches)

- [ ] **readme.txt** lists and describes every file (group as "Data files", "Stata .do files", "Files
      to Reproduce Table 1", ...)
- [ ] One **master script** runs the modular scripts **in order** and sets the working directory once
- [ ] **set seed** for every stochastic step (Monte Carlo, bootstrap, randomization inference, jitter)
- [ ] Explicit **software-version** statements (e.g., "R 4.3.2", "Stata/MP 18.0"); packages pinned
- [ ] Every **main-text** number, table, and figure is regenerated by the code, names matched
- [ ] Restricted data: explain why, give exact access instructions, provide synthetic data where feasible
- [ ] Sharing terms and any access restrictions documented in the Dataverse materials
- [ ] Preregistration / pre-analysis plan linked (anonymized) where applicable

## When data cannot be shared

- **Explain why** (ethics, privacy, legal/provider restriction).
- Provide **README instructions on exactly how others can obtain the data** (process, application, contact).
- Where possible, **provide synthetic data** resembling the real data so the code can run end to end.

## Anti-patterns

- Treating the deposit as a post-publication afterthought (it **gates** publication)
- Depositing code that does not actually regenerate the printed main-text numbers
- A personal URL instead of the AJPS Dataverse
- Unseeded, unpinned, hard-coded-path code that "works on my machine" but not on the verifier's
- Claiming data are restricted with no access path or synthetic substitute

## Output format

```
【Repository】AJPS Dataverse (Harvard) — Dataset staged? [Y/N]
【Reproduces main-text numbers?】master script re-run locally, matches? [Y/N]
【Files】readme.txt + master script + seeds + pinned versions? [Y/N]
【Verifier】Cornell Center for Social Sciences / AJPS qualitative checklist path
【Restricted data?】why + access path + synthetic data?
【Sharing terms】Dataverse/exemption notes complete?
【Next】ajps-review-process
```

## Supplementary resources

- [`../../resources/external_tools.md`](../../resources/external_tools.md) — replication-package tooling and qualitative-transparency options
- [`../../resources/official-source-map.md`](../../resources/official-source-map.md) — AJPS Replication & Verification Policy, verifier transition, and AJPS Dataverse