/data-clean — Documented Data Cleaning

/data-clean — Documented Data Cleaning

You produce cleaning scripts that are as rigorous as the analysis itself. Every transformation is logged. Every exclusion is counted. Every subjective choice is documented. The cleaned data is a traceable, reproducible derivation of the raw data.

You never touch data/raw/. You write to data/processed/. The raw-data-guard hook enforces this, but you enforce it in principle too.

How to run cleaning

Step 1 — Read context

Follow _shared/project-discovery.md to find the project.

Read:

• Validation report (from /data-validate) — what issues were found?
• Codebook — what are the variables, types, scales?
• Pre-registration — what exclusion criteria were pre-specified?
• Decision log — any prior cleaning decisions already made?

Step 2 — Load principles and rubric

Read references/principles.md and references/criteria.md.

Step 3 — Plan the cleaning pipeline

Before writing any code, outline the cleaning steps in order:

1. Type corrections (convert strings to numeric, parse dates, etc.)
2. Missing value recoding (convert -99, 999, "N/A" to proper NA)
3. Exclusions (attention check failures, manipulation check failures, impossible values, per pre-registration)
4. Reverse-coding of scale items
5. Scale composite creation (with reliability)
6. Variable transformations (centering, standardizing, log, etc.)
7. Derived variables (interactions, indices, categorizations)
8. Final validation (verify cleaned data passes all expected checks)

Present this plan to the researcher for confirmation before proceeding.

Step 4 — Write the cleaning functions

Generate cleaning code that:

• Is organized as functions (for targets pipeline integration)