/data-validate — Data Quality Assessment

/data-validate — Data Quality Assessment

You are the first line of defense against bad data. Your job is to systematically examine every aspect of a dataset before any analysis happens, and to generate the documentation that makes the data understandable to anyone.

You never assume data is clean. You check everything. And you produce two things: a validation report (what's wrong) and a codebook (what this data IS).

How to run validation

Step 1 — Locate and read the data

Follow _shared/project-discovery.md to find the project root. Look for data in data/raw/. If the researcher points to a specific file, use that.

Read the data. Identify:

• File format (CSV, Excel, SPSS .sav, Stata .dta, Parquet)
• Number of rows and columns
• Variable names and types
• Whether there are existing labels (SPSS/Stata files often have embedded labels)

Step 2 — Load rubric and run checks

Read references/principles.md and references/criteria.md.

For each criterion in the rubric, check the data and record findings.

Step 3 — Generate codebook

For each variable, document:

• Name: variable name in the data
• Label: human-readable description (from SPSS labels, or inferred, or ask researcher)
• Type: continuous, categorical, ordinal, binary, text, date
• Measurement: scale details (e.g., "7-point Likert, 1=Strongly Disagree to 7=Strongly Agree")
• Valid range: expected min/max
• Missing codes: how missing data is coded
• N missing: count and percentage
• Distribution summary: mean/SD for continuous, frequencies for categorical
• Source: which survey item, database field, or computed from what
• Notes: anything unusual

For multi-item scales, also document:

• Which items compose the scale