Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →brycewang-stanford/data-cleaning-pipeline
Systematic data cleaning workflows for research datasets
Passed 15 of 23 test cases with the skill, and 10 of 23 without it.
and 2 more in the full report
5 fixed10 passed either way8 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 5 test cases the skill fixed. Same model, same prompt, with and without the skill.
Write a Python function impute_missing_values using scikit-learn imputers that handles both numeric and categorical columns. What default strategy should be applied for numeric data, what strategy for categorical data, and how can KNN be supported for numeric features?
| Checked | Without skill | With skill |
|---|---|---|
| Defaults to median strategy for numeric imputation. | Missed | Passed |
| Uses most frequent strategy for categorical imputation. | Passed | Passed |
| Configures KNNImputer with 5 neighbors when KNN strategy is selected. | Passed | Passed |
The instructions your model reads when the skill loads.
A skill for building systematic, reproducible data cleaning pipelines for research datasets. Covers common data quality issues, step-by-step cleaning workflows, handling missing values, detecting and treating outliers, validating data integrity, and documenting cleaning decisions for reproducibility.
Data cleaning should follow a consistent, documented order. Each step builds on the previous one, and the entire pipeline should be scripted for reproducibility.
Data Cleaning Pipeline (recommended order):
1. Initial Assessment
- Load data, check dimensions, inspect dtypes
- Generate summary statistics and missing value report
- Identify structural issues (merged cells, inconsistent delimiters)
2. Structural Fixes
- Standardize column names (snake_case, no spaces)
- Fix data types (strings to numbers, dates, categories)
- Split or merge columns as needed
- Remove completely empty rows/columns
3. Deduplication
- Identify exact duplicates
- Identify near-duplicates (fuzzy matching)
- Decide keep-first, keep-last, or merge strategy
4. Missing Value Treatment
- Classify missingness mechanism (MCAR, MAR, MNAR)
- Apply appropriate imputation or exclusion strategy
- Document and justify missing data decisions
5. Outlier Detection and Treatment
- Statistical methods (IQR, z-score, Mahalanobis)
- Domain-based validation (impossible values)
- Decide: correct, cap, remove, or keep with flag
6. Consistency Checks
test cases passed, vs 10 / 23 without
Published skills whose descriptions closely match this one, with their headline benchmark lift.
More skills from the same source and nearby in the registry, with their headline benchmark lift.