---
name: thomson-li/eda-report
source: https://app.decimal.ai/s/thomson-li-eda-report@1/SKILL.md
source_sha256: f6fc652406c8
---

# EDA Report

Profile an unfamiliar tabular dataset and surface what matters before any modeling.

## When to use

The user has a dataset and wants to understand its shape and quality: new data
file, "what's in here?", "any data issues?", "should I clean anything?", pre-modeling
sanity checks. **Not** for production dashboards (use a dashboard skill) or for the
modeling itself (use `ml-classification-pipeline`).

## Workflow

1. **Load & confirm.** Read the file (`pandas.read_csv/read_excel/read_parquet`).
   Print shape and the first few rows. Confirm the intended **target column** with
   the user if one exists; if unclear, ask before going deep.

2. **Structure.** Report per-column: dtype, non-null count, % missing, # unique.
   Flag columns that are all-null, constant, or near-constant (≥ 99% one value),
   and likely-ID columns (unique ≈ row count).

3. **Missingness.** Rank columns by missing %. Note any rows missing the target.
   Distinguish "missing at random" vs structural missingness if patterns are obvious.

4. **Numeric distributions.** For numeric columns: mean / std / min / quartiles / max,
   skew, and outliers via IQR (values beyond Q1−1.5·IQR or Q3+1.5·IQR). Call out
   suspicious values (negatives where impossible, placeholder codes like -999, 0 as
   missing).

5. **Categorical distributions.** Value counts for low-cardinality columns; for
   high-cardinality, report cardinality and top-k. Flag rare categories.

6. **Relationships.** Correlation matrix for numeric features (note |r| > 0.8 pairs as
   redundancy/multicollinearity risk). If a target is set: feature-vs-target
   relationships (group means for classification, correlations for regression) and
   **class balance** for classification targets.

7. **Leakage / quirks.** Flag anything that looks like target leakage, duplicated
   rows, inconsistent units, or mixed types within a column.

## Output

A concise EDA report (markdown by default; HTML if the user wants something
shareable) with these sections: **Overview → Data quality → Distributions →
Relationships → Flags & recommendations**. End with a short, prioritized
"what to fix / watch before modeling" list. Keep prose tight — tables and bullets
over paragraphs. Save plots only if the user wants visuals; otherwise describe
findings numerically.

## Notes / edge cases

- Large data: sample for plots but compute summary stats on the full set when feasible.
- Datetime columns: parse and report range / gaps rather than treating as numeric.
- Never silently impute or drop — EDA reports findings; cleaning is a separate decision.