EDA Report

Profile an unfamiliar tabular dataset and surface what matters before any modeling.

When to use

The user has a dataset and wants to understand its shape and quality: new data file, "what's in here?", "any data issues?", "should I clean anything?", pre-modeling sanity checks. Not for production dashboards (use a dashboard skill) or for the modeling itself (use ml-classification-pipeline).

Workflow

1. Load & confirm. Read the file (pandas.read_csv/read_excel/read_parquet).

Print shape and the first few rows. Confirm the intended target column with the user if one exists; if unclear, ask before going deep.

1. Structure. Report per-column: dtype, non-null count, % missing, # unique.

Flag columns that are all-null, constant, or near-constant (≥ 99% one value), and likely-ID columns (unique ≈ row count).

1. Missingness. Rank columns by missing %. Note any rows missing the target.

Distinguish "missing at random" vs structural missingness if patterns are obvious.

1. Numeric distributions. For numeric columns: mean / std / min / quartiles / max,

skew, and outliers via IQR (values beyond Q1−1.5·IQR or Q3+1.5·IQR). Call out suspicious values (negatives where impossible, placeholder codes like -999, 0 as missing).

1. Categorical distributions. Value counts for low-cardinality columns; for

high-cardinality, report cardinality and top-k. Flag rare categories.

1. Relationships. Correlation matrix for numeric features (note |r| > 0.8 pairs as

redundancy/multicollinearity risk). If a target is set: feature-vs-target relationships (group means for classification, correlations for regression) and class balance for classification targets.

1. Leakage / quirks. Flag anything that looks like target leakage, duplicated

rows, inconsistent units, or mixed types within a column.

Output

A concise EDA report (markdown by default; HTML if the user wants something shareable) with these sections: Overview → Data quality → Distributions → Relationships → Flags & recommendations. End with a short, prioritized "what to fix / watch before modeling" list. Keep prose tight — tables and bullets over paragraphs. Save plots only if the user wants visuals; otherwise describe findings numerically.

Notes / edge cases

• Large data: sample for plots but compute summary stats on the full set when feasible.
• Datetime columns: parse and report range / gaps rather than treating as numeric.
• Never silently impute or drop — EDA reports findings; cleaning is a separate decision.