Bias Audit

Bias Audit

Overview

Bias in training data produces biased models, full stop. This audit measures representation, outcome disparities, and intersectional gaps so you can fix problems before training.

When to Use

Use when: building models that make decisions about people, deploying in regulated domains, or when protected attributes (gender, race, age, etc.) are available.

Core Metrics

Representation Audit

import pandas as pd
import numpy as np

def representation_audit(df, protected_cols, population_benchmark=None):
    """Check if dataset representation matches population."""
    n = len(df)
    results = {}
    
    for col in protected_cols:
        dist = df[col].value_counts(normalize=True).to_dict()
        results[col] = {
            "distribution": dist,
            "n_groups": len(dist),
            "min_group_pct": min(dist.values()),
            "max_group_pct": max(dist.values()),
            "imbalance_ratio": max(dist.values()) / (min(dist.values()) + 1e-10),
        }
    
    # Intersectional audit
    if len(protected_cols) >= 2:
        intersectional = df.groupby(protected_cols).size() / n
        min_intersection = intersectional.min()
        results["intersectional"] = {
            "n_intersections": len(intersectional),
            "min_pct": min_intersection,