Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →brycewang-stanford/modeling-strategy-guide
Strategic statistical modeling, experimentation, and causal inference
Passed 15 of 22 test cases with the skill, and 13 of 22 without it.
3 fixed12 passed either way6 still failing1 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 3 test cases the skill fixed. Same model, same prompt, with and without the skill.
I am starting a predictive research project on predicting academic dropouts using student demographic and transcript data. My colleague wants to immediately train a deep neural network ensemble. What modeling strategy framework should govern our model progression, and what must be established first before building complex models?
| Checked | Without skill | With skill |
|---|---|---|
| Recommends starting with the simplest model and establishing a baseline prior to building complex models | Passed | Passed |
| Outlines a complexity ladder progressing through descriptive statistics, linear models, regularized regression, tree ensembles, and deep learning | Missed | Passed |
The instructions your model reads when the skill loads.
A skill for strategic statistical modeling applied to academic research. Covers advanced modeling decisions, experimental design, causal inference, feature engineering, and the critical thinking required to move from data to defensible conclusions.
Senior data scientists distinguish themselves not by knowing more algorithms but by asking better questions, designing cleaner experiments, and being honest about what the data can and cannot tell them. This skill translates that professional discipline into a research context, helping academics apply modern data science practices to their empirical work. It covers the strategic decisions that matter most: when to use simple models versus complex ones, how to establish causality rather than mere correlation, and how to communicate uncertainty honestly.
The skill is particularly useful for researchers working with observational data who need causal inference techniques, those designing randomized experiments who need proper power calculations and analysis plans, and anyone building predictive models who needs to avoid common overfitting and leakage pitfalls.
Decision Framework:
1. Start with the simplest model that could answer your question
2. Add complexity only when diagnostics reveal inadequacy
3. Prefer interpretable models unless prediction accuracy is the sole goal
4. Always have a baseline (mean, majority class, last observation)
Model Complexity Ladder:
Level 1: Descriptive statistics, cross-tabulations
Level 2: Linear/logistic regression
Level 3: Regularized regression (Lasso, Ridge, Elastic Net)
Level 4: Tree ensembles (Random Forest, Gradient Boosting)
Level 5: Deep learning (only with sufficient data and clear justification)
import pandas as pd
import numpy as np
def engineer_features(df: pd.DataFrame, config: dict) -> pd.DataFrame:
"""
Apply systematic feature engineering based on domain knowledge.
config example:
test cases passed, vs 13 / 22 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.