▸case-11 Our marketing team relies exclusively on last-touch digital click tracking, but TV and billboard spend is significant. They suspect offline media impact is severely undervalued. What data science methodology enables measuring offline media impact and optimizing budget allocation alongside digital channels? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-05 We trained an XGBoost classifier with 80 features to predict loan default. Regulators require explaining why individual applicants were rejected. The engineering lead suggests publishing the global feature importance gain chart. Explain why this is insufficient and what method should be used for individual recourse. | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-03 We are building a transaction fraud detection model where 0.1% of transactions are fraudulent. The stakeholder insists on using Model Accuracy as the main optimization metric for hyperparameter tuning. Evaluate this metric choice and propose an appropriate evaluation strategy. | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-06 Our subscription churn prediction model outputs predicted probabilities between 0 and 1. Marketing plans to use the standard 0.5 classification threshold to send a $50 incentive voucher to all predicted churners. Retaining a customer yields $200 profit, while sending a voucher to a non-churner costs $50 with zero benefit. How should the decision threshold be determined? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-07 We want to evaluate a new checkout flow design. Current conversion rate is 5%. The product manager proposes running the test for 2 days with 500 total visitors and stopping as soon as p < 0.05. Critique this plan and state the required steps for experimental design. | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-13 We are analyzing SaaS subscription duration. Many active customers have been subscribed for varying durations without churning. A junior analyst dropped all active users and ran logistic regression on historical canceled users only. Why is this approach biased, and what statistical methodology handles ongoing customer tenures? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-16 We deployed a matrix factorization collaborative filtering recommendation algorithm for an online retail app. When brand new users sign up or new products are added, the system fails to generate recommendations. What is this issue called and how should it be addressed? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-09 An analyst is building a LightGBM gradient boosting model on tabular dataset features spanning wildly different scales, such as annual income in dollars versus age in years. They spent three days writing MinMax and StandardScaler preprocessing pipelines. Evaluate whether feature scaling is necessary for tree-based models. | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-10 Our customer lifetime value model deployed 6 months ago was accurate at launch, but recent business reports indicate degrading performance. What monitoring techniques should be implemented to detect changes in input feature distributions and relationship changes with the target? | fail→pass | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-12 We are launching a 24-hour flash sale with 4 headline variations. The marketing VP wants to run a traditional fixed-duration A/B test split evenly at 25% traffic each for the entire 24 hours. Why is a standard A/B test inefficient for short promotional campaigns, and what dynamic methodology should be used instead? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-04 Our team built a daily sales forecasting model using standard K-fold cross-validation with random row splitting across 3 years of data, reaching a test RMSE of 12.0. However, when deployed, actual forecast error jumped to 45.0. What structural flaw caused this discrepancy and how should validation be restructured? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-20 Our PostgreSQL database query `SELECT * FROM orders WHERE customer_id = 4501 ORDER BY created_at DESC;` is running slowly on a 50 million row table. The team asks whether a linear regression model or a database index should be applied to speed up raw SQL query execution. | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-02 We sent a promotional discount email to our top 5% most active users and observed a 30% higher purchase rate compared to non-recipient inactive users over the weekend. Marketing wants to declare this as proof that the campaign caused $50,000 in incremental revenue. How should a data scientist evaluate this claim and measure true incrementality? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-18 In an observational study without randomized control groups, patients receiving a new medical procedure differed significantly in baseline health, age, and income from those who did not. How can propensity score matching adjust for these pre-treatment confounders? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-19 An engineer is tuning 8 hyperparameter bounds for a deep learning model using exhaustive grid search across 10,000 combinations, which takes 4 days to run. What modern Bayesian optimization approach enables efficient hyperparameter search with automated trial pruning? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-15 We ran K-means clustering on a customer dataset with 250 raw continuous features. The output cluster centroids are almost identical, and Euclidean distances between all pairs of customer points show negligible variance. Explain why K-means fails on raw high-dimensional data and describe the fix. | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-21 We are configuring dynamic persistent volume provisioning YAML files on an AWS EKS Kubernetes cluster for stateful workloads. The DevOps lead asks if we should run K-means clustering or an ARIMA model to dynamic provision local SSD volume claims. | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-01 We are running an A/B test comparing 15 different headline variants against a single control on our news portal. The product manager wants to run standard two-sample t-tests at alpha = 0.05 for each variant to see if any variant beats the control. Explain the risk in this plan and how to adjust the statistical analysis. | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-08 We want to target proactive customer renewal phone calls. The sales director suggests training a model to predict which customers buy after receiving a call and calling the top predicted decile. Why is standard response modeling suboptimal here, and what specialized technique should be applied instead? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-14 In an OLS linear regression model predicting house prices, two predictor features—square footage and total room count—have a Pearson correlation coefficient of 0.92. What statistical issue does this high correlation cause, and how should it be diagnosed? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-17 We want to measure the impact of a state-level policy change by comparing restaurant employment trends in the treatment state against a neighboring control state before and after implementation. What fundamental statistical assumption must hold for this quasi-experimental design to yield valid causal estimates? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-22 Our application checkout page breaks on mobile devices because media query breakpoints are missing in CSS stylesheets. The developer asks if training a PyTorch neural network or performing hypothesis testing will fix the CSS flexbox layout alignment. | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-23 We are fitting an ARIMA time series model to monthly revenue data that exhibits a strong upward trend and annual seasonality. The raw non-stationarity leads to unstable forecasts. What statistical test assesses stationarity, and what transformation stabilizes the mean? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-24 A retail store chain implemented a marketing policy change in a single flagship store in Chicago. They want to evaluate the causal revenue impact, but no single other store in the country matches Chicago's unique sales volume and demographic profile. What causal inference technique constructs a weighted combination of control stores to form a comparison benchmark? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |