▸case-16 Management wants us to group 100,000 retail customers based on purchasing history using K-means clustering, but they don't know how to choose the optimal number of clusters or validate cluster stability. Detail an actionable segmentation approach. | fail→fail | 20,600 | 25,408 | +23% | 1 | 1 | 0% | 3,476 | 5,267 | +52% | 0 | 0 | — |
▸case-09 When building a customer lifetime value model using scikit-learn, our junior engineer applied StandardScaler and OneHotEncoder to the entire combined dataset prior to splitting into train and test sets. Detail the correct feature transformation workflow and verification methods. | fail→fail | 21,612 | 18,549 | -14% | 1 | 1 | 0% | 3,752 | 4,032 | +7% | 0 | 0 | — |
▸case-01 We are planning an A/B test to evaluate a new recommendation algorithm on our e-commerce platform. Please provide a structured workflow for experimental design and measurement, including how to define sample sizes, validate outcomes, handle potential bias, and present the statistical results effectively to executive stakeholders. | fail→fail | 32,917 | 29,720 | -10% | 1 | 1 | 0% | 5,764 | 5,859 | +2% | 0 | 0 | — |
▸case-02 I'm working on exploratory data analysis and baseline modeling for a high-dimensional tabular dataset with missing values and class imbalance. Could you outline an actionable, step-by-step methodology to inspect the data quality, transform variables, select appropriate baseline algorithms, and verify model performance before moving to production? | fail→fail | 25,287 | 20,829 | -18% | 1 | 1 | 0% | 4,169 | 4,650 | +12% | 0 | 0 | — |
▸case-03 Our PostgreSQL production database is experiencing high disk I/O during peak hours on the order_items table. Can you write the SQL DDL commands to create B-tree and BRIN indexes on the created_at column and tune max_worker_processes and shared_buffers in postgresql.conf? | pass→pass | 15,425 | 10,736 | -30% | 1 | 1 | 0% | 2,251 | 2,416 | +7% | 0 | 0 | — |
▸case-04 We need a React web component using Tailwind CSS that renders an interactive user dropdown menu with custom avatar icons and dark mode toggle state. Please provide the React JSX code. | pass→pass | 29,876 | 23,342 | -22% | 1 | 1 | 0% | 6,892 | 6,187 | -10% | 0 | 0 | — |
▸case-05 We need a Terraform script to provision an AWS EKS Kubernetes cluster with managed node groups across three availability zones. Please write the HCL configuration file. | pass→pass | 17,347 | 16,124 | -7% | 1 | 1 | 0% | 3,813 | 3,721 | -2% | 0 | 0 | — |
▸case-06 We want to compare user engagement retention between two app onboarding flows. The data team wants to run a standard t-test on daily active minutes, even though the data is heavily right-skewed with many zero values. Outline a statistical testing workflow with non-parametric or bootstrap alternatives, and detail the verification steps. | fail→fail | 23,230 | 24,125 | +4% | 1 | 1 | 0% | 3,730 | 5,389 | +44% | 0 | 0 | — |
▸case-07 We need to forecast weekly store sales across 500 retail locations featuring strong annual seasonality and sudden trend breaks. Should we default to fitting simple linear trend regression models directly across all stores? Provide a structured methodology for time-series modeling. | fail→fail | 23,904 | 24,425 | +2% | 1 | 1 | 0% | 3,996 | 4,175 | +4% | 0 | 0 | — |
▸case-08 We are building a credit card fraud detection classifier where fraud accounts for 0.1% of transactions. A team member suggested evaluating model performance using standard accuracy score. Outline a proper machine learning evaluation and sampling strategy for this dataset. | fail→fail | 16,787 | 20,331 | +21% | 1 | 1 | 0% | 2,507 | 4,089 | +63% | 0 | 0 | — |
▸case-10 Marketing wants to target existing subscribers with a renewal promo code, but we want to avoid sending offers to 'sure things' who would renew anyway. Provide a predictive modeling approach for estimating incremental uplift and measuring treatment effects. | fail→fail | 17,349 | 31,241 | +80% | 1 | 1 | 0% | 3,151 | 5,641 | +79% | 0 | 0 | — |
▸case-11 During exploratory data analysis of a clinical trial dataset, 30% of patient blood pressure measurements are missing. Should we drop all rows with missing values or impute them with column means? Provide an actionable data quality and imputation framework. | fail→fail | 17,595 | 19,947 | +13% | 1 | 1 | 0% | 2,885 | 4,312 | +49% | 0 | 0 | — |
▸case-12 Our SaaS business wants to model customer subscription duration and hazard rates over time, but standard regression models ignore right-censored users who haven't churned yet. Detail a statistical survival analysis framework for customer tenure. | fail→fail | 25,053 | 26,307 | +5% | 1 | 1 | 0% | 5,144 | 6,069 | +18% | 0 | 0 | — |
▸case-13 We trained a Gradient Boosting Tree model on tabular customer data that achieves 99% training accuracy but drops to 62% validation accuracy. Outline an actionable hyperparameter tuning and cross-validation strategy to address over-fitting. | fail→fail | 17,156 | 20,611 | +20% | 1 | 1 | 0% | 2,936 | 3,789 | +29% | 0 | 0 | — |
▸case-14 Our deployed recommendation engine in production has seen declining click-through rates over the last three months. How should we set up continuous model performance and data drift monitoring? Outline a comprehensive operational workflow. | fail→fail | 37,714 | 28,783 | -24% | 1 | 1 | 0% | 6,718 | 5,033 | -25% | 0 | 0 | — |
▸case-15 We want to estimate the causal impact of enabling a new dark mode feature on user session duration using non-experimental observational event logs. Outline a causal inference workflow to control for confounding variables. | fail→fail | 25,343 | 35,888 | +42% | 1 | 1 | 0% | 3,564 | 5,778 | +62% | 0 | 0 | — |
▸case-17 We need to classify customer support tickets into priority categories using text data. A team member suggested using raw TF-IDF with a Logistic Regression model as a starting point. Outline a structured text analytics and model evaluation workflow. | fail→fail | 21,751 | 15,456 | -29% | 1 | 1 | 0% | 3,004 | 3,042 | +1% | 0 | 0 | — |
▸case-18 We have a high-dimensional dataset with 200 numerical features that we want to project into 2D/3D space for executive presentation and anomaly detection. Should we rely solely on PCA? Outline a visualization and dimensionality reduction strategy. | fail→fail | 20,780 | 22,550 | +9% | 1 | 1 | 0% | 3,229 | 3,773 | +17% | 0 | 0 | — |
▸case-19 Our finance team wants to allocate TV, radio, and digital ad spend using regression modeling, but channel spend has lagged carryover effects and diminishing returns. How should we structure a Marketing Mix Model (MMM)? | fail→fail | 22,395 | 22,258 | -1% | 1 | 1 | 0% | 3,556 | 3,727 | +5% | 0 | 0 | — |
▸case-20 We are building a collaborative filtering recommendation engine for new user signups who have zero historical interactions. How do we resolve this cold start problem? Outline an actionable algorithm design and evaluation methodology. | fail→fail | 25,075 | 22,191 | -12% | 1 | 1 | 0% | 4,001 | 3,950 | -1% | 0 | 0 | — |
▸case-21 Our risk management committee rejected a black-box XGBoost loan approval model because loan officers cannot explain rejection reasons to applicants. Outline a model interpretability methodology to provide actionable local and global explanations. | fail→fail | 20,366 | 30,594 | +50% | 1 | 1 | 0% | 3,479 | 4,854 | +40% | 0 | 0 | — |
▸case-22 Our data science team is experiencing inconsistency between offline training feature calculations and online real-time inference features. How should we architect feature engineering pipelines to prevent training-serving skew? | fail→fail | 21,009 | 27,364 | +30% | 1 | 1 | 0% | 2,940 | 4,310 | +47% | 0 | 0 | — |
▸case-23 Instead of a static 50/50 A/B test for headline optimization, product management wants to dynamically route traffic to higher-performing headlines while minimizing regret during the trial. Outline an experimental design using multi-armed bandits. | fail→fail | 20,875 | 22,839 | +9% | 1 | 1 | 0% | 3,469 | 4,592 | +32% | 0 | 0 | — |
▸case-24 We need to forecast inventory demand at national, regional, and store levels simultaneously, ensuring that store forecasts sum up exactly to regional and national aggregates. Outline a hierarchical time-series forecasting approach. | fail→fail | 16,184 | 27,455 | +70% | 1 | 1 | 0% | 2,965 | 5,384 | +82% | 0 | 0 | — |
▸case-25 We are monitoring sensor telemetry from industrial manufacturing equipment to detect impending machine failures before downtime occurs. Most sensor readings are normal, with rare unpredictable failure modes. Detail an anomaly detection methodology. | fail→fail | 19,192 | 28,575 | +49% | 1 | 1 | 0% | 2,956 | 4,833 | +63% | 0 | 0 | — |