▸case-01 A product team compares user session durations in seconds between two independent website layouts, Variant A and Variant B (n = 50 each). The duration data is heavily right-skewed and violates normality. An engineer suggests running a standard two-sample Student's t-test. Which non-parametric hypothesis test should be conducted instead to compare the distributions? | fail→pass | 4,827 | 4,615 | -4% | 1 | 1 | 0% | 963 | 834 | -13% | 0 | 0 | — |
▸case-02 A study measures systolic blood pressure in 30 patients before and after administering an antihypertensive medication. The pairwise differences between post-treatment and baseline readings are strongly skewed. An investigator proposes using a paired Student's t-test. Which non-parametric test is appropriate for paired continuous data when the normality assumption is violated? | pass→pass | 6,366 | 3,901 | -39% | 1 | 1 | 0% | 1,139 | 660 | -42% | 0 | 0 | — |
▸case-03 An analyst evaluates customer satisfaction scores on a 1-to-5 Likert scale across four independent retail store locations. A colleague recommends performing a one-way ANOVA. Which non-parametric test should be selected to compare more than two independent groups with ordinal or non-normal data? | pass→pass | 5,768 | 3,319 | -42% | 1 | 1 | 0% | 732 | 663 | -9% | 0 | 0 | — |
▸case-04 Comparing test scores between two independent student groups where Group 1 has 120 students with variance 12.5 and Group 2 has 28 students with variance 84.2. Group sizes and sample variances are sharply unequal. In Python scientific data analysis, which option flag should be passed to scipy.stats.ttest_ind to perform Welch's t-test instead of Student's t-test? | pass→pass | 3,088 | 2,764 | -10% | 1 | 1 | 0% | 628 | 564 | -10% | 0 | 0 | — |
▸case-05 An RNA-sequencing study tests differential expression across 12,000 genes simultaneously, producing raw p-values for every gene. Applying an uncorrected p < 0.05 threshold yields 1,100 significant genes. To limit false positives among the discovered genes without excessive conservatism, which multiple testing correction approach should be applied? | pass→pass | 3,718 | 4,986 | +34% | 1 | 1 | 0% | 670 | 946 | +41% | 0 | 0 | — |
▸case-06 A clinical trial evaluates 4 primary endpoints for a new drug. The trial protocol strictly demands controlling the probability of making at least one Type I error across all tests at alpha = 0.05. Which conservative correction method adjusts individual significance thresholds by dividing alpha by the number of hypothesis tests? | pass→pass | 3,692 | 2,533 | -31% | 1 | 1 | 0% | 614 | 524 | -15% | 0 | 0 | — |
▸case-07 A medical trial summarizes patient outcomes in a 2x2 contingency table comparing treatment success between Drug A and Drug B. One cell in the contingency table has an expected cell count of 3 patients. An analyst plans to run Pearson's Chi-Square test of independence. Which test should be performed instead due to the small expected cell count? | pass→pass | 3,076 | 2,915 | -5% | 1 | 1 | 0% | 557 | 516 | -7% | 0 | 0 | — |
▸case-08 A political survey records whether 300 voters approve or disapprove of a policy both before and after watching a debate (Yes/No response for each individual). An analyst attempts to test for a change in opinion using a standard Pearson Chi-Square test of independence. Which hypothesis test is designed for paired binary categorical data? | pass→pass | 4,377 | 4,640 | +6% | 1 | 1 | 0% | 792 | 957 | +21% | 0 | 0 | — |
▸case-09 Before running parametric tests on a single sample of 40 continuous measurement observations, you need to conduct a statistical test for normality. An engineer suggests applying the two-sample Kolmogorov-Smirnov test against a fitted distribution. Which statistical test is specifically tailored and most powerful for testing normality on small to moderate continuous samples? | pass→pass | 6,973 | 5,485 | -21% | 1 | 1 | 0% | 1,221 | 995 | -19% | 0 | 0 | — |
▸case-10 You are assessing the monotonic relationship between employee job level (ordinal scale 1 to 7) and annual retention rank. An analyst suggests computing Pearson's correlation coefficient. Which correlation metric should be computed for ordinal or non-linearly monotonic data? | pass→pass | 6,650 | 5,416 | -19% | 1 | 1 | 0% | 1,144 | 1,057 | -8% | 0 | 0 | — |
▸case-11 An e-commerce team plans an A/B test on checkout conversion rates. The baseline conversion is 4.0% and the team wants to detect a 0.5% absolute change with 80% statistical power at alpha = 0.05. A developer suggests collecting 1,000 visits per variant first and calculating required power afterward. When should power analysis and sample size estimation be conducted? | pass→pass | 9,192 | 9,582 | +4% | 1 | 1 | 0% | 1,890 | 2,031 | +7% | 0 | 0 | — |
▸case-12 A one-way ANOVA across four independent teaching methods shows a statistically significant overall difference in exam scores (p = 0.001). An analyst immediately concludes Method A is superior to Method B based solely on the ANOVA result. What post-hoc pairwise comparison procedure should be executed to identify specific differing group pairs while controlling family-wise error? | pass→pass | 7,664 | 5,900 | -23% | 1 | 1 | 0% | 1,484 | 1,155 | -22% | 0 | 0 | — |
▸case-13 A website tests a redesign of its homepage hero banner. The marketing team hypothesizes that the new banner might increase or decrease user click-through rates relative to the control design. Should a one-tailed or two-tailed hypothesis test be conducted? | pass→pass | 4,673 | 4,088 | -13% | 1 | 1 | 0% | 835 | 809 | -3% | 0 | 0 | — |
▸case-14 Before conducting an ANOVA across three treatment groups, you need to verify equal variances across groups. The outcome variable exhibits noticeable skewness and potential outliers. Should you choose Bartlett's test or Levene's test to evaluate homogeneity of variance? | pass→pass | 4,701 | 3,917 | -17% | 1 | 1 | 0% | 905 | 712 | -21% | 0 | 0 | — |
▸case-15 An A/B test with 250,000 users per group yields a p-value of 0.0002 for a tiny difference in average session duration (45.0 seconds vs 45.3 seconds). The product manager assumes the low p-value implies a massive user impact. What effect size metric should be reported alongside the p-value to quantify the standardized magnitude of the difference between the two means? | pass→pass | 6,544 | 5,651 | -14% | 1 | 1 | 0% | 1,275 | 1,121 | -12% | 0 | 0 | — |
▸case-16 An ergonomics study measures task completion times for 20 workers under 4 different lighting conditions, with every worker tested under all 4 conditions. The measured completion times within workers are strongly skewed. Which non-parametric test is appropriate for comparing three or more paired/repeated measurements? | pass→pass | 3,919 | 4,548 | +16% | 1 | 1 | 0% | 680 | 843 | +24% | 0 | 0 | — |
▸case-17 An educator compares test scores of 45 students who took Diagnostic Exam 1 in September and Diagnostic Exam 2 in December. An analyst puts all September scores into Group A and all December scores into Group B and executes scipy.stats.ttest_ind. Why is this independent two-sample test choice statistically improper? | pass→pass | 7,821 | 7,403 | -5% | 1 | 1 | 0% | 1,455 | 1,482 | +2% | 0 | 0 | — |
▸case-18 An online retailer compares purchase conversion rates between two independent traffic sources: Campaign A (150 conversions out of 2,000 visitors) and Campaign B (210 conversions out of 2,000 visitors). What hypothesis test should be conducted to compare these two independent proportions? | pass→pass | 6,361 | 5,629 | -12% | 1 | 1 | 0% | 1,339 | 1,245 | -7% | 0 | 0 | — |
▸case-19 A genetics researcher expects offspring phenotypes to follow a 9:3:3:1 ratio across four categorical traits. After observing counts of 90, 35, 28, and 12 in a single sample of 165 plants, which statistical test should be performed to evaluate whether the observed categorical frequencies match the expected theoretical distribution? | pass→pass | 5,965 | 6,174 | +4% | 1 | 1 | 0% | 1,115 | 1,379 | +24% | 0 | 0 | — |
▸case-20 A dataset contains employee tenure in months for a department: [12, 14, 15, 13, 16, 14, 15, 180]. Summarize the central tendency and dispersion of this variable for an executive dashboard report. | pass→pass | 9,912 | 10,700 | +8% | 1 | 1 | 0% | 2,043 | 2,286 | +12% | 0 | 0 | — |
▸case-21 A machine learning engineer trained a logistic regression model to predict loan default on a test set of 2,000 records. You have the ground-truth binary labels and predicted probabilities. Provide a summary of how to evaluate the model's classification performance. | pass→pass | 14,872 | 12,635 | -15% | 1 | 1 | 0% | 2,824 | 2,347 | -17% | 0 | 0 | — |
▸case-22 In a tabular dataset of 5,000 customer survey responses, 2% of records have missing values in the continuous 'annual_income' column, missing completely at random. Explain the standard pre-processing steps to handle these missing entries prior to model training. | pass→pass | 12,235 | 10,806 | -12% | 1 | 1 | 0% | 2,255 | 1,991 | -12% | 0 | 0 | — |