▸case-01 We are preparing an A/B test for our checkout page redesign. Based on our preliminary metrics, we expect a modest uplift in conversion rate. Could you run a power analysis to calculate the necessary sample size per group so we ensure our experiment is statistically sound? | fail→fail | 18,520 | 30,967 | +67% | 1 | 1 | 0% | 2,673 | 5,701 | +113% | 0 | 0 | — |
▸case-02 I'm setting up a bench experiment comparing two cell culture treatments. We have historical variance data and a target effect size in mind. Please determine the required number of replicates per condition needed to achieve sufficient statistical power. | fail→fail | 14,062 | 24,706 | +76% | 1 | 1 | 0% | 1,662 | 3,908 | +135% | 0 | 0 | — |
▸case-03 We want to evaluate a new model prompt strategy against our baseline performance. Can you perform a sample size estimation calculation based on our anticipated effect size and baseline variance, providing the exact number of evaluation runs required? | fail→fail | 15,700 | 30,273 | +93% | 1 | 1 | 0% | 2,071 | 5,080 | +145% | 0 | 0 | — |
▸case-04 We are designing a phase II clinical trial comparing a new anti-hypertensive drug against standard care. Our baseline mean blood pressure reduction is 12 mmHg with a standard deviation of 8 mmHg, and we want 80% power at alpha = 0.05 to detect a 3 mmHg difference. Please calculate the exact number of patients needed per arm so we can finalize our protocol. | fail→fail | 21,678 | 25,712 | +19% | 1 | 1 | 0% | 2,855 | 4,456 | +56% | 0 | 0 | — |
▸case-05 Our semiconductor fabrication facility is introducing a new wafer cleaning step. Historical defect rate is 2.5%, and we want to detect a drop to 1.8% with 90% power and alpha = 0.01. Please calculate the exact number of wafers we must sample during validation. | fail→fail | 44,073 | 41,248 | -6% | 1 | 1 | 0% | 3,506 | 8,424 | +140% | 0 | 0 | — |
▸case-06 We are benchmarking API latency for a microservices architecture under peak load. Baseline mean latency is 140ms with variance of 400ms^2. We want to detect a 10ms performance regression with 95% power at significance level 0.05. Calculate the exact number of request iterations needed for the test suite. | fail→fail | 20,704 | 31,086 | +50% | 1 | 1 | 0% | 3,350 | 5,963 | +78% | 0 | 0 | — |
▸case-07 We are evaluating a compressed automatic speech recognition model against our uncompressed baseline. Expected word error rate (WER) baseline is 8.0%, standard deviation across audio files is 3.2%, and target effect size is a 0.5% WER reduction. Please calculate the sample size of audio clips needed for 80% power at alpha = 0.05. | fail→fail | 39,093 | 32,956 | -16% | 1 | 1 | 0% | 1,922 | 6,316 | +229% | 0 | 0 | — |
▸case-08 Our e-commerce portal is testing dynamic pricing on high-margin goods. Current average order value (AOV) is $85 with a standard deviation of $30. We want to detect a $5 increase in AOV with 80% statistical power at alpha = 0.05. Compute the required sample size per treatment group. | fail→fail | 18,920 | 22,623 | +20% | 1 | 1 | 0% | 2,776 | 3,694 | +33% | 0 | 0 | — |
▸case-09 In an agricultural field trial comparing bio-fertilizer formulations, expected yield baseline is 6.2 tons/hectare with variance of 0.64 tons^2. We seek to detect a 0.4 ton/hectare improvement with 85% power at alpha = 0.05. Determine the required number of field plot replicates per condition. | fail→fail | 18,754 | 36,398 | +94% | 1 | 1 | 0% | 2,921 | 6,680 | +129% | 0 | 0 | — |
▸case-10 A hardware engineering team is testing thermal cycle durability for a new solder alloy. Historical mean cycles to failure is 1,200 with standard deviation 180. We need 90% power at alpha = 0.05 to detect a 50-cycle difference. Calculate the required sample size of PCB assemblies. | fail→fail | 19,046 | 32,246 | +69% | 1 | 1 | 0% | 3,023 | 6,276 | +108% | 0 | 0 | — |
▸case-16 We are designing an RNA-seq experiment to detect differential gene expression between wild-type and knockout mouse models. Assuming a log-fold change threshold of 1.2 and a biological coefficient of variation of 0.35, determine the number of biological replicates required per group for 80% power at FDR = 0.05. | fail→fail | 30,208 | 45,578 | +51% | 1 | 1 | 0% | 5,054 | 8,433 | +67% | 0 | 0 | — |
▸case-11 We are launching an A/B test for a search ranking algorithm update. Baseline Normalized Discounted Cumulative Gain (NDCG@10) is 0.72 with variance 0.0144. We aim to detect a 0.02 improvement in NDCG@10 with 80% power at alpha = 0.05. Provide the target sample size of query sessions per group. | fail→fail | 15,274 | 24,512 | +60% | 1 | 1 | 0% | 2,119 | 4,214 | +99% | 0 | 0 | — |
▸case-12 Our streaming video platform is evaluating a collaborative filtering recommendation engine. Baseline recommendation click-through rate is 3.2%, and we want to detect an increase to 3.6% with 90% statistical power and alpha = 0.05. Calculate the total user session sample size needed. | fail→fail | 22,615 | 28,442 | +26% | 1 | 1 | 0% | 3,698 | 5,468 | +48% | 0 | 0 | — |
▸case-13 A marketing team is planning a subject line split test. Historical open rate is 18%, and we want to detect a 2 percentage point improvement (to 20%) with 80% power at alpha = 0.05. Determine the required email list sample size per variation. | fail→fail | 21,270 | 26,736 | +26% | 1 | 1 | 0% | 3,275 | 5,075 | +55% | 0 | 0 | — |
▸case-14 We are surveying enterprise clients to measure Net Promoter Score (NPS). Estimated population variance of NPS responses is 1,200, and we want a margin of error within +/- 3 points at a 95% confidence level. Estimate the target sample size of completed survey responses. | fail→fail | 13,699 | 26,077 | +90% | 1 | 1 | 0% | 1,717 | 2,267 | +32% | 0 | 0 | — |
▸case-15 An IoT hardware startup is calibrating industrial temperature sensors. Expected sensor measurement noise standard deviation is 0.45 degrees C. We need to detect a bias of 0.15 degrees C with 95% power at alpha = 0.01. Calculate the required number of measurement repetitions per sensor. | fail→fail | 15,086 | 25,470 | +69% | 1 | 1 | 0% | 2,181 | 4,238 | +94% | 0 | 0 | — |
▸case-17 Our digital ad agency is running a multivariable creative test on a social media platform. Current baseline click-through rate is 1.15%, and we want to detect a relative uplift of 10% with 80% power at alpha = 0.05. Run a power analysis to calculate the required ad impression sample size per variant. | fail→fail | 23,458 | 35,557 | +52% | 1 | 1 | 0% | 3,753 | 6,648 | +77% | 0 | 0 | — |
▸case-18 A game studio is rebalancing a competitive multiplayer character class. Baseline win rate is 48.0%, and we want to verify if the rebalance brings it to 50.0% with 85% statistical power at alpha = 0.05. Calculate the necessary match sample size to achieve this. | fail→fail | 45,549 | 36,431 | -20% | 1 | 1 | 0% | 2,990 | 7,203 | +141% | 0 | 0 | — |
▸case-19 An EV battery laboratory is testing a novel electrolyte additive to reduce capacity degradation. Expected baseline capacity retention after 500 cycles is 88% with standard deviation 3.5%. We want to detect a 1.5% improvement in retention with 90% power at alpha = 0.05. Calculate the required sample size of battery cells. | fail→fail | 20,890 | 28,820 | +38% | 1 | 1 | 0% | 3,259 | 5,422 | +66% | 0 | 0 | — |
▸case-20 We completed our checkout conversion A/B test with 10,000 users per arm. Group A had 520 conversions and Group B had 580 conversions. Please perform a two-proportion Z-test and compute the resulting p-value for our final report. | pass→pass | 17,180 | 35,653 | +108% | 1 | 1 | 0% | 2,772 | 6,988 | +152% | 0 | 0 | — |
▸case-21 Our laboratory finished testing 30 battery batches, recording an average lifespan of 420 hours with a sample standard deviation of 15 hours. Calculate the 95% confidence interval for the mean battery lifespan. | pass→pass | 15,980 | 14,148 | -11% | 1 | 1 | 0% | 2,188 | 2,034 | -7% | 0 | 0 | — |
▸case-22 We are designing a user engagement trial for our reading app and need to decide whether to track mean daily active minutes or 90th percentile session duration as our primary metric. What statistical tradeoffs should guide this choice? | pass→pass | 23,791 | 31,479 | +32% | 1 | 1 | 0% | 3,083 | 4,673 | +52% | 0 | 0 | — |