▸case-01 We are evaluating two studies on drug X for hypertension. Study A is a prospective cohort study with 100,000 subjects showing a 15% risk reduction. Study B is a randomized controlled trial with 500 subjects showing a 12% risk reduction. A team member argues Study A provides stronger causal evidence due to its massive sample size. Evaluate the causal strength of both studies and provide explicit strength ratings for each. | pass→pass | 13,194 | 13,495 | +2% | 1 | 1 | 0% | 2,300 | 2,479 | +8% | 0 | 0 | — |
▸case-02 A clinical trial with 200,000 participants finds a new dietary supplement reduces head colds with p < 0.0001, but the actual reduction is 0.02 days per year. The sponsor claims this p-value proves strong causal impact. Assess this claim and another observational report showing a 3-day reduction in 50 patients, providing explicit strength ratings for both. | pass→pass | 13,504 | 13,231 | -2% | 1 | 1 | 0% | 2,278 | 1,669 | -27% | 0 | 0 | — |
▸case-03 Compare the evidence for Treatment Y: Study A is a single multi-center trial showing positive results (p=0.04). Studies B, C, and D are three independent pre-registered replications by different labs showing consistent moderate effect sizes. Evaluate the strength of evidence provided by the single study versus the replicated body of work, providing explicit strength ratings for both. | pass→pass | 11,754 | 8,127 | -31% | 1 | 1 | 0% | 2,070 | 1,701 | -18% | 0 | 0 | — |
▸case-04 Evaluate two sources of evidence for coffee consumption reducing stroke risk: Source 1 is an observational study that did not adjust for smoking or socioeconomic status. Source 2 is a Mendelian randomization study accounting for genetic variants linked to caffeine metabolism. Rate the strength of evidence for both sources explicitly. | pass→pass | 11,661 | 8,079 | -31% | 1 | 1 | 0% | 1,994 | 1,546 | -22% | 0 | 0 | — |
▸case-05 We have two epidemiological findings of equal statistical association with disease Z. Finding A has no known biological mechanism. Finding B has a fully mapped cellular mechanism verified in vitro. Assess the causal evidence strength for both findings with explicit ratings. | pass→pass | 13,943 | 8,709 | -38% | 1 | 1 | 0% | 2,351 | 1,725 | -27% | 0 | 0 | — |
▸case-06 A chief executive insists a product feature causes higher retention based on 3 customer interviews (anecdotes). A data scientist presents a theoretical user-flow simulation showing predicted retention gains. Assess the evidence strength of these two claims with explicit ratings. | pass→pass | 12,328 | 8,691 | -30% | 1 | 1 | 0% | 1,925 | 1,691 | -12% | 0 | 0 | — |
▸case-07 An executive bases policy on personal stories from two regional managers. A researcher presents an observational cross-sectional survey of 1,200 employees. Assess both items as causal evidence with explicit strength ratings. | pass→pass | 13,220 | 10,400 | -21% | 1 | 1 | 0% | 2,182 | 1,923 | -12% | 0 | 0 | — |
▸case-08 Evaluate two evidence packages for a software latency fix: Package 1 is a kinetic queueing simulation theory predicting 20ms savings. Package 2 is an A/B benchmark experiment on 5% of production traffic showing 18ms savings. Provide explicit strength ratings for both. | pass→pass | 11,472 | 8,953 | -22% | 1 | 1 | 0% | 1,957 | 1,815 | -7% | 0 | 0 | — |
▸case-09 Study 1 reports a significant p=0.01 effect in a post-hoc subgroup analysis of 20 variables. Study 2 attempted a direct pre-registered replication on 1,000 subjects and found no effect (p=0.60). Assess the evidence for the original claim across both studies with explicit ratings. | pass→pass | 10,151 | 7,692 | -24% | 1 | 1 | 0% | 1,808 | 1,646 | -9% | 0 | 0 | — |
▸case-10 Assessing claims that red meat increases heart disease risk: Study A is a food frequency questionnaire survey with high healthy-user bias potential. Study B is a randomized crossover trial measuring downstream lipid biomarkers. Provide explicit strength ratings for both evidence sources. | pass→pass | 13,331 | 8,666 | -35% | 1 | 1 | 0% | 2,075 | 1,653 | -20% | 0 | 0 | — |
▸case-11 Review the causal claim that remote work boosts software developer output based on an internal company survey from 2021. Provide a structured evidence review. | pass→pass | 17,219 | 9,764 | -43% | 1 | 1 | 0% | 2,579 | 1,705 | -34% | 0 | 0 | — |
▸case-12 Claim: Fluoride in water reduces cavities. Source 1 compares cavity rates across whole country averages without individual consumption data. Source 2 is a randomized community trial tracking individual dental records over 5 years. Assess the strength of both evidence sources with explicit ratings. | pass→pass | 8,877 | 5,682 | -36% | 1 | 1 | 0% | 1,426 | 1,240 | -13% | 0 | 0 | — |
▸case-13 Evaluating cancer drug efficacy: Trial 1 shows reduction in tumor shrinkage on MRI (surrogate marker) in 100 patients. Trial 2 shows overall 5-year survival improvement in 500 randomized patients. Evaluate both pieces of evidence with explicit strength ratings. | pass→pass | 12,235 | 7,495 | -39% | 1 | 1 | 0% | 2,122 | 1,540 | -27% | 0 | 0 | — |
▸case-14 Evaluate evidence on mobile phone radiation and brain tumors: Study A is a retrospective case-control study relying on 10-year recall of phone usage. Study B is a prospective cohort study tracking network billing logs. Provide explicit strength ratings for both. | pass→pass | 11,733 | 8,663 | -26% | 1 | 1 | 0% | 1,959 | 1,611 | -18% | 0 | 0 | — |
▸case-15 A city claims a new speed limit reduced accidents because crashes dropped 30% the year after implementation. However, new red-light cameras and severe weather also occurred that year. Compare this before-after study with a stepped-wedge cluster randomized trial in a neighboring state. Provide explicit ratings for both. | pass→pass | 16,096 | 9,247 | -43% | 1 | 1 | 0% | 2,592 | 1,781 | -31% | 0 | 0 | — |
▸case-16 Evaluate two observational studies on alcohol and liver disease: Study 1 shows a clear dose-response curve (higher consumption linearly predicts higher incidence). Study 2 shows erratic, non-monotonic risks across intake groups. Provide explicit strength ratings for both evidence items. | pass→pass | 11,931 | 7,590 | -36% | 1 | 1 | 0% | 1,885 | 1,584 | -16% | 0 | 0 | — |
▸case-17 A world-renowned authority asserts that exercise prevents dementia based on clinical experience. A meta-analysis of prospective cohort studies covering 50,000 participants reports hazard ratios for physical activity. Assess both sources of evidence with explicit ratings. | pass→pass | 12,331 | 9,094 | -26% | 1 | 1 | 0% | 2,047 | 1,830 | -11% | 0 | 0 | — |
▸case-18 Evaluating a new supplement: Dataset A consists of 5 published positive small studies (funnel plot shows missing negative small studies). Dataset B consists of a large, pre-registered negative trial. Rate the strength of causal evidence from Dataset A vs Dataset B explicitly. | pass→pass | 11,064 | 8,200 | -26% | 1 | 1 | 0% | 1,753 | 1,661 | -5% | 0 | 0 | — |
▸case-19 Claim: Molecule X cures Alzheimer's disease. Evidence 1: A paper showing Molecule X binds to amyloid plaques in a test tube. Evidence 2: A phase II randomized double-blind trial showing cognitive score changes in humans. Evaluate both evidence sources with explicit ratings. | pass→pass | 11,756 | 9,652 | -18% | 1 | 1 | 0% | 1,872 | 1,825 | -3% | 0 | 0 | — |
▸case-20 Calculate the required sample size for a two-tailed randomized controlled trial with 80% power, alpha = 0.05, expecting a baseline outcome rate of 20% and a treatment effect reduction to 15%. Show the mathematical formula and step-by-step arithmetic. | pass→fail | 17,708 | 23,335 | +32% | 1 | 1 | 0% | 3,964 | 5,347 | +35% | 0 | 0 | — |
▸case-21 Write a PubMed Boolean search query string to retrieve literature on SGLT2 inhibitors and renal outcomes in diabetic patients for a systematic review. | pass→fail | 12,788 | 23,013 | +80% | 1 | 1 | 0% | 2,538 | 4,508 | +78% | 0 | 0 | — |
▸case-22 Given Study 1 (OR = 1.5, standard error = 0.2) and Study 2 (OR = 1.8, standard error = 0.3), calculate the inverse-variance weighted pooled odds ratio under a fixed-effects model. | pass→fail | 12,451 | 17,703 | +42% | 1 | 1 | 0% | 2,512 | 4,092 | +63% | 0 | 0 | — |