Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →jaechang-hits/scientific-critical-thinking
Evaluating scientific evidence and claims. Covers study design hierarchy (RCT to expert opinion), effect sizes (OR, RR, NNT, Cohen's d), confounding, p-value vs clinical significance, GRADE quality assessment, reproducibility, and bias types (selection, information, confounding, reporting).
Passed 23 of 23 test cases with the skill, and 22 of 23 without it.
1 fixed22 passed either way0 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
A test case the skill fixed. Same model, same prompt, with and without the skill.
A trial for a novel neurodegenerative disease drug shows a 40% reduction in brain amyloid plaque volume on PET scans (p < 0.001). Based on this, the authors state that the drug halts functional cognitive decline in Alzheimer's patients. However, clinical cognitive battery scores showed no statistically significant differences between groups. Evaluate the validity of equating this imaging marker with patient benefit.
| Checked | Without skill | With skill |
|---|---|---|
| Explains that surrogate endpoint changes do not guarantee clinical endpoint improvements. | Passed | Passed |
| Recommends treating the finding as preliminary until clinical cognitive outcomes are demonstrated. | Missed | Passed |
The instructions your model reads when the skill loads.
Scientific critical thinking is the disciplined application of logical and methodological standards to evaluate whether a study's design, analysis, and interpretation support its conclusions. It is the skill that separates a researcher who synthesizes evidence from one who accumulates it. This guide covers the hierarchy of evidence, the mechanics of common biases, effect size interpretation, the p-value controversy, GRADE evidence grading, and common logical fallacies in the interpretation of scientific literature.
Study designs vary in their ability to support causal inference. The hierarchy below applies to questions about the effect of an intervention or exposure on an outcome:
Systematic reviews and meta-analyses of RCTs (highest causal certainty)
↓
Randomized Controlled Trials (RCTs)
↓
Non-randomized controlled trials / cluster-randomized trials
↓
Prospective cohort studies (follow exposure → outcome forward in time)
↓
Retrospective cohort studies
↓
Case-control studies (compare exposed vs. unexposed given outcome)
↓
Cross-sectional studies (measure exposure and outcome simultaneously)
↓
Case series and case reports
↓
Expert opinion, mechanistic reasoning, animal models (lowest causal certainty)
Important exceptions: For questions about rare outcomes, case-control designs are often more efficient than cohort studies. For questions about diagnostic accuracy, randomized designs are usually inappropriate — cross-sectional or cohort designs with verified reference standards are preferred. For harm questions, RCTs are often infeasible (ethical constraints), making large cohort studies the best available evidence.
Effect measures quantify the relationship between an exposure/intervention and an outcome. Confusing them is a leading source of misinterpretation.
| Measure | Formula | Use case | Key interpretation |
|---|
test cases passed, vs 22 / 23 without
Published skills whose descriptions closely match this one, with their headline benchmark lift.
More skills from the same source and nearby in the registry, with their headline benchmark lift.