Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →mims-harvard/tooluniverse-diagnostic-test-evaluation
Diagnostic test / biomarker accuracy — sensitivity, specificity, PPV, NPV, likelihood ratios, accuracy from a 2x2 table; ROC curve, AUC, and the optimal cutoff (Youden) for a continuous biomarker; and post-test probability via Bayes.
Passed 21 of 22 test cases with the skill, and 12 of 22 without it.
and 11 more in the full report
9 fixed12 passed either way1 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 9 test cases the skill fixed. Same model, same prompt, with and without the skill.
A patient with a 15% pre-test probability of Lyme disease tests negative on an ELISA assay that has 90% sensitivity and 95% specificity. Write the exact JSON tool execution request to compute the patient's post-test probability of disease.
| Checked | Without skill | With skill |
|---|---|---|
| Uses Epidemiology_bayesian as the target command or tool. | Missed | Passed |
| Includes operation: 'bayesian', prevalence: 0.15, sensitivity: 0.90, specificity: 0.95, and test_result: 'negative'. | Missed | Passed |
The instructions your model reads when the skill loads.
Judge how well a test or biomarker discriminates disease — at a fixed cutoff (2×2) or across all cutoffs (ROC) — and turn a result into a probability of disease.
| You have… | Go to |
|---|---|
| A 2×2 table (TP/FP/TN/FN) at a fixed cutoff | Step 1 (Epidemiology_diagnostic) |
| A continuous biomarker score + true labels | Step 2 (ROC / AUC / Youden, Python) |
| A test's sens/spec + a patient's pre-test probability | Step 3 (Epidemiology_bayesian) |
tu run Epidemiology_diagnostic '{"operation":"diagnostic","tp":90,"fp":10,"tn":180,"fn":20}'
Returns sensitivity, specificity, PPV, NPV, accuracy, LR_pos, LR_neg, and the sample prevalence.
| Metric | Question it answers | Depends on prevalence? |
|---|---|---|
| Sensitivity = TP/(TP+FN) | Of those WITH disease, what fraction test positive? | No |
| Specificity = TN/(TN+FP) | Of those WITHOUT disease, what fraction test negative? | No |
| PPV = TP/(TP+FP) | If positive, what's the chance of disease? | Yes — strongly |
| NPV = TN/(TN+FN) | If negative, what's the chance of being disease-free? | Yes |
| LR+ = sens/(1−spec) | How much a positive raises the odds of disease | No |
| LR− = (1−sens)/spec | How much a negative lowers the odds | No |
The PPV/NPV trap. Sensitivity and specificity are properties of the test; PPV and NPV depend on the disease prevalence in the tested population. A test with great sens/spec has poor PPV in a low-prevalence (screening) setting. Never quote PPV/NPV from a case-control design (its 50/50 prevalence is artificial) — compute them for the real-world prevalence with
Epidemiology_bayesian(Step 3). Report sensitivity, specificity, and likelihood ratios as the prevalence-independent summary.
When the test is a continuous score, evaluate across all thresholds:
Prefer the ROC_analysis tool — one call returns structured JSON (AUC + bootstrap 95% CI, Youden-optimal cutoff with its sens/spec, optional metrics at a fixed cutoff, and the ROC curve), and works under the MCP server without a shell:
ROC_analysis(scores=[...], labels=[0,1,...]) # inline arrays
ROC_analysis(csv_path="scores.csv", cutoff=0.6) # or a CSV (cols: label, score)
test cases passed, vs 12 / 22 without
Published skills whose descriptions closely match this one, with their headline benchmark lift.
More skills from the same source and nearby in the registry, with their headline benchmark lift.