Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →hashgraph-online/trustworthy-experiment-insights
Assess whether experiment results are credible enough to influence product decisions.
Passed 22 of 23 test cases with the skill, and 16 of 23 without it.
and 4 more in the full report
6 fixed16 passed either way1 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 6 test cases the skill fixed. Same model, same prompt, with and without the skill.
We just wrapped a 3-day checkout test showing a massive 28% increase in completed purchases across 4,000 users, but I'm worried this might be a false positive before rolling it out to 100% of traffic. Please evaluate whether this result is reliable enough to justify a launch call. Output your assessment as a Markdown document starting with '# Experiment Insight Credibility Review'. Include sections for Result Under Review, Credibility Assessment, a structured table under Evidence displaying Check, Finding, and Risk, a bulleted Follow-Up section, and a final Decision Guidance section.
| Checked | Without skill | With skill |
|---|---|---|
| Selects a Credibility Assessment rating from Trust, Trust with caveats, Replicate, Extend, Investigate, or Do not trust | Missed | Passed |
| Includes the sub-category Replication needed in the Follow-Up section | Missed | Passed |
| Includes the sub-category Variance reduction opportunity in the Follow-Up section | Missed | Passed |
The instructions your model reads when the skill loads.
Use this skill to decide whether an experiment result is believable enough to shape a product or engineering decision. It focuses on false positives, false negatives, power, replication, meta-analysis, stratified sampling, covariate adjustment, and suspicious result review.
Primary source: Next-Level A/B Testing by Leemay Nassery. Guidance is transformed and paraphrased from Chapter 6 on false positives and negatives, meta-analysis, metric sensitivity, stratified random sampling, covariate adjustments, replication, longer runs, and statistical power.
Related skills:
ab-test-results-readout for standard experiment reporting.experiment-sensitivity-optimization for improving precision before or
during experiment design.experiment-verification-monitoring for operational validity checks.| Need | Read |
|---|---|
| Insight-quality concepts | references/core/knowledge.md |
| Credibility and follow-up rules | references/core/rules.md |
| Result-review scenarios | references/core/examples.md |
| Step-by-step credibility review | workflows/review-experiment-credibility.md |
test cases passed, vs 16 / 23 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.