Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →borghei/ab-test-setup
Design and analyze A/B tests: sample size, test duration, and statistical significance for conversion experiments.
Passed 15 of 21 test cases with the skill, and 10 of 21 without it.
and 5 more in the full report
6 fixed10 passed either way6 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 6 test cases the skill fixed. Same model, same prompt, with and without the skill.
Write a shell command to generate an experiment plan artifact for the hypothesis 'Adding customer logo wall increases enterprise demo requests' assuming baseline rate of 3%, MDE of 20%, and daily traffic of 1500 visitors, formatted as JSON using the project's planning script.
| Checked | Without skill | With skill |
|---|---|---|
| The command must execute scripts/experiment_planner.py. | Missed | Passed |
| The command must pass -H or --hypothesis 'Adding customer logo wall increases enterprise demo requests'. | Passed | Passed |
The instructions your model reads when the skill loads.
Category: Product Team Tags: A/B testing, experiments, statistical significance, sample size, feature flags, hypothesis testing
A/B Test Setup provides the complete framework for designing experiments that produce statistically valid, actionable results. Most A/B tests fail not because the variant was wrong, but because the test was poorly designed: wrong sample size, wrong metric, or someone peeked at results and stopped early. This skill prevents those mistakes.
Before designing the experiment, confirm these inputs. If any is unknown or vague, ASK — do not assume:
Stop rule: ask only the 2-3 that most change the output. If the user says "just draft it," proceed and list your assumptions at the top of the artifact.
1. HYPOTHESIZE → 2. DESIGN → 3. CALCULATE → 4. IMPLEMENT
↑ │
│ ▼
7. ITERATE ← 6. DOCUMENT ← 5. ANALYZE ← [Run to completion]
Because [observation or data point],
we believe [specific change]
test cases passed, vs 10 / 21 without
Published skills whose descriptions closely match this one, with their headline benchmark lift.
More skills from the same source and nearby in the registry, with their headline benchmark lift.