Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →minhnv0807/19-ab-test-setup-global
Design valid A/B tests for global marketing — hypothesis formulation, sample size calculation, statistical significance, multi-arm testing, primary vs secondary metrics. Tools: Optimizely, VWO, Google Optimize (sunset 2023, alternatives), built-in platform tests (Meta, Google). Trigger: 'A/B test', 'split test', 'multivariate test', 'experiment design', 'statistical significance', 'sample size calculator'.
Passed 19 of 22 test cases with the skill, and 15 of 22 without it.
and 2 more in the full report
4 fixed15 passed either way3 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 4 test cases the skill fixed. Same model, same prompt, with and without the skill.
We want to test changing our checkout button color from gray to bright green to increase purchase completions. Write down the hypothesis for this experiment.
| Checked | Without skill | With skill |
|---|---|---|
| Structures the hypothesis using the format 'If we [change X], [metric Y] will increase by [Z%] because [reason]'. | Missed | Passed |
| Includes a specific numerical percentage for target lift in the hypothesis. | Missed | Passed |
The instructions your model reads when the skill loads.
Run experiments that produce decisions, not noise. Most "A/B tests" in marketing are underpowered, peeked-at, and badly hypothesized — meaning the team learns nothing and ships the louder variant.
A valid A/B test answers one question: "Did this change cause a real improvement, or am I seeing noise?"
To answer it credibly you need four things:
If any one of these is missing, you don't have an A/B test — you have a coin flip with extra steps.
Common newbie mistake: running a test for 3 days, seeing variant B 40% higher, declaring victory, and shipping. Three days is too short to absorb day-of-week effects, and small samples produce wild swings. Variant B may revert (or reverse) by day 14.
Read .agents/product-marketing-context.md if it exists. Audience size, average traffic, and current conversion rate determine whether a test is even feasible.
Ask up to 4 questions:
test cases passed, vs 15 / 22 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.