Ready to supercharge your AI agents?

Install any skill in seconds. Free to start, no credit card required.

A/B Test Setup · sickn33 | DecimalAI

marketingby sickn33

A/B Test Setup

sickn33/ab-testing

Source sickn33/antigravity-awesome-skills ↗License MIT

When the user wants to plan, design, or implement an A/B test or experiment, or build a growth experimentation program. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "should I test this,"...

SkillScore91Maintained

Automatic

Product Management pack

Last verified 8/6/2026

What it changes

Passed 20 of 22 test cases with the skill, and 12 of 22 without it.

Without skill55%

With skill91%

Checks it passed that the model alone missed

Recommends 6,000 visitors per variant
Recommends 1,200 visitors per variant
Recommends 12,000 visitors per variant

and 8 more in the full report

8 fixed12 passed either way2 still failing0 broke

Pass-rate lift: +36 pp
Tokens: +109%
Agent turns: 0%
Response time: -25%

Model gemini-3.6-flashMeasured 8/6/2026v120 of 22 cases passed with skill

Tokens, turns and time describe resource use, not a dollar price.

Use this skill

Save it into your project's skills folder. Claude Code picks it up in its next session.

curl -fsSL --create-dirs -o .claude/skills/sickn33-ab-testing/SKILL.md https://app.decimal.ai/s/sickn33-ab-testing/SKILL.md

Want it pinned and updatable? pip install "decimalai>=0.15", then decimalai skills add sickn33-ab-testing, and start Claude Code with claude --plugin-dir "$(git rev-parse --show-toplevel)/.agents"

1 · One-time — install the CLI (docs ↗)

pip install "decimalai>=0.15"

2 · Get the skill — writes .agents/skills/sickn33-ab-testing/ and pins it in .decimal/skills.lock

decimalai skills add sickn33-ab-testing

No CLI? Get the same file — curl · raw file

curl -fsSL --create-dirs -o .claude/skills/sickn33-ab-testing/SKILL.md https://app.decimal.ai/s/sickn33-ab-testing/SKILL.md

Raw file: /s/sickn33-ab-testing/SKILL.md ↗ · machine-readable summary at /s/sickn33-ab-testing.json ↗

Safety checks: 3 of 3 passed

Code scan CleanIntent PassedContent Clean

See it in action

One of 8 test cases the skill fixed. Same model, same prompt, with and without the skill.

1 of 8

Prompt

For a web form with a 5% baseline conversion rate, how many users per variant are needed to detect a 50% relative lift according to standard experimentation lookup tables?

What was checked, without and with the skill
Checked	Without skill	With skill
Recommends 1,200 visitors per variant	Missed	Passed

case-05 · gemini-3.6-flash · 8/6/2026 · answers are capped and redacted

What's inside

The instructions your model reads when the skill loads.

SKILL.md356 lines3 files

A/B Test Setup

When to Use

Use this skill when you need when the user wants to plan, design, or implement an A/B test or experiment, or build a growth experimentation program. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "should I test this,"...

You are an expert in experimentation and A/B testing. Your goal is to help design tests that produce statistically valid, actionable results.

Initial Assessment

Check for product marketing context first: If .agents/product-marketing.md exists (or .claude/product-marketing.md, or the legacy product-marketing-context.md filename, in older setups), read it before asking questions. Use that context and only ask for information not already covered or specific to this task.

Before designing a test, understand:

Test Context - What are you trying to improve? What change are you considering?
Current State - Baseline conversion rate? Current traffic volume?
Constraints - Technical complexity? Timeline? Tools available?

Core Principles

1. Start with a Hypothesis

Not just "let's see what happens"
Specific prediction of outcome
Based on reasoning or data

2. Test One Thing

Single variable per test
Otherwise you don't know what worked

3. Statistical Rigor

Pre-determine sample size
Don't peek and stop early
Commit to the methodology

4. Measure What Matters

Primary metric tied to business value
Secondary metrics for context

Track record

Benchmark

20 / 22

test cases passed, vs 12 / 22 without

Adoption

Users: 0
Times used: 0

Safety and source details

SkillSafety

✓ 3 of 3 passed

Code scan15 of 15 checks clean

0 critical · 0 warning · 0 info — scanner v4 — 8/2/2026

View 15 scanner checks

live_secretremote_code_executionreverse_shelldata_exfiltrationssrf_metadataagent_snoopinganti_refusaldestructive_commandsobfuscated_unicodeunicode_homoglyphopaque_blobsuspicious_urltool_overreachtrigger_abuseprompt_injection_phrasing

IntentSafe

LLM judge · does the skill do what it claims

ContentClean

LLM judge · policy & harmful-content review

Source

Imported

sickn33 / antigravity-awesome-skills

skills/ab-testing/SKILL.md

View on GitHub ↗

License MITImported 9/11/2026Version v1Bundled 2 files

Other skills for the same job

Published skills whose descriptions closely match this one, with their headline benchmark lift.

More skills from the same source and nearby in the registry, with their headline benchmark lift.

Ready to supercharge your AI agents?

See it in action

What's inside

A/B Test Setup

When to Use

Initial Assessment

Core Principles

1. Start with a Hypothesis

2. Test One Thing

3. Statistical Rigor

4. Measure What Matters

Track record

Benchmark

Adoption

SkillSafety

Source

Other skills for the same job

Related skills