Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →rampstackco/experimentation-platform-orchestrator
A platform decision framework for experimentation. When to use Statsig vs PostHog vs GrowthBook vs Optimizely vs Amplitude vs Eppo vs Kameleoon. How to migrate between them. How to coordinate when multi-platform is genuinely warranted. The decisions that compound for years and the ones you can defer.
Passed 17 of 22 test cases with the skill, and 9 of 22 without it.
and 11 more in the full report
9 fixed8 passed either way4 still failing1 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 9 test cases the skill fixed. Same model, same prompt, with and without the skill.
We are conducting a strategic evaluation across the 11 platform considerations for experimentation tool selection. What are the three valid structured output decisions that this framework produces at the conclusion of an evaluation?
| Checked | Without skill | With skill |
|---|---|---|
| Identifies a single primary recommendation as a valid framework output | Missed | Passed |
| Identifies a short list of two options as a valid framework output | Missed | Passed |
| Identifies deferral as a valid framework output when preconditions are unmet | Missed | Passed |
The instructions your model reads when the skill loads.
A senior product and engineering leader's playbook for making the experimentation platform decision and recovering from making it wrong.
Picking an experimentation platform is one of those decisions that looks easy at the start and compounds for years afterward. The wrong choice costs you in lost experiments (because the team avoids the painful workflow), in cost (because the wrong pricing model penalizes your usage shape), in vendor lock-in (because migration is real engineering work, not a config change), and in cultural drift (because the platform's defaults shape what your team thinks experimentation is).
This skill is the discipline that makes the decision well the first time and the migration plan when you didn't.
When to use this skill: choosing a platform from scratch, evaluating whether to switch, deciding whether to consolidate from multi-platform to single, or planning a migration that has already been approved.
This skill spans platform selection, multi-platform decisions, migration planning, and governance setup. It does not cover experiment design (use experiment-design), result interpretation (use experimentation-analytics), or feature flag operations (use feature-flagging). Pair this skill with the relevant integrations microsite when you need platform-specific MCP details.
The audience is a PM, engineering leader, or data lead who is making the decision or recovering from a previous one. The voice is decisive. There is no "it depends, evaluate them all yourself." The decision space has real shape, and a senior advisor can map your situation to a defensible answer in an afternoon.
Every platform evaluation walks the same seven questions. Answer them honestly first, then read the per-platform profiles, then consult the decision matrix. The order matters: data architecture and statistical rigor are foundational; the rest are layered on top.
Data architecture. Where does experiment data live? Three patterns. Vendor-native (Statsig, Optimizely) keeps the data in the vendor's storage. Product-suite (PostHog, Amplitude) combines analytics and experiments behind one event pipeline. Warehouse-native (GrowthBook, Eppo) runs SQL on your existing data warehouse. The pattern dictates security review depth, residency, statistical depth, and cost shape.
Statistical rigor. Does the platform implement CUPED, sequential testing, the delta method for ratio metrics, and multiple testing corrections? Cheap to verify in a sales call: ask "what variance estimator do you use for ratio metrics?" and "do you support always-valid p-values?" Modern platforms (Statsig, Eppo, parts of PostHog) have these. Older or homegrown platforms often do not.
MCP availability. All seven platforms covered here have a first-party or hosted MCP except Eppo (as of May 2026). MCP availability matters more for agentic workflows where AI agents create and read experiments end to end. It matters less for traditional human-driven experimentation. Worth weighting if your team is AI-forward.
Feature flag integration. Do experiments and feature flags live in the same platform? Statsig, Optimizely, GrowthBook, and PostHog all unify them. Eppo is experiment-only. Kameleoon is creative-personalization-focused. If you also need feature flag operations as production infrastructure, check feature-flagging for the operational discipline; the platform choice has to support both surfaces or you accept a second tool.
Analytics depth. Can you see funnels, retention, and cohorts in the same surface as experiment results? PostHog and Amplitude are strongest here (analytics-first products). Statsig has a strong analytics overlay. Optimizely and GrowthBook are experiment-first, with analytics as a supplementary feature.
Governance and audit. Who can change targeting in production, who can ship experiments, who can read sensitive metrics? Enterprise tiers (Optimizely, LaunchDarkly Federal) handle this with maturity. Open-source platforms (GrowthBook, PostHog self-hosted) require self-built governance. For regulated industries (healthcare, finance, public sector), this question is the deciding factor.
Cost shape. Vendor-native scales with events. Warehouse-native scales with seats and warehouse compute. Product-suite scales with combined event volume across all features. Match the pricing shape to your actual usage shape. A high-traffic startup pays vendor-native pricing differently from a lower-traffic enterprise; pick the shape that is friendly to your trajectory, not just your current month.
test cases passed, vs 9 / 22 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.