Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →hashgraph-online/experiment-verification-monitoring
Verify and monitor running experiments for operational quality.
Passed 13 of 22 test cases with the skill, and 9 of 22 without it.
and 7 more in the full report
10 fixed1 passed either way9 still failing2 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 10 test cases the skill fixed. Same model, same prompt, with and without the skill.
We are preparing to launch a high-stakes redesign of our mobile checkout flow across iOS and Android apps, and we need to ensure the test infrastructure doesn't fail silently. Please build a comprehensive experiment verification and monitoring plan for this release. We need a summary of quality risks, a table of prelaunch verification checks (including method, owner, and criteria), details for active monitoring like canaries and alerts, platform health maintenance, and explicit escalation rules for when to pause or investigate.
| Checked | Without skill | With skill |
|---|---|---|
| The response starts with the main header `# Experiment Verification And Monitoring Plan`. | Missed | Passed |
| The prelaunch checks matrix includes checks for assignment and exposure. | Passed | Passed |
| The escalation rules section includes conditions for pausing and restarting tests. | Passed | Passed |
The instructions your model reads when the skill loads.
Use this skill to prevent misconfigured or unhealthy experiments from producing bad evidence. It focuses on verification before launch, canaries, A/A tests, leakage and interference checks, active monitoring, and quality roadmap metrics.
Primary source: Next-Level A/B Testing by Leemay Nassery. Guidance is transformed and paraphrased from Chapter 5 on experiment effectiveness, prelaunch verification, QA tooling, canaries, A/A tests, spillover effects, and active monitoring.
Related skills:
ab-test-design-brief for planning an experiment before verification.experimentation-throughput-strategy for monitoring overlap conflicts.trustworthy-experiment-insights for statistical credibility after results.| Need | Read |
|---|---|
| Verification concepts | references/core/knowledge.md |
| QA, canary, A/A, and monitoring rules | references/core/rules.md |
| Failure scenarios and examples | references/core/examples.md |
| Step-by-step quality roadmap | workflows/create-experiment-quality-roadmap.md |
test cases passed, vs 9 / 22 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.