Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →agentscope-ai/prompt-regression
Use when the user has changed a prompt (system prompt, RAG template, agent instruction, etc.) and wants to know whether the candidate is better or worse than the baseline. Also use when the user mentions prompt A/B testing, prompt comparison, prompt optimization validation, "did my prompt change help," or prompt regression testing. Outputs per-dimension win rates with statistical significance using OpenJudge PairwiseAnalyzer.
Passed 20 of 22 test cases with the skill, and 14 of 22 without it.
and 9 more in the full report
8 fixed12 passed either way0 still failing2 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 8 test cases the skill fixed. Same model, same prompt, with and without the skill.
After running position-debiased evaluations on 40 queries, the candidate prompt's win rate bootstrap 95% confidence interval for answer relevance is [0.52, 0.66]. What verdict should be assigned?
| Checked | Without skill | With skill |
|---|---|---|
| Assigns the verdict BETTER for the candidate prompt on answer relevance. | Missed | Passed |
The instructions your model reads when the skill loads.
Compare two prompts head-to-head and determine, with statistical rigor, whether the candidate is better, worse, or tied on each evaluation dimension.
You MUST create a task for each item and complete them in order:
Don't hand-write the win-rate + bootstrap math (the swap-aggregation and CI are easy to get
wrong). Run the bundled, tested script (scripts/pairwise.py, standard library only, no
OpenJudge dependency):
python scripts/pairwise.py --comparisons comparisons.jsonl --candidate candidate --baseline baseline
Each comparison row: {"id","model_a","model_b","score","dimension"?} where score >= 0.5
test cases passed, vs 14 / 22 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.