Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →Tactic: perturb scoring weights to test the robustness of the gap ranking against weight choice
.claude/skills/yogsoth-ai-priority-sensitivity-testing/SKILL.md| Test case | Without → With | Effect | Δ tokens | Δ turns |
|---|---|---|---|---|
| case-13 | ✗→✓ | ▲ Improved | -35% | 0% |
| case-10 | ✗→✓ | ▲ Improved | -5% | 0% |
| case-11 | ✗→✓ | ▲ Improved | 22% | 0% |
| case-19 | ✗→✓ | ▲ Improved | 25% | 0% |
| case-21 | ✗→✓ | ▲ Improved | -12% | 0% |
Apply weight perturbations to an existing gap scoring matrix to test whether the final ranking is sensitive to weight choice, thereby judging the credibility of the priority decision.
The final ranking of a scoring matrix depends on the weight settings for each dimension. If a small change in weights drastically changes the ranking, the decision is unreliable; if the ranking remains stable across a reasonable weight range, the priority conclusion is more convincing.
This tactic first establishes baseline weights (AHP or equal weights), then systematically perturbs the weights (±20%), observes the ranking changes, and finally gives a stability verdict.
| SOP | Responsibility | When to call | |-----|------|---------| | ahp-weighting | Use the AHP method to derive a weight vector from dimension-importance judgments | First step, establish baseline weights | | weight-perturbation | Apply ±20% perturbations to each dimension weight and recompute the ranking | Second step, systematic perturbation | | priority-synthesis | Aggregate the ranking results across all perturbation scenarios into a stability report | Third step, synthesize conclusion |
Default (standard flow)
Simplified (S tier or fast mode)
Deep (L tier or high-risk decision)
After execution, report to the calling strategy:
<!-- BEGIN available-tables (generated) -->
Optional, no fixed order; the final leaf is always a sop.
| SOP | When to use | | --- | --- | | ahp-weighting | SOP: Use the AHP (Analytic Hierarchy Process) to determine scoring-dimension weights, outputting a weight vector | | priority-synthesis | SOP: synthesize all scoring data into a final gap priority list and attack-path suggestions | | weight-perturbation | SOP: Perturb weights to test gap-ranking stability, output a stability verdict |
<!-- END available-tables (generated) -->
| Case | Status | Duration (ms) | Turns | Tokens | Tool calls | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Without | With | Δ | Without | With | Δ | Without | With | Δ | Without | With | Δ | ||
case-14 | pass→pass | 19,823 | 8,691 | -56% | 1 | 1 | 0% | 2,431 | 1,412 | -42% | 0 | 0 | — |
case-01 | fail→fail | 38,825 | 39,019 | +0% | 1 | 1 | 0% | 8,306 | 9,112 | +10% | 0 | 0 | — |
case-02 | fail→fail | 37,479 | 34,306 | -8% | 1 | 1 | 0% | 8,297 | 9,103 | +10% | 0 | 0 | — |
case-03 | fail→fail | 38,437 | 39,738 | +3% | 1 | 1 | 0% | 8,286 | 9,092 | +10% | 0 | 0 | — |
case-04 | pass→fail | 27,686 | 24,326 | -12% | 1 | 1 | 0% | 2,336 | 5,663 | +142% | 0 | 0 | — |
case-05 | pass→pass | 23,661 | 15,722 | -34% | 1 | 1 | 0% | 4,685 | 4,343 | -7% | 0 | 0 | — |
case-06 | pass→fail | 16,020 | 29,021 | +81% | 1 | 1 | 0% | 3,048 | 5,986 | +96% | 0 | 0 | — |
case-07 | pass→pass | 13,027 | 13,155 | +1% | 1 | 1 | 0% | 2,956 | 3,568 | +21% | 0 | 0 | — |
case-08 | pass→pass | 12,726 | 14,771 | +16% | 1 | 1 | 0% | 2,612 | 2,856 | +9% | 0 | 0 | — |
case-13 | fail→pass | 17,238 | 12,523 | -27% | 1 | 1 | 0% | 2,752 | 1,797 | -35% | 0 | 0 | — |
case-09 | pass→pass | 8,223 | 11,294 | +37% | 1 | 1 | 0% | 1,397 | 1,825 | +31% | 0 | 0 | — |
case-10 | fail→pass | 20,999 | 14,539 | -31% | 1 | 1 | 0% | 2,632 | 2,498 | -5% | 0 | 0 | — |
case-11 | fail→pass | 14,652 | 11,430 | -22% | 1 | 1 | 0% | 1,563 | 1,908 | +22% | 0 | 0 | — |
case-12 | pass→pass | 14,262 | 12,467 | -13% | 1 | 1 | 0% | 1,650 | 2,383 | +44% | 0 | 0 | — |
case-15 | pass→pass | 14,533 | 4,906 | -66% | 1 | 1 | 0% | 1,602 | 1,617 | +1% | 0 | 0 | — |
case-16 | pass→pass | 16,189 | 10,441 | -36% | 1 | 1 | 0% | 2,076 | 1,464 | -29% | 0 | 0 | — |
case-17 | pass→pass | 20,758 | 8,239 | -60% | 1 | 1 | 0% | 1,442 | 1,336 | -7% | 0 | 0 | — |
case-18 | fail→fail | 14,243 | 9,508 | -33% | 1 | 1 | 0% | 1,464 | 1,549 | +6% | 0 | 0 | — |
case-19 | fail→pass | 28,947 | 8,281 | -71% | 1 | 1 | 0% | 1,119 | 1,401 | +25% | 0 | 0 | — |
case-20 | fail→fail | 16,104 | 9,348 | -42% | 1 | 1 | 0% | 1,943 | 1,455 | -25% | 0 | 0 | — |
case-21 | fail→pass | 14,266 | 11,274 | -21% | 1 | 1 | 0% | 1,858 | 1,630 | -12% | 0 | 0 | — |
case-22 | pass→pass | 20,413 | 10,205 | -50% | 1 | 1 | 0% | 1,114 | 1,625 | +46% | 0 | 0 | — |
case-23 | fail→pass | 16,842 | 5,469 | -68% | 1 | 1 | 0% | 1,805 | 1,822 | +1% | 0 | 0 | — |
DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 23 cases were attempted, and 22 counted toward the lift figure. The other 1 produced results that are not comparable between the two arms, so they are excluded from the headline rather than averaged into it. The headline lift of +17 percentage points is the difference between those two pass rates over the 22 comparable cases. 2 cases got worse with the skill loaded, and they are included in that figure.
Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.
Other measured skills in the registry, with their headline benchmark lift.