Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →yogsoth-ai/deliberative-calibration
Strategy for small-N complete pairwise comparison using Bradley-Terry, Thurstone, AHP, and Borda methods to produce calibrated rankings.
Passed 22 of 22 test cases with the skill, and 17 of 22 without it.
and 5 more in the full report
5 fixed17 passed either way0 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 5 test cases the skill fixed. Same model, same prompt, with and without the skill.
In deliberative calibration, what numerical threshold for rating stability indicates that iterations can stop alongside acceptable consistency?
| Checked | Without skill | With skill |
|---|---|---|
| The response states a rating stability target of at least 95 percent | Missed | Passed |
The instructions your model reads when the skill loads.
Produce a fully calibrated ranking when the candidate set is small enough (5-15 items) to allow complete or near-complete pairwise comparison. Leverages parametric models (Bradley-Terry, Thurstone) and structured weighting (AHP) to extract maximum information from each comparison.
| Resource | Allocation |
|---|---|
| Comparisons | N(N-1)/2 (complete) or ≥ N×log(N) (near-complete) |
| Iterations | 2-4 rounds (initial + consistency repair) |
| Convergence target | CR < 0.1, rating stability ≥ 95% |
candidates: [] # list of items being ranked
comparison_matrix: {} # pair → {winner, confidence, reasoning}
ratings: {} # candidate → score
method: "" # bradley-terry | thurstone | ahp | borda
iteration: 0
convergence: {stable: false, score: 0.0}
consistency: {cr: null, cycles: []}
test cases passed, vs 17 / 22 without
Published skills whose descriptions closely match this one, with their headline benchmark lift.
More skills from the same source and nearby in the registry, with their headline benchmark lift.