Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →yogsoth-ai/discrepancy-identification
Compare same-method scores across sources, flag significant deviations
Passed 21 of 22 test cases with the skill, and 9 of 22 without it.
and 10 more in the full report
12 fixed9 passed either way1 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 12 test cases the skill fixed. Same model, same prompt, with and without the skill.
In an audit of Phi-3-Mini, two discrepancies were found: Dataset A had delta 5.2, and Dataset B had delta 12.8. Generate the flagged_methods JSON entry for Phi-3-Mini.
| Checked | Without skill | With skill |
|---|---|---|
| The field 'max_delta' for Phi-3-Mini in flagged_methods equals 12.8. | Missed | Passed |
The instructions your model reads when the skill loads.
Detect statistically significant discrepancies between scores reported for the same method across different sources. Identifies potential score inflation, implementation bugs, evaluation protocol differences, and unreliable baselines.
| Field | Type | Description |
|---|---|---|
| score_pairs | object[] | Array of {source_a, source_b, method, dataset, metric, score_a, score_b, conditions_a, conditions_b} |
{
"comparisons": [
{
"method": "string",
"dataset": "string",
"metric": "string",
"score_a": 0.0,
"source_a": "string",
"score_b": 0.0,
"source_b": "string",
"absolute_delta": 0.0,
"relative_delta_pct": 0.0,
"is_significant": true,
"likely_cause": "string",
"confidence": "high|medium|low"
}
],
"flagged_methods": [
{
"method": "string",
"num_discrepancies": 0,
"max_delta": 0.0,
"reliability_assessment": "string"
}
test cases passed, vs 9 / 22 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.