Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →yogsoth-ai/headroom-estimation
Estimate theoretical/practical ceiling vs current SOTA gap
Passed 19 of 22 test cases with the skill, and 7 of 22 without it.
and 16 more in the full report
14 fixed5 passed either way1 still failing2 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 14 test cases the skill fixed. Same model, same prompt, with and without the skill.
Quantify the ceiling for Recognizing Textual Entailment on RTE benchmark using accuracy percentage. SOTA scores 93.2%. Human baseline accuracy is 93.6%. Semantic ambiguity limits theoretical ceiling to 97.5%. Return JSON estimation.
| Checked | Without skill | With skill |
|---|---|---|
| Assigns saturating to saturation status because SOTA accuracy 93.2% is within 0.4% of human accuracy 93.6%. | Missed | Passed |
| Computes vs_human headroom as 0.4 percentage points. | Missed | Passed |
The instructions your model reads when the skill loads.
Quantify the remaining improvement potential for a task by estimating various performance ceilings and computing the gap between current SOTA and those ceilings. Distinguishes between theoretical limits (information-theoretic), practical limits (current paradigm), and human performance baselines.
| Field | Type | Description |
|---|---|---|
| task_name | string | The target task |
| current_sota | object | {method, score, metric, dataset, date} |
| human_performance | object | {score, conditions, source} or null |
| theoretical_bounds | object | {bound_type, value, derivation} or null |
{
"task": "string",
"dataset": "string",
"metric": "string",
"ceilings": {
"theoretical": {
"value": null,
"type": "information_theoretic|bayes_optimal|combinatorial",
"derivation": "string",
"confidence": "high|medium|low|speculative"
},
"human": {
"value": null,
"conditions": "string",
"source": "string",
"is_expert": true,
"confidence": "high|medium|low"
},
"practical": {
"value": null,
"assumptions": "string",
test cases passed, vs 7 / 22 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.