Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →romainsimon/learning-from-outcomes
Turns past decisions into calibrated judgment - scoring old predictions against what happened, assigning credit for delayed results, and separating a bad decision from a good decision that lost. Use during a retrospective, postmortem, quarterly or monthly review, when reviewing decisions taken 30+ days ago, when attributing a result to a cause, or when someone claims to have called something.
Passed 18 of 21 test cases with the skill, and 7 of 21 without it.
and 14 more in the full report
12 fixed7 passed either way3 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 11 test cases the skill fixed. Same model, same prompt, with and without the skill.
An engineering refactor was rushed without peer review or automated testing, leading to a major database outage. Identify which cell of the process vs outcome matrix this falls under.
| Checked | Without skill | With skill |
|---|---|---|
| Identifies bad process with bad outcome as the only cell that deserves blame. | Missed | Passed |
The instructions your model reads when the skill loads.
Experience does not automatically produce judgment. Without a written prediction and a scoring step, a review produces a story in which the good outcomes were foreseen and the bad ones were unforeseeable. That story feels like learning and teaches nothing.
Three things make a review compound:
- [ ] 1. Collect the predictions that came due
- [ ] 2. Score them
- [ ] 3. Separate decision quality from outcome quality
- [ ] 4. Assign credit for delayed results
- [ ] 5. Replay two old decisions that are not due
- [ ] 6. Update the priors that were wrong
Pull every prediction from the decision log whose date has passed. Record the outcome as a plain binary before discussing anything, and before anyone explains why it happened.
If there is nothing to collect, the problem is upstream: decisions are being recorded without falsifiable predictions. Fix that in framing-decisions first; nothing here works without it.
Run the calculator. Do not compute these by hand.
node scripts/calc.js calibrate predictions.json
test cases passed, vs 7 / 21 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.