Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →romainsimon/tracking-beliefs
Separates signal from noise in metrics you cannot observe directly, updating a belief from evidence instead of reacting to the latest reading - Bayesian updates over competing explanations, and a filter that says whether this week's move is real.
Passed 20 of 22 test cases with the skill, and 16 of 22 without it.
and 2 more in the full report
4 fixed16 passed either way2 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 4 test cases the skill fixed. Same model, same prompt, with and without the skill.
Organic search sessions dropped 9% week-over-week right after our major frontend release. I need a rigorous diagnosis before we start revert projects. Please evaluate this using a metric tracking and Bayesian updating framework: detail the gap between true traffic state and observed analytics, state if the latest datapoint exceeds typical process and measurement variance, lay out competing explanations with priors before evaluating evidence, show the updated posterior distribution after factoring in our server error logs, and state what action to take based on how concentrated our belief is.
| Checked | Without skill | With skill |
|---|---|---|
| The response states competing explanations and historical priors before evaluating server error log evidence. | Passed | Passed |
| The response evaluates whether the 9% drop exceeds process and observation noise variance. | Passed | Passed |
| The response advises against immediate full revert if posterior belief remains diffuse. | Missed | Passed |
The instructions your model reads when the skill loads.
You never observe the state of the business. You observe noisy readings of it. Revenue lags settlement, refunds and disputes arrive late, annual plans lump, analytics drops traffic to blockers and bots, and attribution is approximate everywhere.
Two operations, and almost every metric argument needs one of them:
- [ ] 1. Say what you are actually measuring, and what the reading is a proxy for
- [ ] 2. Decide whether the latest move is real
- [ ] 3. If it is real, list the explanations before looking for evidence
- [ ] 4. Update on the evidence
- [ ] 5. Act only when the belief is concentrated enough to matter
Write both lines:
The gap between them is the noise model, and naming it usually resolves the argument before any maths. A weekly revenue reading with settlement lag and refunds has several percent of noise on it before anything real has happened.
node scripts/calc.js track metric.json
Input shape: examples/track.json. The filter separates drift in the true level from
test cases passed, vs 16 / 22 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.