Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →romainsimon/valuing-information
Prices research, tests, surveys, dashboards and experiments before running them, by computing how much the result would raise expected utility - which is zero whenever no outcome would change the decision.
Passed 19 of 22 test cases with the skill, and 19 of 22 without it.
1 fixed18 passed either way2 still failing1 broke
Tokens, turns and time describe resource use, not a dollar price.
A test case the skill fixed. Same model, same prompt, with and without the skill.
At Enterprise ERP Corp, enterprise account churn occurs at a rate of 0.2%. The team wants to survey churning enterprise clients to measure dissatisfaction factors. They propose taking a simple random sample across their entire 100,000 user database. Evaluate this sampling design and state the correct methodology.
| Checked | Without skill | With skill |
|---|---|---|
| Recommend deliberately oversampling the rare churning account segment. | Passed | Passed |
| State that sample weights equal to true-to-proposal probability ratios must be applied during analysis. | Missed | Passed |
The instructions your model reads when the skill loads.
The value of an observation is the increase in expected utility from making it:
VOI(O) = sum over outcomes P(o) * max_a EU(a | o) - max_a EU(a)
It is never negative. It is exactly zero whenever the best action is the same whatever you learn. That zero is the entire point of this skill: most proposed research is decoration, and this is the calculation that says so before the money is spent rather than after.
- [ ] 1. Name the decision the information would serve
- [ ] 2. State the actions and what you would do today with no new data
- [ ] 3. State the hidden variable, its states, and the prior
- [ ] 4. Score every action-state pair
- [ ] 5. Compute the perfect-information ceiling before pricing any real study
- [ ] 6. State honestly how discriminating each candidate observation is
- [ ] 7. Run the calculator and read the "changes decision" column
- [ ] 8. If it is worth it, size the study
Put the tables in the answer. The decision table and the likelihood table are not scratch work: write them out. A reader who cannot see the numbers cannot challenge them, and the numbers are the only part of this that is falsifiable.
Write the sentence: "We are trying to decide whether to ___, and we would act differently if we learned ___."
If you cannot complete it, there is no decision, so there is no value of information. That does not make the research illegitimate - curiosity and background understanding are real - but it does mean it should compete for time as a discretionary activity, not be waved through as diligence.
test cases passed, vs 19 / 22 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.