Night Market Research Methodology

Night Market Research Methodology

The discipline that turns a hunch into an accepted result in this repo. An "accepted result" is a change that survived the evidence bar and landed through change control as a rule, a skill module, a config gate, or an ADR. Everything else is either a local working note or a documented retirement. This skill covers the full path: score the idea, experiment behind a default-off flag, meet the evidence bar, land the durable artifact, or retire the idea on the record.

The evidence bar

A claim graduates from hunch to result only when it passes all four tests.

1. One mechanism explains all observations, including negatives. If the hypothesis explains the three failing cases but not why the fourth case passed, it is incomplete. Keep digging until a single mechanism accounts for everything you saw.

2. Predict numbers before running. Write down the expected measurement first, then measure. In-repo anchor: the forced-eval harness labels expected activations in prototypes/forced-eval/activation_cases.json before any run, then compares baseline against treatment with a McNemar paired test (a significance test for paired binary outcomes).

3. Survive assigned adversarial refutation. Assign a reviewer or agent whose explicit job is to break the claim. Use Skill(attune:war-room) for hard-to-reverse decisions and Skill(imbue:rigorous-reasoning) to counter agreement bias. A claim nobody tried to break is unproven.

4. Never let the generator judge itself. The agent that produced the work must not be its sole verifier. See plugins/imbue/skills/proof-of-work/modules/independent-verification.md. Prefer executable checks over an LLM judge, and prove the check can fail before trusting it (Guards 2 and 3 in plugins/imbue/skills/proof-of-work/modules/verifier-integrity.md).