apply-paper — research finding → shipped change

Closes the loop on docs/papers.md: that file captures papers; this skill applies them. A paper only earns its keep when its mechanism changes how a skill or agent behaves. The output is a change (or a tight change plan), not a summary.

Step 1 — Distill the claim (no hype)

In 3-4 sentences: what does the paper actually show, and by what mechanism? Separate the finding (often robust) from the framing (often oversold). If you can't name a mechanism, you can't apply it — stop and say so.

Step 2 — Map it to us

Name the concrete target: which skill / agent / instruction-layer / flow does the mechanism touch? Cite the file. If nothing in our setup is affected, say "no application" and log that in docs/papers.md — that's a valid, honest outcome.

Step 3 — Design the change

The smallest edit that captures the mechanism: a new check, a new mode/step, a tightened contract, a deleted footgun. Prefer scoped edits over broad rewrites (HarnessFix's own lesson). Write it as a diff plan: file → what changes → why it captures the mechanism.

Step 4 — Prove it (the part that makes it real)

State the change's done-condition as a check, then show before/after:

• Behavioral change → a test query / scenario that fails before and passes after.
• New rubric/check → run it on a skill that should fail it and one that should pass.
• "Reliability" claim → measure across runs, not once (per Beyond pass@1).

If you can't show a before/after, you've written a summary, not an application.

Step 5 — Adversarial pass

Red-team with adversarial-review: does the change actually capture the paper's mechanism, or just cite it? Could it be satisfied without the intended effect (cargo-culting the paper)?

Step 6 — Record it

Update the paper's entry in docs/papers.md with → applied: <what changed> (<file>), so the list shows captured-vs-shipped at a glance. File/refresh a Linear JAS issue if it's multi-step.

Output

PAPER: <title + url>
CLAIM: <mechanism in one sentence>
TARGET: <skill/agent/layer + file>
CHANGE: <the diff plan or the landed edit>
PROOF: <before/after evidence, or the check that now passes>

Worked example (shipped)

Offscript (auto-auditing instruction adherence) → added the Adherence audit to meta/linting-and-scoring (behavioral pass: generate adversarial test queries per stated instruction, flag the ones an agent could violate). See that skill's "Adherence audit" section.

Errors

| Issue | Fix | |---|---| | Can't name a mechanism, only a vibe | The paper isn't applicable yet — record "no application" in papers.md; don't force a change. | | Change just cites the paper without its effect | Cargo-culting. Redo Step 4 — if there's no before/after, it didn't apply the mechanism. | | Paper conflicts with an existing skill's approach | Surface both; use adversarial-review to decide, and record the decision (don't silently overwrite). | | Mechanism needs a real model call to test (e.g. behavioral) and no key/tool | Use openrouter for a cheap model, or mark the proof step blocked and don't claim it works. |