This Is Fine

Detect compatibility drift cheaply. Do not turn a canary into a full benchmark.

The coffee is optional. The pinned regression cases are not.

Prepare

1. Read the skill and its last known-good canary record.
2. Pin the old and new model or harness versions when both remain available.

Otherwise pin the new environment and compare against stored expected observations.

1. Select five to ten cheap cases from

references/canary-contract.md.

1. Use fixtures or sandboxes. Do not let a canary mutate production or publish

externally.

Run

Keep task inputs, permissions, tools, reasoning, and timeout stable. Use fresh contexts and capture raw evidence for:

• trigger activation and near-miss non-activation;
• preferred tool route and fallback;
• approval or destructive-action boundary;
• output schema or completion contract;
• one version-sensitive fact or command;
• one known historical failure.

Run the last known-good environment first when practical, then the candidate environment. A missing old runtime is a limitation, not permission to invent a comparison.

Classify

• Trigger drift: activation becomes too broad, too narrow, or ambiguous.
• Capability drift: the same route no longer achieves the outcome.
• Tool drift: a tool, argument, path, or fallback changed.
• Safety drift: approval, validation, or write boundaries weaken.
• Knowledge drift: pinned instructions conflict with current primary

sources.

• Harness drift: discovery, progressive disclosure, or resource loading

changes.

Act

1. Fail the canary on any safety regression or broken core outcome.
2. Propose the smallest correction at the source of drift.
3. Do not rewrite unrelated guidance or silently update expected output to make

the run green.

1. Use $trust-me-bro when available before claiming the new version improves

efficacy. Otherwise compare the same pinned task in isolated baseline and candidate contexts with one declared verifier. A canary alone never proves uplift.

Return

Report the environment delta, case-by-case observation, raw evidence, drift class, severity, smallest proposed fix, and whether release should pass, warn, or block.