Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →mattpocock/diagnosing-bugs
Diagnosis loop for hard bugs and performance regressions.
Passed 12 of 17 test cases with the skill, and 7 of 17 without it.
and 9 more in the full report
8 fixed8 passed either way5 still failing2 broke
Tokens, turns and time describe resource use, not a dollar price.
This benchmark measured an earlier version of the skill.
A newer attempt on gemini-3.6-flash on 10/11/2026 did not produce a publishable comparison (not comparable). The snapshot above is the previously measured run.
One of 7 test cases the skill fixed. Same model, same prompt, with and without the skill.
Here is a stack trace showing an unhandled NullPointerException in `OrderProcessor.java` line 84. Based solely on reading this code snippet, what is your top hypothesis for the bug cause and how should we fix it immediately?
| Checked | Without skill | With skill |
|---|---|---|
| Refuses to propose a root-cause hypothesis before establishing an executable feedback loop. | Missed | Passed |
| Requires running an automated test command that reproduces the failure before proceeding. | Missed | Passed |
The instructions your model reads when the skill loads.
A discipline for hard bugs. Skip phases only when explicitly justified.
When exploring the codebase, read GLOSSARY.md (if it exists) to get a clear mental model of the relevant modules, and check ADRs in the area you're touching.
This skill has you show commands, outputs and captured artifacts. Redact every secret first: write <REDACTED> in its place. Build loops against env vars, so the credential stays in the environment rather than in what you show. Captured artifacts carry auth headers: quote only the lines that carry the signal.
If the redacted output is not enough to diagnose the bug, say so and ask the user.
This is the skill. Everything else is mechanical. If you have a tight pass/fail signal for the bug (one that goes red on this bug), you will find the cause; bisection, hypothesis-testing, and instrumentation all just consume it. If you don't have one, no amount of staring at code will save you.
Spend disproportionate effort here. Be aggressive. Be creative. Refuse to give up.
git bisect run it.scripts/hitl-loop.template.sh so the loop is still structured. Captured output feeds back to you.Build the right feedback loop, and the bug is 90% fixed.
Treat the loop as a product. Once you have a loop, tighten it:
test cases passed, vs 7 / 17 without
Published skills whose descriptions closely match this one, with their headline benchmark lift.
More skills from the same source and nearby in the registry, with their headline benchmark lift.