Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →This skill should be used when the user asks to "verify a fix", "reproduce failure", "diagnose issue", "check BEFORE/AFTER state", "VF task", "reality check", "check test quality", "mock-only tests", or needs guidance on verifying fixes by reproducing failures before and after implementation, or detecting mock-heavy test anti-patterns.
.claude/skills/tzachbon-reality-verification/SKILL.md| Test case | Without → With | Effect | Δ tokens | Δ turns |
|---|---|---|---|---|
| case-03 | ✗→✓ | ▲ Improved | 44% | 0% |
| case-07 | ✗→✓ | ▲ Improved | 1% | 0% |
| case-09 | ✗→✓ | ▲ Improved | -19% | 0% |
| case-10 | ✗→✓ | ▲ Improved | -19% | 0% |
| case-11 | ✗→✓ | ▲ Improved | -19% | 0% |
For fix goals: reproduce the failure BEFORE work, verify resolution AFTER.
Classify user goals to determine if diagnosis is needed. See references/goal-detection-patterns.md for detailed patterns.
Quick reference:
| Goal Keywords | Reproduction Command | |---------------|---------------------| | CI, pipeline | gh run view --log-failed | | test, tests | project test command | | type, typescript | pnpm check-types or tsc --noEmit | | lint | pnpm lint | | build | pnpm build | | E2E, UI | Playwright MCP browser tools | | API, endpoint | WebFetch tool |
For E2E/deployment verification, use MCP tools (Playwright MCP browser tools for UI, WebFetch tool for APIs).
Document in .progress.md under ## Reality Check (BEFORE):
markdown## Reality Check (BEFORE) **Goal type**: Fix **Reproduction command**: `pnpm test` **Failure observed**: Yes **Output**:
FAIL src/auth.test.ts Expected: 200 Received: 401
**Timestamp**: 2026-01-16T10:30:00ZDocument in .progress.md under ## Reality Check (AFTER):
markdown## Reality Check (AFTER) **Command**: `pnpm test` **Result**: PASS **Output**:
PASS src/auth.test.ts All tests passed
**Comparison**: BEFORE failed with 401, AFTER passes
**Verified**: Issue resolvedAdd as task 4.3 (after PR creation) for fix-type specs:
markdown- [ ] 4.3 VF: Verify original issue resolved - **Do**: 1. Read BEFORE state from .progress.md 2. Re-run reproduction command: `<command>` 3. Compare output with BEFORE state 4. Document AFTER state in .progress.md - **Verify**: `grep -q "Verified: Issue resolved" ./specs/<name>/.progress.md` - **Done when**: AFTER shows issue resolved, documented in .progress.md - **Commit**: `chore(<name>): verify fix resolves original issue`
When verifying test-related fixes, check for mock-only test anti-patterns. See references/mock-quality-checks.md for detailed patterns.
Quick reference red flags:
| Without | With | |---------|------| | "Fix CI" spec completes but CI still red | CI verified green before merge | | Tests "fixed" but original failure unknown | Before/after comparison proves fix | | Silent regressions | Explicit failure reproduction | | Manual verification required | Automated verification in workflow | | Tests pass but only test mocks | Tests verify real behavior, not mock behavior | | False sense of security from green tests | Confidence that tests catch real bugs |
| Case | Status | Duration (ms) | Turns | Tokens | Tool calls | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Without | With | Δ | Without | With | Δ | Without | With | Δ | Without | With | Δ | ||
case-03 | fail→pass | 8,885 | 6,561 | -26% | 1 | 1 | 0% | 1,328 | 1,913 | +44% | 0 | 0 | — |
case-01 | fail→fail | 4,781 | 4,609 | -4% | 1 | 1 | 0% | 182 | 1,136 | +524% | 0 | 0 | — |
case-02 | fail→fail | 4,491 | 5,288 | +18% | 1 | 1 | 0% | 164 | 1,002 | +511% | 0 | 0 | — |
case-04 | pass→fail | 7,212 | 24,928 | +246% | 1 | 1 | 0% | 1,248 | 1,076 | -14% | 0 | 0 | — |
case-05 | pass→fail | 6,812 | 4,353 | -36% | 1 | 1 | 0% | 1,246 | 1,033 | -17% | 0 | 0 | — |
case-06 | pass→fail | 12,491 | 7,270 | -42% | 1 | 1 | 0% | 2,126 | 984 | -54% | 0 | 0 | — |
case-07 | fail→pass | 9,432 | 3,127 | -67% | 1 | 1 | 0% | 1,275 | 1,288 | +1% | 0 | 0 | — |
case-08 | pass→pass | 7,822 | 3,852 | -51% | 1 | 1 | 0% | 1,199 | 1,449 | +21% | 0 | 0 | — |
case-09 | fail→pass | 9,099 | 2,635 | -71% | 1 | 1 | 0% | 1,443 | 1,172 | -19% | 0 | 0 | — |
case-10 | fail→pass | 9,104 | 3,119 | -66% | 1 | 1 | 0% | 1,313 | 1,066 | -19% | 0 | 0 | — |
case-11 | fail→pass | 8,592 | 3,649 | -58% | 1 | 1 | 0% | 1,467 | 1,183 | -19% | 0 | 0 | — |
case-12 | fail→pass | 7,040 | 4,671 | -34% | 1 | 1 | 0% | 1,105 | 1,219 | +10% | 0 | 0 | — |
case-13 | fail→fail | 6,019 | 4,241 | -30% | 1 | 1 | 0% | 905 | 1,039 | +15% | 0 | 0 | — |
case-14 | fail→pass | 15,056 | 3,144 | -79% | 1 | 1 | 0% | 2,160 | 1,347 | -38% | 0 | 0 | — |
case-15 | fail→pass | 10,197 | 2,498 | -76% | 1 | 1 | 0% | 1,476 | 1,207 | -18% | 0 | 0 | — |
case-16 | pass→pass | 14,003 | 8,710 | -38% | 1 | 1 | 0% | 1,946 | 2,026 | +4% | 0 | 0 | — |
case-17 | pass→pass | 8,789 | 6,059 | -31% | 1 | 1 | 0% | 1,269 | 1,647 | +30% | 0 | 0 | — |
case-18 | fail→pass | 7,498 | 4,747 | -37% | 1 | 1 | 0% | 1,110 | 1,527 | +38% | 0 | 0 | — |
case-19 | pass→pass | 6,272 | 5,400 | -14% | 1 | 1 | 0% | 919 | 1,616 | +76% | 0 | 0 | — |
case-20 | fail→pass | 9,116 | 2,430 | -73% | 1 | 1 | 0% | 1,286 | 1,127 | -12% | 0 | 0 | — |
case-21 | pass→pass | 6,461 | 1,747 | -73% | 1 | 1 | 0% | 983 | 1,019 | +4% | 0 | 0 | — |
case-22 | pass→pass | 8,901 | 3,283 | -63% | 1 | 1 | 0% | 1,331 | 1,339 | +1% | 0 | 0 | — |
DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 22 cases were attempted, and 16 counted toward the lift figure. The other 6 produced results that are not comparable between the two arms, so they are excluded from the headline rather than averaged into it. The headline lift of +32 percentage points is the difference between those two pass rates over the 16 comparable cases. 3 cases got worse with the skill loaded, and they are included in that figure.
Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.
Other measured skills in the registry, with their headline benchmark lift.