Install any skill in seconds. Free to start, no credit card required.
Get Started Free →Use before claiming CodeWhale release work is done: run the full gate sweep and list the manual QA targets.
.claude/skills/hmbown-codew-release-qa-sweep/SKILL.md| Test case | Without → With | Effect | Δ tokens | Δ turns |
|---|---|---|---|---|
| case-04 | ✗→✓ | ▲ Improved | 34% | 0% |
| case-05 | ✗→✓ | ▲ Improved | 18% | 0% |
| case-10 | ✗→✓ | ▲ Improved | 5% | 0% |
| case-11 | ✗→✓ | ▲ Improved | -11% | 0% |
| case-12 | ✗→✓ | ▲ Improved | -22% | 0% |
Run this before claiming any Codewhale release work is "done." A green automated gate sweep plus the three manual QA targets is the evidence bar. No sweep, no "done" — report exactly what was run and the result of each step.
merge-ready.
(e.g. <release-branch>), which is often local-only.
Run from the repo root, in order. Stop on the first failure and report it.
bash# 0. Confirm you are on the real release head, not a main-based assumption. git branch --show-current # expect e.g. <release-branch> git status --short # working tree should be clean # 1. Formatting + stray whitespace/conflict markers cargo fmt --all --check git diff --check # Core npm workspace tests plus the shared web gate (install web dependencies first). # check:web checks committed facts before regeneration, then docs, design tokens, # lint, TypeScript and the production build. These do not replace the Rust gates. npm test && npm run check:web # 2. Library/protocol/cli/flow/state tests, locked cargo test -p codewhale-config -p codewhale-protocol -p codewhale-cli \ -p codewhale-workflow -p codewhale-state --locked # 3. TUI test binaries, locked cargo test -p codewhale-tui --bins --locked # 4. TUI debug build, locked cargo build -p codewhale-tui --locked # 5. Release build for the shipped binaries, locked cargo build --release --locked -p codewhale-cli -p codewhale-tui # 6. Version-drift gate (workspace ↔ npm ↔ Cargo.lock ↔ changelog ↔ README) ./scripts/release/check-versions.sh # 7. Binary smoke ./target/release/codewhale --version
If you are validating a PR for landing, also test mergeability against the actual release head, never the main-based clean flag:
bashgit merge-tree $(git merge-base <release-branch> <pr-head>) <release-branch> <pr-head>
A PR that is clean against main can still conflict with the release branch.
Unit/build gates do not cover the live TUI. Exercise all three and record what you saw:
Use the built binary with a sealed local home and loopback fixtures, then exercise the relevant scenarios below in an actual terminal. Record dimensions, inputs, visible state, and side effects. Do not substitute a full-screen assertion harness for looking at and using the product.
typing, render, cancel, and the workbar stay live throughout, and that Esc cancels mid-fanout (prompt interrupt, not a wedged ~24s burst or freeze). For the Windows Terminal retest path (ref #3289), start in plan mode, add follow-up input to the plan, press Esc, switch to yolo/accept flow, trigger at least two auto/Fleet worker spawns, and keep typing/cancel/mode-switch checks live for several minutes. Attach logs if the freeze reproduces.
distinct provider/model routes. Confirm zero cross-terminal contamination and no provider+model mismatch — each terminal honors its own route.
queues a typed follow-up, the preview advertises Enter send now, and an empty Enter promotes the oldest queued follow-up. Confirm Ctrl+Enter steers typed text directly, Shift+Enter inserts a newline, and Ctrl+G/Ctrl+S only stash drafts.
Report a checklist: each command, pass/fail, and the salient output line (test counts, the --version string, check-versions.sh verdict). For manual QA, state what you actually observed per target, citing the regression ref where one applies. If a step was skipped or could not be run (e.g. no display for TUI QA), say so explicitly — do not imply coverage you do not have.
Assertions without command output are not acceptable.
git merge-tree against the real head.
route-mismatch, and steering regressions live in the runtime, not the gates.
any PR or issue without maintainer approval. A green sweep is readiness evidence, not permission.
comments, and checks.
keeps the original author, otherwise add Co-authored-by: Name <email> and a Harvested from PR #N by @handle body line (the spaced form the auto-close-at-main workflow greps for).
dry-run/advisory unless the maintainer approves enforcement.
| Case | Status | Duration (ms) | Turns | Tokens | Tool calls | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Without | With | Δ | Without | With | Δ | Without | With | Δ | Without | With | Δ | ||
case-01 | fail→fail | 18,256 | 18,360 | +1% | 1 | 1 | 0% | 337 | 1,867 | +454% | 0 | 0 | — |
case-02 | fail→fail | 15,937 | 20,140 | +26% | 1 | 1 | 0% | 240 | 1,883 | +685% | 0 | 0 | — |
case-03 | fail→fail | 26,241 | 19,822 | -24% | 1 | 1 | 0% | 1,971 | 1,880 | -5% | 0 | 0 | — |
case-04 | fail→pass | 35,675 | 10,707 | -70% | 1 | 1 | 0% | 2,360 | 3,161 | +34% | 0 | 0 | — |
case-05 | fail→pass | 21,972 | 26,443 | +20% | 1 | 1 | 0% | 1,901 | 2,245 | +18% | 0 | 0 | — |
case-06 | pass→pass | 12,684 | 4,373 | -66% | 1 | 1 | 0% | 1,615 | 1,910 | +18% | 0 | 0 | — |
case-07 | fail→fail | 25,310 | 4,872 | -81% | 1 | 1 | 0% | 1,512 | 1,950 | +29% | 0 | 0 | — |
case-08 | pass→pass | 12,094 | 5,353 | -56% | 1 | 1 | 0% | 1,716 | 2,034 | +19% | 0 | 0 | — |
case-09 | pass→pass | 9,013 | 8,713 | -3% | 1 | 1 | 0% | 1,350 | 2,406 | +78% | 0 | 0 | — |
case-10 | fail→pass | 25,676 | 7,050 | -73% | 1 | 1 | 0% | 2,177 | 2,289 | +5% | 0 | 0 | — |
case-11 | fail→pass | 35,002 | 8,336 | -76% | 1 | 1 | 0% | 2,741 | 2,447 | -11% | 0 | 0 | — |
case-12 | fail→pass | 21,917 | 5,637 | -74% | 1 | 1 | 0% | 2,651 | 2,073 | -22% | 0 | 0 | — |
case-13 | fail→pass | 19,977 | 20,948 | +5% | 1 | 1 | 0% | 2,787 | 2,556 | -8% | 0 | 0 | — |
case-14 | fail→pass | 32,095 | 6,826 | -79% | 1 | 1 | 0% | 1,330 | 2,416 | +82% | 0 | 0 | — |
case-15 | fail→fail | 10,954 | 5,590 | -49% | 1 | 1 | 0% | 1,379 | 1,828 | +33% | 0 | 0 | — |
case-16 | fail→pass | 11,985 | 5,819 | -51% | 1 | 1 | 0% | 1,852 | 1,839 | -1% | 0 | 0 | — |
case-17 | fail→pass | 14,972 | 2,483 | -83% | 1 | 1 | 0% | 2,421 | 1,629 | -33% | 0 | 0 | — |
case-18 | pass→pass | 16,771 | 8,430 | -50% | 1 | 1 | 0% | 2,125 | 2,749 | +29% | 0 | 0 | — |
case-19 | fail→pass | 57,538 | 13,593 | -76% | 1 | 1 | 0% | 2,127 | 2,354 | +11% | 0 | 0 | — |
case-20 | fail→fail | 47,450 | 20,354 | -57% | 1 | 1 | 0% | 2,181 | 3,210 | +47% | 0 | 0 | — |
case-21 | fail→fail | 61,366 | 16,731 | -73% | 1 | 1 | 0% | 3,186 | 3,998 | +25% | 0 | 0 | — |
case-22 | fail→pass | 14,474 | 10,223 | -29% | 1 | 1 | 0% | 2,123 | 2,449 | +15% | 0 | 0 | — |
DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 22 cases were attempted, and 19 counted toward the lift figure. The other 3 produced results that are not comparable between the two arms, so they are excluded from the headline rather than averaged into it. The headline lift of +50 percentage points is the difference between those two pass rates over the 19 comparable cases.
Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.
| Model | Method | Date | Lift |
|---|---|---|---|
| gemini-3.6-flash | verified | 9/17/2026 | +63% |
| gemini-3.6-flash | verified | 9/8/2026 | +50% |
| gemini-3.6-flash | verified | 8/24/2026 | +60% |
| gemini-3.6-flash | verified | 8/7/2026 | +50% |
Other measured skills in the registry, with their headline benchmark lift.