Install any skill in seconds. Free to start, no credit card required.
Get Started Free →Teaches agents to reply to PR review comment threads after fixing issues, making resolutions traceable
.claude/skills/github-pr-review-response/SKILL.md| Test case | Without → With | Effect | Δ tokens | Δ turns |
|---|---|---|---|---|
| case-07 | ✗→✓ | ▲ Improved | 160% | 0% |
| case-08 | ✗→✓ | ▲ Improved | -17% | 0% |
| case-13 | ✗→✓ | ▲ Improved | 38% | 0% |
| case-04 | ✓→✗ | ▼ Worse | 276% | 0% |
| case-06 | ✓→✓ | = Same ✓ | 67% | 0% |
When an agent fixes code in response to PR review comments (from Copilot, a human reviewer, or any GitHub reviewer), the fix alone is not enough. The reviewer needs to see — on the PR thread itself — which comments were addressed and how. Without replies, comments stay visually unresolved, reviewers must re-read the entire diff to verify fixes, and there's no traceable link between feedback and resolution.
Use this skill whenever:
✅ THIS SKILL PRODUCES:
❌ THIS SKILL DOES NOT PRODUCE:
Using MCP tools (preferred when available):
github-mcp-server-pull_request_read
method: "get_review_comments"
owner: "{owner}"
repo: "{repo}"
pullNumber: {pr_number}This returns review threads with metadata: isResolved, isOutdated, isCollapsed, and their associated comments. Each comment has an id you'll need for replies.
Using gh CLI (fallback):
bashgh api repos/{owner}/{repo}/pulls/{pr_number}/comments --paginate
Each comment object contains id, body, path, line, and in_reply_to_id. Top-level comments have no in_reply_to_id — those are the ones you reply to.
Make the actual code changes. This is your normal domain work — the skill doesn't prescribe how to fix, only how to communicate the fix.
Track what you changed. For each review comment, note:
id (top-level, not a reply)After fixing and committing, reply to each review comment thread individually.
REST API call (via gh CLI):
bashgh api repos/{owner}/{repo}/pulls/{pr_number}/comments/{comment_id}/replies \ -f body="Fixed in {sha_short} — {brief description of what was changed}"
Important: {comment_id} must be the ID of the top-level comment in the thread. You cannot reply to a reply — only to the original review comment.
Example replies:
bash# Specific and traceable gh api repos/bradygaster/squad/pulls/42/comments/18234/replies \ -f body="Fixed in a1b2c3d — switched to path.dirname(squadDirInfo.path) for worktree consistency" # When applying a suggested code change gh api repos/bradygaster/squad/pulls/42/comments/18235/replies \ -f body="Applied suggestion — updated error message to include the file path for debuggability" # When pushing back on a suggestion gh api repos/bradygaster/squad/pulls/42/comments/18236/replies \ -f body="Considered but not applied — this path needs to stay absolute because worktree resolution depends on it. See detectSquadDir() in detect-squad-dir.ts."
Thread resolution is only available via the GitHub GraphQL API. Use this when your fix fully addresses the comment and no further discussion is needed.
First, get the thread IDs (they're different from comment IDs):
bashgh api graphql -f query=' query { repository(owner: "{owner}", name: "{repo}") { pullRequest(number: {pr_number}) { reviewThreads(first: 100) { nodes { id isResolved comments(first: 1) { nodes { body databaseId } } } } } } } '
Match thread IDs to comment IDs using databaseId, then resolve:
bashgh api graphql -f query=' mutation { resolveReviewThread(input: {threadId: "{thread_node_id}"}) { thread { id isResolved } } } '
When to resolve vs. leave open:
Rule of thumb: Agent-to-agent threads (e.g., Copilot review → agent fix) can be resolved by the fixer. Human reviewer threads should be left for the human to resolve.
Commit messages should reference the PR context:
fix: address review feedback on PR #{pr_number}
- Switched to path.dirname() for worktree path resolution (comment #18234)
- Updated error message to include file path (comment #18235)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>For single-comment fixes, a shorter format works:
fix: use path.dirname() for worktree consistency (PR #{pr_number} review)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>gh apigh api .../repliesReview comment (id: 55123): > squadDir could be undefined here. Consider adding a null check.
Agent workflow:
get_review_commentssrc/cli/core/detect-squad-dir.tsfix: add null check for squadDir (PR #99 review)bash gh api repos/bradygaster/squad/pulls/99/comments/55123/replies \ -f body="Fixed in f4e5d6c — added early return when squadDir is undefined, matching the pattern in loadConfig()"
Comments:
detect-squad-dir.ts:42detect-squad-dir.ts:58output.ts:15Agent handles each individually:
bash# Fix all three, commit git add packages/squad-cli/src/cli/core/detect-squad-dir.ts packages/squad-cli/src/cli/core/output.ts git commit -m "fix: address 3 review comments on PR #99 - Added null check for squadDir (comment #55123) - Switched to path.join() for cross-platform paths (comment #55124) - Reduced log verbosity to debug level (comment #55125)" git push # Reply to each thread individually gh api repos/bradygaster/squad/pulls/99/comments/55123/replies \ -f body="Fixed — added early return when squadDir is undefined" gh api repos/bradygaster/squad/pulls/99/comments/55124/replies \ -f body="Fixed — switched to path.join(squadDir, 'config.json') for cross-platform consistency" gh api repos/bradygaster/squad/pulls/99/comments/55125/replies \ -f body="Fixed — changed from console.log to debug() so it only shows with --verbose flag"
Copilot sometimes provides suggestion blocks with exact code to apply:
Review comment (id: 55130):
`Consider using optional chaining:
const name = config?.agent?.name ?? 'default';
Reply format when applying:
bashgh api repos/bradygaster/squad/pulls/99/comments/55130/replies \ -f body="Applied suggestion — using optional chaining with nullish coalescing"
Reply format when not applying:
bashgh api repos/bradygaster/squad/pulls/99/comments/55130/replies \ -f body="Not applied — config is guaranteed non-null at this point (validated on line 12). Optional chaining would mask errors."
Not every review comment should be accepted. When a suggestion is incorrect or doesn't apply:
bashgh api repos/bradygaster/squad/pulls/99/comments/55140/replies \ -f body="Considered but not applied — this file is in the zero-dependency bootstrap set (see copilot-instructions.md § Protected Files). Adding path.join() would require importing from the SDK, which breaks the bootstrap constraint."
Do NOT resolve the thread when pushing back. Leave it open for the reviewer to confirm.
| Case | Status | Duration (ms) | Turns | Tokens | Tool calls | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Without | With | Δ | Without | With | Δ | Without | With | Δ | Without | With | Δ | ||
case-01 | fail→fail | 20,995 | 8,968 | -57% | 1 | 1 | 0% | 3,378 | 3,161 | -6% | 0 | 0 | — |
case-02 | fail→fail | 30,145 | 9,245 | -69% | 1 | 1 | 0% | 3,638 | 3,190 | -12% | 0 | 0 | — |
case-03 | fail→fail | 11,272 | 10,720 | -5% | 1 | 1 | 0% | 855 | 3,371 | +294% | 0 | 0 | — |
case-04 | pass→fail | 6,688 | 28,002 | +319% | 1 | 1 | 0% | 1,085 | 4,075 | +276% | 0 | 0 | — |
case-05 | fail→fail | 17,475 | 23,348 | +34% | 1 | 1 | 0% | 2,129 | 4,909 | +131% | 0 | 0 | — |
case-06 | pass→pass | 16,576 | 10,362 | -37% | 1 | 1 | 0% | 2,635 | 4,399 | +67% | 0 | 0 | — |
case-07 | fail→pass | 9,848 | 4,803 | -51% | 1 | 1 | 0% | 1,321 | 3,432 | +160% | 0 | 0 | — |
case-08 | fail→pass | 21,714 | 3,842 | -82% | 1 | 1 | 0% | 4,064 | 3,391 | -17% | 0 | 0 | — |
case-09 | pass→pass | 9,983 | 7,603 | -24% | 1 | 1 | 0% | 1,774 | 4,035 | +127% | 0 | 0 | — |
case-10 | pass→pass | 13,582 | 6,872 | -49% | 1 | 1 | 0% | 1,971 | 3,933 | +100% | 0 | 0 | — |
case-11 | pass→pass | 5,915 | 4,096 | -31% | 1 | 1 | 0% | 981 | 3,255 | +232% | 0 | 0 | — |
case-12 | pass→pass | 8,731 | 4,701 | -46% | 1 | 1 | 0% | 1,316 | 3,516 | +167% | 0 | 0 | — |
case-13 | fail→pass | 24,305 | 3,348 | -86% | 1 | 1 | 0% | 2,376 | 3,268 | +38% | 0 | 0 | — |
case-14 | pass→pass | 6,016 | 5,556 | -8% | 1 | 1 | 0% | 840 | 3,520 | +319% | 0 | 0 | — |
case-15 | pass→pass | 8,179 | 14,052 | +72% | 1 | 1 | 0% | 1,001 | 3,593 | +259% | 0 | 0 | — |
case-16 | pass→pass | 14,972 | 5,048 | -66% | 1 | 1 | 0% | 1,282 | 3,416 | +166% | 0 | 0 | — |
case-17 | pass→pass | 10,440 | 6,088 | -42% | 1 | 1 | 0% | 1,923 | 3,741 | +95% | 0 | 0 | — |
case-18 | pass→pass | 7,352 | 6,016 | -18% | 1 | 1 | 0% | 1,045 | 3,581 | +243% | 0 | 0 | — |
case-19 | pass→pass | 14,455 | 9,069 | -37% | 1 | 1 | 0% | 2,649 | 4,182 | +58% | 0 | 0 | — |
case-20 | pass→pass | 4,672 | 3,119 | -33% | 1 | 1 | 0% | 620 | 3,228 | +421% | 0 | 0 | — |
case-21 | pass→pass | 9,641 | 6,076 | -37% | 1 | 1 | 0% | 1,261 | 3,533 | +180% | 0 | 0 | — |
case-22 | pass→pass | 7,382 | 2,798 | -62% | 1 | 1 | 0% | 1,172 | 3,116 | +166% | 0 | 0 | — |
DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 22 cases were attempted, and 18 counted toward the lift figure. The other 4 produced results that are not comparable between the two arms, so they are excluded from the headline rather than averaged into it. The headline lift of +9 percentage points is the difference between those two pass rates over the 18 comparable cases. 3 cases got worse with the skill loaded, and they are included in that figure.
Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.
Other measured skills in the registry, with their headline benchmark lift.