▸case-10 In PR #501, the CI job fails on a shared integration test that was recently fixed on the `main` branch. PR #501's own code changes do not touch that integration test at all. What step should be taken to address this merge-blocking CI failure? | pass→pass | 5,998 | 2,327 | -61% | 1 | 1 | 0% | 1,259 | 791 | -37% | 0 | 0 | — |
▸case-01 Can you take over PR #142 and handle everything needed to make it merge-ready? Please clear any open merge conflicts, address feedback or bug reports from review comments, and resolve failing CI runs until all checks pass and the pull request is ready for merging. | fail→fail | 3,798 | 4,337 | +14% | 1 | 1 | 0% | 238 | 494 | +108% | 0 | 0 | — |
▸case-02 During a git merge operation for branch feature/auth-redesign into main, you find that the base branch removed the old session table entirely, while feature/auth-redesign heavily refactors index queries on that same session table. The business intent of the two branches directly contradicts each other. Should you pick one side and commit the merge, or abort and request team clarification? | pass→pass | 9,116 | 2,398 | -74% | 1 | 1 | 0% | 1,497 | 697 | -53% | 0 | 0 | — |
▸case-03 While resolving merge conflicts in `src/utils/logger.ts`, main added structured JSON output logging while feature/telemetry added a custom log level filter. Both changes are complementary. How should the conflict be resolved? | pass→fail | 7,703 | 2,996 | -61% | 1 | 1 | 0% | 1,602 | 404 | -75% | 0 | 0 | — |
▸case-04 When invoking the GitHub REST API or CLI to retrieve review comments on pull request #88, you need to decide how to fetch comment threads without getting overwhelmed by old conversation threads that authors already marked as resolved. What filter parameters or strategy should you apply when fetching comments? | pass→pass | 8,751 | 10,557 | +21% | 1 | 1 | 0% | 2,082 | 1,918 | -8% | 0 | 0 | — |
▸case-05 When fetching review comments on PR #55 via tool calls, you receive API responses. To minimize token consumption and avoid context overflow, should you load the full nested JSON schema payload for all review comments across the repo, or extract only specific fields? | pass→pass | 3,967 | 3,725 | -6% | 1 | 1 | 0% | 697 | 938 | +35% | 0 | 0 | — |
▸case-06 Automated code analysis tool Bugbot flagged a warning on PR #102, claiming `user.age` could be undefined. However, line 12 explicitly checks `if (user.age === undefined) return;` right before the flagged line. Should you apply Bugbot's suggested patch to wrap it in an extra check? | pass→pass | 9,380 | 2,739 | -71% | 1 | 1 | 0% | 1,773 | 832 | -53% | 0 | 0 | — |
▸case-07 Bugbot left a comment on PR #204 suggesting that a lock in `db/pool.go` might lead to a deadlock under high concurrency, but the concurrency semantics are complex and you cannot definitively verify whether the deadlock is possible. How should you handle this comment? | pass→pass | 11,662 | 2,943 | -75% | 1 | 1 | 0% | 1,901 | 784 | -59% | 0 | 0 | — |
▸case-08 In PR #305, the `e2e-tests` CI job is failing because a flaky test `test_billing_retry` timed out. To get the pull request status to green quickly, should you edit `.github/workflows/ci.yml` to remove `e2e-tests` or pass `--ignore-failures`? | pass→pass | 7,404 | 3,816 | -48% | 1 | 1 | 0% | 1,295 | 951 | -27% | 0 | 0 | — |
▸case-09 While trying to get CI to pass on PR #412 (which adds a user bio field), you notice an unhandled exception in an unrelated payment module `services/stripe.ts` that causes a linter error in main. Should you fix `services/stripe.ts` inside this pull request to fix the linter error? | pass→pass | 8,007 | 3,676 | -54% | 1 | 1 | 0% | 1,378 | 993 | -28% | 0 | 0 | — |
▸case-11 You have just pushed a commit fixing a scoping bug requested by reviewers on PR #610. What is the immediate next step in managing the pull request state? | pass→pass | 4,927 | 4,928 | +0% | 1 | 1 | 0% | 886 | 616 | -30% | 0 | 0 | — |
▸case-12 Bugbot flagged a missing null check on line 45 of `src/api/user.ts` in PR #115. Upon inspection, `response.data` can indeed be null when the API returns HTTP 204. How should this feedback be handled? | pass→fail | 7,624 | 4,973 | -35% | 1 | 1 | 0% | 1,522 | 663 | -56% | 0 | 0 | — |
▸case-13 A human reviewer left a change request asking to move database seed scripts out of `src/controllers` and into `scripts/db`. Is this actionable review feedback, and what should be done? | pass→pass | 8,693 | 5,280 | -39% | 1 | 1 | 0% | 1,418 | 806 | -43% | 0 | 0 | — |
▸case-14 When triaging comments on PR #789, you see 15 resolved threads from last week and 2 open unresolved threads from today. Which comments should be reviewed and acted upon? | pass→fail | 7,092 | 1,547 | -78% | 1 | 1 | 0% | 1,059 | 499 | -53% | 0 | 0 | — |
▸case-15 A test failure on PR #220 is directly caused by a typo in a newly added function parameter in `src/helpers/format.ts` introduced by this PR. How should this failure be addressed? | pass→fail | 5,506 | 7,846 | +42% | 1 | 1 | 0% | 1,189 | 1,539 | +29% | 0 | 0 | — |
▸case-16 In `config/settings.json`, main added a field `"timeout": 30` and the feature branch added `"maxRetries": 3`. How should the merge conflict in `config/settings.json` be resolved? | pass→fail | 5,587 | 3,126 | -44% | 1 | 1 | 0% | 1,149 | 465 | -60% | 0 | 0 | — |
▸case-17 The CI setup step fails because the GitHub Actions runner environment changed its Node.js default version across the whole repository, requiring an update to `.github/workflows/build.yml`. Making this change is outside the feature scope of PR #888. What action should be taken? | pass→pass | 7,782 | 7,308 | -6% | 1 | 1 | 0% | 1,391 | 1,081 | -22% | 0 | 0 | — |
▸case-18 After making changes to address reviewer comments on PR #330, you commit the changes locally. What must be done before marking the PR ready to merge? | pass→fail | 5,672 | 3,815 | -33% | 1 | 1 | 0% | 1,058 | 937 | -11% | 0 | 0 | — |
▸case-19 You need to locate the file line referenced in a GitHub pull request review comment. Should you download the entire commit history and complete pull request JSON body, or extract only the target file path and line reference? | pass→fail | 3,592 | 1,612 | -55% | 1 | 1 | 0% | 613 | 545 | -11% | 0 | 0 | — |
▸case-20 Can you create a brand new git feature branch named `feature/dark-mode`, write a toggle component in `src/components/ThemeToggle.tsx`, commit the changes, and open a new pull request against `main`? | pass→fail | 7,848 | 3,831 | -51% | 1 | 1 | 0% | 1,767 | 462 | -74% | 0 | 0 | — |
▸case-21 Please act as a peer code reviewer for PR #990. Evaluate whether the architectural design pattern in `src/services/auth.ts` follows clean architecture principles, assign a score out of 10, and submit an official GitHub review approval or rejection. | fail→fail | 8,256 | 8,647 | +5% | 1 | 1 | 0% | 1,293 | 476 | -63% | 0 | 0 | — |
▸case-22 We need to set up GitHub Actions for a new repository from scratch. Please write a `.github/workflows/ci.yml` file that runs `npm test` and `npm run lint` on every push to `main`. | pass→fail | 4,791 | 3,520 | -27% | 1 | 1 | 0% | 1,008 | 499 | -50% | 0 | 0 | — |