▸case-01 The test suite is broken after we updated our database schema dependencies. Please run the tests, fix all the failing specs across the modules, and do a final check to ensure everything is green. | fail→fail | 13,689 | 7,263 | -47% | 1 | 1 | 0% | 164 | 486 | +196% | 0 | 0 | — |
▸case-02 In our PyTest repo, 40 tests under tests/api/ are failing with 'alembic.util.exc.CommandError: Can't locate revision' during fixture setup. The junior dev suggests adding mock return values inside each of the 40 test functions. How should we approach diagnosing and fixing these failures step-by-step? Provide a concrete plan. | pass→pass | 20,066 | 20,009 | -0% | 1 | 1 | 0% | 2,624 | 2,769 | +6% | 0 | 0 | — |
▸case-03 Our Jest CI run failed with 15 broken tests scattered across user, order, and billing test suites. Should we open each test file alphabetically and fix errors file-by-file, or how should we structure the debugging process? Outline the action plan. | fail→pass | 17,966 | 16,763 | -7% | 1 | 1 | 0% | 2,227 | 2,288 | +3% | 0 | 0 | — |
▸case-04 We ran our test suite and 25 tests are failing. 20 of them throw 'Error: JWT_SECRET is not defined' in test setup, and 5 fail in discount_calculator.test.js with expected 15 to equal 12. Should we start by tuning the discount math assertions? Detail the fix ordering strategy. | pass→pass | 10,451 | 14,686 | +41% | 1 | 1 | 0% | 1,828 | 1,914 | +5% | 0 | 0 | — |
▸case-05 In our Rails project, 30 RSpec specs fail with 'ActiveRecord::RecordNotUnique: PG::UniqueViolation' on user creation. A team member wants to change all user factory emails to use random UUIDs in every spec file. How should we systematically group and resolve this? | fail→fail | 20,680 | 17,297 | -16% | 1 | 1 | 0% | 2,771 | 2,405 | -13% | 0 | 0 | — |
▸case-06 We modified our Go backend interfaces and now 'go test ./...' fails with 12 errors in pkg/auth and pkg/storage. Should we edit all failing test files at once and run the full suite at the end, or what step-by-step methodology should we follow? Describe the process. | fail→pass | 11,320 | 14,953 | +32% | 1 | 1 | 0% | 1,998 | 1,941 | -3% | 0 | 0 | — |
▸case-07 Our Spring Boot test suite shows 20 failing test classes. 18 throw 'Failed to load ApplicationContext' and 2 throw 'AssertionFailedError: expected <foo> but was <bar>'. The author wants to address the two string assertion tests first because they look easier. Detail the systematic fix sequence. | pass→fail | 21,144 | 23,745 | +12% | 1 | 1 | 0% | 2,597 | 3,284 | +26% | 0 | 0 | — |
▸case-08 Our Cypress E2E suite has 10 specs failing with 404 status codes on API requests. Should we add cy.wait(5000) sleeps to all failing spec files? Provide the step-by-step diagnostic and remediation process. | fail→fail | 18,291 | 20,051 | +10% | 1 | 1 | 0% | 3,285 | 2,882 | -12% | 0 | 0 | — |
▸case-09 8 PyTest files are failing with AttributeError after a refactoring of our HTTP client module. Someone suggested modifying each test to catch AttributeError and skip the assertion. Explain how to group and fix these systematic failures properly. | fail→pass | 19,706 | 17,325 | -12% | 1 | 1 | 0% | 2,474 | 2,411 | -3% | 0 | 0 | — |
▸case-10 We just added a new payment gateway integration in payment_gateway.py. We need you to write comprehensive unit tests covering success, decline, and network timeout scenarios with PyTest mocks. Provide the test code. | pass→fail | 23,310 | 4,584 | -80% | 1 | 1 | 0% | 4,164 | 560 | -87% | 0 | 0 | — |
▸case-11 We want to set up a GitHub Actions workflow file .github/workflows/test.yml that runs pytest across Python 3.9, 3.10, and 3.11 with PostgreSQL service containers. Provide the complete YAML configuration. | pass→pass | 14,877 | 12,660 | -15% | 1 | 1 | 0% | 2,121 | 1,859 | -12% | 0 | 0 | — |
▸case-12 Our test suite is 100% green and passing, but running all unit tests takes 12 minutes. How can we refactor our PyTest fixtures and parallelize execution with pytest-xdist to reduce runtime? Provide a performance optimization plan. | pass→pass | 19,245 | 24,591 | +28% | 1 | 1 | 0% | 3,428 | 3,633 | +6% | 0 | 0 | — |
▸case-13 After updating @mantine/core, 14 Vitest tests fail across 5 component specs. Should we edit all 5 component test files simultaneously and push directly to main? Describe the systematic protocol for fixing and verifying these component tests. | pass→pass | 18,176 | 15,972 | -12% | 1 | 1 | 0% | 2,312 | 2,067 | -11% | 0 | 0 | — |
▸case-14 In our Playwright web test suite, 10 tests fail with selector timeouts on the user avatar header, and 2 fail on visual diff snapshots. Should we update the visual diff baseline images first? Detail the fix ordering and grouping steps. | pass→pass | 20,521 | 16,570 | -19% | 1 | 1 | 0% | 2,305 | 2,194 | -5% | 0 | 0 | — |
▸case-15 16 PHPUnit tests fail with 'PDOException: Integrity constraint violation: 1452 Foreign key constraint fails'. A developer suggests turning off foreign key checks globally in phpunit.xml. Provide the diagnostic strategy and correct remediation steps. | fail→pass | 14,783 | 18,721 | +27% | 1 | 1 | 0% | 2,459 | 2,509 | +2% | 0 | 0 | — |
▸case-16 We refactored our payment processing service object signature from .call(user, amount) to .call(account:, amount_cents:). Now 10 MiniTest specs throw NoMethodError. Should we suppress exceptions in test setup? Explain the systematic fix workflow. | pass→pass | 14,768 | 9,322 | -37% | 1 | 1 | 0% | 2,596 | 1,855 | -29% | 0 | 0 | — |
▸case-17 Running 'cargo test' yields 18 compilation and runtime test failures across 3 Rust crates. 15 errors mention 'unknown field user_id'. How should we structure the debugging and fixing process instead of randomly patching files? | pass→pass | 19,653 | 11,014 | -44% | 1 | 1 | 0% | 2,474 | 2,197 | -11% | 0 | 0 | — |
▸case-18 After converting our Swift NetworkManager to async/await, 12 XCTests are failing with nil unwrap errors because completion handlers were removed. Should we delete these failing XCTest cases and write new ones from scratch? Provide the step-by-step test fixing protocol. | fail→pass | 14,257 | 15,598 | +9% | 1 | 1 | 0% | 2,493 | 2,058 | -17% | 0 | 0 | — |
▸case-19 We added a 'tenant_id NOT NULL' column to our postgres schema, and now 22 integration tests fail with 'null value in column tenant_id violates not-null constraint'. A colleague wants to edit each test file individually to insert random tenant IDs. Describe the systematic fix workflow. | pass→pass | 20,446 | 15,752 | -23% | 1 | 1 | 0% | 2,673 | 2,111 | -21% | 0 | 0 | — |
▸case-20 15 Jest tests pass individually when run with -t, but fail when running npm test on the whole suite. A dev suggests running Jest sequentially with --runInBand as a permanent fix. How should we systematically identify and fix this issue? | pass→pass | 20,000 | 21,349 | +7% | 1 | 1 | 0% | 2,641 | 3,227 | +22% | 0 | 0 | — |
▸case-21 8 Mocha tests are failing with 'Timeout of 2000ms exceeded'. The junior dev increased test timeout to 60000ms in mocha.opts to make them pass. Is this the right way to fix failing tests? Explain how to approach fixing these properly. | pass→pass | 19,446 | 20,113 | +3% | 1 | 1 | 0% | 2,536 | 2,881 | +14% | 0 | 0 | — |
▸case-22 In our .NET 8 solution, 14 xUnit tests fail with NullReferenceException during DatabaseFixture initialization. Should we wrap every test method body in a try-catch block? Provide the step-by-step systematic fixing plan. | pass→pass | 18,753 | 15,281 | -19% | 1 | 1 | 0% | 2,529 | 2,034 | -20% | 0 | 0 | — |
▸case-23 Running pytest shows 7 total test failures: 2 fail with 'botocore.exceptions.NoCredentialsError' in fixture setup, and 5 fail on dictionary value assertions in data processing functions. Detail the exact fix order strategy and verification steps. | pass→pass | 22,313 | 16,754 | -25% | 1 | 1 | 0% | 2,610 | 2,228 | -15% | 0 | 0 | — |