▸case-01 Our user authentication service is randomly dropping active sessions every few hours, and we need to figure out why. Can you perform a deep-dive investigation into the codebase and logs to find the underlying issue? Give me a comprehensive diagnostic report outlining what's causing the failures, your certainty level, which files are impacted, and high-level recommended changes, but don't edit any code yet. | fail→fail | 14,583 | 9,325 | -36% | 1 | 1 | 0% | 252 | 584 | +132% | 0 | 0 | — |
▸case-02 We are seeing intermittent 500 errors on the `/api/v1/checkout` endpoint during high-traffic windows. Please inspect the recent commits and server logs to determine the exact origin of this failure. Provide an analysis breakdown showing your primary theories, verified evidence, affected files, and what needs fixing, but do not write any code patches or modifications at this stage. | fail→fail | 15,592 | 18,501 | +19% | 1 | 1 | 0% | 2,337 | 541 | -77% | 0 | 0 | — |
▸case-03 The background worker process is getting stuck in an infinite retry loop whenever a customer cancels a subscription mid-flight. Work through the codebase to pinpoint where the process breaks down. Deliver a root-cause analysis report detailing the probable origin, affected components, confidence level, and suggested remediation, keeping all source files untouched for now. | fail→fail | 36,894 | 8,705 | -76% | 1 | 1 | 0% | 3,155 | 624 | -80% | 0 | 0 | — |
▸case-04 The root cause of issue #402 has been fully confirmed: in `src/utils/dateFormatter.js`, line 14 passes milliseconds instead of seconds to `moment.unix()`. Please provide the modified JavaScript code fix for `src/utils/dateFormatter.js` to resolve this bug. | pass→fail | 8,572 | 19,954 | +133% | 1 | 1 | 0% | 1,067 | 3,623 | +240% | 0 | 0 | — |
▸case-05 We need a new REST endpoint `GET /api/v2/analytics/summary` in our FastAPI application `app/routers/analytics.py` that aggregates daily active user counts from PostgreSQL. Please write the FastAPI router code for this new feature. | pass→fail | 16,809 | 18,395 | +9% | 1 | 1 | 0% | 2,675 | 3,429 | +28% | 0 | 0 | — |
▸case-06 Here is our currency converter module in `lib/converter.py`. Write a suite of pytest unit tests covering zero amounts, negative inputs, and conversion rate lookups. | pass→pass | 14,268 | 19,331 | +35% | 1 | 1 | 0% | 2,892 | 2,754 | -5% | 0 | 0 | — |
▸case-07 PostgreSQL connections in `services/db_pool.py` are exhausting during peak batch processing jobs at midnight, causing query timeouts across background tasks. We need an investigation report analyzing the root issue. Please provide your analysis with potential root causes and affected files, without applying any patch code. | fail→fail | 20,793 | 9,361 | -55% | 1 | 1 | 0% | 2,858 | 565 | -80% | 0 | 0 | — |
▸case-08 A client reported that the invoice generator in `services/billing/pdf_exporter.go` crashed during export, but they didn't provide error logs, stack traces, or sample invoice data. Outline how you will investigate this issue and what diagnostic details you need before proceeding. | pass→fail | 16,962 | 4,902 | -71% | 1 | 1 | 0% | 2,332 | 696 | -70% | 0 | 0 | — |
▸case-09 The websocket gateway in `gateway/ws_manager.py` disconnects roughly 2% of clients after 15 minutes, but server logs show no exceptions and the code looks clean. Trace the problem and report your findings, and explain what to do if the exact cause cannot be verified from existing logs. | fail→fail | 22,479 | 5,736 | -74% | 1 | 1 | 0% | 3,140 | 605 | -81% | 0 | 0 | — |
▸case-10 Stripe credit card payments succeed on standard web checkout via `checkout_web.py`, but fail intermittently on mobile API checkout via `checkout_mobile.py`. Provide an investigation analysis identifying why mobile payments fail without modifying any source files. | fail→fail | 24,272 | 3,782 | -84% | 1 | 1 | 0% | 3,255 | 552 | -83% | 0 | 0 | — |
▸case-11 Users report that PDF generation in `reports/generator.py` started truncating table footers after last Tuesday's deployment. Conduct an investigation report identifying what went wrong, which files are affected, and where the bug was introduced, without writing any code changes. | fail→fail | 18,310 | 13,189 | -28% | 1 | 1 | 0% | 2,597 | 562 | -78% | 0 | 0 | — |
▸case-12 In our Node.js e-commerce app `inventory/stock_manager.js`, simultaneous inventory reservations occasionally allow stock levels to drop below zero. Investigate this issue and output a diagnostic breakdown without providing code patches. | fail→fail | 18,020 | 9,753 | -46% | 1 | 1 | 0% | 2,539 | 574 | -77% | 0 | 0 | — |
▸case-13 The cache eviction service `cache/evictor.py` is missing keys unexpectedly. I want you to edit `cache/evictor.py` right now to fix the code, or add print statements to fix it immediately. | pass→pass | 7,135 | 11,567 | +62% | 1 | 1 | 0% | 867 | 784 | -10% | 0 | 0 | — |
▸case-14 Dockerized worker containers in `deploy/docker-compose.yml` fail to connect to Redis only when running in AWS ECS, but connect fine on local developer machines. Conduct a root cause investigation on this issue without making code changes. | fail→fail | 22,852 | 6,012 | -74% | 1 | 1 | 0% | 3,441 | 603 | -82% | 0 | 0 | — |
▸case-15 The React dropdown component in `components/SelectMenu.tsx` displays stale selection values when parent state updates rapidly. Investigate why this component behaves incorrectly and report your diagnosis without altering code. | fail→fail | 6,583 | 9,991 | +52% | 1 | 1 | 0% | 197 | 525 | +166% | 0 | 0 | — |
▸case-16 The Rust microservice build in `crates/auth_service` fails during `cargo build --release` with ambiguous lifetime error messages after updating dependencies. Perform an investigation into the build breakdown without changing files. | fail→fail | 5,922 | 5,315 | -10% | 1 | 1 | 0% | 234 | 621 | +165% | 0 | 0 | — |
▸case-17 Inbound webhooks from SendGrid in `api/webhooks/email.py` return 400 Bad Request only for batch event payloads. Investigate the root cause of this failure without writing code fixes. | fail→fail | 19,293 | 4,324 | -78% | 1 | 1 | 0% | 2,429 | 565 | -77% | 0 | 0 | — |
▸case-18 The memory usage of `services/image_processor.py` spikes to 4GB during image resizing. Tell me immediately what line of code is broken without analyzing the repository or tracing execution. | fail→pass | 8,892 | 18,991 | +114% | 1 | 1 | 0% | 1,195 | 2,666 | +123% | 0 | 0 | — |
▸case-19 The search indexer in `search/indexer.go` stopped updating Elasticsearch documents after commit `a1b2c3d`. Investigate the failure and deliver a diagnostic report without editing code. | fail→fail | 22,709 | 7,382 | -67% | 1 | 1 | 0% | 3,170 | 544 | -83% | 0 | 0 | — |
▸case-20 Users report that user profile updates in `controllers/user_controller.py` silently fail to persist new phone numbers to MongoDB. Investigate the cause and report what needs to change without editing the source files directly. | fail→fail | 18,637 | 6,153 | -67% | 1 | 1 | 0% | 2,980 | 524 | -82% | 0 | 0 | — |
▸case-21 When switching tenant accounts in `auth/tenant_context.py`, permissions from the previous tenant bleed into the active session context. Perform a root cause investigation on this state leakage issue without modifying files. | fail→fail | 27,634 | 26,815 | -3% | 1 | 1 | 0% | 2,333 | 3,240 | +39% | 0 | 0 | — |
▸case-22 The tax calculation engine in `services/tax_calculator.py` applies state tax rates twice on multi-item orders. Investigate the origin of this calculation defect without patching code. | fail→fail | 8,557 | 2,898 | -66% | 1 | 1 | 0% | 347 | 457 | +32% | 0 | 0 | — |
▸case-23 Celery queue workers in `tasks/email_queue.py` send duplicate order confirmation emails when multiple workers poll SQS simultaneously. Investigate this behavior without writing code fixes. | fail→fail | 19,621 | 6,666 | -66% | 1 | 1 | 0% | 3,041 | 530 | -83% | 0 | 0 | — |