▸case-01 We're seeing intermittent API gateway timeouts during peak traffic hours, but we don't know if it's caused by database connection pooling issues or a memory leak in the router service. Furthermore, product leadership hasn't defined acceptable latency limits or fallback behavior under heavy load. Could you run a two-part investigation to trace the underlying breakdown and clarify the missing requirements for a proper resolution before we begin planning a fix? | pass→pass | 16,324 | 18,317 | +12% | 1 | 1 | 0% | 2,581 | 3,473 | +35% | 0 | 0 | — |
▸case-02 Our checkout microservice is silently dropping payment confirmation webhooks for about 2% of transactions. We need to diagnose the actual bug behind these lost events, but we also lack clear acceptance criteria for retry policies and customer notification behavior when a webhook fails. Please perform a deep dive to analyze the root failure and articulate the scope and functional specifications for fixing it. | pass→pass | 28,839 | 20,842 | -28% | 1 | 1 | 0% | 4,725 | 3,762 | -20% | 0 | 0 | — |
▸case-03 Enterprise users are getting randomly logged out mid-session following our recent auth service patch, and the cause remains unexplained. On top of that, team leads disagree on whether session persistence should be extended or if silent token refresh is required. Please investigate why these disconnections are happening and crystallize the exact boundary requirements for the fix so we have a clear target. | pass→pass | 16,094 | 18,174 | +13% | 1 | 1 | 0% | 2,456 | 3,358 | +37% | 0 | 0 | — |
▸case-04 We have a well-understood race condition in the signup worker thread where two database writes execute out of order. We already have detailed acceptance criteria for fixing the locking mechanism. Please write a step-by-step implementation plan for refactoring this thread safety issue. | pass→pass | 11,462 | 11,480 | +0% | 1 | 1 | 0% | 1,912 | 2,262 | +18% | 0 | 0 | — |
▸case-05 The design team provided a complete specification with explicit criteria for adding dark mode support to all account settings screens, and there are no backend errors or unknown bugs. Please generate the CSS component implementation tasks directly. | fail→pass | 3,161 | 9,349 | +196% | 1 | 1 | 0% | 499 | 1,841 | +269% | 0 | 0 | — |
▸case-06 We know exactly why the cache is failing: key memory limits are exhausted due to missing TTL parameters, and product agreed to set eviction to LRU with 24-hour TTLs. Please update the cache configuration parameters directly. | pass→pass | 9,848 | 4,864 | -51% | 1 | 1 | 0% | 1,578 | 1,162 | -26% | 0 | 0 | — |
▸case-07 Real-time communication drops sporadically during live events, but logs show no clear stack traces. Concurrently, product management has not specified fallback protocols or disconnections SLA limits. Conduct a full diagnostic trace and requirements definition. | pass→pass | 22,812 | 25,395 | +11% | 1 | 1 | 0% | 3,686 | 4,357 | +18% | 0 | 0 | — |
▸case-08 Nightly ETL jobs are intermittently storing corrupted JSON payloads in S3 without throwing fatal errors. The pipeline team also lacks clear specifications on data validation checks and alert thresholds. Perform a comprehensive inquiry into the cause and scope. | pass→pass | 24,541 | 20,550 | -16% | 1 | 1 | 0% | 3,816 | 3,600 | -6% | 0 | 0 | — |
▸case-09 Some users report double billing on renewal days, but internal server metrics show normal 200 HTTP responses. Furthermore, business operations has not clarified refund automation boundaries or retry limits. Please analyze the root issue and establish missing specifications. | pass→pass | 17,786 | 22,496 | +26% | 1 | 1 | 0% | 3,010 | 4,041 | +34% | 0 | 0 | — |
▸case-10 The search indexing service crashes under sustained load without clear stack traces, and stakeholders cannot agree on partial index freshness targets or resource caps. Run a systematic analysis to discover why it fails and establish exact fix boundaries. | fail→pass | 21,275 | 22,493 | +6% | 1 | 1 | 0% | 3,245 | 4,028 | +24% | 0 | 0 | — |
▸case-11 Push notifications arrive up to three hours late for push subscribers, but queue lengths remain low. Additionally, product managers have not defined target delivery windows or retry strategies. Investigate the failure cause and define missing product criteria. | pass→pass | 18,103 | 23,610 | +30% | 1 | 1 | 0% | 2,909 | 4,076 | +40% | 0 | 0 | — |
▸case-12 Tenant data occasionally leaks into search queries across customer boundaries, but cache isolation seems correct. Simultaneously, legal has not defined compliance requirements for cross-tenant isolation enforcement. We need a dual-phase analysis before planning a fix. | pass→pass | 16,673 | 24,494 | +47% | 1 | 1 | 0% | 2,434 | 3,409 | +40% | 0 | 0 | — |
▸case-13 Production storage volumes fill up unexpectedly every third weekend, yet log file rotators report healthy operations. Management hasn't decided on retention policies or automated purging rules. Please trace the cause and define precise fix boundaries. | pass→pass | 21,077 | 15,442 | -27% | 1 | 1 | 0% | 2,986 | 2,850 | -5% | 0 | 0 | — |
▸case-14 OAuth authentication occasionally enters an infinite loop for federated users, but browser cookies look valid. At the same time, security hasn't finalized session expiration rules or fallback login paths. Perform a deep inquiry to trace the bug and clarify scope. | pass→pass | 18,492 | 23,590 | +28% | 1 | 1 | 0% | 2,830 | 3,886 | +37% | 0 | 0 | — |
▸case-15 Complex nested GraphQL queries crash the engine without producing server logs, and product management has not established depth limits or rate-limiting tiers. Trace the root failure and clarify explicit spec limits. | pass→pass | 18,116 | 19,287 | +6% | 1 | 1 | 0% | 2,783 | 3,079 | +11% | 0 | 0 | — |
▸case-16 Background job processing halts entirely every few days without trigger logs, while operations has not specified worker timeout policies or dead-letter queue behavior. Execute an investigation into the issue cause and functional requirements. | pass→pass | 23,798 | 23,219 | -2% | 1 | 1 | 0% | 3,889 | 3,241 | -17% | 0 | 0 | — |
▸case-17 Kafka events reach consumers out of order during deployment rollouts, yet partition keys match. Furthermore, stakeholders haven't specified whether strict ordering or idempotent processing is required. Trace the breakdown and define acceptance boundaries. | pass→pass | 19,231 | 27,590 | +43% | 1 | 1 | 0% | 3,269 | 4,892 | +50% | 0 | 0 | — |
▸case-18 Large file uploads complete with 200 OK statuses but store truncated blobs in cloud storage. Meanwhile, engineering leads lack clear requirements regarding chunked uploading or checksum verification steps. Analyze the root cause and crystallize specs. | pass→pass | 19,800 | 30,817 | +56% | 1 | 1 | 0% | 3,282 | 5,515 | +68% | 0 | 0 | — |
▸case-19 Automated traffic bypasses global API rate limits under heavy load, but redis keys show normal counts. In addition, API governance hasn't defined burst allowances or HTTP response codes for rate-limited clients. Execute a systematic probe and specification process. | pass→pass | 21,120 | 22,059 | +4% | 1 | 1 | 0% | 2,287 | 3,745 | +64% | 0 | 0 | — |
▸case-20 Transactional emails are silently marked as spam by major email providers despite valid DKIM signatures. The product team has not established unsubscribe behavior or retry schedules. Investigate why this happens and crystallize acceptance criteria. | pass→pass | 18,054 | 24,516 | +36% | 1 | 1 | 0% | 3,110 | 4,627 | +49% | 0 | 0 | — |
▸case-21 Internal service discovery intermittently fails to resolve container names during pod scaling, and network architects have not decided on DNS caching TTLs or fallback nameservers. Conduct a two-stage investigation. | pass→pass | 24,034 | 20,858 | -13% | 1 | 1 | 0% | 3,879 | 3,827 | -1% | 0 | 0 | — |
▸case-22 User session data becomes desynchronized between edge nodes and database replicas under regional failover, while product leads have not agreed on eventual consistency grace periods. Trace the root issue and establish spec boundaries. | pass→pass | 23,723 | 24,144 | +2% | 1 | 1 | 0% | 3,455 | 4,351 | +26% | 0 | 0 | — |