▸case-01 We are building a real-time collaborative document editor for our enterprise team. We are undecided between using WebSockets with operational transformation (OT) versus CRDTs over WebSockets/WebRTC. Could you evaluate our situation, break down the choices, give us a clear technical direction, explain your reasoning, and list out actionable steps to get started? | fail→pass | 20,016 | 17,016 | -15% | 1 | 1 | 0% | 3,188 | 3,139 | -2% | 0 | 0 | — |
▸case-02 Our e-commerce checkout service experiences severe latency spikes during flash sales when inventory queries hit the primary SQL database. We need advice on resolving this bottleneck. Please analyze the issue, compare potential remedies, suggest the best solution with justification, and outline the sequence of steps for implementation. | pass→pass | 16,281 | 13,271 | -18% | 1 | 1 | 0% | 2,802 | 2,685 | -4% | 0 | 0 | — |
▸case-03 We have a monolithic Node.js application that is becoming difficult to scale and maintain across three feature teams. We are debating whether to migrate to microservices or adopt a modular monolith approach. Can you look at our setup, weigh the options, recommend an architecture strategy with your rationale, and provide concrete rollout phases? | fail→pass | 16,189 | 16,791 | +4% | 1 | 1 | 0% | 2,989 | 3,095 | +4% | 0 | 0 | — |
▸case-04 We are developing a mobile app that needs to fetch nested user profile data over low-bandwidth mobile networks, currently choosing between REST APIs and GraphQL. Please analyze our situation, present the options, recommend an approach with rationale, and provide next steps. | pass→pass | 16,174 | 13,655 | -16% | 1 | 1 | 0% | 2,639 | 2,681 | +2% | 0 | 0 | — |
▸case-05 Our startup processes bursty batch data processing jobs twice a day. We are choosing between AWS Lambda serverless execution and running a persistent Amazon EKS Kubernetes cluster. Analyze our workload, compare the options, give a recommendation with rationale, and list next steps. | fail→pass | 13,524 | 11,499 | -15% | 1 | 1 | 0% | 2,448 | 2,292 | -6% | 0 | 0 | — |
▸case-06 Our payment processing service uses synchronous HTTP calls between six microservices, causing cascading failures when one service is down. Analyze our current architecture, present alternative communication patterns, recommend a resilient solution with rationale, and outline next steps. | fail→pass | 17,792 | 13,442 | -24% | 1 | 1 | 0% | 3,173 | 2,678 | -16% | 0 | 0 | — |
▸case-07 We are designing a financial ledger application requiring ACID compliance and strict schema enforcement, but our team is considering MongoDB due to developer familiarity. Analyze our requirements, compare relational vs document store approaches, recommend a database paradigm with rationale, and list next steps. | pass→pass | 18,061 | 13,962 | -23% | 1 | 1 | 0% | 3,002 | 2,631 | -12% | 0 | 0 | — |
▸case-08 Our web application experiences slow page loads because identical complex aggregation queries are re-computed on every user request. Analyze the performance bottleneck, evaluate caching strategies, recommend an architecture with rationale, and outline next steps. | pass→pass | 17,143 | 16,051 | -6% | 1 | 1 | 0% | 2,725 | 2,912 | +7% | 0 | 0 | — |
▸case-09 We are architecting authentication for a multi-domain web application and cannot decide between stateless JWT tokens stored in browser local storage and centralized session storage using HTTP-only cookies. Analyze our setup, weigh the options, recommend a secure approach with rationale, and provide next steps. | pass→pass | 19,323 | 14,323 | -26% | 1 | 1 | 0% | 3,572 | 2,964 | -17% | 0 | 0 | — |
▸case-10 Our enterprise web platform has four independent product teams working in the same React codebase, leading to deployment collisions. Analyze our organization context, compare micro-frontends against a modular SPA build pipeline, give a recommendation with rationale, and outline next steps. | pass→pass | 17,765 | 17,294 | -3% | 1 | 1 | 0% | 3,017 | 3,177 | +5% | 0 | 0 | — |
▸case-11 We receive telemetry data from 50,000 IoT devices and need to generate real-time alerting alongside historical reporting. Analyze our data velocity needs, compare streaming architectures against scheduled batch processing, recommend a pipeline approach with rationale, and list next steps. | pass→pass | 19,768 | 16,170 | -18% | 1 | 1 | 0% | 3,487 | 3,180 | -9% | 0 | 0 | — |
▸case-12 Our microservices backend exposes individual endpoints directly to external clients, leading to security inconsistencies and rate limiting challenges. Analyze our current setup, compare implementing a centralized API gateway against service-level controls, recommend a solution with rationale, and provide next steps. | pass→pass | 16,452 | 12,569 | -24% | 1 | 1 | 0% | 2,867 | 2,608 | -9% | 0 | 0 | — |
▸case-13 Our relational database search queries using SQL LIKE operations have become too slow as product catalog rows reached 10 million. Analyze the bottleneck, evaluate full-text search engines versus SQL query tuning, recommend an architecture with rationale, and list next steps. | pass→pass | 14,717 | 13,946 | -5% | 1 | 1 | 0% | 2,742 | 2,758 | +1% | 0 | 0 | — |
▸case-14 We are building a B2B SaaS platform and must decide between separate databases per tenant versus a shared database with tenant identifier columns. Analyze our security and scalability requirements, compare isolation models, recommend a multi-tenant strategy with rationale, and outline next steps. | pass→pass | 19,342 | 16,041 | -17% | 1 | 1 | 0% | 3,592 | 2,524 | -30% | 0 | 0 | — |
▸case-15 We need an event infrastructure to support message replay for audit logs as well as lightweight task queues for background job processing. Analyze our requirements, compare distributed commit log systems with traditional message queues, recommend a queueing architecture with rationale, and list next steps. | pass→pass | 19,441 | 18,450 | -5% | 1 | 1 | 0% | 3,358 | 3,494 | +4% | 0 | 0 | — |
▸case-16 Our software development team struggles with slow end-to-end integration tests holding up deployment pipelines for hours. Analyze our deployment bottleneck, compare testing strategies like parallelization, test splitting, and staging environments, recommend an architecture with rationale, and outline next steps. | pass→pass | 20,158 | 14,621 | -27% | 1 | 1 | 0% | 3,375 | 2,967 | -12% | 0 | 0 | — |
▸case-17 Our primary cloud region experienced an outage causing 4 hours of downtime. Analyze our reliability requirements, evaluate active-passive multi-region vs active-active multi-region replication, recommend a disaster recovery architecture with rationale, and provide next steps. | pass→pass | 19,252 | 15,341 | -20% | 1 | 1 | 0% | 3,666 | 3,138 | -14% | 0 | 0 | — |
▸case-18 Our React mobile app is suffering from re-render performance drops because all component state is stored in a single centralized global store. Analyze our state management bottleneck, compare global state stores against localized state and context, recommend an architecture with rationale, and list next steps. | pass→pass | 19,109 | 16,409 | -14% | 1 | 1 | 0% | 3,244 | 3,093 | -5% | 0 | 0 | — |
▸case-19 Users upload large video files through our Node.js application server, causing severe memory spikes and process timeouts. Analyze our media ingestion pipeline, compare direct object storage presigned uploads against server-proxied uploads, recommend a pipeline with rationale, and outline next steps. | fail→pass | 23,414 | 14,183 | -39% | 1 | 1 | 0% | 4,060 | 2,776 | -32% | 0 | 0 | — |
▸case-20 I am getting a SyntaxError: Unexpected token '}' at line 42 in my JavaScript function. Here is the code snippet: `function calculateTotal(items) { return items.reduce((a, b) => a + b, 0); }}`. Can you fix this error for me? | pass→fail | 2,472 | 4,766 | +93% | 1 | 1 | 0% | 516 | 1,261 | +144% | 0 | 0 | — |
▸case-21 Write a Python function named `is_palindrome` that takes a string, ignores non-alphanumeric characters and casing, and returns True if it reads the same forward and backward. | pass→fail | 4,198 | 8,209 | +96% | 1 | 1 | 0% | 908 | 1,996 | +120% | 0 | 0 | — |
▸case-22 Write a SQL query for PostgreSQL that finds the top 5 customers by total spending from an `orders` table containing `customer_id` and `amount` columns. | pass→fail | 2,501 | 7,408 | +196% | 1 | 1 | 0% | 567 | 1,762 | +211% | 0 | 0 | — |