▸case-01 Our sales team claims the CRM is too slow and asked us to rebuild the backend database right away. I suspect jumping straight into a database overhaul is premature, but I need to figure out what's really driving this request. Can you help me unpack this request and frame the actual core problem before we take any action? | fail→pass | 16,518 | 11,509 | -30% | 1 | 1 | 0% | 2,422 | 1,516 | -37% | 0 | 0 | — |
▸case-02 A client just submitted a ticket asking for a custom real-time analytics dashboard with predictive AI models, but they haven't shared what operational bottleneck they are facing or why they need it. Can you help me deconstruct this vague requirement and generate different ways to frame the problem so I can align with them? | fail→fail | 15,301 | 11,550 | -25% | 1 | 1 | 0% | 2,418 | 2,061 | -15% | 0 | 0 | — |
▸case-03 Our product management team wants to implement an automated PDF export feature because users complained about reporting being tedious. I'm worried we are leaping to a specific feature fix without understanding the actual user friction. Please walk me through analyzing and reframing this issue into a clear problem definition. | pass→pass | 15,378 | 12,773 | -17% | 1 | 1 | 0% | 2,222 | 2,055 | -8% | 0 | 0 | — |
▸case-04 Our executive sponsor wants us to build a dedicated native React Native mobile app for internal expense approvals because managers take too long to approve expenses. I'm tempted to immediately start wireframing the native app UI. How should we analyze this request before starting development? | fail→pass | 18,610 | 10,221 | -45% | 1 | 1 | 0% | 2,666 | 1,790 | -33% | 0 | 0 | — |
▸case-05 The finance team requested that we migrate all legacy reporting tables to Snowflake by next month because reports are running slow. Moving everything is a 6-month effort, but migrating just end-of-month financial summary tables would take 2 weeks. Help me frame this problem before we commit resources. | fail→pass | 14,456 | 8,500 | -41% | 1 | 1 | 0% | 2,244 | 1,539 | -31% | 0 | 0 | — |
▸case-06 The lead engineer told our product team: 'We must refactor the monolith into an event-driven CQRS GraphQL microservice architecture to solve latency.' Everyone wants to start coding. Frame this request into plain business language so stakeholders can understand the core issue. | fail→pass | 14,582 | 12,119 | -17% | 1 | 1 | 0% | 2,204 | 1,530 | -31% | 0 | 0 | — |
▸case-07 A client requested a complete redesign of the user onboarding flow because conversion dropped by 10%. I have 10 different questions about metrics, design, tech stack, and user cohorts. What clarifying questions should I ask them first to frame the problem without overwhelming them? | fail→pass | 12,320 | 10,608 | -14% | 1 | 1 | 0% | 1,771 | 1,783 | +1% | 0 | 0 | — |
▸case-08 The marketing director insists we must add LLM-powered vector search to our e-commerce site because customers aren't finding items. Frame this problem for our discovery meeting, making sure to surface what we are taking for granted. | pass→pass | 17,018 | 9,961 | -41% | 1 | 1 | 0% | 2,399 | 1,689 | -30% | 0 | 0 | — |
▸case-09 We explored three ways to frame our customer churn issue, and the team selected the framing focused on high initial setup friction rather than missing features. Formulate the final confirmation statement to send to the executive sponsor. | pass→pass | 8,900 | 3,822 | -57% | 1 | 1 | 0% | 1,341 | 763 | -43% | 0 | 0 | — |
▸case-10 Our support head asked us to implement an AI chatbot immediately to reduce ticket volume. I'm ready to select a chatbot vendor today. What should I do first before picking a vendor? | fail→pass | 16,087 | 8,025 | -50% | 1 | 1 | 0% | 2,380 | 1,431 | -40% | 0 | 0 | — |
▸case-11 A key enterprise customer asked for 'custom RBAC role creation with granular field-level permission matrices.' We are tempted to build a complex permission builder immediately. How should we frame this requirement before engineering begins? | pass→pass | 17,557 | 9,461 | -46% | 1 | 1 | 0% | 2,463 | 1,607 | -35% | 0 | 0 | — |
▸case-12 Our VP of Sales wants us to add a 1-click checkout button on every product page because cart abandonment is 68%. Help me unpack this solution request and reframe the core problem. | pass→pass | 16,773 | 12,202 | -27% | 1 | 1 | 0% | 2,432 | 2,010 | -17% | 0 | 0 | — |
▸case-13 Operations asked us to write a Python script to auto-generate weekly CSV reports because manual data entry takes them 4 hours every Monday. We are about to write the script. Help us frame the underlying problem first. | pass→fail | 14,634 | 10,345 | -29% | 1 | 1 | 0% | 2,078 | 1,661 | -20% | 0 | 0 | — |
▸case-14 Community feedback heavily demands 'Dark Mode' for our desktop web app. Developers want to start building CSS themes right away. How do we frame this request to ensure we understand the underlying need? | pass→pass | 14,513 | 8,626 | -41% | 1 | 1 | 0% | 1,983 | 1,526 | -23% | 0 | 0 | — |
▸case-15 Users are complaining that they miss important team updates, so engineering proposed building a full real-time notification center with push notifications and email digests. Frame this request to reveal whether simpler or alternative approaches exist. | pass→pass | 15,353 | 8,961 | -42% | 1 | 1 | 0% | 2,107 | 1,590 | -25% | 0 | 0 | — |
▸case-16 Customer success reported that clients are cancelling subscriptions during month 3, and asked us to build an automated discount popup at day 75. Help us structure our initial problem-framing inquiry. | pass→pass | 16,338 | 7,490 | -54% | 1 | 1 | 0% | 2,376 | 1,267 | -47% | 0 | 0 | — |
▸case-17 A client requested 'asynchronous webhook pub/sub callbacks for real-time order status mutation events'. Frame this request for our non-technical product review committee. | fail→pass | 11,172 | 7,622 | -32% | 1 | 1 | 0% | 1,686 | 1,253 | -26% | 0 | 0 | — |
▸case-18 Our growth team requested sending daily push notifications to all inactive users to boost daily active users (DAU). Before launching push campaigns, how should we reframe this problem? | fail→pass | 14,207 | 9,352 | -34% | 1 | 1 | 0% | 2,073 | 1,694 | -18% | 0 | 0 | — |
▸case-19 Engineering proposed rewriting the billing module from scratch to fix occasional rounding errors on international invoices. Rewriting takes 4 months; fixing the floating-point calculation takes 1 day. Frame this situation for product alignment. | fail→pass | 11,023 | 9,493 | -14% | 1 | 1 | 0% | 1,625 | 1,639 | +1% | 0 | 0 | — |
▸case-20 Our primary PostgreSQL cluster crashed for 45 minutes yesterday due to connection pool exhaustion during a marketing promotion. The incident root cause is known and documented. Please generate a standard blameless post-mortem report outlining the timeline, root cause, short-term fix, and long-term remediation items. | pass→fail | 19,357 | 9,918 | -49% | 1 | 1 | 0% | 2,944 | 1,744 | -41% | 0 | 0 | — |
▸case-21 We have already aligned on the problem statement: enterprise clients need SAML 2.0 single sign-on to comply with security audits. Please write a detailed Product Requirement Document (PRD) specifying user stories, functional requirements, API payloads, and acceptance criteria for SAML 2.0 integration. | pass→fail | 29,923 | 11,734 | -61% | 1 | 1 | 0% | 5,581 | 2,026 | -64% | 0 | 0 | — |
▸case-22 We have five fully scoped, clearly defined user features with estimated reach, impact, confidence, and effort scores. Calculate the RICE score for each feature and rank them in priority order based strictly on the scores provided. | fail→fail | 6,597 | 9,378 | +42% | 1 | 1 | 0% | 1,046 | 1,678 | +60% | 0 | 0 | — |