▸case-01 I am building a new retention dashboard in Tableau for our executive team and need a comprehensive technical design spec. Could you create a complete BI dashboard specification covering key SaaS metrics, segment dimensions, chart layouts, alerting mechanisms, and context annotation workflows? | fail→fail | 35,936 | 39,679 | +10% | 1 | 1 | 0% | 5,767 | 6,430 | +11% | 0 | 0 | — |
▸case-02 Design a dimensional model for our Snowflake retention data mart. A team member suggested only partitioning cohorts by subscription plan tier and geographic region to keep the schema minimal. Define the complete set of required segment dimensions for holistic retention tracking. | fail→fail | 24,883 | 21,059 | -15% | 1 | 1 | 0% | 3,930 | 3,444 | -12% | 0 | 0 | — |
▸case-03 We are preparing our quarterly SaaS metrics documentation for executive lifecycle reviews. Our team currently measures overall revenue retention, but debate exists on how to handle expansion and logo counts. Provide the complete standard metric definitions required for retention reviews. | fail→fail | 19,266 | 15,417 | -20% | 1 | 1 | 0% | 3,136 | 2,514 | -20% | 0 | 0 | — |
▸case-04 Our UI/UX designer is drafting wireframes for a new retention monitoring tool and proposed putting all metric trend lines into a simple grid of line charts. Specify the required visualization layout components needed for effective drillable retention dashboards. | fail→pass | 16,459 | 15,424 | -6% | 1 | 1 | 0% | 2,496 | 2,591 | +4% | 0 | 0 | — |
▸case-05 We want to configure automated notifications in our operational telemetry pipeline whenever account retention drops. A developer proposed sending a daily summary PDF email regardless of metric changes. Define the operational requirements for an automated retention alerting layer. | pass→pass | 15,230 | 19,066 | +25% | 1 | 1 | 0% | 2,272 | 3,010 | +32% | 0 | 0 | — |
▸case-06 Specify the database table schema and workflow requirements for a context annotation feature on executive retention reports. The product manager suggested only storing timestamp, chart_id, and free-text comment. | fail→pass | 16,914 | 19,599 | +16% | 1 | 1 | 0% | 2,688 | 3,315 | +23% | 0 | 0 | — |
▸case-07 When presenting account retention across enterprise vs mid-market tiers in Looker, business analysts often mix raw ARR amounts with account headcounts on the same axis, causing confusion. What core guidelines should be followed to prevent metric misinterpretation? | fail→fail | 16,227 | 14,951 | -8% | 1 | 1 | 0% | 2,385 | 2,422 | +2% | 0 | 0 | — |
▸case-08 Our customer success leadership team complains that current retention charts show declining usage signals but nobody takes action. How should chart visualizations be structured to ensure immediate operational response? | pass→pass | 18,364 | 18,996 | +3% | 1 | 1 | 0% | 2,438 | 3,011 | +24% | 0 | 0 | — |
▸case-09 When retention signals detect an account entering high churn risk, what secondary mapping system should be paired with the retention dashboard to record and track triggered intervention steps? | pass→pass | 12,911 | 3,733 | -71% | 1 | 1 | 0% | 1,765 | 682 | -61% | 0 | 0 | — |
▸case-10 Draft an architectural template for a BI dashboard specification document intended for analytics engineers building retention dashboards. Ensure all standard administrative and technical governance sections are covered. | pass→pass | 24,402 | 26,805 | +10% | 1 | 1 | 0% | 3,826 | 4,574 | +20% | 0 | 0 | — |
▸case-11 Create an operational template for communicating regular retention performance updates to department heads without requiring them to log into Tableau directly every morning. | pass→pass | 28,259 | 17,573 | -38% | 1 | 1 | 0% | 2,498 | 2,830 | +13% | 0 | 0 | — |
▸case-12 Design a structured JSON schema for payload notifications sent when account metrics breach risk limits. The initial draft only contained account_id and current_score. | fail→pass | 17,569 | 17,686 | +1% | 1 | 1 | 0% | 3,159 | 3,478 | +10% | 0 | 0 | — |
▸case-13 Our VP of Customer Success wants to know the primary operational use cases suitable for automated retention review systems. List the core business scenarios where automated retention tracking should be deployed. | fail→pass | 15,425 | 14,774 | -4% | 1 | 1 | 0% | 2,345 | 2,359 | +1% | 0 | 0 | — |
▸case-14 We are evaluating whether our recent customer save plays and adoption drives were effective. How should retention reporting be applied to evaluate these short-term initiatives? | pass→pass | 17,602 | 16,804 | -5% | 1 | 1 | 0% | 2,584 | 2,591 | +0% | 0 | 0 | — |
▸case-15 How should BI dashboards be structured so that individual product line managers and regional directors can investigate root causes of churn within their specific business units? | pass→pass | 21,606 | 24,410 | +13% | 1 | 1 | 0% | 3,018 | 3,739 | +24% | 0 | 0 | — |
▸case-16 When displaying net revenue retention dynamics over a fiscal quarter, which specific chart format best illustrates the net movement between starting ARR, expansions, contractions, churn, and ending ARR? | pass→pass | 9,384 | 8,019 | -15% | 1 | 1 | 0% | 1,355 | 1,350 | -0% | 0 | 0 | — |
▸case-17 What specific visualization format should be used on a retention dashboard to display customer group retention percentages over elapsed month-indexed intervals? | pass→pass | 9,909 | 7,463 | -25% | 1 | 1 | 0% | 1,571 | 1,382 | -12% | 0 | 0 | — |
▸case-18 What high-level visual components should sit at the top of an executive retention dashboard to convey overall program health at a glance before users dive into deep breakdowns? | pass→pass | 18,048 | 10,928 | -39% | 1 | 1 | 0% | 2,597 | 1,771 | -32% | 0 | 0 | — |
▸case-19 In addition to customer demographic traits like industry or persona, which product-level dimensions must be integrated into retention cohort analysis schemas? | pass→fail | 17,204 | 13,469 | -22% | 1 | 1 | 0% | 2,523 | 2,083 | -17% | 0 | 0 | — |
▸case-20 We are designing a post-interaction CSAT survey for our technical support desk in Qualtrics. What question types and rating scale lengths should we select to maximize response rates? | pass→pass | 14,849 | 15,573 | +5% | 1 | 1 | 0% | 2,239 | 2,551 | +14% | 0 | 0 | — |
▸case-21 Write a dbt SQL model that calculates a 30-day rolling average of API calls per user from raw segment event logs stored in Snowflake. | pass→pass | 19,615 | 16,396 | -16% | 1 | 1 | 0% | 3,182 | 2,881 | -9% | 0 | 0 | — |
▸case-22 Outline a incentive compensation structure for enterprise account executives closing multi-year software contracts with upfront annual billing. | pass→pass | 19,084 | 22,389 | +17% | 1 | 1 | 0% | 3,094 | 3,940 | +27% | 0 | 0 | — |