▸case-01 We're launching a new feature next month and need to create a feedback collection form for our beta users. Can you build a comprehensive survey strategy document that outlines the goals, target user selection rules, question distribution structure, and branching flow before we start building it in our tool? | fail→fail | 22,701 | 44,085 | +94% | 1 | 1 | 0% | 3,534 | 2,817 | -20% | 0 | 0 | — |
▸case-02 I need to equip my customer success team with a standard framework and a categorized repository of survey questions for measuring product adoption and user satisfaction. Please generate a complete design blueprint along with structured question sets categorized by outcome focus. | fail→fail | 25,537 | 29,026 | +14% | 1 | 1 | 0% | 4,010 | 4,593 | +15% | 0 | 0 | — |
▸case-03 We want to send an un-incentivized B2B customer feedback questionnaire to busy C-level executives. We drafted a 15-minute, 35-question survey to cover every department's questions thoroughly. Please refine our survey plan. | pass→pass | 14,332 | 17,936 | +25% | 1 | 1 | 0% | 2,095 | 2,748 | +31% | 0 | 0 | — |
▸case-04 We drafted this core metric question: 'How satisfied are you with our software's speed and pricing transparency?' Evaluate this question structure for our product satisfaction survey. | pass→pass | 10,954 | 7,563 | -31% | 1 | 1 | 0% | 1,703 | 1,435 | -16% | 0 | 0 | — |
▸case-05 In our multi-choice feature request survey, we list 8 product features in alphabetical order. Respondents keep selecting the top 2 features. How should we structure the choice display to fix this ordering bias? | pass→pass | 12,588 | 15,139 | +20% | 1 | 1 | 0% | 1,888 | 2,433 | +29% | 0 | 0 | — |
▸case-06 We have finalized our quarterly customer churn survey draft and logic. We want to publish it immediately from our design tool. What operational step should we take before hitting launch? | fail→fail | 10,245 | 8,085 | -21% | 1 | 1 | 0% | 1,444 | 1,384 | -4% | 0 | 0 | — |
▸case-07 We are designing a survey targeted specifically at enterprise admin users who manage over 500 licenses, but we are blasting the email link to our entire broad subscriber mailing list. How do we ensure our question architecture screens out non-qualifying respondents? | pass→pass | 17,276 | 12,873 | -25% | 1 | 1 | 0% | 2,632 | 2,190 | -17% | 0 | 0 | — |
▸case-08 Our survey software guarantees responsive CSS layouts out of the box. Do we need to include device testing in our pre-launch QA script, or is automatic rendering sufficient? | pass→pass | 13,957 | 10,529 | -25% | 1 | 1 | 0% | 1,912 | 1,682 | -12% | 0 | 0 | — |
▸case-09 We are building a standard question repository for product managers who want respondent feedback on unreleased features and future feature priorities. Which standard category should these questions belong to? | fail→pass | 10,985 | 5,566 | -49% | 1 | 1 | 0% | 1,595 | 979 | -39% | 0 | 0 | — |
▸case-10 We want to sample our active user base for a new feature feedback study. We plan to send the invite to all users who logged in this week, including users currently working through high-severity support tickets. How should we adjust our audience sampling design? | pass→pass | 14,064 | 11,939 | -15% | 1 | 1 | 0% | 1,997 | 1,890 | -5% | 0 | 0 | — |
▸case-11 Our research team proposed a 20-question survey consisting entirely of 1-to-5 Likert scale rating grids to keep data collection clean. Review this question architecture. | pass→pass | 17,834 | 16,296 | -9% | 1 | 1 | 0% | 2,346 | 2,530 | +8% | 0 | 0 | — |
▸case-12 We are drafting a survey about our onboarding flow. Our team agreed on the goal 'understand user feelings about onboarding'. Is this objective definition complete for a survey brief? | fail→pass | 11,420 | 7,771 | -32% | 1 | 1 | 0% | 1,632 | 1,388 | -15% | 0 | 0 | — |
▸case-13 What artifact should we provide to internal business reviewers to confirm survey text, logic branching, and localized language translations before publishing? | fail→pass | 15,403 | 5,712 | -63% | 1 | 1 | 0% | 2,131 | 1,068 | -50% | 0 | 0 | — |
▸case-14 When defining the logic and branching architecture for a new survey, how should completion time expectations be established? | pass→pass | 18,410 | 14,431 | -22% | 1 | 1 | 0% | 2,414 | 2,219 | -8% | 0 | 0 | — |
▸case-15 Our product marketing team needs standardized survey questions to evaluate customer willingness-to-pay and tier packaging acceptance. Which section of the survey question bank covers this? | pass→pass | 11,937 | 6,230 | -48% | 1 | 1 | 0% | 1,810 | 1,125 | -38% | 0 | 0 | — |
▸case-16 We are measuring how frequently team leads utilize our new workflow automation module after day 30. Which category of the survey question bank should we draw from? | pass→pass | 9,708 | 5,653 | -42% | 1 | 1 | 0% | 1,458 | 1,062 | -27% | 0 | 0 | — |
▸case-17 We are worried that automated bots or unattentive respondents will fill out our long-form research survey. What specific question architecture element guards against low-quality garbage submissions? | pass→pass | 17,055 | 12,516 | -27% | 1 | 1 | 0% | 2,333 | 1,926 | -17% | 0 | 0 | — |
▸case-18 Our existing customer health survey program has seen response rates plummet and data quality degrade over the past 3 quarters. What operational procedure should we run? | pass→pass | 15,339 | 14,811 | -3% | 1 | 1 | 0% | 2,217 | 2,358 | +6% | 0 | 0 | — |
▸case-19 Create a survey brief template for our upcoming annual user conference feedback initiative. What key components must be included in the brief header? | pass→pass | 17,108 | 15,881 | -7% | 1 | 1 | 0% | 2,568 | 2,527 | -2% | 0 | 0 | — |
▸case-20 We collected 1,200 response rows from our recent customer survey in CSV format. Can you run a statistically significant Chi-Square test of independence and generate a regression analysis on column C vs column F in Python? | pass→pass | 13,994 | 15,477 | +11% | 1 | 1 | 0% | 2,551 | 3,030 | +19% | 0 | 0 | — |
▸case-21 Our survey invitation emails are bouncing and going to the spam folder. Can you help us configure SPF, DKIM, and DMARC DNS records for our email domain on Cloudflare? | pass→pass | 12,666 | 13,125 | +4% | 1 | 1 | 0% | 2,139 | 2,497 | +17% | 0 | 0 | — |
▸case-22 Write a Zapier webhook handler function in Node.js that listens for submission payloads from Typeform and writes contact properties directly into the HubSpot CRM API. | pass→pass | 19,566 | 16,052 | -18% | 1 | 1 | 0% | 3,508 | 3,109 | -11% | 0 | 0 | — |