▸case-01 I need to process an email thread from our product launch update. Here is the text:
Subject: Q3 App Launch Readiness
From: Mark (VP Product)
Body: 'Hey Alex, we need to make sure the API staging environment is ready before Friday. Can you also ask Sam to check the database migration scripts? I'll review the security audit tomorrow. Also, someone should handle the analytics integration if we have time. Alex mentioned earlier that he would update the API documentation by Wednesday.'
My name is Alex, and my role is Lead Developer. I track my action items in Jira. Please convert this email thread into a complete task breakdown relative to my role. Group the actionable points into my direct tasks, outbound commitments I declared, items delegated to or owned by others, and any vague asks that need clarifying questions. Make sure every entry includes exact quote references from the source email, assigned owners with actions and target completion dates, clear finish criteria showing when it's done, and draft a brief confirmation email I can send back to Mark and the team to lock in responsibilities. | fail→fail | 14,399 | 10,675 | -26% | 1 | 1 | 0% | 2,923 | 3,012 | +3% | 0 | 0 | — |
▸case-02 Please help me organize this vendor negotiation thread into an actionable task digest:
Subject: Office Relocation Vendor Contracts
From: Rachel (COO)
Body: 'Sarah, please review the logistics contract and negotiate the delivery terms with Apex Moving by Tuesday. Marcus should handle the IT cabling vendor quote. Sarah, didn't you mention you'd send the updated floor plan to HR today? Also, we need to deal with the storage facility situation soon. Marcus, please confirm if you can take care of the access badges.'
I am Sarah, the Operations Manager, and we use Asana for task tracking. Process this thread relative to my role. Categorize the tasks into my direct responsibilities, promises I made in the thread, tasks belonging to others, and ambiguous requests that need a follow-up question. For every item, provide quoted source lines, owner-assigned actions with deadlines, explicit observable completion tests, and draft a quick reply message I can post back to Rachel to confirm deadlines and assignments. | pass→pass | 11,056 | 8,815 | -20% | 1 | 1 | 0% | 2,305 | 2,802 | +22% | 0 | 0 | — |
▸case-03 I just received a planning email from our Director of Marketing and need to turn it into an actionable work plan:
Subject: Q4 Campaign Rollout Plan
From: Elena (Marketing Director)
Recipients: David, Priya, Tom
Body: 'David, could you draft the copy for the landing page by Nov 10? Priya is taking care of the social media graphics. David, you said yesterday that you would prepare the budget deck for finance by Friday. Also, can someone oversee the influencer outreach initiative? Tom should get us the email template designs.'
I am David (Marketing Lead), and my team manages tasks in Trello. Extract the work from this thread specifically for my perspective. Show my direct to-dos, the commitments I made earlier, items handled by teammates, and unclear requests requiring clarification. Include exact quote citations for each item, who owns what with target dates, observable done criteria, and a concise draft response to Elena confirming who is doing what and when. | fail→pass | 9,214 | 9,514 | +3% | 1 | 1 | 0% | 2,023 | 2,810 | +39% | 0 | 0 | — |
▸case-04 I have a thread where three team members agree to meet next Tuesday at 2 PM EST for a post-mortem review. Can you send out Google Calendar invites to all participants and update their calendar statuses? | pass→fail | 6,266 | 5,303 | -15% | 1 | 1 | 0% | 930 | 1,757 | +89% | 0 | 0 | — |
▸case-05 Here is a transcript from our engineering sync meeting: 'John presented the new architecture. Mary raised concerns about memory leaks. We agreed to move the release to Friday.' Please write a high-level executive summary narrative of the meeting for our newsletter. | pass→pass | 4,205 | 9,681 | +130% | 1 | 1 | 0% | 698 | 2,439 | +249% | 0 | 0 | — |
▸case-06 I am writing an end-of-week status report email to our client regarding the website redesign project. Here are our updates: frontend is 80% done, backend deployment succeeded, QA started today. Please write a polished status report email addressed to the client. | pass→pass | 8,112 | 11,947 | +47% | 1 | 1 | 0% | 1,373 | 3,020 | +120% | 0 | 0 | — |
▸case-07 Convert this email from Sarah to a task list: 'Hey Jim, please review the vendor agreement before 5 PM.' Many assistants paraphrase this as 'Jim needs to look over the contract'. How should the source reference be formatted? | fail→pass | 8,344 | 5,748 | -31% | 1 | 1 | 0% | 1,313 | 1,781 | +36% | 0 | 0 | — |
▸case-08 In an email thread, the VP writes: 'We need to handle the cloud database situation by Friday.' Should I guess that 'handle' means upgrading instances, or should I create a task 'Jim upgrades database by Friday'? | pass→pass | 8,361 | 7,607 | -9% | 1 | 1 | 0% | 1,339 | 2,187 | +63% | 0 | 0 | — |
▸case-09 An email thread sent to me (Lead QA) says: 'Someone should test the payment gateway before launch on Friday.' How should this be handled regarding task ownership and deadlines? | fail→pass | 11,291 | 9,403 | -17% | 1 | 1 | 0% | 1,691 | 2,311 | +37% | 0 | 0 | — |
▸case-10 In an email thread, my boss asked me 'Can you send the draft report by Wednesday?' and earlier in the thread I stated 'I'll email the client intake notes today by 3 PM.' Model defaults often lump all user tasks together. How should my self-declared commitment be categorized relative to inbound requests? | fail→pass | 12,346 | 8,035 | -35% | 1 | 1 | 0% | 1,885 | 2,305 | +22% | 0 | 0 | — |
▸case-11 An email from Tom reads: 'Please update the security policy document when you get a chance.' Common tools output 'Deadline: None' or 'TBD'. How should the target completion date be handled? | pass→pass | 10,244 | 6,564 | -36% | 1 | 1 | 0% | 1,699 | 2,068 | +22% | 0 | 0 | — |
▸case-12 When extracting a task from the email 'Please upload the updated brand guidelines to the shared drive by Tuesday', what should be specified as the done-test? | pass→pass | 5,307 | 2,802 | -47% | 1 | 1 | 0% | 837 | 1,425 | +70% | 0 | 0 | — |
▸case-13 In an email thread, I am delegating the API testing to Carlos: 'Carlos, please run the API regression suite by Thursday.' How should this delegation be reflected in the confirming reply sent to the team? | fail→pass | 8,885 | 6,322 | -29% | 1 | 1 | 0% | 1,323 | 1,937 | +46% | 0 | 0 | — |
▸case-14 Extract tasks from this thread: 'Subject: Design Review. From: Art Director. Body: Mark, please finish the logo specs by Monday. I'll send the typography guide today. Someone needs to fix the color palette. Mark: I'll prepare the client deck by Friday.' I am Mark. I want to organize all categories without dropping any section types. | fail→pass | 5,359 | 6,126 | +14% | 1 | 1 | 0% | 1,087 | 2,226 | +105% | 0 | 0 | — |
▸case-15 In an email, the manager asks: 'Could you find out why the customer onboarding drop-off increased in Q2?' Is this a question to answer in prose or an actionable task? | fail→pass | 7,364 | 6,830 | -7% | 1 | 1 | 0% | 1,040 | 2,077 | +100% | 0 | 0 | — |
▸case-16 An email thread between Alice (Product Manager), Bob (Engineer), and Charlie (Designer) contains tasks for all three. If Bob processes this thread relative to his role, how should Alice's task 'Alice will write the PRD by Tuesday' be categorized? | pass→pass | 5,944 | 3,860 | -35% | 1 | 1 | 0% | 889 | 1,660 | +87% | 0 | 0 | — |
▸case-17 After extracting tasks from a complex project thread, what critical components must the draft confirming reply contain to lock in responsibilities across team members? | fail→pass | 13,314 | 11,138 | -16% | 1 | 1 | 0% | 2,120 | 2,646 | +25% | 0 | 0 | — |
▸case-18 An email states: 'Can you secure vendor quotes for office catering before Friday?' An automated tool rewrote the task as 'Research food options'. Why does this violate extraction rules and how should the task description be formatted? | fail→pass | 8,456 | 7,199 | -15% | 1 | 1 | 0% | 1,317 | 2,113 | +60% | 0 | 0 | — |
▸case-19 An email paragraph says: 'Please send the Q3 financial audit to accounting by Wednesday EOD, update the expense sheet, and confirm with Sarah that the budget was approved.' How should these asks be extracted? | fail→pass | 7,328 | 7,270 | -1% | 1 | 1 | 0% | 1,340 | 2,120 | +58% | 0 | 0 | — |
▸case-20 An email thread contains the line: 'We need to deal with the vendor situation ASAP.' How should this line be processed into output format? | fail→fail | 6,985 | 8,154 | +17% | 1 | 1 | 0% | 1,210 | 2,163 | +79% | 0 | 0 | — |
▸case-21 In an email thread, Manager says 'Please review the PR by tomorrow' and Developer replies 'I'll fix the login bug by 2 PM today'. Process this thread for Developer. | fail→pass | 6,474 | 5,962 | -8% | 1 | 1 | 0% | 1,059 | 2,020 | +91% | 0 | 0 | — |
▸case-22 An email says: 'Please inform the executive committee about the launch delay by EOD.' What is an appropriate observable done-test for this communication task? | pass→pass | 7,598 | 5,017 | -34% | 1 | 1 | 0% | 1,136 | 1,675 | +47% | 0 | 0 | — |
▸case-23 If an email thread specifies an action without any date ('Sam needs to submit the expense report'), how must the deadline column represent this date? | fail→pass | 6,663 | 3,963 | -41% | 1 | 1 | 0% | 955 | 1,538 | +61% | 0 | 0 | — |