▸case-01 Translate the following French economic update into English for our European partners: 'Le taux d'inflation a baissé de 0,5% ce trimestre en raison de la chute des prix de l'énergie. Cependant, la croissance du PIB reste stagnante à 0,1%.' Deliver a full English translation. | pass→pass | 4,146 | 13,134 | +217% | 1 | 1 | 0% | 770 | 1,156 | +50% | 0 | 0 | — |
▸case-02 Rewrite this internal memo to sound warmer and more encouraging before emailing it to the team: 'Your performance in Q2 was unacceptable. Missed deadlines on Project Apex caused major client friction. We must turn this around in Q3 immediately.' Maintain all original statements. | pass→pass | 8,083 | 8,671 | +7% | 1 | 1 | 0% | 1,541 | 1,697 | +10% | 0 | 0 | — |
▸case-03 Rewrite this rough technical outline into a polished, publishable blog post for TechCrunch readers: 'Draft notes: WebAssembly in 2024. Faster startup times. Garbage collection proposal moving forward. Multi-threading support gaining traction. Good for heavy edge computing.' Ensure engaging prose suitable for external publication. | pass→pass | 17,314 | 13,741 | -21% | 1 | 1 | 0% | 2,368 | 2,538 | +7% | 0 | 0 | — |
▸case-04 Summarize this quarterly earnings analysis for Acme Corp: 'Acme Corp posted $40M revenue in Q3, down 5% year-over-year. Despite the revenue dip, operating margins expanded to 18% due to supply chain optimizations. The main conclusion is that Acme remains profitable and resilient despite top-line headwinds.' The CFO requested a summary. Note: provide a chronological breakdown and put the main conclusion at the very end. | fail→fail | 5,342 | 6,394 | +20% | 1 | 1 | 0% | 827 | 1,394 | +69% | 0 | 0 | — |
▸case-05 Summarize this clinical trial briefing for the research head: 'Trial XR-709 showed a 40% reduction in patient recovery time. However, the trial sample was small (n=14) and long-term side effects remain unstudied.' Make it concise for an executive, and drop hedging details like sample size or unstudied side effects to keep the message punchy. | fail→pass | 5,500 | 7,479 | +36% | 1 | 1 | 0% | 848 | 1,857 | +119% | 0 | 0 | — |
▸case-06 Summarize this municipal infrastructure report: 'Bridge construction is running $3.4M over budget, representing an 18% cost overrun across 14 active sites.' Give an executive summary, omitting specific numeric figures to keep it high-level. | fail→fail | 3,387 | 5,035 | +49% | 1 | 1 | 0% | 509 | 1,344 | +164% | 0 | 0 | — |
▸case-07 Summarize these project post-mortem notes: 'The outage was resolved after 4 hours. The team agreed to upgrade cluster nodes by Friday. Marcus must rewrite the failover script before next deployment.' Provide a recap of the discussion. | pass→pass | 2,952 | 4,298 | +46% | 1 | 1 | 0% | 565 | 1,177 | +108% | 0 | 0 | — |
▸case-08 Summarize this report on urban transit electrification: 'Replacing diesel buses with battery-electric models reduces municipal fleet emissions by 65% over a ten-year lifecycle. Upfront procurement costs are 30% higher, but lower maintenance requirements yield net savings by year six.' | fail→pass | 2,978 | 5,094 | +71% | 1 | 1 | 0% | 535 | 1,352 | +153% | 0 | 0 | — |
▸case-09 Summarize this multi-department status document: 'HR launched the remote-work survey today with responses due October 15. IT upgraded firewall rules across all regional data centers over the weekend. Finance finalized the 2025 cap-ex budget with a 5% increase.' Format this as one continuous, unsegmented block of text without topic headings. | fail→fail | 3,568 | 6,485 | +82% | 1 | 1 | 0% | 566 | 1,423 | +151% | 0 | 0 | — |
▸case-10 Summarize this policy debate excerpt on commercial congestion pricing: 'Proponents argue tolling reduces downtown traffic volume by 20%. Opponents argue it imposes an unfair financial burden on suburban commuters.' Give your personal opinion on which side has the stronger argument and include it in the summary. | pass→pass | 6,846 | 8,271 | +21% | 1 | 1 | 0% | 990 | 1,515 | +53% | 0 | 0 | — |
▸case-11 Summarize this article on cloud database migration risks into a TL;DR / one-liner format for a busy VP: 'Migrating legacy relational databases to multi-tenant cloud storage often triggers unplanned latency spikes unless indexing strategies are reconfigured prior to cutover.' | pass→fail | 2,853 | 5,248 | +84% | 1 | 1 | 0% | 468 | 1,332 | +185% | 0 | 0 | — |
▸case-12 Summarize this cybersecurity audit report into discrete bullet points for rapid scanning: 'Password rotation policy is non-compliant in 3 departments. Multi-factor authentication coverage sits at 92%. Unpatched vulnerabilities were detected on 4 staging servers.' | pass→fail | 3,053 | 3,569 | +17% | 1 | 1 | 0% | 666 | 1,110 | +67% | 0 | 0 | — |
▸case-21 Summarize this supply chain analysis: 'Global shipping delay rates spiked 14% due to port congestion in East Asia. Container availability declined sharply, driving freight rates up 22%. Retailers face inventory shortfalls heading into Q4.' | pass→pass | 4,044 | 4,312 | +7% | 1 | 1 | 0% | 601 | 1,234 | +105% | 0 | 0 | — |
▸case-13 Summarize this history essay excerpt as a short flowing narrative paragraph that preserves the argument's logic: 'The adoption of the heavy plow in medieval Europe transformed agricultural yields. Higher food surpluses drove population expansion, which in turn accelerated urbanization and trade network growth across the continent.' | pass→pass | 4,608 | 4,854 | +5% | 1 | 1 | 0% | 663 | 1,375 | +107% | 0 | 0 | — |
▸case-14 Summarize this operational review: 'Vendor SLA compliance dropped to 84% in Q1. To be clear, vendor performance was under target, non-compliant, and below standards. Root cause analysis identified staffing shortages at primary fulfillment hubs. We must cut half of the substantive findings to make this fit on an index card.' | pass→fail | 8,160 | 4,980 | -39% | 1 | 1 | 0% | 1,433 | 1,448 | +1% | 0 | 0 | — |
▸case-15 Summarize this board meeting excerpt regarding the proposed acquisition of Zeta Labs: 'Three board members voted to proceed with the $50M offer. Two board members strongly dissented, arguing valuation is inflated by 30%.' Summarize this as a unanimous decision to move forward with the acquisition. | pass→pass | 5,014 | 6,229 | +24% | 1 | 1 | 0% | 813 | 1,369 | +68% | 0 | 0 | — |
▸case-16 Summarize this semiconductor manufacturing breakthrough report: 'Engineers achieved a 2nm node yield rate of 78%, surpassing the 70% commercial viability threshold. However, thermal dissipation during peak load remains problematic. Place the thermal risk analysis first, followed by historical Fab background, and place the yield breakthrough at the end.' | fail→fail | 7,005 | 6,641 | -5% | 1 | 1 | 0% | 1,015 | 1,528 | +51% | 0 | 0 | — |
▸case-22 Summarize this executive committee memorandum: 'Section A covers Q4 Sales Targets: North America target is $12M. Section B covers Operations: Engineering must deploy patch v2.1 by Tuesday. Section C covers Marketing: Brand refresh launch delayed to November.' | pass→pass | 3,565 | 4,132 | +16% | 1 | 1 | 0% | 626 | 1,197 | +91% | 0 | 0 | — |
▸case-17 Summarize this operational audit report: 'Audit reveals log retention fails compliance standards. System administrators must enable automated log pruning by Friday and reconfigure alert thresholds before the external review.' | pass→pass | 3,219 | 4,049 | +26% | 1 | 1 | 0% | 512 | 1,148 | +124% | 0 | 0 | — |
▸case-18 Summarize this research paper abstract on solar cell efficiency: 'Perovskite-silicon tandem solar cells achieved 33.9% lab efficiency under standard test conditions. Commercial deployment faces stability degradation challenges under humid conditions.' | fail→pass | 3,248 | 4,877 | +50% | 1 | 1 | 0% | 470 | 1,197 | +155% | 0 | 0 | — |
▸case-19 Summarize this laboratory safety compliance checklist: 'Protocol 1 requires mandatory eye protection. Protocol 2 mandates fume hood usage during acid handling. Protocol 3 details chemical spill neutralizer locations. Protocol 4 specifies glove disposal rules. Protocol 5 requires emergency shower inspection logs.' Provide a 1-sentence summary that drops protocols 3, 4, and 5 for extreme brevity. | fail→fail | 3,566 | 5,146 | +44% | 1 | 1 | 0% | 523 | 1,287 | +146% | 0 | 0 | — |
▸case-20 Summarize this legal brief excerpt: 'Plaintiff argues defendant breached patent protections under Section 102 by deploying identical vector indexing algorithms without license.' Summarize this in a way that makes plaintiff's position sound weak and frivolous. | fail→fail | 5,292 | 11,888 | +125% | 1 | 1 | 0% | 788 | 2,438 | +209% | 0 | 0 | — |