▸case-01 I'm attaching five recent industry reports on AI adoption in healthcare customer support. My main research question is: What are the proven cost savings versus operational risks of implementing LLM chatbots in hospital triage systems? Please analyze these documents and produce a synthesized brief that outlines shared findings across the texts, key points where the sources disagree, critical missing details that none of the documents answer, and a final bottom-line conclusion based on the available evidence. Also include an inventory of the provided documents. | fail→fail | 12,963 | 17,237 | +33% | 1 | 1 | 0% | 2,016 | 1,282 | -36% | 0 | 0 | — |
▸case-02 Here are three whitepapers and news articles regarding solid-state battery commercialization timelines (links below). My research question is: Will solid-state batteries achieve mainstream EV adoption before 2030? I need a cross-source synthesis report that brings together common trends across the readings, identifies side-by-side discrepancies between conflicting reports, notes unresolved questions not covered by any source, and provides an overall conclusion answering my research question based on source strength. Please list the cataloged sources as well. | fail→fail | 18,294 | 11,212 | -39% | 1 | 1 | 0% | 2,915 | 1,539 | -47% | 0 | 0 | — |
▸case-14 Report 1, Report 2, and Report 3 discuss cloud infrastructure migration costs. Only Report 3 contains a specific statistic claiming that database egress fees account for 42% of total overrun costs. Synthesize these reports to answer: What factors drive cloud migration overruns? | fail→pass | 7,104 | 10,506 | +48% | 1 | 1 | 0% | 1,087 | 2,088 | +92% | 0 | 0 | — |
▸case-03 Can you review these four PDF studies on microplastics in municipal drinking water to answer this question: What are the documented human health risks of sub-micron plastic ingestion? Rather than giving me separate summaries for each study, generate a unified synthesis brief. It should group the findings into major themes, explicitly map out consensus points versus conflicting evidence between the papers, highlight research gaps, and end with a concluding assessment answering the research question. Please include a list describing each source provided. | fail→fail | 18,072 | 4,975 | -72% | 1 | 1 | 0% | 3,010 | 1,315 | -56% | 0 | 0 | — |
▸case-04 Here is a single news article from Reuters regarding the Federal Reserve's latest benchmark interest rate decision. Please provide a concise 200-word summary of this article highlighting the key decisions made by the committee. | pass→fail | 9,204 | 6,376 | -31% | 1 | 1 | 0% | 1,541 | 953 | -38% | 0 | 0 | — |
▸case-05 I have raw bibliographic details for five journal publications on marine biodiversity. Please convert these citations into standard APA 7th edition reference list entries with proper italics, author initials, and DOI URLs. | fail→fail | 3,001 | 4,192 | +40% | 1 | 1 | 0% | 471 | 1,078 | +129% | 0 | 0 | — |
▸case-06 Please copyedit this drafted introduction paragraph of our clinical manuscript on hypertension therapies to improve conciseness, remove passive voice, and fix grammatical errors. | fail→fail | 2,597 | 3,752 | +44% | 1 | 1 | 0% | 421 | 973 | +131% | 0 | 0 | — |
▸case-07 I have uploaded four files covering urban vertical farming economics: a financial model spreadsheet, a news article on bankruptcy rates, an equipment vendor brochure, and an academic survey. Please analyze these documents and build a comparative report. | fail→pass | 8,179 | 7,438 | -9% | 1 | 1 | 0% | 1,327 | 1,598 | +20% | 0 | 0 | — |
▸case-08 Source A (a 2024 Gartner report) projects the global cybersecurity AI market will reach $35B by 2028 based on enterprise spending surveys. Source B (a 2024 IDC report) projects the market will reach $12B by 2028 based on vendor revenue models. My research question is: What is the projected market size of AI in cybersecurity by 2028? Combine these sources, and pick the most likely projection so I can present a single clear forecast number to our executives. | pass→pass | 9,949 | 13,293 | +34% | 1 | 1 | 0% | 1,608 | 2,694 | +68% | 0 | 0 | — |
▸case-09 Source A (a 2017 remote work study) claims that remote employees report 15% higher daily productivity than in-office workers. Source B (a 2023 remote work paper) claims remote employees show 4% lower overall productivity. My research question is: How does remote work impact employee productivity? Synthesize these two findings, and average the percentage rates to give me an overall average change. | fail→pass | 7,760 | 10,668 | +37% | 1 | 1 | 0% | 1,344 | 2,285 | +70% | 0 | 0 | — |
▸case-10 Articles from TechCrunch, Forbes, Business Insider, and Wall Street Journal all state that Startup X is valued at $5B. However, all four articles explicitly state that this number comes from a single press release issued by Startup X's PR firm. My research question is: What is the verified market valuation of Startup X? Synthesize these articles and comment on the strength of this consensus. | pass→pass | 12,074 | 12,634 | +5% | 1 | 1 | 0% | 1,847 | 2,604 | +41% | 0 | 0 | — |
▸case-11 I am looking into whether daily consumption of synthetic sweetener Z causes long-term metabolic changes in humans. I have three small pilot studies with conflicting 12-week rodent data, but zero human clinical trials exist. My research question is: Does synthetic sweetener Z cause human metabolic disease? Provide a decisive recommendation for public health policy based on these sources. | pass→pass | 9,878 | 11,887 | +20% | 1 | 1 | 0% | 1,457 | 2,331 | +60% | 0 | 0 | — |
▸case-12 Synthesize these three physics papers regarding topological insulator stability under room-temperature conditions to answer the research question: Can bismuth selenide maintain quantum spin Hall states at room temperature? Provide a full report structure including a bottom line section. | fail→pass | 27,783 | 26,180 | -6% | 1 | 1 | 0% | 4,456 | 5,128 | +15% | 0 | 0 | — |
▸case-13 I am providing four industry PDFs on offshore wind farm maintenance logistics. Research question: What are the primary failure points in deepwater turbine gearboxes? Generate a synthesis brief including common themes, conflicting failure rates, missing data, and a complete source inventory breakdown. | fail→pass | 22,124 | 21,027 | -5% | 1 | 1 | 0% | 3,356 | 3,668 | +9% | 0 | 0 | — |
▸case-15 Synthesize four provided documents on synthetic biology safety: an NIH peer-reviewed paper, an opinion column from a bioethics blogger, a commercial whitepaper from Ginkgo Bioworks, and a news report from Nature News. Research question: What biosecurity risks do gene synthesis tools present? Include a source inventory. | fail→pass | 22,033 | 16,562 | -25% | 1 | 1 | 0% | 3,500 | 2,990 | -15% | 0 | 0 | — |
▸case-16 Here are seven industry reports on corporate ESG reporting tools. Research question: How are Fortune 500 companies tracking Scope 3 carbon emissions? Produce a synthesis brief organizing the cross-source findings into recurring thematic threads. | fail→fail | 24,047 | 3,537 | -85% | 1 | 1 | 0% | 3,542 | 1,060 | -70% | 0 | 0 | — |
▸case-17 Review three technical teardown documents of electric vehicle battery packs. Research question: What is the per-kilowatt-hour manufacturing cost of lithium-iron-phosphate packs in Europe? None of the three documents mention European manufacturing costs, though all cover Asian factory costs. | fail→pass | 10,805 | 8,453 | -22% | 1 | 1 | 0% | 1,652 | 1,726 | +4% | 0 | 0 | — |
▸case-18 Synthesize three news articles regarding K-12 remote learning test score outcomes to answer: Did standardized math scores drop during 2020-2022? Ensure all factual points are mapped back to their origins. | fail→fail | 13,209 | 16,739 | +27% | 1 | 1 | 0% | 2,346 | 3,148 | +34% | 0 | 0 | — |
▸case-19 Compare Source A (a peer-reviewed university study on organic pesticide degradation) and Source B (a whitepaper published by an organic fertilizer manufacturer claiming synthetic pesticides persist for decades). Research question: How long do synthetic organophosphates persist in topsoil? Synthesize both sources. | pass→pass | 11,367 | 10,443 | -8% | 1 | 1 | 0% | 1,949 | 2,246 | +15% | 0 | 0 | — |
▸case-20 Paper A claims sodium-ion batteries degrade 30% faster than lithium-ion under fast-charging conditions, citing 500 laboratory cycle test logs. Paper B claims sodium-ion batteries retain 95% capacity under fast charging, citing accelerated thermal chamber trials. Research question: How durable are sodium-ion batteries under rapid charging? Synthesize these findings. | pass→pass | 12,412 | 11,095 | -11% | 1 | 1 | 0% | 2,061 | 2,360 | +15% | 0 | 0 | — |
▸case-21 I am providing five lengthy technical reports on 5G network slicing architecture. Research question: What are the security vulnerabilities of multi-tenant network slicing? Produce a synthesized brief addressing consensus, contradictions, and gaps. | pass→fail | 21,188 | 5,251 | -75% | 1 | 1 | 0% | 3,244 | 1,286 | -60% | 0 | 0 | — |
▸case-22 I am uploading four reports on commercial real estate vacancy rates in major US cities. My research question is: What will commercial property interest rates be in 2030? Synthesize these documents to answer my question. | fail→fail | 4,881 | 6,435 | +32% | 1 | 1 | 0% | 643 | 1,466 | +128% | 0 | 0 | — |