▸case-12 We want to pull recent AI paper preprints from arXiv specifically focused on computer vision, avoiding general physics or quantitative biology papers. Which arXiv category subject code should be appended to the search query? | pass→pass | 2,263 | 2,755 | +22% | 1 | 1 | 0% | 397 | 622 | +57% | 0 | 0 | — |
▸case-01 I am conducting a systematic literature review on CRISPR gene editing in human hematopoietic stem cells. I need to formulate a search strategy to catch the latest un-reviewed manuscript preprints before formal journal publication. Which major preprint server dedicated to biology should be prioritized for indexing these early-stage biological studies, and what filtering parameter distinguishes preprints from published literature? | pass→pass | 9,018 | 6,783 | -25% | 1 | 1 | 0% | 1,672 | 1,445 | -14% | 0 | 0 | — |
▸case-02 When querying PubMed for articles on hypertension treatment using ACE inhibitors, a naive search for 'ACE inhibitor' misses papers using specific drug names like lisinopril or captopril. How should the query be structured using NLM's controlled vocabulary to automatically capture all sub-terms and drug variants? | pass→pass | 10,137 | 8,692 | -14% | 1 | 1 | 0% | 2,006 | 1,919 | -4% | 0 | 0 | — |
▸case-03 I want to search literature databases for articles discussing machine learning applied to either lung cancer or breast cancer, but excluding reviews. I constructed the query: `machine learning AND lung cancer OR breast cancer NOT review`. Why does this query produce erratic results, and how should operators and parenthetical syntax be formatted? | pass→pass | 11,195 | 8,608 | -23% | 1 | 1 | 0% | 2,149 | 1,882 | -12% | 0 | 0 | — |
▸case-04 We are tracking unpublished early results of phase 2 mRNA vaccine clinical trials in human cohorts. Should we target bioRxiv or medRxiv for these clinical health science preprints, and why? | pass→pass | 8,986 | 6,516 | -27% | 1 | 1 | 0% | 1,685 | 1,277 | -24% | 0 | 0 | — |
▸case-05 When querying an academic literature API to retrieve papers published strictly between January 1, 2020 and December 31, 2023, what standard date filter parameter format should be submitted instead of free-text strings like 'last 3 years'? | pass→pass | 8,602 | 5,834 | -32% | 1 | 1 | 0% | 2,080 | 1,547 | -26% | 0 | 0 | — |
▸case-06 Given the persistent identifier `10.1038/s41586-020-2649-2`, what web service endpoint or REST API retrieves structured bibliographic JSON metadata directly without HTML scraping? | pass→pass | 6,188 | 6,882 | +11% | 1 | 1 | 0% | 1,519 | 1,789 | +18% | 0 | 0 | — |
▸case-07 We need to search academic literature for 'type 2 diabetes'. To ensure high recall across historical and international papers, how should the search string expand this single phrase to include clinical synonyms and alternative nomenclature? | pass→pass | 14,299 | 9,486 | -34% | 1 | 1 | 0% | 2,947 | 2,163 | -27% | 0 | 0 | — |
▸case-08 A researcher is searching for papers about 'how plants adapt to extreme drought', but relevant papers might use terms like 'desiccation tolerance', 'xerophytic adaptations', or 'water deficit signaling'. Which search paradigm should be leveraged alongside keyword search to retrieve conceptually similar papers without exact word matches? | pass→pass | 5,208 | 3,758 | -28% | 1 | 1 | 0% | 1,012 | 850 | -16% | 0 | 0 | — |
▸case-09 An automated literature harvesting pipeline needs to check whether a retrieved DOI has a legally accessible, free full-text PDF available before attempting download. What standard open-access database API should be queried using the paper's DOI? | pass→pass | 5,365 | 6,343 | +18% | 1 | 1 | 0% | 931 | 1,467 | +58% | 0 | 0 | — |
▸case-10 To discover seminal foundational papers that influenced a landmark 2018 paper on transformer architectures, as well as subsequent studies that built upon it, which two distinct citation direction techniques must be executed? | pass→pass | 3,400 | 3,453 | +2% | 1 | 1 | 0% | 713 | 812 | +14% | 0 | 0 | — |
▸case-11 When searching for the multi-word phrase 'single-cell RNA sequencing batch effect correction', how should the search string be formatted to prevent the search engine from splitting the words across different sections of the document? | pass→pass | 6,036 | 5,223 | -13% | 1 | 1 | 0% | 1,127 | 1,097 | -3% | 0 | 0 | — |
▸case-13 When aggregating search results from PubMed, arXiv, and IEEE Xplore, the same paper often appears multiple times with slight variation in title formatting. What unique canonical identifier field should be prioritized as the primary key for deduplicating these entries? | pass→pass | 5,964 | 4,650 | -22% | 1 | 1 | 0% | 1,094 | 1,029 | -6% | 0 | 0 | — |
▸case-14 A medical guideline panel requires literature search results to distinguish rigorously between peer-reviewed journal articles and non-peer-reviewed preprints. How should an automated literature search pipeline classify publications from bioRxiv versus MEDLINE? | pass→pass | 11,521 | 9,519 | -17% | 1 | 1 | 0% | 2,287 | 1,938 | -15% | 0 | 0 | — |
▸case-15 An automated research assistant needs to export metadata of retrieved papers directly into a standardized format readable by LaTeX reference managers like JabRef or Overleaf. What standard export format specification should be requested from paper APIs? | fail→pass | 6,093 | 7,635 | +25% | 1 | 1 | 0% | 1,128 | 1,673 | +48% | 0 | 0 | — |
▸case-16 To search PubMed or Web of Science for variations of the word 'therapy' (such as therapeutic, therapies, therapist), what wildcard symbol is standardly appended to the root word stem `therap`? | pass→pass | 2,381 | 2,409 | +1% | 1 | 1 | 0% | 499 | 583 | +17% | 0 | 0 | — |
▸case-17 A research team wants to quickly identify the most influential foundational papers in deep reinforcement learning from the last decade. How should the search result sorting order be configured in academic search queries? | pass→pass | 8,070 | 7,334 | -9% | 1 | 1 | 0% | 1,700 | 1,699 | -0% | 0 | 0 | — |
▸case-18 How can a search query in PubMed or Semantic Scholar filter for papers published specifically by researchers at Stanford University working on Alzheimer's disease? | pass→pass | 8,781 | 10,172 | +16% | 1 | 1 | 0% | 1,562 | 1,935 | +24% | 0 | 0 | — |
▸case-19 I have a CSV file containing patient blood pressure measurements before and after clinical treatment (`bp_data.csv`). Please write a Python script using pandas and scipy.stats to perform a paired t-test, calculate the p-value, and plot a boxplot. | pass→pass | 11,081 | 10,338 | -7% | 1 | 1 | 0% | 2,368 | 2,459 | +4% | 0 | 0 | — |
▸case-20 Draft independent and dependent patent claims under 35 U.S.C. 112 for a novel microfluidic diagnostic chip that detects viral RNA using isothermal amplification. | pass→pass | 14,458 | 10,777 | -25% | 1 | 1 | 0% | 2,659 | 2,132 | -20% | 0 | 0 | — |
▸case-21 Provide a complete LaTeX document template using the `article` class with `amsmath`, `graphicx`, and `hyperref` packages, including sections for Abstract, Introduction, Methods, and Results. | pass→pass | 9,282 | 8,424 | -9% | 1 | 1 | 0% | 1,863 | 1,905 | +2% | 0 | 0 | — |
▸case-22 A systematic review protocol requires filtering search results strictly to Randomized Controlled Trials studying statin therapy in cardiovascular disease. What PubMed Publication Type filter tag should be attached to the query term? | pass→pass | 3,346 | 4,055 | +21% | 1 | 1 | 0% | 540 | 742 | +37% | 0 | 0 | — |