▸case-05 Format a book citation in Chicago Author-Date style (17th ed.) for 'Quantum Computation' by Robert Taylor, published in Chicago by Academic Press in 2020. Many writers put the publication year at the very end as in Chicago Notes-Bibliography; place it according to Chicago Author-Date rules. | pass→pass | 4,742 | 4,179 | -12% | 1 | 1 | 0% | 832 | 855 | +3% | 0 | 0 | — |
▸case-01 Format a citation for a journal article by Jane Doe and John Smith published in 2023 titled 'Advanced Neural Networks' in Journal of AI, volume 12, issue 3, pages 45-50, with DOI 10.1016/j.ai.2023.01.002. Ensure standard APA 7th edition formatting rules are followed, especially regarding author names and DOI links. | fail→pass | 4,803 | 4,654 | -3% | 1 | 1 | 0% | 1,039 | 1,089 | +5% | 0 | 0 | — |
▸case-02 Format an in-text citation and reference list entry for an IEEE conference paper by Alice Johnson presented at the 2022 International Conference on Robotics. Writers often use author-year parens like (Johnson, 2022); convert this into the proper IEEE standard standard in-text marker. | pass→pass | 7,744 | 4,973 | -36% | 1 | 1 | 0% | 1,271 | 893 | -30% | 0 | 0 | — |
▸case-03 Generate a valid BibTeX entry for a conference paper titled 'Scaling Transformer Models' presented at NeurIPS 2021 by Alan Turing. A common default is to use @article, but correct the entry type and required BibTeX fields for a published conference paper. | pass→pass | 4,373 | 4,736 | +8% | 1 | 1 | 0% | 880 | 1,002 | +14% | 0 | 0 | — |
▸case-04 Format a reference list entry in Vancouver style for a biomedical article authored by Smith A, Jones B, Williams C, Brown D, Davis E, Miller F, and Wilson G published in 2021. Most styles list all authors or cut off at 3, but apply Vancouver's rule for more than 6 authors. | pass→pass | 5,515 | 6,075 | +10% | 1 | 1 | 0% | 798 | 1,144 | +43% | 0 | 0 | — |
▸case-06 Construct a RIS formatted string for a journal article titled 'Gene Editing via CRISPR' by Elena Rostova published in 2022 in Nature Biotechnology, volume 40, pages 100-108. Avoid using generic tag names like TX or NT for primary metadata. | pass→pass | 4,314 | 6,316 | +46% | 1 | 1 | 0% | 861 | 1,177 | +37% | 0 | 0 | — |
▸case-07 Create an MLA 9th edition works cited entry for an article by Sarah Jenkins titled 'Historical Linguistics' in Language Studies, vol. 15, no. 2, 2019, pp. 12-30, accessed via JSTOR. Do not include the access date unless required, but properly identify the container and location. | pass→pass | 11,064 | 4,829 | -56% | 1 | 1 | 0% | 1,991 | 966 | -51% | 0 | 0 | — |
▸case-08 Clean and format a set of raw DOI strings ('doi:10.1038/s41586-020-2649-2', 'http://dx.doi.org/10.1038/s41586-020-2649-2', '10.1038/s41586-020-2649-2') into modern Crossref standard URL presentation. | pass→pass | 8,509 | 8,755 | +3% | 1 | 1 | 0% | 1,589 | 1,734 | +9% | 0 | 0 | — |
▸case-09 Create a BibTeX entry for an article by François Müller titled 'Über Machine Learning' in 2022. Raw unicode accents are often pasted directly, but format the special characters for maximum compatibility with classic LaTeX/BibTeX engines. | pass→pass | 6,188 | 6,332 | +2% | 1 | 1 | 0% | 1,066 | 1,261 | +18% | 0 | 0 | — |
▸case-10 Format in-text citations in APA 7th edition for two distinct 2021 papers by Carol Danvers. Standard in-text parenthetical citations would both appear as (Danvers, 2021), creating ambiguity. | pass→pass | 6,545 | 4,599 | -30% | 1 | 1 | 0% | 1,276 | 992 | -22% | 0 | 0 | — |
▸case-11 Represent a book published in 2018 titled 'Deep Learning' by Ian Goodfellow, Yoshua Bengio, and Aaron Courville in CSL-JSON format. Avoid using arbitrary key names like 'writers' or 'pub_date'. | pass→pass | 5,586 | 5,362 | -4% | 1 | 1 | 0% | 1,115 | 1,183 | +6% | 0 | 0 | — |
▸case-12 Format a reference list item in AMA (American Medical Association 11th ed.) style for an article by Patel NK, Smith JR in JAMA 2020;323(4):300-305. Avoid inserting periods after author initials or using full month names. | pass→pass | 8,222 | 7,849 | -5% | 1 | 1 | 0% | 1,390 | 1,269 | -9% | 0 | 0 | — |
▸case-13 Construct a Harvard-style reference for a chapter titled 'Data Structures' by Mark Lee in the edited book 'Computer Science Handbook' edited by Ray Adams, published by CRC Press in London, 2019, pp. 45-67. | fail→fail | 3,995 | 4,646 | +16% | 1 | 1 | 0% | 775 | 896 | +16% | 0 | 0 | — |
▸case-14 Write a BibTeX entry for a paper titled 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'. BibTeX style files often convert titles to lower case; ensure proper capitalisation preservation for acronyms. | pass→pass | 4,461 | 4,914 | +10% | 1 | 1 | 0% | 876 | 1,115 | +27% | 0 | 0 | — |
▸case-15 Format a citation according to The Bluebook (Law Review style) for an article titled 'AI and Privacy' by Michael Green, published in Volume 130 of the Harvard Law Review in 2021, starting at page 500. | pass→pass | 7,136 | 7,791 | +9% | 1 | 1 | 0% | 1,301 | 1,489 | +14% | 0 | 0 | — |
▸case-16 Format an APA 7th edition reference for an arXiv preprint titled 'Prompt Engineering Strategies' by David Zhang, posted in 2023 with identifier arXiv:2301.12345 [cs.CL]. Preprints are often incorrectly cited as journal articles. | pass→pass | 7,671 | 4,462 | -42% | 1 | 1 | 0% | 1,455 | 998 | -31% | 0 | 0 | — |
▸case-17 Format a reference for Nature journal style for a paper by Watson, J. D. & Crick, F. H. published in Nature volume 171, pages 737-738 in 1953. Nature uses specific bolding and ordering rules. | pass→pass | 6,101 | 5,882 | -4% | 1 | 1 | 0% | 1,116 | 1,207 | +8% | 0 | 0 | — |
▸case-18 Format a citation record in EndNote tagged format (Refer format) for a journal article titled 'Robotics in Surgery' by Arthur Pendelton, 2021, Journal of Medicine, Vol 5, No 2. | pass→pass | 4,182 | 2,745 | -34% | 1 | 1 | 0% | 570 | 594 | +4% | 0 | 0 | — |
▸case-19 Show the first and subsequent in-text parenthetical citations in APA 7th edition for a report by the World Health Organization published in 2022. | pass→pass | 5,897 | 3,505 | -41% | 1 | 1 | 0% | 969 | 820 | -15% | 0 | 0 | — |
▸case-20 I have a CSV dataset containing 10,000 papers with their publication years and citation counts. Run a linear regression in Python using scikit-learn to predict citation count based on publication year. | pass→pass | 8,829 | 10,654 | +21% | 1 | 1 | 0% | 1,944 | 2,255 | +16% | 0 | 0 | — |
▸case-21 Write a 200-word scientific abstract summarizing novel research on a prospective randomized controlled trial of a new antihypertensive medication called Cardiovex. | pass→pass | 13,976 | 8,096 | -42% | 1 | 1 | 0% | 2,388 | 1,671 | -30% | 0 | 0 | — |
▸case-22 My TeX Live installation fails with 'pdflatex: command not found' when running a GitHub Actions workflow on ubuntu-latest. Provide the bash commands to install TeX Live packages in the workflow. | pass→pass | 7,278 | 6,966 | -4% | 1 | 1 | 0% | 1,213 | 1,267 | +4% | 0 | 0 | — |