▸case-20 Outline a literature review structure with major themes and subtopics for recent advances in transformer compression techniques. | pass→pass | 19,102 | 13,555 | -29% | 1 | 1 | 0% | 3,141 | 2,438 | -22% | 0 | 0 | — |
▸case-21 Draft a budget justification paragraph explaining the need for 2 post-doctoral researchers and an H100 GPU cluster for our 3-year AI safety grant. | pass→fail | 12,326 | 9,507 | -23% | 1 | 1 | 0% | 1,763 | 1,806 | +2% | 0 | 0 | — |
▸case-22 Suggest 5 catchy titles for my upcoming IEEE ICRA paper on underwater swarm robotics localization. | pass→fail | 11,320 | 8,612 | -24% | 1 | 1 | 0% | 1,474 | 1,510 | +2% | 0 | 0 | — |
▸case-01 I've been brainstorming my machine learning thesis on federated learning for sparse medical datasets. Can you distill all our discussion into a single overarching North Star sentence that outlines my core objective, methodology, key problem, and long-term impact? | fail→fail | 6,526 | 4,908 | -25% | 1 | 1 | 0% | 1,152 | 930 | -19% | 0 | 0 | — |
▸case-02 We need to define the main research mission for our upcoming NeurIPS submission on efficient attention mechanisms. Please synthesize our ideas into a one-sentence core directional statement that explicitly covers the goal, approach, problem addressed, and overall significance. | fail→fail | 9,411 | 5,447 | -42% | 1 | 1 | 0% | 1,438 | 1,090 | -24% | 0 | 0 | — |
▸case-03 I'm writing a grant proposal for my robotics lab's work on adaptive manipulation. Could you formulate a single-sentence guiding vision statement summarizing what we plan to accomplish, the technical path we'll take, the specific challenge we're fixing, and the final impact? | fail→pass | 7,359 | 5,861 | -20% | 1 | 1 | 0% | 1,241 | 1,183 | -5% | 0 | 0 | — |
▸case-04 I need a single overarching directional statement for my quantum computing error correction project. We are working on surface code optimization to reduce physical qubit overhead for fault-tolerant quantum computation. Please provide one concise guiding sentence detailing our goal, technique, target obstacle, and ultimate goal. | fail→pass | 3,808 | 5,223 | +37% | 1 | 1 | 0% | 708 | 1,043 | +47% | 0 | 0 | — |
▸case-05 Synthesize my bioengineering research on CRISPR-Cas12a off-target reduction into a single master direction statement. We use engineered guide RNA hairpin structures to solve high off-target cleavage rates, aiming to enable safer gene therapies in vivo. Give me one sentence combining goal, method, problem, and impact. | fail→fail | 6,410 | 5,457 | -15% | 1 | 1 | 0% | 1,148 | 1,156 | +1% | 0 | 0 | — |
▸case-06 Draft a single core mission statement for our autonomous driving perception paper on radar-camera sensor fusion. We want to achieve reliable object detection in heavy fog by fusing millimeter-wave radar with thermal imaging to solve visibility loss, ultimately enabling safe level-4 autonomy. Express this in one structured sentence. | fail→pass | 4,793 | 3,410 | -29% | 1 | 1 | 0% | 897 | 831 | -7% | 0 | 0 | — |
▸case-07 Summarize our hardware research direction on edge-device neuromorphic processors. We build event-based spiking neural network accelerators to resolve high micro-watt power consumption in continuous keyword spotting, enabling battery-free IoT devices. Express this as a single directional sentence capturing goal, path, issue, and effect. | pass→pass | 5,925 | 3,382 | -43% | 1 | 1 | 0% | 1,139 | 806 | -29% | 0 | 0 | — |
▸case-08 Create a one-sentence vision statement for our sim-to-real transfer project in quadrupeds. We use domain randomization with learned system identification to fix reality gap policy failure in rough terrain locomotion, aiming for resilient field deployment. | fail→pass | 5,585 | 4,415 | -21% | 1 | 1 | 0% | 872 | 850 | -3% | 0 | 0 | — |
▸case-09 Formulate a single directional research sentence for my climate science manuscript. We apply spatial-temporal graph neural networks to resolve low resolution in extreme weather forecasting models, with the goal of improving regional disaster preparedness. | fail→fail | 4,006 | 4,798 | +20% | 1 | 1 | 0% | 637 | 1,033 | +62% | 0 | 0 | — |
▸case-10 Distill our privacy-preserving data sharing project into a single guiding research sentence. We use local differential privacy mechanisms with generative adversarial networks to overcome utility degradation in tabular health records, enabling safe cross-institutional clinical research. | fail→pass | 5,348 | 4,081 | -24% | 1 | 1 | 0% | 933 | 873 | -6% | 0 | 0 | — |
▸case-11 Write a single-sentence primary objective for our NLP study on morphologically rich low-resource languages. We leverage cross-lingual transfer learning with adapter units to combat data scarcity in machine translation, striving for global language equity in digital services. | pass→fail | 4,105 | 4,519 | +10% | 1 | 1 | 0% | 726 | 871 | +20% | 0 | 0 | — |
▸case-12 Provide a single directional sentence for our materials science initiative on lithium-metal batteries. We formulate ceramic-polymer composite electrolytes to mitigate dendrite short-circuiting at fast charge rates, with the long-term objective of doubling electric vehicle driving range. | fail→pass | 4,434 | 4,519 | +2% | 1 | 1 | 0% | 747 | 993 | +33% | 0 | 0 | — |
▸case-13 Summarize our 6G wireless communication research into a single vision sentence. We deploy programmable intelligent reflecting surfaces to tackle high-frequency path loss in urban dense environments, enabling terabit-per-second cellular connectivity. | fail→pass | 4,362 | 4,108 | -6% | 1 | 1 | 0% | 825 | 867 | +5% | 0 | 0 | — |
▸case-14 Synthesize our computational biology project into one core objective sentence. We apply geometric deep learning on molecular graphs to bypass combinatorial search bottlenecks in small-molecule drug design, accelerating therapeutic discovery for rare diseases. | fail→pass | 4,464 | 4,938 | +11% | 1 | 1 | 0% | 802 | 1,038 | +29% | 0 | 0 | — |
▸case-15 Distill our optical computing architecture project into a single sentence. We construct silicon photonic tensor cores to overcome thermal throttling limitations in matrix multiplication workloads, paving the way for ultra-fast energy-efficient AI datacenters. | fail→fail | 4,922 | 5,222 | +6% | 1 | 1 | 0% | 862 | 1,017 | +18% | 0 | 0 | — |
▸case-16 Write a single overarching research sentence for our software verification paper. We integrate neural-guided heuristics into CDCL SAT solvers to eliminate execution path explosion in formal verification, ensuring zero-day vulnerability elimination in critical infrastructure software. | fail→fail | 5,067 | 5,164 | +2% | 1 | 1 | 0% | 932 | 1,184 | +27% | 0 | 0 | — |
▸case-17 Formulate a one-sentence research vision for our plasma physics initiative. We implement real-time magnetic feedback control via deep reinforcement learning to prevent plasma disruption instabilities in tokamaks, moving closer to commercially viable fusion energy. | fail→pass | 4,079 | 5,525 | +35% | 1 | 1 | 0% | 692 | 1,064 | +54% | 0 | 0 | — |
▸case-18 Create a single research direction sentence for our biomedical device project. We design deterministic lateral displacement microfluidic channels to overcome low concentration yield in circulating tumor cell isolation, enabling early non-invasive cancer diagnostics. | fail→pass | 4,168 | 6,022 | +44% | 1 | 1 | 0% | 732 | 1,221 | +67% | 0 | 0 | — |
▸case-19 Write a complete 150-word abstract for my paper on graph neural networks in drug discovery, covering background, method, results, and conclusion. | pass→pass | 18,055 | 10,296 | -43% | 1 | 1 | 0% | 4,079 | 1,871 | -54% | 0 | 0 | — |