▸case-07 We are selecting an enterprise CRM platform among Salesforce, HubSpot, and Dynamics. We want to maximize sales representative adoption speed and maximize customer retention rate. However, total implementation cost must remain under $80,000 and data migration must complete within 60 days. Please formalize our decision model. | fail→fail | 21,198 | 24,376 | +15% | 1 | 1 | 0% | 2,990 | 3,732 | +25% | 0 | 0 | — |
▸case-01 We are choosing a cloud vendor to migrate our legacy database systems. Our budget cap is $150k/year, migration must finish within 4 months, and we want to maximize uptime while minimizing latency across 3 global regions. Attached are candidate proposals from AWS, Azure, and GCP. Please analyze this background and applicant data to generate a formal optimization specification detailing our primary target objectives with measurement criteria, explicit binding constraints with limits, and relative stakeholder trade-offs. | fail→fail | 38,731 | 22,567 | -42% | 1 | 1 | 0% | 4,845 | 3,391 | -30% | 0 | 0 | — |
▸case-02 Our logistics team needs to determine the best fleet mix of electric versus gas delivery vehicles for last-mile fulfillment. We aim to maximize daily delivery throughput and minimize total maintenance costs, but capital expenditure cannot exceed $2M and delivery time slots are non-negotiable. Please review this operational context and vehicle candidate set to build a structured optimization framework that establishes measurable objectives, hard constraint thresholds, and overall stakeholder trade-off priorities. | fail→fail | 96,643 | 30,376 | -69% | 1 | 1 | 0% | 6,753 | 2,801 | -59% | 0 | 0 | — |
▸case-03 I am evaluating three candidate real estate locations for a new manufacturing facility based on labor costs, tax incentives, port proximity, and strict environmental limits. Could you take this site dataset and project background to produce a formal optimization model defining our key performance objectives, hard boundary constraints, and stakeholder preference weightings? | fail→fail | 56,719 | 32,926 | -42% | 1 | 1 | 0% | 1,540 | 3,690 | +140% | 0 | 0 | — |
▸case-04 We are assessing four potential marketing channels (Search, Social, Display, Video). The team only mentioned that we want to maximize total conversions. No budget limits, timing constraints, or other objectives were discussed. Please process this input and generate the formal optimization framework. | fail→pass | 20,142 | 16,640 | -17% | 1 | 1 | 0% | 2,553 | 2,201 | -14% | 0 | 0 | — |
▸case-05 We are evaluating three API gateways (Kong, Apigee, Tyk). Our goals are to minimize request latency and maximize request throughput. No financial, compliance, or operational constraints were mentioned anywhere in our notes. Please build the formal optimization model for gateway selection. | fail→pass | 14,260 | 29,109 | +104% | 1 | 1 | 0% | 1,811 | 4,896 | +170% | 0 | 0 | — |
▸case-06 Our infrastructure engineering team is selecting a Kubernetes backup tool. The only rule specified by management is that backup restore time must be under 15 minutes. No optimization goals or key performance indicators were defined. Analyze this context and output the formal optimization specification. | fail→pass | 23,888 | 29,956 | +25% | 1 | 1 | 0% | 3,177 | 3,618 | +14% | 0 | 0 | — |
▸case-08 Our engineering team is deciding between gRPC, REST, and GraphQL for internal microservice communication. Context: we want to minimize network payload size and minimize CPU serialization overhead. Constraints: maximum allowable P99 latency is 15ms, and all services must support TLS 1.3 encryption. Produce a complete formal decision spec with trade-off preferences. | fail→fail | 44,764 | 21,521 | -52% | 1 | 1 | 0% | 7,037 | 4,016 | -43% | 0 | 0 | — |
▸case-09 We are evaluating Snowflake, BigQuery, and Databricks for our analytics platform. Context: maximize query execution speed and maximize developer satisfaction score. Constraints: monthly compute expenditure cannot exceed $12,000, and compliance requires SOC2 Type II certification. Formulate our optimization criteria and constraints. | fail→fail | 20,149 | 15,576 | -23% | 1 | 1 | 0% | 2,633 | 3,015 | +15% | 0 | 0 | — |
▸case-10 Our procurement team is selecting raw material suppliers across 4 candidates in North America and Asia. Context: maximize shipment reliability percentage and minimize lead time in days. Hard limit: defect rate must stay below 0.5% and minimum order quantity cannot exceed 5,000 units. Please output the structured optimization definition. | fail→fail | 17,417 | 14,964 | -14% | 1 | 1 | 0% | 2,404 | 2,011 | -16% | 0 | 0 | — |
▸case-11 We are deciding between Flutter, React Native, and Swift/Kotlin native development for a new fintech application. Context: minimize time-to-market in weeks and maximize cross-platform code reuse percentage. Hard limits: app initial launch time must be under 1.5 seconds, and memory footprint must remain under 120MB. Generate our formal optimization framework. | fail→fail | 48,014 | 20,014 | -58% | 1 | 1 | 0% | 7,591 | 2,919 | -62% | 0 | 0 | — |
▸case-12 Our hospital network is evaluating Epic, Cerner, and MEDITECH. Context: maximize clinician satisfaction ratings and minimize system downtime during transition. Constraints: HIPAA compliance is mandatory, and total contract cost must not exceed $5M over 3 years. Produce the formal objective and constraint specification. | fail→fail | 16,595 | 11,262 | -32% | 1 | 1 | 0% | 3,035 | 2,298 | -24% | 0 | 0 | — |
▸case-13 We are choosing between GitHub Actions, GitLab CI, and CircleCI for our monorepo. Context: minimize build duration and maximize developer onboarding speed. Constraints: monthly build cost must stay under $3,000 and self-hosted runner support is mandatory. Formalize our optimization criteria. | fail→fail | 19,644 | 20,444 | +4% | 1 | 1 | 0% | 3,345 | 3,721 | +11% | 0 | 0 | — |
▸case-14 Our SOC team is selecting an endpoint detection and response (EDR) vendor among CrowdStrike, SentinelOne, and Defender. Context: maximize threat detection accuracy and minimize false positive alert rate. Constraints: agent RAM usage must be under 200MB and licensing cost cannot exceed $45 per endpoint annually. Formulate our decision model. | fail→fail | 24,772 | 20,765 | -16% | 1 | 1 | 0% | 4,408 | 3,041 | -31% | 0 | 0 | — |
▸case-15 We are choosing between Stripe, Adyen, and Checkout.com for global e-commerce payment processing. Context: maximize authorization success rate and minimize processing fee percentage. Constraints: must support 3D Secure 2.0 and payout settlement time must not exceed 2 business days. Generate the optimization specification. | fail→fail | 19,754 | 18,044 | -9% | 1 | 1 | 0% | 3,822 | 3,522 | -8% | 0 | 0 | — |
▸case-16 Our clean energy project is evaluating Lithium-iron-phosphate vs Flow batteries vs Sodium-ion. Context: maximize round-trip energy efficiency and maximize cycle life count. Constraints: upfront capital expenditure must be under $10M and footprint area cannot exceed 2,000 square meters. Formulate our formal optimization goals. | fail→fail | 21,208 | 14,092 | -34% | 1 | 1 | 0% | 3,976 | 3,027 | -24% | 0 | 0 | — |
▸case-17 Our HR department is choosing among Blue Cross, Aetna, and UnitedHealth for employee coverage. Context: maximize employee coverage satisfaction and minimize annual premium cost per employee. Constraints: out-of-pocket maximum must not exceed $4,000 and nationwide network coverage is mandatory. Produce the formal decision model. | fail→fail | 15,346 | 15,777 | +3% | 1 | 1 | 0% | 3,005 | 3,384 | +13% | 0 | 0 | — |
▸case-18 We are selecting between Algolia, Elasticsearch, and Constructor.io for site search. Context: minimize search result latency in milliseconds and maximize conversion rate from search queries. Constraints: monthly subscription fee must be under $5,000 and index update frequency must be under 5 seconds. Define our optimization framework. | fail→fail | 21,768 | 13,041 | -40% | 1 | 1 | 0% | 3,748 | 2,533 | -32% | 0 | 0 | — |
▸case-19 We are evaluating three candidate VP of Engineering profiles. Context: maximize team retention rate and maximize product feature velocity. Constraints: base salary cap of $350k and mandatory candidate availability within 30 days. Please formalize the candidate evaluation optimization criteria. | fail→fail | 13,028 | 20,441 | +57% | 1 | 1 | 0% | 2,435 | 2,833 | +16% | 0 | 0 | — |
▸case-20 Our dev team wants to choose a logging service and we only care about minimizing log retention costs. No constraints or secondary goals exist in our notes. Please draft the optimization output. | fail→pass | 14,940 | 10,459 | -30% | 1 | 1 | 0% | 2,494 | 1,998 | -20% | 0 | 0 | — |
▸case-21 We are selecting server instance types (c6i, m6i, r6i) for a real-time analytics pipeline. Context: maximize query throughput per second and minimize hourly instance cost. Constraints: peak memory usage must remain under 64GB and vCPU count must not exceed 16 per instance. Please build our optimization model. | fail→fail | 20,154 | 17,159 | -15% | 1 | 1 | 0% | 3,927 | 3,736 | -5% | 0 | 0 | — |
▸case-22 We are picking a video streaming CDN between Fastly, Cloudflare, and Akamai. Context: maximize video startup speed and minimize rebuffering ratio. However, our infrastructure team provided no budget limit, contract length constraint, or performance thresholds. Please generate the formal decision specification. | fail→pass | 25,402 | 11,606 | -54% | 1 | 1 | 0% | 4,083 | 2,227 | -45% | 0 | 0 | — |
▸case-23 Here is a formulated linear program for our warehouse allocation: maximize Z = 30x1 + 45x2 subject to 2x1 + 3x2 <= 120 and x1 + x2 <= 50. Please write a Python script using scipy.optimize.linprog to solve for the exact values of x1 and x2. | pass→pass | 7,925 | 14,408 | +82% | 1 | 1 | 0% | 2,076 | 3,252 | +57% | 0 | 0 | — |
▸case-24 Given this completed simplex tableau for our product mix decision where the optimal objective value is $50,000 and the dual price for labor hours is $15/hour: calculate how much total profit changes if available labor hours increase from 400 to 450 hours. | pass→pass | 5,851 | 6,737 | +15% | 1 | 1 | 0% | 1,168 | 1,466 | +26% | 0 | 0 | — |
▸case-25 We need to extract past vendor performance data from our PostgreSQL database. Write a SQL query against the vendor_evaluations table to select vendor_id, average latency_ms, and uptime_percentage where evaluation_year is 2023. | pass→pass | 5,416 | 4,670 | -14% | 1 | 1 | 0% | 901 | 1,185 | +32% | 0 | 0 | — |