▸case-01 I need a comprehensive API reference guide for our payment processing SDK. Please draft the documentation covering every endpoint, input parameter, return object, edge case constraint, and runnable code sample for quick developer integration. | fail→fail | 49,752 | 39,026 | -22% | 1 | 1 | 0% | 8,229 | 6,622 | -20% | 0 | 0 | — |
▸case-02 We are launching a new telemetry agent and need a full configuration guide. Could you write out the reference documentation including all configuration flags, nested YAML schema definitions, table layouts for defaults, and troubleshooting steps for invalid setups? | fail→fail | 42,035 | 36,332 | -14% | 1 | 1 | 0% | 8,191 | 7,799 | -5% | 0 | 0 | — |
▸case-03 Can you assemble a technical reference document for migrating from version 1 to version 2 of our data pipeline library? Make sure to include a quick start section, detailed method signatures, side-by-side code examples, warning callouts for breaking changes, and cross-references between related features. | fail→fail | 42,075 | 33,723 | -20% | 1 | 1 | 0% | 8,126 | 5,991 | -26% | 0 | 0 | — |
▸case-04 We need to document our decision to switch our primary database from PostgreSQL 15 to MongoDB 7.0 for our chat application. Write an Architecture Decision Record (ADR) outlining the context, decision driver, considered options, and chosen solution. | pass→pass | 22,614 | 23,655 | +5% | 1 | 1 | 0% | 3,724 | 4,443 | +19% | 0 | 0 | — |
▸case-05 Our Kubernetes cluster experienced a 45-minute API server outage on May 12th due to memory exhaustion. Draft an incident post-mortem report covering incident timeline, root cause analysis, impact summary, and corrective action items. | pass→pass | 24,187 | 23,187 | -4% | 1 | 1 | 0% | 3,367 | 4,333 | +29% | 0 | 0 | — |
▸case-06 Draft a Product Requirements Document (PRD) for adding two-factor authentication (2FA) via TOTP to our user account settings page, including user stories, target persona, success metrics, and high-level wireframe requirements. | pass→pass | 19,535 | 25,366 | +30% | 1 | 1 | 0% | 3,482 | 5,312 | +53% | 0 | 0 | — |
▸case-07 Write a high-level announcement blog post for our enterprise sales software release, highlighting new AI lead-scoring features and customer benefits to drive webinar signups. | pass→pass | 11,922 | 11,817 | -1% | 1 | 1 | 0% | 1,953 | 2,136 | +9% | 0 | 0 | — |
▸case-08 Here are the raw endpoint specs for the OpenWeatherMap One Call API v3.0: endpoint /data/3.0/onecall, parameters lat (float, required), lon (float, required), appid (string, required), units (string, optional: standard, metric, imperial), exclude (string, optional). Generate a parameter reference entry. Most developers prefer narrative prose descriptions over tabular formatting, so feel free to write it as a paragraph. | fail→pass | 8,133 | 12,903 | +59% | 1 | 1 | 0% | 1,618 | 3,035 | +88% | 0 | 0 | — |
▸case-09 Create a reference guide for the Redis 7.2 maxmemory-policy and maxmemory configuration directives. Just summarize what they do in general terms—no need to list exact default values, memory limit units, or edge cases like OOM error conditions. | fail→pass | 8,784 | 16,995 | +93% | 1 | 1 | 0% | 1,696 | 3,684 | +117% | 0 | 0 | — |
▸case-10 Document the JSON Schema for an e-commerce order payload (schema fields: order_id string, items array of objects with sku string and quantity integer, total_amount number). Just give me a quick informal summary of the fields without specifying constraints, nullability, or verification steps. | fail→pass | 3,213 | 18,109 | +464% | 1 | 1 | 0% | 620 | 4,342 | +600% | 0 | 0 | — |
▸case-11 Provide a quick-start reference for the GitHub REST API v3 user profile endpoint GET /user. Give pseudo-code snippets that sketch out the concept rather than functional, executable code samples. | fail→fail | 8,401 | 17,306 | +106% | 1 | 1 | 0% | 1,711 | 3,970 | +132% | 0 | 0 | — |
▸case-12 Document the deprecation of the OAuth2 password grant type in Keycloak 24. Write it as standard body text in line with regular usage descriptions so it doesn't stand out too drastically. | fail→fail | 9,763 | 14,003 | +43% | 1 | 1 | 0% | 1,361 | 3,056 | +125% | 0 | 0 | — |
▸case-13 Create a troubleshooting reference entry for AWS S3 HTTP 403 Access Denied and 409 BucketNotEmpty errors. A simple list of possible causes is sufficient—no need for verification commands or structured resolution steps. | fail→fail | 9,608 | 14,173 | +48% | 1 | 1 | 0% | 1,445 | 2,618 | +81% | 0 | 0 | — |
▸case-14 I have a sprawling CLI reference for AWS CLI v2 with over 100 subcommands. Should I put all subcommands into one single, flat Markdown file without table of contents or breadcrumbs to keep navigation simple? | fail→fail | 12,430 | 13,392 | +8% | 1 | 1 | 0% | 2,177 | 3,035 | +39% | 0 | 0 | — |
▸case-15 Write a quick start reference for initializing a Docker container running PostgreSQL 16. Skip the verification step at the end since developers can figure out if PostgreSQL is running on their own. | fail→fail | 6,372 | 9,348 | +47% | 1 | 1 | 0% | 1,358 | 2,330 | +72% | 0 | 0 | — |
▸case-16 Write a reference entry for updating an application from Python 3.9 typing.List / typing.Dict to Python 3.10 native type hint syntax list[] / dict[]. Standard narrative descriptions are enough—do not include side-by-side before and after code comparisons. | fail→fail | 15,104 | 26,130 | +73% | 1 | 1 | 0% | 2,361 | 4,532 | +92% | 0 | 0 | — |
▸case-17 We need a reference page for Docker Compose v2 environment file directives (.env). Can you format this in raw HTML tables embedded inside Microsoft Word binary docx format? Also, skip top matter metadata like title and description. | fail→pass | 29,527 | 33,994 | +15% | 1 | 1 | 0% | 6,219 | 5,959 | -4% | 0 | 0 | — |
▸case-18 Document the Stripe API Charges endpoint limit constraints (rate limits, maximum charge amount, currency limits). Is it okay to omit rate limit retry headers (429 status) and decimal precision bounds since edge cases rarely happen? | fail→fail | 17,391 | 17,244 | -1% | 1 | 1 | 0% | 2,750 | 3,830 | +39% | 0 | 0 | — |
▸case-19 Create a reference entry for Apache Kafka 3.6 topic configuration parameters cleanup.policy and retention.ms. You don't need to link or reference related topics like segment.bytes or log compaction mechanics. | fail→fail | 7,885 | 14,688 | +86% | 1 | 1 | 0% | 1,382 | 3,413 | +147% | 0 | 0 | — |
▸case-20 We are building a reference catalog for standard HTTP status codes (200, 400, 401, 403, 404, 500) for REST API services. Write each status code entry using a completely different structure and layout so the page stays visually interesting and non-repetitive. | fail→fail | 24,328 | 19,944 | -18% | 1 | 1 | 0% | 4,403 | 4,454 | +1% | 0 | 0 | — |
▸case-21 Draft a reference guide for the Git 2.43 CLI commands git rebase, git merge, and git cherry-pick. Skip command summaries and quick-lookup tables, going straight into long prose explanations. | fail→fail | 46,804 | 48,853 | +4% | 1 | 1 | 0% | 8,232 | 8,769 | +7% | 0 | 0 | — |
▸case-22 We are documenting environment variables for a Node.js 20 Express application (PORT, NODE_ENV, DATABASE_URL, JWT_SECRET). Write them out as an unstructured list of bullet points without specifying types, defaults, or required status. | fail→fail | 3,132 | 4,902 | +57% | 1 | 1 | 0% | 617 | 1,400 | +127% | 0 | 0 | — |
▸case-23 Draft a reference entry for configuring NGINX 1.24 reverse proxy headers (X-Forwarded-For, X-Real-IP, Host). Leave out any testing or validation instructions like nginx -t or curl -I. | fail→fail | 15,388 | 13,010 | -15% | 1 | 1 | 0% | 2,461 | 3,004 | +22% | 0 | 0 | — |