▸case-06 We want to decide on a naming convention for our internal git feature branches like feature/ticket-123. Can you give us a quick recommendation based on standard git workflow best practices? | pass→pass | 7,354 | 10,915 | +48% | 1 | 1 | 0% | 1,303 | 2,066 | +59% | 0 | 0 | — |
▸case-01 We need to upgrade our payment service to the latest Stripe Billing API version. Could you research the external documentation and give me an implementation breakdown? Please structure your response to cover recent changes, how it impacts our project, a practical step-by-step implementation checklist, and any potential risks or unresolved items. | fail→pass | 19,161 | 12,785 | -33% | 1 | 1 | 0% | 3,254 | 2,242 | -31% | 0 | 0 | — |
▸case-02 I'm updating our Node.js service to use AWS SDK v3 for DynamoDB instead of v2. Can you review the migration guides and summarize what we need to do? I'd like a breakdown covering what is new, key takeaways for our repository, a clear checklist of code steps to follow, and any unknown factors or edge-case risks. | fail→fail | 16,757 | 9,126 | -46% | 1 | 1 | 0% | 3,380 | 2,178 | -36% | 0 | 0 | — |
▸case-03 We are adopting Next.js 14 Server Actions for our form submissions. Please check the online documentation and provide technical guidance organized into four sections: what modified, how it applies to our codebase, a concrete checklist for implementation, and remaining uncertainties or risks. | fail→pass | 14,666 | 10,652 | -27% | 1 | 1 | 0% | 2,633 | 2,511 | -5% | 0 | 0 | — |
▸case-04 I have a local helper function formatUserDisplayName(user) in src/utils/user.js that needs a simple refactor to handle null middle names. Can you clean up this internal function to handle null values cleanly? | pass→pass | 4,779 | 6,170 | +29% | 1 | 1 | 0% | 897 | 1,319 | +47% | 0 | 0 | — |
▸case-05 Our team wants to rename the local variable user_role to account_role across our internal controller files in src/controllers/. Can you write a quick local search-and-replace summary for our team? | pass→pass | 7,755 | 9,612 | +24% | 1 | 1 | 0% | 1,459 | 1,770 | +21% | 0 | 0 | — |
▸case-07 We are migrating our single-page React app from Auth0 Implicit Flow to Authorization Code Flow with PKCE. Can you review Auth0's official documentation and prepare a migration plan covering API updates, impact on our frontend codebase, actionable implementation code steps, and any unresolved risks? | fail→fail | 18,437 | 14,226 | -23% | 1 | 1 | 0% | 3,650 | 2,815 | -23% | 0 | 0 | — |
▸case-08 We plan to upgrade our database cluster to PostgreSQL 16 and enable logical replication from standby servers. Please consult PostgreSQL documentation and break down the changes, relevance to our cluster setup, concrete migration commands, and potential edge-case risks. | fail→pass | 17,702 | 14,775 | -17% | 1 | 1 | 0% | 3,275 | 2,906 | -11% | 0 | 0 | — |
▸case-09 We have legacy Kubernetes Deployment manifests using deprecated apiVersions. Research official Kubernetes reference docs and provide technical migration guidance outlining what changed in API versions, impact on our YAML files, step-by-step kubectl commands, and unknown cluster runtime risks. | fail→pass | 18,501 | 11,166 | -40% | 1 | 1 | 0% | 3,516 | 2,413 | -31% | 0 | 0 | — |
▸case-10 Our repository uses Docker Compose file format v2 with deprecated mem_limit attributes. Review official Docker documentation and detail schema changes, codebase impacts, concrete file edit commands, and unknown runtime memory constraints. | fail→fail | 19,413 | 13,373 | -31% | 1 | 1 | 0% | 3,807 | 2,583 | -32% | 0 | 0 | — |
▸case-11 We are updating our Python backend to use OpenAI's Chat Completions API instead of legacy Completions. Consult official OpenAI API reference docs and outline API modifications, adjustments needed for our prompt pipeline, concrete python code steps, and remaining risks. | fail→fail | 18,789 | 13,532 | -28% | 1 | 1 | 0% | 3,683 | 2,818 | -23% | 0 | 0 | — |
▸case-12 Our Node.js service needs to upgrade @elastic/elasticsearch from v7 to v8. Review official Elastic guides and detail client breaking changes, impact on our query mapping files, concrete npm install and code update commands, and unknown cluster compatibility risks. | pass→pass | 18,865 | 17,672 | -6% | 1 | 1 | 0% | 3,698 | 3,462 | -6% | 0 | 0 | — |
▸case-13 We are upgrading our Terraform infrastructure modules to AWS Provider v5. Check HashiCorp AWS provider documentation and supply guidance covering provider attribute changes, impacts on our HCL files, concrete terraform CLI commands, and unverified provider behavior risks. | fail→fail | 20,567 | 12,117 | -41% | 1 | 1 | 0% | 3,838 | 2,470 | -36% | 0 | 0 | — |
▸case-14 We are upgrading our Redis instance to Redis 7.0 and enabling ACL v2. Review official Redis documentation and break down new ACL rules, repository configuration requirements, concrete redis-cli commands, and unknown server setup risks. | fail→fail | 17,344 | 13,498 | -22% | 1 | 1 | 0% | 3,102 | 2,630 | -15% | 0 | 0 | — |
▸case-15 Our custom GitHub Actions JavaScript actions must migrate from Node.js 16 to Node.js 20. Consult official GitHub Actions documentation and provide details on runtime changes, action.yml impacts, concrete YAML modifications, and unknown workflow runner compatibility risks. | fail→pass | 16,648 | 12,081 | -27% | 1 | 1 | 0% | 3,365 | 2,435 | -28% | 0 | 0 | — |
▸case-16 We want to upgrade our frontend build pipeline from Tailwind CSS v2 to v3 JIT mode. Research official Tailwind CSS migration guides and summarize core framework changes, impacts on tailwind.config.js, concrete terminal install steps, and unknown CSS plugin risks. | fail→fail | 20,476 | 12,939 | -37% | 1 | 1 | 0% | 3,646 | 2,691 | -26% | 0 | 0 | — |
▸case-17 We need to upgrade our web client from Firebase v8 namespace SDK to v9 modular SDK. Review official Firebase web docs and break down tree-shaking API changes, impacts on our service imports, concrete code import refactoring steps, and unknown auth session risks. | fail→pass | 16,680 | 10,332 | -38% | 1 | 1 | 0% | 3,324 | 2,023 | -39% | 0 | 0 | — |
▸case-18 Our backend service is upgrading mongodb driver from v3 to v4. Consult official MongoDB driver documentation and supply technical breakdown of connection option changes, impacts on our database model, concrete code update commands, and unverified replica set connection risks. | fail→pass | 18,096 | 13,879 | -23% | 1 | 1 | 0% | 3,419 | 2,747 | -20% | 0 | 0 | — |
▸case-19 We are upgrading our Node.js GraphQL API from Apollo Server 3 to Apollo Server 4 @apollo/server. Review official Apollo GraphQL docs and present changes in server lifecycle, impacts on express middleware, concrete npm commands, and unknown plugin risks. | fail→fail | 15,367 | 12,999 | -15% | 1 | 1 | 0% | 3,032 | 2,665 | -12% | 0 | 0 | — |
▸case-20 We need to update our FastAPI backend from Pydantic v1 to Pydantic v2. Consult official Pydantic documentation and detail core model validation changes, impacts on our schema classes, concrete code refactoring steps using BaseModel, and unknown validation edge cases. | fail→fail | 21,323 | 15,005 | -30% | 1 | 1 | 0% | 4,518 | 3,183 | -30% | 0 | 0 | — |
▸case-21 We are upgrading our Android application project to Android Gradle Plugin 8.0 with Java 17. Review official Android Developer documentation and outline build system changes, build.gradle.kts impacts, concrete gradle commands, and unknown library dependency conflicts. | fail→pass | 16,470 | 11,410 | -31% | 1 | 1 | 0% | 3,237 | 2,710 | -16% | 0 | 0 | — |
▸case-22 We need to upgrade our Java microservices from Spring Boot 2.7 to 3.0. Check official Spring migration documentation and summarize framework updates, impacts on javax imports, concrete pom.xml modification steps, and unknown third-party starter risks. | fail→fail | 16,423 | 11,134 | -32% | 1 | 1 | 0% | 3,233 | 2,452 | -24% | 0 | 0 | — |