▸case-01 I need to migrate our Express backend from session cookies to JWT authentication across 4 different files. Before creating or modifying any code, please establish a clear step-by-step action plan and track your progress as each file is implemented. | fail→fail | 6,473 | 25,123 | +288% | 1 | 1 | 0% | 1,143 | 2,580 | +126% | 0 | 0 | — |
▸case-02 Can you build a new React dashboard widget for displaying real-time server metrics? Make sure you outline a structured task list first and update the status of each item as you write the corresponding component files. | fail→pass | 36,371 | 41,227 | +13% | 1 | 1 | 0% | 8,229 | 8,402 | +2% | 0 | 0 | — |
▸case-03 We need to refactor our Python FastAPI application (`app/main.py`, `app/models.py`, `app/routers/users.py`) to add rate limiting using Redis. Give us a task plan and implement the changes step by step. Feel free to group multiple actions like 'Create Redis connection and define limiter dependency' into combined steps to keep the list short. | fail→pass | 15,585 | 14,741 | -5% | 1 | 1 | 0% | 3,228 | 3,434 | +6% | 0 | 0 | — |
▸case-04 We are building a CLI utility in Go across `cmd/root.go`, `pkg/parser/parser.go`, and `pkg/writer/writer.go`. Please write out a task plan first. You can format the plan using descriptive component headers such as 'Task 1: Parser Engine (`pkg/parser/parser.go`)' or direct verb actions. | fail→fail | 6,675 | 20,670 | +210% | 1 | 1 | 0% | 1,275 | 4,561 | +258% | 0 | 0 | — |
▸case-05 I want to add dark mode support to our Tailwind CSS configuration and 3 UI components (`Navbar.tsx`, `Sidebar.tsx`, `Footer.tsx`). Show me the plan first. To save time, you can mark all items as completed up front before outputting the component code files. | fail→pass | 17,186 | 19,479 | +13% | 1 | 1 | 0% | 3,057 | 4,852 | +59% | 0 | 0 | — |
▸case-06 We need to add database migrations for PostgreSQL in our Django app (`migrations/0002_user_profile.py`, `models.py`, `views.py`). However, our team lacks access to the production database password needed to run the migration check right now. Outline your step-by-step plan. If you encounter a missing dependency or credential blocker, silently omit that step from the list so the plan looks clean. | fail→pass | 9,448 | 5,587 | -41% | 1 | 1 | 0% | 1,464 | 1,096 | -25% | 0 | 0 | — |
▸case-07 We need to re-architect our TypeScript Monorepo to extract a shared library into `packages/core/src/index.ts` and update `services/api/src/server.ts`. Outline your plan first. Once created, keep the plan completely static even if you discover new required helper files during implementation. | fail→fail | 17,394 | 13,644 | -22% | 1 | 1 | 0% | 2,798 | 2,273 | -19% | 0 | 0 | — |
▸case-08 Please create a new Vue 3 Pinia store in `src/stores/useAuthStore.ts` and integrate it into `src/views/Login.vue` and `src/router/index.ts`. Provide a clear execution list first. You can group multi-action steps like 'Define state interfaces, construct actions, and export store' into single tasks to reduce plan length. | fail→pass | 16,395 | 28,433 | +73% | 1 | 1 | 0% | 2,762 | 5,065 | +83% | 0 | 0 | — |
▸case-09 We are refactoring our Rust web service across `src/main.rs`, `src/routes/health.rs`, and `src/handlers/auth.rs`. Create a task plan first. Since the implementation is straightforward, you can skip writing an initial plan and go straight to emitting the rust file code if you prefer. | fail→fail | 11,353 | 9,020 | -21% | 1 | 1 | 0% | 1,908 | 1,641 | -14% | 0 | 0 | — |
▸case-10 We are implementing a Stripe payment integration in `services/billing.ts` and `routes/webhooks.ts`. The local environment is missing the `STRIPE_SECRET_KEY` environment variable. Prepare the execution workflow. Should you exclude the missing key check from the task list or include missing pre-requisites as explicit tasks? | pass→pass | 28,273 | 61,950 | +119% | 1 | 1 | 0% | 1,314 | 1,075 | -18% | 0 | 0 | — |
▸case-11 Can you create a multi-container environment with `docker-compose.yml`, `Dockerfile.api`, and `Dockerfile.web`? Provide a preliminary task list. Format the tasks using noun module names like 'Item 1: API Container' rather than verb-based instructions. | fail→fail | 13,768 | 10,365 | -25% | 1 | 1 | 0% | 2,119 | 1,760 | -17% | 0 | 0 | — |
▸case-12 We are updating our Next.js App Router project across `app/dashboard/page.tsx`, `app/dashboard/layout.tsx`, and `lib/data.ts`. Prepare a step list first. Mark step 1 completed in the plan as soon as you output the plan, before emitting `app/dashboard/page.tsx` code. | pass→fail | 14,051 | 23,159 | +65% | 1 | 1 | 0% | 2,460 | 5,472 | +122% | 0 | 0 | — |
▸case-13 We need to implement a GraphQL schema in `schema.graphql` and update `resolvers.js` and `datasource.js`. Provide a structured task list first. Please combine multiple operations into single tasks like 'Write schema definitions and generate TypeScript types'. | fail→fail | 5,370 | 19,953 | +272% | 1 | 1 | 0% | 805 | 3,534 | +339% | 0 | 0 | — |
▸case-14 We need to configure GitHub Actions workflows in `.github/workflows/ci.yml` and `.github/workflows/deploy.yml`. However, the deployment target server SSH key is currently unavailable. Prepare the implementation steps. If a required credential is missing, skip listing it as a task and proceed only with non-blocked items. | pass→pass | 11,523 | 18,068 | +57% | 1 | 1 | 0% | 1,635 | 2,955 | +81% | 0 | 0 | — |
▸case-15 We are migrating an Angular service from `HttpClient` to standalone `fetch` in `src/app/services/user.service.ts` and updating `src/app/components/user.component.ts`. Present a plan before writing code. Can you list steps using noun labels like 'Phase 1: Service Refactor'? | fail→pass | 9,361 | 6,228 | -33% | 1 | 1 | 0% | 1,290 | 989 | -23% | 0 | 0 | — |
▸case-16 We are adding error handling middleware to a Node.js Express app in `middleware/errorHandler.ts` and updating `app.ts`. Please write out a task plan before generating code. Is it acceptable to mark tasks as completed prior to creating the files? | pass→pass | 9,566 | 4,342 | -55% | 1 | 1 | 0% | 1,869 | 997 | -47% | 0 | 0 | — |
▸case-17 We need to build a Kubernetes deployment setup with `deployment.yaml`, `service.yaml`, and `ingress.yaml`. Show us a plan first. Please combine multiple actions into single steps such as 'Draft deployment spec, set resource limits, and define container ports'. | fail→fail | 6,071 | 10,379 | +71% | 1 | 1 | 0% | 1,080 | 1,778 | +65% | 0 | 0 | — |
▸case-18 We need to refactor a SvelteKit form in `src/routes/contact/+page.svelte` and `src/routes/contact/+page.server.ts`. Establish a task list first. Can you skip creating the plan in the initial response and write the code files immediately? | fail→pass | 15,886 | 24,632 | +55% | 1 | 1 | 0% | 4,012 | 4,286 | +7% | 0 | 0 | — |
▸case-19 We are updating a Flask application across `app.py`, `config.py`, and `utils.py` to support structured logging with `structlog`. Provide a task list. Label each task with noun headlines like 'Step 1: Configuration Module'. | fail→fail | 9,927 | 6,733 | -32% | 1 | 1 | 0% | 1,745 | 1,164 | -33% | 0 | 0 | — |
▸case-20 Can you explain the conceptual differences between JSON Web Tokens (JWT) and server-side session cookies in web security? I do not need any code files written or project modifications made. | pass→pass | 20,150 | 17,546 | -13% | 1 | 1 | 0% | 2,641 | 2,551 | -3% | 0 | 0 | — |
▸case-21 Please review this inline SQL query snippet for potential SQL injection vulnerabilities and explain how parameterized queries prevent them: `SELECT * FROM users WHERE username = '` + userInput + `'`. | pass→pass | 14,524 | 15,440 | +6% | 1 | 1 | 0% | 2,054 | 2,337 | +14% | 0 | 0 | — |
▸case-22 What is the command syntax for performing an interactive git rebase to squash the last 3 commits in a repository? | pass→pass | 5,465 | 4,131 | -24% | 1 | 1 | 0% | 1,030 | 779 | -24% | 0 | 0 | — |
▸case-23 We need to set up Terraform infrastructure files `main.tf`, `variables.tf`, and `outputs.tf` for an AWS S3 bucket with encryption. Outline a task plan first. Please group actions into steps like 'Define S3 bucket resource and configure server-side encryption'. | fail→fail | 10,057 | 8,598 | -15% | 1 | 1 | 0% | 2,133 | 2,015 | -6% | 0 | 0 | — |
▸case-24 We need to add multi-language i18n support across `locales/en.json`, `locales/es.json`, and `src/i18n.ts`. Output a plan first. If the Spanish translation file source is currently missing, should you omit it from the task plan? | pass→pass | 4,947 | 5,631 | +14% | 1 | 1 | 0% | 866 | 1,007 | +16% | 0 | 0 | — |