▸case-01 We are introducing a major architectural change to our payment gateway that involves multi-service updates and high integration risks. Since this task is too large to execute in one go, please process this request using a structured RFC orchestration workflow. I need you to decompose the feature into clear work units with dependencies, quality checks, and merge queue rules, and output an execution log, a unit scorecard, a dependency snapshot, and an integration risk summary. | fail→pass | 31,393 | 20,479 | -35% | 1 | 1 | 0% | 4,063 | 3,950 | -3% | 0 | 0 | — |
▸case-02 Our team is implementing a complex real-time notification engine spanning multiple backend modules and database schemas. Please break down this RFC through a multi-stage unit pipeline, outlining the task units, complexity assessments, stalled-unit recovery strategies, and safe merge validation steps. Ensure the final output includes a dependency map, task scorecards, an execution trail, and a summary of integration risks. | fail→pass | 37,371 | 56,663 | +52% | 1 | 1 | 0% | 6,217 | 6,591 | +6% | 0 | 0 | — |
▸case-03 I have an extensive RFC for migrating our user identity provider across all mobile and web services. Could you run this through an agent-oriented RFC decomposition process? Break it down into modular units with acceptance criteria and rollback plans, enforce strict integration queue management, and deliver an RFC execution log, unit scorecards, a dependency graph snapshot, and an overall risk evaluation report. | fail→pass | 40,150 | 31,007 | -23% | 1 | 1 | 0% | 6,214 | 6,383 | +3% | 0 | 0 | — |
▸case-04 We have an urgent bug fix where a helper function `filterActiveUsers` in `utils/user.js` has an array index off-by-one error. The team wants you to quickly provide the corrected JavaScript code snippet and explanation. Should this bug fix be submitted through a full multi-stage RFC orchestration pipeline with DAG decomposition, unit scorecards, and a merge queue? | pass→pass | 9,535 | 13,869 | +45% | 1 | 1 | 0% | 1,587 | 2,224 | +40% | 0 | 0 | — |
▸case-05 Can you perform a code review on PR #142, which changes variable names from snake_case to camelCase across three helper files? We need feedback on naming clarity and syntax cleanliness. | fail→fail | 3,754 | 9,568 | +155% | 1 | 1 | 0% | 598 | 1,766 | +195% | 0 | 0 | — |
▸case-06 Our product manager needs a draft Product Requirements Document (PRD) detailing user personas, market positioning, and high-level feature ideas for an upcoming marketing landing page. | pass→fail | 22,843 | 19,898 | -13% | 1 | 1 | 0% | 3,161 | 3,357 | +6% | 0 | 0 | — |
▸case-07 We need to update a single validation regex inside `validators/email.ts` to allow subdomains, and we have existing unit tests that deterministically verify this single file. Standard team workflows suggest rating task complexity on a story-point scale from 1 to 13 or a 5-tier matrix. How should this task be classified in terms of complexity tier? | pass→pass | 7,265 | 5,095 | -30% | 1 | 1 | 0% | 1,140 | 1,286 | +13% | 0 | 0 | — |
▸case-08 We are modifying the JSON payload structure returned by three API controller files (`user.ts`, `order.ts`, `billing.ts`), which alters runtime payload structure across modules and carries moderate integration risk. Some team members recommend treating this as Level 1 since no DB schema changed, while others recommend Level 4. How should this task be categorized? | fail→pass | 14,992 | 12,508 | -17% | 1 | 1 | 0% | 2,238 | 2,350 | +5% | 0 | 0 | — |
▸case-09 We are replacing our session-cookie auth with a cross-service OAuth2/OIDC token service, changing architecture, security boundaries, and authorization logic across 12 microservices. The project board defaults to labeling everything as high-priority Tier 5. What complexity tier applies to this change? | fail→pass | 13,779 | 6,970 | -49% | 1 | 1 | 0% | 2,171 | 1,528 | -30% | 0 | 0 | — |
▸case-10 We are drafting the metadata template for decomposing a large search engine overhaul into execution units. Team members proposed using Jira-style fields (`summary`, `assignee`, `story_points`, `priority`, `comments`). Specify the required fields for each unit specification template block. | fail→pass | 16,601 | 10,294 | -38% | 1 | 1 | 0% | 2,768 | 2,037 | -26% | 0 | 0 | — |
▸case-11 When executing an individual task unit within a decomposed RFC, developers usually jump straight from reading the ticket to writing implementation code, followed by writing tests and opening a pull request. Outline the required quality pipeline steps for each individual unit from start to finish. | fail→pass | 15,392 | 13,129 | -15% | 1 | 1 | 0% | 2,497 | 2,490 | -0% | 0 | 0 | — |
▸case-12 During an integration run, Unit B fails its unit tests. Unit C depends on Unit B, but Unit C's code is already complete and passing isolated checks. Is Unit C permitted to merge into the integration queue with an override flag? | pass→pass | 10,657 | 6,257 | -41% | 1 | 1 | 0% | 1,536 | 1,412 | -8% | 0 | 0 | — |
▸case-13 When a developer completes a work unit branch and wants to join the merge queue, standard git practice in our team is to perform a direct merge commit or a squash merge directly into the main branch. What is the required git branch update strategy before queue integration? | pass→pass | 10,901 | 4,380 | -60% | 1 | 1 | 0% | 1,650 | 1,053 | -36% | 0 | 0 | — |
▸case-14 To save CI compute time, developers suggest running full integration tests only at the end of the day after all queue merges complete. What is the rule regarding integration test execution during merge queue processing? | fail→pass | 12,434 | 3,769 | -70% | 1 | 1 | 0% | 1,777 | 875 | -51% | 0 | 0 | — |
▸case-15 Unit #104 in the payment processing DAG has been stuck for 48 hours due to an unexpected deadlock issue. The engineer wants to leave it in the active queue while attempting 3 force-push retries. What are the first two mandatory steps when handling a stalled unit? | pass→pass | 8,471 | 3,485 | -59% | 1 | 1 | 0% | 1,332 | 941 | -29% | 0 | 0 | — |
▸case-16 After a stalled unit has been removed from the queue and its findings snapshotted, what are the remaining two steps to recover the stalled unit and proceed with work? | fail→pass | 6,455 | 2,739 | -58% | 1 | 1 | 0% | 976 | 849 | -13% | 0 | 0 | — |
▸case-17 In our standard developer workflow, engineers write code, run unit tests, write documentation, and ask for code review. Does the unit quality pipeline allow skipping the research phase if the developer believes they understand the task? | fail→pass | 11,504 | 13,326 | +16% | 1 | 1 | 0% | 1,557 | 2,325 | +49% | 0 | 0 | — |
▸case-18 When starting a complex RFC processing workflow, developers often jump directly into allocating tasks to developers or writing code. What are the first two mandatory stages of the RFC pipeline workflow? | fail→pass | 7,757 | 2,539 | -67% | 1 | 1 | 0% | 1,231 | 821 | -33% | 0 | 0 | — |
▸case-19 Once all work units have passed their individual quality checks and merged through the integration queue, what is the final stage of the overall RFC pipeline workflow before signing off? | pass→pass | 8,550 | 2,432 | -72% | 1 | 1 | 0% | 1,235 | 768 | -38% | 0 | 0 | — |
▸case-20 A team lead asks what deliverables must be produced after running an RFC through the pipeline. They expect a Git diff summary and a Slack notification. What artifact records the step-by-step history of the RFC process? | pass→pass | 6,568 | 3,160 | -52% | 1 | 1 | 0% | 1,025 | 846 | -17% | 0 | 0 | — |
▸case-21 When presenting the structural breakdown of a multi-unit RFC, what visual or structured artifact must be generated to document how units relate to each other? | fail→pass | 10,573 | 4,091 | -61% | 1 | 1 | 0% | 1,523 | 946 | -38% | 0 | 0 | — |
▸case-22 Before declaring an RFC execution complete, what specific output report must detail the overall risk profile across merged units and systems? | fail→pass | 7,419 | 1,976 | -73% | 1 | 1 | 0% | 1,181 | 671 | -43% | 0 | 0 | — |
▸case-23 In addition to execution logs and dependency graphs, what specific output artifact tracks the status, metrics, and quality results for individual units? | fail→pass | 10,150 | 2,733 | -73% | 1 | 1 | 0% | 1,633 | 769 | -53% | 0 | 0 | — |