▸case-01 We need to break down our monolith migration project across three separate services (authentication, payment processing, and email notifications). I'm planning to spin up multiple sub-AI workers to execute this faster. Please formulate a clear subagent execution strategy for me. I need to see your initial evaluation of whether delegating these parts makes sense or if I should handle any directly, a breakdown table listing each worker lane with their assigned subtask, designated file boundaries/target areas, forbidden touchpoints, and expected deliverable structures. Also include any shared conventions we must enforce across all workers before starting, and the step-by-step merge verification pipeline for bringing their outputs together. | fail→pass | 29,504 | 24,934 | -15% | 1 | 1 | 0% | 4,050 | 4,402 | +9% | 0 | 0 | — |
▸case-02 I want to delegate a full audit of our open-source AI platform to sub-agents, covering three distinct areas: security vulnerability scanning, performance benchmarking, and API documentation cleanup. Please generate an agent orchestration plan for this effort. Format the output to first evaluate whether any of these tasks should be kept in-house instead of delegated. Then, provide a lane allocation table showing each agent's subtask goal, primary file/directory targets, restricted areas they must avoid, and the exact artifact format expected. Finish with a section listing all baseline assumptions every agent must share upfront, along with a detailed strategy for checking conflicts and integrating their final deliverables. | fail→pass | 26,347 | 27,881 | +6% | 1 | 1 | 0% | 2,782 | 4,866 | +75% | 0 | 0 | — |
▸case-03 We are preparing to analyze three competing enterprise SaaS products by dispatching parallel LLM research instances. Before launching them, I need a comprehensive subagent coordination document. Please outline whether delegating all three targets is practical versus retaining parts for direct handling. Structure your response with a clear lane breakdown table capturing each sub-agent's core deliverable, scope of investigation, explicit boundaries on what not to touch, and output specifications. Conclude with the universal constraints all instances must adopt, followed by a systematic procedure for combining their findings and resolving any contradictory data. | fail→fail | 29,037 | 26,428 | -9% | 1 | 1 | 0% | 4,344 | 4,507 | +4% | 0 | 0 | — |
▸case-04 We want to speed up work on our Python REST API repository `backend-core`. Agent A will refactor `src/auth/jwt.py` to use a new crypto library, while Agent B will write new unit tests for `src/auth/jwt.py`. Should we launch both agents simultaneously in parallel lanes to save time, or arrange them differently? Provide the execution plan and rationale. | pass→pass | 19,219 | 20,369 | +6% | 1 | 1 | 0% | 2,571 | 3,596 | +40% | 0 | 0 | — |
▸case-05 I have a tiny bug in my React app `frontend-dashboard` where a button text needs changing from 'Submit' to 'Save', and its CSS padding increased by 2px in `src/components/Button.tsx`. I'm thinking of creating a subagent to handle this while I work on backend tasks. Should I delegate this fix to a subagent? | fail→pass | 12,680 | 13,770 | +9% | 1 | 1 | 0% | 1,268 | 2,140 | +69% | 0 | 0 | — |
▸case-06 We have two subagents building components for our TypeScript service `data-pipeline`. Agent 1 builds the ingest module in `src/ingest/` and Agent 2 builds the processing engine in `src/engine/`. They need to communicate over an internal event interface. Should we let each agent design its own side of the interface during execution and resolve discrepancies during merge? | pass→pass | 17,061 | 17,654 | +3% | 1 | 1 | 0% | 2,006 | 3,259 | +62% | 0 | 0 | — |
▸case-07 Three parallel research subagents gathered financial metrics for `Acme Corp`. Agent 1 reports annual revenue as $50M, Agent 2 reports $52M, and Agent 3 reports $75M. Should the integration step average these figures to $59M, or how should this discrepancy be handled? | pass→pass | 20,452 | 16,327 | -20% | 1 | 1 | 0% | 2,356 | 2,373 | +1% | 0 | 0 | — |
▸case-08 We have four parallel feature branches created by isolated subagents for the `mobile-app` project. Should we merge all four branches into `main` simultaneously at the end and run a single test suite, or follow a different merge strategy? | pass→pass | 12,604 | 16,236 | +29% | 1 | 1 | 0% | 2,010 | 2,735 | +36% | 0 | 0 | — |
▸case-09 I am writing a brief for a subagent that will update database migration scripts in `db/migrations/`. In my main conversation, I already discussed our custom SQL naming rules and project context. Should I instruct the subagent 'follow the naming conventions as discussed earlier', or how should context be provided? | pass→pass | 13,906 | 7,749 | -44% | 1 | 1 | 0% | 1,502 | 2,121 | +41% | 0 | 0 | — |
▸case-10 We are instructing a subagent to fix a bug in `src/parser/json.ts`. If the subagent discovers while fixing the bug that `src/parser/xml.ts` also needs a redesign, should we encourage the agent to go ahead and refactor `xml.ts` as well to be helpful? | pass→pass | 15,624 | 14,222 | -9% | 1 | 1 | 0% | 1,629 | 2,252 | +38% | 0 | 0 | — |
▸case-11 We are designing a strategic overhaul of our company's product pricing model. Should we delegate the final pricing strategy choice and business synthesis to an autonomous subagent lane while we handle the data collection scripts ourselves? | pass→pass | 18,677 | 17,909 | -4% | 1 | 1 | 0% | 1,940 | 2,486 | +28% | 0 | 0 | — |
▸case-12 I am facing a Git merge conflict in my terminal while merging branch `feature/auth` into `main` on repository `web-app`. Git flags conflicting lines in `package.json` around the `lodash` version dependency. How do I manually resolve this Git conflict using standard git CLI commands? | pass→pass | 13,488 | 8,728 | -35% | 1 | 1 | 0% | 1,838 | 2,537 | +38% | 0 | 0 | — |
▸case-13 How do I write a `docker-compose.yml` file to spin up a PostgreSQL 15 database alongside a Redis 7 instance for local microservice development? | pass→pass | 20,137 | 22,762 | +13% | 1 | 1 | 0% | 2,209 | 3,676 | +66% | 0 | 0 | — |
▸case-14 Write a Python `pytest` fixture in `conftest.py` that sets up a temporary SQLite database connection in memory and tears it down after test completion. | pass→pass | 8,462 | 8,193 | -3% | 1 | 1 | 0% | 1,528 | 2,278 | +49% | 0 | 0 | — |
▸case-15 We are planning two parallel workstreams for `e-commerce-api`. Workstream 1 updates user profile validation in `src/users/`. Workstream 2 adds logging to all routes under `src/`. Is this directory allocation safe for parallel subagent execution? | pass→pass | 15,368 | 8,781 | -43% | 1 | 1 | 0% | 1,698 | 2,118 | +25% | 0 | 0 | — |
▸case-16 When drafting a brief for an agent working on `src/components/Header.tsx`, what explicit restriction section must be included regarding other UI components like `src/components/Footer.tsx`? | pass→pass | 9,811 | 10,361 | +6% | 1 | 1 | 0% | 1,521 | 1,791 | +18% | 0 | 0 | — |
▸case-17 Two subagents finished their isolated modules: Subagent A built `src/auth/service.ts` and passed its unit tests; Subagent B built `src/gateway/router.ts` and passed its unit tests. Can we mark the task complete immediately after individual unit tests pass? | pass→pass | 16,312 | 7,430 | -54% | 1 | 1 | 0% | 1,831 | 2,127 | +16% | 0 | 0 | — |
▸case-18 We are briefing a subagent to audit dependencies in `package.json`. Should we let the subagent choose its output format (such as freeform narrative or markdown bullet points) freely during execution? | pass→pass | 10,805 | 7,846 | -27% | 1 | 1 | 0% | 1,733 | 1,981 | +14% | 0 | 0 | — |
▸case-19 An agent needs to refactor `src/utils/format.ts` according to our internal company code styling rules. Should we link to an internal wiki URL inside the prompt or embed the relevant rules directly in the brief? | pass→pass | 15,421 | 12,232 | -21% | 1 | 1 | 0% | 1,564 | 2,066 | +32% | 0 | 0 | — |
▸case-20 We have 5 small documentation typos across 5 separate Markdown files in `docs/`. Should we launch 5 parallel subagents to fix one typo each? | pass→pass | 13,809 | 5,388 | -61% | 1 | 1 | 0% | 1,336 | 1,775 | +33% | 0 | 0 | — |
▸case-21 I need to orchestrate two parallel subagents to build an internal CLI tool: Agent 1 writes `src/cli/args.ts` and Agent 2 writes `src/cli/output.ts`. Produce the full orchestration design for this build task including do-it-yourself assessment, lane breakdown, shared pins, and merge protocol. | fail→pass | 29,075 | 23,276 | -20% | 1 | 1 | 0% | 5,104 | 4,931 | -3% | 0 | 0 | — |
▸case-22 During a competitive analysis task, Agent A reports Product X supports OAuth2, while Agent B reports Product X only supports Basic Auth. How should the integration step record this in the final report? | pass→pass | 14,054 | 9,099 | -35% | 1 | 1 | 0% | 1,749 | 2,280 | +30% | 0 | 0 | — |
▸case-23 Why does integrating subagent work in step-by-step dependency order lower overall project risks compared to batch-merging all agent branches at the very end of a sprint? | pass→pass | 15,323 | 14,564 | -5% | 1 | 1 | 0% | 2,370 | 2,650 | +12% | 0 | 0 | — |
▸case-24 We are dividing work on a web server project. We have two proposals: Proposal 1 slices tasks by topic ('database work' vs 'API route work'). Proposal 2 slices tasks by physical directory ownership (`src/db/` vs `src/api/`). Which slicing principle prevents agent collisions? | pass→pass | 8,391 | 6,797 | -19% | 1 | 1 | 0% | 1,320 | 1,949 | +48% | 0 | 0 | — |
▸case-25 When generating a subagent brief for updating `src/config/settings.json`, what specific element guarantees the agent stops execution rather than making unprompted architectural guesses? | pass→pass | 10,844 | 6,436 | -41% | 1 | 1 | 0% | 1,590 | 1,692 | +6% | 0 | 0 | — |