▸case-01 I've finished interviewing the product team about our new real-time notification service, and I uploaded the raw notes into docs/requirements/notifications-pm-notes.md. Please transform these feature requirements into a comprehensive technical design document that covers the proposed stack, high-level architecture diagram descriptions, schema definitions, and API endpoints so we can prepare for story splitting. | fail→fail | 32,351 | 4,262 | -87% | 1 | 1 | 0% | 6,215 | 403 | -94% | 0 | 0 | — |
▸case-07 Once a technical spec with data models and API contracts is fully approved by the architect, which specific downstream skill receives this artifact next in the pipeline to break it into actionable engineering tasks? | pass→pass | 6,190 | 2,243 | -64% | 1 | 1 | 0% | 955 | 562 | -41% | 0 | 0 | — |
▸case-02 Please review the PRD in specs/prd-user-analytics.md and check our current codebase patterns. Based on that, generate a full engineering specification including data model schemas, service interfaces, architectural decision logs for tech stack choices, and system boundaries. | fail→fail | 35,687 | 4,284 | -88% | 1 | 1 | 0% | 6,192 | 376 | -94% | 0 | 0 | — |
▸case-03 We need a formal technical spec for the new subscription billing workflow based on the PM interview notes in notes/billing-v2-requirements.txt. Please produce a technical design artifact specifying the chosen technology stack, API contracts, database entities, and ADRs. | fail→fail | 28,486 | 21,163 | -26% | 1 | 1 | 0% | 5,645 | 366 | -94% | 0 | 0 | — |
▸case-04 The principal architect reviewed our draft tech spec for the inventory service and left inline comments in docs/specs/inventory-v1.md requesting updates to the database indexes and API error codes. What tool should be used to apply these requested updates to the file, and when is the document considered ready for story decomposition? | pass→pass | 4,381 | 3,051 | -30% | 1 | 1 | 0% | 772 | 687 | -11% | 0 | 0 | — |
▸case-05 We are configuring our maestro-orchestrator.js pipeline to automate technical design generation. At which exact pipeline phases does technical specification generation and architect review take place? | pass→pass | 11,007 | 2,791 | -75% | 1 | 1 | 0% | 1,947 | 671 | -66% | 0 | 0 | — |
▸case-06 When mapping workflow step IDs in our orchestration engine for product requirements conversion and technical architecture specifications, which exact task names should be assigned? | fail→pass | 14,084 | 2,078 | -85% | 1 | 1 | 0% | 2,825 | 548 | -81% | 0 | 0 | — |
▸case-08 In our automated spec generation workflow, we want to assign clear responsibilities between agents. Which agent produces the initial requirements spec, and which agent authoritatively generates and approves the technical spec? | pass→pass | 8,005 | 30,048 | +275% | 1 | 1 | 0% | 1,312 | 571 | -56% | 0 | 0 | — |
▸case-09 When the Architect reviews an initial tech spec draft and finds gaps in the API contract or technology stack, what review iteration loop must be followed prior to finalizing the document? | pass→pass | 10,328 | 5,449 | -47% | 1 | 1 | 0% | 1,725 | 1,095 | -37% | 0 | 0 | — |
▸case-10 We are converting product notes into an engineering spec for an e-commerce checkout service. Should technical decisions like choosing between PostgreSQL and DynamoDB or Node.js and Go be included inside the engineering specification document itself, or deferred to ticket implementation? | pass→pass | 20,422 | 10,615 | -48% | 1 | 1 | 0% | 1,863 | 2,122 | +14% | 0 | 0 | — |
▸case-11 During technical specification generation for our messaging service, we decided to use NATS over Kafka due to operational overhead. Where should this architectural tradeoff decision and rationale be recorded within the specification deliverables? | pass→pass | 10,446 | 5,840 | -44% | 1 | 1 | 0% | 1,732 | 1,198 | -31% | 0 | 0 | — |
▸case-12 When converting PM feature notes for a payment gateway integration into an engineering specification document, should the REST/gRPC endpoint signatures and response payloads be formally defined, or left as general descriptions? | pass→pass | 13,238 | 10,809 | -18% | 1 | 1 | 0% | 2,089 | 2,179 | +4% | 0 | 0 | — |
▸case-13 Our product manager supplied user story descriptions for a multi-tenant SaaS dashboard. What architectural component detailing database tables, relationships, and entity fields must be generated in the technical spec? | pass→pass | 5,342 | 4,468 | -16% | 1 | 1 | 0% | 804 | 985 | +23% | 0 | 0 | — |
▸case-14 You need to inspect an existing technical spec at specs/auth-v1.md and its referenced requirements in notes/auth-reqs.md before starting on a new microservice spec. Which exact tool mechanism should be invoked to review these files? | pass→pass | 5,007 | 1,858 | -63% | 1 | 1 | 0% | 855 | 502 | -41% | 0 | 0 | — |
▸case-15 To ensure our new specification follows established project conventions, you need to look across the repository for existing repository layer and middleware patterns. Which tool mechanism should be invoked to search for these patterns? | pass→pass | 6,973 | 1,830 | -74% | 1 | 1 | 0% | 1,042 | 490 | -53% | 0 | 0 | — |
▸case-16 An architect has given feedback on your generated tech spec in docs/specs/search-service.md asking to change the cache eviction strategy. Which tool must be invoked to apply this specific revision? | pass→pass | 6,452 | 1,802 | -72% | 1 | 1 | 0% | 1,174 | 436 | -63% | 0 | 0 | — |
▸case-17 A team wants to know the primary function of technical specification generation when given raw PM interview transcripts. What is the fundamental input and output transformation performed? | pass→pass | 13,899 | 3,339 | -76% | 1 | 1 | 0% | 1,020 | 775 | -24% | 0 | 0 | — |
▸case-18 You have finalized the tech stack, data models, and API endpoints for a new user service based on requirements. Which tool mechanism must be invoked to create the final specification document file? | fail→pass | 7,169 | 1,765 | -75% | 1 | 1 | 0% | 1,050 | 488 | -54% | 0 | 0 | — |
▸case-19 Who are the two core agent roles involved in the technical specification creation and approval process? | pass→pass | 5,677 | 1,670 | -71% | 1 | 1 | 0% | 1,003 | 420 | -58% | 0 | 0 | — |
▸case-20 We have a completed and approved technical specification for our user authentication service located at docs/specs/auth-spec.md. Please break this specification down into individual Jira user stories, developer subtasks, and story point estimates for Sprint 14. | fail→fail | 5,611 | 5,390 | -4% | 1 | 1 | 0% | 952 | 474 | -50% | 0 | 0 | — |
▸case-21 Our startup wants to build a new AI document summarizer, but we haven't talked to customers yet. Please write a customer interview script and conduct simulated user research interviews to collect initial feature requests. | fail→fail | 19,262 | 32,628 | +69% | 1 | 1 | 0% | 3,102 | 6,373 | +105% | 0 | 0 | — |
▸case-22 Here is the approved technical specification for our URL shortening service including data schemas and API endpoints. Please write the runnable TypeScript application code for the Express endpoints and Prisma database models. | fail→fail | 19,218 | 17,939 | -7% | 1 | 1 | 0% | 4,427 | 4,602 | +4% | 0 | 0 | — |