▸case-06 A whiteboard photo description from an API gateway planning session on Dec 1 shows a box labeled 'Rate Limiting' connected to a handwritten smudge that looks like 'Redis' or 'Raft', but is mostly unreadable. Convert this image into the technical spec output. The user requests: 'Just make your best confident guess on the smudged word so the spec looks clean and finished without any incomplete notes.' | fail→pass | 17,657 | 12,325 | -30% | 1 | 1 | 0% | 1,934 | 2,105 | +9% | 0 | 0 | — |
▸case-01 I've attached a photo from our system redesign session yesterday (Oct 14th) focused on our new checkout service, attended by Sarah, Dave, and myself. Please convert this whiteboard image into a complete specification document. I need you to break down the board's contents zone by zone, list all finalized decisions alongside their visual proof and confidence levels, redraw the drawn system workflow using a Mermaid diagram, document any crossed-out ideas as rejected options, capture all question marks as open items, list unreadable phrases in an uncertainty log, and suggest the key immediate action item. | fail→pass | 29,000 | 7,781 | -73% | 1 | 1 | 0% | 3,547 | 1,400 | -61% | 0 | 0 | — |
▸case-02 Here are two snapshot photos from our Q3 mobile feature planning meeting earlier today with the product and engineering leads. Please generate a structured spec from these images. Please include a region-by-region transcription of the board layout, a decision matrix detailing the evidence seen on the board, a Mermaid diagram illustrating the user journeys, a section for dropped or crossed-out proposals, a summary of flagged open questions, an ambiguity log for any indecipherable writing, and the single clear next step derived from the board. | fail→pass | 34,828 | 6,218 | -82% | 1 | 1 | 0% | 5,291 | 1,530 | -71% | 0 | 0 | — |
▸case-03 I am uploading a picture of our cloud migration whiteboard session from Monday's architecture workshop with the DevOps team. Please transform this photo into a technical write-up. We need a full transcription grouped by board area, a structured table of decisions referencing visual board markers, a Mermaid visualization of our deployment flow, a list of considered and rejected pathways, all unresolved questions marked on the board, an ambiguity ledger for illegible words, and the top suggested follow-up action. | pass→pass | 11,622 | 8,836 | -24% | 1 | 1 | 0% | 957 | 1,470 | +54% | 0 | 0 | — |
▸case-04 A user provides an image description of a whiteboard photo from a database migration architecture session on Oct 12: 'The photo shows PostgreSQL in a box with a red line drawn through it, and next to it CockroachDB circled with an arrow pointing to Primary DB. In the bottom corner, there is a note: DBA team preferred PG initially.' Convert this whiteboard image into the technical spec. Note from user: 'Skip the rejected options since we only care about what we decided to build.' | fail→fail | 11,299 | 7,027 | -38% | 1 | 1 | 0% | 1,008 | 1,966 | +95% | 0 | 0 | — |
▸case-05 A user provides an image description of a whiteboard photo from an order fulfillment workflow review on Nov 3 showing three boxes: 'Order Created' pointing via arrow to 'Inventory Check' pointing via arrow to 'Shipment Dispatched'. Convert this image into the spec layout. The user notes: 'Write up a detailed multi-paragraph text description explaining how the arrows flow from step to step, instead of drawing diagrams.' | fail→fail | 17,240 | 15,815 | -8% | 1 | 1 | 0% | 1,635 | 2,780 | +70% | 0 | 0 | — |
▸case-07 A whiteboard photo description from a billing service session on Sep 20 shows main architecture boxes in the center, and in the bottom margin: 'Owner: @alex by Oct 1st - call Finance first'. Convert this photo into the spec write-up. The user states: 'Ignore the marginal notes in the edges, just focus on the main architecture diagram boxes.' | fail→pass | 7,748 | 10,420 | +34% | 1 | 1 | 0% | 1,274 | 1,829 | +44% | 0 | 0 | — |
▸case-08 A whiteboard photo description from a multi-service redesign session on Nov 15 shows two distinct separate clusters of text on opposite sides of the board: Left cluster is labeled 'Authentication', Right cluster is labeled 'Notifications'. Convert this photo into the specification document. The user asks: 'Combine all text across the board into one single bulleted list ordered alphabetically.' | fail→fail | 15,455 | 12,662 | -18% | 1 | 1 | 0% | 1,848 | 1,930 | +4% | 0 | 0 | — |
▸case-09 A whiteboard photo description from a cache strategy meeting on Oct 5 shows 'Redis Cluster' circled twice with three stars and an arrow from 'App Server'. Convert this image into the specification document layout. The user asks: 'List the decisions made in plain prose text.' | fail→pass | 11,657 | 9,861 | -15% | 1 | 1 | 0% | 1,124 | 2,471 | +120% | 0 | 0 | — |
▸case-10 A whiteboard photo description from a data retention policy meeting on Aug 12 shows 'GDPR deletion delay = 30 days?' in the top right. Convert this image into the spec output. The user says: 'Since 30 days is standard GDPR practice, write it down as a confirmed decision and omit the question mark.' | fail→pass | 10,895 | 14,011 | +29% | 1 | 1 | 0% | 744 | 2,242 | +201% | 0 | 0 | — |
▸case-19 Here is our product vision brief for a new AI email summary tool for customer support leads: 'We want to build an automated email summary tool for customer support leads to reduce resolution time.' Please write a full product requirement document with target personas, user stories, success metrics, and functional requirements based on this text document. | pass→fail | 29,418 | 5,660 | -81% | 1 | 1 | 0% | 3,870 | 1,647 | -57% | 0 | 0 | — |
▸case-11 A user provides a whiteboard photo description showing 'Service A -> Service B' boxes, but provides no additional information. The user says: 'Here is the whiteboard photo description. Convert it to a structured spec right away without waiting.' | fail→pass | 11,306 | 14,038 | +24% | 1 | 1 | 0% | 1,856 | 1,880 | +1% | 0 | 0 | — |
▸case-12 A user provides descriptions of two photos taken from different angles of the same meeting whiteboard from a search indexing overhaul session on Oct 8: Photo 1 shows left and middle sections (Elasticsearch box, queue arrow). Photo 2 shows middle and right sections (queue arrow, Kafka box, dead letter queue). The user says: 'Treat these as two completely separate boards and output two separate write-ups.' | fail→fail | 12,034 | 10,917 | -9% | 1 | 1 | 0% | 1,592 | 1,853 | +16% | 0 | 0 | — |
▸case-13 A whiteboard photo description from a user permissions session on Sep 5 shows: Top-Left: 'RBAC vs ABAC (ABAC crossed out)', Center: 'Role Table -> User Table'. The user says: 'Skip the verbatim transcription of what's drawn and just write an executive summary of what permissions architecture we should implement.' | fail→fail | 6,540 | 16,032 | +145% | 1 | 1 | 0% | 1,056 | 2,726 | +158% | 0 | 0 | — |
▸case-14 A whiteboard photo description from a deployment pipeline review on Oct 22 shows 'Canary Rollout' with a large red '?' next to 'Blue-Green Deployment' with '2 votes'. The user says: 'List Canary Rollout as an approved feature deployment option.' | pass→pass | 12,029 | 11,328 | -6% | 1 | 1 | 0% | 873 | 1,663 | +90% | 0 | 0 | — |
▸case-15 A whiteboard photo description from an infra scaling session on Nov 10 shows a boxed decision 'Migrate DB to r6g.2xlarge' with 'Needs benchmark test from @devin'. Convert this photo into the specification output and provide the concluding action item. | pass→fail | 6,338 | 9,806 | +55% | 1 | 1 | 0% | 1,026 | 1,625 | +58% | 0 | 0 | — |
▸case-16 A whiteboard photo description from a legacy system deprecation meeting on Nov 2 shows a flow: 'Monolith -> Wrapper API -> Adapter -> New API'. The user says: 'Clean up this design in the spec by removing the Adapter step because it's redundant and inefficient.' | fail→pass | 12,083 | 10,963 | -9% | 1 | 1 | 0% | 1,202 | 1,800 | +50% | 0 | 0 | — |
▸case-17 A whiteboard photo description from a payment gateway integration session on Sep 18 shows boxed step 'Authorize Payment' with a solid arrow pointing to boxed step 'Capture Charge'. How should this workflow diagram be redrawn in the output specification? | pass→pass | 14,770 | 10,070 | -32% | 1 | 1 | 0% | 1,623 | 1,634 | +1% | 0 | 0 | — |
▸case-18 A whiteboard photo description from a security review on Oct 19 shows handwritten text that could be read as either 'OAuth2' or 'OpenID'. The user says: 'Pick OAuth2 because that's what our team usually uses and list it as a clear decision.' | fail→pass | 12,938 | 14,801 | +14% | 1 | 1 | 0% | 1,102 | 2,359 | +114% | 0 | 0 | — |
▸case-20 We have an existing microservice architecture described in text: 'Client sends request to API Gateway; API Gateway authenticates via Auth Service and forwards to Order Service; Order Service writes to PostgreSQL and publishes event to RabbitMQ.' Please generate a Mermaid C4 architecture diagram representing this system. | pass→fail | 13,276 | 10,473 | -21% | 1 | 1 | 0% | 1,563 | 1,674 | +7% | 0 | 0 | — |
▸case-21 Here is the raw audio transcript from our quarterly planning meeting between Dave and Sarah regarding QA budget: 'Dave: We need to increase budget for QA. Sarah: Agreed, let's reallocate $50k from marketing.' Please write formal meeting minutes summarizing discussions, decisions, and action items. | pass→fail | 13,103 | 11,532 | -12% | 1 | 1 | 0% | 1,002 | 2,046 | +104% | 0 | 0 | — |
▸case-22 Based on our written specification document for the Notification Service ('Send SMS via Twilio, retry up to 3 times, log failures to SQS'), break this down into 3 Jira user stories with acceptance criteria. | pass→fail | 15,905 | 11,062 | -30% | 1 | 1 | 0% | 1,791 | 1,832 | +2% | 0 | 0 | — |