▸case-15 Video streaming platform project frame: We need to handle video uploads efficiently. Context map: Web portal, storage bucket, transcoding worker pool. No data on file sizes, expected concurrency, or business cost of delays is available. Discover quality attributes and rank them. | fail→fail | 17,362 | 12,383 | -29% | 1 | 1 | 0% | 2,634 | 2,463 | -6% | 0 | 0 | — |
▸case-01 We are preparing to build an automated patient notification system for urgent lab results. Here is our problem background, context map, and initial assumptions regarding high volume spikes and HIPAA compliance requirements. Before our engineering team meets to make any technology selections or write ADRs, please analyze our context and provide a prioritized list of relevant non-functional requirements/quality attributes ranked by business impact and operational risk. Make sure to call out any areas where information is too vague to determine a priority. | fail→pass | 22,316 | 18,966 | -15% | 1 | 1 | 0% | 3,488 | 3,591 | +3% | 0 | 0 | — |
▸case-02 Our team is about to rewrite our core payment processing service to handle international expansion. Attached are our root-cause analysis documents from recent outages and our system context map. Before we begin drafting architecture options or recording design decisions, can you help us discover and rank the key quality attributes for this system based on the evidence provided? Please deliver a structured discovery summary and highlight any gaps where additional clarification is needed. | pass→pass | 19,986 | 20,742 | +4% | 1 | 1 | 0% | 2,937 | 3,850 | +31% | 0 | 0 | — |
▸case-03 We have finished our quality attribute discovery and need to write a formal Architectural Decision Record (ADR 004) selecting Apache Kafka over RabbitMQ for our real-time analytics pipeline. Please draft the ADR including Title, Context, Decision, and Consequences. | pass→fail | 15,138 | 8,132 | -46% | 1 | 1 | 0% | 2,189 | 1,858 | -15% | 0 | 0 | — |
▸case-04 We are starting a new e-commerce project and need a C4 System Context diagram description to show how our web shop interacts with external payment gateways, logistics providers, and customer databases. Please generate the C4 System Context map text description for this system. | pass→fail | 13,799 | 18,375 | +33% | 1 | 1 | 0% | 2,498 | 3,645 | +46% | 0 | 0 | — |
▸case-05 Our quality attribute discovery identified extreme burst scalability (100k req/sec) and low operational overhead as our top priorities. Please evaluate whether we should deploy our microservices on AWS Lambda or Amazon EKS, select the best option, and justify the design choice. | pass→fail | 21,395 | 11,356 | -47% | 1 | 1 | 0% | 3,093 | 2,268 | -27% | 0 | 0 | — |
▸case-06 Here is our e-commerce platform problem frame: During last year's Black Friday sale, database lock contention during checkout caused 40% of transactions to fail, resulting in $2M lost revenue. Context map shows 5 third-party payment gateways. Assumptions state traffic will triple this year. Perform quality attribute discovery for our checkout rewrite, recommend whether we should use Redis caching or Event Sourcing, and provide an ADR. | fail→fail | 24,008 | 13,006 | -46% | 1 | 1 | 0% | 3,581 | 2,734 | -24% | 0 | 0 | — |
▸case-07 We are designing a telehealth video consultation platform. Context map: doctor app, patient mobile app, WebRTC server. Problem frame states: 'Patients complain that video calls disconnect occasionally.' No financial or legal impact data is provided. Rank the non-functional requirements for this platform and state whether security or latency is priority number one. | fail→pass | 15,567 | 11,587 | -26% | 1 | 1 | 0% | 2,342 | 2,284 | -2% | 0 | 0 | — |
▸case-08 Our fintech core ledger frame: Audit compliance requires immutable transaction records with 100% precision. Recent root-cause analysis showed zero double-spend incidents but highlighted slow reconciliation batches taking 6 hours. Context map shows integration with 3 central banks. Perform quality attribute discovery and select whether an append-only event log or relational schema with shadow tables is the right architecture. | fail→pass | 24,364 | 9,876 | -59% | 1 | 1 | 0% | 3,604 | 2,166 | -40% | 0 | 0 | — |
▸case-09 We are building an IoT telemetry ingest platform for smart meters handling 50,000 events per second. Here is our problem frame and context map showing meter agents, MQTT broker, and time-series database. Please provide a standard ISO/IEC 25010 checklist containing all 8 quality attribute categories ranked generically. | pass→pass | 21,736 | 14,047 | -35% | 1 | 1 | 0% | 3,405 | 2,696 | -21% | 0 | 0 | — |
▸case-10 Our fleet management system context map shows 10,000 trucks sending GPS coordinates every 5 seconds. Root cause analysis shows map rendering lag on dispatch screens. Problem frame: Dispatchers lose 15 minutes per shift waiting for map updates. Provide the quality attribute discovery summary for this issue. Make sure your output clearly states its boundary relative to architecture selection. | fail→pass | 16,063 | 9,876 | -39% | 1 | 1 | 0% | 2,587 | 2,055 | -21% | 0 | 0 | — |
▸case-11 We are modernizing an insurance claims system. Background: Claims adjusters want the system to be fast and secure. Context map: Web portal, legacy mainframe backend. Assumptions: Claims volume might grow. Rank Performance, Security, and Scalability by business risk and select the target response time threshold. | fail→fail | 13,505 | 11,330 | -16% | 1 | 1 | 0% | 1,913 | 2,183 | +14% | 0 | 0 | — |
▸case-12 Warehouse picking system. Frame: Mobile scanners freeze when walking between Wi-Fi access points in steel-shielded aisles, causing 2 hours of worker idle time daily. Context map: Handheld scanner app, local edge server, central ERP. Assumptions: Wi-Fi dead zones cannot be eliminated physically. Perform quality attribute discovery and map each candidate directly to the evidence. | pass→pass | 18,308 | 14,417 | -21% | 1 | 1 | 0% | 2,613 | 2,756 | +5% | 0 | 0 | — |
▸case-13 Ride-sharing app location service frame: Real-time driver matching fails when location updates exceed 500ms latency, causing user drop-off. Context map: Driver app, Geo-index microservice, Rider app. Assumptions: 50,000 active drivers per city. Discover and prioritize non-functional requirements, and write a design decision section choosing GeoHash vs H3 spatial indexing. | fail→pass | 25,428 | 11,110 | -56% | 1 | 1 | 0% | 3,853 | 2,396 | -38% | 0 | 0 | — |
▸case-14 Mobile banking app context map: iOS and Android app, OAuth2 gateway, Core Banking API. Frame: Regulatory requirement mandates biometrics and multi-factor auth for transactions over $1,000. RCA shows 5% of users fail biometric auth due to UI timeouts. Assumptions: Security failure leads to heavy banking fines. Perform quality attribute discovery and propose a solution architecture. | fail→pass | 22,731 | 13,800 | -39% | 1 | 1 | 0% | 3,223 | 2,773 | -14% | 0 | 0 | — |
▸case-16 Retail point-of-sale system frame: Store internet outages occur twice a month per store, lasting 30 minutes. During outages, stores currently close registers, losing $5,000 per hour per store across 200 stores. Context map: POS register terminal, local database, cloud backend. Prioritize candidate quality attributes by business risk. | pass→pass | 16,055 | 12,478 | -22% | 1 | 1 | 0% | 2,650 | 2,529 | -5% | 0 | 0 | — |
▸case-17 EHR interoperability gateway frame: Hospital network needs to export patient records to state health registry within 15 minutes of discharge per state law SB-402. RCA: Manual CSV exports take 4 hours and resulted in $50k compliance penalty last month. Context map: EHR database, HL7 FHIR transformer, State API endpoint. Discover quality attributes. | pass→pass | 12,717 | 12,413 | -2% | 1 | 1 | 0% | 2,245 | 2,525 | +12% | 0 | 0 | — |
▸case-18 Media asset management system frame: Photographers upload 50GB RAW image bundles after sports events. Search index updates take 3 hours, delaying photo agency publication and missing news deadlines. Context map: Ingestion endpoint, ElasticSearch index, S3 store. Perform quality attribute discovery and include an explicit boundary statement. | fail→pass | 17,677 | 12,420 | -30% | 1 | 1 | 0% | 2,680 | 2,417 | -10% | 0 | 0 | — |
▸case-19 Smart grid SCADA monitor frame: Sensor nodes send grid frequency readings. System must react in real time to prevent blackouts. Context map: Substation sensors, edge aggregator, central dashboard. No latency numerical threshold or blackout cost data is provided. Discover and prioritize non-functional requirements. | fail→fail | 21,555 | 13,027 | -40% | 1 | 1 | 0% | 2,931 | 2,510 | -14% | 0 | 0 | — |
▸case-20 Developer API gateway frame: External partner API requests spike during morning syncs, causing 504 gateway timeouts for internal mobile apps. RCA: No rate limiting on partner tier. Context map: Partner client, API Gateway, Internal Microservices. Discover quality attributes and write the ADR for Kong Gateway rate-limiting policy. | fail→pass | 23,010 | 13,204 | -43% | 1 | 1 | 0% | 3,493 | 2,734 | -22% | 0 | 0 | — |
▸case-21 Global news site frame: Article pages load in 4.5 seconds in South America due to single-region origin server, causing 35% bounce rate ($100k ad revenue loss monthly). Security audits show zero security breaches in 3 years. Context map: Origin server, global users. Perform quality attribute discovery and order attributes by stakeholder impact. | pass→pass | 16,175 | 10,543 | -35% | 1 | 1 | 0% | 2,483 | 2,235 | -10% | 0 | 0 | — |
▸case-22 Supply chain tracking system frame: We plan to support a large number of shipment updates daily. Context map: Scanner app, Kafka event bus, Postgres backend. Assumptions: Logistics operations run 24/7. Discover quality attributes and prioritize scalability. | fail→fail | 19,969 | 12,678 | -37% | 1 | 1 | 0% | 2,819 | 2,546 | -10% | 0 | 0 | — |