▸case-01 We need to translate our e-commerce dimensional warehouse schema into a Looker LookML setup. Please generate the corresponding semantic data model, including custom calculated measures, drill-down hierarchies, row-level security filters based on regional access policies, and explanatory documentation for our BI developers. | fail→pass | 26,563 | 18,984 | -29% | 1 | 1 | 0% | 6,098 | 4,968 | -19% | 0 | 0 | — |
▸case-02 I have a star-schema dimension model for retail sales along with our standard enterprise business glossary terms. Could you convert this into a Power BI semantic definition? Make sure the output provides the core data model, calculated measures, analytical hierarchies, security filter objects, and comprehensive documentation. | fail→pass | 26,410 | 20,184 | -24% | 1 | 1 | 0% | 6,196 | 5,126 | -17% | 0 | 0 | — |
▸case-03 Please convert our financial data model into a Cube.js schema using our company glossary and access control rules. We require an output that includes the semantic model structure, metric calculation rules, hierarchy definitions, security filter configurations, and user documentation. | fail→pass | 27,413 | 17,631 | -36% | 1 | 1 | 0% | 5,978 | 4,393 | -27% | 0 | 0 | — |
▸case-04 We are provisioning a PostgreSQL data warehouse for our sales team. Please write the SQL DDL statements (CREATE TABLE, PRIMARY KEY, FOREIGN KEY, and CREATE INDEX) for the fact_sales and dim_customer tables. | pass→pass | 11,889 | 10,276 | -14% | 1 | 1 | 0% | 2,533 | 2,696 | +6% | 0 | 0 | — |
▸case-05 We need an orchestration workflow to pull incremental orders from Postgres and populate our dimensional model daily. Please write an Apache Airflow DAG in Python using PostgresOperator. | pass→pass | 16,470 | 17,414 | +6% | 1 | 1 | 0% | 3,469 | 4,129 | +19% | 0 | 0 | — |
▸case-06 Our web application needs a visual dashboard screen for regional managers. Please write a React JSX frontend component using Chart.js that renders a bar chart for monthly sales. | pass→pass | 15,797 | 12,213 | -23% | 1 | 1 | 0% | 3,836 | 3,035 | -21% | 0 | 0 | — |
▸case-07 We have a healthcare patient admissions dimensional model. Generate a Tableau data model semantic definition in JSON containing calculated metrics, patient location hierarchies, row-level security filters for hospital region staff, and documentation. | pass→pass | 18,089 | 16,647 | -8% | 1 | 1 | 0% | 3,773 | 4,075 | +8% | 0 | 0 | — |
▸case-08 Convert our SaaS subscription dimensional model into a dbt metrics layer specification. Supply the response in JSON format including semantic definitions, MRR calculations, customer drill-down hierarchies, security filters, and user documentation. | fail→pass | 21,987 | 19,319 | -12% | 1 | 1 | 0% | 5,434 | 4,853 | -11% | 0 | 0 | — |
▸case-09 Create a Cube.js semantic schema for a multi-tenant B2B analytics platform where tenants can only view their own organization's records. Output JSON schema format with semantic model, calculations, hierarchies, security filters, and documentation. | fail→pass | 19,053 | 16,218 | -15% | 1 | 1 | 0% | 4,234 | 4,316 | +2% | 0 | 0 | — |
▸case-10 Build a Power BI semantic definition JSON for an HR salary analytics model with dynamic row-level security based on user principal names (UPN). Ensure the output contains model structures, measures, hierarchies, security filters, and documentation. | pass→pass | 23,078 | 22,078 | -4% | 1 | 1 | 0% | 5,551 | 5,808 | +5% | 0 | 0 | — |
▸case-11 We have a complex supply chain model with multiple paths between Orders, Shipments, and Warehouses. Generate a Looker semantic layer JSON that resolves fan-out join issues and defines calculations, hierarchies, security, and documentation. | pass→pass | 21,943 | 20,213 | -8% | 1 | 1 | 0% | 4,800 | 4,839 | +1% | 0 | 0 | — |
▸case-12 Translate a Fintech transaction star schema into a dbt metrics layer JSON object. Include calculated measures for Net Revenue Retention and Churn Rate, date-based drill hierarchies, department security filters, and setup documentation. | pass→pass | 20,411 | 18,007 | -12% | 1 | 1 | 0% | 4,733 | 4,743 | +0% | 0 | 0 | — |
▸case-13 We need a Cube.js semantic layer definition JSON for an IoT telemetry stream. Define 7-day rolling average measures, temporal granularity hierarchies, device group security rules, and user documentation. | pass→pass | 19,548 | 17,103 | -13% | 1 | 1 | 0% | 3,514 | 4,233 | +20% | 0 | 0 | — |
▸case-14 Define a Tableau semantic model JSON for retail sales with multi-level product hierarchies (Category -> Subcategory -> SKU) and geographic hierarchies (Region -> State -> Store), alongside calculated profit margins and documentation. | pass→pass | 17,344 | 15,320 | -12% | 1 | 1 | 0% | 3,776 | 3,872 | +3% | 0 | 0 | — |
▸case-15 Translate a multi-currency global sales model into a LookML semantic definition JSON. Include multi-currency conversion calculation rules, region/currency security filters, customer dimension hierarchies, and developer documentation. | pass→pass | 24,602 | 18,886 | -23% | 1 | 1 | 0% | 5,394 | 4,914 | -9% | 0 | 0 | — |
▸case-16 Convert a logistics dimensional model into a Power BI semantic definition JSON using a business glossary that maps 'shipment_id' to 'Consignment Number' and 'lat_long' to 'Delivery Location'. Include calculations, hierarchies, security, and documentation. | pass→pass | 19,195 | 23,817 | +24% | 1 | 1 | 0% | 4,538 | 4,800 | +6% | 0 | 0 | — |
▸case-17 Generate a dbt metrics layer definition JSON for a telecom subscriber churn model. Focus on comprehensive metric calculation documentation, subscriber plan hierarchies, region security filters, and overall setup guides. | pass→pass | 26,591 | 25,708 | -3% | 1 | 1 | 0% | 5,995 | 6,443 | +7% | 0 | 0 | — |
▸case-18 Create a Cube.js semantic model JSON for industrial machinery sensors. Include 95th-percentile temperature calculations, plant location hierarchies, operator access security filters, and model documentation. | pass→pass | 19,214 | 21,260 | +11% | 1 | 1 | 0% | 3,976 | 5,328 | +34% | 0 | 0 | — |
▸case-19 Generate a Looker LookML semantic model JSON for marketing funnel attribution (First Touch, Last Touch, Multi-Touch). Include measure calculations, channel hierarchies, user privacy security filters, and documentation. | pass→pass | 24,446 | 22,745 | -7% | 1 | 1 | 0% | 6,009 | 5,571 | -7% | 0 | 0 | — |
▸case-20 Translate an embedded analytics database for a SaaS portal into a Tableau data model JSON with row-level security enforced per organization account ID, complete with metrics, drill paths, and documentation. | pass→pass | 23,852 | 19,592 | -18% | 1 | 1 | 0% | 5,381 | 4,625 | -14% | 0 | 0 | — |
▸case-21 Generate a Power BI semantic definition JSON for daily inventory snapshot balances. Define semi-additive measures (closing balance), warehouse location hierarchies, facility security rules, and user documentation. | pass→pass | 26,927 | 18,026 | -33% | 1 | 1 | 0% | 4,622 | 4,654 | +1% | 0 | 0 | — |
▸case-22 Convert a mobile gaming user retention dataset into a dbt metrics layer JSON object. Include D1/D7/D30 retention calculation definitions, cohort acquisition date hierarchies, publisher security filters, and documentation. | pass→pass | 13,500 | 11,139 | -17% | 1 | 1 | 0% | 3,139 | 2,919 | -7% | 0 | 0 | — |
▸case-23 Construct a Looker LookML semantic definition JSON for financial P&L reporting. Define account balance rollup hierarchies (Gross Revenue -> Net Revenue -> EBITDA), financial measures, entity security filters, and maintainer documentation. | pass→pass | 25,434 | 18,098 | -29% | 1 | 1 | 0% | 5,800 | 4,554 | -21% | 0 | 0 | — |
▸case-24 Translate a high-throughput ad-tech bid stream dimensional model into a Cube.js schema JSON. Define win rate calculations, advertiser/campaign drill hierarchies, agency security filters, and deployment documentation. | pass→fail | 26,929 | 26,877 | -0% | 1 | 1 | 0% | 6,180 | 6,501 | +5% | 0 | 0 | — |
▸case-25 Generate a Tableau semantic data model JSON for insurance claims processing. Include loss ratio calculations, policy type hierarchies, adjuster region security filters, and technical documentation. | pass→pass | 34,208 | 20,270 | -41% | 1 | 1 | 0% | 5,690 | 4,975 | -13% | 0 | 0 | — |