▸case-01 I need to build an automated pipeline that takes a long-form video transcript, runs a summarization step, extracts key insights, and then generates social media posts and blog drafts from those insights. Can you assemble the right sequence of catalog skills to handle this flow? Please map out the step-by-step data handoffs, include logic for handling errors, show a visual workflow diagram, and define the complete composite skill setup. Go ahead and run this pipeline on a sample transcript, presenting the final output using a timestamped header followed by clear Results and Recommendations sections. | fail→pass | 32,293 | 55,406 | +72% | 1 | 1 | 0% | 5,665 | 4,825 | -15% | 0 | 0 | — |
▸case-02 Can you design and execute a multi-stage workflow for automated competitor analysis? I want to take a competitor's domain name, gather their product specs, analyze their pricing tier structure, and generate a battlecard for our sales team. Please chain together the necessary capabilities so that each step passes its transformed data to the next, incorporating conditional branch logic if data is missing. Provide a diagram of the workflow, the full composite skill definition, and the run results along with actionable next steps. Format the final response under a timestamped header with separate sections for Results and Recommendations. | fail→pass | 52,623 | 28,355 | -46% | 1 | 1 | 0% | 3,960 | 5,438 | +37% | 0 | 0 | — |
▸case-03 I want to orchestrate a customer feedback processing system. The sequence should ingest raw survey comments, perform sentiment analysis, classify issues into categories, and draft escalation emails for negative responses. Help me combine the relevant skills into a unified procedure with automatic output-to-input handoffs, error recovery points, and a structural diagram. Please execute the entire composite pipeline on sample survey data, and display the integrated outputs accompanied by next-step advice, formatted with a timestamp tag above distinct Results and Recommendations sections. | fail→pass | 20,709 | 21,172 | +2% | 1 | 1 | 0% | 3,776 | 4,363 | +16% | 0 | 0 | — |
▸case-04 I have a Python function for calculating Fibonacci numbers: 'def fib(n): return n if n <= 1 else fib(n-1) + fib(n-2)'. Please write a standard pytest suite for this function. Just produce the pytest script directly without setting up multi-step catalog workflows or skill composition frameworks. | pass→fail | 4,639 | 11,771 | +154% | 1 | 1 | 0% | 1,009 | 2,681 | +166% | 0 | 0 | — |
▸case-05 I am building an ETL process using Apache Airflow in Python. Write an Airflow DAG definition script that runs daily at midnight, fetches data from an HTTP API, and writes it to a PostgreSQL database using PostgresHook. Provide raw Python code for Airflow, not skill composition orchestrations. | pass→fail | 11,904 | 14,493 | +22% | 1 | 1 | 0% | 2,202 | 3,285 | +49% | 0 | 0 | — |
▸case-06 In our internal git repo holding the catalog of skills, what JSON schema key should be updated in the repository manifest catalog.json file to add a new category tag 'devops' to an existing skill entry? Answer the schema question directly. | pass→pass | 36,982 | 8,375 | -77% | 1 | 1 | 0% | 985 | 1,768 | +79% | 0 | 0 | — |
▸case-07 We need to integrate an API specification generator with a documentation builder and a client SDK generator for OpenAPI 3.0. A common mistake is just listing the skills side-by-side without defining how data passes from step to step. Make sure to define explicit data handoff transformations between each step in the chain where output from step N feeds step N+1, along with error handling logic. | pass→pass | 37,846 | 34,634 | -8% | 1 | 1 | 0% | 6,420 | 7,245 | +13% | 0 | 0 | — |
▸case-08 Create a composite lead qualification workflow that processes inbound web forms. If the lead score is above 80, route the lead to high-priority CRM sync; if the score is below 80, trigger a nurture email sequence. Base models often forget to include explicit if-then-else branching. Please define this workflow with explicit conditional branch definitions based on step output values. | pass→pass | 22,149 | 22,676 | +2% | 1 | 1 | 0% | 4,772 | 5,025 | +5% | 0 | 0 | — |
▸case-09 Design a technical documentation pipeline that ingests Markdown files, runs spell-checking, checks broken links, and generates HTML artifacts. Base models often omit visual diagrams and just write paragraphs. Make sure to include a visual workflow diagram showing steps and handoffs, followed by full skill composition specs and timestamped header formatting. | fail→pass | 33,463 | 22,474 | -33% | 1 | 1 | 0% | 6,824 | 4,937 | -28% | 0 | 0 | — |
▸case-10 Build a composite workflow for scraping news articles, summarizing key themes, and publishing to Slack. Web scraping often fails due to rate limits or 404 errors, so people usually forget to specify error recovery. Define explicit error handling and fallback points for each step in the multi-step chain. | pass→pass | 36,706 | 26,932 | -27% | 1 | 1 | 0% | 5,888 | 5,317 | -10% | 0 | 0 | — |
▸case-11 I need a composite skill definition for security vulnerability remediation that combines CVE scanning, patch draft generation, and pull request creation. Provide a complete composite skill definition structure that packages these building blocks into a reusable unit, complete with handoff points. | pass→pass | 23,012 | 26,365 | +15% | 1 | 1 | 0% | 3,663 | 4,545 | +24% | 0 | 0 | — |
▸case-12 Compose and execute a workflow for database schema migration safety: step 1 parses a SQL file, step 2 checks for breaking table alterations, and step 3 generates migration rollback scripts. Run this on a sample table schema alter statement, displaying integrated execution results and recommendations. | fail→pass | 36,411 | 21,987 | -40% | 1 | 1 | 0% | 8,243 | 3,871 | -53% | 0 | 0 | — |
▸case-13 Create a composite skill workflow for log file anomaly detection and alerting. Standard LLMs often format section headings as H3 (### Results) or custom titles like # Output. Strictly adhere to standard Markdown H2 tags for the Results and Recommendations sections under the required top-level title. | pass→pass | 15,431 | 21,204 | +37% | 1 | 1 | 0% | 2,522 | 3,695 | +47% | 0 | 0 | — |
▸case-14 Orchestrate a podcast processing pipeline: Step 1 transcribes audio, Step 2 cleans up filler words, Step 3 extracts timestamps for chapters. Explicitly state the data transformation required between Step 1 raw text output and Step 2 text input format, as well as Step 2 to Step 3. | fail→pass | 22,521 | 36,422 | +62% | 1 | 1 | 0% | 3,910 | 6,566 | +68% | 0 | 0 | — |
▸case-15 Design a content repurposing workflow chaining 3 steps: 1) Extract text from PDF whitepapers, 2) Translate text into Spanish, 3) Generate key summary bullet points. Show how the translated text from Step 2 directly feeds as input into Step 3. | fail→pass | 16,810 | 15,681 | -7% | 1 | 1 | 0% | 3,322 | 3,246 | -2% | 0 | 0 | — |
▸case-16 Set up a composite workflow for handling e-commerce order dispute tickets: Step 1 fetches order logs, Step 2 determines refund eligibility based on purchase date, Step 3 drafts customer response. Include conditional logic for eligible versus ineligible orders. | fail→pass | 17,337 | 22,632 | +31% | 1 | 1 | 0% | 3,369 | 4,057 | +20% | 0 | 0 | — |
▸case-17 Build a 3-step performance refactoring pipeline: Step 1 benchmarks Python script execution time, Step 2 rewrites slow functions, Step 3 re-runs benchmarks to verify optimization. Define the handoffs, error recovery if re-run fails benchmark criteria, and visual diagram. | pass→pass | 27,859 | 27,593 | -1% | 1 | 1 | 0% | 4,562 | 4,834 | +6% | 0 | 0 | — |
▸case-18 Construct a multi-step GitHub issue triage pipeline that labels issues, assigns owners based on topic, and auto-posts responses for duplicate issues. Include conditional if-then-else logic for duplicates vs new issues, and wrap everything in a composite skill structure. | pass→pass | 33,311 | 24,528 | -26% | 1 | 1 | 0% | 5,857 | 4,324 | -26% | 0 | 0 | — |
▸case-19 Orchestrate a financial report auditing composite workflow: Step 1 parses PDF balance sheets, Step 2 verifies arithmetic totals, Step 3 flags discrepancies. Show data transformation formats between steps and execute on sample balance sheet figures. | fail→pass | 33,671 | 25,224 | -25% | 1 | 1 | 0% | 7,428 | 5,786 | -22% | 0 | 0 | — |
▸case-20 Combine localization skills into a workflow: Step 1 translates software strings, Step 2 checks string length constraints, Step 3 flags truncated items. Define error recovery when string length exceeds UI limits. | fail→pass | 16,668 | 19,329 | +16% | 1 | 1 | 0% | 2,690 | 3,895 | +45% | 0 | 0 | — |
▸case-21 Design a candidate screening workflow: Step 1 extracts resume skills, Step 2 compares skills against job description requirements, Step 3 categorizes candidate fit (Strong, Moderate, Low). Include conditional handoffs to trigger interview scheduling for 'Strong' candidates. | fail→pass | 20,136 | 22,994 | +14% | 1 | 1 | 0% | 3,406 | 4,394 | +29% | 0 | 0 | — |
▸case-22 Build an incident postmortem generation workflow: Step 1 aggregates Datadog alert logs, Step 2 identifies root cause timeline, Step 3 formats postmortem Markdown document. Provide a visual workflow diagram and data transformation mapping. | pass→pass | 23,316 | 32,445 | +39% | 1 | 1 | 0% | 3,885 | 5,669 | +46% | 0 | 0 | — |