Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →growthxai/output-eval-error-analysis
Systematically review workflow traces to identify failure modes before building evaluators.
Passed 19 of 21 test cases with the skill, and 9 of 21 without it.
and 12 more in the full report
11 fixed9 passed either way2 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 10 test cases the skill fixed. Same model, same prompt, with and without the skill.
We want to download execution details for 20 recent production runs of our email_summarizer workflow into test dataset YAML files. What CLI command should we run to directly generate dataset files pre-populated with input and last_output?
| Checked | Without skill | With skill |
|---|---|---|
| The response specifies using npx output workflow dataset generate email_summarizer --download --limit 20. | Missed | Passed |
| The response notes that generated dataset YAML files are created in tests/datasets/. | Missed | Passed |
The instructions your model reads when the skill loads.
Review real workflow traces and categorize how your workflow fails before writing any evaluators. Evaluators built without error analysis target generic qualities ("is this good?") instead of the specific ways your workflow actually breaks. This skill walks you through the process.
Gather 50-100 representative workflow executions. More traces = more reliable failure categories.
List recent workflow executions and pull their traces:
# List recent runs for a workflow
npx output workflow runs list <workflowName>
# Pull a specific trace as JSON
npx output workflow debug <workflowId> --json
Download production traces directly into dataset YAML files:
# Download up to 20 recent traces as dataset files
npx output workflow dataset generate <workflowName> --download --limit 20
This creates YAML files in tests/datasets/ with the input and last_output fields populated from real executions.
test cases passed, vs 9 / 21 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.