Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →miosa-osa/error-analysis
Guided analysis of LLM traces to categorize and pattern-match failures. Reads execution traces, classifies failure modes, detects recurring patterns, and produces actionable taxonomy of errors. For diagnosing why an LLM system fails and where to focus improvement effort.
Passed 18 of 21 test cases with the skill, and 10 of 21 without it.
and 6 more in the full report
9 fixed11 passed either way3 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 9 test cases the skill fixed. Same model, same prompt, with and without the skill.
When searching for recurring failure triggers in `evals/results/`, we want to ignore isolated glitches and only report patterns that happen at least 5 times. Which flag configures this minimum occurrence threshold?
| Checked | Without skill | With skill |
|---|---|---|
| Uses --min-pattern 5 to set the pattern detection threshold. | Missed | Passed |
The instructions your model reads when the skill loads.
Categorize LLM failures by reading traces and detecting patterns.
Guide through systematic analysis of LLM execution traces (eval results, agent logs, or production errors) to categorize failure modes, detect recurring patterns, and quantify their frequency. Produces an error taxonomy with root cause hypotheses and prioritized fix recommendations. Turns a pile of failures into an actionable improvement roadmap.
# Analyze eval results
/error-analysis --traces evals/results/run-42/
# Analyze with existing taxonomy (extend it)
/error-analysis --traces evals/results/ --taxonomy errors/taxonomy.yaml
# Focus on specific failure type
/error-analysis --traces evals/results/ --filter "score < 0.5"
# Analyze agent execution logs
/error-analysis --traces logs/agent-runs/ --format agent-trace
# Compare error patterns between two runs
/error-analysis --traces evals/results/run-42/ --compare evals/results/run-41/
# Sample from large trace sets
/error-analysis --traces evals/results/ --sample 100
| Flag | Type | Default | Description |
|---|---|---|---|
--traces | string | required | Path to trace files or results directory |
--taxonomy | string | — | Existing error taxonomy to extend |
--filter | string | — | Expression to filter traces (e.g., "score < 0.5") |
--format | enum | auto | Trace format: auto, eval-result, agent-trace, jsonl, log |
--compare | string | — | Second trace set for differential analysis |
--sample | int | all | Random sample size for large trace sets |
test cases passed, vs 10 / 21 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.