Ready to supercharge your AI agents?

Install any skill in seconds. Free to start, no credit card required.

Task: Triage System Verification Failures · bartsoj | DecimalAI

testingby bartsoj

Task: Triage System Verification Failures

bartsoj/TRIAGE.md

Source bartsoj/skill.md ↗License Not recorded

Analyze system verification failures, trace each to its originating design artifact, group related issues into fix batches, and produce an actionable fix plan.

SkillScore86Maintained

Automatic

QA & Test Engineering pack

Last verified 8/6/2026

What it changes

Passed 17 of 20 test cases with the skill, and 10 of 20 without it.

Without skill50%

With skill86%

Checks it passed that the model alone missed

The failure is categorized as a third-party library issue.
The response declines to write source code fixes or patches.
The YAML frontmatter includes the failures_analyzed metric key.

and 11 more in the full report

8 fixed11 passed either way3 still failing0 broke

Pass-rate lift: +36 pp
Tokens: +242%
Agent turns: 0%
Response time: -3%

Model gemini-3.6-flashMeasured 8/6/2026v119 of 22 cases passed with skill · 20 comparable

Tokens, turns and time describe resource use, not a dollar price.

Use this skill

Save it into your project's skills folder. Claude Code picks it up in its next session.

curl -fsSL --create-dirs -o .claude/skills/bartsoj-triage-md/SKILL.md https://app.decimal.ai/s/bartsoj-triage-md/SKILL.md

Want it pinned and updatable? pip install "decimalai>=0.15", then decimalai skills add bartsoj-triage-md, and start Claude Code with claude --plugin-dir "$(git rev-parse --show-toplevel)/.agents"

1 · One-time — install the CLI (docs ↗)

pip install "decimalai>=0.15"

2 · Get the skill — writes .agents/skills/bartsoj-triage-md/ and pins it in .decimal/skills.lock

decimalai skills add bartsoj-triage-md

No CLI? Get the same file — curl · raw file

curl -fsSL --create-dirs -o .claude/skills/bartsoj-triage-md/SKILL.md https://app.decimal.ai/s/bartsoj-triage-md/SKILL.md

Raw file: /s/bartsoj-triage-md/SKILL.md ↗ · machine-readable summary at /s/bartsoj-triage-md.json ↗

Safety checks: 3 of 3 passed

Code scan CleanIntent PassedContent Clean

See it in action

One of 8 test cases the skill fixed. Same model, same prompt, with and without the skill.

1 of 8

Prompt

During integration testing, the web server crashed on startup with a database SSL handshake error because `pg` driver requires `ssl: { rejectUnauthorized: false }` for local staging, which was omitted in `ARCHITECTURE.md` § Database Conventions. What category and root artifact apply?

What was checked, without and with the skill
Checked	Without skill	With skill
The failure is categorized as a third-party library issue.	Missed	Passed
The root artifact is identified as ARCHITECTURE.md § Database Conventions.	Passed	Passed

case-19 · gemini-3.6-flash · 8/6/2026 · answers are capped and redacted

What's inside

The instructions your model reads when the skill loads.

SKILL.md280 linessingle file

Task: Triage System Verification Failures

Objective

Given a SYSTEM_VERIFICATION.md (or equivalent failure report) documenting integration failures in a composed application, produce a TRIAGE.md that traces every failure to the specific design artifact that, if correct, would have prevented the bug — then groups related issues into ordered fix batches and provides specific artifact update instructions. The output enables a human to update the right documents (architecture, interface contract, specs) so the pipeline can re-run and produce correct implementations.

Inputs

System verification report (required) — SYSTEM_VERIFICATION.md or equivalent document listing failures with symptoms, evidence, categories, and affected components.
Architecture documents (auto-discovered) — system structure, endpoint tables, component descriptions, conventions. Used to trace failures to architectural gaps.
INTERFACES.md (auto-discovered) — wire-format contracts. Used to identify missing or incorrect contract entries.
Unit SPECs (auto-discovered) — per-unit specifications. Used to trace whether a spec was wrong, incomplete, or correctly specified but incorrectly implemented.
Unit IMPLEMENTATION.md files (optional) — implementation records. Used to check whether implementations deviated from their specs.
Decision documents (optional) — architectural decisions. Used to understand the rationale behind choices that may have contributed to failures.
units/<area>/ folder structure (auto-discovered) — the existing per-area unit catalog. Used to identify gaps where no unit covers a needed capability; missing capabilities become candidates for new triggers (roadmap/<NNN>-<slug>/ROADMAP.md or issues/<NNN>-<slug>/ISSUE.md).

Workflow

Triage proceeds in four phases: failure analysis, artifact tracing, batch grouping, and fix plan generation.

Phase 1: Failure Analysis

Read the system verification report and build a complete picture of every failure.

For each failure:

Understand the symptom. What specifically went wrong? (HTTP 400, wrong field names, missing endpoint, service won't start, empty response, type error at runtime)
Understand the evidence. What was the expected behavior vs actual behavior? What HTTP responses, error messages, or screenshots were captured?
Identify the boundary. Which components are involved? Which side of the boundary is wrong — or are both wrong because they independently derived an unspecified contract?
Classify the root cause category:

Category	What went wrong	Typical artifact gap
Contract mismatch	Client and server disagree on wire format — field names, casing, structure, types	INTERFACES.md missing or incorrect entry; SPEC derived wire format from prose instead of interface contract
Missing configuration	Env var, migration, library setup, dev script not present	SPEC for infrastructure/scaffold unit incomplete; ARCHITECTURE.md missing config section
Logic bug	Code does the wrong thing despite correct contracts	SPEC was wrong (spec bug) or IMPLEMENTATION deviated from spec (implementation bug)

Track record

Benchmark

17 / 20

test cases passed, vs 10 / 20 without

Adoption

Users: 0
Times used: 0

Safety and source details

SkillSafety

✓ 3 of 3 passed

Code scan15 of 15 checks clean

0 critical · 0 warning · 0 info — scanner v4 — 8/2/2026

View 15 scanner checks

live_secretremote_code_executionreverse_shelldata_exfiltrationssrf_metadataagent_snoopinganti_refusaldestructive_commandsobfuscated_unicodeunicode_homoglyphopaque_blobsuspicious_urltool_overreachtrigger_abuseprompt_injection_phrasing

IntentSafe

LLM judge · does the skill do what it claims

ContentClean

LLM judge · policy & harmful-content review

Source

Imported

bartsoj / skill.md

skills/triage/SKILL.md

View on GitHub ↗

Imported 8/2/2026Version v1

More skills from the same source and nearby in the registry, with their headline benchmark lift.

Ready to supercharge your AI agents?

See it in action

What's inside

Task: Triage System Verification Failures

Objective

Inputs

Workflow

Phase 1: Failure Analysis

Track record

Benchmark

Adoption

SkillSafety

Source

Related skills