Ready to supercharge your AI agents?

Install any skill in seconds. Free to start, no credit card required.

Get Started Free →

Triage CI Failure · asymmetric-al | DecimalAI

testingby asymmetric-al

Triage CI Failure

asymmetric-al/triage-ci-flake

Source asymmetric-al/core ↗License AGPL-3.0

Use when CI tests fail on main branch after PR merge, or when investigating flaky test failures in CI environments

SkillScore68Maintained

Automatic

QA & Test Engineering pack

Last verified 8/20/2026

What it changes

Passed 10 of 15 test cases with the skill, and 7 of 15 without it.

Without skill45%

With skill68%

Checks it passed that the model alone missed

Identifies blocks as the target test suite directory.
Uses pnpm dev:prod fields to start the production server.
Uses pnpm test:int admin with the -t flag specifying the test name.

and 6 more in the full report

8 fixed7 passed either way4 still failing3 broke

Pass-rate lift: +23 pp
Tokens: +251%
Agent turns: 0%
Response time: -51%

Model gemini-3.6-flashMeasured 8/20/2026v115 of 22 cases passed with skill · 15 comparable

Tokens, turns and time describe resource use, not a dollar price.

Use this skill

Save it into your project's skills folder. Claude Code picks it up in its next session.

curl -fsSL --create-dirs -o .claude/skills/asymmetric-al-triage-ci-flake/SKILL.md https://app.decimal.ai/s/asymmetric-al-triage-ci-flake/SKILL.md

Want it pinned and updatable? pip install "decimalai>=0.15", then decimalai skills add asymmetric-al-triage-ci-flake, and start Claude Code with claude --plugin-dir "$(git rev-parse --show-toplevel)/.agents"

1 · One-time — install the CLI (docs ↗)

pip install "decimalai>=0.15"

2 · Get the skill — writes .agents/skills/asymmetric-al-triage-ci-flake/ and pins it in .decimal/skills.lock

decimalai skills add asymmetric-al-triage-ci-flake

No CLI? Get the same file — curl · raw file

curl -fsSL --create-dirs -o .claude/skills/asymmetric-al-triage-ci-flake/SKILL.md https://app.decimal.ai/s/asymmetric-al-triage-ci-flake/SKILL.md

Raw file: /s/asymmetric-al-triage-ci-flake/SKILL.md ↗ · machine-readable summary at /s/asymmetric-al-triage-ci-flake.json ↗

Safety checks: 3 of 3 passed

Code scan CleanIntent PassedContent Clean

See it in action

One of 7 test cases the skill fixed. Same model, same prompt, with and without the skill.

1 of 7

Prompt

I am triaging an integration test failure (Vitest) in the `admin` suite for the test named `should create localized document`. What exact pnpm command should I execute to run only this integration test?

What was checked, without and with the skill
Checked	Without skill	With skill
Uses pnpm test:int admin with the -t flag specifying the test name.	Missed	Passed

case-19 · gemini-3.6-flash · 8/20/2026 · answers are capped and redacted

What's inside

The instructions your model reads when the skill loads.

SKILL.md414 linessingle file

Triage CI Failure

Overview

Systematic workflow for triaging and fixing test failures in CI, especially flaky tests that pass locally but fail in CI. Tests that made it to main are usually flaky due to timing, bundling, or environment differences.

CRITICAL RULE: You MUST run the reproduction workflow before proposing any fixes. No exceptions.

When to Use

CI test fails on main branch after PR was merged
Test passes locally but fails in CI
Test failure labeled as "flaky" or intermittent
E2E or integration test timing out in CI only

MANDATORY First Steps

YOU MUST EXECUTE THESE COMMANDS. Reading code or analyzing logs does NOT count as reproduction.

Extract suite name, test name, and error from CI logs
EXECUTE: Kill port 3000 to avoid conflicts
EXECUTE: pnpm dev $SUITE_NAME (use run_in_background=true)
EXECUTE: Wait for server to be ready (check with curl or sleep)
EXECUTE: Run the specific failing test with Playwright directly (npx playwright test test/TEST_SUITE_NAME/e2e.spec.ts:31:3 --headed -g "TEST_DESCRIPTION_TARGET_GOES_HERE")
If test passes, EXECUTE: pnpm prepare-run-test-against-prod
EXECUTE: pnpm dev:prod $SUITE_NAME and run test again

Only after EXECUTING these commands and seeing their output can you proceed to analysis and fixes.

"Analysis from logs" is NOT reproduction. You must RUN the commands.

Core Workflow

digraph triage_ci {
    "CI failure reported" [shape=box];
    "Extract details from CI logs" [shape=box];
    "Identify suite and test name" [shape=box];
    "Run dev server: pnpm dev $SUITE" [shape=box];
    "Run specific test by name" [shape=box];

Track record

Benchmark

10 / 15

test cases passed, vs 7 / 15 without

Adoption

Users: 0
Times used: 0

Safety and source details

SkillSafety

✓ 3 of 3 passed

Code scan15 of 15 checks clean

0 critical · 0 warning · 0 info — scanner v4 — 8/3/2026

View 15 scanner checks

live_secretremote_code_executionreverse_shelldata_exfiltrationssrf_metadataagent_snoopinganti_refusaldestructive_commandsobfuscated_unicodeunicode_homoglyphopaque_blobsuspicious_urltool_overreachtrigger_abuseprompt_injection_phrasing

IntentSafe

LLM judge · does the skill do what it claims

ContentClean

LLM judge · policy & harmful-content review

Source

Imported

asymmetric-al / core

vendor/payload-upstream/.claude/skills/triage-ci-flake/SKILL.md

View on GitHub ↗

License AGPL-3.0Imported 8/3/2026Version v1

More skills from the same source and nearby in the registry, with their headline benchmark lift.

Ready to supercharge your AI agents?

See it in action

What's inside

Triage CI Failure

Overview

When to Use

MANDATORY First Steps

Core Workflow

Track record

Benchmark

Adoption

SkillSafety

Source

Related skills