Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →datadog/dd-triage-flaky-test
Load when investigating a specific flaky test. Gets history, failure pattern, and category, then recommends fix, quarantine, or escalate.
Passed 17 of 20 test cases with the skill, and 9 of 20 without it.
and 13 more in the full report
10 fixed10 passed either way3 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 10 test cases the skill fixed. Same model, same prompt, with and without the skill.
We want to triage the test `TestPaymentRefund` in Datadog, but when searching for flakiness under repository `github.com/acme/store-api` no results were returned. To verify whether the repo ID in Datadog CI Visibility might be different, what pup query search fallback should be executed?
| Checked | Without skill | With skill |
|---|---|---|
| The search query executes pup cicd tests search with @test.name:"TestPaymentRefund". | Missed | Passed |
| The query parameter sets a 30-day lookback window with --from 30d. | Missed | Passed |
| The repo ID is extracted from the @git.repository.id_v2 field in the returned search results. | Missed | Passed |
The instructions your model reads when the skill loads.
One-line summary: Investigate a specific flaky test — get history, failure pattern, and category, then recommend fix, quarantine, or escalate.
Requires: dd-pup skill (pup CLI installed and authenticated).
| Parameter | Description |
|---|---|
| Test name | Fully qualified test name (e.g. TestMyFunc or com.example.MyTest) |
| Repository | Lowercase, no-schema URL (e.g. github.com/org/repo). Derive from git remote get-url origin if not provided. |
Derive repository ID from git if not provided:
git remote get-url origin
# Strip protocol and trailing .git, then lowercase the result
# e.g. https://github.com/DataDog/my-repo.git → github.com/datadog/my-repo
Validation fallback: If STEP 1 returns no results, confirm the correct repository by searching without a repo filter:
pup cicd tests search \
--query "@test.name:\"<test-name>\"" \
--from 30d \
--limit 5
Extract @git.repository.id_v2 from results and retry STEP 1 with the confirmed value.
Preferred — use fingerprint_fqn if known (fingerprint_fqn is a valid CI Visibility search facet, distinct from flaky_state):
test cases passed, vs 9 / 20 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.