Ready to supercharge your AI agents?

Install any skill in seconds. Free to start, no credit card required.

Get Started Free →

Create Skill Test · dotnet | DecimalAI

Registry skillby dotnet

Create Skill Test

dotnet/create-skill-test

Source dotnet/skills ↗License MIT

Scaffolds eval.yaml test files for agent skills in the dotnet/skills repository.

SkillScore96Maintained

Automatic

QA & Test Engineering pack

Last verified 9/1/2026

What it changes

Passed 22 of 23 test cases with the skill, and 7 of 23 without it.

Without skill30%

With skill96%

Checks it passed that the model alone missed

Uses the top-level defaults block instead of config.
Specifies using stimuli instead of legacy scenarios.
Appends || exit 0 to guard the failing build command.

and 16 more in the full report

15 fixed7 passed either way1 still failing0 broke

Pass-rate lift: +65 pp
Tokens: +156%
Agent turns: 0%
Response time: -29%

Model gemini-3.6-flashMeasured 9/1/2026v422 of 23 cases passed with skill

Tokens, turns and time describe resource use, not a dollar price.

Use this skill

Save it into your project's skills folder. Claude Code picks it up in its next session.

curl -fsSL --create-dirs -o .claude/skills/dotnet-create-skill-test/SKILL.md https://app.decimal.ai/s/dotnet-create-skill-test/SKILL.md

Want it pinned and updatable? pip install "decimalai>=0.15", then decimalai skills add dotnet-create-skill-test, and start Claude Code with claude --plugin-dir "$(git rev-parse --show-toplevel)/.agents"

1 · One-time — install the CLI (docs ↗)

pip install "decimalai>=0.15"

2 · Get the skill — writes .agents/skills/dotnet-create-skill-test/ and pins it in .decimal/skills.lock

decimalai skills add dotnet-create-skill-test

No CLI? Get the same file — curl · raw file

curl -fsSL --create-dirs -o .claude/skills/dotnet-create-skill-test/SKILL.md https://app.decimal.ai/s/dotnet-create-skill-test/SKILL.md

Raw file: /s/dotnet-create-skill-test/SKILL.md ↗ · machine-readable summary at /s/dotnet-create-skill-test.json ↗

Safety checks: 3 of 3 passed

Code scan CleanIntent PassedContent Clean

See it in action

One of 15 test cases the skill fixed. Same model, same prompt, with and without the skill.

1 of 15

Prompt

I am updating an old test spec for `analyze-deps` in `dotnet-deps`. It currently starts with `config: { timeout: 3m }`. I want to add `defaults: { runs: 3 }` right below it. Is having both `config:` and `defaults:` allowed?

What was checked, without and with the skill
Checked	Without skill	With skill
Replaces config entirely with defaults rather than keeping both blocks.	Missed	Passed

case-08 · gemini-3.6-flash · 9/1/2026 · answers are capped and redacted

What's inside

The instructions your model reads when the skill loads.

SKILL.md344 linessingle file

Create Skill Test

Scaffold an evaluation spec (eval.yaml) for a skill or agent so it conforms to the Vally schema, passes skill-validator check and check_eval_quality.py, is powerful enough to return a verdict, and does not overfit to the skill's own wording.

When to Use

Creating a new eval.yaml for a skill or agent
Adding stimuli to an existing eval
Sizing an eval so the pass gate can actually be reached
Setting up or repairing fixture files alongside an eval
Reviewing whether rubric items and graders risk overfitting

When Not to Use

Diagnosing a failing or regressed eval — use improve-skill-quality
Modifying the skill-validator or the evaluation workflows
Creating or editing SKILL.md files — use create-skill

Inputs

Input	Required	Description
Skill or agent name	Yes	Must exist under `plugins/<plugin>/skills/` or `plugins/<plugin>/agents/`
Plugin name	Yes	e.g. `dotnet-msbuild`
Skill content	Yes	Read it — you cannot write non-overfitted rubric items without it
Failure modes to discriminate	Recommended	Each becomes one stimulus

Workflow

Step 1: Locate the target and the test directory

tests/<plugin>/<skill-name>/eval.yaml          # skills
tests/<plugin>/agent.<agent-name>/eval.yaml    # agents (the agent. prefix disambiguates)

Verify the target exists at plugins/<plugin>/skills/<skill-name>/SKILL.md or plugins/<plugin>/agents/<agent-name>.agent.md, and read it.

Track record

Benchmark

22 / 23

test cases passed, vs 7 / 23 without

Adoption

Users: 1
Times used: 16

Safety and source details

SkillSafety

✓ 3 of 3 passed

Code scan15 of 15 checks clean

0 critical · 0 warning · 0 info — scanner v5 — 9/1/2026

View 15 scanner checks

live_secretremote_code_executionreverse_shelldata_exfiltrationssrf_metadataagent_snoopinganti_refusaldestructive_commandsobfuscated_unicodeunicode_homoglyphopaque_blobsuspicious_urltool_overreachtrigger_abuseprompt_injection_phrasing

IntentSafe

LLM judge · does the skill do what it claims

ContentClean

LLM judge · policy & harmful-content review

Source

Imported

dotnet / skills

.agents/skills/create-skill-test/SKILL.md

View on GitHub ↗

License MITImported 9/1/2026Version v4

More skills from the same source and nearby in the registry, with their headline benchmark lift.

Ready to supercharge your AI agents?

See it in action

What's inside

Create Skill Test

When to Use

When Not to Use

Inputs

Workflow

Step 1: Locate the target and the test directory

Track record

Benchmark

Adoption

SkillSafety

Source

Related skills