Install any skill in seconds. Free to start, no credit card required.
Get Started Free →Runs automated tests to validate plugin integrity across 14 categories. Use before creating PRs, after making changes to skills or templates, or to verify plugin health.
.claude/skills/bitwize-music-studio-test/SKILL.md| Test case | Without → With | Effect | Δ tokens | Δ turns |
|---|---|---|---|---|
| case-14 | ✗→✓ | ▲ Improved | 27% | 0% |
| case-07 | ✗→✓ | ▲ Improved | 26% | 0% |
| case-08 | ✗→✓ | ▲ Improved | 34% | 0% |
| case-10 | ✗→✓ | ▲ Improved | 130% | 0% |
| case-11 | ✗→✓ | ▲ Improved | 55% | 0% |
Input: $ARGUMENTS
Run automated tests to validate plugin integrity. Execute each test methodically and report results clearly.
Default: Run all tests if no argument provided.
You are the plugin's automated test runner. Execute each test, track pass/fail, and report actionable results.
/test quick)For fast automated validation, run the pytest suite. Call get_python_command() first to get $PYTHON (the venv interpreter path — correct on every OS, including native Windows):
bash$PYTHON -m pytest ${CLAUDE_PLUGIN_ROOT}/tests/ -v
This covers:
tests/plugin/) - Frontmatter, templates, references, links, terminology, consistency, config, state, genres, integrationtests/unit/) - State parsers/indexer, shared utilities, mastering functionsRun specific categories:
bash$PYTHON -m pytest ${CLAUDE_PLUGIN_ROOT}/tests/plugin/test_skills.py -v # Skills only $PYTHON -m pytest ${CLAUDE_PLUGIN_ROOT}/tests/plugin/ -v # All plugin tests $PYTHON -m pytest ${CLAUDE_PLUGIN_ROOT}/tests/unit/ -v # All unit tests $PYTHON -m pytest ${CLAUDE_PLUGIN_ROOT}/tests/ -m "not slow" -v # Skip slow tests
Pytest catches common issues fast. For deep behavioral tests, use the full test suite below.
════════════════════════════════════════
CATEGORY: Test Category Name
════════════════════════════════════════
[PASS] Test name
[FAIL] Test name
→ Problem: what's wrong
→ File: path/to/file:line
→ Fix: specific fix instruction
────────────────────────────────────────
Category: X passed, Y failed
────────────────────────────────────────At the end:
════════════════════════════════════════
FINAL RESULTS
════════════════════════════════════════
config: X passed, Y failed
skills: X passed, Y failed
templates: X passed, Y failed
...
────────────────────────────────────────
TOTAL: X passed, Y failed, Z skipped
════════════════════════════════════════All test definitions are in test-definitions.md.
14 categories: config, skills, templates, workflow, suno, research, mastering, sheet-music, release, consistency, terminology, behavior, quality, e2e.
Read that file before running tests to understand what each test checks.
| Command | Description | |---------|-------------| | /test or /test all | Run all tests | | /test quick | Run Python test runner (fast automated checks) | | /test config | Configuration system tests | | /test skills | Skill definitions and docs | | /test templates | Template file tests | | /test workflow | Album workflow documentation | | /test suno | Suno integration tests | | /test research | Research workflow tests | | /test mastering | Mastering workflow tests | | /test sheet-music | Sheet music generation tests | | /test release | Release workflow tests | | /test consistency | Cross-reference checks | | /test terminology | Consistent language tests | | /test behavior | Scenario-based tests | | /test quality | Code quality checks | | /test e2e | End-to-end integration test |
For rapid validation during development, use pytest directly. Call get_python_command() first to get $PYTHON (the venv interpreter path — correct on every OS):
bash# Run all tests $PYTHON -m pytest ${CLAUDE_PLUGIN_ROOT}/tests/ -v # Run specific test modules $PYTHON -m pytest ${CLAUDE_PLUGIN_ROOT}/tests/plugin/test_skills.py ${CLAUDE_PLUGIN_ROOT}/tests/plugin/test_templates.py -v # Verbose with short tracebacks $PYTHON -m pytest ${CLAUDE_PLUGIN_ROOT}/tests/ -v --tb=short # Quiet mode (for CI/logs) $PYTHON -m pytest ${CLAUDE_PLUGIN_ROOT}/tests/ -q --tb=line
Test modules in tests/plugin/:
test_skills.py - Frontmatter, required fields, model validationtest_templates.py - Template existence and structuretest_references.py - Reference doc existencetest_links.py - Internal markdown linkstest_terminology.py - Deprecated terms checktest_consistency.py - Version sync, skill countstest_config.py - Config file validationtest_state.py - State cache tool validationtest_genres.py - Genre directory cross-referencetest_integration.py - Cross-skill prerequisite chainsWhen bugs are found:
/test [category] to verify test fails/test [category] to verify test passesRule: Every bug fix should add a regression test.
output_mode: content and -n for line numbers| Case | Status | Duration (ms) | Turns | Tokens | Tool calls | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Without | With | Δ | Without | With | Δ | Without | With | Δ | Without | With | Δ | ||
case-14 | fail→pass | 8,809 | 2,439 | -72% | 1 | 1 | 0% | 1,339 | 1,707 | +27% | 0 | 0 | — |
case-03 | fail→fail | 9,955 | 4,056 | -59% | 1 | 1 | 0% | 1,942 | 1,632 | -16% | 0 | 0 | — |
case-01 | fail→fail | 13,713 | 4,762 | -65% | 1 | 1 | 0% | 2,310 | 1,676 | -27% | 0 | 0 | — |
case-02 | fail→fail | 11,823 | 3,756 | -68% | 1 | 1 | 0% | 2,413 | 1,613 | -33% | 0 | 0 | — |
case-04 | pass→fail | 21,979 | 6,458 | -71% | 1 | 1 | 0% | 4,668 | 1,561 | -67% | 0 | 0 | — |
case-05 | pass→pass | 15,627 | 12,910 | -17% | 1 | 1 | 0% | 2,968 | 3,691 | +24% | 0 | 0 | — |
case-06 | pass→pass | 8,803 | 7,991 | -9% | 1 | 1 | 0% | 1,832 | 2,700 | +47% | 0 | 0 | — |
case-07 | fail→pass | 8,345 | 2,411 | -71% | 1 | 1 | 0% | 1,363 | 1,713 | +26% | 0 | 0 | — |
case-08 | fail→pass | 7,927 | 2,288 | -71% | 1 | 1 | 0% | 1,325 | 1,778 | +34% | 0 | 0 | — |
case-09 | pass→pass | 7,083 | 2,823 | -60% | 1 | 1 | 0% | 1,295 | 1,748 | +35% | 0 | 0 | — |
case-10 | fail→pass | 4,917 | 2,435 | -50% | 1 | 1 | 0% | 763 | 1,754 | +130% | 0 | 0 | — |
case-11 | fail→pass | 6,895 | 2,263 | -67% | 1 | 1 | 0% | 1,071 | 1,660 | +55% | 0 | 0 | — |
case-12 | fail→pass | 4,776 | 2,459 | -49% | 1 | 1 | 0% | 883 | 1,729 | +96% | 0 | 0 | — |
case-13 | fail→pass | 17,293 | 2,678 | -85% | 1 | 1 | 0% | 3,020 | 1,726 | -43% | 0 | 0 | — |
case-15 | fail→pass | 10,442 | 3,409 | -67% | 1 | 1 | 0% | 1,773 | 1,920 | +8% | 0 | 0 | — |
case-16 | fail→pass | 6,786 | 2,200 | -68% | 1 | 1 | 0% | 1,109 | 1,690 | +52% | 0 | 0 | — |
case-17 | fail→pass | 7,128 | 3,080 | -57% | 1 | 1 | 0% | 1,226 | 1,751 | +43% | 0 | 0 | — |
case-18 | fail→pass | 9,769 | 2,082 | -79% | 1 | 1 | 0% | 1,576 | 1,645 | +4% | 0 | 0 | — |
case-19 | fail→pass | 12,801 | 2,588 | -80% | 1 | 1 | 0% | 2,005 | 1,842 | -8% | 0 | 0 | — |
case-20 | fail→pass | 7,615 | 2,786 | -63% | 1 | 1 | 0% | 1,348 | 1,791 | +33% | 0 | 0 | — |
case-21 | fail→pass | 9,902 | 2,206 | -78% | 1 | 1 | 0% | 1,649 | 1,669 | +1% | 0 | 0 | — |
case-22 | fail→pass | 13,584 | 1,905 | -86% | 1 | 1 | 0% | 2,324 | 1,623 | -30% | 0 | 0 | — |
DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 22 cases were attempted, and 18 counted toward the lift figure. The other 4 produced results that are not comparable between the two arms, so they are excluded from the headline rather than averaged into it. The headline lift of +64 percentage points is the difference between those two pass rates over the 18 comparable cases. 3 cases got worse with the skill loaded, and they are included in that figure.
Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.
Other measured skills in the registry, with their headline benchmark lift.