Skill Tester

Skill Tester

Validate skill packages for structure compliance, test Python scripts for syntax and stdlib-only imports, and score quality across four dimensions (documentation, code quality, completeness, usability) with letter grades and improvement recommendations. Supports BASIC, STANDARD, and POWERFUL tier classification.

Core Capabilities

• Structure validation — check required files, directory layout, YAML frontmatter, and required SKILL.md sections against tier thresholds.
• Script testing — AST syntax checks, stdlib-only import analysis, argparse and __main__-guard detection, runtime --help and sample-data execution.
• Quality scoring — four equally weighted dimensions producing an overall score, letter grade (A+ through F), and a prioritized improvement roadmap.
• Tier classification — BASIC / STANDARD / POWERFUL requirements for SKILL.md depth, script count/LOC, argparse, output formats, and error handling.
• Dual output — human-readable reports and --json for CI/CD gating with meaningful exit codes.

When to Use

• Creating a new skill and validating it before publishing.
• Auditing an existing skill's structure, scripts, and quality.
• Embedding a quality gate into a CI/CD pipeline.

Clarify First

Before validating, confirm these inputs. If any is unknown or vague, ASK — do not assume:

• Target skill path — the skill directory to validate, test, and score (the subject of all three tools)
• Target tier — BASIC / STANDARD / POWERFUL (--tier; sets the required sections, script count, and structural thresholds)
• Pass bar — the minimum quality score / whether failures gate CI (--minimum-score, exit codes)

Stop rule: ask only the 2-3 that most change the output. If the user says "just draft it," proceed and list your assumptions at the top of the artifact.

Quick Start

# Validate skill structure and documentation
python skill_validator.py engineering/my-skill --tier POWERFUL --json

# Test all Python scripts in a skill
python script_tester.py engineering/my-skill --timeout 30

# Score quality with improvement roadmap
python quality_scorer.py engineering/my-skill --detailed --minimum-score 75