--- name: 0xjitsu/pii-sweep source: https://app.decimal.ai/s/0xjitsu-pii-sweep@1/SKILL.md source_sha256: eb2a438e233b --- # pii-sweep Automated PII and secrets detection for repositories being prepared for public release. --- ## When to Trigger Activate this skill when the user: - Asks to **scrub PII** or **remove personal data** - Wants to **prepare a repo for open source** or **make a repo public** - Says "check for secrets" or "run a PII audit" - Is about to **publish** or **transfer ownership** of a repository - Asks to verify a repo is "clean" before release --- ## Exclusions Always skip these paths during scanning: - `.git/` -- repository internals - `node_modules/` -- third-party dependencies - `vendor/` -- third-party dependencies - `*.min.js`, `*.min.css` -- minified bundles - Binary files (images, fonts, compiled assets) - `package-lock.json`, `bun.lockb`, `yarn.lock` -- lockfiles --- ## Sweep Patterns ### Email Addresses ```regex [a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,} ``` **Exclude safe patterns:** `@example.com`, `@example.org`, `@your-domain.com`, `@users.noreply.github.com`, `noreply@` Flag everything else as a potential real email. ### Phone Numbers US formats: ```regex \b\d{3}[-.]?\d{3}[-.]?\d{4}\b ``` International formats: ```regex \+\d{1,3}[-.\s]?\(?\d{1,4}\)?[-.\s]?\d{1,4}[-.\s]?\d{1,9} ``` **Exclude:** Numbers that are clearly not phones (port numbers like `3000`, `8080`, `5432`; hex values; timestamps). ### API Keys and Tokens | Pattern | Service | |---------|---------| | `sk_live_[a-zA-Z0-9]+` | Stripe live key | | `sk_test_[a-zA-Z0-9]+` | Stripe test key | | `ghp_[a-zA-Z0-9]+` | GitHub PAT | | `gho_[a-zA-Z0-9]+` | GitHub OAuth token | | `sntryu_[a-zA-Z0-9]+` | Sentry user token | | `xnd_[a-zA-Z0-9]+` | Xendit API key | | `Bearer [a-zA-Z0-9._-]+` | Bearer tokens | | `token:\s*"[^"]+"` | Generic token assignments | | `AKIA[0-9A-Z]{16}` | AWS access key | | `AIza[0-9A-Za-z_-]{35}` | Google API key | | `xoxb-[0-9a-zA-Z-]+` | Slack bot token | | `xoxp-[0-9a-zA-Z-]+` | Slack user token | ### Spreadsheet and Document IDs ```regex spreadsheetId.*['\"][a-zA-Z0-9_-]{20,}['\"] ``` Google Sheets URLs: ```regex docs\.google\.com/spreadsheets/d/[a-zA-Z0-9_-]+ ``` Also check for `SHEET_ID`, `DOC_ID`, `SPREADSHEET_ID` with hardcoded values. ### Internal URLs Flag domains that are NOT in the safe list: **Safe list:** `example.com`, `example.org`, `localhost`, `127.0.0.1`, `0.0.0.0`, `github.com`, `npmjs.com`, `vercel.app`, `shields.io`, `img.shields.io`, `creativecommons.org`, `opensource.org`, `keepachangelog.com`, `semver.org` Any other `https?://` URL with a real-looking domain should be flagged for review. ### Physical Addresses Basic US address pattern: ```regex \d{1,5}\s+[A-Z][a-z]+(\s+[A-Z][a-z]+)*\s+(St|Ave|Blvd|Dr|Rd|Ln|Way|Ct|Pl|Cir)\b ``` Also check for: - ZIP codes near identifiable context: `\b\d{5}(-\d{4})?\b` - City + State patterns: `[A-Z][a-z]+,\s*[A-Z]{2}\s+\d{5}` ### Known Team Member Names Ask the user for a list of real names to search for. Search case-insensitively across all text files. Common locations where names hide: - Code comments (`// Author: ...`) - Package.json `author` and `contributors` fields - README acknowledgments sections - License file headers - Git commit messages (note: these persist in `.git/` and are excluded from file sweep, but warn the user about `git log`) --- ## Sample Data Replacement Standards When replacing PII, use these safe substitutes: | Data Type | Replacement | Rationale | |-----------|-------------|-----------| | Phone numbers | `555-XXX-XXXX` (e.g., `555-012-3456`) | FCC reserved range | | Email addresses | `user@example.com`, `admin@example.com` | IANA reserved domain | | Business names | Generic descriptors: "Sample Cabinet Shop", "Metro Kitchen & Bath", "Acme Corp" | Clearly fictional | | Person names | "Jane Doe", "John Smith", "Alex Johnson" | Obvious placeholders | | Physical addresses | "123 Main St, Anytown, CA 90210" | Recognizable fake | | Spreadsheet IDs | `your-sheet-id` with note `(set SHEET_ID in .env)` | Guides user to configure | | API keys | `your-api-key-here` with note `(set in .env)` | Guides user to configure | | Internal URLs | `https://your-app.example.com` | Safe placeholder | --- ## Workflow ### Phase 1: First Pass Scan Run all patterns across the repository. For each finding, report: ``` [CATEGORY] file/path:line_number Matched: Context: ``` Group findings by category. Show a summary count: ``` Sweep Results: Emails: 3 findings (2 files) Phone numbers: 1 finding (1 file) API keys: 2 findings (1 file) Spreadsheet IDs: 0 findings Internal URLs: 4 findings (3 files) Addresses: 0 findings Names: 1 finding (1 file) -------------------------------- Total: 11 findings ``` ### Phase 2: User Review Present each finding and ask the user to classify: - **True positive** -- real PII, needs replacement - **False positive** -- safe to ignore (e.g., test fixture, example data) - **Move to .env** -- secret that should become an environment variable Do NOT auto-replace anything. Wait for explicit user confirmation on each finding or batch. ### Phase 3: Replace For true positives: - Replace with the appropriate sample data from the standards table above - If moving to `.env`, add the variable to `.env` (gitignored) and `.env.example` (with placeholder) - Update the code to read from `process.env.VARIABLE_NAME` or equivalent For each replacement, show a before/after diff. ### Phase 4: Verification Pass Re-run all sweep patterns. The goal is zero findings (excluding acknowledged false positives). If new findings appear (e.g., a replacement introduced a new pattern match), flag and resolve. ### Phase 5: Environment File Check If the project uses `.env`: 1. Verify `.env` is listed in `.gitignore` 2. Verify `.env.example` exists with all required variables (placeholder values only) 3. Verify no `.env` file is tracked in git: `git ls-files .env` ### Phase 6: Final Report ``` PII Sweep Complete ================== Files scanned: 142 Patterns checked: 7 categories Findings: 11 total True positives: 8 (replaced) False positives: 3 (acknowledged) Moved to .env: 2 Replacements made: - src/config.ts:14 email -> user@example.com - src/config.ts:15 api_key -> moved to .env - README.md:45 phone -> 555-012-3456 ... Remaining warnings: - Git history contains real names in commit messages (consider: git filter-branch or BFG Repo Cleaner) - 2 false positives acknowledged by user Status: READY FOR PUBLIC RELEASE ``` --- ## Safety Rules | Rule | Rationale | |------|-----------| | Never auto-replace without user confirmation | Automated replacement can break functionality or remove intentional data | | Never delete content -- always replace | Deletions can break code; replacements keep the app functional | | Exclude `.git/`, `node_modules/`, binaries | These are managed externally and would produce false positives | | Warn about git history | `git log` and reflog retain PII even after file-level scrubbing | | Run verification pass after all replacements | Ensures no PII was missed or reintroduced | | Check `.env` is gitignored | Prevents secrets from being committed in the future | --- ## Git History Warning File-level scrubbing does not clean git history. If the repository previously contained real PII in committed files, warn the user about these options: 1. **BFG Repo-Cleaner** -- fastest option for removing specific strings from history 2. **git filter-repo** -- more flexible, can rewrite paths and content 3. **Fresh repo** -- if history is not important, create a new repo with a clean initial commit This is outside the scope of pii-sweep (destructive git operations require explicit user action) but must always be mentioned in the final report if any true positives were found in tracked files.