a5c-ai/verification-suite

• Is this plan structurally valid? (XML task format, required sections, frontmatter)
• Is this phase complete? (all plans executed, summaries written, state updated)
• Do all references resolve? (no broken links, missing files, stale references)
• Do expected artifacts exist? (outputs from each phase are present)
• Is requirements coverage at 100%?
• Is the .planning/ directory healthy?

This corresponds to the original lib/verify.cjs module and the verification patterns in references/verification-patterns.md.

Capabilities

1. Plan Structure Validation

Validate a PLAN.md file for structural correctness:

Checks:
  [PASS] Frontmatter present with required fields (status, phase, wave, task_count)
  [PASS] XML task blocks present and well-formed
  [PASS] Each task has <task>, <context>, <acceptance_criteria> elements
  [PASS] Task count matches frontmatter task_count
  [FAIL] Task 3 missing <acceptance_criteria> element
  [WARN] No depends_on field in frontmatter (optional but recommended)

XML task format:

xml
<task id="1" title="Implement login endpoint">
  <context>
    Build POST /api/auth/login endpoint with email/password validation.
  </context>
  <acceptance_criteria>
    - Endpoint accepts POST with email and password
    - Returns JWT token on success
    - Returns 401 on invalid credentials
    - Rate limits to 5 attempts per minute
  </acceptance_criteria>
</task>

2. Phase Completeness Checks

Verify a phase has all expected artifacts:

Phase 72 completeness:
  [PASS] PLAN-1.md exists and status=executed
  [PASS] PLAN-2.md exists and status=executed
  [PASS] SUMMARY.md exists with commits and files lists
  [PASS] STATE.md updated with phase 72 in completed_phases
  [FAIL] VERIFICATION.md missing (verification not run)
  [WARN] No CONTEXT.md (discuss-phase was skipped)

Completeness: 4/6 checks passed (67%)
Required for completion: VERIFICATION.md must be created

3. Reference Integrity Verification

Check that all file references in GSD artifacts are valid:

Reference integrity scan:
  .planning/ROADMAP.md
    [PASS] Phase 72 directory exists: .planning/phase-72/
    [PASS] Phase 73 directory exists: .planning/phase-73/

  .planning/phase-72/PLAN-1.md
    [PASS] Referenced file src/auth/oauth.ts exists
    [PASS] Referenced file src/auth/tokens.ts exists
    [FAIL] Referenced file src/auth/middleware.ts does NOT exist

  .planning/STATE.md
    [PASS] Current phase 72 exists in ROADMAP.md
    [PASS] All completed_phases exist in ROADMAP.md

4. Artifact Existence Verification

Verify expected outputs from a phase exist:

Phase 72 expected artifacts:
  [PASS] src/auth/oauth.ts created (from PLAN-1.md task 1)
  [PASS] src/auth/tokens.ts created (from PLAN-1.md task 2)
  [PASS] src/middleware/auth.ts created (from PLAN-2.md task 1)
  [FAIL] src/auth/__tests__/oauth.test.ts missing (from PLAN-2.md task 3)

5. Requirements Coverage Analysis

Map requirements to phase deliverables:

Requirements coverage:
  R1 (project scaffolding) -> Phase 70 [completed] -> PASS
  R5 (user auth)          -> Phase 72 [completed] -> PASS
  R7 (REST API)           -> Phase 73 [planned]   -> PENDING
  R12 (admin dashboard)   -> NOT MAPPED            -> FAIL

Coverage: 12/13 mapped (92%)
Gap: R12 needs a phase assignment

6. Planning Directory Health Check

Comprehensive health assessment:

.planning/ health check:
  [PASS] config.json exists and valid
  [PASS] PROJECT.md exists
  [PASS] REQUIREMENTS.md exists
  [PASS] ROADMAP.md exists
  [PASS] STATE.md exists and parseable
  [WARN] phase-70/ has no SUMMARY.md (orphaned phase?)
  [FAIL] phase-72/PLAN-3.md references non-existent wave 4
  [INFO] 2 debug sessions found in debug/
  [INFO] 5 quick tasks found in quick/

Health: 5 pass, 1 warn, 1 fail
Recommendation: Run gsd health --fix to repair issues

7. Milestone Audit Report

Generate comprehensive milestone readiness report:

markdown
# Milestone Audit: v1.0

## Phase Completion
| Phase | Status | Plans | Summary | Verification |
|-------|--------|-------|---------|--------------|
| 70 | verified | 2/2 executed | present | passed |
| 71 | verified | 3/3 executed | present | passed |
| 72 | completed | 2/2 executed | present | NOT RUN |
| 73 | completed | 2/2 executed | present | passed |

## Requirements Coverage
12/13 requirements covered (92%)
Gap: R12 (admin dashboard) - not in v1.0 scope

## Integration Points
3 integration points identified, 2 verified

## Audit Result: CONDITIONAL PASS
- Phase 72 needs verification
- R12 confirmed out of v1.0 scope

8. Test Coverage Analysis

For add-tests process, analyze what needs testing:

Phase 72 test coverage:
  src/auth/oauth.ts
    Functions: loginUser, refreshToken, revokeToken
    Existing tests: none
    Recommended: unit tests for all 3 functions

  src/middleware/auth.ts
    Functions: requireAuth, requireRole
    Existing tests: none
    Recommended: unit tests + integration tests

Tool Use Instructions

Validating Plan Structure

1. Use Read to load the plan file
2. Check frontmatter presence and required fields
3. Use regex patterns to find and validate XML task blocks
4. Count tasks and compare to frontmatter task_count
5. Check each task has required elements (context, acceptance_criteria)

Checking Phase Completeness

1. Use Glob to find all files in the phase directory
2. Check for expected files (PLAN-.md, SUMMARY.md, VERIFICATION.md)
3. Use Read to check plan statuses in frontmatter
4. Use Grep to check STATE.md for phase in completed_phases
5. Return completeness report

Scanning Reference Integrity

1. Use Read to load each GSD artifact
2. Extract file paths and references from content
3. Use Glob or Bash(ls) to verify each reference exists
4. Categorize as PASS/FAIL/WARN

Health Check

1. Use Glob to inventory all .planning/ contents
2. Run structure, completeness, and integrity checks
3. Identify orphaned files and missing links
4. Generate health report with recommendations

Process Integration

• plan-phase.js - Validate plan structure after planner generates plans
• execute-phase.js - Verify phase completeness after execution
• verify-work.js - Artifact existence verification during UAT
• audit-milestone.js - Full milestone audit report generation
• add-tests.js - Test coverage analysis for test generation
• health command - Full planning directory health check

Output Format

json
{
  "operation": "validate_plan|check_phase|check_refs|check_artifacts|coverage|health|audit|test_coverage",
  "status": "pass|fail|warn",
  "checks": [
    { "name": "frontmatter_present", "status": "pass", "detail": "" },
    { "name": "xml_tasks_valid", "status": "fail", "detail": "Task 3 missing acceptance_criteria" }
  ],
  "summary": {
    "total": 6,
    "passed": 4,
    "failed": 1,
    "warnings": 1
  },
  "recommendations": ["Fix Task 3 in PLAN-1.md"]
}

Configuration

| Setting | Default | Description | |---------|---------|-------------| | strictMode | false | Treat warnings as failures | | requireVerification | true | Require VERIFICATION.md for phase completion | | requireContext | false | Require CONTEXT.md for phase completion | | coverageThreshold | 100 | Minimum requirements coverage percentage |

Error Handling

| Error | Cause | Resolution | |-------|-------|------------| | Plan parse error | Malformed XML tasks or YAML frontmatter | Show specific syntax error location | | Phase directory missing | Expected phase directory does not exist | Create directory or update roadmap | | Stale reference | Referenced file was moved or deleted | Update reference or remove | | Coverage gap | Requirement not mapped to any phase | Add phase or update existing phase mapping |

Constraints

• Verification is read-only; it never modifies files (only reports issues)
• Plan validation requires XML task format (not arbitrary markdown)
• Phase completeness checks are configurable (strict vs relaxed)
• Health check scans the entire .planning/ directory recursively
• Milestone audit must verify all phases in the milestone scope
• Test coverage analysis examines actual source files, not just plan references