Mutation Testing (Experimental)

Mutation Testing (Experimental)

Status: EXPERIMENTAL — advisory report only. Never wire this as a merge gate.

Coverage proves a line ran under test; mutation testing proves a test would notice if the line were wrong. Muter mutates the source (flips > to <, && to ||, deletes statements), reruns the suite per mutant, and reports the mutants that survived — code that is covered but effectively unasserted. It is the truest answer to "did the agent write real tests or coverage theater," and also the slowest and most fragile tool in the gauntlet, which is why it stays advisory.

When This Skill Activates

Use this skill when:

• Coverage is healthy but bugs still slip through covered code
• Auditing agent-written test suites for assertion-free tests
• A periodic (nightly/weekly) quality audit of engine/logic targets

Do NOT use as a per-commit or merge-blocking check — a full mutation run multiplies the suite's runtime by the mutant count.

Setup

1. Install muter and confirm it runs against the current Xcode before investing further:

brew install muter-mutation-testing/formulae/muter
muter --version

2. Copy templates/muter.conf.yml to the repo root as muter.conf.yml and scope it (see below).
3. First run:

muter run    # writes an HTML/console report of surviving mutants

Scoping: Engines Only

Mutation-test the code where logic density is highest and the suite claims real coverage — engine/store/service types. Exclude UI, generated code, and test files: mutating a SwiftUI body mostly produces noise, and every excluded file cuts the runtime multiplier.

mutateSourcesInDirectories:
  - App/Engines
  - App/Stores
excludeList: