Spec-driven test rebuild

Spec-driven test rebuild

Derive a behavior specification from a package's implementation, then rebuild the unit test suite as an expression of that spec. Every rule gets a stable ID; every test cites the rule it expresses, so coverage is greppable in both directions.

The models are PR #9158 (packages/state, spec at packages/state/SPEC.md) and PR #9172 (packages/store, spec at packages/store/SPEC.md). Read both PRs and both specs before starting — they set the format, tone, and PR structure. (#9172's spec lives on its branch if not yet merged.)

Process

1. Baseline

If the working tree has uncommitted changes that aren't yours, stop and ask before branching — never stash or carry someone else's work silently.

Create a branch. Run pnpm test run in the package and record the file and test counts, noting how many belong to fuzz suites — fuzz suites stay untouched and their (often large) generated counts are excluded from every count you report later. Map the existing test files and note any duplicated or parallel suites that should merge (the store package had two whole generations of tests).

2. Read the implementation

Read all of it before writing anything. The spec is derived from the source, not from the existing tests or the docs.

Key principle: You're documenting observed behavior, not prescribing ideal behavior. If the code does something unexpected, unintuitive, or even wrong-looking but deliberate, spec it as-is. Flag quirks as "internal" or call them out in the PR body for human judgment—don't silently "fix" the spec to match how you think the code should work. Let the spec-writing process surface latent inconsistencies; then decide whether to fix the implementation or document the behavior.

3. Write SPEC.md

Write SPEC.md at the package root: numbered rules with stable IDs (e.g. S3, MG4) grouped into sections, one rule per observable behavior, internal machinery marked internal. Keep the header framing from the existing specs ("When a test and this document disagree, one of them is wrong — figure out which and fix it.").

• Spec what the code DOES, not what it should do. If behavior looks wrong but intent is ambiguous, document it as the contract.
• Before baking in any rule you inferred rather than observed, verify it with a throwaway test. Delete the scratch file after.
• Every rule must be observable by a runtime test. Don't write rules for pure type-level behavior (utility types, inference) unless the test suite actually exercises them with type assertions; otherwise leave types out of the spec.

4. Audit the spec adversarially

Audit the draft before writing tests. The failure mode is overstatement. Specifically hunt for:

• claimed symmetry between two code paths (create vs update, top-level vs nested, up vs down) — check each path independently;
• claims that two implementations of the same idea agree (a predicate vs an index, from-scratch vs incremental) — feed both the edge cases, especially missing/undefined values;
• blanket "X never happens" or "Y is always Z" claims — find the branch that falsifies them;
• side effects on inputs (mutation, freezing) that a happy-path test wouldn't notice.

5. Rebuild the test suite

Test files mirror the spec's sections; every test name cites its rule like it('[S3] ...'). Merge duplicated coverage, port good existing tests rather than rewriting them, and add tests for uncovered rules.