Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →Snapshot delta between two cost-summary JSON outputs. PR-level cost regression detection — answers "what changed between these two specific snapshots?". Pairs with cost-summary's stable JSON contract.
.claude/skills/ruvnet-cost-diff/SKILL.md| Test case | Without → With | Effect | Δ tokens | Δ turns |
|---|---|---|---|---|
| case-03 | ✗→✓ | ▲ Improved | -28% | 0% |
| case-04 | ✗→✓ | ▲ Improved | -18% | 0% |
| case-05 | ✗→✓ | ▲ Improved | -24% | 0% |
| case-06 | ✗→✓ | ▲ Improved | -39% | 0% |
| case-09 | ✗→✓ | ▲ Improved | 29% | 0% |
PR-level cost regression detection. Where cost-counterfactual compares to HYPOTHETICAL baselines (always-haiku/sonnet/opus) and cost-burn compares latest bucket to PRIOR MEAN, cost-diff compares two SPECIFIC known-good snapshots.
| Question | Skill | |---|---| | "What would we have spent at always-X?" | cost-counterfactual | | "Is daily burn accelerating vs prior mean?" | cost-burn | | "Did THIS PR add spend vs main?" | cost-diff ← this |
Implementation: scripts/diff.mjs. Consumes the stable JSON contract from cost summary --format json.
--baseline and --current JSON snapshots.total_cost_usd + sessionCount (cost-summary shape).byTier (haiku/sonnet/opus) and byModel (each model).added / removed / changed based onbaseline / current zero-ness.
|delta| descending so the biggest movers are at the top.--alert-on-pct N: exit 1 when total_pct > N.--alert-on-usd N: exit 1 when total_delta_usd > N.Both can be set; first to trigger wins.
bash# Capture baseline (e.g. on main, via the cost-tracker-smoke CI workflow) cost summary --format json > baseline.json # On the PR branch, capture current state cost summary --format json > current.json # Compare; fail the PR if total spend grew >10% OR >$5 cost diff --baseline baseline.json --current current.json \ --alert-on-pct 10 --alert-on-usd 5.00
The combination of both flags catches:
(e.g. doubling from $0.10 to $0.20 hits +100% but only +$0.10).
(e.g. growing from $100 to $110 is only +10% but +$10).
Either signal can fail the PR independently — they're OR'd.
The two USD-level thresholds above miss a regression class: when ONE token type grows disproportionately even though total spend grows modestly. Example: a PR introduces a verbose context-cache pattern, total spend grows only 10% (under --alert-on-pct 50), but cache_write tokens grow 900%. The iter-82 driver hides inside the USD signal.
--alert-on-class-pct cache_write:50 exits 1 when cache_write tokens grow more than 50% baseline → current. Multiple classes can be checked in one flag (comma-separated):
bashcost diff --baseline baseline.json --current current.json \ --alert-on-class-pct cache_write:50,output:25
First class to breach wins. Valid classes: input | output | cache_write | cache_read.
Recommended PR-gate triad:
bashcost diff --baseline ... --current ... \ --alert-on-pct 25 \ --alert-on-usd 5.00 \ --alert-on-class-pct cache_write:100
Three orthogonal signals — pct (total grew), usd (large absolute jump), class-pct (composition shifted). Each catches what the others miss; AND-of-OR semantics means any one firing fails the PR.
| Total spend | $1.000000 | $1.500000 | +$0.500000 | 50.00% |
| Sessions | 10 | 13 | +3 | 30.00% |
## By tier
| opus | $0 | $0.60 | +$0.600000 | new | added |
| sonnet | $0.70 | $0.50 | -$0.200000 | -28.57% | changed |
| haiku | $0.30 | $0.40 | +$0.100000 | 33.33% | changed |Notice the table is sorted by absolute delta, not alphabetically — the biggest mover (opus newly added) bubbles to the top. Operators reading top-down see "what mattered" first.
| Exit | Meaning | |---|---| | 0 | No alert, OR no thresholds set | | 1 | --alert-on-pct or --alert-on-usd threshold exceeded | | 2 | Config error (missing files, invalid JSON, malformed snapshot) |
| Status | Meaning | |---|---| | added | This tier/model was $0 in baseline, >$0 in current | | removed | This tier/model was >$0 in baseline, $0 in current | | changed | Both baseline and current >$0; delta is the difference |
Entries with baseline === 0 && current === 0 are dropped (nothing to report).
cost-diff is the SECOND HALF of a contract that cost-summary started: the stable JSON shape from cost summary --format json. Both pieces have been frozen — adding fields to summary is fine; renaming or removing isn't.
If you're consuming snapshots elsewhere (dashboards, alerting), the same shape works — cost-diff is just one consumer.
| Case | Status | Duration (ms) | Turns | Tokens | Tool calls | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Without | With | Δ | Without | With | Δ | Without | With | Δ | Without | With | Δ | ||
case-12 | pass→pass | 10,089 | 2,605 | -74% | 1 | 1 | 0% | 1,662 | 1,819 | +9% | 0 | 0 | — |
case-01 | pass→pass | 4,921 | 3,687 | -25% | 1 | 1 | 0% | 1,092 | 2,051 | +88% | 0 | 0 | — |
case-02 | pass→pass | 13,709 | 4,759 | -65% | 1 | 1 | 0% | 2,638 | 2,279 | -14% | 0 | 0 | — |
case-03 | fail→pass | 12,154 | 2,645 | -78% | 1 | 1 | 0% | 2,660 | 1,919 | -28% | 0 | 0 | — |
case-04 | fail→pass | 12,291 | 3,515 | -71% | 1 | 1 | 0% | 2,660 | 2,187 | -18% | 0 | 0 | — |
case-05 | fail→pass | 12,640 | 3,851 | -70% | 1 | 1 | 0% | 3,061 | 2,330 | -24% | 0 | 0 | — |
case-06 | fail→pass | 17,108 | 3,134 | -82% | 1 | 1 | 0% | 3,276 | 2,013 | -39% | 0 | 0 | — |
case-07 | pass→pass | 8,803 | 1,590 | -82% | 1 | 1 | 0% | 1,522 | 1,707 | +12% | 0 | 0 | — |
case-08 | pass→pass | 5,352 | 1,547 | -71% | 1 | 1 | 0% | 963 | 1,695 | +76% | 0 | 0 | — |
case-09 | fail→pass | 7,242 | 1,471 | -80% | 1 | 1 | 0% | 1,296 | 1,673 | +29% | 0 | 0 | — |
case-10 | pass→pass | 9,702 | 1,943 | -80% | 1 | 1 | 0% | 1,611 | 1,716 | +7% | 0 | 0 | — |
case-11 | pass→pass | 5,550 | 3,489 | -37% | 1 | 1 | 0% | 907 | 1,676 | +85% | 0 | 0 | — |
case-13 | pass→pass | 8,963 | 2,053 | -77% | 1 | 1 | 0% | 1,544 | 1,721 | +11% | 0 | 0 | — |
case-14 | fail→pass | 9,017 | 2,945 | -67% | 1 | 1 | 0% | 1,798 | 1,981 | +10% | 0 | 0 | — |
case-15 | fail→pass | 9,976 | 4,318 | -57% | 1 | 1 | 0% | 1,675 | 1,740 | +4% | 0 | 0 | — |
case-16 | fail→pass | 9,870 | 2,076 | -79% | 1 | 1 | 0% | 1,847 | 1,781 | -4% | 0 | 0 | — |
case-17 | pass→pass | 13,204 | 3,679 | -72% | 1 | 1 | 0% | 2,242 | 2,101 | -6% | 0 | 0 | — |
case-18 | fail→pass | 14,229 | 2,798 | -80% | 1 | 1 | 0% | 2,979 | 2,035 | -32% | 0 | 0 | — |
case-19 | pass→pass | 7,673 | 1,305 | -83% | 1 | 1 | 0% | 1,319 | 1,620 | +23% | 0 | 0 | — |
case-20 | fail→pass | 10,870 | 4,429 | -59% | 1 | 1 | 0% | 1,899 | 2,198 | +16% | 0 | 0 | — |
case-21 | pass→pass | 8,537 | 2,895 | -66% | 1 | 1 | 0% | 1,481 | 1,951 | +32% | 0 | 0 | — |
case-22 | pass→pass | 14,787 | 7,354 | -50% | 1 | 1 | 0% | 2,502 | 2,703 | +8% | 0 | 0 | — |
DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 22 cases were attempted. The headline lift of +45 percentage points is the difference between those two pass rates over the 22 comparable cases.
Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.
Other measured skills in the registry, with their headline benchmark lift.