Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →Campaign: Counterfactual reasoning to identify load-bearing factors. Core question: If key factors were different, would the conclusion still hold? Methods: Pearl SCM Three-Step, Lewis Possible Worlds, Tetlock & Belkin, PNS/PS.
.claude/skills/yogsoth-ai-counterfactual-probing/SKILL.md| Test case | Without → With | Effect | Δ tokens | Δ turns |
|---|---|---|---|---|
| case-10 | ✗→✓ | ▲ Improved | 49% | 0% |
| case-11 | ✗→✓ | ▲ Improved | -24% | 0% |
| case-19 | ✗→✓ | ▲ Improved | 163% | 0% |
| case-05 | ✓→✗ | ▼ Worse | -36% | 0% |
| case-15 | ✓→✗ | ▼ Worse | 0% | 0% |
Core question: If key factors were different, would the conclusion still hold?
| Artifact Type | Primary Strategy | Fallback Strategy | |---|---|---| | hypothesis, claim | structural-counterfactual | necessity-sufficiency | | research-question | thought-experiment | closest-worlds | | idea, approach | factor-removal | closest-worlds | | experiment-design | necessity-sufficiency | structural-counterfactual | | gap | closest-worlds | factor-removal |
| Parameter | S (Quick) | M (Standard) | L (Deep) | |---|---|---|---| | Factors examined | 5 | 10 | 20 | | Counterfactual scenarios | 3 | 8 | 15 | | Necessity tests | 3 | 6 | 12 | | Flip-point search depth | 2 | 4 | 8 |
Each subagent operates in isolated context. Factor enumeration precedes all counterfactual reasoning. Scenarios are constructed with minimal deviation from actuality. Fragility measurements aggregate across all tested factors.
Produces CounterfactualMap containing: load-bearing factors ranked by necessity, fragility index per factor, flip-points identified, robustness assessment, and recommended sensitivity analyses.
<!-- BEGIN available-tables (generated) -->
Optional, no fixed order; the final leaf is always a sop.
| Strategy | When to use | | --- | --- | | closest-worlds | Strategy: Lewis Possible Worlds — find the minimal change to reality that would flip the conclusion, measuring how close the nearest world where the conclusion fails. | | factor-removal | Strategy: Systematic factor removal — remove factors one at a time and observe whether the conclusion remains stable, identifying which factors are load-bearing. | | necessity-sufficiency | Strategy: Probability of Necessity and Sufficiency (PNS/PS) — systematically evaluate whether each factor is necessary, sufficient, both, or neither for the conclusion. | | structural-counterfactual | Strategy: Pearl Three-Step counterfactual — Abduction (fit model to evidence), Action (intervene on factor), Prediction (derive counterfactual outcome). | | thought-experiment | Strategy: Williamson-style precise thought experiments — construct carefully specified counterfactual scenarios to test whether conclusions depend on contingent features. |
Optional, no fixed order; the final leaf is always a sop.
| Tactic | When to use | | --- | --- | | causal-necessity-testing | Tactic: Extract causal claims, evaluate probability of necessity (PN) and sufficiency (PS) for each, classify into necessity-sufficiency quadrants. | | minimal-change-search | Tactic: Generate candidate changes, detect flip-points where conclusion reverses, measure fragility as distance to nearest flip. | | systematic-factor-ablation | Tactic: List all factors, remove one at a time, assess conclusion stability, rank factors by load-bearing importance. |
Optional, no fixed order; the final leaf is always a sop.
| SOP | When to use | | --- | --- | | context-checkpoint | Append research process and results to the current Phase's context file. Covers both process and results with genuine substance. Use this skill at plan-designated checkpoint points — typically after each strategy completes or at key decision nodes within a research Phase. | | context-init | Create a new context file for a research Phase. Called once at Phase start to initialize the file that subsequent context-checkpoint calls will append to. Use this skill whenever a new research Phase begins and a fresh context file is needed. | | stress-test-saturation-detection | Determines whether validation has reached saturation — no new weaknesses or failure modes being discovered. Used by all 5 campaigns as termination signal. | | verdict-synthesis | Synthesizes findings from a completed campaign into typed verdict reports. Produces DebateVerdict, RedTeamReport, FailureAnticipationReport, CounterfactualMap, or AdversarialStressReport depending on campaign. Also supports cross-campaign StressTestSummary. |
<!-- END available-tables (generated) -->
| Case | Status | Duration (ms) | Turns | Tokens | Tool calls | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Without | With | Δ | Without | With | Δ | Without | With | Δ | Without | With | Δ | ||
case-01 | pass→pass | 26,972 | 39,093 | +45% | 1 | 1 | 0% | 4,014 | 7,168 | +79% | 0 | 0 | — |
case-07 | fail→fail | 32,181 | 6,840 | -79% | 1 | 1 | 0% | 848 | 1,597 | +88% | 0 | 0 | — |
case-02 | fail→fail | 23,549 | 8,291 | -65% | 1 | 1 | 0% | 4,040 | 1,599 | -60% | 0 | 0 | — |
case-03 | fail→fail | 23,065 | 9,307 | -60% | 1 | 1 | 0% | 3,755 | 1,860 | -50% | 0 | 0 | — |
case-04 | fail→fail | 16,027 | 7,305 | -54% | 1 | 1 | 0% | 3,083 | 1,675 | -46% | 0 | 0 | — |
case-05 | pass→fail | 14,047 | 7,861 | -44% | 1 | 1 | 0% | 2,577 | 1,640 | -36% | 0 | 0 | — |
case-06 | fail→fail | 37,441 | 8,632 | -77% | 1 | 1 | 0% | 1,013 | 1,662 | +64% | 0 | 0 | — |
case-08 | fail→fail | 55,298 | 7,952 | -86% | 1 | 1 | 0% | 4,795 | 1,638 | -66% | 0 | 0 | — |
case-09 | pass→pass | 11,082 | 5,128 | -54% | 1 | 1 | 0% | 1,892 | 2,040 | +8% | 0 | 0 | — |
case-10 | fail→pass | 10,148 | 8,622 | -15% | 1 | 1 | 0% | 1,697 | 2,522 | +49% | 0 | 0 | — |
case-11 | fail→pass | 12,037 | 2,708 | -78% | 1 | 1 | 0% | 2,045 | 1,548 | -24% | 0 | 0 | — |
case-12 | pass→pass | 8,916 | 3,204 | -64% | 1 | 1 | 0% | 1,591 | 1,680 | +6% | 0 | 0 | — |
case-13 | pass→pass | 17,273 | 4,122 | -76% | 1 | 1 | 0% | 1,626 | 1,814 | +12% | 0 | 0 | — |
case-14 | pass→pass | 12,145 | 3,477 | -71% | 1 | 1 | 0% | 2,280 | 1,741 | -24% | 0 | 0 | — |
case-15 | pass→fail | 8,240 | 1,880 | -77% | 1 | 1 | 0% | 1,325 | 1,327 | +0% | 0 | 0 | — |
case-16 | pass→pass | 13,744 | 2,783 | -80% | 1 | 1 | 0% | 2,138 | 1,575 | -26% | 0 | 0 | — |
case-17 | fail→fail | 21,459 | 8,957 | -58% | 1 | 1 | 0% | 3,873 | 2,758 | -29% | 0 | 0 | — |
case-18 | pass→pass | 9,697 | 5,715 | -41% | 1 | 1 | 0% | 1,603 | 1,974 | +23% | 0 | 0 | — |
case-19 | fail→pass | 35,020 | 11,568 | -67% | 1 | 1 | 0% | 1,153 | 3,036 | +163% | 0 | 0 | — |
case-20 | pass→pass | 21,370 | 33,090 | +55% | 1 | 1 | 0% | 4,063 | 7,279 | +79% | 0 | 0 | — |
case-21 | fail→fail | 6,579 | 7,245 | +10% | 1 | 1 | 0% | 658 | 1,724 | +162% | 0 | 0 | — |
case-22 | pass→fail | 21,798 | 31,197 | +43% | 1 | 1 | 0% | 3,941 | 6,289 | +60% | 0 | 0 | — |
DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 22 cases were attempted, and 14 counted toward the lift figure. The other 8 produced results that are not comparable between the two arms, so they are excluded from the headline rather than averaged into it. The headline lift of 0 percentage points is the difference between those two pass rates over the 14 comparable cases. 6 cases got worse with the skill loaded, and they are included in that figure.
Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.
Other measured skills in the registry, with their headline benchmark lift.