Ready to supercharge your AI agents?

Install any skill in seconds. Free to start, no credit card required.

Get Started Free →

Strategy: Robustness Design · yogsoth-ai | DecimalAI

yogsoth-ai/robustness-design

SkillScore74Maintained

Design experiments to identify failure boundaries and robustness limits

AI/MLby yogsoth-ai

Links this skill to your agents and keeps it current with the author’s updates.

Automatic

AI Engineer pack

1 · One-time — install the CLI (docs ↗)

pip install decimalai

2 · Get the skill — writes .claude/skills/yogsoth-ai-robustness-design/SKILL.md

decimalai skills pull yogsoth-ai-robustness-design --out .claude/skills/

No CLI? Get the same file — curl · raw file

curl -fsSL --create-dirs -o .claude/skills/yogsoth-ai-robustness-design/SKILL.md https://app.decimal.ai/s/yogsoth-ai-robustness-design/SKILL.md

Raw file: /s/yogsoth-ai-robustness-design/SKILL.md ↗ · machine-readable summary at /s/yogsoth-ai-robustness-design.json ↗

In productionGlobal

how this skill behaves on real traffic · last 30 days

No production data for this skill yet — usage numbers appear here once agents start running it.

Anonymized aggregate across every org using this skill — proof from real production use, not a demo.

Does this skill help? · Benchmark: with vs. withoutsign in to run this on your agents

27 test cases from the publisher’s eval.yaml · each run twice on gemini-3.6-flash — with the skill injected, then without · last run 18d ago

Yes — +15% lift · at higher cost+40% tokens0% turns74% of whole cases pass with skill (20/27)

Test case	Without → With	Effect	Δ tokens	Δ turns
case-08	✗→✓	▲ Improved	-10%	0%
case-09	✗→✓	▲ Improved	-18%	0%
case-10	✗→✓	▲ Improved	9%	0%
case-12	✗→✓	▲ Improved	27%	0%
case-18	✗→✓	▲ Improved	0%	0%

Showing 5 of 27 cases

SkillSafety

✓ 3 of 3 passed

Code scan15 of 15 checks clean

0 critical · 0 warning · 0 info — scanner v4 — 8/10/2026

live_secretremote_code_executionreverse_shelldata_exfiltrationssrf_metadataagent_snoopinganti_refusaldestructive_commandsobfuscated_unicodeunicode_homoglyphopaque_blobsuspicious_urltool_overreachtrigger_abuseprompt_injection_phrasing

IntentSafe

LLM judge · does the skill do what it claims

ContentClean

LLM judge · policy & harmful-content review

Source

Imported

yogsoth-ai / de-anthropocentric-research-engine

skills/robustness-design/SKILL.md

View on GitHub ↗

License Apache-2.0Imported 8/10/2026Version v1

Skill contents

SKILL.md

Strategy: Robustness Design

Question: Under what conditions does the method fail?

Methodology

Distribution Shift Testing: Evaluate under covariate shift, label shift, domain shift.
Adversarial Robustness: Perturbation-based attacks (PGD, AutoAttack) at varying epsilon.
Cross-Domain Transfer: Test on domains not seen during training.
Noise Injection: Gaussian noise, label noise, missing data at varying severity.
Stress Testing: Push inputs to boundary conditions (extreme lengths, rare categories, edge cases).

Execution Flow

factor-identification → Identify robustness dimensions (noise type, shift type, severity)
level-specification → Define severity levels for each perturbation
baseline-selection → Select robust baselines for comparison
metric-specification → Define degradation metrics (absolute and relative to clean)
design-matrix-construction → Build perturbation grid
sample-size-estimation → Determine samples needed per condition
statistical-method-selection (tactic) → Choose tests for degradation significance

Budget Gate

| Robustness Type | Conditions | Severities | Min Runs | Notes | |----------------|-----------|-----------|----------|-------| | Single perturbation | 1 | 3-5 | 3-5 | Quick sanity check | | Multi-perturbation | 3-5 | 3 each | 9-15 | Standard robustness eval | | Adversarial sweep | 1 attack | 5-10 epsilon | 5-10 | Adversarial robustness curve | | Comprehensive | 5+ types | 3-5 each | 50+ | Publication-ready robustness | | Cross-domain | N domains | 1 | N | Transfer evaluation |

Available Tactics

Optional, no fixed order; the final leaf is always a sop.

| Tactic | When to use | | --- | --- | | statistical-method-selection | Select appropriate statistical methods for experiment analysis |

Available SOPs

Optional, no fixed order; the final leaf is always a sop.

| SOP | When to use | | --- | --- | | baseline-selection | Select appropriate baselines for experimental comparison | | design-matrix-construction | Build the experiment design matrix with proper orthogonality and balance | | factor-identification | Identify independent, dependent, and control variables for an experiment | | level-specification | Determine appropriate levels for each experimental factor | | metric-specification | Define experiment metrics and significance standards | | sample-size-estimation | SOP: power analysis and required experiment count estimation |

Illustrative Want to see what yogsoth-ai/robustness-design would put in front of your model? Preview the rendered body and check its trigger phrases — no LLM call, no signup.

Latest benchmark · publisher runVerified

9/3/2026, 22:53 UTC · 27 cases

Lift

+15%

74% vs 59%

Agent turns

0%

1.0 vs 1.0

Tokens

+40%

3,761.58 vs 2,694.23

fail → pass · 7pass → pass · 13fail → fail · 4pass → fail · 3

Per-case results (27)

Case	Status	Duration (ms)			Turns			Tokens			Tool calls
Case	Status	Without	With	Δ	Without	With	Δ	Without	With	Δ	Without	With	Δ
case-01 We are preparing an LLM classifier for publication and need a rigorous robustness evaluation plan to identify edge-case failures and adversarial weaknesses. Please map out a comprehensive robustness design process covering factor analysis, baseline selections, degradation metric definitions, and a detailed test grid with estimated run budgets so we can guarantee statistical significance across multiple perturbation types.	fail→fail	53,185	47,963	-10%	1	1	0%	8,254	8,756	+6%	0	0	—
case-02 We are designing a benchmark for the ResNet-50 image classification model to evaluate its stability under corruptions. Team members suggest jumping straight to building the 2x2 grid matrix of noise parameters before picking control baselines or defining degradation metrics. Outline the correct sequential execution flow for structuring this evaluation.	pass→pass	38,349	18,632	-51%	1	1	0%	2,773	3,036	+9%	0	0	—
case-03 For an audio keyword spotter baseline evaluation, our engineer wants to choose baseline architectures first, then figure out what noise factors and severity levels to test. How should the initial steps of the robustness execution flow be ordered?	pass→pass	20,601	12,430	-40%	1	1	0%	2,202	1,797	-18%	0	0	—
case-04 When running perturbation tests on a BERT sentiment model, should we pick our statistical significance tests before estimating sample sizes, or after sample size estimation?	fail→fail	19,138	22,967	+20%	1	1	0%	2,252	3,635	+61%	0	0	—
case-05 We are benchmarking an IoT telemetry anomaly detector against corrupted sensor data. The engineering team only wants to evaluate additive Gaussian noise because it is easiest to implement. What other noise injection categories must be included to satisfy a comprehensive noise robustness evaluation?	fail→fail	23,222	25,054	+8%	1	1	0%	2,778	3,562	+28%	0	0	—
case-06 We need to test the security boundaries of an Autonomous Vehicle Yolov8 object detector against adversarial input perturbations. A developer recommended testing only random uniform pixel jitter. What specific gradient-based adversarial attack algorithms should be specified for epsilon-bounded robustness testing?	pass→fail	22,596	27,548	+22%	1	1	0%	2,758	4,277	+55%	0	0	—
case-07 A team evaluating a clinical diagnostic model under dataset shift plans to only measure domain shift across different hospital systems. What key distribution shift categories are missing from this plan?	pass→fail	19,700	20,203	+3%	1	1	0%	2,327	2,992	+29%	0	0	—
case-08 We want to perform a quick sanity check on a speech recognition model using a single background noise condition. Our lead engineer wants to run 20 different severity levels across 50 runs. According to standard budget constraints for a quick single-perturbation sanity check, what are the recommended severity levels and minimum run counts?	fail→pass	28,828	3,806	-87%	1	1	0%	1,371	1,230	-10%	0	0	—
case-09 For a multi-perturbation robustness evaluation of a financial fraud detection model across 4 noise conditions, the team proposed testing 10 severity levels with 100 total runs. What are the standard recommended severities per condition and total minimum run budget for multi-perturbation evaluations?	fail→pass	20,035	9,972	-50%	1	1	0%	2,729	2,248	-18%	0	0	—
case-10 We are setting up an adversarial perturbation sweep for a facial recognition model using PGD. The security team wants to test only 2 epsilon values with 2 total runs. What are the recommended epsilon severity levels and minimum run counts for an adversarial sweep curve?	fail→pass	48,534	20,113	-59%	1	1	0%	2,891	3,164	+9%	0	0	—
case-11 We are preparing a paper on model robustness for NeurIPS and need a publication-ready evaluation across 6 distinct perturbation types. The lead author suggested running 1 severity level per type for a total of 6 runs. What budget parameters are required for a publication-ready comprehensive evaluation?	pass→pass	25,364	30,761	+21%	1	1	0%	3,263	5,148	+58%	0	0	—
case-12 We need to evaluate cross-domain transfer for a document OCR model across 5 target unseen domains. How many severity levels per domain are required, and what is the minimum run count calculation for transfer evaluation?	fail→pass	21,798	16,589	-24%	1	1	0%	2,866	3,642	+27%	0	0	—
case-13 To stress-test a loan default prediction model under extreme conditions, the QA lead suggests only testing standard Gaussian noise around the mean feature values. What input boundary conditions should stress testing target instead?	fail→fail	15,248	28,755	+89%	1	1	0%	2,518	4,285	+70%	0	0	—
case-14 In defining performance degradation metrics for an e-commerce recommendation system under covariate shift, the team plans to report only the absolute accuracy on the shifted test set. What critical metric framing must be included alongside absolute metrics?	pass→pass	14,092	17,029	+21%	1	1	0%	2,226	2,606	+17%	0	0	—
case-15 When starting a robustness assessment for an autonomous drone navigation system, the project manager asks: 'What is the average accuracy of the system across standard test datasets?' How should the core strategy question for robustness design be framed instead?	pass→pass	13,646	17,030	+25%	1	1	0%	1,471	3,313	+125%	0	0	—
case-16 During the factor identification phase for a medical imaging diagnostic model, the team only listed 'perturbation intensity'. What core dimensions must be defined when identifying robustness factors?	pass→pass	19,913	17,873	-10%	1	1	0%	2,293	2,645	+15%	0	0	—
case-17 When evaluating domain generalization for a satellite image segmentation model trained on European imagery, should cross-domain transfer testing be conducted on European sub-regions included in training or on unobserved domains?	pass→pass	11,105	17,865	+61%	1	1	0%	1,780	2,703	+52%	0	0	—
case-18 For a toxic text classifier, after factor identification and level specification are completed, what step immediately follows level specification before metrics are defined?	fail→pass	8,385	9,381	+12%	1	1	0%	1,263	1,261	-0%	0	0	—
case-19 We are building the test grid for an autonomous driving perception stack involving lighting changes, fog density, and motion blur. What structured step converts identified factor levels into an executable experiment grid?	pass→pass	20,667	14,093	-32%	1	1	0%	2,447	3,146	+29%	0	0	—
case-20 For a customer support chatbot entity extraction model, the product manager wants to test robustness using only common 10-word queries. What edge case conditions should stress testing target?	pass→pass	23,156	34,655	+50%	1	1	0%	3,019	5,326	+76%	0	0	—
case-21 A machine learning pipeline for fraud detection is experiencing changes where the underlying class prior P(Y) changes over time while P(X\|Y) remains constant. What specific type of distribution shift must be evaluated for this scenario?	pass→pass	12,277	16,764	+37%	1	1	0%	1,286	2,602	+102%	0	0	—
case-22 After determining the sample size estimation for a timeseries forecasting robustness test, what is the final execution step in the robustness design flow?	fail→pass	14,518	8,857	-39%	1	1	0%	1,423	1,267	-11%	0	0	—
case-23 For a single perturbation sanity check on a recommendation model's embedding vectors using Gaussian noise, how many minimum runs should be planned?	pass→pass	17,515	10,312	-41%	1	1	0%	1,998	1,411	-29%	0	0	—
case-24 In addition to label noise and missing data, what continuous noise type should be included when evaluating sensor telemetry robustness?	fail→pass	16,230	21,533	+33%	1	1	0%	2,377	4,179	+76%	0	0	—
case-25 We are tuning the learning rate schedule, weight decay, and dropout parameters for a Vision Transformer trained on ImageNet. Please provide a hyperparameter optimization grid search plan to maximize validation accuracy.	pass→pass	26,010	75,507	+190%	1	1	0%	3,592	6,091	+70%	0	0	—
case-26 We are training a multi-class image classifier on a heavily imbalanced dataset where 90% of samples belong to one class. Recommend appropriate loss function formulations to handle the severe class imbalance during model training.	pass→pass	24,642	35,601	+44%	1	1	0%	3,158	6,021	+91%	0	0	—
case-27 We are collecting and cleaning a new raw web-scraped dataset for training an LLM. How should we perform deduplication, outlier filtering, and quality scoring to create a clean training dataset?	pass→fail	30,954	53,872	+74%	1	1	0%	4,106	8,688	+112%	0	0	—

These cases were also screened by the skill’s intent / safety pass.See the Trust review ↗Reproduce this benchmark in Colab ↗

How this skill was measured

DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 27 cases were attempted, and 26 counted toward the lift figure. The other 1 produced results that are not comparable between the two arms, so they are excluded from the headline rather than averaged into it. The headline lift of +15 percentage points is the difference between those two pass rates over the 26 comparable cases. 3 cases got worse with the skill loaded, and they are included in that figure.

Model measured: gemini-3.6-flash
Graded by: gemini-3.6-flash
Grading method: judged
Verification: verified
Pass rate without the skill: 59.3%
Pass rate with the skill: 74.1%
Cases attempted: 27
Cases counted toward lift: 26
Cases that regressed: 3
Skill version measured: v1
Run completed: 9/3/2026, 22:53 UTC

Measured example — same prompt, with vs. without the skill · from the benchmark run · case case-09

Prompt

For a multi-perturbation robustness evaluation of a financial fraud detection model across 4 noise conditions, the team proposed testing 10 severity levels with 100 total runs. What are the standard recommended severities per condition and total minimum run budget for multi-perturbation evaluations?

Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.

Show both answers

SkillScore blends 4 signals1 of 4 complete

BenchmarkPass rate WITH skill on a verified run74

Live evalNeeds ≥10 evaluated traces in 30 days

AI ratingNeeds ≥10 rated traces

AdoptionNeeds ≥10 serves across ≥2 orgs in 30 days

Efficiency · separate axis (at equal correctness)+0% turns+46% tokens

Shown for transparency; not credited unless a verified run shows no correctness regression.

Backed by 1 of 4 signals — each added signal makes the score more robust.

Verified on gemini-3.6-flash · re-tested 17 days ago

Other measured skills in the registry, with their headline benchmark lift.