Ready to supercharge your AI agents?

Install any skill in seconds. Free to start, no credit card required.

Get Started Free →

Strategy: Scaling Design · yogsoth-ai | DecimalAI

yogsoth-ai/scaling-design

SkillScore82Maintained

Design scaling experiments to characterize performance-resource relationships

AI/MLby yogsoth-ai

Links this skill to your agents and keeps it current with the author’s updates.

Automatic

AI Engineer pack

1 · One-time — install the CLI (docs ↗)

pip install decimalai

2 · Get the skill — writes .claude/skills/yogsoth-ai-scaling-design/SKILL.md

decimalai skills pull yogsoth-ai-scaling-design --out .claude/skills/

No CLI? Get the same file — curl · raw file

curl -fsSL --create-dirs -o .claude/skills/yogsoth-ai-scaling-design/SKILL.md https://app.decimal.ai/s/yogsoth-ai-scaling-design/SKILL.md

Raw file: /s/yogsoth-ai-scaling-design/SKILL.md ↗ · machine-readable summary at /s/yogsoth-ai-scaling-design.json ↗

In productionGlobal

how this skill behaves on real traffic · last 30 days

No production data for this skill yet — usage numbers appear here once agents start running it.

Anonymized aggregate across every org using this skill — proof from real production use, not a demo.

Does this skill help? · Benchmark: with vs. withoutsign in to run this on your agents

22 test cases from the publisher’s eval.yaml · each run twice on gemini-3.6-flash — with the skill injected, then without · last run 18d ago

Yes — +32% lift+17% tokens0% turns82% of whole cases pass with skill (18/22)

Test case	Without → With	Effect	Δ tokens	Δ turns
case-02	✗→✓	▲ Improved	40%	0%
case-11	✗→✓	▲ Improved	76%	0%
case-12	✗→✓	▲ Improved	21%	0%
case-17	✗→✓	▲ Improved	29%	0%
case-19	✗→✓	▲ Improved	-8%	0%

Showing 5 of 22 cases

SkillSafety

✓ 3 of 3 passed

Code scan15 of 15 checks clean

0 critical · 0 warning · 0 info — scanner v4 — 8/10/2026

live_secretremote_code_executionreverse_shelldata_exfiltrationssrf_metadataagent_snoopinganti_refusaldestructive_commandsobfuscated_unicodeunicode_homoglyphopaque_blobsuspicious_urltool_overreachtrigger_abuseprompt_injection_phrasing

IntentSafe

LLM judge · does the skill do what it claims

ContentClean

LLM judge · policy & harmful-content review

Source

Imported

yogsoth-ai / de-anthropocentric-research-engine

skills/scaling-design/SKILL.md

View on GitHub ↗

License Apache-2.0Imported 8/10/2026Version v1

Skill contents

SKILL.md

Strategy: Scaling Design

Question: How does performance scale with resources?

Methodology

Neural Scaling Laws (Kaplan 2020, Hoffmann 2022): Power-law relationships between compute/data/parameters and loss.
Compute-Optimal Scaling (Chinchilla): Find optimal allocation between model size and data.
Data Scaling: Characterize learning curves as function of dataset size.
Model Scaling: Performance vs. parameter count at fixed data.
Inference Scaling: Throughput/latency vs. batch size, sequence length, model size.

Execution Flow

factor-identification → Identify scaling axes (data, compute, parameters, time)
level-specification → Define scale points (geometric progression, typically 4-8 points)
metric-specification → Define metrics at each scale (loss, downstream task, efficiency)
design-matrix-construction → Build scaling experiment grid
sample-size-estimation → Determine replicates needed for reliable curve fitting
budget-constrained-design (tactic) → Optimize which scale points to run given budget

Budget Gate

| Scaling Type | Scale Points | Replicates | Min Runs | Typical Cost | |-------------|-------------|------------|----------|--------------| | Data scaling | 4-6 | 3 | 12-18 | Low (same model, subset data) | | Model scaling | 4-8 | 2-3 | 8-24 | High (different model sizes) | | Compute-optimal | 6-10 per iso-FLOP | 1-2 | 12-20 | Very high | | Inference scaling | 5-10 | 5 | 25-50 | Low (inference only) |

Available Tactics

Optional, no fixed order; the final leaf is always a sop.

| Tactic | When to use | | --- | --- | | budget-constrained-design | Optimize experiment design under compute and time budget constraints | | statistical-method-selection | Select appropriate statistical methods for experiment analysis |

Available SOPs

Optional, no fixed order; the final leaf is always a sop.

| SOP | When to use | | --- | --- | | design-matrix-construction | Build the experiment design matrix with proper orthogonality and balance | | factor-identification | Identify independent, dependent, and control variables for an experiment | | level-specification | Determine appropriate levels for each experimental factor | | metric-specification | Define experiment metrics and significance standards | | sample-size-estimation | SOP: power analysis and required experiment count estimation |

Illustrative Want to see what yogsoth-ai/scaling-design would put in front of your model? Preview the rendered body and check its trigger phrases — no LLM call, no signup.

Latest benchmark · publisher runVerified

9/3/2026, 22:47 UTC · 22 cases

Lift

+32%

82% vs 50%

Agent turns

0%

1.0 vs 1.0

Tokens

+17%

3,827.95 vs 3,268.23

fail → pass · 7pass → pass · 11fail → fail · 4

Per-case results (22)

Case	Status	Duration (ms)			Turns			Tokens			Tool calls
Case	Status	Without	With	Δ	Without	With	Δ	Without	With	Δ	Without	With	Δ
case-01 We are preparing to train a domain-specific language model and want to characterize how performance scales with dataset size. Could you walk us through a complete scaling experiment protocol? Please detail how we should select independent and control variables, choose scale points, define evaluation metrics, build the experiment design matrix, estimate sample sizes or replicates, and adjust the layout if budget constraints become tight.	fail→fail	48,660	29,156	-40%	1	1	0%	7,665	4,826	-37%	0	0	—
case-02 Our AI research team needs to conduct a compute-optimal scaling study similar to Chinchilla to find the ideal balance between model parameters and training tokens. Please provide a structured experimental design plan that outlines factor identification, scale point determination, metric definitions, the construction of a design matrix, run/replicate estimation, and budget optimization strategies.	fail→pass	35,650	41,785	+17%	1	1	0%	5,323	7,437	+40%	0	0	—
case-03 We are preparing an inference benchmarking suite for our production vision models to analyze throughput and latency across varying batch sizes and sequence lengths. Can you create a step-by-step benchmark design that breaks down the key factors, specifies scale levels, defines performance metrics, formats a full design matrix, and estimates necessary replicate runs to ensure reliable curve fitting?	fail→fail	45,465	28,555	-37%	1	1	0%	7,556	4,748	-37%	0	0	—
case-04 We are setting up a distributed training pipeline on an 8-GPU node for a 7B parameter transformer. What values should we choose for the AdamW optimizer's learning rate, beta parameters, and warmup steps to prevent loss spikes during early training?	pass→pass	22,737	33,329	+47%	1	1	0%	3,084	6,251	+103%	0	0	—
case-05 We want to redesign the attention architecture of our language model to reduce KV cache memory consumption during inference on edge devices. Should we adopt Multi-Query Attention (MQA) or Grouped-Query Attention (GQA), and what ratio of query heads to key-value heads provides the best tradeoff?	pass→pass	21,523	35,878	+67%	1	1	0%	3,432	5,988	+74%	0	0	—
case-06 We are fine-tuning Llama-3-8B on specialized medical literature using Low-Rank Adaptation (LoRA). How should we set the rank r, alpha scaling factor, and target modules (e.g. q_proj, v_proj) for optimal fine-tuning performance?	pass→pass	22,840	27,668	+21%	1	1	0%	2,921	4,584	+57%	0	0	—
case-07 We are evaluating model scaling for a family of decoder-only transformers ranging from 100M to 10B parameters at a fixed dataset size of 500B tokens. An engineer suggested running 12 different model sizes with 10 replicates each to fit the power-law curve. What parameters should we actually use for model scale points and replicates under standard scaling protocol?	fail→fail	25,821	32,931	+28%	1	1	0%	3,544	4,969	+40%	0	0	—
case-08 We are starting a new empirical scaling study. Our team lead wants to immediately jump into building the matrix grid and running power analysis before listing any independent or dependent variables. What is the correct initial execution order for scaling design studies?	pass→pass	21,026	13,701	-35%	1	1	0%	2,494	2,066	-17%	0	0	—
case-09 Our infrastructure manager thinks data scaling experiments require training entirely distinct model architectures and will cost millions in compute. How should a data scaling study actually be conducted, and what is its expected relative cost level?	pass→pass	24,542	24,409	-1%	1	1	0%	3,013	3,504	+16%	0	0	—
case-10 We need to decide whether to run a compute-optimal scaling study or an inference benchmarking study. Compute-optimal studies sound inexpensive because they use smaller models. How does compute-optimal scaling rank in typical cost, and what fundamental question does it answer?	pass→pass	18,450	12,957	-30%	1	1	0%	2,105	1,924	-9%	0	0	—
case-11 When defining scale points (such as token counts or parameter sizes) for power-law curve fitting, a researcher suggested selecting scale points spaced linearly (e.g., 1B, 2B, 3B, 4B, 5B tokens). How should scale levels actually be spaced across the experimental range?	fail→pass	17,780	24,548	+38%	1	1	0%	2,444	4,290	+76%	0	0	—
case-12 In our scaling study pipeline, where does budget-constrained design fit into the workflow, and what role does it play when full grid exploration is too expensive?	fail→pass	21,364	21,019	-2%	1	1	0%	2,470	2,998	+21%	0	0	—
case-13 Our research paper needs to ground its methodology for parameter and compute scaling analysis in foundational literature. Which classic power-law scaling law frameworks establish relationships between compute, data, parameters, and loss?	pass→pass	23,466	23,324	-1%	1	1	0%	3,356	3,928	+17%	0	0	—
case-14 We have a limited cloud budget and want to run both model scaling during training and inference scaling across batch sizes. How do the relative costs of inference scaling compare to model scaling experiments?	pass→pass	22,661	20,304	-10%	1	1	0%	2,855	3,229	+13%	0	0	—
case-15 Can you clarify what model scaling measures in contrast to data scaling, specifically keeping training data volume constant?	pass→pass	21,071	22,966	+9%	1	1	0%	2,378	3,390	+43%	0	0	—
case-16 We are ready to build our experiment grid for a multi-factor scaling study involving parameter count and sequence length. What structural properties must our design matrix maintain according to experimental design standards?	pass→pass	23,553	21,306	-10%	1	1	0%	2,947	2,965	+1%	0	0	—
case-17 Why is sample size estimation performed prior to executing scaling runs, and what statistical analysis technique should be used to determine required replicate counts?	fail→pass	21,434	22,029	+3%	1	1	0%	2,594	3,343	+29%	0	0	—
case-18 When initiating the factor identification phase for an empirical scaling law study, how should variables be categorized across the experiment?	pass→pass	21,676	20,813	-4%	1	1	0%	2,527	3,002	+19%	0	0	—
case-19 What three distinct categories of performance metrics must be defined during the metric specification step of scaling design?	fail→pass	14,725	10,675	-28%	1	1	0%	1,621	1,489	-8%	0	0	—
case-20 A team member proposes running a data scaling study with 10 scale points and 1 single run per dataset size to save compute. Why is 1 run insufficient for data scaling, and what is the standard replicate count and scale point range for data scaling?	fail→pass	20,141	19,482	-3%	1	1	0%	2,391	2,491	+4%	0	0	—
case-21 We are setting up a benchmark matrix for inference scaling across sequence lengths and batch sizes. An engineer recommends 20 scale points with 1 single inference run per point. What are the recommended scale points and replicate counts for inference scaling studies?	fail→fail	16,964	13,847	-18%	1	1	0%	2,818	3,160	+12%	0	0	—
case-22 In compute-optimal scaling experiments (like Chinchilla iso-FLOP curves), compute is extremely constrained. How many replicates per point should be used given the high cost per run?	fail→pass	15,335	24,323	+59%	1	1	0%	2,363	3,633	+54%	0	0	—

These cases were also screened by the skill’s intent / safety pass.See the Trust review ↗Reproduce this benchmark in Colab ↗

How this skill was measured

DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 22 cases were attempted. The headline lift of +32 percentage points is the difference between those two pass rates over the 22 comparable cases.

Model measured: gemini-3.6-flash
Graded by: gemini-3.6-flash
Grading method: judged
Verification: verified
Pass rate without the skill: 50%
Pass rate with the skill: 81.8%
Cases attempted: 22
Cases counted toward lift: 22
Cases that regressed: 0
Skill version measured: v1
Run completed: 9/3/2026, 22:47 UTC

Measured example — same prompt, with vs. without the skill · from the benchmark run · case case-02

Prompt

Our AI research team needs to conduct a compute-optimal scaling study similar to Chinchilla to find the ideal balance between model parameters and training tokens. Please provide a structured experimental design plan that outlines factor identification, scale point determination, metric definitions, the construction of a design matrix, run/replicate estimation, and budget optimization strategies.

Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.

Show both answers

SkillScore blends 4 signals1 of 4 complete

BenchmarkPass rate WITH skill on a verified run82

Live evalNeeds ≥10 evaluated traces in 30 days

AI ratingNeeds ≥10 rated traces

AdoptionNeeds ≥10 serves across ≥2 orgs in 30 days

Efficiency · separate axis (at equal correctness)+0% turns+31% tokens

Shown for transparency; not credited unless a verified run shows no correctness regression.

Backed by 1 of 4 signals — each added signal makes the score more robust.

Verified on gemini-3.6-flash · re-tested 17 days ago

Other measured skills in the registry, with their headline benchmark lift.