Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →orchestra-research/ml-training-recipes
Battle-tested PyTorch training recipes for all domains — LLMs, vision, diffusion, medical imaging, protein/drug discovery, spatial omics, genomics. Covers training loops, optimizer selection (AdamW, Muon), LR scheduling, mixed precision, debugging, and systematic experimentation.
Passed 23 of 23 test cases with the skill, and 15 of 23 without it.
and 7 more in the full report
8 fixed15 passed either way0 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 8 test cases the skill fixed. Same model, same prompt, with and without the skill.
Our PyTorch CUDA training job runs out of GPU memory due to memory fragmentation despite total allocated memory being within hardware limits. What specific environment variable and value should be set in Python before importing `torch` to enable expandable memory segments?
| Checked | Without skill | With skill |
|---|---|---|
| Sets PYTORCH_ALLOC_CONF to expandable_segments:True. | Missed | Passed |
The instructions your model reads when the skill loads.
Battle-tested patterns for PyTorch training across domains. Drawn from production codebases (Karpathy's autoresearch/nanochat, torchvision, HuggingFace) and modern training practice.
references/architecture.md — Transformer/LLM architecture code patterns, weight initreferences/optimizers.md — Muon, AdamW hybrid, per-group LR, compiled optimizer stepsreferences/domain-specific.md — Vision, diffusion, contrastive, distributed, checkpointing, data loadingreferences/scaling-and-selection.md — Scaling laws, compute budget tables, decision trees, DGX Sparkreferences/biomedical.md — Drug discovery, protein models, medical imaging, genomics, clinical NLPreferences/experiment-loop.md — Autonomous experiment loop (autoresearch keep/discard/revert)Pick the right model by data type and data scale:
| Data Type | < 10K samples | 10K-100K | > 100K |
|---|---|---|---|
| Images | Pretrained CNN + fine-tune | Fine-tune ViT or CNN | ViT from scratch |
| Text (gen) | Few-shot prompting | Fine-tune GPT/LLaMA (LoRA) | Pretrain from scratch |
| Tabular | XGBoost/LightGBM | Still XGBoost | Neural viable |
| Audio | Pretrained Whisper | Fine-tune AST | Train from scratch |
| Molecules | Pretrained GNN | Fine-tune molecular LM | Train GNN from scratch |
| Proteins | ESM-2 embeddings + head | Fine-tune ESM-2 | Train protein LM |
| Medical img | Pretrained CNN | nnU-Net (auto-config) | Swin-UNETR / MedSAM |
Key principle: architecture matters less than training recipe at equal compute. A well-tuned ResNet beats a poorly-tuned ViT (ref: "ResNet Strikes Back", Wightman 2021).
For biomedical domains, see references/biomedical.md.
For sequence model selection and compute planning, see references/scaling-and-selection.md.
test cases passed, vs 15 / 23 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.