Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →nvidia/nemo-mbridge-perf-megatron-fsdp
Operational guide for enabling Megatron FSDP in Megatron-Bridge, including config knobs, code anchors, pitfalls, and verification.
Passed 18 of 23 test cases with the skill, and 6 of 23 without it.
and 20 more in the full report
13 fixed5 passed either way4 still failing1 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 13 test cases the skill fixed. Same model, same prompt, with and without the skill.
Where is the 2-GPU Megatron FSDP pretraining functional test located in the NeMo Bridge repository, and what command runs it?
| Checked | Without skill | With skill |
|---|---|---|
| specifies test path tests/functional_tests/training/test_megatron_fsdp.py | Missed | Passed |
| instructs running the smoke test via pytest under torchrun with two processes per node | Passed | Passed |
The instructions your model reads when the skill loads.
For stable background and recommendation level, see:
Minimal Megatron FSDP override in Bridge:
cfg.dist.use_megatron_fsdp = True
cfg.ddp.use_megatron_fsdp = True
cfg.ddp.data_parallel_sharding_strategy = "optim_grads_params"
cfg.ddp.average_in_collective = False
cfg.checkpoint.ckpt_format = "fsdp_dtensor"
Example recipe fixup:
cfg = llama3_8b_pretrain_config()
cfg.dist.use_megatron_fsdp = True
cfg.ddp.use_megatron_fsdp = True
cfg.ddp.data_parallel_sharding_strategy = "optim_grads_params"
cfg.ddp.average_in_collective = False
cfg.checkpoint.ckpt_format = "fsdp_dtensor"
cfg.checkpoint.save = "/tmp/fsdp_ckpts"
cfg.checkpoint.load = None
Performance harness note:
python scripts/performance/launch.py --use_megatron_fsdp true
test cases passed, vs 6 / 23 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.