Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →nvidia/nemo-mbridge-perf-moe-vlm-training
Practical guidance for training MoE VLMs in Megatron Bridge. Compares FSDP and 3D-parallel approaches, using rounded lessons from Qwen3-VL, Qwen3-Next, and other multimodal experiments.
Passed 20 of 21 test cases with the skill, and 11 of 21 without it.
and 13 more in the full report
11 fixed10 passed either way0 still failing1 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 10 test cases the skill fixed. Same model, same prompt, with and without the skill.
We are enabling CUDA graphs on a stable MoE VLM training run to reduce launch overhead. A developer suggests enabling CUDA graphs globally across the entire model training step. What specific CUDA-graph scope should be applied as the safe MoE default?
| Checked | Without skill | With skill |
|---|---|---|
| Advises capturing CUDA graphs on attention, router, and MoE preprocessing submodules rather than full model graphs | Missed | Passed |
The instructions your model reads when the skill loads.
Stable docs: @docs/training/moe-optimization.md Card: @skills/nemo-mbridge-perf-moe-vlm-training/card.yaml
| Approach | Strength | Best fit |
|---|---|---|
| FSDP | Simplest path to a working multimodal run | first bring-up, memory-first tuning, awkward PP boundaries |
| 3D parallel | Higher ceiling after tuning | stable models with a clean PP layout and time for deeper sweeps |
For MoE VLMs, the practical workflow is usually:
The main patterns were consistent across the tracker:
Mock-data VLM runs are not trustworthy performance proxies. In the experiments, image-free mock runs looked closer to "roughly twice as fast" than "slightly optimistic" when compared with real multimodal input.
Use real or realistic image payloads before drawing any conclusion about VLM throughput.
test cases passed, vs 11 / 21 without
Published skills whose descriptions closely match this one, with their headline benchmark lift.
More skills from the same source and nearby in the registry, with their headline benchmark lift.