Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →nvidia/nemo-mbridge-perf-moe-dispatcher-selection
Choose the right MoE token dispatcher (`alltoall`, DeepEP, or HybridEP) for the hardware, EP degree, and optimization stage. Summarizes patterns from DSV3, Qwen3, Qwen3-Next, and VLM bring-up work.
Passed 22 of 22 test cases with the skill, and 9 of 22 without it.
and 21 more in the full report
14 fixed9 passed either way0 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 13 test cases the skill fixed. Same model, same prompt, with and without the skill.
When enabling the flex dispatcher backend with DeepEP in NeMo Megatron, we notice default SM allocation might not be optimal for our model scale. What exact CLI flag tunes the number of streaming multiprocessors reserved for DeepEP communication kernels, and what is its default numerical setting?
| Checked | Without skill | With skill |
|---|---|---|
| Identifies the command-line argument as --moe-deepep-num-sms | Passed | Passed |
| States that the default allocation is 20 SMs | Missed | Passed |
The instructions your model reads when the skill loads.
Stable docs: @docs/training/moe-optimization.md Card: @skills/nemo-mbridge-perf-moe-dispatcher-selection/card.yaml
| Hardware | First choice | Why |
|---|---|---|
| H100 | DeepEP, if the runtime package is installed | Strong default for cross-node EP on Hopper |
| B200 | DeepEP, if the runtime package is installed | Good first choice unless a platform-specific HybridEP path is available |
| GB200 / GB300 NVL72 | HybridEP, if the runtime package is installed | Best fit for NVLink-domain-aware dispatch and lower memory pressure |
| Unknown or first bring-up | alltoall | Easiest path for correctness and debugging |
| EP size | Guidance |
|---|---|
| Small EP | Dispatcher choice is usually second-order; start with alltoall or DeepEP |
| Medium EP | DeepEP often becomes worthwhile |
| Large EP | HybridEP is usually the best target on NVL72 systems |
| Workload | Common best path | Notes |
|---|---|---|
| DSV3 at large scale | HybridEP on GB200 or GB300, DeepEP on H100 | Dispatcher choice matters more as EP and PP both grow |
| Qwen3 235B | DeepEP on H100, HybridEP on GB200 | HybridEP usually wins on GB200 and often uses less memory |
| Qwen3 30B | DeepEP | Smaller models still benefit, but the absolute gap is smaller |
| Qwen3-Next | Close race in BF16, HybridEP stronger in FP8 or memory-tight runs | Good reminder to test, not assume |
| MoE VLMs | Start simple, then test HybridEP on GB200-class systems | Vision workloads are sensitive to both memory and host overhead |
Do not interpret a dispatcher timing until the container has proven that the
selected backend package is available. --moe_flex_dispatcher_backend None
test cases passed, vs 9 / 22 without
Published skills whose descriptions closely match this one, with their headline benchmark lift.
More skills from the same source and nearby in the registry, with their headline benchmark lift.