Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →nvidia/nemo-mbridge-perf-cuda-graphs
Validate and use CUDA graph capture in Megatron Bridge, including local full-iteration graphs and Transformer Engine scoped graphs for attention, MLP, and MoE modules.
Passed 20 of 20 test cases with the skill, and 7 of 20 without it.
and 21 more in the full report
14 fixed8 passed either way0 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 12 test cases the skill fixed. Same model, same prompt, with and without the skill.
When using the NeMo performance harness CLI (`run_script.py`), if I set `--cuda_graph_impl` to something other than `none`, what two RNG-related configuration flags does the harness automatically enable for me?
| Checked | Without skill | With skill |
|---|---|---|
| States that model.use_te_rng_tracker is auto-enabled | Missed | Passed |
| States that rng.te_rng_tracker is auto-enabled | Missed | Passed |
The instructions your model reads when the skill loads.
Stable documentation: @docs/training/cuda-graphs.md Card: @skills/nemo-mbridge-perf-cuda-graphs/card.yaml
CUDA graphs capture GPU operations once and replay them with minimal host-driver overhead. Bridge supports two implementations:
cuda_graph_impl | Mechanism | Scope support |
|---|---|---|
"local" | MCore FullCudaGraphWrapper wrapping entire fwd+bwd | full_iteration |
"transformer_engine" | TE make_graphed_callables() per layer | attn, mlp, moe, moe_router, moe_preprocess, mamba |
Start with TE-scoped graphs for most training workloads, then verify replay timing against eager on the same dispatcher, layout, and container:
attn, then optionally mlpattn moe_router moe_preprocessUse local + full_iteration only when you specifically want full-iteration
capture and can satisfy the tighter constraints.
For recompute-heavy workloads:
local full-iteration graphs or away
from graphs entirelyRelated docs:
test cases passed, vs 7 / 20 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.