Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →nvidia/nemo-mbridge-perf-cpu-offloading
Validate and use CPU offloading in Megatron Bridge, including layer-level activation offloading and fractional optimizer state offloading with HybridDeviceOptimizer.
Passed 21 of 21 test cases with the skill, and 8 of 21 without it.
and 21 more in the full report
14 fixed8 passed either way0 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 13 test cases the skill fixed. Same model, same prompt, with and without the skill.
When `optimizer_cpu_offload` is enabled in NeMo Megatron Bridge, what underlying Megatron-LM optimizer class is instantiated to manage host and device parameters?
| Checked | Without skill | With skill |
|---|---|---|
| Identifies HybridDeviceOptimizer as the instantiated class | Missed | Passed |
The instructions your model reads when the skill loads.
Two independent mechanisms to move data from GPU to CPU memory:
| Mechanism | Config namespace | What gets offloaded | PP restriction |
|---|---|---|---|
| Activation offloading | model.cpu_offloading* | Activations (and optionally weights) per transformer layer | PP must be 1 |
| Optimizer offloading | optimizer.optimizer_cpu_offload | Adam optimizer states (momentum + variance) via HybridDeviceOptimizer | None |
| Situation | Recommendation |
|---|---|
| Large MoE model (30B+), needs PP > 1 | Optimizer offloading — activation offloading is blocked by PP=1 |
| Small/medium model, PP=1 fits, activation memory dominates | Activation offloading |
| Want tunable memory-speed tradeoff | Optimizer offloading with fractional optimizer_offload_fraction |
| Throughput is top priority | Don't enable — offloading always adds overhead |
| CUDA graphs are needed | Only optimizer offloading — activation offloading is incompatible |
| Memory pressure is moderate | Optimizer offload at 25–50% fraction for best efficiency |
cfg.optimizer.optimizer_cpu_offload = True
cfg.optimizer.optimizer_offload_fraction = 1.0
cfg.optimizer.overlap_cpu_optimizer_d2h_h2d = True
CLI overrides:
test cases passed, vs 8 / 21 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.