ai ml by nvidia
TP / DP / PP Communication Overlap Skill nvidia/nemo-mbridge-perf-tp-dp-comm-overlap
Operational guide for enabling TP, DP, and PP communication overlap in Megatron-Bridge, including config knobs, code anchors, pitfalls, and verification.
What it changes View full report → Passed 21 of 22 test cases with the skill, and 8 of 22 without it.
Without skill 36%
With skill 95%
Checks it passed that the model alone missed and 21 more in the full report
13 fixed8 passed either way1 still failing0 broke
Pass-rate lift +59 pp
Tokens -9%
Agent turns 0%
Response time -60% Model gemini-3.6-flashMeasured 8/12/2026 v1 21 of 22 cases passed with skill
Use this skill Claude Code DecimalAI agent CLI & files
Save it into your project's skills folder. Claude Code picks it up in its next session.
curl -fsSL --create-dirs -o .claude/skills/nvidia-nemo-mbridge-perf-tp-dp-comm-overlap/SKILL.md https://app.decimal.ai/s/nvidia-nemo-mbridge-perf-tp-dp-comm-overlap/SKILL.md Copy command Copy SKILL.md
Want it pinned and updatable? pip install "decimalai>=0.15", then decimalai skills add nvidia-nemo-mbridge-perf-tp-dp-comm-overlap, and start Claude Code with claude --plugin-dir "$(git rev-parse --show-toplevel)/.agents"
Linked stays up to date: your agents get the author's new versions automatically.
Editable copy is yours to change; the author's updates won't reach it.
Install linked skill Create editable copy
1 · One-time — install the CLI (
docs ↗ )
$ pip install "decimalai>=0.15"Copy
2 · Get the skill — writes .agents/skills/nvidia-nemo-mbridge-perf-tp-dp-comm-overlap/ and pins it in .decimal/skills.lock
$ decimalai skills add nvidia-nemo-mbridge-perf-tp-dp-comm-overlapCopy
Copy link Copy README badge Copy SKILL.md No CLI? Get the same file — curl · raw file $ curl -fsSL --create-dirs -o .claude/skills/nvidia-nemo-mbridge-perf-tp-dp-comm-overlap/SKILL.md https://app.decimal.ai/s/nvidia-nemo-mbridge-perf-tp-dp-comm-overlap/SKILL.mdCopy
Safety checks: 3 of 3 passed
Code scan Clean Intent Passed Content Clean
Overview Benchmark Contents Activity Versions 1 Ratings 0
See it in action One of 13 test cases the skill fixed. Same model, same prompt, with and without the skill.
Prompt We want to disable FP32 gradient reduction precision and disable FP8 parameter gathering in Megatron Bridge. Show the exact two lines of code to configure these options.
What was checked, without and with the skill Checked Without skill With skill Sets cfg.mixed_precision.grad_reduce_in_fp32 = False Sets cfg.mixed_precision.fp8_param_gather = False
Show both answers
What's inside The instructions your model reads when the skill loads.
SKILL.md 112 lines single file
TP / DP / PP Communication Overlap Skill
For stable background and recommendation level, see:
@docs/training/communication-overlap.md
Enablement
Minimal Bridge override:
from megatron.bridge.training.comm_overlap import CommOverlapConfig
cfg.model.tensor_model_parallel_size = 4
cfg.model.sequence_parallel = True
cfg.model.pipeline_model_parallel_size = 4
cfg.model.virtual_pipeline_model_parallel_size = 2
cfg.comm_overlap = CommOverlapConfig(
tp_comm_overlap=True,
)
cfg.ddp.use_distributed_optimizer = True
cfg.ddp.overlap_grad_reduce = True
cfg.ddp.overlap_param_gather = True
Optional TP preset:
from megatron.bridge.training.comm_overlap import userbuffers_bf16_h100_h12288_tp4_mbs1_seqlen2048
cfg.comm_overlap.tp_comm_overlap_cfg = userbuffers_bf16_h100_h12288_tp4_mbs1_seqlen2048
Precision knobs belong to mixed precision:
cfg.mixed_precision.grad_reduce_in_fp32 = False
cfg.mixed_precision.fp8_param_gather = False
Track record Benchmark 21 / 22
test cases passed, vs 8 / 22 without
Adoption
Users 0
Times used 0 View activity → Safety and source details
SkillSafety ✓ 3 of 3 passed ✓ Code scan 15 of 15 checks clean
0 critical · 0 warning · 0 info — scanner v4 — 8/3/2026
View 15 scanner checks live_secret remote_code_execution reverse_shell data_exfiltration ssrf_metadata agent_snooping anti_refusal destructive_commands obfuscated_unicode unicode_homoglyph opaque_blob suspicious_url tool_overreach trigger_abuse prompt_injection_phrasing
✓ Intent Safe
LLM judge · does the skill do what it claims
✓ Content Clean
LLM judge · policy & harmful-content review
Source Imported nvidia / skills
skills/nemo-mbridge-perf-tp-dp-comm-overlap/SKILL.md
View on GitHub ↗ License Apache-2.0Imported 8/3/2026Version v1