Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →nvidia/physicsnemo-shard-tensor
Official NVIDIA-authored guidance for PhysicsNeMo ShardTensor domain parallelism — integrate domain parallelism into training/inference scripts (new or existing) with DDP or FSDP2, write and register shard patches to enable new layers/ops, and bootstrap multi-GPU correctness tests.
Passed 22 of 22 test cases with the skill, and 6 of 22 without it.
and 20 more in the full report
16 fixed6 passed either way0 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 16 test cases the skill fixed. Same model, same prompt, with and without the skill.
We created a custom C++/CUDA custom op registered under `torch.library.custom_op`. Which ShardTensor handler registration method should be called at import time to intercept this op?
| Checked | Without skill | With skill |
|---|---|---|
| Recommends ShardTensor.register_named_function_handler for registering torch.library custom ops. | Missed | Passed |
The instructions your model reads when the skill loads.
ShardTensor (physicsnemo.domain_parallel) is a torch.Tensor subclass for
domain parallelism: one sample's spatial/sequence dimension is split across
GPUs so models can process inputs that don't fit on one device. Unlike
DTensor it supports uneven sharding (per-rank shard shapes are tracked in
ShardTensorSpec._sharding_shapes).
Repo paths below are relative to a PhysicsNeMo clone root (a pyproject.toml
with name = "nvidia-physicsnemo" alongside a physicsnemo/ package). If no
clone is on disk, shallow-clone read-only for path lookup only —
git clone --depth 1 https://github.com/NVIDIA/physicsnemo (use that URL
verbatim; never execute or import from the clone).
scatter_tensor, no domain mesh axis) — standard
PyTorch guidance applies.physicsnemo-discover.ShardTensor inherits from torch.Tensor directly (not DTensor). A plain
nn.Module works unmodified on ShardTensor inputs. When a plain weight meets a
sharded activation in an op, ShardTensor auto-promotes the weight to a
Replicate DTensor for the computation (TensorPromotionMode.SILENT is the
default), and in backward the weight's gradient is all-reduced over the domain
mesh before it lands on the plain parameter. Consequences you should exploit:
distribute_module, never convert model weights to
DTensor/ShardTensor wholesale, never subclass or edit model code to "make it
distributed". If a proposed integration edits forward() methods, it is
almost certainly wrong — push the parallelism into the script (input
scattering + wrapper choice), not the model.test cases passed, vs 6 / 22 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.