Count the network in the model budget.

Video caption

Count the network in the model budget.

Two equal GPU counts need not have equal connectivity.

Topology, link, collective, application.

Match connected capacity to the real job.

#EricFieldNotes

Full written post / accessibility read

Distributed inference and training move information between devices. If the communication path cannot support the job, buying more cards may make the queue or run slower rather than faster. The fabric is part of the capacity you are buying.

NVIDIA Fabric Manager documents asymmetric NVLink behavior for some virtualized four-GPU H100 and H200 layouts. A dashboard listing the same accelerator type does not reveal peer path, NIC affinity or competing traffic. Those are observable properties of the actual allocation.

Ask for the provider's placement class before the pilot. On a scoped allocation, capture nvidia-smi topology, local bandwidth and network probes, a representative NCCL collective, then the model's actual step or inference trace at target concurrency. Keep device IDs and timestamps through all four checks.

Specify the placement and bandwidth class your workload needs, then validate completion time and cost under representative load. Do this because a disconnected bargain can be more expensive than a well-connected allocation that finishes predictably.

#EricFieldNotes

Four-beat scene transcript

1. Count the network in the model budget.

Distributed inference and training move information between devices. If the communication path cannot support the job, buying more cards may make the queue or run slower rather than faster. The fabric is part of the capacity you are buying.

Visual: A supernode is more than a pile of accelerators.

2. Inventory can conceal asymmetric paths.

NVIDIA Fabric Manager documents asymmetric NVLink behavior for some virtualized four-GPU H100 and H200 layouts. A dashboard listing the same accelerator type does not reveal peer path, NIC affinity or competing traffic. Those are observable properties of the actual allocation.

Visual: Two equal GPU counts need not have equal connectivity.

3. Use four checks in a scoped pilot.

Ask for the provider's placement class before the pilot. On a scoped allocation, capture nvidia-smi topology, local bandwidth and network probes, a representative NCCL collective, then the model's actual step or inference trace at target concurrency. Keep device IDs and timestamps through all four checks.

Visual: Topology, link, collective, application.

4. Buy a communication envelope.

Specify the placement and bandwidth class your workload needs, then validate completion time and cost under representative load. Do this because a disconnected bargain can be more expensive than a well-connected allocation that finishes predictably.

Visual: Match connected capacity to the real job.

Research and claim limits

• NVIDIA NCCL performance and tuning (S122)
• NVIDIA Fabric Manager (S126)

The examples identified as illustrative or simulated are design probes, not reported incidents. Vendor specifications do not establish workload performance.