Comment on The Nvidia DGX-1 Deep Learning Supercomputer in a BoxparentComments−pinewurst10yThe diagram is confusing, but the GPUs are connected to the NVLink matrix which is connected to the motherboard via the PLX PCIe switches. The quad IB/dual 10GbE are separate IO attached to the motherboard.https://devblogs.nvidia.com/parallelforall/inside-pascal/−virtuallynathan10yThat would make much more sense. Thanks! The PCI bandwidth must be fairly limited. 4x 100G Infiniband is 64x PCIe lanes, out of 80x lanes available.
Comments
The diagram is confusing, but the GPUs are connected to the NVLink matrix which is connected to the motherboard via the PLX PCIe switches. The quad IB/dual 10GbE are separate IO attached to the motherboard.
https://devblogs.nvidia.com/parallelforall/inside-pascal/
That would make much more sense. Thanks! The PCI bandwidth must be fairly limited. 4x 100G Infiniband is 64x PCIe lanes, out of 80x lanes available.